Search NASA⌕ Search

SEARCH · Search NASA

Results for “data processing steps”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Gross and Net Soil Methane Flux and Ancillary Data, Edgewater, MD, USA, summer 2022

This data package contains measurements used to quantify methane cycling and environmental conditions in coastal forest soils during the 2022 growing season. It includes time‑series data of soil methane flux, soil respiration, soil temperature, and volumetric water content collected from soil monoliths transplanted along an inundation and salinity gradient. The package also provides one‑time measurements from a stable‑isotope pool‑dilution incubation, including gravimetric water content, methane headspace concentrations, and ¹³CH₄ enrichment over time. Data files are provided in comma‑separated values (CSV) format, with accompanying metadata and readme documentation in PDF and plain‑text formats. All files can be opened with standard software such as R, Python, or spreadsheet programs capable of handling CSV files. The metadata file describes variable definitions, units, processing steps, and the structure of each data table to support reuse and integration with other datasets.

13-C↗

Experience gained from computer processing of physical experimental data during the restoration of measured values

The processing stage in which the restored values of the physical parameters are received is described. The following main steps are discussed: estimation of the state of the telemetry data, processing of the calibration data, and determination of the errors in the data; data decommutation and analysis of the structure of measurement cycles for each instrument; decoding, estimates of the reliability of the restored data, and their agreement with the models adopted for the measurement process; and analysis of errors due to deterministic and random factors. A block diagram of the method is presented.

Mamotko, Z. N.↗

System for Automated Calibration of Vector Modulators

Vector modulators are used to impose baseband modulation on RF signals, but non-ideal behavior limits the overall performance. The non-ideal behavior of the vector modulator is compensated using data collected with the use of an automated test system driven by a LabVIEW program that systematically applies thousands of control-signal values to the device under test and collects RF measurement data. The technology innovation automates several steps in the process. First, an automated test system, using computer controlled digital-to-analog converters (DACs) and a computer-controlled vector network analyzer (VNA) systematically can apply different I and Q signals (which represent the complex number by which the RF signal is multiplied) to the vector modulator under test (VMUT), while measuring the RF performance specifically, gain and phase. The automated test system uses the LabVIEW software to control the test equipment, collect the data, and write it to a file. The input to the Lab - VIEW program is either user-input for systematic variation, or is provided in a file containing specific test values that should be fed to the VMUT. The output file contains both the control signals and the measured data. The second step is to post-process the file to determine the correction functions as needed. The result of the entire process is a tabular representation, which allows translation of a desired I/Q value to the required analog control signals to produce a particular RF behavior. In some applications, corrected performance is needed only for a limited range. If the vector modulator is being used as a phase shifter, there is only a need to correct I and Q values that represent points on a circle, not the entire plane. This innovation has been used to calibrate 2-GHz MMIC (monolithic microwave integrated circuit) vector modulators in the High EIRP Cluster Array project (EIRP is high effective isotropic radiated power). These calibrations were then used to create correction tables to allow the commanding of the phase shift in each of four channels used as a phased array for beam steering of a Ka-band (32-GHz) signal. The system also was the basis of a breadboard electronic beam steering system. In this breadboard, the goal was not to make systematic measurements of the properties of a vector modulator, but to drive the breadboard with a series of test patterns varying in phase and amplitude. This is essentially the same calibration process, but with the difference that the data collection process is oriented toward collecting breadboard performance, rather than the measurement of output from a network analyzer.

Lux, James↗

Investigation of field induced trapping on floating gates

The development of a technology for building electrically alterable read only memories (EAROMs) or reprogrammable read only memories (RPROMs) using a single level metal gate p channel MOS process with all conventional processing steps is outlined. Nonvolatile storage of data is achieved by the use of charged floating gate electrodes. The floating gates are charged by avalanche injection of hot electrodes through gate oxide, and discharged by avalanche injection of hot holes through gate oxide. Three extra diffusion and patterning steps are all that is required to convert a standard p channel MOS process into a nonvolatile memory process. For identification, this nonvolatile memory technology was given the descriptive acronym DIFMOS which stands for Dual Injector, Floating gate MOS.

Gosney, W. M.↗

Intercalibration of AMSR2 NASA Team 2 Algorithm Sea Ice Concentrations with AMSR-E Slow Rotation Data

Sea ice estimates from AMSR2 are intercalibrated with AMSR-E fields through a two-step process. First, slow rotation 2 r/min AMSR-E data is used to derive regression equations from colocated pairs of AMSR2 and AMSR-E brightness temperatures (Tb s). The regression equations are used to modify AMSR2 Tb s into AMSR-E equivalent Tb s that are then input into the NASA Team 2 (NT2) sea ice concentration algorithm used for the AMSR-E standard products. The regressed Tb s result in changes in sea ice concentration of a few percent compared to using the original un-regressed AMSR2 Tb s. Next, sea ice estimates from the F17 SSMIS sensor are used as a bridge to compare AMSR-E total sea ice extent estimates in 2010 with AMSR2 total sea ice extent estimates in 2013. Based on this comparison, a further adjustment is made to a weather filter threshold used in the NT2 algorithm to minimize the total extent bias between AMSR2 and AMSR-E using a double-differencing approach. The adjustments reduced apparent bias with AMSR-E from 200 000 km2 for the original unmodified AMSR2 Tb s to –700 and 4700 km2 for the Arctic and Antarctic, respectively. These differences are within the range of previous passive microwave sea ice intercalibrations. The adjusted AMSR2 sea ice fields provide a nearly 15-year time series of sea ice change; depending on the lifetime of AMSR2 and possible follow on sensors, AMSR2 has the potential to be part of a multidecadal record of sea ice change.

remote sensing↗

Scalable Data Management of the Uintah Simulation Framework for Next-Generation Engineering Problems with Radiation

The need to scale next-generation industrial engineering problems to the largest computational platforms presents unique challenges. This paper focuses on data management related problems faced by the Uintah simulation framework at a production scale of 260K processes. Uintah provides a highly scalable asynchronous many-task runtime system, which in this work is used for the modeling of a 1000 megawatt electric (MWe) ultra-supercritical (USC) coal boiler. At 260K processes, we faced both parallel I/O and visualization related challenges, e.g., the default file-per-process I/O approach of Uintah did not scale on Mira. In this paper we present a simple to implement, restructuring based parallel I/O technique. We impose a restructuring step that alters the distribution of data among processes. The goal is to distribute the dataset such that each process holds a larger chunk of data, which is then written to a file independently. This approach finds a middle ground between two of the most common parallel I/O schemes–file per process I/O and shared file I/O–in terms of both the total number of generated files, and the extent of communication involved during the data aggregation phase. To address scalability issues when visualizing the simulation data, we developed a lightweight renderer using OSPRay, which allows scientists to visualize the data interactively at high quality and make production movies. Finally, this work presents a highly efficient and scalable radiation model based on the sweeping method, which significantly outperforms previous approaches in Uintah, like discrete ordinates. The integrated approach allowed the USC boiler problem to run on 260K CPU cores on Mira.

97 MATHEMATICS AND COMPUTING↗

Space qualified hybrid superconductor/semiconductor planar oscillator circuit

We report on the space qualification of a hybrid superconductor/semiconductor planar local oscillator (LO) at 8.4 GHz. This oscillator was designed, fabricated, and tested as a component for the High Temperature Superconductivity Space Experiment 2 (HTSSE-2). The LO consisted of a GaAs MESFET and microstrip circuitry patterned onto a YBa2Cu3O(7-delta) high temperature superconducting (HTS) thin film on a 1.0 x 1.0 sq cm lanthanum aluminate (LaAlO3) substrate. At 77 K, this oscillator achieved power output levels up to 10 dBm into a 50 Ohm load. When incorporated into a full cryogenic receiver, the LO provided output powers within 0.0-3.0 dBm with less than 50 mW of dc power dissipation. Space qualification data on the sensitivity of the HTS films to the processing steps involved in the fabrication of HTS-based components are presented. Data on ohmic contacts, strength of wire bonds made to such contacts, and aging effects as well as vibration test results are discussed.

Miranda, F. A.↗

Progress in Operational Analysis of Launch Vehicles in Nonstationary Flight

This paper presents recent results in an ongoing effort to understand and develop techniques to process launch vehicle data, which is extremely challenging for modal parameter identification. The primary source of difficulty is due to the nonstationary nature of the situation. The system is changing, the environment is not steady, and there is an active control system operating. Hence, the primary tool for producing clean operational results (significant data lengths and data averaging) is not available to the user. This work reported herein uses a correlation-based two step operational modal analysis approach to process the relevant data sets for understanding and development of processes. A significant drawback for such processing of short time histories is a series of beating phenomena due to the inability to average out random modal excitations. A recursive correlation process coupled to a new convergence metric (designed to mitigate the beating phenomena) is the object of this study. It has been found in limited studies that this process creates clean modal frequency estimates but numerically alters the damping.

James, George↗

Custom Accessors: Enabling Scalable Data Ingestion, (Re-)Organization, and Analysis on Distributed Systems

The emerging class of high velocity and high volume data analytic workflows comprise interwoven data ingestion, organization, and processing stages, with ingestion and organization steps often contributing comparable or even higher computational costs than actual processing steps. Since complex workflows consist of a variety of phases that view and use data differently, being able to construct efficient, scalable, distributed data structures (arrays, vectors, sets, maps, and multi-maps) is essential and requires custom methods to extend and shrink containers, analyze and position data, and, maintain globallyconsistent meta-data. In this paper, we propose a novel datastructure access paradigm based on the concept of Accessors. At a high level, accessors are customizable callable objects that can modify the behavior of insert, read, update, and delete operations for distributed containers while preserving atomicity guarantees. Accessors provide a very clean and natural way to implement a variety of programming patterns, e.g., conditional insertion/deletion and cascading computations, which would be otherwise hard (or even impossible) to express in parallel and distributed settings without using locks. We demonstrate the practicality and usefulness of our approach with two representative use cases and study the performance of these applications on a distributed High-Performance Computing system. Our analysis highlights that our proposed abstraction allows for an effective overlapping and concurrent execution of different workflow steps (e.g., data ingestion and analysis), which in a conventional analytics pipeline would execute sequentially, contributing cumulatively to the overall latency.

Castellana, Vito G. [BATTELLE (PACIFIC NW LAB)] (O↗

Monitoring the defoliation of hardwood forests in Pennsylvania using LANDSAT

An automated system for conducting annual gypsy moth defoliation surveys using LANDSAT MSS data and digital processing techniques is described. A two-step preprocessing procedure was developed that uses multitemporal data sets representing forest canopy conditions before and after defoliation to create a digital image in which all nonforest cover types are eliminated or masked out of a LANDSAT image that exhibits insect defoliation. A temporal window for defoliation assessment was identified and a statewide data base was established. A data management system to interface image analysis software with the statewide data base was developed and a cost benefit analysis of this operational system was conducted.

Dottavio, C. L.↗

Thermal contouring of forestry data: Wallops Island

The contouring of 8-13.5 micrometer thermal data collected over a forestry site in Virginia is described. The data were collected at an altitude of 1000 ft above terrain on November 4, 1970. The site was covered on three approximately parallel lines. The purpose of the contouring was to attempt to delineate pine trees attacked by southern pine bark beetle, and to map other important terrain categories. Special processing steps were required to achieve the correct aspect ratio of the thermal data. The reference for the correction procedure was color infrared photography. Data form and quality are given, processing steps are outlined, a brief interpretation of results is given, and conclusion are presented.

Thomson, F.↗

Asynchronous in situ connected-components analysis for complex fluid flows

Fluid flow in porous media is at the heart of problems such as groundwater contamination and carbon sequestration, and presents an important challenge for scientific computing. For this class of problem, large three-dimensional simulations are performed to advance scientific inquiry. On massively parallel computing systems, the volume of data generated by such approaches can become a productivity bottleneck if the raw data generated from the simulation is analyzed in a post-processing step. We present a physics-based framework for in situ data reduction that is theoretically grounded in multiscale averaging theory. We show how task parallelism can be exploited to concurrently perform a variety of analysis tasks with data-dependent costs, including the generation of iso-surfaces, morphological analyses, and connected components analysis. A task management framework is constructed to launch asynchronous analysis threads, manage dependencies between different tasks, promote data locality and hide the impact of data transfers. The framework is applied to analyze GPU-based simulations of two-fluid flow in porous media, generating a set of averaged measures that represents the overall system behavior. We demonstrate how the approach can be applied to perform physically-consistent averaging over fluid subregions using connected components analysis. Simulations performed on Oak Ridge National Lab s Titan supercomputer are profiled to demonstrate the performance of the associated multi-threaded in situ analysis for typical production simulation of two-fluid flow.

Mcclure, James↗

Ptychography: Pushing the Limits of X-Ray Microscopy

Ptychography is an advanced diffraction based imaging technique that can achieve resolution of 5 nm, and below. It is done by scanning a sample through a beam of focused x-rays using discrete yet overlapping scan steps. Scattering data is collected on a CCD camera, and the phase of the scattered light is reconstructed with sophisticated iterative algorithms. Because the experimental setup is similar, ptychography setups can be created by retrofitting existing STXM beam lines with new hardware. The other challenge comes in the reconstruction of the collected scattering images. Scattering data must be adjusted and packaged with experimental parameters to calibrate the reconstruction software. The necessary pre-processing of data prior to reconstruction is unique to each beamline setup, and even the optical alignments used on that particular day. Pre-processing software must be developed to be flexible and efficient in order to allow experimenters appropriate control and freedom in the analysis of their hard-won data. This paper will describe the implementation of pre-processing software which successfully connects data collection steps to reconstruction steps, letting the user accomplish accurate and reliable ptychography.

47 OTHER INSTRUMENTATION↗

Coupling fracture facies with in-situ permeability measurements to generate stochastic simulations of tight carbonate aquifer properties: Example from the Lower Cretaceous aquifer, Northern Provence, SE France

The relationships between fracture facies and permeability distribution have been explored using a well-constrained 3D implicit structural model of hemipelagic low porosity/low permeability carbonate from the Northern Provence region, SE France. Fracture and permeability facies were determined using an extensive dataset of 99 hydrogeological wells. Data processing was undertaken using a step-by-step approach, involving: (i) identification of the fracture facies based on well logs detailing structure and stratigraphy; (ii) determination of permeability facies from the a priori correlation between the dimension of the hydraulic radius of influence (deduced from slug test analyses) and the type of reservoir heterogeneity (fissure, fracture, fault zone, etc.); (iii) three dimensional plot of fracture and permeability facies in the geological model using a variographic analysis of data. Thirty-three sequential indicator simulations (SIS) based on geostatistic analyses were realised on both fracture and permeability facies. Finally, a connectivity algorithm was developed to compute the probability of connection between selected infiltration areas and the major aquifer springs via moderate- to high-permeability geological bodies. Key results are: (i) fault zones have a greater role in controlling permeability facies distribution than on fracture facies repartition; (ii) there is little correlation between permeability and fracture facies distributions; (iii) connectivity results highlight the compartmentalisation of aquifers in the Cadarache area, the extensions of permeable geological bodies being limited by the N130 faults.

58 GEOSCIENCES↗

Optimizing a Small RNAseq Analysis Pipeline for NASA GeneLab Using Open-Source Tools and Libraries

Small RNA sequencing (small RNAseq) is a powerful tool for studying the regulation of gene expression in various organisms. Small RNAseq has been leveraged in space biology research to study how expression of small RNAs, e.g. micro RNAs (miRNAs), small interfering RNAs (siRNAs), and piwi-interacting RNAs (piRNAs), change upon exposure to the space environment. NASA GeneLab currently hosts small RNAseq raw data derived from space-relevant experiments on the Open Science Data Repository (OSDR). To maximize the accessibility of these data to the scientific community, in addition to hosting raw data, which is only interpretable by bioinformaticians, GeneLab plans to process all small RNAseq datasets and make those processed data available to the scientific community via the OSDR. In this study, we present the development of the GeneLab standardized pipeline for processing small RNAseq datasets. Using human, plant, and synthetic small RNAseq datasets, we interrogate various open-source software and publicly available databases to evaluate their accuracy and reproducibility in each step of the pipeline. For quality control and adapter detection and trimming, we evaluated TrimGalore!, FASTX, SeqKit, and DNApi methods to optimize alignment to reference genomes. We compared BWA, Bowtie, and Bowtie2 to determine the optimal alignment tool. For each alignment tool we also assessed various reference databases, including Ensembl reference genomes and different types of small RNA reference databases, including genome, hairpin, and miRNA references from the miRbase and MirGeneDB databases. To quantify the aligned data, we compared SAMtools, HTSeq, and RSEM for counting alignment events from each alignment tool used. Finally, we evaluated various tools, including DESeq2 and EdgeR, for data normalization and subsequent differential expression analysis. We will present the results from our comparative analyses for each pipeline step and propose a consensus pipeline for processing small RNAseq data derived from various organisms exposed to the space environment.

SmallRNAseq, NASA GeneLab, quality control, adapte↗

Performance characterization of irregular I/O at the extreme scale

Here, this paper reports our experience with irregular I/O and describes lessons learned when running applications with such I/O on supercomputers at the extreme scale. Specifically, we study how irregularities in I/O patterns (i.e., irregular amount of data written per process at each I/O step) in scientific simulations can cause increasing I/O times and substantial loss in scalability. To this end, we quantify the impact of irregular I/O patterns on the I/O performance of scientific applications at the extreme scale by statistically modeling the irregular I/O behavior of two scientific applications: the Monte Carlo application QMCPack and the adaptive mesh refinement application ENZO. For our testing, we feed our model into I/O kernels of two well-known I/O data models (i.e., ADIOS and HDF) to measure the performance of the two applications’ I/O under different I/O settings. Empirically, we show how the growing data sizes and the irregular I/O patterns in these applications are both relevant factors impacting performance.

97 MATHEMATICS AND COMPUTING↗

Thermal Infrared Radiometric Calibration of the Entire Landsat 4, 5, and 7 Archive (1982-2010)

Landsat's continuing record of the thermal state of the earth's surface represents the only long term (1982 to the present) global record with spatial scales appropriate for human scale studies (i.e., tens of meters). Temperature drives many of the physical and biological processes that impact the global and local environment. As our knowledge of, and interest in, the role of temperature on these processes have grown, the value of Landsat data to monitor trends and process has also grown. The value of the Landsat thermal data archive will continue to grow as we develop more effective ways to study the long term processes and trends affecting the planet. However, in order to take proper advantage of the thermal data, we need to be able to convert the data to surface temperatures. A critical step in this process is to have the entire archive completely and consistently calibrated into absolute radiance so that it can be atmospherically compensated to surface leaving radiance and then to surface radiometric temperature. This paper addresses the methods and procedures that have been used to perform the radiometric calibration of the earliest sizable thermal data set in the archive (Landsat 4 data). The completion of this effort along with the updated calibration of the earlier (1985 1999) Landsat 5 data, also reported here, concludes a comprehensive calibration of the Landsat thermal archive of data from 1982 to the present

Schott, John R.↗