Search NASA⌕ Search

SEARCH · Search NASA

Results for “data ingest”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Data Product: Ka ARM Zenith Radar (KAZR), in CfRadial format: precipitation mode, corrected

The KAZRCFRCOR Value Added Product (VAP) performs several corrections to the ingested KAZR data and also creates a significant detection mask for each radar mode. The VAP computes gaseous attenuation as a function of time and radial distance from the radar antenna, based on ambient relative humidity, and corrects observed reflectivities for that effect. KAZR reflectivities are also corrected in the antenna ‘nearfield’ range to compensate for the near-field antenna pattern and gain. KAZRCFRCOR also dealiases mean Doppler velocities, to correct velocities whose magnitudes exceed the radar’s Nyquist velocity. This dataset contains the 'kazrpr' (precipitation) mode output from this VAP.

54 ENVIRONMENTAL SCIENCES↗

Data Product: Ka ARM Zenith Radar (KAZR), in CfRadial format: general mode, corrected

The KAZRCFRCOR Value Added Product (VAP) performs several corrections to the ingested KAZR data and also creates a significant detection mask for each radar mode. The VAP computes gaseous attenuation as a function of time and radial distance from the radar antenna, based on ambient relative humidity, and corrects observed reflectivities for that effect. KAZR reflectivities are also corrected in the antenna ‘nearfield’ range to compensate for the near-field antenna pattern and gain. KAZRCFRCOR also dealiases mean Doppler velocities, to correct velocities whose magnitudes exceed the radar’s Nyquist velocity. This dataset contains the 'kazrge' (general) mode output from this VAP.

54 ENVIRONMENTAL SCIENCES↗

Navigating Exascale Operational Data Analytics: From Inundation to Insight

In this paper, we address the challenges in achieving sustainable data-driven efficiency by providing a detailed exploration of the end-to-end operational data analytics (ODA) framework that evolved through two generations of supercomputer systems at the Oak Ridge Leadership Computing Facility (OLCF). This framework addresses large data streams ingested from heavily instrumented HPC environment that accumulates multi-terabytes per day. We outline the multifaceted data life cycle across HPC procurement, operations, and research & development, identifying key obstacles and design decisions that shape effective strategies in building and supporting data pipelines end-to-end. By sharing key insights and lessons learned from our experience, we offer recommendations for the HPC community on enabling sustainable operational data analytics and beyond. Our contributions aim to bridge the gap between potential and real benefits of operational data, guiding future efforts towards integrated and sustainable operational intelligence in high-performance computing environments.

Shin, Woong↗

Data Product: Ka ARM Zenith Radar (KAZR), in CfRadial format: moderate sensitivity mode, corrected

The KAZRCFRCOR Value Added Product (VAP) performs several corrections to the ingested KAZR data and also creates a significant detection mask for each radar mode. The VAP computes gaseous attenuation as a function of time and radial distance from the radar antenna, based on ambient relative humidity, and corrects observed reflectivities for that effect. KAZR reflectivities are also corrected in the antenna ‘nearfield’ range to compensate for the near-field antenna pattern and gain. KAZRCFRCOR also dealiases mean Doppler velocities, to correct velocities whose magnitudes exceed the radar’s Nyquist velocity. This dataset contains the 'kazrmd' (moderate sensitivity) mode output from this VAP.

54 ENVIRONMENTAL SCIENCES↗

Design and implementation of a data analysis pipeline for paper spray ionization mass spectrometry of UO 2 Cl 2

Ambient mass spectrometry is an emerging characterization method for nuclear materials analysis. SRNL has recently acquired a new JEOL AccuTOF™-DART® LP 4G with a Filter Spray™ front end sample injection system, to develop new methods for ambient mass spectrometry of uranium analytes. Preliminary measurements of uranium-containing samples have demonstrated a need to rapidly evaluate large datasets generated with tailored data analysis scripts. To address this gap a data analysis pipeline written in R within the R Studio environment has been developed. The pipeline comprises a series of processing steps where the data are ingested, analyte peaks of interest are extracted, subsequent ratio and normalization calculations are performed, and results are presented in a series of graphical plots. In detail, the data files containing up to several hundred chronological spectra of a single sample in netCDF format were first extracted into RStudio. Then, initial 3D plots of the data were created in respect to the time, intensity, and mass to charge domains. These graphics serve as an initial easily visualized survey of instrument data. Next, a .csv format file containing the names, masses, and abundances of isotopes of interest were ingested into RStudio and were used to automatically collect intensity values at the mass value. Uranium isotope ratios were plotted against gadolinium isotope ratios (gadolinium was used as an internal standard) and assessed for ion beam stability and intensity. The spectra were filtered by applying a threshold intensity for the 158GdO+ mass channel. A Monte Carlo experiment was performed by creating a theoretical distribution of the data in order to determine the ratio of the uranium isotopes. The results show that the developed data analysis pipeline successfully filters high quality data from overall poor quality runs. Our results also support the idea that paper spray ionization mass spectrometry can be used for both detection and quantification of trace amounts of uranium compounds.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Low cost, microcomputer-based interactive analysis system for direct-reception and archived remote sensing data

A hardware/software system has been developed for use in meteorological field experiments and by developing countries which need low-cost remote sensing data analysis capabilities. The system can directly receive satellite transmissions of the AVHRR, TOVS, Argos, GOES, GMS, and Meteosat systems. Also, the system can process computer tapes from Landsat. Received data are ingested directly into the computer's disk storage, eliminating the need for a wide-band analog tape drive. Data may be interpreted in terms of cloud imagery, cloudtop temperature, sea surface temperature, vegetation index, and temperature/humidity profiles with overlays of geographic boundaries and latitude/longitude grids. The image processing, enhancement functions, and modeling capabilities of the system are discussed.

Vermillion, Charles H.↗

The North Alabama Lightning Warning Product

The North Alabama Lightning Mapping Array NALMA has been collecting total lightning data on storms in the Tennessee Valley region since 2001. Forecasters from nearby National Weather Service (NWS) offices have been ingesting this data for display with other AWIPS products. The current lightning product used by the offices is the lightning source density plot. The new product provides a probabalistic, short-term, graphical forecast of the probability of lightning activity occurring at 5 min intervals over the next 30 minutes . One of the uses of the current lightning source density product by the Huntsville National Weather Service Office is to identify areas of potential for cloud-to-ground flashes based on where LMA total lightning is occurring. This product quantifies that observation. The Lightning Warning Product is derived from total lightning observations from the Washington, D.C. (DCLMA) and North Alabama Lightning Mapping Arrays and cloud-to-ground lightning flashes detected by the National Lightning Detection Network (NLDN). Probability predictions are provided for both intracloud and cloud-to-ground flashes. The gridded product can be displayed on AWIPS workstations in a manner similar to that of the lightning source density product.

Buechler, Dennis E.↗

NASA Earth Sciences Data Support System and Services for the Northern Eurasia Earth Science Partnership Initiative

The presentation describes data management of NASA remote sensing data for Northern Eurasia Earth Science Partnership Initiative (NEESPI). Many types of ground and integrative (e.g., satellite, GIs) data will be needed and many models must be applied, adapted or developed for properly understanding the functioning of Northern Eurasia cold and diverse regional system. Mechanisms for obtaining the requisite data sets and models and sharing them among the participating scientists are essential. The proposed project targets integration of remote sensing data from AVHRR, MODIS, and other NASA instruments on board US- satellites (with potential expansion to data from non-US satellites), customized data products from climatology data sets (e.g., ISCCP, ISLSCP) and model data (e.g., NCEPNCAR) into a single, well-architected data management system. It will utilize two existing components developed by the Goddard Earth Sciences Data & Information Services Center (GES DISC) at the NASA Goddard Space Flight Center: (1) online archiving and distribution system, that allows collection, processing and ingest of data from various sources into the online archive, and (2) user-friendly intelligent web-based online visualization and analysis system, also known as Giovanni. The former includes various kinds of data preparation for seamless interoperability between measurements by different instruments. The latter provides convenient access to various geophysical parameters measured in the Northern Eurasia region without any need to learn complicated remote sensing data formats, or retrieve and process large volumes of NASA data. Initial implementation of this data management system will concentrate on atmospheric data and surface data aggregated to coarse resolution to support collaborative environment and climate change studies and modeling, while at later stages, data from NASA and non-NASA satellites at higher resolution will be integrated into the system.

Leptoukh, Gregory↗

NASA Earth Sciences Data Support System and Services for the Northern Eurasia Earth Science Partnership Initiative

The presentation describes the recently awarded ACCESS project to provide data management of NASA remote sensing data for the Northern Eurasia Earth Science Partnership Initiative (NEESPI). The project targets integration of remote sensing data from MODIS, and other NASA instruments on board US-satellites (with potential expansion to data from non-US satellites), customized data products from climatology data sets (e.g., ISCCP, ISLSCP) and model data (e.g., NCEP/NCAR) into a single, well-architected data management system. It will utilize two existing components developed by the Goddard Earth Sciences Data & Information Services Center (GES DISC) at the NASA Goddard Space Flight Center: (1) online archiving and distribution system, that allows collection, processing and ingest of data from various sources into the online archive, and (2) user-friendly intelligent web-based online visualization and analysis system, also known as Giovanni. The former includes various kinds of data preparation for seamless interoperability between measurements by different instruments. The latter provides convenient access to various geophysical parameters measured in the Northern Eurasia region without any need to learn complicated remote sensing data formats, or retrieve and process large volumes of NASA data. Initial implementation of this data management system will concentrate on atmospheric data and surface data aggregated to coarse resolution to support collaborative environment and climate change studies and modeling, while at later stages, data from NASA and non-NASA satellites at higher resolution will be integrated into the system.

Leptoukh, Gregory↗

A New ML-Based Adaptive Thinning Methodology to Improve the Impact of AIRS and CrIS Assimilation on Global Tropical Cyclone Forecasts

This work builds on previous research performed by this team to improve the forecast of Tropical Cyclones (TCs) by assimilating AIRS and CrIS radiances into the NASA Global Earth Observing System (GEOS). Past published work demonstrated that the assimilation of radiances with variable density was beneficial to TC forecasting in the GEOS. In the previous setup, a fixed-size moving square named 'TC domain' was activated by the so-called TC-vitals, an international real-time message accessible to all NWP forecasting centers, that documents the existence of a TC, its estimated position, and its size. The information from TC-vitals activated a switch in the GEOS, which allowed to reduce the distance used for thinning AIRS and CrIS data inside a 15 degrees by 15 degrees moving TC domain centered on the storm, so that more data were assimilated in the vicinity of the TC during its lifetime. The methodology produced improved TC analyses and led to better forecasts, particularly related to intensity, without damaging the global forecast skill. In the new version, the adaptive thinning methodology is based on a machine-learning technique. The technique searches for TCs and creates TC masks by using cloud-top temperatures from all geostationary satellites without the need for additional information. It is being trained against the International Best Track Archive for Climate Stewardship (IBTrACS) data base. Once a TC mask is created, a switch identical to the one used in the previous adaptive thinning method is activated, allowing the GEOS to ingest more data in the TC-shaped size-changing domain that follows the storm. As of today, the team has been able to successfully assimilate data inside the ML-detected TC domains. Future work includes an improved capability of reducing false alarm rates (i.e., cloud systems that are erroneously labeled as TCs).

Oreste Reale↗

Heliophysical Modeling at the CCMC - Community Modeling Activities to Compliment the VHGO

The Community Coordinated Modeling Center (CCMC) hosts an ever growing inventory of models to support the research activities of the Heliophysics community.In this poster we detail this model inventory. We describe the manner in which the CCMC provides access to these models to the community. This support includes model runs driven with archived data and 'realtime' runs which update as the latest data is ingested by the models. It includes runs for individual researchers and in support of observational planning and analysis for a number of flight missions. Our need to integrate the data streams into and out of numerous models and graphics packages has led to the development of a number of infra-structure component that are also highly relevant to the design of the VHGO. We discuss this issue and the natural and vital link that must develop between the VHGO and modeling centers such as the CCMC, if the usefulness of the VHGO is to be maximized.

MacNeice, P.↗

Airspace Technology Demonstration 2 (ATD-2) Phase 2 Technology Description Document (TDD)

This Technology Description Document (TDD) provides an overview of the technology for the Phase 1 Baseline and Phase 2 Fused demonstrations of the Integrated Arrival, Departure, and Surface (IADS) prototype system of the National Aeronautics and Space Administration’s (NASA) Airspace Technology Demonstration 2 (ATD-2) project, which has been demonstrated since October of 2017 at Charlotte Douglas International Airport (CLT). Development, integration, and field demonstration of relevant technologies of the IADS system directly address recommendations made by the Next Generation Air Transportation System (NextGen) Integration Working Group (NIWG) on Surface and Data Sharing and the Surface Collaborative Decision Making (Surface CDM) concept of operations developed jointly by the Federal Aviation Administration (FAA) and aviation industry partners. NASA has been developing the IADS traffic management system under the ATD-2 project in coordination with the FAA, flight operators, CLT airport operators, and the National Air Traffic Controllers Association (NATCA). The primary goal of ATD-2 is to improve the predictability and operational efficiency of the air traffic system in metroplex environments, through the enhancement, development, and integration of the nation’s most advanced and sophisticated arrival, departure, and surface prediction, scheduling, and management systems. The ATD-2 project is a 5-year research activity beginning in 2015 and extending through 2020. The Phase 1 Baseline IADS capability resulting from the ATD-2 research has been demonstrated at the CLT airport since the beginning of October 2017. Phase 1 provided data exchange and integration with the initial demonstration of the integrated system of strategic and tactical scheduling, collaborative tactical surface metering, tactical departure scheduling to an en route meter point and overhead stream insertion, initial integration of a TFDM SWIM prototype feed, and an early implementation prototype of a Terminal Flight Data Manager (TFDM) Electronic Flight Data (EFD) system. The strategic surface scheduling element of the capability is consistent with the Surface CDM Concept of Operations published in 2014 by the FAA Surface Operations Directorate.1 The Phase 2 Fused IADS system provided substantial updates to baseline IADS Demonstration capability. Improvements were made to the IADS modeler; tactical surface scheduling, and metering; tactical departure scheduling for overhead stream insertion; Electronic Flight Data (EFD) and the integration with the FAA Advanced Electronic Flight Strip (AEFS); RTC/RMTC, and departure trajectories. New capabilities included a TFDM SWIM prototype feed and delivery of IADS data via the TFDM Terminal Publication (TTP) service of the FAA’s SWIM system; strategic surface scheduling and metering as well as fusion with tactical surface metering; integration and ingestion of data from TTP-connected Mobile App and texting industry for General Aviation (GA) flights; and expansion of airspace deployments to include adjacent Center automation, in particular interface with the Atlanta Center (ZTL) arrival metering TBFM system to evaluate pre-scheduling of flights. Improvements in Data Analysis and System Health (DASH) monitoring and updates were completed as well.

Talebi, Ehsan↗

Program for Analysis and Enhancement of Images

Land Analysis System (LAS) is collection of image-analysis computer programs designed to manipulate and analyze multispectral image data. Provides user with functions ingesting various sensor data, radiometric and geometric corrections, image registration, training site selection, supervised and unsupervised classification, Fourier domain filtering, and image enhancement. Sufficiently modular and includes extensive library of subroutines to permit inclusion of new algorithmic programs. Commercial package International Mathematical & Statistical Library (IMSL) required for full implementation of LAS. Written in VAX FORTRAN 77, C, and Macro assembler for DEC VAX operating under VMS 4.0.

Lu, Yun-Chi↗

24-hour human urine and serum profiles of Bisphenol A following ingestion in soup: Individual pharmacokinetic data and emographics

Here we present data to evaluate potential absorption of Bisphenol A through non-metabolizing tissues of the upper digestive tract. Concurrent serum and urine concentrations of d6-BPA, and its glucuronide and sulfate conjugates, were measured over a 24 h period in 10 adult male volunteers following ingestion of 30 μg d6-BPA/kg body weight in soup. The pharmacokinetic behavior of BPA and its metabolites in this cohort (rapid absorption, complete elimination, evidence against sublingual absorption) was reported. This Data in Brief article contains the corresponding individual pharmacokinetic data, reports the demographics of the cohort and provides additional details related to the analytical methods employed and is related to [4].

59 BASIC BIOLOGICAL SCIENCES↗

Enhancing Clean Energy Innovation Ecosystem Discovery Tool

During March 2016, the DOE’s Office of Energy Policy and Systems Analysis (EPSA) sought a methodology to identify and quantify the strength of the existing clean energy innovation ecosystems (IE) in the U.S. and their characteristics. Oak Ridge National Lab (ORNL) did a pilot study (Phase 1) and demonstrated the feasibility of an application comprised of natural language processing, link analysis, and other computational techniques to transform text and numerical data into metrics on clean energy innovation activity and geography. The data-collection, ingest and analytics pipelines were combined with an advanced user interface, together known as the Ecosystem Discovery Tool, to enhance DOE’s understanding of existing geographic innovation clusters. During Phase 2 of this project, this tool has been further enhanced by integrating new data sets and through backend software architecture changes, validation and bug fixing, making it a much more robust and powerful application. The tool’s user interface is a lot more intuitive, which enables a user to visualize the IE rankings for various geographic regions(CBSA, state level) for different clean energy technologies and seek automated insights. ORNL also created a DOE-specific dashboard that allows the user to visualize and analyze federal funding from various offices including Energy Efficiency and Renewable Energy(EERE), Office of Electricity(OE), Fossil Energy(FE), and Nuclear Energy(NE). During Phase 1 of this project, EPSA defined a clean energy innovation ecosystem as the overlap of five Ecosystem Components: 1) nascent clean energy Indicators, 2) investors, 3) enabling environment, 4) networking assets and 5) large companies. EPSA and ORNL worked together to collect data for each component: 1) small and medium companies, ARPA-E awardees, SBIR awardees, patents, publications, ERCs for Nascent Clean Energy Indicators; 2) qualified investors; 3) the Clean Edge Policy Index/DSIRE for the Enabling Environment; 4) universities, national laboratories, ERCs and incubators for Networking Assets; and 5) large companies and a subset of the Russell 1000 list for Large Companies. The ORNL team created a visual tool based on Tableau that integrated ecosystem component data to score, rank and map IEs. The tool was created with the flexibility to allow the user to choose the weights of each of the five ecosystem components and the subcomponents. This flexibility allows the user to visualize different subsets and to use the underlying data for different types of analysis. During Phase 2 of this project, the backend database and computation process have undergone major changes to provide a much more refined user-interface and added new features into the scoring algorithm. This phase also involved addition of newer databases such as a list of Energy Research Centers (ERC) ,DSIRE, and funding data from EERE, FE, NE, and OE to provide more granular information and more accurate/better quantification.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Data Model Management for Space Information Systems

The Reference Architecture for Space Information Management (RASIM) suggests the separation of the data model from software components to promote the development of flexible information management systems. RASIM allows the data model to evolve independently from the software components and results in a robust implementation that remains viable as the domain changes. However, the development and management of data models within RASIM are difficult and time consuming tasks involving the choice of a notation, the capture of the model, its validation for consistency, and the export of the model for implementation. Current limitations to this approach include the lack of ability to capture comprehensive domain knowledge, the loss of significant modeling information during implementation, the lack of model visualization and documentation capabilities, and exports being limited to one or two schema types. The advent of the Semantic Web and its demand for sophisticated data models has addressed this situation by providing a new level of data model management in the form of ontology tools. In this paper we describe the use of a representative ontology tool to capture and manage a data model for a space information system. The resulting ontology is implementation independent. Novel on-line visualization and documentation capabilities are available automatically, and the ability to export to various schemas can be added through tool plug-ins. In addition, the ingestion of data instances into the ontology allows validation of the ontology and results in a domain knowledge base. Semantic browsers are easily configured for the knowledge base. For example the export of the knowledge base to RDF/XML and RDFS/XML and the use of open source metadata browsers provide ready-made user interfaces that support both text- and facet-based search. This paper will present the Planetary Data System (PDS) data model as a use case and describe the import of the data model into an ontology tool. We will also describe the current effort to provide interoperability with the European Space Agency (ESA)/Planetary Science Archive (PSA) which is critically dependent on a common data model.

RDF↗

Calibrating constitutive models with full‐field data via physics informed neural networks

Abstract The calibration of solid constitutive models with full‐field experimental data is a long‐standing challenge, especially in materials that undergo large deformations. In this paper, we propose a physics‐informed deep‐learning framework for the discovery of hyperelastic constitutive model parameterizations given full‐field surface displacement data and global force‐displacement data. Contrary to the majority of recent literature in this field, we work with the weak form of the governing equations rather than the strong form to impose physical constraints upon the neural network predictions. The approach presented in this paper is computationally efficient, suitable for irregular geometric domains, and readily ingests displacement data without the need for interpolation onto a computational grid. A selection of canonical hyperelastic material models suitable for different material classes is considered including the Neo–Hookean, Gent, and Blatz–Ko constitutive models as exemplars for general non‐linear elastic behaviour, elastomer behaviour with finite strain lock‐up, and compressible foam behaviour, respectively. We demonstrate that physics informed machine learning is an enabling technology and may shift the paradigm of how full‐field experimental data are utilized to calibrate constitutive models under finite deformations.

Hamel, Craig M.↗

Towards A Representation of Vertically Resolved Ozone Changes in Reanalyses

The Solar Backscatter Ultraviolet Radiometer (SBUV) instruments on NASA and NOAA spacecraft provide a long-term record of total-column ozone and deep-layer partial columns since about 1980. These data have been carefully processed to extract long-term trends and offer a valuable resource for ozone monitoring. Studies assimilating limb-sounding observations in the Goddard Earth Observing System (GEOS) data assimilation system (DAS) demonstrate that vertical ozone gradients in the upper troposphere and lower stratosphere (UTLS) are much better represented than with the deep-layer SBUV observations. This is exemplified by the use of retrieved ozone from the EOS Microwave Limb Sounder (EOS-MLS) instrument in the MERRA-2 reanalysis, for the period after 2004. This study examines the potential for extending the use of limb-sounding observations at earlier times and into the future, so that future reanalyses may be more applicable to the study of long-term ozone changes.Historical data are available from NASA instruments: the Limb Infrared Monitor of the Stratosphere (LIMS: 1978-1979); the Upper Atmospheric Research Satellite (UARS: 1991-1995); Sounding of the Atmosphere using Broadband Emission Radiometry (SABER: 2000-onwards). For the post EOS-MLS period, the joint NASA-NOAA Ozone Monitoring and Profiling Suite Limb Profiler (OMPS-LP) instrument was launched on the Suomi-NPP platform in 201x and is planned for future platforms. This study will examine two aspects of these data pertaining to future reanalyses. First, the feasibility of merging the EOS-MLS and OMPS-LP instruments to provide a long-term record that extends beyond the potential lifetime of EOS-MLS. If feasible, this would allow for long-term monitoring of ozone recovery in a three-dimensional reanalysis context. Second, the skill of the GEOS DAS in ingesting historical data types will be investigated. Because these do not overlap with EOS-MLS, use will be made of system statistics and evaluation using independent datasets. Impacts of using a complete ozone chemistry module will also be considered.

ML↗