Search NASA⌕ Search

SEARCH · Search NASA

Results for “data ingest”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

FInd Gas Source

FIGS is a neural network software that ingests real time synchronized field data on environmental flow fields and turbulence and gas concentration variations at high frequency and uses an error minimization algorithm to locate the gas source and quantify its strength. The software can be interfaced with atmospheric, oceanic and subsurface instruments in a variety of platforms stationary or mobile (e.g. cars, UAVs, submersible vehicles or boreholes) and used to find gas sources by smart use of data and phenomenology. FIGS can be trained by phenomenological model of the flow fields in the environment of interest and/or be calibrated by controlled release. After initial deployment the FIGS learning will grow with time as it accumulates data on source quantification. FIGS can be installed on any computer from small beagle-bones for field deployment/end-use to PC/MACs/main-frame for training/analysis. FIGS has been trained (using LANL's high resolution atmospheric simulations) and calibrated, tested and evaluated in the field and shown to perform well in finding and quantifying methane leaks at 10-100m scales at well pads by ingesting atmospheric measurements. The code is applicable to gas and particle source location at large scales.

Travis, Bryan↗

Impact of Assimilating AIRS Cloud-Cleared Radiances on Atmospheric Dynamics and Polar Low Representation at High Latitudes

This study explores the sensitivity of planetary boundary layer height and related atmospheric dynamics to the assimilation of cloud-cleared AIRS (Atmospheric Infrared Sounder) radiances in the Goddard Earth Observing System (GEOS, version 5) data assimilation and forecast system during the boreal fall 2014 season using observing system experiments (OSEs). Examined here are comparisons between the current, operational approach of assimilating AIRS clear-sky radiances against the assimilation of cloud-cleared radiances (CCR). In polar regions, assimilation of AIRS CCRs is particularly beneficial because of the sparsity of conventional observations and the prevalence of extended low-level stratus cloud cover, which limit the ingestion of clear-sky data. Assimilation of hyperspectral infrared information from AIRS over the Arctic region slightly modifies the lower midtropospheric temperature structure, which in turn contributes to adjustments in geopotential height, affecting the baroclinic instability properties over the entire hemisphere and explaining the overall improvement in global forecast skill. In addition, it is shown that the assimilation of CCRs benefits the representation of convectively-driven small-scale cyclones at high latitudes in the same way as previously noted for tropical cyclones. Specifically, assimilation of CCRs create a temperature dipole over the top of meteorologically active and strongly convective systems such as polar, arctic, and antarctic lows, which helps constrain the analyzed representation of their scale and vertical structure.

Mcgrath-Spangler, E. L.↗

An Automated Approach to Labelling Datasets in Earth Science Publications

NASA Data Active Archive Centers, orDAACs, ingest, store, and distribute dataacquired from satellites, ground systems as well asreanalysis models. Many authors use this datain their research. However, most of the datasets usedin Earth Science Publications are not citedcorrectly or not cited at all. Thus, there is no directlink between the datasets used and thescientific publications which reference them. Thisleads to issues with reproducibility of theresults, attribution of the research results, anddiscovery of new datasets. This project began byexploring various methods of automatically labellingGoddard Earth Sciences Data andInformation Services Center (GES DISC) datasets usingSupervised Machine Learning and EarthData Search Common Metadata Repository (CMR) queries.The ultimate goal was to create alibrary of citations that utilized automated citationlabeling to directly link the researchpublications to the data they use. Supervised MachineLearning approaches struggled due to thelimited amount of labelled training data to learnfrom. Increasing the volume of training data isdifficult as it requires subject matter experts todevote time to manually reviewing journalarticles and determining the datasets used. The CMRqueries were inconsistent because theunderlying metadata is continuously being updated.Thus, it is hard to generalize theeffectiveness of the CMR results as they are dependenton the internal state of CMR. Theseapproaches helped inform the decision to transitionthe project into using a Knowledge Graph.Another key aspect of this project focused on theautomated extraction of features (platform,instrument, variables, etc) and explicit citationsfrom within Earth Science Publications. Theseautomated extractions were used to classify researchpapers based on their platform/instrumentcouples. This information was input into the CitationManagement System for GES DISC. Theseplatform/instrument couples also provide an additionalfacet that can be searched on the GESDISC website.

Edward Jahoda↗

FY19Q3 LL17-GEOVIS-PD3SS

NFIX has been licensed to ORNL, LANL, SNL and LBNL for use on the MINOS venture. We continue to meet with the MINOS data team to receive feedback and suggestions for additional features. We are accommodating feature requests within our resources. • A new feature was added to NFIX that lists out sensor metadata files (json files) that NFIX is not able to ingest due to formatting errors. This feature has greatly eased the effort required by the MINOS team to identify errant json files. Our summer student used this new feature to determine the number of sensor data metafiles that NFIX was not ingesting (a small number) and found the root cause to be a particular set of formatting problems. The MINOS team is using this information to correct the mis-formatted files. Our summer student also developed a tutorial for the NFIX users. She was in a unique position to develop a tutorial as she had to come up to speed on the tool in order to use it for our data ingest study. The tutorial is planned to appear as an appendix in the project final report. A white paper on using GeoVisipedia technology for detecting changes at portals using satellite imagery was submitted to NA21. The technique uses GeoVisipedia’s model projection technology as a view-invariant segment technique thereby greatly enhancing the robustness of detecting changes. We demonstrated NFIX to our program manager and his technical advisor in April. Per headquarters direction we were not required to present at a DNN R&D Joint Program Review this year.

97 MATHEMATICS AND COMPUTING↗

A Summary of Proposed Changes to the Current ICARTT Format Standards and their Implications to Future Airborne Studies

The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. The ASDC specializes in atmospheric data that is important to understanding the causes and processes of global climate change and the consequences of human activities on the climate. The ASDC currently supports more than 44 projects and has over 1,700 archived data sets, which increase daily. ASDC customers include scientists, researchers, federal, state, and local governments, academia, industry, and application users, the remote sensing community, and the general public.

Northup, Emily↗

Toolsets for Airborne Data (TAD): Improving Machine Readability for ICARTT Data Files

The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. The ASDC specializes in atmospheric data that is important to understanding the causes and processes of global climate change and the consequences of human activities on the climate. The ASDC currently supports more than 44 projects and has over 1,700 archived data sets, which increase daily. ASDC customers include scientists, researchers, federal, state, and local governments, academia, industry, and application users, the remote sensing community, and the general public.

Early, Amanda Benson↗

TPSAS-NF1676L-16802-DND

The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. Currently, the ASDC supports more than 44 projects and has over 1,700 archived data sets, which increase daily. The ASDC’s implementation mission with iRODS is to establish a consolidated, shared services capability that will provide higher quality, more cost effective and efficient services, and greater access to our current science community users and future diverse customers.

Tiffany Mathews↗

TPSAS-NF1676L-16824-DND

The Atmospheric Science Data Center (ASDC) at NASA Langley Research Center is responsible for the ingest, archive, and distribution of NASA Earth Science data in the areas of radiation budget, clouds, aerosols, and tropospheric chemistry. Currently, the ASDC supports more than 44 projects and has over 1,700 archived data sets, which increase daily. The ASDC?s implementation mission with iRODS is to establish a consolidated, shared services capability that will provide higher quality, more cost effective and efficient services, and greater access to our current science community users and future diverse customers.

Tiffany Mathews↗

The NASA Open Science Data Repository: Biomedical Data, Analysis Tools, and Informatic Collaborations

Increased biomedical risks and challenges associated with deep space missions require knowledge discovery, health countermeasures, and biomedical support capabilities. Maximally open-access and reusable data is needed by developers, scientists, and engineers to develop these systems. The NASA Open Science Data Repository (OSDR) is a maximally open access and FAIR database (ie., findable, accessible, interoperable, and reusable), and meets various scientific, technical, and operational needs. It offers users and submitters the ability to upload, download, search, share, analyze, cite, and visualize data across ‘omics, physiological, phenotypic, payload, hardware, behavioral, bioimaging, video, and environmental monitoring telemetry datasets. OSDR is an expanded database, based upon the successes of NASA GeneLab. OSDR has >460 studies with datasets covering model organisms to non-NASA human astronauts. There are ~12 datasets from the Inspiration 4 (I4) mission, spanning metagenomics, comprehensive metabolic panels, clonal hematopoiesis, spatial transcriptomics, proteomics, and cytokine panels. In the interest of data privacy, two I4 datasets with raw files relating to the epitranscriptome, and a new request feature is live in OSDR (with a backend review process established) developed from industry norms. OSDR is collecting and curating biomedical human data from a new sub-orbital research flight and is open to more space life science/biomedical submissions from the international and commercial sectors. OSDR also recently began a collaboration with the European Space Agency (ESA) to collect and curate >200 terabytes of human and model organism data. The OSDR submission portal is designed to ingest and curate ~25 ‘omics and ~50 physiological-phenotypic-imaging assay data types. Tools available for OSDR users include: 1) an Environmental Data Application to compare radiation, CO2, relative humidity, temperature, and other telemetry across missions and subjects, 2) the RadLab database, a collaboration between NASA, ESA, the German and Italian Space Agencies, and the Bulgarian Academy of Sciences, and 3) a Multi-study visualization tool which enables users to look across and combine ‘omics datasets. There are ~600 volunteer OSDR Analysis Working Group (AWG) members providing feedback on scientific data/metadata standards and collaborating to mine-reuse OSDR in research. OSDR/GeneLab has enabled ~60 publications reusing data as of October 2023.

space biology↗

The Experimental Regional Ensemble Forecast System (ExREF): Its Use in NWS Forecast Operations and Preliminary Verification

The Experimental Regional Ensemble Forecast (ExREF) system is a tool for the development and testing of new Numerical Weather Prediction (NWP) methodologies. ExREF is run in near‐realtime by the Global Systems Division (GSD) of the NOAA Earth System Research Laboratory (ESRL) and its products are made available through a website, an ftp site, and via the Unidata Local Data Manager (LDM). The ExREF domain covers most of North America and has 9‐km horizontal grid spacing. The ensemble has eight members, all employing WRF‐ARW. The ensemble uses a variety of initial conditions from LAPS and the Global Forecasting System (GFS) and multiple boundary conditions from the GFS ensemble. Additionally, a diversity of physical parameterizations is used to increase ensemble spread and to account for the uncertainty in forecasting extreme precipitation events. ExREF has been a component of the Hydrometeorology Testbed (HMT) NWP suite in the 2012‐2013 and 2013‐2014 winters. A smaller domain covering just the West Coast was created to minimize band‐width consumption for the NWS. This smaller domain has and is being distributed to the National Weather Service (NWS) Weather Forecast Office and California Nevada River Forecast Center in Sacramento, California, where it is ingested into the Advanced Weather Interactive Processing System (AWIPS I and II) to provide guidance on the forecasting of extreme precipitation events. This paper will review the cooperative effort employed by NOAA ESRL, NASA SPoRT (Short‐term Prediction Research and Transition Center), and the NWS to facilitate the ingest and display of ExREF data utilizing the AWIPS I and II D2D and GFE (Graphical Software Editor) software. Within GFE is a very useful verification software package called BoiVer that allows the NWS to utilize the River Forecast Center's 4 km gridded QPE to compare with all operational NWP models 6‐hr QPF along with the ExREF mean 6‐hr QPF so the forecasters can build confidence in the use of the ExREF in preparing their rainfall forecasts. Preliminary results will be presented.

Reynolds, David↗

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗

DMTN-165: A Hybrid Notification and Alert Retrieval Service

We consider the implications of providing a subset of alert packet contents in the alert stream, with the full alert packets available for rate-limited retrieval using an identifier included in the "lightweight" alert. This hybrid approach would enable much wider dissemination of the complete lightweight alert stream, potentially to thousands more users than is possible in the current baseline. In turn this "alerts on your laptop" access would allow users to conduct more sophisticated filtering and analysis than the current Alert Filtering Service, increasing the overall scientific returns of the alert stream. Community brokers would still be able to retrieve all of the contents of the full alert stream within the 60 second latency window and would gain greater control over their ingestion of the bulk alert data. Technically, this hybrid system may represent a modest increase in complexity over the current baseline but is likely to be more operationally robust.

79 ASTRONOMY AND ASTROPHYSICS↗

Unlocking the Full Potential of Earth Observation During the 2015 Texas Flood Disaster

Intense rainfall during late April and early May 2015 in Texas and Oklahoma led to widespread and sustained flooding in several river basins. Texas state agencies relevant to emergency response were activated when severe weather then ensued for 6 weeks from 8 May until 19 June following Tropical Storm Bill. An international team of scientists and flood response experts assembled and collaborated with decision-making authorities for user-driven high-resolution satellite acquisitions over the most critical areas; while experimental automated flood mapping techniques provided daily ongoing monitoring. This allowed mapping of flood inundation from an unprecedented number of spaceborne and airborne images. In fact, a total of 27,174 images have been ingested to the USGS Hazards Data Distribution System (HDDS) Explorer, except for the SAR images used. Based on the Texas flood use case, we describe the success of this effort as well as the limitations in fulfilling the needs of the decision-makers, and reflect upon these. In order to unlock the full potential for Earth observation data in flood disaster response, we suggest in a call for action(i) stronger collaboration from the onset between agencies, product developers, and decision-makers;(ii) quantification of uncertainties when combining data from different sources in order to augment information content; (iii) include a default role for the end-user in satellite acquisition planning; and(iv) proactive assimilation of methodologies and tools into the mandated agencies.

Texas↗

Total Temperature Measurements using a Rearward Facing Probe in Supercooled Liquid Droplet and Ice Crystal Clouds

This paper presents an analysis of local total temperature and humidity experimental measurement taken in atmospheric ice cloud flows. The measurements were obtained in a series of tests in NASA's Propulsion Systems Laboratory. The probe used in the tests is referred to as the Rearward Facing Probe which was designed to mitigate the contamination effects of ice accretion and ingestion into the probe. The data provided important insights in the interaction of the ice cloud and the atmospheric flow. For the majority of the test runs, small temperature drops in the range of 0.6 to 2.8 C and up to 1.5 g/kg of water vapor rise were found as a result of the interaction. Under certain very low temperature or high TWC conditions, the interaction with the cloud produced a warming of the airflow. A thermal model based on evaporative and convective heat transfer mechanisms between the spray droplets and the airflow showed good agreement with the experimental data. Detailed analyses of the response of the probe under various flow, thermodynamic, and cloud conditions, are provided in the paper.

ice crystal↗

Updates to the Spatial Analysis and Decision Assistance (SADA) Freeware Tool - 16583

SADA is a collaboration between the University of Tennessee and Oak Ridge National Laboratory to present a freeware desktop application integrating environmental risk analytics, spatial modeling, and decision sciences for use at EPA, NRC, DOE, and DoD contaminated sites. The software embeds risk assessment, uncertainty modeling, and downstream decision processes within a spatial context. Updates to Version 6 will provide will include a rich GIS environment and a new 3-D visualization model which permits immersive visualization of contamination, with topology, and the built environment. The risk models presented are developed in conjunction with Superfund to implement all scenarios in their Regional Screening Levels (RSLs) and radionuclide preliminary remediation goals (PRGs). In addition, SADA will be able to ingest high resolution LandScan population data providing insights into the proximity of population to contamination. A brief overview of the progress in provided along with anticipated next steps. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Archi: Agentic Operations at the CMS Experiment

We present Archi, an open-source, end-to-end framework for scientific collaborations that combines the systematic ingestion and organization of heterogeneous data sources with the deployment of configurable, private, and extensible agents that retrieve and reason over them. An instance of Archi has been deployed for the Computing Operations team of the CMS experiment at CERN's LHC since February 2026 as a support agent for technical operators, offering retrieval and analysis capabilities by combining documentation, historical data, and live monitoring systems. We evaluate the system on operator feedback and a question set collected from production usage, graded by human and automated panels. The system proves effective at operational tasks, resolving real-world queries posed by CMS operators. We also observe that locally-hosted, open-weight models perform competitively, enabling fully private management of sensitive data.

Lugato, Pietro [MIT; CERN]↗

VIBE 2.0: Visual Integration for Bayesian Evaluation

Data fusion methods are powerful tools for evaluating experiments designed to discover measurable features of directly unobservable systems. We describe an interactive software platform, Visual Integration for Bayesian Evaluation (VIBE), which ingests Bayesian posterior probability matrices, performs data fusion, and allows the user to interactively evaluate the classification power of fusing various combinations of data sources, such as transcriptomic, proteomics, metabolomics, biochemistry and function.

97 MATHEMATICS AND COMPUTING↗