Search NASA⌕ Search

SEARCH · Search NASA

Results for “INGESTION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗

White Paper: Scalable Digital Twin Capabilities for Aging and Surveillance of Engineered Systems

This white paper presents a multi-year initiative to develop practical, secure, and scalable digital twin capabilities for engineered systems in aging and surveillance contexts—an approach pioneered at the National Nuclear Security Administration (NNSA) Lawrence Livermore National Laboratory (LLNL) that maps directly onto the needs and ambitions of the Navy for ship- and fleet-level digital twins. LLNL’s work in building part- and process-level digital twins for advanced manufacturing, with a vision to scale up to entire factory floors and, ultimately, enterprise-wide digital twins, offers an adaptable pathway for the Navy as it seeks to modernize lifecycle management, readiness, and predictive maintenance across ships and fleets. For our application, we integrate physics-based modeling with automated data ingestion, processing, and AI-driven calibration, creating hybrid models that are both interpretable and data responsive. We modernized legacy workflows, established centralized data infrastructure, automated experimental pipelines, and demonstrated end-to-end coupling of accelerated aging data with finite element simulations via optimization and surrogate modeling. The result is a generalizable framework that supports part-level digital twins today and lays the groundwork for future system-level twins suitable for Navy applications.

36 MATERIALS SCIENCE↗

Shallow cumulus entrainment: observational retrieval, physical interpretation, and climate impacts (Final grant report)

This document constitutes a final technical report for DE-SC0020083, a research grant awarded by the US Department of Energy Office of Science, to Prof. Daniel Kirshbaum at McGill University. The original research proposal, entitled "Shallow cumulus entrainment: observational retrieval, physical interpretation, and climate impacts", defined four scientific objectives that synthesized ARM observations with turbulence-permitting large-eddy simulations (LES) to quantify and understand the process of cumulus entrainment, whereby a cumulus cloud ingests surrounding air through its lateral edges. These objectives included developing observational climatologies of cumulus bulk entrainment rate at two ARM observatories, physical interpretation of these findings using LES, quantification of the impacts of subcloud circulations on entrainment, and the climate impacts of flow-dependent entrainment variations. In the six years that have elapsed since the grant was awarded (three funded years and three no-cost annual extensions), all four of the objectives were successfully addressed. Four personnel were trained (one B.Sc., one M.Sc. to completion, five years of Ph.D. training, and 2.5 years of postdoctoral training), eight peer-reviewed articles were published in high-impact journals, and 20 presentations were delivered (three invited).

54 ENVIRONMENTAL SCIENCES↗

Evaluation of Data Lake Design for the Accelerator Control System

In this modern world, the user expects faster processing and real-time response to operate accelerator control devices. The existing framework with its infrastructure does not have the ability to satisfy these future needs. Therefore, modernization is required to reach industry standards and develop a modular framework that can provide flexibility and dynamic scalability. The data lake architecture, comprising three layers, ingestion, processing, and data consumption, provides flexibility and scalability to meet current and future demands for the accelerator control system.

Jaikar, Amol [Fermilab]↗

Precursor Analysis Report: Remote Access Attack on Oldsmar Water Treatment Facility 2021

The Remote Access Attack on Oldsmar Water Treatment Facility 2021 Precursor Analysis Report leverages publicly available information about the Oldsmar cyber attack and catalogs anomalous observables for each technique employed in the attack. This analysis is based upon the methodology of the Cybersecurity for the Operational Technology Environment (CyOTE) program. On 5 February 2021, an adversary gained unauthorized remote access to Bruce T. Haddock Water Treatment Plant in Oldsmar, Florida, which provides treated water to 15,000 customers. The adversary accessed the facility’s Supervisory Control and Data Acquisition (SCADA) workstation and human machine interface (HMI) to change the chemical concentration of sodium hydroxide, commonly referred to as lye and used to regulate acidity levels, from 100 parts per million (PPM) to 11,100 PPM. The chemical was raised to lethal levels that if ingested could lead to serious soft tissue damage, burns, or even death. The facility, however, had redundancies and alarms in place to alert personnel of dangerous chemical levels, and facility officials stated it would have taken 24 to 36 hours for the chemical changes to affect the water supply. Researchers and analysts identified six unique techniques utilized during the attack with a total of 23 observables using MITRE ATT&CK® for Industrial Control Systems. The CyOTE program assesses observables accompanying techniques used prior to the triggering event to identify opportunities to detect malicious activity. If observables accompanying the attack techniques are perceived and investigated prior to the triggering event, earlier comprehension of malicious activity can take place. Four of the identified techniques used during the Oldsmar cyber attack were precursors to the triggering event. Analysis identified 21 observables associated with these precursor techniques, 20 of which were assessed to have an increased likelihood of being perceived in the minutes preceding the triggering event. The response and comprehension time could have been reduced if the observables had been identified earlier. The information gathered in this report contributes to a library of observables tied to a repository of artifacts, data sources, and technique detection references for practitioners and developers to support the comprehension of indicators of attack. Asset owners and operators can use these products if they experience similar observables or to prepare for comparable scenarios.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Trident Smyth: Near Real Time Sensor Fusion and Track Stitching Application

Sandia’s Sensor Data Fusion application is a NGA sponsored project that fuses tracks from different sensors to provide more accuracy than any single track. The application serves as the unclassified architecture for data ingestion, processing, and near real-time sensor fusion.

Sorroche, Joseph John [Sandia National Laboratorie↗

Radiological Safety Analysis Computer (RSAC) Program Version 7.4 Users’ Manual

The Radiological Safety Analysis Computer (RSAC) Program Version 7.4 (RSAC-7) is the newest version of the RSAC legacy code. RSAC-7 calculates the consequences of a release of radionuclides to the atmosphere. Users generates a fission product inventory from either reactor operating history or a nuclear criticality event. RSAC-7 models the effects of high-efficiency particulate air filters or other cleanup systems and calculates the decay and ingrowth during transport through processes, facilities, and the environment. Doses are calculated for inhalation, air immersion, ground surface, ingestion, and cloud gamma pathways. RSAC-7 is used as a tool to evaluate accident conditions in emergency response scenarios, radiological sabotage events, and safety basis accident consequences. This users’ manual contains the mathematical models and operating instructions for RSAC-7. Instructions, screens, and examples are provided to guide the user through the functions provided by RSAC-7. This program is designed for users who are familiar with radiological dose assessment methods.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Grid Architecture Mapping to Understand Transformation (GAMUT): Methods and Framework Architecture

Grid architecture (GA) is a concept that was developed to address the need for a comprehensive view of power grid challenges. GA can be viewed as a relatively consistent and fixed high-level approach; however, for any instantiation of grid structures, a combinatorial explosion results from each lower-layer expansion. This constitutes the main challenge with GA—it is a grid architect’s view of the system, which might not be very informative at the implementation level. Grid Architecture Mapping to Understand Transformation (GAMUT project) seeks to bridge that gap by integrating subject matter expertise across GA structures, providing users who lack expertise in GA approaches with valuable insights and informational materials. GAMUT seeks to answer feasibility questions for the approach. System-level expectations are that a GA baseline needs to be established in order for GA to be the common framework to which any lower layer approach is tied. This report explores a potential information ingestion and documentation framework to support GAMUT. The main concepts that enable the solution domain of GAMUT are discussed, and examples are provided. The solution domain leverages already-existing technology and concepts related to GA, knowledge management, and other relevant areas. To assess GAMUT building blocks and the overall approach, a feasibility assessment is proposed, rooted in systems engineering and GA architecture evaluation concepts.

24 POWER TRANSMISSION AND DISTRIBUTION↗

APOLLO: a facility-scale differentiable virtual accelerator for Fermilab

As the design complexity of modern accelerators grows, there is more interest in using advanced simulations that have fast execution time or yield additional insights like gradients. The FAST/IOTA facility has been working on implementing and experimentally validating an end-to-end digital twin that is both fast and gradient-aware, allowing for rapid prototyping of new software and experiments with minimal beam time costs. Our framework integrates physics and ML codes for linac and ring simulation through a set of generic interfaces between surrogate and physics-based sections. To reproduce device inputs and outputs, system state is exposed as a deterministic discrete event simulator. Because Fermilab is undergoing control system transition, both EPICS and ACNET frontends are supported. Recently, we have begun transitioning to a new community lattice standard, PALS, as well as developing standardized infrastructure for data ingest and normalization to prepare for model calibration during FAST proton injector commissioning. We discuss implementation details as well as challenges, and future plans to extend modelling to main complex proton accelerators like PIPII and Booster.

Kuklev, Nikita [Fermilab]↗

C-HER Metadata Overview: Approach, Standards, and Rigor for the Centralized Health and Exposomic Resource

The Centralized Health and Exposomic Resource (C-HER) unifies environmental, demographic, geographic, and health-related data for exposomic research. The source data differ in format, geographic coverage, time period, resolution, terminology, and documentation. We use a common metadata framework to describe those differences and to record how each data resource has been processed, documented, and ingested. This document relates only to the C-HER metadata framework. It explains the information that is recorded for each resource, the standards used to organize that information, the conditions for metadata completeness, and the relationship between metadata and quality review. It is intended for those who need to understand what C-HER metadata communicates and how it supports appropriate use of the data. It is not an implementation specification or procedure. It does not document the database schema, source code, deployment configuration, transformation algorithms, or dataset-specific QA/QC thresholds. Those materials are maintained separately.

MacFarland, Midgie [ORNL] (ORCID:0009000807354078)↗

Atmospheric Radiation Measurement program climate research facility operations quarterly report July 1 - Sep. 30, 2009.

Individual raw data streams from instrumentation at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility (ACRF) fixed and mobile sites are collected and sent to the Data Management Facility (DMF) at Pacific Northwest National Laboratory (PNNL) for processing in near-real time. Raw and processed data are then sent approximately daily to the ACRF Archive, where they are made available to users. For each instrument, we calculate the ratio of the actual number of data records received daily at the Archive to the expected number of data records. The results are tabulated by (1) individual data stream, site, and month for the current year and (2) site and fiscal year (FY) dating back to 1998. The U.S. Department of Energy (DOE) requires national user facilities to report time-based operating data. The requirements concern the actual hours of operation (ACTUAL); the estimated maximum operation or uptime goal (OPSMAX), which accounts for planned downtime; and the VARIANCE [1 - (ACTUAL/OPSMAX)], which accounts for unplanned downtime. The OPSMAX time for the fourth quarter of FY 2009 for the Southern Great Plains (SGP) site is 2,097.60 hours (0.95 ? 2,208 hours this quarter). The OPSMAX for the North Slope Alaska (NSA) locale is 1,987.20 hours (0.90 ? 2,208) and for the Tropical Western Pacific (TWP) locale is 1,876.8 hours (0.85 ? 2,208). The ARM Mobile Facility (AMF) was officially operational May 1 in Graciosa Island, the Azores, Portugal, so the OPSMAX time this quarter is 2,097.60 hours (0.95 x 2,208). The differences in OPSMAX performance reflect the complexity of local logistics and the frequency of extreme weather events. It is impractical to measure OPSMAX for each instrument or data stream. Data availability reported here refers to the average of the individual, continuous data streams that have been received by the Archive. Data not at the Archive result from downtime (scheduled or unplanned) of the individual instruments. Therefore, data availability is directly related to individual instrument uptime. Thus, the average percentage of data in the Archive represents the average percentage of the time (24 hours per day, 92 days for this quarter) the instruments were operating this quarter. Table 1 shows the accumulated maximum operation time (planned uptime), actual hours of operation, and variance (unplanned downtime) for the period July 1 - September 30, 2009, for the fixed sites. Because the AMF operates episodically, the AMF statistics are reported separately and not included in the aggregate average with the fixed sites. The fourth quarter comprises a total of 2,208 hours for the fixed and mobile sites. The average of the fixed sites well exceeded our goal this quarter. The AMF data statistic requires explanation. Since the AMF radar data ingest software is being modified, the data are being stored in the DMF for data processing. Hence, the data are not at the Archive; they are anticipated to become available by the next report.

54 ENVIRONMENTAL SCIENCES↗

Design and Implementation of a Metadata-rich File System

Despite continual improvements in the performance and reliability of large scale file systems, the management of user-defined file system metadata has changed little in the past decade. The mismatch between the size and complexity of large scale data stores and their ability to organize and query their metadata has led to a de facto standard in which raw data is stored in traditional file systems, while related, application-specific metadata is stored in relational databases. This separation of data and semantic metadata requires considerable effort to maintain consistency and can result in complex, slow, and inflexible system operation. To address these problems, we have developed the Quasar File System (QFS), a metadata-rich file system in which files, user-defined attributes, and file relationships are all first class objects. In contrast to hierarchical file systems and relational databases, QFS defines a graph data model composed of files and their relationships. QFS incorporates Quasar, an XPATH-extended query language for searching the file system. Results from our QFS prototype show the effectiveness of this approach. Compared to the de facto standard, the QFS prototype shows superior ingest performance and comparable query performance on user metadata-intensive operations and superior performance on normal file metadata operations.

99 GENERAL AND MISCELLANEOUS↗

NWTC Site 4.0 - NREL ASSIST (SN10) / Thermodynamic retrievals TROPoe

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 10) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a Vaisala CL51 ceilometer. The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data were not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Denver, CO.

17 WIND ENERGY↗

NWTC Site 3.2 - NREL ASSIST (SN12) / Thermodynamic retrievals TROPoe

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 12) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a Vaisala CL51 ceilometer. The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data were not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Denver, CO.

17 WIND ENERGY↗

Title NWTC Site 3.2 - NREL ASSIST (SN11) / Thermodynamic retrievals TROPoe

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 11) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height from a Vaisala CL51 ceilometer. The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data were not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at Denver, CO.

17 WIND ENERGY↗

Site B - NREL ASSIST (SN11) Thermodynamic Retrievals TROPoe / Derived Data

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 11) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height (CBH), which is a combined data product that uses data from ceilometers at sites A1 and H and scanning lidars from ARM sites C1 and E37. The CBH is weighted inversely proportionally to the distance to the respective site to take into account the spatial variability of clouds (see https://github.com/StefanoWind/ASSIST_analysis/blob/main/awaken_processing/combine_cbh.py). The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data was not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at ARM SGP, OK.

17 WIND ENERGY↗

Site G - NREL ASSIST (SN10) Thermodynamic Retrievals TROPoe / Derived Data

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 10) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height (CBH), which is a combined data product that uses data from ceilometers at sites A1 and H and scanning lidars from ARM sites C1 and E37. The CBH is weighted inversely proportionally to the distance to the respective site to take into account the spatial variability of clouds (see https://github.com/StefanoWind/ASSIST_analysis/blob/main/awaken_processing/combine_cbh.py). The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data was not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at ARM SGP, OK.

17 WIND ENERGY↗

Site C1a - NREL ASSIST (SN12) Thermodynamic Retrievals TROPoe / Derived Data

This dataset contains daily files with thermodynamic profiles retrieved with the optimal estimation physical retrieval TROPoe v0.12 (Turner and Löhnert 2014; Turner and Blumberg 2019; Turner and Löhnert 2021). The profiles are retrieved every 10 minutes from instantaneous observations from the NREL ASSIST-II (SN 12) infrared spectrometer. Observations are noise-filtered but not averaged in time to minimize errors due to non-uniform clouds. Additional input data in TROPoe are cloud base height (CBH), which is a combined data product that uses data from ceilometers at sites A1 and H and scanning lidars from ARM sites C1 and E37. The CBH is weighted inversely proportionally to the distance to the respective site to take into account the spatial variability of clouds (see https://github.com/StefanoWind/ASSIST_analysis/blob/main/awaken_processing/combine_cbh.py). The full pipeline for running the retrieval is available at https://github.com/StefanoWind/TROPoe_processor. Met data was not ingested. In addition to these temporally resolved input data, TROPoe requires an a priori dataset (prior) that provides mean climatological estimates of thermodynamic profiles and specifies how temperature and humidity covary with height as an input (for details see, e.g., Djalalova et al. 2022). The prior is a key component of the retrieval and provides a constraint on the ill-posed inversion problem. A monthly prior was computed from operational radiosonde launches at ARM SGP, OK.

17 WIND ENERGY↗