Field scale application of deep learning-based data assimilation for parameter estimation and uncertainty quantification at a geologic carbon storage site
Explore the source record for details and available documents.
Yoon, Hongkyu↗
SEARCH · Search NASA
Results for “DATA BASES”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Latent space-based data assimilation for real-time forecasting during geologic carbon storage
Explore the source record for details and available documents.
Yoon, Hongkyu [Sandia National Laboratories (SNL-N↗
Automatically Recognizing Semantics of List-based Data Structures in Source Code
Explore the source record for details and available documents.
Computer science↗
Data for Selected Ion Monitoring for Orbitrap-Based Metabolomics
Orbitrap mass spectrometry in full scan mode enables the simultaneous detection of hundreds of metabolites and their isotope-labeled forms. Yet, sensitivity remains limiting for many metabolites, including low-concentration species, poor ionizers, and low-fractional-abundance isotope-labeled forms in isotope-tracing studies. Here, we explore selected ion monitoring (SIM) as a means of sensitivity enhancement. The analytes of interest are enriched in the orbitrap analyzer by using the quadrupole as a mass filter to select particular ions. In tissue extracts, SIM significantly enhances the detection of ions of low intensity, as indicated by improved signal-to-noise (S/N) ratios and measurement precision. In addition, SIM improves the accuracy of isotope-ratio measurements. SIM, however, must be deployed with care, as excessive accumulation in the orbitrap of similar m/z ions can lead, via space-charge effects, to decreased performance (signal loss, mass shift, and ion coalescence). Ion accumulation can be controlled by adjusting settings including injection time and target ion quantity. Overall, we suggest using a full scan to ensure broad metabolic coverage, in tandem with SIM, for the accurate quantitation of targeted low-intensity ions, and provide methods deploying this approach to enhance metabolome coverage.f
Mass Spectrometry↗
Automated Signal Timing Plan Reconstruction Using High-Resolution Event-Based Controller Data for Digital Twins
Transportation digital twins are essential tools for evaluating emerging technologies such as connected and automated vehicles, adaptive traffic signal control, and mobility optimization strategies. Realistic digital twins require accurate emulation of real-world signal controllers and detailed signal timing plans. However, signal timing plans are often unavailable or difficult to access, forcing researchers and modelers to rely on assumed fixed timings or halt their analysis. To overcome this challenge, we present a method that directly estimates signal timing plan parameters using high-resolution, event-based data from traffic signal controllers. The proposed method extracts key parameters, including cycle length, offset, phase sequence, coordinated phases, phase-specific minimum and maximum green durations, vehicle extensions, and splits under coordination. A rule-based deterministic signal timing reconstruction algorithm based on traffic signal operation rules, such as those outlined in the Signal Timing Manual, is developed and validated. We evaluate this method, which uses high-resolution controller event logs and verified signal timing plans, on 94 signalized intersections in Nashville, Tennessee, demonstrating their ability to generate accurate, simulation-ready signal timing plans for tools such as SUMO and Vissim.
Saroj, Abhilasha [ORNL] (ORCID:0000000191178063)↗
The influence of cloud cover on the reliability of satellite-based solar resource data
Satellite-based solar resource data are often developed and validated by using binary cloudiness categories: clear sky or overcast cloudy sky. To investigate the reliability of solar resource data in partially cloudy conditions, we estimate cloud fraction using two distinct algorithms: a physical retrieval model using surface observed global horizontal irradiance (GHI) and direct normal irradiance (DNI) and a temporal average of cloud mask data estimated by the observed DNI. Our analysis reveals a significant presence of scattered clouds, broken clouds, and mismatches between satellite- and surface-based cloud data at 17 surface sites across the contiguous United States, though confidently clear and cloudy conditions collectively account for more than 70 % of the data. Solar radiation is computed using the National Solar Radiation Database (NSRDB) algorithm and validated using surface observations. Here, our findings suggest that, in the presence of scattered clouds, NSRDB data for clear-sky conditions can be subject to significant overestimation. In cloudy-sky conditions classified by satellite data, DNI computed by the Fast All-sky Radiation Model for Solar applications with DNI (FARMS-DNI) can be underestimated when limited clouds are detected by surface observations. The bias observed in several cloudiness categories indicates that the NSRDB is exceptionally accurate in confidently clear conditions. However, clear-sky conditions with scattered clouds and mismatched cloud data contribute significantly to the overall uncertainties in the NSRDB. Therefore, future improvements in solar resource data should involve development and implementation of satellite-derived cloud fraction and should consider a novel radiative transfer model accounting for amplified cloud reflection. The evaluation within cloudiness categories also provides a physical rationale for the superior performance of FARMS-DNI compared to the Direct Insolation Simulation Code (DISC) in both cloudy-sky and all-sky conditions.
14 SOLAR ENERGY↗
Machine Learning-Based Extreme Data Reduction for Prompt Supernova Pointing at DUNE
One of the goals of the Deep Underground Neutrino Experiment (DUNE) is to use the massive underground liquid argon time projection chamber (LArTPC) detectors at its far site for multimessenger astronomy (MMA), in the detection of neutrinos from core-collapse supernovae (SNe). Its current baseline trigger strategy detects activity in the detector that is consistent with supernova (SN) neutrinos and saves the raw data for further offline analysis but provides no prompt pointing information crucial for optical follow-ups by other observatories. This approach is based on the assumption that prompt pointing determination using raw data is computationally prohibitive. In this article, we demonstrate a proof-of-concept based on applying extreme data reduction on the buffered SN data in the DUNE data acquisition (DAQ) system’s front-end computers using a machine learning (ML) workflow. This reduces the data by ~5 orders of magnitude, allowing a full track reconstruction to be carried out quickly on a single server. The total time to perform the ML-based data reduction and the full track reconstruction is less than the time to transfer the SN data back to Fermilab or a high-performance computing (HPC) center. This shows that prompt processing of raw SN data is possible and, in fact, trivial once the data have been reduced to reject radiological backgrounds, paving the way to a high-quality SN pointing trigger that is based on fully reconstructed data instead of trigger primitives (TPs).
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
CLPNets: Coupled Lie–Poisson neural networks for multi-part Hamiltonian systems with symmetries
To accurately compute data-based prediction of Hamiltonian systems, it is essential to utilize methods that preserve the structure of the equations over time. We consider a particularly challenging case of systems with interacting parts that do not reduce to pure momentum evolution. Such systems are essential in scientific computations, such as discretization of a continuum elastic rod, which can be viewed as the group of rotations and translations $SE(3)$. The evolution involves not only the momenta but also the relative positions and orientations of the particles. The presence of Lie group-valued elements, such as relative positions and orientations, poses a problem for applying previously derived methods for data-based computing. We develop a novel method of data-based computation and complete phase space learning of such systems. We follow the original framework of SympNets (Jin et al., 2020) and LPNets (Eldred et al., 2024), building the neural network from phase space mappings that preserve the Lie–Poisson structure. We derive a novel system of mappings that are built into neural networks describing the evolution of such systems. We call such networks Coupled Lie–Poisson Neural Networks, or CLPNets. We consider increasingly complex examples for the applications of CLPNets, starting with the rotation of two rigid bodies about a common axis, progressing to the free rotation of two rigid bodies, and finally to the evolution of two connected and interacting $SE(3)$ components, describing the discretization of an elastic rod into two elements. Our method preserves all Casimir invariants to machine precision, preserves energy to high accuracy, and shows good resistance to the curse of dimensionality, requiring only a few thousand data points for all cases studied (three to eighteen dimensions). Additionally, the method is highly economical in memory requirements, requiring only about 200 parameters for the most complex case considered.
Data-based modeling↗
Conservative projection-based data-driven model order reduction of a fluid-kinetic spectral solver
Kinetic simulations are computationally intensive due to six-dimensional phase space discretization. Many kinetic spectral solvers use the asymmetrically weighted Hermite expansion due to its conservation and fluid-kinetic coupling properties, i.e., the lower-order Hermite moments capture and describe the macroscopic fluid dynamics, and higher-order Hermite moments describe the microscopic kinetic dynamics. We leverage this structure by developing a parametric data-driven reduced-order model based on the proper orthogonal decomposition, which projects the higher-order kinetic moments while retaining the fluid moments intact. We demonstrate analytically and numerically that the method ensures local and global mass, momentum, and energy conservation. The numerical results show that the proposed method effectively replicates the high-dimensional spectral simulations at a fraction of the computational cost and memory, as validated on the weak Landau damping and two-stream instability benchmark problems.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Global tuning of hadronic interaction models with accelerator-based and astroparticle data
In high-energy and astroparticle physics, event generators play an essential role, even in the simplest data analyses. As analysis techniques become more sophisticated, e.g. based on deep neural networks, their correct description of the observed event characteristics becomes even more important. Physical processes occurring in hadronic collisions are simulated within a Monte Carlo framework. A major challenge is the modeling of hadron dynamics at low momentum transfer, which includes the initial and final phases of every hadronic collision. QCD-inspired phenomenological models used for these phases cannot guarantee completeness or correctness over the full phase space. These models usually include parameters which must be tuned to suitable experimental data. Until now, event generators have been developed and tuned mainly on the basis of data from high-energy physics experiments at accelerators. The wealth of data available from the latest generation of astroparticle experiments has not yet been fully exploited, and in many cases is not satisfactorily described. Both kinds of data sets are complementary as astroparticle experiments provide sensitivity especially to hadrons produced nearly parallel to the collision axis and cover center-of-mass energies up to several hundred TeV, well beyond those reached at colliders so far. In this report, we provide an overview of state-of-the-art event generators and their tuning, including the most relevant inputs from high-energy accelerator and astroparticle experiments. We present a road map that shows, for the first time, how the unified tuning of event generators with accelerator-based and astroparticle data can be performed.
Albrecht, J. [Ruhr U., Bochum, RAPP Ctr.; Ruhr U.,↗
MaTableGPT: GPT‐Based Table Data Extractor from Materials Science Literature
Abstract Efficiently extracting data from tables in the scientific literature is pivotal for building large‐scale databases. However, the tables reported in materials science papers exist in highly diverse forms; thus, rule‐based extractions are an ineffective approach. To overcome this challenge, the study presents MaTableGPT, which is a GPT‐based table data extractor from the materials science literature. MaTableGPT features key strategies of table data representation and table splitting for better GPT comprehension and filtering hallucinated information through follow‐up questions. When applied to a vast volume of water splitting catalysis literature, MaTableGPT achieves an extraction accuracy (total F1 score) of up to 96.8%. Through comprehensive evaluations of the GPT usage cost, labeling cost, and extraction accuracy for the learning methods of zero‐shot, few‐shot, and fine‐tuning, the study presents a Pareto‐front mapping where the few‐shot learning method is found to be the most balanced solution owing to both its high extraction accuracy (total F1 score >95%) and low cost (GPT usage cost of 5.97 US dollars and labeling cost of 10 I/O paired examples). The statistical analyses conducted on the database generated by MaTableGPT revealed valuable insights into the distribution of the overpotential and elemental utilization across the reported catalysts in the water splitting literature.
Yi, Gyeong Hoon [Computational Science Research Ce↗
Automated vehicle microscopic energy consumption study (AV-Micro): Data collection and model development
While the Adaptive Cruise Control (ACC) system in automated vehicles (AVs) is expected to impact transportation energy significantly, existing AV energy consumption models only directly adopt those developed with Human-driven Vehicle (HV) data without even slight adaptation or calibration to accommodate unique AV energy consumption features. This study will investigate how accurately HV data-based models can predict the energy consumption of AVs. Empirical trajectory data and corresponding instantaneous energy consumption rates from both AVs and HVs were collected. We adopted two classical HV data-based models to fit these data. The calibration results indicated that these models yield around 20 30% prediction errors for AVs. To further improve the prediction accuracy, this study designed an AV-Micro model by incorporating components of multiple classic energy consumption models that better capture ACC energy consumption features, including piecewise driving behavior. With this, the AV-Micro model achieves lower than 10% prediction errors. The AV-Micro model’s high consistency across different test runs was verified with statistical significance tests, demonstrating its adaptability in different driving profiles. To confirm the discrepancies between the energy consumption features of AVs and HVs, more statistical significance tests were conducted to show that the AV-Micro model cannot be directly applied to HV data. The findings by calibrated AV-Micro models revealed that AVs consume approximately 80.5–146.4 J more energy than HVs for each meter traveled. Furthermore, the frequency analysis of energy consumption indicates that there is still some room for AVs to improve energy efficiency, particularly given their larger amplitude high-frequency fluctuations.
33 ADVANCED PROPULSION SYSTEMS↗
Optimization-Based Data-Driven Approach for Detecting Fault Location in Power Systems
In grids with large penetration of converterinterfaced resources (CIRs), measurements of voltage, current, and line parameters can fluctuate significantly during fault conditions. These fluctuations, combined with complex network topologies and extensive system branching, make accurate fault location challenging. Faults, such as short circuits, can cause prolonged outages with serious socio-economic impacts, highlighting the need for rapid fault identification to minimize downtime. However, current fault detection methods—such as relays and digital fault recorders—often relay information too slowly, impeding swift corrective action. Given the limited availability of high-resolution phasor measurement units, this paper introduces an optimization-based observer to estimate fault locations, grid line parameters, and voltages using local CIR measurements. To preserve the confidentiality of CIRs and enhance estimation accuracy, this study uses a black-box model of CIRs. This bottom-up, event-driven approach can enhances protection and control systems through optimized and real-time fault detection. Simulation results show that the optimization-based data-driven observer can accurately detect fault locations and estimate grid states and parameters, providing valuable insights for utilities and operators in grid applications.
Subedi, Sunil [ORNL] (ORCID:000000034069090X)↗
Consumer safety-oriented scheduling of rotating power outages during heat waves
Extreme heat events have widespread effects on power systems, reducing available generation capacity, limiting transmission capabilities, and causing unusual demand patterns on the consumer side. As these combined effects expose bulk transmission systems to potential large-scale blackouts, utilities may be required to schedule and apply rotating outages, by temporarily and alternately disconnecting distribution substations to reduce overload. However, utilities lack mechanisms to inform these events, exacerbating the negative effects of heat waves on affected communities. This paper introduces a novel framework for scheduling rotating outages during heat waves while considering impacts on consumers’ safety. Instead of random sequential load shedding, we propose a methodology to rotate power outages considering a metric that quantifies the indoor overheating risk of groups of consumers during a power outage. The overheating risk is derived from a detailed building simulation using CityBES, where the buildings are modeled based on available data—use type, year built, floor area, number of stories, location—while presence of air conditioning and occupancy are calibrated from smart meter data. Based on the metric, an algorithm to schedule the rotating outages is applied to prioritize feeders for disconnection at each hour according to their overheating risk to meet a utility load reduction target. Applied to two substations and seven feeders in the Portland General Electric territory, the results show that this approach effectively leads to the lowest overheating risk during the resulting outage schedules, with an average 10.1% lower overheating compared to uninformed schedules.
Building thermal simulation↗
Redis-Based Streaming Architecture for Accelerator Beam Instrumentation DAQ Systems
The Fermilab Acceleraor Division, Beam Instrumentation Department, is always adopting modern and current software methodologies for complex DAQ architectures. This paper highlights the Redis Adapter (RA) as the key software component enabling high performance, modular communication between digitizers and distributed control systems by leveraging Redis and containerization. The RA provides a unified, efficient interface between Redis based data streams and consumer systems. In the legacy architecture, digitized data flowed through the custom, UDP based Distributed Data Communication Protocol in the middle layer. In the current system, DDCP remains the ingestion path, while the RA serves as the decoupling layer. The proposed system replaces old VME digitizers with a SOM-based digitizer that communicates with Redis using the RA. The RA acts as both a performance-critical bridge and a protocol-agnostic adapter, ensuring compatibility with legacy control frameworks while enabling future scalability and modularity. This restructuring of the middle layer also helps the system achieve high throughput, reduce latency, and simplify the data path. Finally, we will demonstrate how RA is utilized in our two core products to deliver both legacy compatibility and future flexibility.
Joshi, S. [Fermilab]↗
Leveraging large language models to address data scarcity in machine learning for graphene synthesis
Machine learning in experimental materials science faces significant challenges due to the scarcity of data, which are costly and time-consuming to generate, particularly when relying on in-house experiments. Literature data mining offers a potential solution but introduces issues like mixed data quality, inconsistent formats, and non-uniform reporting of synthesis parameters, resulting in partially missing and heterogeneous features across the dataset. Here, we propose data imputation and feature engineering methods that employ pre-trained large language models (LLMs) to enhance machine learning performance on scarce, heterogeneous datasets, demonstrated on graphene CVD synthesis data and the ML-HydPARK hydrogen storage dataset. GPT models perform data imputation via tailored prompting and semantic normalization of inconsistently reported features through embeddings, for example, to harmonize the complex nomenclature of CVD substrates. Beyond yielding more diverse and richer feature representations than traditional methods such as K-nearest neighbors (KNN) and Multivariate Imputation by Chained Equations (MICE), LLM-based data imputation is evaluated against dataset characteristics and prompting strategies. We vary the level of autonomy granted to the LLM, from generic prompting that leverages pre-trained knowledge for autonomous data generation to data-informed prompting that constrains outputs using target-specific information, and demonstrate which level of autonomy yields superior imputation performance across datasets and feature types. The proposed data engineering methods markedly improve downstream performance; for example, in graphene layer number classification using a support vector machine (SVM), binary accuracy increases from 39% to 65% and ternary accuracy from 52% to 72%. Fine-tuning experiments on both datasets show that combining our proposed LLM-based data imputation and feature encoding methods with numerical machine learning predictors outperforms standalone fine-tuned LLM predictors in data-scarce settings. The proposed strategies emphasize data enhancement techniques rather than refining learning architectures or regularizing loss functions, offering a broadly applicable framework for improving machine learning performance on scarce, inhomogeneous datasets.
Chemical vapor deposition↗
Monthly Mean In Situ Surface Flux Observations Paired with Satellite-Derived and Reanalysis-Based Flux Data for the Great Lakes Region, 2001–2020
Surface radiative and turbulent heat fluxes over the Great Lakes strongly influence regional hydrological and meteorological processes, and their accurate representation is critical for numerical weather prediction and coupled atmosphere–lake modeling. However, direct flux observations are spatially sparse across the region, so gridded reanalysis and satellite-derived products are often used for climatological analyses and model evaluation despite differences in their flux representations. This dataset provides processed, quality-controlled, monthly mean surface flux observations from the Great Lakes Evaporation Network (GLEN), AmeriFlux, and the National Data Buoy Center, paired with spatiotemporally matched flux estimates from two reanalysis products, the fifth generation European Centre for Medium-Range Weather Forecasts (ECMWF) reanalysis dataset (ERA5) and the Modern Era Reanalysis for Research and Applications, version 2 (MERRA-2), and two satellite-derived products, the Clouds and Earth's Radiant Energy Systems Energy Balanced and Filled (CERES-EBAF) and the Cloud, Albedo and Surface Radiation dataset from AVHRR data - Edition 3 (CLARA-A3). The dataset includes sixteen observational stations with variable temporal coverage within 2001–2020. For each station, a CSV file contains monthly time series of available flux variables, including surface downwelling shortwave radiation (SW), surface downwelling longwave radiation (LW), sensible heat (SH) flux, and latent heat flux (LH), alongside matched gridded product values where available. Columns in the CSV file correspond to different variables sourced from each dataset, with column titles structured as "{dataset}_{variable}". Columns with relevant metadata are also provided in each CSV file, including station latitude and longitude, monthly timestamps, and the name of the sourced observational data. These files are structured for direct use in common analysis tools, including Microsoft Excel, Python pandas, and Python matplotlib. This dataset supports climatological analysis of the Great Lakes regional surface energy budget, evaluation of satellite-derived and reanalysis-based flux products, and development or validation of flux representations in numerical weather prediction and coupled atmosphere–lake models.
Great Lakes↗
APOLLO: a facility-scale differentiable virtual accelerator at Fermilab FAST/IOTA
As the design complexity of modern accelerators grows, there is more interest in using advanced simulations that have fast execution time or yield additional insights like gradients. The FAST/IOTA facility has been working on implementing and experimentally validating an end-to-end digital twin that is both fast and gradient-aware, allowing for rapid prototyping of new software and experiments with minimal beam time costs. Our framework integrates physics and ML codes for linac and ring simulation through a set of generic interfaces between surrogate and physics-based sections. To reproduce device inputs and outputs, system state is exposed as a deterministic event loop in a specialized discrete event simulator architecture. Because Fermilab is undergoing control system transition, several APIs were implemented as final user interfaces - a fully asynchronous EPICS soft IOC, a gRPC-based Data Pool Manager (DPM), and legacy ACNET protocols. We discuss implementation details as well as challenges handling live data assimilation and future plans to extend modelling to main complex proton accelerators like PIPII and Booster.
Kuklev, Nikita [Fermilab]↗