Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Imaging Bragg Edge Analysis TooLs for Engineering Structures (iBeatles)

The Spallation Neutron Source (SNS) at Oak Ridge National Laboratory (ORNL) provides pulsed neutrons with energies varying from epithermal to cold. In preparation for VENUS, the neutron imaging beamline to be located at beam port 10, we have performed a series of experiments focused on wavelength-dependent radiography and computed tomography for a broad range of applications, from materials science to biological tissues.One of the time-of-flight (TOF) techniques that is of interest to the scientific community is the 2-dimensional mapping of phases and average crystalline plane orientation in samples both ex-situ and during applied stresses such as tensile loading and heating. This technique is known as Bragg edgeimaging and relies on the identification of changes of transmission values, fitting of the edge to measure its displacement, and thus identify the shift in lattice parameter due to stresses. One of the challenges of TOF imaging measurements is the amount of data and the inability to observe Bragg edge shifts in real time during an experiment. Thus, we have been focusing on creating a Python-based interface that allows fast data processing and instantaneous mapping and fitting of the Bragg edges, and their evolution through time. Python libraries and Jupyter notebooks have been implemented to facilitate decision making during an experiment. The advantage of the notebooks is the possibility to guide an experiment as they can quickly process and display Bragg edge data. These notebooks can be used independently, or can be combined in a Python Graphical User Interface (GUI) tool called iBeatles. This interface permits visualization and fitting of the Bragg edges, and ultimately back-projects the fitting results onto the radiographs to display a strain map. Assuming data collection has sufficient statistics, the strain mapping analysis can be performed on a pixel-by-pixel basis. This development is a step forward toward a better user experience at the future VENUS beamline in terms of live feedback and productivity. Analysis that used to take days of switching between different applications can now be done in minutes within the

Bilheux, JeanChristophe [Oak Ridge National Labora↗
DOE OSTI2026

Data and scripts associated with “Moisture content modulates DOM thermodynamic regulation of oxygen consumption in drying streambed sediments”

This data package is associated with the publication “Moisture content modulates DOM thermodynamic regulation of oxygen consumption in drying streambed sediments” published in Scientific Reports (Garayburu-Caruso et al., 2026). The package contains processed data products and scripts used to quantify how drying and re-inundation of riverbed sediments influence dissolved organic matter (DOM) thermodynamic properties and their relationship with sediment oxygen (O₂) consumption across 33 stream sites in the contiguous United States. The data package contains DOM thermodynamic metrics (e.g., Gibbs free energy of carbon oxidation and thermodynamic efficiency), and O₂ consumption along with watershed-scale climate and land-cover metrics used as explanatory variables in the analyses. Underlying unprocessed and processed ultrahigh-resolution mass spectrometry data, oxygen consumption rates from laboratory moisture-manipulation experiments, within-sample environmental properties, sediment moisture content and contextual field measurements are archived separately at https://data.ess-dive.lbl.gov/datasets/doi:10.15485/2428003 (Laan et al., 2024) and https://data.ess-dive.lbl.gov/datasets/doi:10.15485/1923689 (Forbes et al.,2023). A preliminary version of this data package was published in February 2026 at the time of manuscript submission. It was updated in June 2026, at the time of manuscript acceptance, to include the finalized data and additional metadata (readme, data dictionary, and file level metadata). For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. At the top level, the data package is organized into five main folders: (1) Data, (2)Figures, (3) Map, (4) GAM_Reulsts, and (5) src. The Data folder contains analysis-ready tabular files with oxygen consumption rates, DOM thermodynamic properties by site and treatment, site-level environmental variables, watershed-scale metrics, and other derived variables referenced in the manuscript. The Figures folder contains static image files associated with the main text and supplemental figures, while the Map folder includes spatial data and map-layer files used to create the sampling-location map. The GAM results folder contains the results for each of the general additive model (GAM).The src folder contains R scripts used to perform data processing, statistical analyses (including clustering, generalized additive models, and threshold analysis), and figure generation. This data package is associated with a GitHub repository found at https://github.com/WHONDRS-Hub/ECA_DOM_Thermodynamics.

Dissolved organic matter↗
DOE OSTI2024

Dataset for scientific paper "Simulated plant‑mediated oxygen input has strong impacts on fine‑scale porewater biogeochemistry and weak impacts on integrated methane fluxes in coastal wetlands", a modeling study based on field observation at the tidal salt marshes of the Parker River Estuary, Massachusetts, United States

This dataset is the raw and processed data for the paper "Simulated plant ‑ mediated oxygen input has strong impacts on fine ‑ scale porewater biogeochemistry and weak impacts on integrated methane fluxes in coastal wetlands". This study investigated how plant-mediated oxygen input affects subsurface biogeochemical reactions of organic carbon degradation and the resulting methane emissions of coastal wetlands by model simulation. We used the subsurface geochemical simulator PFLOTRAN for the modeling, which produced the simulated changes in porewater chemical substances and methane emissions over 10 days under different scenarios of plant-mediated oxygen input.Specifically, this dataset contains: 1) the input files for PFLOTRAN of all simulation runs conducted in this study. Those files are with an extension of ".in", containing information of the biogeochemical reaction network (stoichiometry, reaction rate, Monod constants, etc), fluid flow rate and oxygen concentration in the fluid which together simulated the plant-mediated oxygen input, the configuration of artificial reactions that simulated the methane fluxes, etc. The PFLOTRAN input files are text files, which can be opened by NotePad, but running these input files will require proper installation of PFLOTRAN (instruction: https://documentation.pflotran.org/user_guide/how_to/installation/installation.html). 2) the raw and processed model output from PFLOTRAN of all simulation runs, and 3) the python scripts used to process the raw model output, including random allocation of root cells, converting raw data into organized formats, calculating the methane fluxes based on the model output, data visualization, etc. The raw and processed model output from PFLOTRAN are in .spydata format, which can be viewed with Python. and 3) the python scripts for data processing and analysis are programming scripts, which can be opened with Python.This modeling work, in particular the model parameterization of root density and initial conditions of porewater concentrations of biogeochemical substances, was based on field measurements at the salt marsh of the Upper Parker River Estuary, Massachusetts, United States.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Pyrogenic Organic Matter Laboratory Experiment: Aerobic Respiration and Geochemistry from Variably Inundated Stream Sediments (v3)

This dataset supports a broader study examining the effects of variable inundation and pyrogenic organic matter on ecosystem respiration. The dataset provides data generated from a laboratory batch experiment investigating the interaction between variable inundation conditions (wet and dry sediment) and pyrogenic organic matter (burned and unburned treatments). The contents include time series dissolved oxygen, sediment geochemistry data, and field metadata (including qualitative information on instream and river corridor characteristics). This data package was originally published in November 2025. It was updated in April 2026 (v2; new and modified files) and May 2026 (v3; modified files). See the change history section in the readme for more details For details on how to navigate data packages generated by this project, see https://data.ess-dive.lbl.gov/portals/PNNLRiverCorridorSFA/About. In addition to a readme, this data package also includes a file-level metadata (FLMD) file that describes each file and a data dictionary (DD) that describes all column/row headers and variable definitions. This dataset is comprised of (1) file-level metadata; (2) data dictionary; (3) field metadata; (4) international generic sample number (IGSN) mapping file; (5) readme; (6) field protocol; (7) sample name metadata; (8) an environmental context picture for the dry and inundated sampling locations; and (9) a subfolder with sample data from the sediment incubation experiment. The sample data subfolder contains (1) dissolved organic carbon (DOC, measured as non-purgeable organic carbon, NPOC); (2) total nitrogen (TN); (3) gravimetric moisture; (4) partial pressure and production rates of carbon dioxide, methane, and nitrous oxide; (5) field wet sediment mass, dry sediment mass, water mass, and field wet sediment volume in incubation and sediment NPOC/TN vials; (6) methods codes; (7) respiration rates, pH, and temperature from after the incubation, raw time series dissolved oxygen and temperature, and a subfolder containing associated plots and scripts; (8) ions; (9) FTICR-MS methods; and (10) a subfolder of 12 Tesla (12T) FTICR-MS data. This folder contains the CoreMS processed data and three subfolders, one containing the .xml files, one containing the CoreMS output files, and the other containing instructions and scripts for processing the files in CoreMS (https://github.com/EMSL-Computing/CoreMS). All files are .csv, .pdf, .R, .xml, .html, .Rmd, .py, .cal, .json, or .jpg.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Leveraging Pre-Built Catalogs and Object-Level Scheduling to Eliminate I/O Bottlenecks in HPC Environments

Modern High-Performance Computing (HPC) environments face mounting challenges due to the shift from large to small file datasets, along with an increasing number of users and parallelized applications. As HPC systems rely on Parallel File Systems (PFS), such as Lustre for data processing, performance bottlenecks stemming from Object Storage Target (OST) contention have become a significant concern. Existing solutions, such as LADS with its object-level scheduling approach, fall short in large-scale HPC environments due to their inability to effectively address metadata I/O bottlenecks and the growing number of I/O processes. This study highlights the pressing need for a comprehensive solution that tackles both OST contention and metadata I/O challenges in diverse HPC workloads. To address these challenges, we propose SwiftLoad, an object-level I/O scheduling framework that leverages a metadata catalog to enhance the performance and efficiency of parallel HPC utilities. The adoption of the metadata catalog mitigates the metadata I/O bottlenecks that commonly occur in HPC utilities, a challenge that is particularly pronounced in object-level I/O scheduling. SwiftLoad addresses OST contention and the uneven distribution of I/O processes across different OSTs through mathematical modeling and incorporates a Loader Configuration Module to regulate the number of I/O processes. Evaluated with two representative utilities—data deduplication profiling and data augmentation—SwiftLoad achieved performance improvements of up to 5.63x and 11.0x, respectively, on a production supercomputer.

HPC↗
DOE OSTI2024

Connected Traffic Signal Coordination Optimization Framework through Network-Wide Adaptive Linear Quadratic Regulator–Based Control Strategy

Traffic congestion in metropolitan areas causes several significant challenges, such as longer travel times, decreased productivity, increased fuel consumption and vehicle emissions, and even severe injuries during crashes. Traffic signal control is a management approach to reduce traffic congestion and allocate the appropriate right of way for safety and mobility efficiency, both in temporal and spatial domains. Here, this study proposes a network-wide adaptive signal control coordination optimization framework based on the linear quadratic regulator algorithm. The traffic flow conditions driven by signal control inputs are formulated based on their network-wide state-space representation. After modeling traffic control regulation constraints, an adaptive linear quadratic regulator algorithm is designed to maximize the network-wide total throughput under the current conditions. Optimal signal control split time durations for multiple intersections in the network are derived by solving the algebraic Riccati equation. Furthermore, the recursive least square parameter estimation method is employed to quantify dynamic traffic condition changes. To verify the effectiveness of this proposed signal control framework, both simulation and real-world experimental tests are conducted for multiple intersections in downtown Chattanooga, Tennessee, United States. In preparation for real-world experimental tests, pipelines for real-time data processing implementation and historical traffic flow data analysis are conducted. The test results demonstrate that the proposed control framework achieves a decrease in travel time by up to 19.4%, total time spent (TTS) by up to 11.9%, and relative queue balance (RQB) by up to 15.6%. The research findings indicate that the proposed signal control framework can be generalized to handle large scale signal control optimization network-wide.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Online learning of quadratic manifolds from streaming data for nonlinear dimensionality reduction and nonlinear model reduction

Here, this work introduces an online greedy method for constructing quadratic manifolds from streaming data, designed to enable in situ analysis of numerical simulation data on the Petabyte scale. Unlike traditional batch methods, which require all data to be available upfront and take multiple passes over the data, the proposed online greedy method incrementally updates quadratic manifolds in one pass as data points are received, eliminating the need for expensive disk input/output operations as well as storing and loading data points once they have been processed. A range of numerical examples demonstrate that the online greedy method learns accurate quadratic manifold embeddings while being capable of processing data that far exceed common disk input/output capabilities and volumes as well as main-memory sizes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

A network of soil moisture, soil temperature, air temperature, net radiation, ground heat flux and ground water for Chicago, Illinois

This dataset contains environmental monitoring data collected using solar-powered Multi-Function Research (MFR) Long Range Wide Area (LoRaWAN)-enabled nodes at 11 sites in Chicago, Illinois, as part of the DOE Urban Integrated Field Lab CROCUS project. The MFR node system consists of an Input/Output Digital Input Module (IB8) interface box (ICT International) providing wired connections for environmental sensors and an MFR-Node-L data logger that manages power, data processing, and LoRaWAN communication. The wireless data are ingested via Sage network (https://sagecontinuum.org/) nodes that contain LoRaWAN antennae. Measurements were collected from 11 MFR nodes deployed across Chicago State University (CSU), Northeastern Illinois University (NEIU), Northwestern University (NU), University of Illinois Chicago (UIC), West Woodlawn "Blacks in Green" (BIG), and Indian Boundary Prairies (IBP). Each MFR node supports a consistent suite of sensors measuring atmospheric, soil, and hydrological variables. Atmospheric measurements include 2m air temperature (°C), 2m vapor pressure deficit (kPa), and 2m shortwave/longwave radiation (incoming and outgoing, W/m²) measured using ATH-VPD and Apogee SN500 sensors. Soil measurements include volumetric water content (VWC, %) and temperature (°C) at four depths (15, 30, 45, and 60 cm below surface) using Meter Teros54 sensors, and heat flux (W/m²) at 10 cm depth using Huske HFP01-05 sensors. At selected locations, Meter Hydros21 sensors measure groundwater depth (mm), specific conductivity (dS/m), and temperature (°C). The dataset includes timestamps, site identifiers with location names, device IDs, Global Positioning System (GPS) coordinates, variable names with units, measurement depths, values, sensor names, and Sage node identifiers. All timestamps are in local Chicago time (CDT/CST). Quality control flags are provided using a 6-bit binary system indicating physical range violations, step spikes, 24-hour flat-line conditions, 6-hour jitter, 7-day ultra-low variance, and persistent high offset. Data is provided in CSV and CF-compliant NetCDF formats. This dataset is part of a larger collection of CROCUS environmental monitoring data, including linked datasets from Air Quality Transmitter (AQT) sensors, Weather Transmitter (WXT) sensors, and Sap Flow Meter (SFM1x) sensors.

Chicago↗
DOE OSTI2025

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Lignin molecular weights of Populus trichocarpa residues after CELF pretreatment

Here we present a dataset of molecular weights of lignin from a woody energy crop (Populus trichocarpa) residues after a series of co-solvent enhanced lignocellulosic fractionation (CELF) pretreatment. The natural poplar variant GW-9947 from the Center for Bioenergy Innovation (CBI) was used. The poplar was knife milled and passed through a 1 mm sieve and CELF pretreatment was performed in a Parr autoclave reactor with 7.5 wt % solids loading, 0.5 wt% H2SO4 as catalyst at 150°C with various time. Tetrahydrofuran was added in a 1:1 mass ratio with water as the pretreatment solvent. Lignin was isolated from the pretreated samples after ball-milling in a porcelain jar with ceramic balls via Retsch PM 200 at 580 rpm for 2.5 h followed by enzymatic hydrolysis in acetate buffer (pH 4.8, 50 °C) for 48 h. The solid residue was isolated by centrifugation and hydrolyzed again with freshly added buffer and enzymes for another 48 h. After filtration, the solid residue was extracted twice with 96% (v/v) 1,4-dioxane/water mixture at room temperature overnight. The extracts were combined, rotary evaporated, and freeze-dried to recover lignin. The lignin samples were then derivatized in an acetic anhydride/pyridine (1:1, v/v) mixture and stirred at room temperature for 24 h. Ethanol was added to the reaction mixture, left for 30 min and then removed with a rotary evaporator. The addition and removal of ethanol was repeated at least 3 times until all traces of acetic acid were removed. Acetylated lignin samples were then dissolved in tetrahydrofuran (THF) at a concentration of 1.0 mg/mL. The molecular weight of acetylated lignin was measured by a gel permeation chromatography (GPC) on a PSS-Polymer Standards Service (Warwick, RI, USA) GPC SECurity 1200 system featuring Agilent HPLC 1200 components equipped with four Waters Styragel columns (HR1, HR2, HR4 and HR6) and an UV detector (270 nm). Tetrahydrofuran was used as the mobile phase and flow rate was 0.3 mL/min. The Polymer Standards Service WinGPC Unity software (Build 6807) was used for data processing for all the samples. The data provides information about the effects of CELF pretreatment time at 150 ºC on lignin molecular weights.

09 BIOMASS FUELS↗
DOE OSTI2026

Comparison of CNN-Based Image Classification Approaches for Implementation of Low-Cost Multispectral Arcing Detection

Camera-based sensing has benefited in recent years from developments in machine learning data processing methods, as well as improved data collection options such as Unmanned Aerial Vehicles (UAV) mounted sensors. However, cost considerations, both for the initial purchase of sensors as well as updates, maintenance, or potential replacement if damaged, can limit adoption of more expensive sensing options for some applications. To evaluate more affordable options with less expensive, more available, and more easily replaceable hardware, we examine the use of machine learning-based image classification with custom datasets, utilizing deep learning based-image classification and the use of ensemble models for sensor fusion. Utilizing the same models for each camera to reduce technical overhead, we showed that for a very representative training dataset, camera-based detection can be successful for detection of electrical arcing. We also use multiple validation datasets, based on conditions expected to be of varying difficulty, to evaluate custom data. These results show that ensemble models of different data sources can mitigate risks from gaps in training data, though the system will be less redundant for those cases unless other precautions are taken. We found that with good quality custom datasets, data fusion models can be utilized without specialization in design to the specific cameras utilized, allowing for less specialized, more accessible equipment to be utilized as multispectral camera components. This approach can provide an alternative to expensive sensing equipment for applications in which lower-cost or more easily replaceable sensing equipment is desirable.

convolutional neural networks↗
DOE OSTI2024

Modular Subsurface Sensors and Integrated Software for Advanced Subsurface Characterization and Monitoring using Unoccupied Vehicles

The advent and subsequent proliferation of autonomous airborne, waterborne, and groundbased vehicles (i.e., “drones”) promises to broadly transform the geosciences and associated industries, including fossil energy exploration and development, mineral resource exploration and development, water-resource management, and environmental remediation. For geophysical characterization and monitoring, the prospect of programming highly repeatable and low-cost drone missions for subsurface imaging will allow for deployments in hazardous and previously inaccessible areas. Coupled with autonomous workflows for data processing, management, and visualization, drone-based geophysical characterization and monitoring will enable unprecedented, real-time insight into diverse subsurface properties and processes of scientific and engineering importance. Toward this end, the objectives of this Lab Directed Research and Development (LDRD) project were to develop new (1) instrumentation for dronebased electromagnetic induction (EMI) geophysical imaging, including separated transmitter and receivers and associated electronics, (2) software for real-time data telemetry, processing, management, and visualization. Although EMI has been previously deployed using unoccupied aerial systems (UASs), these applications failed to capitalize on the game-changing capabilities of drone platforms. Whereas drone-based data acquisition allows for collection of rich, three-dimensional (3D) multi-offset/multi-angle configurations between transmitters and receivers, past efforts have relied on conventional instrumentation that was designed for ground-based data collection with the transmitter and a single receiver housed in the same unit; nor did these previous applications demonstrate real-time delivery of results to support rapid management decisions in the field. In this 1-year project, we (1) designed and constructed new lightweight independent transmitter and receiver antenna platforms that communicate with a laptop computer; (2) developed software to control data acquisition, manage/transfer data, and visualize data as its collected; and (3) demonstrated the operation of the new hardware and software systems in a ground-based field test. Our work entails major technological advances for EMI and established a foundation on which to build a new drone-based, real-time geophysical EMI imaging capability to support diverse challenges facing the nation.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

High-count rate effects in event processing for XRISM/ Resolve X-ray microcalorimeter: I. Ground test

The spectroscopic performance of an X-ray microcalorimeter is compromised at high count rates. We utilize the Resolve X-ray microcalorimeter onboard the XRISM satellite to examine the effects observed during high-count rate measurements and propose modeling approaches to mitigate them. We specifically address the following instrumental effects that impact performance: CPU limit, pile-up, and untriggered electrical cross-talk. Experimental data at high count rates were acquired during ground testing using the flight model instrument and a calibration X-ray source. In the experiment, data processing not limited by the performance of the onboard CPU was run in parallel, which cannot be done in orbit. This makes it possible to access the data degradation caused by limited CPU performance. We use these data to develop models that allow for a more accurate estimation of the aforementioned effects. To illustrate the application of these models in observation planning, we present a simulated observation of GX 13+1. Understanding and addressing these issues is crucial to enhancing the reliability and precision of X-ray spectroscopy in situations characterized by elevated count rates.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Carbon dioxide, water vapor and methane soil efflux (soil respiration) in a Pinus palustris root exclusion in Georgetown, SC

This dataset contains processed data from a combination of survey flux chambers and long-term automated flux chambers. Soil flux measurements were conducted from June 2023 through December 2025 in a mature longleaf pine forest in Georgetown, SC. Soil respiration measurements were conducted approximately biweekly for two and a half years, before and after a root exclusion that took place on May 5, 2024. Processed, QAQC’d data for the treatment (root exclusion) and control (roots intact) before and after the root exclusion can be found in the file: 1_DATA_ESS_DOE_HR_RS_HB2_QAQC_Survey_Data_20260223.csv. Two multiday deployments were also conducted prior to the root exclusion using long-term automated chambers to continuously monitor greenhouse gas soil efflux. Processed, QAQC’d data for both long-term deployments can be found in the file: 2_DATA_ESS_DOE_HR_RS_HB2_QAQC_Longterm_Data_20260209.csv. Raw and working data files (.json, .81x, & .82z format) from LI-COR equipment are included for reference and can be accessed using SoilFluxPro software. CSV metadata files describe the raw data and modifications made using SoilFluxPro v5 and Matlab R2024b, as well as formatting and units for processed CSVs. Matlab code is included for reading in the processed CSVs, with sample figures comparing treatment and control. This research was performed as part of the project: “Improving models of stand and watershed carbon and water fluxes with more accurate representations of soil-plant-water dynamics in southern pine ecosystems”, which examines in part the effects hydraulic redistribution on soil efflux of carbon dioxide, water vapor and methane, as well as soil moisture and temperature in a southern pine ecosystem with sandy soils and high water table.

CARBON DIOXIDE FLUX↗
DOE OSTI2025

A systematic review of machine learning in groundwater monitoring

With increasing concerns about water scarcity, groundwater has become crucial since this resource provides most of the freshwater needs. However, various human and natural activities often contaminate the groundwater, making it unsuitable for use. Over the years, scientists and engineers have used many methods to predict and track groundwater contamination as part of environmental monitoring. Consequently, there is an urgent need for improved methods, particularly in the face of increasing contamination. Machine learning has sometimes been used to monitor groundwater, air quality, and climate. Traditional methods must be improved due to the complexity and large amount of environmental data. This includes using hybrid models that combine traditional and new techniques. Despite the use of machine learning in many scientific areas, there is a lack of comprehensive reviews focusing on its use in environmental monitoring, especially groundwater monitoring. We aim to fill this gap by exploring machine-learning applications in groundwater monitoring. We discuss relevant methods, their limitations, and future potential. We summarize research on automating data processing and model training using groundwater sensor data. Our research underscores the transformative potential of machine learning to revolutionize long-term groundwater monitoring and contamination detection, providing valuable insights for future research and practical applications.

AI/ML↗
DOE OSTI2025

Accounting for electron-beam-induced warping of molecular nanocrystals in MicroED structure determination

High-energy electrons induce sample damage and motion at the nanoscale to fundamentally limit the determination of molecular structures by electron diffraction. Using a fast event-based electron counting (EBEC) detector, we characterize beam-induced, dynamic, molecular crystal lattice reorientations (BIRs). These changes are sufficiently large to bring reciprocal lattice points entirely in or out of intersection with the sphere of reflection, occur as early events in the decay of diffracted signal due to radiolytic damage, and coincide with beam-induced migrations of crystal bend contours within the same fluence regime and at the same illuminated location on a crystal. These effects are observed in crystals of biotin, a series of amino acid metal chelates, and a six-residue peptide, suggesting that incident electrons inevitably warp molecular lattices. The precise orientation changes experienced by a given microcrystal are unpredictable but are measurable by indexing individual diffraction patterns during beam-induced decay. Reorientations can often tilt a crystal lattice several degrees away from its initial position before irradiation, and for an especially beam-sensitive Zn(II)-methionine chelate, are associated with dramatic crystal quakes prior to 1 e − Å −2 electron beam fluence accumulates. Since BIR coincides with the early stages of beam-induced damage, it echoes the beam-induced motion observed in single-particle cryoEM. As with motion correction for cryoEM imaging experiments, accounting for BIR-induced errors during data processing could improve the accuracy of MicroED data.

Vlahakis, Niko (ORCID:0000000250920265)↗
DOE OSTI2025

HPDR: High-Performance Portable Scientific Data Reduction Framework

The rapid growth in scientific data generation is outpacing advancements in computing systems necessary for efficient storage, transfer, and analysis, particularly in the context of exascale computing. With the deployment of first-generation exascale computing systems and next-generation experimental facilities, this gap is widening and necessitates effective data reduction techniques to manage enormous data volumes. Over the past decade, various data reduction methods, including lossless compression, error-controlled lossy compression, and data refactoring, have been developed to accelerate I/O in scientific workflows. Despite significant reductions in data volume, these methods introduce considerable computational overhead, which can become the new bottleneck in data processing. To mitigate this, GPU-accelerated data reduction algorithms have been introduced. However, challenges remain in their integration into exascale workflows, including limited portability across different GPU architectures, substantial memory transfer overhead, and reduced scalability on dense multi-GPU systems. To address these challenges, we propose HPDR, a high-performance and portable data reduction framework. HPDR is designed to enable the execution of state-of-the-art reduction algorithms across diverse processor architectures while reducing memory transfer overhead to 2.3 % of the original, resulting in up to 3.5× faster throughput compared to existing solutions. It also achieves up to 96% of the theoretical speedup in multi-GPU settings. In addition, evaluations on accelerating I/O operations at scale up to 1,024 nodes of the Frontier supercomputer demonstrate that HPDR can achieve up to 103 TB/s reduction throughput, providing up to 4× acceleration in parallel I/O performance compared to existing data reduction routines. This work highlights the potential of HPDR to significantly enhance data reduction efficiency in exascale computing environments.

Chen, Jieyang [University of Oregon]↗
DOE OSTI2024

Road Lidar Dataset for the TxDOT Austin District

This is a road lidar data collection for developing road elevation models and road inundation mapping methodologies, a joint work between ORNL and The University of Texas at Austin. This dataset is generated as part of the flood transportation infrastructure, partly funded by the NOAA CIROH project. ORNL is a project partner for high-performance computing-empowered flood inundation mapping methodology R&D. The dataset is computed using a GPU-accelerated lidar data processing workflow developed at ORNL. The lidar data source is from TxGIO, the state lidar data collection site. The output dataset is in two formats: laz and copc. It is organized by TxDOT's maintenance sections, covering the Austin District. Data size: 3.86 billion road lidar points, 1.67% of the entire lidar data input Projection: EPSG:32614 (WGS84/UTM zone 14N) Website: https://web.corral.tacc.utexas.edu/nfiedata/road3d/austin_district/AustinMaintenanceSections_H_epsg6343_V_epsg5703/ LICENSE FOR USE -- MAPS AND DATA DISCLAIMER This resource is shared under the Creative Commons Attribution CC BY, http://creativecommons.org/licenses/by/4.0/ MAPS AND DATA DISCLAIMER The Oak Ridge National Laboratory (ORNL) shall not be held liable for improper or incorrect use of the data described or information contained on this map or associated series of maps. The data and related map graphics are not legal, land survey or engineering documents and are not intended to be used as such. ORNL gives no warranty, express or implied, as to the accuracy, reliability, utility or completeness of this information. The user of these maps and data assumes all responsibility and risk for the use of the maps and data. ORNL disclaims all warranties, representations or endorsements either express or implied, with regard to the information contained in this map product, including, but not limited to, all implied warranties of merchantability, fitness for a particular purpose or non-infringement. This preliminary map product is for research and review purposes only. It is not intended to be used for emergency management operational or life safety decisions at the local or regional governmental level or by the general public. Users requiring information regarding hazardous conditions or meteorological conditions for specific geographic areas should consult directly with their city or county emergency management office.

54 ENVIRONMENTAL SCIENCES↗