Search NASA⌕ Search

SEARCH · Search NASA

Results for “data pipelines”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 919 records · Page 51
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

White-Rabbit-Disciplined FPGA Readout for Fermilab Timing Events

Fermilab's accelerator timing links broadcast short event codes to thousands of devices at once, but the links themselves carry no absolute notion of time; that comes separately from a White Rabbit reference. This work builds the piece that ties the two together on a single board. On a Xilinx Kria KR260 (Zynq UltraScale+), the programmable logic decodes a real Fermilab TCLK link, stamps every event with an absolute White-Rabbit \{sec, ns\} UTC time, and reads the timestamped stream out over AXI4-Lite; a thin Linux process on the same die publishes each event into a Redis stream on the control network. To exercise the full chain on one board, the decoded events are re-encoded as gigabit ACLK, transmitted out an SFP+ optical port, looped back over a short fiber jumper, and decoded again on the same timeline, and are additionally mirrored as an ACLK-Lite Manchester waveform for benchtop probing. Across sustained, multi-day testing against real Fermilab TCLK, the pipeline has decoded, timestamped, and published hundreds of millions of events with practically zero loss, and folding the timestamped stream on the 60-second accelerator supercycle recovers the machine's periodic structure directly from the published data.

Rossel, Jacob [Fermilab; UC, Berkeley (main)] (ORC↗
DOE OSTI2025

Dark Energy Survey Year 6 Results: Synthetic-source Injection Across the Full Survey Using Balrog

Synthetic source injection (SSI), the insertion of sources into pixel-level on-sky images, is a powerful method for characterizing object detection and measurement in wide-field, astronomical imaging surveys. Within the Dark Energy Survey (DES), SSI plays a critical role in characterizing all necessary algorithms used in converting images to catalogs, and in deriving quantities needed for the cosmology analysis, such as object detection rates, galaxy redshift estimation, galaxy magnification, star-galaxy classification, and photometric performance. We present here a source injection catalog of 146 million injections spanning the entire 5000 deg 2 DES footprint, generated using the Balrog SSI pipeline. Through this SSI sample, we demonstrate that the DES Year 6 (Y6) image processing pipeline provides accurate estimates of the object properties, for both galaxies and stars, at the percent-level, and we highlight specific regimes where the accuracy is reduced. We then show the consistency between SSI and data catalogs, for all galaxy samples developed within the weak lensing and galaxy clustering analyses of DES Y6. The consistency between the two catalogs also extends to their correlations with survey observing properties (seeing, airmass, depth, extinction, etc.). Lastly, we highlight a number of applications of this catalog to the DES Y6 cosmology analysis, such as estimates of the redshift distribution and lens magnification. This dataset is the largest SSI catalog produced at this fidelity and will serve as a key testing ground for exploring the utility of SSI catalogs in upcoming surveys such as the Vera C. Rubin Observatory Legacy Survey of Space and Time.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

The Geothermal Data Repository: Ten Years of Supporting the Geothermal Industry with Open Access to Geothermal Data

The Department of Energy's (DOE) Geothermal Data Repository (GDR) is celebrating its tenth anniversary! Over the last decade it has grown from the simple idea of storing public data in a centralized location to a valuable tool at the center of the US geothermal scientific community and an integral part of the DOE Geothermal Technologies Office (DOE GTO) project management strategy. Researchers funded by the DOE GTO have contributed over 1,300 data submissions to the GDR. These data have been used to further advancements in geothermal science, economic analysis, exploration, research, development, and operational efficiency. The adoption of open data methodologies and a data management strategy that prioritizes universal open access and standardized, interoperable data have further increased the value of GDR data, making them available across a distributed network of data sharing partners and improving their utility to other industries and related fields, including material science and space exploration. Incorporating feedback from users has been critical to the GDR's success, allowing it to grow over the years to meet the evolving needs of the geothermal community. This paper will explore some of many changes that occurred throughout the GDRs tenure and the lessons learned along the way, as well as highlight some of the new features and recent improvements that been implemented to support innovation, reduce duplication of effort, and advance the geothermal industry as a whole.

access↗
DOE OSTI2026

A Single-Board Fermilab Timing Pipeline on the Xilinx KR260: Decode, Timestamp, Publish, Mirror

Full-width abstract \renewcommand{\maketitlehookd}{% \begin{abstract} \noindent Fermilab's accelerator timing links broadcast short event codes to thousands of devices at once, but the links themselves carry no absolute notion of time; that comes separately from a White Rabbit reference. This work builds the piece that ties the two together on a single board. On a Xilinx Kria KR260 (Zynq UltraScale+), the programmable logic decodes a real Fermilab TCLK link, stamps every event with an absolute White-Rabbit \{sec, ns\} UTC time, and reads the timestamped stream out over AXI4-Lite; a thin Linux process on the same die publishes each event into a Redis stream on the control network. To exercise the full chain on one board, the decoded events are re-encoded as gigabit ACLK, transmitted out an SFP+ optical port, looped back over a short fiber jumper, and decoded again on the same timeline, and are additionally mirrored as an ACLK-Lite Manchester waveform for benchtop probing. Across sustained, multi-day testing against real Fermilab TCLK, the pipeline has decoded, timestamped, and published hundreds of millions of events with practically zero loss, and folding the timestamped stream on the 60-second accelerator supercycle recovers the machine's periodic structure directly from the published data.

Rossel, Jacob [UC, Berkeley; Fermilab] (ORCID:0009↗
DOE OSTI2025

Attention-based functional-group coarse-graining: a deep learning framework for molecular prediction and design

Machine learning (ML) offers considerable promise for the design of new molecules and materials. In real-world applications, the design problem is often domain-specific, and suffers from insufficient data, particularly labeled data, for ML training. In this study, we report a data-efficient, deep-learning framework for molecular discovery that integrates a coarse-grained functional-group representation with a self-attention mechanism to capture intricate chemical interactions. Our approach exploits group-contribution concepts to create a graph-based intermediate representation of molecules, serving as a low-dimensional embedding that substantially reduces the data demands typically required for training. Using a self-attention mechanism to learn the subtle but highly relevant chemical context of functional groups, the method proposed here consistently outperforms existing approaches for predictions of multiple thermophysical properties. In a case study focused on adhesive polymer monomers, we train on a limited dataset comprising only 6,000 unlabeled and 600 labeled monomers. The resulting chemistry prediction model achieves over 92% accuracy in forecasting properties directly from SMILES strings, exceeding the performance of current state-of-the-art techniques. Furthermore, the latent molecular embedding is invertible, enabling the design pipeline to automatically generate new monomers from the learned chemical subspace. We illustrate this functionality by targeting several properties, including high and low glass transition temperatures (Tg), and demonstrate that our model can identify new candidates with values that surpass those in the training set. The ease with which the proposed framework navigates both chemical diversity and data scarcity offers a promising route to accelerate and broaden the search for functional materials.

Han, Ming [Univ. of Chicago, IL (United States)]↗
DOE OSTI2022

Dark Energy Survey Year 3 results: A 2.7% measurement of baryon acoustic oscillation distance scale at redshift 0.835

We present angular diameter measurements obtained by measuring the position of baryon acoustic oscillations (BAO) in an optimized sample of galaxies from the first three years of Dark Energy Survey data (DES Y3). The sample consists of 7 million galaxies distributed over a footprint of 4100 deg 2 with 0.6 < z photo < 1.1 and a typical redshift uncertainty of 0.03 ( 1 + z ) . The sample selection is the same as in the BAO measurement with the first year of DES data, but the analysis presented here uses three times the area, extends to higher redshift, and makes a number of improvements, including a fully analytical BAO template, the use of covariances from both theory and simulations, and an extensive preunblinding protocol. We used two different statistics; angular correlation function and power spectrum, and validate our pipeline with an ensemble of over 1500 realistic simulations. Both statistics yield compatible results. We combine the likelihoods derived from angular correlations and spherical harmonics to constrain the ratio of comoving angular diameter distance D M at the effective redshift of our sample to the sound horizon scale at the drag epoch. We obtain D M ( z eff = 0.835 ) / r d = 18.92 ± 0.51 , which is consistent with, but smaller than, the Planck prediction assuming flat Λ CDM , at the level of 2.3 σ . The analysis was performed blind and is robust to changes in a number of analysis choices. It represents the most precise BAO distance measurement from imaging data to date, and is competitive with the latest transverse ones from spectroscopic samples at z > 0.75 . When combined with DES 3 x 2 pt + SNIa , they lead to improvements in H 0 and Ω m constraints by ∼ 20 % .

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Dark Energy Survey Year 3 Results: Galaxy mock catalogs for BAO analysis

The calibration and validation of scientific analysis in simulations is a fundamental tool to ensure unbiased and robust results in observational cosmology. In particular, mock galaxy catalogs are a crucial resource to achieve these goals in the measurement of baryon acoustic oscillation (BAO) in the clustering of galaxies. Here we present a set of 1952 galaxy mock catalogs designed to mimic the Dark Energy Survey Year 3 BAO sample over its full photometric redshift range 0.6 < zphoto < 1.1. The mocks are based upon 488 ICE-COLA fast N-body simulations of full-sky light cones and were created by populating halos with galaxies, using a hybrid halo occupation distribution – halo abundance matching model. This model has ten free parameters, which were determined, for the first time, using an automatic likelihood minimization procedure. We also introduced a novel technique to assign photometric redshift for simulated galaxies, following a two-dimensional probability distribution with VIMOS Public Extragalactic Redshift Survey data. The calibration was designed to match the observed abundance of galaxies as a function of photometric redshift, the distribution of photometric redshift errors, and the clustering amplitude on scales smaller than those used for BAO measurements. An exhaustive analysis was done to ensure that the mocks reproduce the input properties. Finally, mocks were tested by comparing the angular correlation function w(θ), angular power spectrum Cℓ, and projected clustering ξp(r⊥) to theoretical predictions and data. The impact of volume replication in the estimate of the covariance is also investigated. The success in accurately reproducing the photometric redshift uncertainties and the galaxy clustering as a function of redshift render this mock creation pipeline as a benchmark for future analyses of photometric galaxy surveys.Key words: catalogs / large-scale structure of Universe / galaxies: distances and redshifts / Galaxy: halo / methods: numerical

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Identifying Adversarial Cyber-Activity in Operational Technology Environments Using Bayesian Networks

Critical infrastructure and other operational technology (OT) environments face increasing cybersecurity risks from adversarial behavior. This paper describes the development of a risk model using a Bayesian network to enhance the comprehension of observable cyber events caused by malicious activity in OT environments. The core of the Bayesian network is a process model that describes the stages of adversary behavior. The remainder of the model is based on the MITRE ATT&CK® for Industrial Control Systems (ICS) taxonomy, which includes tactics and techniques that may be used by the adversary. The observables provide evidence for adversary behavior through the intermediary technique and tactic nodes. One challenge in constructing this model is a lack of open-source data from cyber-attacks on OT systems. This paper discusses learning from limited data, the elicitation of expert opinion to construct the conditional probability tables when data is scarce, and the refinement of the most difficult conditional probabilities tables using several forms of sensitivity analyses. Finally, the Bayesian network is demonstrated using two historical case studies: the DarkSide ransomware attack on the Colonial Pipeline and the destructive cyberattack targeting the ThyssenKrupp blast furnace. Index Terms—Cybersecurity, industrial control systems, operational technology

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Unveiling the Arsenal of Apple Bitter Rot Fungi: Comparative Genomics Identifies Candidate Effectors, CAZymes, and Biosynthetic Gene Clusters in Colletotrichum Species

The bitter rot of apple is caused by Colletotrichum spp. and is a serious pre-harvest disease that can manifest in postharvest losses on harvested fruit. In this study, we obtained genome sequences from four different species, C. chrysophilum, C. noveboracense, C. nupharicola, and C. fioriniae, that infect apple and cause diseases on other fruits, vegetables, and flowers. Our genomic data were obtained from isolates/species that have not yet been sequenced and represent geographic-specific regions. Genome sequencing allowed for the construction of phylogenetic trees, which corroborated the overall concordance observed in prior MLST studies. Bioinformatic pipelines were used to discover CAZyme, effector, and secondary metabolic (SM) gene clusters in all nine Colletotrichum isolates. We found redundancy and a high level of similarity across species regarding CAZyme classes and predicted cytoplastic and apoplastic effectors. SM gene clusters displayed the most diversity in type and the most common cluster was one that encodes genes involved in the production of alternapyrone. Our study provides a solid platform to identify targets for functional studies that underpin pathogenicity, virulence, and/or quiescence that can be targeted for the development of new control strategies. With these new genomics resources, exploration via omics-based technologies using these isolates will help ascertain the biological underpinnings of their widespread success and observed geographic dominance in specific areas throughout the country.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

SynthEsizing Novel H2 Sensors for Operational Resilience in Pipeline Infrastructure (SENSOR)

A flexible and extensible computational framework acts as a black-box materials discovery engine, capable of screening, predicting, and designing advanced materials with minimal manual intervention was developed. While developed for hydrogen sensing, the approach can be readily adapted to other materials challenges, offering a powerful tool for data-driven materials innovation.

08 HYDROGEN↗
DOE OSTI2026

Dark Energy Survey Year 3 results: optimized $w$CDM simulation-based inference with weak lensing map-level hybrid statistics

We present cosmological constraints from the Dark Energy Survey Year 3 (DES Y3) weak lensing data using hierarchical hybrid statistics within a Bayesian simulation-based inference framework that is based on the Gower Street simulations. To maximize the precision of the inference, we have developed a new, information-theory based, data compression of the weak lensing maps to just seven highly informative summary statistics. The hybrid scheme exploits the high information content of the power spectrum, compressing both the power spectrum and neural-based summaries that are designed to extract further information. Our simulation-based approach enables principled forward modelling of all major sources of systematic uncertainty and survey properties into realistic mock observations, including the survey mask, photometric redshift uncertainties, intrinsic galaxy alignments, multiplicative shear calibration bias, source galaxy clustering, non-Gaussian shape noise, and non-linear structure formation. The summary statistics are then used in a Bayesian simulation-based inference pipeline. The inference is validated through coverage tests and checks for robustness against baryonic feedback. Assuming a $w$CDM cosmology, our analysis yields $S_8 = 0.808 \pm 0.017$, $Ω_{\rm m} = 0.325 \pm 0.024$, and $w < -0.766$ (marginalized posterior 68 per cent credible intervals). This rigorous combination of information theory, physics- and neural network-based extreme data compression, and principled Bayesian analysis improves the figure of merit for $(Ω_{\rm m}, S_8, w)$ by 60 per cent over the previous state-of-the-art, and by almost a factor of 3 over two-point analyses of the same data. They are the most precise joint constraints on $(Ω_{\rm m}, S_8, w)$ from weak gravitational lensing data alone of any survey to date. We intend to apply this analysis to the more recent DES Y6 data.

Williamson, J. [University Coll. London]↗
DOE OSTI2021

Detecting optical transients using artificial neural networks and reference images from different surveys

ABSTRACT We present a technique to detect optical transients based on an artificial neural networks method. We describe the architecture of two networks capable of comparing images of the same part of the sky taken by different telescopes. One image corresponds to the epoch in which a potential transient could exist; the other is a reference image of an earlier epoch. We use data obtained by the Dr. Cristina V. Torres Memorial Astronomical Observatory and archival reference images from the Sloan Digital Sky Survey. We trained a convolutional neural network and a dense layer network on simulated source samples and then tested the trained networks on samples created from real image data. Autonomous detection methods replace the standard process of detecting transients, which is normally achieved by source extraction of a difference image followed by human inspection of the detected candidates. Replacing the human inspection component with an entirely autonomous method would allow for a rapid and automatic follow-up of interesting targets of opportunity. The toy-model pipeline that we present here is not yet able to replace human inspection, but it might provide useful hints to identify potential candidates. The method will be further expanded and tested on telescopes participating in the Transient Optical Robotic Observatory of the South Collaboration.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Fixed-target serial crystallography at the Structural Biology Center

Serial synchrotron crystallography enables the study of protein structures under physiological temperature and reduced radiation damage by collection of data from thousands of crystals. The Structural Biology Center at Sector 19 of the Advanced Photon Source has implemented a fixed-target approach with a new 3D-printed mesh-holder optimized for sample handling. The holder immobilizes a crystal suspension or droplet emulsion on a nylon mesh, trapping and sealing a near-monolayer of crystals in its mother liquor between two thin Mylar films. Data can be rapidly collected in scan mode and analyzed in near real-time using piezoelectric linear stages assembled in an XYZ arrangement, controlled with a graphical user interface and analyzed using a high-performance computing pipeline. Here, the system was applied to two β-lactamases: a class D serine β-lactamase from Chitinophaga pinensis DSM 2588 and L1 metallo-β-lactamase from Stenotrophomonas maltophilia K279a.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Fast Algorithms for Scientific Data Compression

Many scientific simulations and experiments generate terabytes to petabytes of data daily, necessitating data compression techniques. Unlike video and image compression, scientists require methods that accurately preserve primary data (PD) and derived quantities of interest (QoIs). In our previous work, we demonstrated the effectiveness of hybrid compression techniques that combine machine learning with traditional approaches. This paper presents innovative computational techniques aimed at expediting the compression pipeline. Our experiments, conducted on two distinct platforms with a large-scale XGC-based fusion simulation, demonstrate that the overhead incurred by these new approaches is less than one percent of the computational resources needed for the simulation.

Banerjee, Tania↗
DOE OSTI2020

An Overview of the Waste-to-Energy System Simulation (WESyS) Model

The leveraging of waste streams for energy and chemical production could add revenue to waste disposal operations, and it presents opportunities for addressing a variety of economic and environmental objectives at the local, state, and national levels. The Waste-to-Energy System Simulation (WESyS) model is a system dynamics model that was created to simulate the development of the U.S. waste-to-energy industry over time. For each of the three primary waste resources modeled (landfills, concentrated animal feeding operations, and publicly owned treatment works), WESyS simulates technically feasible scenarios for use of the waste, including direct conversion to fuels, and anaerobic digestion followed by flaring, electricity generation, combined heat and power, cleanup and compression to compressed natural gas, and cleanup and injection into an existing pipeline. The model allows users to explore numerous plausible future scenarios for the development of the U.S. waste-to-energy industry. This report provides an overview of the WESyS model and documents the key assumptions, equations, and data sources used to create the model.

09 BIOMASS FUELS↗
DOE OSTI2024

Effect of Drying on Corrosion Mitigation of Hanford Transfer Lines

Radioactive waste is stored in underground, carbon-steel double-shell tanks at the Department of Energy Hanford site. The waste is transferred between the tanks and other assets using the transfer lines spanning throughout the various tank farms at Hanford. The transfer lines consist of a pipe-in-pipe design, small diameter pipes, and are not piggable. Recent inspection data of the transfer lines have shown areas with corrosion on both interior of the encasements and exterior of the primary pipes, with nearly 50 percent wall loss on the primary pipes and nearly 25% wall loss on the encasement pipes due to pitting corrosion. The visual inspections of the transfer lines have shown presence of corrosion products near the pipeline risers and beyond. It has been hypothesized that the corrosion is predominantly due to the high humidity conditions and in some cases is driven by the presence of residual hydrotest water in the encasement and the associated contact with the safety significant primary pipe. Therefore, drying of the transfer lines could lead to corrosion mitigation. Experimental studies are being conducted to understand the effect of environmental conditions, especially, relative humidity and temperature, on transfer line grade carbon steel corrosion and on mitigating corrosion. The experimental conditions are selected based on the seasonal temperature changes, and relative humidity conditions ranging from 30 to 100 percent. The experimental data will be used as guidance for maintaining a dry environment that will help mitigate the transfer-line corrosion caused by the high humidity conditions.

Shukla, Pavan K.↗
DOE OSTI2021

Could on-site fuel storage economically reduce power plant-gas grid dependence in pipeline constrained areas like New England?

In the Northeastern United States, natural gas supply constraints have led to periods when gas shortages have caused up to a quarter of all unscheduled power plant outages. Dual fuel oil/gas generators or local gas storage might mitigate gas supply shortages. We use historical power plant operational and availability data to develop a supply curve of the costs required for generators to mitigate fuel shortage failures in New England. Based on 2012–2018 data, we find that the historical fuel shortages at approximately 2 GW worth of gas-fired capacity could be mitigated using on-site fuel storage. For comparison, New England’s average reserve margin was 1.7–2.8 GW over our sample period. Oil dual fuel plants would recoup their investment if compensated with a reliability adder of $\$ 3-7$/MWh during their normal operations, while $\$ 7-16$/MWh would incentivize using onsite, compressed natural gas storage. We estimate that the capital expenses associated with the fuel storage options would be less expensive than installing battery backup for resource adequacy at current battery prices.

03 NATURAL GAS↗
DOE OSTI2025

Bridging the gap: Deploying AI-based Models in Real-Time Fusion Plasma Control Systems

Achieving reliable real-time control in fusion plasma experiments requires strict timing guarantees across entire control algorithms. In earlier work by Abbate et al. (2023), we demonstrated the feasibility of neural-network-based control algorithms on the DIII-D tokamak using the internally developed open-source Keras2C library for model conversion into C (Conlin et al. (2021)). However, the initial implementations relied on data buffering and branching logic outside the neural network code, causing variability in execution times. Subsequent deployments on DIII-D and KSTAR—including the RTCAKENN algorithm for kinetic profile reconstruction—proved that minimizing branching and buffering throughout the pipeline yields consistent millisecond-level cycle times under real experimental conditions (Shousha et al. (2023)). However, keeping pace with rapidly evolving AI frameworks (e.g. PyTorch) is challenging. Finally, we, therefore, propose a community-driven open-source effort to expand the tool, enabling real-time deployment across diverse systems that require strictly bounded execution times.

AI-based models↗