Search NASA⌕ Search

SEARCH · Search NASA

Results for “data cleaning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Reconstruction framework advancements to support streaming for the ePIC detector at the EIC

The ePIC collaboration adopted the JANA2 framework to manage its reconstruction algorithms. This framework has since evolved substantially in response to ePIC’s needs. There have been three main design drivers: integrating cleanly with the Podio-based data models and other layers of the key4hep stack, enabling external configuration of existing components, and supporting timeframe splitting for streaming readout. The result is a unified component model featuring a new declarative interface for specifying inputs, outputs, parameters, services, and resources. This interface enables the user to instantiate, configure, and wire components via an external file. One critical new addition to the component model is a hierarchical decomposition of data boundaries into levels such as Run, Timeframe, PhysicsEvent, and Subevent. Two new component abstractions, Folder and Unfolder, are introduced in order to traverse this hierarchy, e.g. by splitting or merging. The pre-existing components can now operate at different event levels, and JANA2 will automatically construct the corresponding parallel processing topology. This means that a user may write an algorithm once, and configure it at runtime to operate on timeframes or on physics events. Overall, these changes mean that the user requires less knowledge about the framework internals, obtains greater flexibility with configuration, and gains the ability to reuse the existing abstractions in new streaming contexts.

Brei, Nathan [Thomas Jefferson National Accelerato↗
DOE OSTI2026

Database of virus genomes from ultra-deep sequencing of wastewater

Researchers at University of Missouri have conducted ultra-deep RNA sequencing of viral concentrates from wastewater (1 billion Illumina reads per sample). The resulting dataset spans 321 samples collected weekly from 11 cities between 2023-2025. As part of a tri-lab collaboration, scientists at LLNL and LANL cleaned, assembled, and annotated this metagenomic data, identifying nearly 200,000 viral genomes. Careful data curation resulted in a database containing 21,015 high-quality, near-complete viral genomes from wastewater. This database contains viruses predicted to infect a range of hosts including bacteria (most common viruses), plants (most abundant viruses), and vertebrates (rarest viruses). There are also numerous novel viruses that could not be well identified and whose host(s) are unknown. Just 7% of all genomes in the wastewater virus database had genus-level matches in the public NCBI database, and 17% matched to a recently created metagenomic virus database at that level (metaVR). The database will provide baseline information about viruses in wastewater that may be used to additional identify novel viruses during ongoing monitoring

Allen, Jonathan [Lawrence Livermore National Labor↗
DOE OSTI2025

A DECADE of dwarfs: first detection of weak lensing around spectroscopically confirmed low-mass galaxies

We present the first detection of weak gravitational lensing around spectroscopically confirmed dwarf galaxies, using the large overlap between DESI DR1 spectroscopic data and DECADE/DES weak lensing catalogs. A clean dwarf galaxy sample with well-defined redshift and stellar mass cuts enables excess surface mass density measurements in two stellar mass bins ($\log \rm{M}_*=[8.2, 9.2]~M_\odot$ and $\log \rm{M}_*=[9.2, 10.2]~M_\odot$), with signal-to-noise ratios of $5.6$ and $12.4$ respectively. This signal-to-noise drops to $4.5$ and $9.2$ respectively for measurements without applying individual inverse probability (IIP) weights, which mitigates fiber incompleteness from DESI's targeting. The measurements are robust against variations in stellar mass estimates, photometric shredding, and lensing calibration systematics. Using a simulation-based modeling framework with stellar mass function priors, we constrain the stellar mass-halo mass relation and find a satellite fraction of $\simeq 0.3$, which is higher than previous photometric studies but $1.5σ$ lower than $Λ$CDM predictions. We find that IIP weights have a significant impact on lensing measurements and can change the inferred $f_{\rm{sat}}$ by a factor of two, highlighting the need for accurate fiber incompleteness corrections for dwarf galaxy samples. Our results open a new observational window into the galaxy-halo connection at low masses, showing that future massively multiplexed spectroscopic observations and weak lensing data will enable stringent tests of galaxy formation models and $Λ$CDM predictions.

To, Chun-Hao [Chicago U., Astron. Astrophys. Ctr.;↗
DOE OSTI2026

wastewater_virus

This repo contains software used to clean and assemble high-throughput sequencing data containing viruses. The input is raw illumina sequencing reads and the output is a database of high-quality viral genomes. The specific application is to wastewater viral concentrates but it is not restricted to that sample type. The software is composed of Nextflow workflows and a set of custom Python and bash scripts that call publicly available bioinformatics tools to accomplish obvious tasks in data analysis in a high performance computing environment. For detailed information, please see the repo's README file.

Kantor, Rose [Lawrence Livermore National Laborato↗
DOE OSTI2024

Low-Income Energy Affordability Data - LEAD Tool - 2022 Update

The Low-Income Energy Affordability Data (LEAD) Tool was created by the Better Building's Clean Energy for Low Income Communities Accelerator (CELICA) to help state and local partners understand housing and energy characteristics for the low- and moderate-income (LMI) communities they serve. The LEAD Tool provides estimated LMI household energy data based on income, energy expenditures, fuel type, housing type, and geography, which stakeholders can use to make data-driven decisions when planning for their energy goals. From the LEAD Tool website, users can also create and download customized heat-maps and charts for various geographies, housing, energy characteristics, and population demographics and educational attainment. Datasets are available for 50 states plus Puerto Rico and Washington D.C., along with their cities, counties, and census tracts, as well as tribal areas. The file below, "01. Description of Files," provides a list of all files included in this dataset. A description of the abbreviations and units used in the LEAD Tool data can be found in the file below titled "02. Data Dictionary 2022". A list of geographic regions used in the LEAD Tool can be found in files 04-11. The Low-Income Energy Affordability Data comes primarily from the 2022 U.S. Census American Community Survey 5-Year Public Use Microdata Samples and is calibrated to 2022 U.S. Energy Information Administration electric utility (Survey Form-861) and natural gas utility (Survey Form-176) data. The methodology for the LEAD Tool can viewed below (3. Methodology Document). For more information, and to access the interactive LEAD Tool platform, please visit the "10. LEAD Tool Platform" resource link below. For more information on the Better Building's Clean Energy for Low Income Communities Accelerator (CELICA), please visit the "11. CELICA Website" resource below.

AMI↗
DOE OSTI2025

Dynamic Low-Rank Training with Spectral Regularization: Achieving Robustness in Compressed Representations

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94\% compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗
DOE OSTI2025

Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks

Deployment of neural networks on resource-constrained devices demands models that are both compact and robust to adversarial inputs. However, compression and adversarial robustness often conflict. In this work, we introduce a dynamical low-rank training scheme enhanced with a novel spectral regularizer that controls the condition number of the low-rank core in each layer. This approach mitigates the sensitivity of compressed models to adversarial perturbations without sacrificing clean accuracy. The method is model- and data-agnostic, computationally efficient, and supports rank adaptivity to automatically compress the network at hand. Extensive experiments across standard architectures, datasets, and adversarial attacks show the regularized networks can achieve over 94 compression while recovering or improving adversarial accuracy relative to uncompressed baselines.

Schotthoefer, Steffen [ORNL] (ORCID:00000002156965↗
DOE OSTI2024

U.S. State Renewables Portfolio & Clean Electricity Standards: 2024 Status Update [Slides]

This report provides an overview and status update on U.S. state renewables portfolio standards (RPS) and has been expanded from previous editions to also cover 100% clean electricity standards (CES) adopted by a growing number of states. The report, published in slide-deck form along with accompanying data files, describes recent legislative revisions, key policy design features, compliance with interim targets, past and projected impacts on clean electricity development, and compliance costs. The 2023 edition presents historical data through year-end 2023 and projections out to 2050. Key trends from this edition of the report include the following: -Evolution of state RPS and CES programs: States continue to refine and revise their RPS policies, often by adopting higher targets and/or broader CES policies. Among the 29 states plus DC with an RPS, 16 have RPS targets of at least 50% of retail sales, and 4 states have a 100% RPS. An additional 16 states have adopted a broader 100% CES. -Historical impacts on renewables development: Almost half of all growth in U.S. renewable electricity (RE) generation and capacity since 2000 is nominally associated with state RPS requirements. That percentage has declined over time to 35% of all U.S. RE capacity additions in 2023, though in certain regions RPS policies continue to play a dominant role in driving RE growth. -Future RPS and CES demand and incremental needs: The combined demand for clean electricity from RPS and CES policies will grow from roughly 500 TWh today to 1700 TWh by 2050. Accounting for current supplies—including existing nuclear and hydroelectric generation eligible for CES targets—RPS and CES policies will require 900 TWh of new clean electricity by 2050, equivalent to roughly 3x the historical rate of RPS-buildout. -RPS target achievement to-date: States have generally met their interim RPS targets in recent years, with only a few exceptions reflecting unique, state-specific issues. Most CES targets are not yet in force, and so little compliance experience to-date. -REC pricing trends: Prices for NEPOOL Class I RECs remained at roughly $\$40$/MWh over the past year, just below ACP rates in the larger state markets, while PJM Tier I REC prices continued to rise, reaching $\$35$/MWh by year-end 2023 and surpassing ACP levels in some states. Prices for solar RECs remained relatively stable, and continue to exhibit wide variation across states, with the highest prices ($200-450/MWh) in NJ, MA, and DC. -RPS compliance costs: RPS compliance costs average roughly 4% of retail electricity bills across RPS states, though vary widely from state to state, with the highest costs (11-12% of retail bills) in states with solar carve-outs and high SREC prices.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

Estimating value of information for heliostat washing operations at solar thermal plants

Concentrating solar power (CSP) plants depend on thousands of heliostats whose reflectance declines as dust accumulates. Operators routinely measure reflectance to estimate soiling and, in turn, inform cleaning schedules, but the value of collecting more frequent or more accurate data has not been formally quantified. This study introduces a Monte Carlo discrete event simulation framework that integrates stochastic models of soiling, weather, and measurement error with a dynamic cleaning dispatch policy to estimate annual energy production and operations costs. Applied to two representative central-receiver field configurations, the results show that both the frequency and accuracy of reflectance measurements can meaningfully impact plant performance. In both case studies, reducing measurement intervals yields significant returns, with the energy gains greatly exceeding the cost of more frequent data collection. The simulation framework serves as a decision-support tool for CSP operators, allowing them to input site-specific soiling conditions, measurement accuracy, and survey frequency to evaluate the tradeoffs between data collection cost and energy recovery, and to identify measurement strategies that maximize plant profit.

14 SOLAR ENERGY↗
DOE OSTI2024

A new Monte Carlo generator for BSM physics in B → K*ℓ+ℓ− decays with an application to lepton non-universality in angular distributions

Abstract Within the widely used EvtGen framework, we have added a new event generator model forB → K * ℓ + ℓ − with improved standard model (SM) decay amplitudes and possible BSM physics contributions, which are implemented in the operator product expansion in terms of Wilson coefficients. This event generator can then be used to estimate the statistical sensitivity of a simulated experiment to the most general BSM signal resulting from dimension-six operators. We describe the advantages and potential of the newly developed ‘Sibidanov Physics Generator’ in improving the experimental sensitivity of searches for lepton non-universal BSM physics and clarifying signatures. The new generator can properly simulate BSM scenarios, interference between SM and BSM amplitudes, and correlations between different BSM observables as well as acceptance bias. We show that exploiting such correlations substantially improves experimental sensitivity. As a demonstration of the utility of the MC generator, we examine the prospects for improved measurements of lepton non-universality in angular distributions forB→K * ℓ + ℓ − decays from the expected 50 ab −1 data set of the Belle II experiment, using a four-dimensional unbinned maximum likelihood fit. We describe promising experimental signatures and correlations between observables. The use of lepton-universality violating ∆-observables significantly reduces uncertainties in the SM expectations due to QCD and resonance effects and is ideally suited for Belle II with the large data sets expected in the next decade. Thanks to the clean experimental environment of ane + e − machine, Belle II should be able to probe BSM physics in the Wilson coefficientsC 7 and$$ {C}_7^{\prime } $$ C 7 ′ , which appear at lowq 2 in the di-electron channel.

Physics↗
DOE OSTI2024

EDX ClaiMM: Digital Resources for the Critical Minerals and Materials Community

Securing critical mineral supply chains is essential for transitioning to a clean energy economy and for maintaining national security. Big-data analytics can serve as a cost-effective means of identifying new domestic critical mineral resources but only if data can be easily located and digested. Using ArcGIS Enterprise Sites, EDX ClaiMM was developed to increase the accessibility of critical minerals data, reducing time spent on data collection and integration. Hosted tools provide rapid visualization and exploration of key datasets, unlocking insights to support resource assessments.

Yesenchak, Rachel↗
DOE OSTI2025

Future grid mix impacts on whole-building life cycle assessment

Building construction and operation are a significant contribution to global greenhouse gas emissions, so understanding and mitigating emissions is crucial for reliable and realistic emissions accounting. Whole-building Life Cycle Assessment (WBLCA) is an emissions accounting method that considers lifetime environmental impacts of a building during its construction, operation, and eventual end-of-life. When performing WBLCAs, emission calculations from the building's operation over the entire building lifespan are typically based on today's energy grid mixes. This method does not consider changes or advancements in the clean energy proportion within the grid mix and can over or under-inflate results, skewing the ratio of embodied vs. operational environmental impacts. While a variety of prediction tools estimate what future grid emissions might be, predictions can vary widely. To predict the clean energy ratio within future grid mixes and the potential impact these changes might have on WBLCA, annual data from several existing U.S. grid models was averaged and probabilistic modeling was used to extend the usable projections of shorter forecasts. Results show that clean energy sources will likely continue to increase over time, although the rate of growth varies by model. On average, by 2085, the clean energy penetration of the grid is projected to reach ~81% and renewable energy is projected to reach ~71%, although no widespread consensus is reached. To understand how the future grid mix impacts lifetime building emissions within a WBLCA context, the team analyzed two 2021 IECC-compliant all-electric residential buildings: one built from traditional materials and construction processes and the other built with carbon sequestering materials and modular assembly, with a portion of energy generated on site. The results indicate that a moderate estimate of future electricity grid mixes shows a reduction of yearly operational emissions for traditional residential buildings of 55% between 2025 and 2085, and a corresponding reduction of 48% of total emissions over a 60 year building lifespan. This study offers a nuanced approach to account for the variability of future grid mix models and provides an average trend-line based on a robust collection of scenarios.

Life Cycle Assessment (LCA)↗
DOE OSTI2025

Boosting Noise2Inverse via enhanced model selection for denoising computed tomography data

Synchrotron-based x-ray tomographic imaging enables the examination of the internal structure of materials at high spatial and temporal resolution. Experimental constraints can impose dose and time limits on the measurements, introducing a higher level of noise and artifacts in the reconstructed images. Deep learning has emerged as a powerful tool to remove noise from reconstructed images. Recently, the Noise2Inverse method was designed specifically for denoising reconstructed images without requiring paired noisy and clean images. This method creates multiple statistically independent reconstructions used to pair the data in which training involves transforming one reconstruction into the other, and vice versa. Originally designed to be used after a fixed number of epochs, we see in practice that this approach may not produce the optimal model and may unnecessarily waste computational resources. Therefore, we propose an alternative method of identifying the best model during training that aligns with the Noise2Inverse method. During validation, we compare the model output of the multiple reconstructions among each other. We hypothesize that the best model is the one that produces images with the highest similarity, implying a convergence in the predicted material properties and absorption values. To compare model outputs, we consider the absolute error, square error, structural similarity index (SSIM), peak signal-to-noise ratio (PSNR), and cosine similarity. We evaluate our method on two simulated tomography datasets and two, real-world, low-contrast, high-energy x-ray tomography datasets. We show our approach is more effective at determining the best model, up to an increase of 12.50% and 12.53% in SSIM and PSNR, respectively, while only requiring a fifth of the training time compared to the original approach.

CT↗
DOE OSTI2024

Analysis Program (2023 Annual Progress Report)

This document summarizes the progress of VTO Analysis projects supported during the fiscal year 2023. The VTO Analysis Program supports mission-critical technological, economic, and interdisciplinary analyses to assist in prioritizing VTO technology investments and to inform research portfolio planning. These efforts provide essential vehicle and market data, modeling and simulation, and integrated and applied analyses, using the unique capabilities, analytical tools, and expertise resident in the DOE’s national laboratory system. VTO Analysis projects also demonstrate additional capabilities and expertise provided by research partnerships that may include academia, the private sector, and non-profit organizations.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Obtaining Real Production Data Through On-Component Printed SIR Patterns

Cleanliness validation of printed circuit assemblies has become increasingly important as electronic assemblies become smaller, denser, and more challenging to clean, prompting an increase in reliance in custom solutions. With increasingly fast paced development cycles, long lead times and costs of quality custom surrogate test boards become prohibitive in obtaining cleanliness data that is a true representative of the final product. In this study, aerosol jet printing, an additive manufacturing technology for electronics, was used to manufacture surface insulation resistance (SIR) test structures that were evaluated on their ability to detect cleanliness defects and remain stable when clean, as well as their survivability to standard electronics wash processes. Additionally, in an effort to further increase the agility of true product testing, SIR test structures were printed directly onto ball-grid-arrays (BGAs) and tested the survivability of conformally printed leads for data acquisition. SIR test structures displayed good wash survivability on standard FR4 and BGAs, including wraparounds for on-component prints, and high sensitivity to cleanliness defects. The presented results demonstrate the feasibility of printed SIR structures as a quick-turn, highly customizable solution for wash validation with potential for increased fidelity cleanliness testing.

SIR, Additive Manufacturing, Printed Electronics, ↗
DOE OSTI2025

Life Cycle Analysis of Greenhouse Gas Emissions of Clean Fuels with the R&D GREET 2024 Model

This document summarizes research on the life cycle greenhouse gas (GHG) emissions rates from the production and use of clean fuels to support a new version of the Research and Development Greenhouse Gases, Regulated Emissions, and Energy Use in Technologies (R&D GREET) model, R&D GREET 2024 In this effort, Argonne National Laboratory (ANL) focuses on clean fuel pathways that are readily available in the market or are emerging in the near term. The selected pathways represent clean fuel technologies that convert biomass- and/or waste-based feedstocks to liquid and/or gaseous fuels for the transportation sector and other potential uses. The pathways are configured in R&D GREET 2024 with up-to-date feedstock-to-fuel life cycle inventory (LCI) data. Additionally, a new tab has been added to R&D GREET 2024 called “Clean Fuels” which allows the user to easily change inputs and access LCA results. Argonne does not warrant that the results presented in this report are consistent with the requirements of any particular regulatory or incentive program. Users interested in specific programs that reference GREET are encouraged to review guidance specific to those programs if and when it is available to determine appropriate means of compliance and contact the relevant responsible agencies for those specific policies or programs.

09 BIOMASS FUELS↗
DOE OSTI2024

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗
DOE OSTI2025

Ten questions on future and extreme weather data for building simulation and analysis in a changing climate

Weather plays a significant role in building operations as it directly influences HVAC loads and in turn the building energy and thermal performance. In a changing climate, future trends and extreme weather events become critical concerns in the global building decarbonization and clean energy transition. This paper aims to address ten key questions concerning extreme and future weather data for building applications, and more importantly to identify research gaps and guide the curation and selection of future and extreme weather data for use in building performance simulation and assessment. The paper intends to inform architects and engineers, operators, owners, policy makers, and other stakeholders on considering the impacts of future and extreme weather data and adopting strategies for selecting and applying this data in various use cases related to building design, operation, and retrofit for energy efficiency, electrification, and climate resilience.

Yan, Da↗