Search NASA⌕ Search

SEARCH · Search NASA

Results for “data challenges”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Efficient Data Query for Gaussian Process Compressed Data through Value Range Estimation [Slides]

When the resolution of the data increases, data reduction methods are applied to simulation output, including Gaussian process, neural representation and compression algorithms. Lots of data analysis/visualization techniques requires data query, but data query from reduced representation is still challenging. This report will provide examples and provide possible answers to why data query from reduced representation is still challenging.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

DEEP Solar: Data DrivEn Modeling and Analytics for Enhanced System Layer ImPlementation

Realizing the SETO 2030 mission of reducing solar energy costs to 3-5 c/kWh will require innovative enabling research on effective, cost-efficient integration of local PV within distribution systems. However, the intermittent and variable nature of PVs compels operators to impose conservative hosting capacity constraints. Given the extremely high variability of (intermittent and unpredictable) solar energy generation, relaxing the capacity constraints (which are currently around 15%) and achieving 100% or greater integration of renewables will require a fundamental transformation of the power grid via the utilization of exponentially larger amounts of AMI enabled fine-grained data. To address the challenges in increasing the penetration of renewable energy based DERs, this project envisions an Enhanced System Layer (ESL) at the distribution network level that is reliable, cost-effective and scalable to millions of Distributed Energy Resources (DERs)/devices. This includes developing: 1) Transformative and highly scalable machine learning based predictive analytics tools that plug into distribution system planning and provide real-time situational awareness at the distribution level for short and long-term operational planning. The tools will be built using novel data-driven energy models of millions of active nodes with AMI, 2) Adaptive stochastic analysis and optimization algorithms for real-time grid operations, 3) Dynamic Scenario Analysis using parallel Cloudenabled implementations with < 1 minute computational cycle times.

14 SOLAR ENERGY↗
DOE OSTI2024

Primordial nucleosynthesis with non-extensive statistics

The conventional Big Bang model successfully anticipates the initial abundances of 2 H(D), 3 He, and 4 He, aligning remarkably well with observational data. However, a persistent challenge arises in the case of 7 Li, where the predicted abundance exceeds observations by a factor of approximately three. Despite numerous efforts employing traditional nuclear physics to address this incongruity over the years, the enigma surrounding the lithium anomaly endures. In this context, we embark on an exploration of Big Bang nucleosynthesis (BBN) of light element abundances with the application of Tsallis non-extensive statistics. A comparison is made between the outcomes obtained by varying the non-extensive parameter q away from its unity value and both observational data and abundance predictions derived from the conventional big bang model. Here, a good agreement is found for the abundances of 4 He, 3 He and 7 Li, implying that the lithium abundance puzzle might be due to a subtle fine-tuning of the physics ingredients used to determine the BBN. However, the deuterium abundance deviates from observations.

Bertulani, Carlos A.↗
DOE OSTI2020

Big Data Analytic for Cascading Failure Analysis

With the challenges of increased grid dynamics and more variability of power generation from renewable energy sources, rapidly increasing complexity in the grid model, and abundant data from measurements and simulations, the requirements for computational analysis have also increased dramatically. Power system operators and engineers need to analyze more scenarios, extract meaningful information from a larger set of data, and respond more quickly when faced with these new challenges for large-scale applications, such as contingency analysis. This paper proposes a novel big data analysis approach for power system cascading analysis, prevention, and remediation. The developed techniques will be capable of cascading analysis, better assessment of the system’s vulnerability level, as well as proposing potential remediation. Case studies using IEEE 118-bus system and a 500-bus system, with a comparison against a commercial tool, validate the advantages of the developed big data approach: accurate prediction, and more importantly, faster and effective correction actions.The developed techniques could be further used for other power system applications.

big data, cascading analysis, corrective action, m↗
DOE OSTI2025

A Bottom-Up Cost Estimation Tool for Nuclear Microreactors

The rising interest in nuclear microreactors has highlighted the need for comprehensive technoeconomic assessments. However, the scarcity of publicly available designs and cost data has posed significant challenges. To address this issue, the Microreactor Optimization Using Simulation and Economics (MOUSE) tool is developed. MOUSE is a tool that integrates nuclear microreactor design with reactor economics. The design calculations encompass core simulations using the OpenMC Monte Carlo Particle Transport Code [romano2015], along with simplified balance of plant calculations. On the economic side, MOUSE provides detailed bottom-up cost estimates, calculating both the total capital cost and the levelized cost of energy for first-of-a-kind and nth-of-a-kind microreactors. The cost estimation correlations are developed using data from the MARVEL project and additional literature sources. MOUSE has released as an open-source tool on GitHub (MOUSE Tool). By combining design calculations with cost equations, MOUSE enables users to evaluate the impact of various technological consideration, advanced moderators, design changes, material/fuel changes, and geometry modifications—as well as economic parameters like interest rates and construction duration. This comprehensive framework can guide stakeholders towards technological solutions that enhance microreactor competitiveness. Additionally, powered by the WATTS toolkit [romano2022], MOUSE supports optimization studies, parametric analyses, and uncertainty calculations/propagation. Currently, preconceptual designs of three microreactor types are included in MOUSE: a liquid metal thermal microreactor (LTMR), gas cooled TRISO-fueled microreactor (GCMR) and heat-pipe TRISO fueled microreactor (HPMR). To showcase its ability, MOUSE was used to conduct detailed bottom-up cost estimates for the first of a kind (FOAK) and Nth of a kind (NOAK) of the following microreactors • A 20MWt LTMR that is built on the ongoing MARVEL demonstration at Idaho National Laboratory (INL) • A 15 MWt GCMR that was designed to be more representative of the typical commercial microreactor • A 7 MWt HPMR that was built on previous work (Choi 2024) The The reader should note that these three designs and corresponding cost estimates are examples to demonstrate the MOUSE capability. The designs are pre-conceptual, the reactor designs were not optimized, and the cost estimates were developed with incomplete information. Additionally, stakeholders might be interested in a variety of designs that may differ from the examples provided in this report. The MOUSE tool can also be used to study how design choices affect economics. To demonstrate its capability, MOUSE was used conduct parametric studies such as examining the economic impact of the reflector's material and thickness, the moderator's booster material and dimensions, fuel composition and enrichment, core size, and power level. Several insights were gained from these parametric studies.

Hanna, Botros↗
DOE OSTI2025

Remote Sensing and Fluxes Upscaling for Real-world Impact (Workshop Report)

The "Remote Sensing and Fluxes Upscaling for Real-world Impact" workshop, held on July 9-10, 2024, at Lawrence Berkeley National Lab, was a collaborative effort led by the AmeriFlux Management Project, NEON, and the Carbon Dew Community of Practice. The event brought together over 200 registrants and approximately 100 attendees each day, including leading experts, researchers, and practitioners. The primary focus was on bridging the gap between cutting-edge research and practical applications in environmental monitoring by integrating remote sensing and flux data. Key themes included the importance of site-level measurements for validating remote sensing products, providing nature-based climate solutions, and addressing challenges such as instrument costs and the need for standardized methods. At the regional scale, discussions centered on addressing spatial heterogeneity and using high-resolution remote sensing and machine learning methods to enhance data interpretation. Global scale challenges included data consistency, gap filling, and accurate emission source identification, with opportunities for international collaboration and standardized practices to improve global carbon budget assessments. The workshop emphasized the critical need for integrating data across local, regional, and global scales through explicit scale-matching and developed a workflow for scaling flux data using "straight shot" and "explicit nesting" approaches. The event highlighted the importance of connecting scientific research with real-world applications in carbon, energy, and water management, ensuring that advancements translate into tangible societal benefits. These insights will guide future research, technology transfer, and collaboration, maximizing the potential of environmental fluxes to address real-world challenges.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Physics-informed machine learning modeling for predictive control using noisy data

Due to the occurrence of over-fitting at the learning phase, the modeling of chemical processes via artificial neural networks (ANN) by using corrupted data (i.e., noisy data) is an ongoing challenge. Therefore, this work investigates the effect of both Gaussian and non-Gaussian noise on the performance of process-structure based recurrent neural networks (RNN) models, which take the form of partially-connected RNN models in this work, that are used to approximate a class of multi-input-multi-outputs nonlinear systems. Furthermore, two different techniques, specifically Monte Carlo dropout and co-teaching, are utilized in the development of partially-connected RNN models. Here, these two techniques are employed to reduce the over-fitting in ANNs when noisy data is used in the training process and, hence, to improve the open-loop accuracy as well as the closed-loop performance under a Lyapunov-based model predictive controller (MPC). Aspen Plus Dynamics, a well-known high-fidelity process simulator, is used to simulate a large-scale chemical process application in order to demonstrate the anticipated improvements in both open-loop approximation and closed-loop controller performance in the presence of Gaussian and non-Gaussian noise in the data set using physics-informed RNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

CSAPR2 cell-tracking data collected during TRACER

One of the challenges of analyzing convective cell properties is quick evolution of the individual convective cells. While the operational radar data provide great a data set to analyze the evolution of radar observables of convective precipitation clouds statistically, previous studies also suggested that, because of the quick evolution of cell life cycle, conventional radar volume scan strategies taking ~5-7 minutes might not capture the detailed evolution. The TRACER campaign deployed CSAPR2, which performed frequent update of RHI and sector PPI scans to track convective cells every < 2 minutes guided by a new cell-tracking framework, Multisensor Agile Adaptive Sampling (MAAS; Kollias et al. 2020). This allows for capturing fast-evolving radar observables. The submitted data files are CSAPR2 data in CfRadial format collected during the TRACER field campaign from June to September 2020. The data files include processed radar variables including: noise-masked reflectivity and differential reflectivity corrected for rain attenuation and systematic biases, noise-masked dealiased radial velocity, specific differential phase, locations of target cells (latitude, longitude, radar range), and radar-echo classification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Advanced data science toolkit for non-data scientists – A user guide

Emerging modern data analytics attracts much attention in materials research and shows great potential for enabling data-driven design. Data populated from the high-throughput CALPHAD approach enables researchers to better understand underlying mechanisms and to facilitate novel hypotheses generation, but the increasing volume of data makes the analysis extremely challenging. Here in this paper, we introduce an easy-to-use, versatile, and open-source data analytics frontend, ASCENDS (Advanced data SCiENce toolkit for Non-Data Scientists), designed with the intent of accelerating data-driven materials research and development. The toolkit is also of value beyond materials science as it can analyze the correlation between input features and target values, train machine learning models, and make predictions from the trained surrogate models of any scientific dataset. Various algorithms implemented in ASCENDS allow users performing quantified correlation analyses and supervised machine learning to explore any datasets of interest without extensive computing and data science background. The detailed usage of ASCENDS is introduced with an example of experimental high-temperature alloy data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

A generative modeling approach to reconstructing 21 cm tomographic data

Abstract Analyses of the cosmic 21 cm signal are hampered by astrophysical foregrounds that are far stronger than the signal itself. These foregrounds, typically confined to a wedge-shaped region in Fourier space, often necessitate the removal of a vast majority of modes, thereby degrading the quality of the data anisotropically. To address this challenge, we introduce a novel deep generative model based on stochastic interpolants to reconstruct the 21 cm data lost to wedge filtering. Our method leverages the non-Gaussian nature of the 21 cm signal to effectively map wedge-filtered 3D lightcones to samples from the conditional distribution of wedge-recovered lightcones. We demonstrate how our method is able to restore spatial information effectively, considering both varying cosmological initial conditions and astrophysics. Furthermore, we discuss a number of future avenues where this approach could be applied in analyses of the 21 cm signal, potentially offering new opportunities to improve our understanding of the Universe during the epochs of cosmic dawn and reionization. Code, pre-trained models, and scripts for making plots in this paper can be found here .

Sabti, Nashwan (ORCID:000000027924546X)↗
DOE OSTI2021

Machine Learning Driven Contouring of High-Frequency Four-Dimensional Cardiac Ultrasound Data

Automatic boundary detection of 4D ultrasound (4DUS) cardiac data is a promising yet challenging application at the intersection of machine learning and medicine. Using recently developed murine 4DUS cardiac imaging data, we demonstrate here a set of three machine learning models that predict left ventricular wall kinematics along both the endo- and epi-cardial boundaries. Each model is fundamentally built on three key features: (1) the projection of raw US data to a lower dimensional subspace, (2) a smoothing spline basis across time, and (3) a strategic parameterization of the left ventricular boundaries. Model 1 is constructed such that boundary predictions are based on individual short-axis images, regardless of their relative position in the ventricle. Model 2 simultaneously incorporates parallel short-axis image data into their predictions. Model 3 builds on the multi-slice approach of model 2, but assists predictions with a single ground-truth position at end-diastole. To assess the performance of each model, Monte Carlo cross validation was used to assess the performance of each model on unseen data. For predicting the radial distance of the endocardium, models 1, 2, and 3 yielded average R2 values of 0.41, 0.49, and 0.71, respectively. Monte Carlo simulations of the endocardial wall showed significantly closer predictions when using model 2 versus model 1 at a rate of 48.67%, and using model 3 versus model 2 at a rate of 83.50%. These finding suggest that a machine learning approach where multi-slice data are simultaneously used as input and predictions are aided by a single user input yields the most robust performance. Subsequently, we explore the how metrics of cardiac kinematics compare between ground-truth contours and predicted boundaries. We observed negligible deviations from ground-truth when using predicted boundaries alone, except in the case of early diastolic strain rate, providing confidence for the use of such machine learning models for rapid and reliable assessments of murine cardiac function. To our knowledge, this is the first application of machine learning to murine left ventricular 4DUS data. Future work will be needed to strengthen both model performance and applicability to different cardiac disease models.

4D ultrasound↗
DOE OSTI2024

Tank Waste Characterization: History, Challenges, and Success Stories

The preparation and chemical and radiochemical analysis of Hanford tank waste samples can be performed with standard laboratory equipment and instruments as relatively routine processes that are not particularly challenging. Rather, the main challenges of tank waste characterization are associated with radiological dose and sampling limitations. Accurate, representative and effective sampling techniques are difficult with the waste tanks because they were not designed for routine sampling. There are a finite number of sampling locations for each tank based on riser positioning, depth and the operational functionality of the sampling riser. For example, in one recently emptied SST, there was one riser that was found to have had concrete dumped down it, thereby eliminating that sampling port. Additionally, the waste within the tank; especially true for the saltcake and sludge, is not homogenous. The ability to adequately mix a million-gallon double shell tank (DST) is a concern for data reproducibility. Another real challenge that must be addressed for sampling single shell tanks, is how to dissolve the salt cake waste in a compromised (leaking) SST. These physical constraints mean that uncertainty in the representativeness of samples must be considered when applying analytical results to the bulk contents of the tank. The tank waste is highly radioactive and thus can only be handled initially by facilities that can receive samples into concrete-shielded hot cells with remote operation with an example provided in Figure 1. The shielding protects the worker from the radiological dose while mineral oil windows and remotely operated manipulators enables the samples to be handled. At Hanford, analytical laboratories with these hot cell capabilities are limited to the Pacific Northwest National Laboratory and the main Hanford operations support laboratory, 222-S Laboratory. Because of their highly radioactive nature, samples must be sufficiently diluted to facilitate their analysis outside of a shielded cell. In some cases, this means some accuracy must be compromised to complete the analysis beyond that normally encountered for non-radioactive material.

Waste Characterization, BBI, PHOENIX: Tank Farms: ↗
DOE OSTI2020

Heterogeneous data-processing optimization with CLARA’s adaptive workflow orchestrator

The hardware landscape used in HEP and NP is changing from homogeneous multi-core systems towards heterogeneous systems with many different computing units, each with their own characteristics. To achieve maximum performance with data processing, the main challenge is to place the right computing on the right hardware. In this paper, we discuss CLAS12 charge particle tracking workflow orchestration that allows us to utilize both CPU and GPU to improve the performance. The tracking application algorithm was decomposed into micro-services that are deployed on CPU and GPU processing units, where the best features of both are intelligently combined to achieve maximum performance. In this heterogeneous environment, CLARA aims to match the requirements of each micro-service to the strength of a CPU or a GPU architecture. A predefined execution of a micro-service on a CPU or a GPU may not be the most optimal solution due to the streaming data-quantum size and the data-quantum transfer latency between CPU and GPU. So, the CLARA workflow orchestrator is designed to dynamically assign micro-service execution to a CPU or a GPU, based on the online benchmark results analyzed for a period of real-time data-processing.

Gyurjyan, Vardan↗
DOE OSTI2026

DESI DR1 Ly α 1D power spectrum: Validation of estimators

The Data Release 1 (DR1) of the Dark Energy Spectroscopic Instrument (DESI) is the largest sample to date for small-scale Lyα forest cosmology, accessed through its one-dimensional power spectrum (P 1D ). The Lyα forest P 1D is extracted from quasar spectra that are highly inhomogeneous (both in wavelength and between quasars) in noise properties due to intrinsic properties of the quasar, atmospheric and astrophysical contamination, and also sensitive to low-level details of the spectral extraction pipeline. We employ two estimators in DR1 analysis to measure P 1D : the optimal estimator and the fast Fourier transform (FFT) estimator. To ensure robustness of our DR1 measurements, we validate these two power spectrum and covariance matrix estimation methodologies against the challenging aspects of the data. First, using a set of 20 synthetic 1D realizations of DR1, we derive the masking bias corrections needed for the FFT estimator and the continuum fitting bias needed for both estimators. We demonstrate that both estimators, including their covariances, are unbiased with these corrections using the Kolmogorov-Smirnov test. Second, we substantially extend our previous suite of CCD image simulations to include 675,000 quasars, allowing us to accurately quantify the pipeline's performance. This set of simulations reveals biases at the highest k values, corresponding to a resolution error of a few percent. We base the resolution systematics error budget of DR1 P 1D on these values, but do not derive corrections from them since the simulation fidelity is insufficient for precise corrections.

Lyman alpha forest↗
DOE OSTI2020

Probing and manipulating magnetic and 2D quantum materials using ultrafast laser and high harmonic sources

The ability to probe the full dynamic response of quantum materials on the length- and time-scales (Å to attoseconds on up) fundamental to charge, spin and phonon interactions is leading to a host of new discoveries. The coupled interactions between charges, spins, orbital and lattice degrees of freedom are key to determining the state of a material, whether metallic, insulating, superconducting or magnetic. Under thermal equilibrium conditions these states can be tuned by varying the temperature, pressure, chemical doping or dimensionality. However, their inherent complexity and rich phase landscape make them challenging to understand or manipulate deterministically. Fortunately, ultrafast light sources have undergone remarkable advances in recent years, achieving what was merely a dream three decades ago, i.e., full coherent control of light fields spanning the THz to the X-ray regions. Here, these new capabilities are providing powerful new tools for coherently manipulating and probing quantum materials using light. Recent advances in high harmonic generation (HHG) have provided new techniques for uncovering new phases and couplings in 2D and magnetic materials, using femtosecond lasers to coherently manipulate and tune the properties and interactions in the material, and HHG as a probe to map out these new phases and couplings. There are several beautiful recent observations of exciting new states in materials using light, but in some cases, the limited experimental data points made it challenging to validate an interpretation. The new ability of ultrafast HHG-based photoemission and magneto-optic spectroscopies to precisely probe multiple phases, electronic and magnetic order parameters, when combined with scanning the laser excitation fluence, is allowing us to map how material phases, magnetic states, and electron-phonon-spin couplings in these delicate materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Leveraging generative adversarial networks to create realistic scanning transmission electron microscopy images

Abstract The rise of automation and machine learning (ML) in electron microscopy has the potential to revolutionize materials research through autonomous data collection and processing. A significant challenge lies in developing ML models that rapidly generalize to large data sets under varying experimental conditions. We address this by employing a cycle generative adversarial network (CycleGAN) with a reciprocal space discriminator, which augments simulated data with realistic spatial frequency information. This allows the CycleGAN to generate images nearly indistinguishable from real data and provide labels for ML applications. We showcase our approach by training a fully convolutional network (FCN) to identify single atom defects in a 4.5 million atom data set, collected using automated acquisition in an aberration-corrected scanning transmission electron microscope (STEM). Our method produces adaptable FCNs that can adjust to dynamically changing experimental variables with minimal intervention, marking a crucial step towards fully autonomous harnessing of microscopy big data.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Autonomous hybrid optimization of a SiO 2 plasma etching mechanism

Computational modeling of plasma etching processes at the feature scale relevant to the fabrication of nanometer semiconductor devices is critically dependent on the reaction mechanism representing the physical processes occurring between plasma produced reactant fluxes and the surface, reaction probabilities, yields, rate coefficients, and threshold energies that characterize these processes. The increasing complexity of the structures being fabricated, new materials, and novel gas mixtures increase the complexity of the reaction mechanism used in feature scale models and increase the difficulty in developing the fundamental data required for the mechanism. This challenge is further exacerbated by the fact that acquiring these fundamental data through more complex computational models or experiments is often limited by cost, technical complexity, or inadequate models. In this paper, we discuss a method to automate the selection of fundamental data in a reduced reaction mechanism for feature scale plasma etching of SiO 2 using a fluorocarbon gas mixture by matching predictions of etch profiles to experimental data using a gradient descent (GD)/Nelder–Mead (NM) method hybrid optimization scheme. These methods produce a reaction mechanism that replicates the experimental training data as well as experimental data using related but different etch processes.

36 MATERIALS SCIENCE↗