Search NASA⌕ Search

SEARCH · Search NASA

Results for “ERROR ANALYSIS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)

This data release provides all data and code used in the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)" to model stream temperature, evaluate, and assess results. The associated manuscript explores current open questions in prediction in ungauged and unmonitored basins concerning top-down versus bottom-up approaches, tradeoffs between data available and input requirements, and the appropriate representation of catchment attributes as inputs to deep learning models. Modeling was done primarily with long short-term memory (LSTM) models, and stream site coverage spans 1362 locations across the conterminous United States. The data is organized into these items items:Code repository and data for the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)".Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code: - data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- error_analysis_attribute_and_groundwater_dir.zip - workflows for the extended error analysis by stream attribute and groundwater influenceData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2024streamdata, author = {Jared Willard and Fabio Ciulla and Helen Weierbach and Vipin Kumar and Charuleka Varadharajan}, title = {Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models"}, year = {2024}, doi = {10.15485/2448016}, publisher = {ESS-DIVE Repository}, url = {https://doi.org/10.15485/2448016}}MLA: Willard, Jared, et al. Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models". 2024. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

CrossCheck: Rapid, Reproducible, and Interpretable Model Evaluation

Evaluation beyond aggregate performance metrics, e.g. F1-score, is crucial to both establish an appropriate level of trust in machine learning models and identify future model improvements. In this paper we demonstrate CrossCheck, an interactive visualization tool for rapid crossmodel comparison and reproducible error analysis. We describe the tool and discuss design and implementation details. We then present three use cases (named entity recognition, reading comprehension, and clickbait detection) that show the benefits of using the tool for model evaluation. CrossCheck allows data scientists to make informed decisions to choose between multiple models, identify when the models are correct and for which examples, investigate whether the models are making the same mistakes as humans, evaluate models’ generalizability and highlight models’ limitations, strengths and weaknesses. Furthermore, CrossCheck is implemented as a Jupyter widget, which allows rapid and convenient integration into data scientists’ model development workflows.

Arendt, Dustin L.↗
DOE OSTI2021

Quarter 4 Report: Report on Final Findings and Opportunities for Future Work in the Use of Mixed Precision in Iterative Solvers

The fourth quarter of the project was spent developing an error analysis of the s-step Lanczos and CG algorithms. Our theoretical bounds and numerical experiments show that the numerical behavior of the algorithm can be significantly improved by using extra precision in a small part of the computation related to the computation and application of the Gram matrix. We have published a technical report which includes all steps of the analysis [8]; a shortened version for journal submission is in preparation. We plan to submit this paper in the following weeks. Activities related to this also include a collaboration with Ichitaro Yamazaki on gathering performance results for these new mixed precision s-step Krylov subspace methods using single/double precision on GPUs. Namely, we would like to obtain performance results that show that the performance overhead of using double the working precision in these select computations is minimal. Other activities include attending biweekly xSDK meetings and presenting a pitch talk on this work to the group on February 25, 2021. In the remainder of the document, we summarize our findings on the potential for mixed precision in classical Krylov subspace methods and s-step Krylov subspace methods, as well as key opportunities for future work.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

ASCEND: Asymptotically compatible strong form foundations for nonlocal discretization

Nonlocal models naturally handle a range of physics of interest to SNL, but discretization of their underlying integral operators poses mathematical challenges to realize the accuracy and robustness commonplace in discretization of local counterparts. This project focuses on the concept of asymptotic compatibility, namely preservation of the limit of the discrete nonlocal model to a corresponding well-understood local solution. We address challenges that have traditionally troubled nonlocal mechanics models primarily related to consistency guarantees and boundary conditions. For simple problems such as diffusion and linear elasticity we have developed complete error analysis theory providing consistency guarantees. We then take these foundational tools to develop new state-of-the-art capabilities for: lithiation-induced failure in batteries, ductile failure of problems driven by contact, blast-on-structure induced failure, brittle/ductile failure of thin structures. We also summarize ongoing efforts using these frameworks in data-driven modeling contexts. This report provides a high-level summary of all publications which followed from these efforts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

GraphAlign: Graph-Enabled Machine Learning for Seismic Event Filtering

This report summarizes results from a 2 year effort to improve the current automated seismic event processing system by leveraging machine learning models that can operated over the inherent graph data structure of a seismic sensor network. Specifically, the GraphAlign project seeks to utilize prior information on which stations are more likely to detect signals originating from particular geographic regions to inform event filtering. To date, the GraphAlign team has developed a Graphical Neural Network (GNN) model to filter out false events generated by the Global Associator (GA) algorithm. The algorithm operates directly on waveform data that has been associated to an event by building a variable sized graph of station waveforms nodes with edge relations to an event location node. This builds off of previous work where random forest models were used to do the same task using hand crafted features. The GNN model performance was analyzed using an 8 week IMS/IDC dataset, and it was demonstrated that the GNN outperforms the random forest baseline. We provide additional error analysis of which events the GNN model performs well and poorly against concluded by future directions for improvements.

58 GEOSCIENCES↗
DOE OSTI2022

Doppler-Free Saturation Spectroscopy (DFSS) for Magnetic and Electric Field Measurements in an FRC plasma

The Cooperative Research and Development Agreement (CRADA) between Oak Ridge National Laboratory (ORNL) and TAE Technologies (TAE) focused on a feasibility study of Doppler-free saturation spectroscopy (DFSS) as a diagnostic strategy to measure the internal magnetic and electric fields of TAE’s C-2W field-reversed configuration (FRC) fusion device. Non-perturbative measurements of magnetic and electric fields in the core of FRCs can provide critical information needed to constrain equilibrium models and for the continued advancement of the fusion research program at TAE. DFSS is a well-established, laser-based absorption spectroscopy technique used to suppress inhomogeneous (Doppler) broadening linewidths and reveal Zeeman/Stark split hyperfine structures of target emission lines. The collaboration involved the use of ORNL’s suite of experimentally validated quantum mechanical modelling codes to simulate theoretical atomic spectra for a variety of machine parameters and laser beam configurations. TAE’s expertise was heavily used to provide measurements/models of the required machine parameters needed as inputs to the simulations. Results of an all-encompassing error analysis demonstrate the favourable capabilities of the diagnostic and provided invaluable insights as to how DFSS can be best used to accurately measure internal B/E-fields in C-2W.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Commissioning of SRF Cavity Analyzer System

Research and development in Superconducting Radio Frequency (SRF) cavities manufacturing, surface treatment, preparation, and doping techniques for the improvement of cavities performance necessitate reliable cavities’ intrinsic quality factor measurement with accurate and repeatable results. Fermilab Vertical Test Stand facility (VTS) performs ~270 cavities tests per year, including R&D cavities as well as production cavities at each step of their preparation for the placement in accelerator’s cryomodules. Current Fermilab VTS testing system is based on the scalar approach of power measurement, which neglects potential error factors of the RF networks with distributed parameters, hence the assessed measurement errors of such a system might be slightly underestimated. Alternative approach, utilizing a vector measurement (amplitude and phase) of RF signals in the setup with a cavity as a device under test, promises an enhanced accuracy and results traceability. New cavity an alyser system based on this approach (Vector VTS) was developed by D. Frolov et al. and currently undergoes a commissioning process. Basic underlying principles of the system will be presented with the methods of quality factor extraction from S-parameters measurement of multiport cavities and brief error analysis will be discussed. The system architecture will be shown with main components parameters and ensuing testing capabilities. Multiple tests results will be discussed with a cross-check comparison of the results obtained with a conventional testing method. System advantages such as automated calibration process and additional features as a built- in microphonics measurement capability will be presented.

Netepenko, A.↗
DOE OSTI2020

Fast Evaluation of Aircraft Icing Severity Using Machine Learning Based on XGBoost

Aircraft icing represents a serious hazard in aviation which has caused a number of fatal accidents over the years. In addition, it can lead to substantial increase in drag and weight, thus reducing the aerodynamics performance of the airplane. The process of ice accretion on a solid surface is a complex interaction of aerodynamic and environmental variables. The complex relationship makes machine learning-based methods an attractive alternative to traditional numerical simulation-based approaches. In this study, we introduce a purely data-driven approach to find the complex pattern between different flight conditions and aircraft icing severity prediction. The supervised learning algorithm Extreme Gradient Boosting (XGBoost) is applied to establish the prediction framework which makes prediction based on any set of observations. The input flight conditions for the proposed prediction framework are liquid water content, droplet diameter and exposure time. The proposed approach is demonstrated in three cases: maximum ice thickness prediction, icing area prediction and icing severity level evaluation. Performance comparison studies and error analysis are also conducted to verify the effectiveness and performance of the proposed method. Results show that the proposed method has reasonable capability in evaluating aircraft icing severity.

Li, Sibo (ORCID:000000021705844X)↗
DOE OSTI2020

A Dynamic Amplitude-Correcting Gradient Estimation Technique to Align X-ray Focusing Optics

High-brightness X-rays, as produced at synchrotrons and X-ray free electron laser (XFEL) facilities, are used to characterize materials in a variety of scientific experiments. In most cases, effective use of the high-energy light requires precisely-aligned focusing optics; one example being a compound refractive lens (CRL). To align a CRL, the position and rotation must be optimized along four axes. In practice, this is a labor-intensive, time-consuming manual process that can monopolize scarce experimental time at the necessary X-ray facilities. Models of the expected Xray transmission function suggest that this task can be automated; however, the temporally-varying intensity at X-ray free electron laser facilities preclude the direct use of standard implementations of optimization solvers such as steepest descent algorithms. In this paper, we propose a novel technique to estimate the gradient of noisy functions with temporally-varying amplitudes. We construct this dynamicamplitude correction by systematically sampling a fixed central location within the standard finite difference stencil, accounting for the observed changes in time, and normalizing the difference quotients against those fluctuations. In addition to a rigorous error analysis of the sampling technique, we demonstrate its efficacy in stochastic descent optimization methods. Further, we demonstrate how this approach may be implemented to optimize X-ray focusing optics at synchrotrons or XFEL facilities

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Design Space Exploration of Emerging Memory Technologies for Machine Learning Applications

Memory design space exploration methods study memory systems’ performances and limitations before implementation. The computer memory design space has grown exponentially because of the enormous growth of memory types, memory controllers, and application software. Computer simulators are commonly used for memory design space exploration. However, complex memory simulations take an enormous amount of time. Hence, in this paper, we proposed a machine learning-based design space exploration method for dynamic random-access memory and non-volatile memory systems. We applied our method to the CosmoGAN and LeNet applications to predict the following six memory response parameters: (i) bandwidth, (ii) power, (iii) average latency, (iv) average total latency, (v) memory reads, and (vi) memory writes. Our experimental results show that machine learning models can predict memory response parameter values faster than simulations. We used support vector machine, random forest, and gradient boosting machine learning models. We observed that the support vector machine provides better performance for bandwidth, average latency, and average total latency. The random forest model works better for memory reads and writes. The gradient boosting model provides superior prediction performance for power. We provide a detailed discussion on learning curve characteristics, error analysis, and memory type recommendation.

Hasan, S M Shamimul↗
DOE OSTI2024

Error and Correction Analysis for the FFA@CEBAF Energy Upgrade

An energy upgrade design for the Continuous Electron Beam Accelerator Facility (CEBAF) is under development, using fixed field alternating gradient (FFA) return arcs to recirculate electron beam up to an additional five times through the accelerating structures at CEBAF. A necessary component of any large accelerator is a beam steering and optical correction system. Small environmental changes and system errors can lower beam quality or even shut down the machine; and in pursuit of the scientific mission of JLab, high quality electron beams must be delivered to the experimental halls on a predictable schedule. Correction in the novel FFA arcs of the current upgrade design is complicated by several factors. These complexities inform the choice of correction algorithm structure and parameter values. A baseline algorithm in addition to diagnostic and correction hardware configuration is presented. The effect of this correction protocol is shown with respect to estimated errors, and several possible extensions of the algorithm are discussed. This work presents an important proof of concept for the FFA@CEBAF design effort, and provides a functional correction strategy which may be simply adjusted and optimized for future design changes.

Coxe, Alex [Old Dominion Univ., Norfolk, VA (Unite↗
DOE OSTI2025

Ice storage model-predictive control in an office building with PV: scenario, error and sensitivity analysis

Thermal energy storage (TES) can enable more building-sited renewable electricity generation and lower utility bill costs for buildings owners and occupants, especially when there are high demand and variable time-of-use (TOU) charges. A model predictive control (MPC) strategy can offer additional savings over a schedule-based control with added complexity and reliance on forecasts. Here, this study examines savings for medium office buildings with chiller plants in three locations with building-installed solar photovoltaics (PV) to understand the impact of MPC. Control setpoints are fixed by a schedule-based control or optimized by nonlinear MPC. These control setpoints are actuated within EnergyPlus building models to simulate the utility cost of the chiller plant. NLP solutions can be unstable or unrealistic, but our results show that by regularizing the NLP, the solutions can be reasonably followed by the building model. MPC models make simplifications that lead to errors once the controller is participating in and changing the operation of the building. These errors average 9 % across the cases, showing that the most important parts of the system are represented. The no-thermal load costs are computed to show that the optimization can in some cases achieve both the minimum TOU and minimum monthly demand costs by demand management while reducing TOU energy costs by energy arbitrage. The MPC saves 35–66 % in the annual chiller plant operating costs, which is an additional savings above the schedule by 1–33 %. PV and TES are complementary and mostly independent, but a load with PV often results in better performance for the schedule. Our case study and sensitivity analysis show the importance of modeling and optimization for complex rates, but also the circumstances wherein a simpler strategy achieves the same performance with less potential for error.

14 SOLAR ENERGY↗
DOE OSTI2023

Unified analysis of finite-size error for periodic Hartree-Fock and second order Møller-Plesset perturbation theory

Despite decades of practice, finite-size errors in many widely used electronic structure theories for periodic systems remain poorly understood. For periodic systems using a general Monkhorst-Pack grid, there has been no comprehensive and rigorous analysis of the finite-size error in the Hartree-Fock theory (HF) and the second order Møller-Plesset perturbation theory (MP2), which are the simplest wavefunction based method, and the simplest post-Hartree-Fock method, respectively. Such calculations can be viewed as a multi-dimensional integral discretized with certain trapezoidal rules. Due to the Coulomb singularity, the integrand has many points of discontinuity in general, and standard error analysis based on the Euler-Maclaurin formula gives overly pessimistic results. The lack of analytic understanding of finite-size errors also impedes the development of effective finite-size correction schemes. We propose a unified analysis to obtain sharp convergence rates of finite-size errors for the periodic HF and MP2 theories. Our main technical advancement is a generalization of the result of Lyness [Math. Comp. 30 (1976), pp. 1–23] for obtaining sharp convergence rates of the trapezoidal rule for a class of non-smooth integrands. Our result is applicable to three-dimensional bulk systems as well as low dimensional systems (such as nanowires and 2D materials). Our unified analysis also allows us to prove the effectiveness of the Madelung-constant correction to the Fock exchange energy, and the effectiveness of a recently proposed staggered mesh method for periodic MP2 calculations (see X. Xing, X. Li, and L. Lin [J. Chem. Theory Comput. 17 (2021), pp. 4733–4745]). In conclusion, our analysis connects the effectiveness of the staggered mesh method with integrands with removable singularities, and suggests a new staggered mesh method for reducing finite-size errors of periodic HF calculations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The rise and fall of stretched bond errors: Extending the analysis of Perdew–Zunger self-interaction corrections of reaction barrier heights beyond the LSDA

Incorporating self-interaction corrections (SIC) significantly improves chemical reaction barrier height predictions made using density functional theory methods. We present a detailed orbital-by-orbital analysis of these corrections for three semi-local density functional approximations (DFAs) situated on the three lowest rungs of Jacob’s ladder of approximations. The analysis is based on Fermi–Löwdin Orbital Self-Interaction Correction (FLOSIC) calculations performed at several steps along the reaction pathway from the reactants (R) to the transition state (TS) to the products (P) for four representative reactions selected from the BH76 benchmark set. For all three functionals, the major contribution to self-interaction corrections of the barrier heights can be traced to stretched bond orbitals that develop near the TS configuration. The magnitude of the ratio of the self-exchange–correlation energy to the self-Hartree energy (XC/H) for a given orbital is introduced as an indicator of one-electron self-interaction error. XC/H = 1.0 implies that an orbital’s self-exchange–correlation energy exactly cancels its self-Hartree energy and that the orbital, therefore, makes no contribution to the SIC in the FLOSIC scheme. For the practical DFAs studied here, XC/H spans a range of values. Here, the largest values are obtained for stretched or strongly lobed orbitals. We show that significant differences in XC/H for corresponding orbitals in the R, TS, and P configurations can be used to identify the major contributors to the SIC of barrier heights and reaction energies. Based on such comparisons, we suggest that barrier height predictions made using the strongly constrained and appropriately normed meta-generalized gradient approximation may have attained the best accuracy possible for a semi-local functional using the Perdew–Zunger SIC approach.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

On the Representativity of Electrode Microstructure Parameters and Their Electrochemical Response for Lithium Ion Batteries

Lithium-ion battery electrochemical models require an accurate description of the electrodes microstructures to be predictive that can be achieved through nanoscale imaging. Such observations are however limited by their field of view (FOV), as they provide only a subset of the whole electrode volume that does not necessarily represent the whole electrode microstructure heterogeneity, and therefore can bias the microstructure analysis. A microstructure scale electrochemical model was used to investigate lithium plating onset, material non-uniform utilization, and in-plane heterogeneities for an NMC-graphite full cell. To evaluate the representativeness, and thus relevance, of these model predictions, a coupled representativity analysis has been performed on the microstructure parameters and, in a novel way, on the full cell electrochemical response. Electrode microstructure parameters representativeness has been first quantified using the representative volume element (RVE) methodology. The RVE major flaw is that ultimately it can only conclude if a FOV contains representative subvolumes of the FOV, but not if the FOV itself is representative of the electrode volume. Analysis can conclude negatively ('FOV is not representative'), but not positively ('FOV is representative'). One major contribution of this work was to quantify the convergence of the RVE size with the FOV, to actually investigate the FOV representativeness and thus partly remedy this intrinsic limitation. The analysis determined that performing a standard RVE calculation, without exploring its FOV convergence, is likely to strongly underestimate the actual RVE size. The new RVE methodology has been automated in the NREL open-source Microstructure Analysis Toolbox (MATBOX) and is available to the battery community. Representativeness of microstructure parameters is however only an intermediate step, as the end-results of an electrochemical model are performances predictions. Indeed, what is the practical consequence of a given deviation for a microstructure parameter? The microstructure parameter deviation propagations to the 3D microstructure scale electrochemical response have been then quantified for different charge rates. This defines a threshold for the microstructure parameters FOV for a desired maximum deviation of the electrochemical response. Such deviation propagation analysis is analogous to error propagation analysis and is necessary to determine the relevance of microstructure scale model predictions for macroscale predictions. Electrochemical model shows cell representative section areas are increasing with C-rate, due to higher in-plane heterogeneities, indicating larger FOVs are required specifically for fast charge modeling. Therefore, we introduced the novel concept of electrochemical RVE (eRVE) that is a function of the operating conditions (thus defined as a dynamic RVE), with an increasing dependence with the C-rate. Representativity analysis of the investigated cell determined a FOV of 144.4 x 54.4 m2 is large enough to establish a convergence on the representative section areas for low to intermediate C-rate (=2.5C), but not large enough to conclude for higher rates. This work aims to emphasize the importance of representativity analysis for LIB electrode microstructures, as it is required to estimate the error, and thus the relevance, of microstructure parameters intended to be used in macroscale models. The methodology and results can help researchers to select the relevant imaging and associated FOV required to provide accurate enough microstructure parameters.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2022

Using an Explainable Machine Learning Approach to Characterize Earth System Model Errors: Application of SHAP Analysis to Modeling Lightning Flash Occurrence

Abstract Computational models of the Earth System are critical tools for modern scientific inquiry. Efforts toward evaluating and improving errors in representations of physical and chemical processes in these large computational systems are commonly stymied by highly nonlinear and complex error behavior. Recent work has shown that these errors can be effectively predicted using modern Artificial Intelligence (A.I.) techniques. In this work, we go beyond these previous studies to apply an explainable A.I. technique to not only predict model errors but also move toward understanding the underlying reasons for successful error prediction. We use XGBoost classification trees and SHapley Additive exPlanations analysis to explore the errors in the prediction of lightning occurrence in the NASA Goddard Earth Observing System model, a widely used Earth System Model. This explainable error prediction system can effectively predict the model error and indicates that the errors are strongly related to convective processes and the characteristics of the land surface.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Investigating Resilience of Loops in HPC Programs: A Semantic Approach with LLMs

Soft errors have become one of the major concerns for the error resilience of the HPC applications as those errors may cause HPC applications to generate serious outcomes such as silent data corruptions (SDCs). Protecting the applications from soft errors is an essential while challenging task. Among different approaches, obtaining a profound understanding of the resilience proneness of an application is very important to devise efficient error detection and recovery strategies. Given the scale of the HPC applications both in the code size and execution time, there are often cases that the error propagation analysis on such applications would produce a massive volume of unstructured data, which requires a significant amount of efforts, to process and to obtain indicating actions towards error protection. In this paper, we present a control-flow based visual analysis framework to help the users conduct error propagation analysis and identify the critical sections of a program that may have a higher likelihood of leading to erroneous outcomes when affected by the control flow related errors. We also design and implement the scalable visualization framework - ResilienceVis that efficiently and effectively visualizes the affected program states under errors and the propagation traces for an application in a user-friendly manner, and eventually, we combine the analysis and visualization to exhibit the error-proneness of the different sections of applications.

Jiang, Hailong↗