SEARCH · Search NASA

Results for “evaluation methodology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Exact wave solver for nonparaxial laser beam propagation

Simulations of inertial confinement fusion (ICF) experiments require high-fidelity models for laser beam propagation in a nonuniform plasma with varying index of refraction. We describe a new numerical wave solver that is applicable to centimeter-scale length plasmas encountered in indirect drive ICF applications. The one-way Helmholtz equation (OHE) generalizes the time-harmonic paraxial wave equation to large angles. Here, we present a methodology to numerically evaluate the exact solution to the OHE. This solution is computed by analytically advancing eigenfunctions of the one-way Helmholtz operator along a propagation direction and is applicable to any given index of a refraction profile. We compare our exact method with a commonly used approximate split-step technique for solving the OHE. As a test problem, we consider nonparaxial propagation of Gaussian and speckled beams in a plasma density channel with internal reflection. We find that the split-step approach incurs significant errors compared to the exact solution computed using the novel algorithm.

Belyaev, Mikhail A. (ORCID:0000000224908887)↗

DOE OSTI2024

LHC EFT WG note: SMEFT predictions, event reweighting, and simulation

This note provides a comprehensive overview of tools for predicting observables in the Standard Model effective field theory (SMEFT) at both tree level and one loop using event generators. We evaluate three primary methodologies–event reweighting, separate simulation of squared matrix elements, and full SMEFT process simulation–focusing on their statistical performance, computational efficiency, and potential biases. Each approach is assessed in terms of its accuracy, highlighting trade-offs between precision and resource demands. Practical insights into their applicability for high-energy physics analyses are offered, with particular attention to processes where SMEFT effects are significant. Additionally, we discuss the role of helicity in reweighting strategies and its impact on the quality of predictions. By comparing the methods across various LHC processes, this note provides guidance for selecting the most effective strategy for various SMEFT studies, ensuring robust predictions while optimizing computational resources.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Introduction to Special Section: Machine Learning for Image-based Geologic Interpretation

Image-based geological interpretation has been a labor-intensive and time-consuming process because it requires well-trained geoscientists to identify geological structures, features, and textures from various types of images. These images include scanning electron microscopic images, optical microscopic images, optical photos, resistivity images, seismic volumes, remote-sensing images, etc. With fast-evolving machine learning (ML) technology and computing power in recent decades, computers can achieve nearhuman-level to super-human-level performance with scalable high efficiency in the computer vision field. These technological revolutions facilitated image-based geological interpretation in petroleum exploration and production. For example, a fault picking method applied to 3-D seismic volume data using deep learning can achieve superior performance in comparison to conventional auto-picking methods. In addition, under the new normal of low oil prices, the petroleum industry seeks cost-effective strategies such as automating traditionally labor-intensive processes. Nevertheless, the potential of applying ML to geological image interpretation is still facing a few key challenges including data scarcity, data distribution, poor data and/or label quality, data leakage, learning algorithms, model architecture, training methodologies, testing and evaluation metrics, hyper-parameters optimization, model drift, production deployment, and the like.

58 GEOSCIENCES↗

DOE OSTI2023

Assessment of Modeling and Simulation Technical Gaps in Safety Analysis of High Burnup Accident-Tolerant Fuels

The United States nuclear industry is facing a strong challenge to maintain regulatory-required levels of safety while ensuring economic competitiveness to stay in business. Safety remains a key parameter for all aspects related to the operation of light water reactor (LWR) nuclear power plants (NPPs), and it can be achieved more economically by using a risk-informed ecosystem, such as that being developed by the Risk-Informed Systems Analysis (RISA) Pathway under the U.S. Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program. The LWRS Program is promoting a wide range of research and development activities to maximize both the safety and economically efficient performance of NPPs through improved scientific understanding, especially given that many plants are considering second license renewal. The RISA Pathway has two main goals: The deployment of methodologies and technologies that enable better representation of safety margins and the factors that contribute to cost and safety, and; The development of advanced applications that enable cost-effective plant operation. As part of the RISA Pathway, the Enhanced Resilient Plant (ERP) project refers to an NPP where safety is improved by implementing various measures, such as accident-tolerant fuels (ATF), diverse and flexible coping strategy (FLEX), enhancements to plant components and systems, incorporation of augmented or new passive cooling systems, and utilization of advanced battery technologies. The objective of the ERP research is to use novel methods and computational tools to enhance existing reactors’ safety while reducing operational costs. Many U.S. utilities are targeting implementation of ATFs instead of traditional fuel in the near future since ATFs offer benefits in terms of improved performance and cost savings. The robust properties of ATF make it possible to extend the refueling cycle from 18 to 24 months in addition to the opportunity to use less of fuel. Extensive safety assessments are required to support regulatory requirements and obtain the approvals to use ATFs and the ERP project support the industry by developing novel effective methodologies for safety evaluations. In this project, the technical gaps in the modeling and simulation of the high burnup (HBU) ATF were assessed in terms of the fuel cladding behavior during the postulated accident events. The issues were identified in modeling the cladding deformation, the hydrodynamic change due to cladding deformation and the critical heat flux (CHF). The RELAP5-3D cladding deformation model was assessed by multiple verification tests and validation with the instrumented fuel assembly (IFA) experiment.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2026

Demonstration of Algorithm for Sensor Placement Optimization using Simulation Data

This deliverable reports FY26 progress in advancing a neural-network-based Green’s-function framework for reconstructing neutron-flux distributions from ex-core measurements and for translating reconstruction requirements into a practical detector-layout strategy. Building on the FY25 formulation, the present work had two main objectives: (1) refine and re-evaluate the reconstruction methodology on an updated Purdue University Reactor Number One (PUR-1) model, and (2) develop a systematic, Green’s-function-guided procedure for boundary detector placement.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2024

Personalized, disease-stage specific, rapid identification of immunosuppression in sepsis

Introduction Data overlapping of different biological conditions prevents personalized medical decision-making. For example, when the neutrophil percentages of surviving septic patients overlap with those of non-survivors, no individualized assessment is possible. To ameliorate this problem, an immunological method was explored in the context of sepsis. Methods Blood leukocyte counts and relative percentages as well as the serum concentration of several proteins were investigated with 4072 longitudinal samples collected from 331 hospitalized patients classified as septic (n=286), non-septic (n=43), or not assigned (n=2). Two methodological approaches were evaluated: (i) a reductionist alternative, which analyzed variables in isolation; and (ii) a non-reductionist version, which examined interactions among six (leukocyte-, bacterial-, temporal-, personalized-, population-, and outcome-related) dimensions. Results The reductionist approach did not distinguish outcomes: the leukocyte and serum protein data of survivors and non-survivors overlapped. In contrast, the non-reductionist alternative differentiated several data groups, of which at least one was only composed of survivors (a finding observable since hospitalization day 1). Hence, the non-reductionist approach promoted personalized medical practices: every patient classified within a subset associated with 100% survival subset was likely to survive. The non-reductionist method also revealed five inflammatory or disease-related stages (provisionally named ‘early inflammation, early immunocompetence, intermediary immuno-suppression, late immuno-suppression, or other’). Mortality data validated these labels: both ‘suppression’ subsets revealed 100% mortality, the ‘immunocompetence’ group exhibited 100% survival, while the remaining sets reported two-digit mortality percentages. While the ‘intermediary’ suppression expressed an impaired monocyte-related function, the ‘late’ suppression displayed renal-related dysfunctions, as indicated by high concentrations of urea and creatinine. Discussion The data-driven differentiation of five data groups may foster early and non-overlapping biomedical decision-making, both upon admission and throughout their hospitalization. This approach could evaluate therapies, at personalized level, earlier. To ascertain repeatability and investigate the dynamics of the ‘other’ group, additional studies are recommended.

Immunology↗

DOE OSTI2022

Metrics for Intercomparison of Remapping Algorithms (MIRA) protocol applied to Earth system models

Abstract. Strongly coupled nonlinear phenomena such as those described by Earth system models (ESMs) are composed of multiple component models with independent mesh topologies and scalable numerical solvers. A common operation in ESMs is to remap or interpolate component solution fields defined on their computational mesh to another mesh with a different combinatorial structure and decomposition, e.g., from the atmosphere to the ocean, during the temporal integration of the coupled system. Several remapping schemes are currently in use or available for ESMs. However, a unified approach to compare the properties of these different schemes has not been attempted previously. We present a rigorous methodology for the evaluation and intercomparison of remapping methods through an independently implemented suite of metrics that measure the ability of a method to adhere to constraints such as grid independence, monotonicity, global conservation, and local extrema or feature preservation. A comprehensive set of numerical evaluations is conducted based on a progression of scalar fields from idealized and smooth to more general climate data with strong discontinuities and strict bounds. We examine four remapping algorithms with distinct design approaches, namely ESMF Regrid (Hill et al., 2004), TempestRemap (Ullrich and Taylor, 2015), generalized moving least squares (GMLS) (Trask and Kuberry, 2020) with post-processing filters, and WLS-ENOR (Li et al., 2020). By repeated iterative application of the high-order remapping methods to the test fields, we verify the accuracy of each scheme in terms of their observed convergence order for smooth data and determine the bounded error propagation using challenging, realistic field data on both uniform and regionally refined mesh cases. In addition to retaining high-order accuracy under idealized conditions, the methods also demonstrate robust remapping performance when dealing with non-smooth data. There is a failure to maintain monotonicity in the traditional L2-minimization approaches used in ESMF and TempestRemap, in contrast to stable recovery through nonlinear filters used in both meshless GMLS and hybrid mesh-based WLS-ENOR schemes. Local feature preservation analysis indicates that high-order methods perform better than low-order dissipative schemes for all test cases. The behavior of these remappers remains consistent when applied on regionally refined meshes, indicating mesh-invariant implementations. The MIRA intercomparison protocol proposed in this paper and the detailed comparison of the four algorithms demonstrate that the new schemes, namely GMLS and WLS-ENOR, are competitive compared to standard conservative minimization methods requiring computation of mesh intersections. The work presented in this paper provides a foundation that can be extended to include complex field definitions, realistic mesh topologies, and spectral element discretizations, thereby allowing for a more complete analysis of production-ready remapping packages.

58 GEOSCIENCES↗

DOE OSTI2021

Call to Action for Global Access to and Harmonization of Quality Information of Individual Earth Science Datasets

Knowledge about the quality of data and metadata is important to support informed decisions on the (re)use of individual datasets and is an essential part of the ecosystem that supports open science. Quality assessments reflect the reliability and usability of data. They need to be consistently curated, fully traceable, and adequately documented, as these are crucial for sound decision- and policy-making efforts that rely on data. Quality assessments also need to be consistently represented and readily integrated across systems and tools to allow for improved sharing of information on quality at the dataset level for individual quality attribute or dimension. Although the need for assessing the quality of data and associated information is well recognized, methodologies for an evaluation framework and presentation of resultant quality information to end users may not have been comprehensively addressed within and across disciplines. Global interdisciplinary domain experts have come together to systematically explore needs, challenges and impacts of consistently curating and representing quality information through the entire lifecycle of a dataset. This paper describes the findings of that effort, argues the importance of sharing dataset quality information, calls for community action to develop practical guidelines, and outlines community recommendations for developing such guidelines. Practical guidelines will allow for global access to and harmonization of quality information at the level of individual Earth science datasets, which in turn will support open science.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2025

Common practices for quantifying methane emissions from plumes detected by remote sensing

This document provides a set of community-accepted practices for quantifying methane emissions based on plumes detected via spectroscopic remote sensing. Its primary goal is to promote consistency in the generation, validation, reporting, and quality assessment of methane emission estimates derived from remote sensing radiances. Developed by subject matter experts with deep experience across all stages of the measurement process, this guidance reflects a critical evaluation of current methodologies and highlights key practices needed to produce reliable, interoperable, and traceable products. The focus is specifically on methane emissions quantified from distinct plumes originating from localized sources, rather than diffuse emissions spread over large regions, which are beyond the scope of this work. This document is intended to serve both data producers and users. For producers, it offers a framework for aligning with field-recognized standards to ensure their outputs meet rigorous quality and transparency criteria. For users, it provides a reference to assess dataset fitness-for-purpose by highlighting essential metadata, assumptions, and methodological choices that underpin emission estimates. By fostering a shared understanding of best practices, this work aims to enhance comparability, confidence, and utility of remotely sensed methane emission products.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Optimization and Performance Characterization of the Second Generation Fermilab Constant Fraction Discriminator Readout ASIC

We present the optimization and performance characterization of the second-generation Fermilab Constant Fraction Discriminator ASIC (FCFD), designed for the readout of AC-coupled low-gain avalanche detector (LGAD) strip-sensors. The FCFD is explicitly engineered to be insensitive to signal-amplitude variations, thereby removing the need for time-walk correction that is required in other LGAD time-stamping readout ASICs. This updated version, referred to as FCFD1.1, incorporates several enhancements over the first iteration to address key challenges in AC-coupled LGAD front-end design. We outline the primary readout-ASIC design considerations for these applications, describe the methodology used to evaluate critical sensor and system parameters, and summarize the additionally implemented features. Performance measurements using injected charge signals and minimum-ionizing particles in test-beams demonstrate a time resolution of approximately 40 ps and a position resolution of roughly \SI{15}{\micro\meter} when tested with beam particles uniformly over the active area of the sensor.

Apresyan, Artur [Fermilab] (ORCID:0000000261860130↗

DOE OSTI2024

Enhancing Site Screening for Underground Hydrogen Storage: Qualitative Site Quality Assessment - SHASTA: Subsurface Hydrogen Assessment, Storage, and Technology Acceleration Project

The global shift towards renewable energy sources to mitigate fossil fuel dependency and meet carbon emission targets by 2050 has underscored the importance of innovative solutions to address energy supply-demand imbalances. Underground Hydrogen Storage (UHS) has emerged as a promising strategy to store excess renewable energy in subsurface formations for future retrieval and utilization. This report focuses on enhancing the site screening process for UHS facilities. By drawing on insights from prior research to identify key criteria influencing site suitability, our objective is to present a comprehensive set of sixteen specific criteria essential for refining the selection of UHS sites. These criteria cover various aspects such as reservoir performance, legal access, regulatory compliance, economic viability, public acceptance, and safety and security considerations. Our proposed methodology allows users to evaluate potential sites using binary responses, transitioning from isolated factors to broader considerations that facilitate a qualitative assessment of site suitability. This method enables comparative analysis and informed decision-making, supporting stakeholders in the site selection process. Our approach is designed as a guiding framework rather than a rigid template for site screening, emphasizing the importance of customizing approaches to individual circumstances. As UHS site development project evolves, adopting a comprehensive selection methodology that integrates holistic evaluations will be essential for ensuring efficient and effective site screening processes tailored to specific project needs.

08 HYDROGEN↗

DOE OSTI2024

Enhancing Site Screening for Underground Hydrogen Storage: Qualitative Site Quality Assessment - SHASTA: Subsurface Hydrogen Assessment, Storage, and Technology Acceleration Project

08 HYDROGEN↗

DOE OSTI2025

Identifying rolling resistance and air resistance simultaneously for an electric truck

Accurately estimating rolling and air resistance is essential for predicting the energy consumption of vehicles. This study presents a field-based approach using a rolldown test to simultaneously determine rolling and air resistance coefficients. Unlike prior methods that frequently used simulations or models, we employ a goal programming methodology to improve precision and evaluate the actual vehicle and environmental conditions. Our methodology was tested using a Class 8 Freightliner eCascadia on a surveyed road section, ensuring controlled conditions for data collection. By analyzing the time–velocity relationship across multiple test runs, we derived resistance coefficients for both loaded and unloaded conditions. The study confirms that rolling resistance is largely independent of velocity at low speeds but exhibits a nonlinear dependency at higher speeds. Additionally, road surface conditions, tire condition, axle configuration, aerodynamic properties, and weather conditions significantly impact resistance values, emphasizing the need for real-world testing rather than relying solely on standardized projections. Our results align with existing literature while demonstrating the efficacy of the goal programming approach in refining resistance estimates. This work contributes to improved vehicle energy modeling, offering practical insights for fleet operators and policymakers seeking accurate energy consumption predictions for electric trucks operating under varying environmental conditions.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

An Evaluation of The Dynamic Physical Security Risk Assessment Methodology for Fleet-Wide Applications

The requirements for U.S. nuclear power plants to maintain a large onsite physical security force contribute to their high operational costs. The cost of maintaining the current physical security posture is approximately 10% of the overall operation and maintenance budget for commercial nuclear power plants. The goal of the Light Water Reactor Sustainability (LWRS) program’s physical security pathway is to develop tools, methods, and technologies and provide the technical basis for an optimized physical security posture. The conservatisms built into current security postures may be analyzed and minimized to reduce security costs while still ensuring adequate security and operational safety. The research performed at Idaho National Laboratory within LWRS program’s physical security pathway has successfully developed a dynamic force-on-force modeling framework using various computer simulation tools and integrating them with the dynamic assessment Event Modeling Risk Assessment using Linked Diagrams (EMRALD) tool. This integrated process for physical security analysis is named Modeling and Analysis for Safety Security using Dynamic EMRALD Framework (MASS-DEF). This document provides an update on the progress in applying the MASS-DEF process to an operating commercial nuclear power plant as well as additional industry feedback regarding use of the tool for other physical security risk-informed topics. This report is only a summary of the progress and does not contain specific modeling results as those contain sensitive security information. Previous reports described how a user could integrate their plant-specific force-on-force models with the dynamic simulation tool EMRALD, model operator actions, and integrate with probabilistic risk assessment tools, such as CAFTA (Computer Aided Fault Tree Analysis System) or SAPHIRE (Systems Analysis Programs for Hands-on Integrated Reliability Evaluations), and with thermal-hydraulic tools, such as RELAP-5 or MAAP. Previous reports applied various combinations of available simulations codes with EMRALD using generic plant models to demonstrate how to perform the analysis. This report is an update the progress of applying the dynamic computational framework to an actual nuclear facility using their security scenarios and timelines. This report also provides an update to the procedural guidance for the MASS-DEF process and an overview of the generic models available for use by utilities. This report does not contain any plant’s sensitive information and/or safeguards information. This study’s purpose was to verify that the results achieved using generic models are similar to actual plant results and refine our guidance on the use of the framework. This assessment enables further analysis, such as what-if scenarios and staff-reduction evaluation, thereby optimizing physical security at plants.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Evaluation of Mesoscale Convective Systems in Climate Simulations: Methodological Development and Results from MPAS-CAM over the United States

Explore the source record for details and available documents.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Validating automated resonance evaluation with synthetic data

The integrity and precision of nuclear data are crucial for a broad spectrum of applications, from national security and nuclear reactor design to medical diagnostics, where the associated uncertainties can significantly impact outcomes. A substantial portion of uncertainty in nuclear data originates from the subjective biases in the evaluation process, a crucial phase in the nuclear data production pipeline. Recent advancements indicate that automation of certain routines can mitigate these biases, thereby standardizing the evaluation process and enhancing reproducibility. This research aims to provide a methodology, framework, and metrics for the validation of automated nuclear data evaluation software leveraging high-quality synthetic data that closely mimic real experimental observables. An introduced error metric provides a scale and intuitive measure of the evaluation quality by quantifying the estimate’s accuracy and performance across the specified energy range. Synthetic data provides access to experimental observables and underlying resonance parameters, enabling comparison of different evaluations. The methodology is demonstrated using Ta-181 isotope data in the resolved resonance region. The Automated Resonance Identification Subroutine (ARIS), which operates without prior resonance information, was used to test and showcase the framework’s capabilities utilizing the proposed error metrics. The results demonstrate the effectiveness of the proposed approach and framework for optimizing software parameters and testing hypotheses through “what-if” controlled experiments, such as modifying assumptions about experimental conditions or average resonance parameters.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

Dynamic Probabilistic Safety Assessment Studies for Advanced Reactor Using RAVEN

Probabilistic Safety Assessment (PSA) is used extensively to evaluate the risks associated with complex engineering systems like Nuclear Power Plants (NPPs). Current PSA models are based on the Event-Tree/Fault-Tree (ET/FT) methodology. ET and FT models are static and are based on Boolean logic approaches. In the past, concerns have been raised in the literature regarding the capability of the traditional static modelling approaches to adequately account for the impact of process, hardware, software, firmware and human interactions on the stochastic system behaviour. To overcome the limitations of the traditional approach to PSA, several dynamic PSA methodologies have been proposed. One of the dynamic PSA methodologies used for dynamic evaluations is Dynamic Event Tree (DET) framework which can be used to assess the impact of the parameter variability and scenario dynamics on the PSA model for the initiating event. The DET framework couples the stochastic model (number of component/trains that start on demand, operator action timing, etc.) with a Thermal-Hydraulic (TH) model of the plant. This paper explores the use of DET along with a case study on advanced reactor. The initiating event selected for the study was Class IV power supply failure event. The TH analysis considering uncertainty in various parameters was performed using RELAP5 and Reactor Analysis and Virtual control ENvironment (RAVEN) tool. Based on the uncertainty analysis, it is concluded that the peak clad temperatures (PCT) are within the limits in all the code runs implying a high-degree of safety margin. However, variation in time to reach the PCT was observed among the code runs and the mean time to reach the PCT was found to be around 8590sec (approximately 2.4 hours). Hence, sufficient time margin is available for human intervention and the operator might have a relatively stress-free state during such an accident scenario. Due to the static nature of the traditional PSA models, the safety margin available was lesser, whereas, with the help of dynamic PSA models, one can demonstrate that the actual available safety margin is more in the present case study and is valuable input from the design point of view.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

SIMULATION CLONING FOR DIGITAL TWINS: A SCALABLE APPROACH

Digital Twin (DT) methods represent an important technology in which a simulated model of the operations of a physical system uses real-time sensor data to simulate, monitor, and consequently, improve its operations. One of the primary objectives of such a DT is to inform the physical system of measures to be taken in response to one or multiple intervening events that can change the state of the physical system. As such, a capability that is able to carry out multiple scenario assessments in real time in readiness for such events is a very effective tool in the use of simulations as DTs. However, continuous evaluation with highly probable event simulation scenarios are challenging due to the constraints of finite memory and a large exploration space. This paper reports a novel methodology for the continuous evaluation of $k$ probabilistic \textit{what-if} event scenarios under finite resource constraints and demonstrates its use as a digital-twin for a real-world application.

Yoginath, Srikanth↗