Search NASA⌕ Search

SEARCH · Search NASA

Results for “model diagnostics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Infrastructure Team at Lawrence Livermore National Laboratory (LLNL) to facilitate the use of long-term, high-frequency measurements from the ARM Facility in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The Coupled Model Intercomparison Project (CMIP) model data sets are also included in the package to enable model intercomparison as demonstrated in Zhang et al. (2017). The mean of the CMIP model can serve as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, and radiative fluxes, with plan to extend to more fields, such as aerosol and microphysics properties. Process-oriented diagnostics focusing on individual cloud- and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. The version 1.0 package is designed based on data collected at ARM’s Southern Great Plains (SGP) Research Facility, with the plan to extend to other ARM sites. The metrics and diagnostics package is currently built upon standard Python libraries and additional Python packages developed by DOE (such as CDMS and CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first present the input data, which constitutes the core content of the metrics and diagnostics package in section 2, and a user's guide documenting the workflow/structure of the version 1.0 codes, and including step-by-step instruction for running the package in section 3.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Multiscale modeling and laser diagnostics to reveal non-equilibrium reaction chemistry at a plasma-liquid interface

Low-temperature, atmospheric-pressure plasmas in contact with liquid are at the core of a wide range of applications including water treatment, medicine, materials synthesis, and chemical transformation. In general, plasma-liquid processes are scientifically compelling because reactivity can be produced without a catalyst, in relatively inert molecules, such as air or nitrogen and liquid water, in their native states at ambient conditions. However, reactions at a plasma-liquid interface are extremely complex, occurring at a multiphase, gas-liquid boundary where excited or dissociate gaseous species dissolve and react with solution-phase species, and unique reaction pathways are induced by non-equilibrium chemistry. In particular, detailed knowledge of the physical and chemical processes, including what species are produced in the gas phase and how these species are subsequently transported across and react near the interfacial region, remains largely unanswered. In this project, modeling of the non-equilibrium reaction chemistry at the interface of low-temperature, atmospheric-pressure plasmas and liquid water is developed, supported by advanced laser diagnostics.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

ARM Data-Oriented Metrics and Diagnostics Package for Climate Model Evaluation

A Python-based metrics and diagnostics package is currently being developed by the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term, high-frequency measurements from ARM in evaluating the regional climate simulation of clouds, radiation, and precipitation. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Transfer learning for probabilistic localization of hidden cracks in concrete structures

Abstract The utility of discriminative supervised learning models built using multiple training-data sources is investigated for hidden crack localization in concrete. Feed-forward neural network (FFNN) is chosen as the model architecture, and transfer learning is used to assimilate the information obtained from different sources (computational physics simulations and laboratory experiments). The labeled training data consists of values of a damage index and the known locations of hidden cracks. The classification models need to learn how the presence of damage (hidden cracks) affects the damage index at different sensors for different test conditions. To this end, diagnostic FFNN models are built by sequentially adding and training new hidden layers to assimilate labeled information from computer models (different model geometries, test conditions, crack lengths, crack locations) and laboratory experiments on a plain cement slab. These transfer learning-based models are then used to localize damage in concrete specimens that reflect real-world conditions (i.e., specimens with steel reinforcement and randomly distributed aggregate). The actual damage state in these specimens is determined by extracting cores and performing petrographic studies on the extracted cores. The damage probability estimated by transfer learning-based models is compared with the petrographic damage rating index (DRI) to identify the most suitable approach to train the diagnostic models. The transfer learning-based diagnostic methodology shows promise and could be used in various structural health monitoring applications, where sufficient labeled data are typically not available from a single data source.

Miele, S.↗
DOE OSTI2022

ARM data-oriented metrics and diagnostics package for climate model evaluation (ARM-DIAGS-V3) version 3

A Python-based metrics and diagnostics package is currently being developed by the ARM Infrastructure Team at Lawrence Livermore National Laboratory to facilitate the use of long-term high frequency measurements from the ARM program in evaluating the regional climate simulation of clouds, radiation, precipitation, and aerosols. This metrics and diagnostics package computes climatological means of targeted climate model simulation and generates tables and plots for comparing the model simulation with ARM observational data. The CMIP model data sets are also included in the package to enable model inter-comparison as demonstrated in Zhang et al. (2018) and Zhang et al. (2020). The mean of the CMIP model can be served as a reference for individual models. Basic performance metrics are computed to measure the accuracy of mean state and variability of climate models. The evaluated physical quantities include cloud fraction, temperature, relative humidity, cloud liquid water path, total column water vapor, precipitation, sensible and latent heat fluxes, aerosol optical depth, and radiative fluxes, with plan to extend to more fields, such as the evaluation of model simulated aerosol physicochemical properties and cloud microphysics properties. Process-oriented diagnostics focusing on aerosol, cloud, and precipitation-related phenomena are also being developed for the evaluation and development of specific model physical parameterizations. In addition to the Southern Great Plains (SGP), North Slope of Alaska (NSA) and Tropical Western Pacific (TWP) atmospheric observatories in the ARMDIAGS version 2.0, the version 3.0 package have extended to the data collected at the ARM Eastern North Atlantic (ENA) atmospheric observatory and the Observation and Modeling of the Green Ocean Amazon (GOAMAZON) field campaign. The metrics and diagnostics package are currently built upon standard Python libraries and additional Python packages developed by DOE (CDAT). The ARM metrics and diagnostic package is available publicly with the hope that it can serve as an easy entry point for climate modelers to compare their models with ARM data. In this report, we first provide an overview of major metrics in section 2. The input data, which constitutes the core content of the metrics and diagnostics package, is summarized in section 3. A user's guide documenting the workflow/structure of the version 3.0 codes and including step-by-step instruction for running the package is described in section 4.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

The ARM Data-oriented Metrics and Diagnostics Package for Climate Models - A New Tool for Evaluating Climate Models with Field Data

The U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) program User Facility produces ground-based long-term continuous unique measurements for atmospheric state, precipitation, turbulent fluxes, radiation, aerosol, cloud and the land surface, which are collected at multiple sites. These comprehensive datasets have been widely used to calibrate climate models and are proven to be invaluable for climate model development and improvement. This article introduces an evaluation package to facilitate the use of ground-based ARM measurements in climate model evaluation. The ARM data-oriented metrics and diagnostics package (ARM-DIAGS) includes both ARM observational datasets and a Python-based analysis toolkit for computation and visualization. The observational datasets are compiled from multiple ARM data products and specifically tailored for use in climate model evaluation. In addition, ARM-DIAGS also includes simulation data from models participating the Coupled Model Inter-comparison Project (CMIP), which will allow climate-modeling groups to compare a new, candidate version of their model to existing CMIP models. The analysis toolkit is designed to make the metrics and diagnostics quickly available to the model developers.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Improving Reliability of Large Language Models for Nuclear Power Plant Diagnostics Technical Presentation

Large Language Models (LLMs) struggle out of the box when answering factually about detailed questions, especially in domains that are sparsely represented in their training data. This causes hallucinations and reduces reliability making it difficult for them to be used in practice. This work shows that using RAG techniques can improve factual accuracy and reliability, allowing for the application of LLMs in specialized areas, even when those areas that aren’t extensively covered in their initial training.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

Integrated Data Analysis to Improve Te and Zeff Measurement on NSTX-U

The multi-energy soft x-ray (ME-SXR) camera is a new concept for a fusion plasma diagnostic based on a pixelated detector with adjustable photon cutoff thresholds. It is a single, compact, solid-state detector which combines spatial and spectral sensitivity and can be configured remotely at run time, making it ideal for a fusion environment. The detector can be configured for application to a wide range of physics problems including impurity transport, electron temperature profiles, and non-thermal effects. The detector has been installed on the Madison Symmetric Torus (MST) as part of a collaboration between Princeton Plasma Physics Laboratory and the University of Wisconsin–Madison to develop the technology and analysis techniques. A full diagnostic forward model of the ME-SXR diagnostic has been developed to enable analysis of the raw photon count data. This model incorporates the atomic physics underlying the plasma emission, diagnostic geometry, and detector response characteristics. Employing this model, a Bayesian analysis approach is used to extract electron temperature from the raw data. This analysis approach uses all available information, clearly quantifies prior information, and handles uncertainties transparently. The ME-SXR camera is being used to study the 3D (helical, stellarator-like) state that forms in the MST device. This helical core structure has improved energy confinement. The transition to the helical state is accompanied by a rise in electron temperature in the central part of the helical structure, but then this enhancement fluctuates. The characteristics of this fluctuation are being compared to a magneto-hydrodynamics model.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

A Pacific Tropical Decadal Variability Challenge for Climate Models

Understanding and forecasting Tropical Pacific Decadal-scale Variability (TPDV) strongly rely on climate model simulations. Using a Linear Inverse Modeling (LIM) diagnostic approach, we reveal Coupled Model Intercomparison Project Phase 6 models have significant challenges in reproducing the spatial structure and dominant mechanisms of TPDV. Specifically, while the models' ensemble mean pattern of TPDV resembles that of observations, the spread across models is very large and most models show significant differences from observations. In observations, removing the coupling between extratropics and tropics reduces TPDV by ~60%–70%, and removing the tropical thermocline variability makes the central tropical Pacific a key center of action for TPDV and El Niño Southern Oscillation variability. These characteristics are only confirmed in a subset of models. Differences between observations and simulations are outside the range of natural internal TPDV noise and pose important questions regarding our ability to model the impacts of natural internal low-frequency variability superimposed on long-term climate change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

The kinetic analog of the pressure–strain interaction

Energy transport in weakly collisional plasma systems is often studied with fluid models and diagnostics. However, the applicability of fluid models is limited when collisions are weak or absent, and using a fluid approach can obscure kinetic processes that provide key insights into the physics of energy transport. Kinetic diagnostics retain all of the information in 3D-3V phase space and thereby reach beyond the insights of fluid models to elucidate the mechanisms responsible for collisionless energy transport. In this work, we derive the Kinetic Pressure–Strain (KPS): a kinetic analog of the pressure–strain interaction, which is the channel between flow energy density and internal energy density in fluid models. Through two case studies of electron Landau damping, we demonstrate that the KPS diagnostic can elucidate kinetic mechanisms that are responsible for energy transport in this channel, just as the related field–particle correlation is known to identify kinetic mechanisms of transport between electromagnetic field energy density and kinetic energy density in particle flows. In addition, we show that resonant electrons play a major role in transferring energy between fluid flows and internal energy during the process of Landau damping.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Scalable Technologies Achieving Risk-Informed Condition-Based Predictive Maintenance Enhancing the Economic Performance of Operating Nuclear Power Plants

The primary objective of the research presented in this report is to develop scalable technologies that are deployable across plant assets and across the nuclear fleet to achieve risk-informed predictive maintenance (PdM) strategies at commercial nuclear power plants (NPPs). Over the years, the nuclear fleet has relied on labor-intensive and time-consuming preventive maintenance (PM) programs, driving up operation and maintenance (O&M) costs to achieve high capacity factors. A well-constructed risk-informed PdM approach for an identified plant asset has been developed in this research, taking advantage of advancements in data analytics, machine learning (ML), artificial intelligence (AI), physics-informed modeling, and visualization. These technologies would allow commercial NPPs to reliably transition from current labor-intensive PM programs to a technology driven PdM program, eliminating unnecessary O&M costs. The work presented in the report is being developed as part of a collaborative research effort between Idaho National Laboratory and Public Service Enterprise Group Nuclear, LLC. This report (1) reflects the results of work by LWRS Program researchers with PSEG, Nuclear LLC-owned Salem and Hope Creek Nuclear Power Plants; (2) presents utilization of circulating water system (CWS) heterogeneous data and fault modes from both the Salem and Hope Creek nuclear power plant sites to develop salient fault signatures associated with each fault mode; (3) describes the integration of component-level predictive models into a robust system-level model enabled by the federated-transfer learning; (4) describes the development of physics-informed model of circulating water pump and motor; (5) develops a scalable risk and economic model; and (6) outlines the development of a user-centric visualization application. The outcomes presented in this report lays the foundation and provides a much-needed technical basis to focus on explainability and trustworthiness of ML and AI-based technologies, as part of future research.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

eagles-project/asediag

Aerosol process analysis in model-native Spectral Element (SE) grid. Aerosol Diagnostics on Model Native Grid is a Python-based tool designed for diagnosing aerosol processes in E3SM. It is particularly designed to analyze simulation data on the model-native spectral element (SE) grid, soit is also known as Aerosol SE Diagnostics or “asediag”

Hassan, Taufiq↗
DOE OSTI2024

Results from a synthetic model of the ITER XRCS-Core diagnostic based on high-fidelity x-ray ray tracing

A high-fidelity synthetic diagnostic has been developed for the ITER core x-ray crystal spectrometer diagnostic based on x-ray ray tracing. This synthetic diagnostic has been used to model expected performance of the diagnostic, to aid in diagnostic design, and to develop engineering tolerances. The synthetic model is based on x-ray ray tracing using the recently developed xicsrt ray tracing code and includes a fully three-dimensional representation of the diagnostic based on the computer aided design. The modeled components are: plasma geometry and emission profiles, highly oriented pyrolytic graphite pre-reflectors, spherically bent crystals, and pixelated x-ray detectors. Plasma emission profiles have been calculated for Xe 44+ , Xe 47+ , and Xe 51+ , based on an ITER operational scenario available through the Integrated Modelling & Analysis Suite database, and modeled within the ray tracing code as a volumetric x-ray source; the shape of the plasma source is determined by equilibrium geometry and an appropriate wavelength distribution to match the expected ion temperature profile. All individual components of the x-ray optical system have been modeled with high-fidelity producing a synthetic detector image that is expected to closely match what will be seen in the final as-built system. Particular care is taken to maintain preservation of photon statistics throughout the ray tracing allowing for quantitative estimates of diagnostic performance.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Systematic benchmarking demonstrates large language models have not reached the diagnostic accuracy of traditional rare-disease decision support tools

Large language models (LLMs) show promise in supporting differential diagnosis, but their performance is challenging to evaluate due to the unstructured nature of their responses, and their accuracy compared to existing diagnostic tools is not well characterized. To assess the current capabilities of LLMs to diagnose genetic diseases, we benchmarked these models on 5213 previously published case reports using the Phenopacket Schema, the Human Phenotype Ontology and Mondo disease ontology. Prompts generated from each phenopacket were sent to seven LLMs, including four generalist models and three LLMs specialized for medical applications. The same phenopackets were used as input to a widely used diagnostic tool, Exomiser, in phenotype-only mode. The best LLM ranked the correct diagnosis first in 23.6% of cases, whereas Exomiser did so in 35.5% of cases. While the performance of LLMs for supporting differential diagnosis has been improving, it has not reached the level of commonly used traditional bioinformatics tools. Future research is needed to determine the best approach to incorporate LLMs into diagnostic pipelines.

Reese, Justin T. [Lawrence Berkeley National Labor↗
DOE OSTI2024

Evaluation of Deep Learning Model Architectures for Point-of-Care Ultrasound Diagnostics

Point-of-care ultrasound imaging is a critical tool for patient triage during trauma for diagnosing injuries and prioritizing limited medical evacuation resources. Specifically, an eFAST exam evaluates if there are free fluids in the chest or abdomen but this is only possible if ultrasound scans can be accurately interpreted, a challenge in the pre-hospital setting. In this effort, we evaluated the use of artificial intelligent eFAST image interpretation models. Widely used deep learning model architectures were evaluated as well as Bayesian models optimized for six different diagnostic models: pneumothorax (i) B- or (ii) M-mode, hemothorax (iii) B- or (iv) M-mode, (v) pelvic or bladder abdominal hemorrhage and (vi) right upper quadrant abdominal hemorrhage. Models were trained using images captured in 27 swine. Using a leave-one-subject-out training approach, the MobileNetV2 and DarkNet53 models surpassed 85% accuracy for each M-mode scan site. The different B-mode models performed worse with accuracies between 68% and 74% except for the pelvic hemorrhage model, which only reached 62% accuracy for all model architectures. These results highlight which eFAST scan sites can be easily automated with image interpretation models, while other scan sites, such as the bladder hemorrhage model, will require more robust model development or data augmentation to improve performance. With these additional improvements, the skill threshold for ultrasound-based triage can be reduced, thus expanding its utility in the pre-hospital setting.

47 OTHER INSTRUMENTATION↗