SEARCH · Search NASA

Results for “Integral Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Search for a heavy pseudoscalar Higgs boson decaying into a 125 GeV Higgs boson and a Z boson in final states with two tau and two light leptons at $\sqrt{s}=$ 13 TeV

A search is performed for a pseudoscalar Higgs boson, A, decaying into a 125 GeV Higgs boson h and a Z boson. The h boson is specifically targeted in its decay into a pair of tau leptons, while the Z boson decays into a pair of electrons or muons. A data sample of proton-proton collisions collected by the CMS experiment at the LHC at $ \sqrt{s} $ = 13 TeV is used, corresponding to an integrated luminosity of 35.9 fb$^{−1}$. No excess above the standard model background expectations is observed in data. A model-independent upper limit is set on the product of the gluon fusion production cross section for the A boson and the branching fraction to Zh → ℓℓττ. The observed upper limit at 95% confidence level ranges from 27 to 5 fb for A boson masses from 220 to 400 GeV, respectively. The results are used to constrain the extended Higgs sector parameters for two benchmark scenarios of the minimal supersymmetric standard model.[graphic not available: see fulltext]

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

DFAT: A web-based toolkit for estimating demand flexibility in building-to-grid integration

Demand Flexibility Assessment Tool (DFAT) is an open source web-based tool that estimates the demand flexibility potential of common control strategies in commercial buildings. The toolkit features a demand flexibility estimation tool that contains two calculators, basic and advanced, based on the level of input of customer data. The basic version calculates demand shed metrics for the control strategy “global temperature adjustment” and “cycle on/off compressors” using customer building information, local weather data, and electrical meter data. The advanced version, which uses detailed HVAC equipment data, calculates demand flexibility metrics for control strategies such as static pressure reset, global temperature adjustment, and cycle on/off compressors. In addition to the demand flexibility estimation tool, this toolkit offers a benchmarking tool that helps facility operators, aggregators, and utility resource managers assess demand flexibility opportunities, quantify/verify performance, and compare their performance against that of their peers.

Leong, Michael↗

DOE OSTI2020

Modern Approaches to Exact Diagonalization and Selected Configuration Interaction with the Adaptive Sampling CI Method

Recent advances in selected configuration interaction methods have made them competitive with the most accurate techniques available and, hence, creating an increasingly powerful tool for solving quantum Hamiltonians. In this work, we build on recent advances from the adaptive sampling configuration interaction (ASCI) algorithm. We show that a useful paradigm for generating efficient selected CI/exact diagonalization algorithms is driven by fast sorting algorithms, much in the same way iterative diagonalization is based on the paradigm of matrix vector multiplication. We present several new algorithms for all parts of performing a selected CI, which includes new ASCI search, dynamic bit masking, fast orbital rotations, fast diagonal matrix elements, and residue arrays. The ASCI search algorithm can be used in several different modes, which includes an integral driven search and a coefficient driven search. The algorithms presented here are fast and scalable, and we find that because they are built on fast sorting algorithms they are more efficient than all other approaches we considered. After introducing these techniques, we present ASCI results applied to a large range of systems and basis sets to demonstrate the types of simulations that can be practically treated at the full-CI level with modern methods and hardware, presenting double- and triple-ζ benchmark data for the G1 data set. The largest of these calculations is Si$_2$H$_6$ which is a simulation of 34 electrons in 152 orbitals. We also present some preliminary results for fast deterministic perturbation theory simulations that use hash functions to maintain high efficiency for treating large basis sets.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Superdiffusion resilience in Heisenberg chains with two-dimensional interactions on a quantum processor

Superdiffusive spin transport in the one-dimensional (1D) Heisenberg model is a key theoretical discovery in nonequilibrium quantum many-body physics. Although extensively studied in 1D systems, the breakdown and sustenance of superdiffusion in two-dimensional (2D) lattices with integrability-breaking terms, as found in real materials, remains an open question. To address this, we develop a toy model that extends the 1D Heisenberg model with a representative set of 2D interaction types and tunable strengths. Our model exhibits varying degrees of superdiffusion breakdown depending on the interaction type, spanning ballistic to diffusive regimes. We establish and justify a hierarchy of 2D interactions based on their resilience against superdiffusion breakdown: Heisenberg >𝑋⁢𝑋 > Ising. This precise control over the superdiffusive behavior also enables rigorous benchmarking of quantum hardware, and our simulations on IBM's Heron devices confirm the hardware's ability to accurately capture these many-body nonequilibrium phenomena. Overall, our results are relevant not only to simulating superdiffusion in real materials, such as the 1D Heisenberg compound KCuF3, which contains modest nonintegrable 2D terms, but also to extending superdiffusive behavior to larger 2D qubit lattices and other 2D materials.

Alagarsamy Manikandan, Keerthi Kumaran [ORNL]↗

DOE OSTI2024

Coherency-Aware Learning Control of Inverter-Dominated Grids: A Distributed Risk-Constrained Approach

Here, this letter investigates the importance of integrating the coherency knowledge for designing controllers to dampen sustained oscillations in wide-area power networks with significant penetration of inverter-interfaced resources. Coherency is a fundamental property of power systems, where time-scale separation in frequency dynamics leads to clustered behavior among generators of different groups. Large-scale penetration of inverter-driven low inertia resources replacing conventional synchronous generators (SGs) can lead to perturbation in the coherent partitioning; hence, integrating such information is of utmost importance for oscillation control designs. We present the coherency-aware design of a distributed output feedback-based reinforcement learning method that additionally incorporates risk constraints to capture the uncertainties related to net-load fluctuations. The use of domain-aware coherency information has produced improved training and oscillation performance than the coherency-agnostic control design, hence proving to be effective in controller design. Finally, we validated the proposed method with numerical experiments on the benchmark IEEE 68-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2026

Forensic characterization of surrogate nuclear explosion debris: radiochemical and spectroscopic strategies for method validation

Surrogate nuclear explosion debris (SNED) has emerged as a critical platform for advancing post-detonation nuclear forensic analysis in the absence of readily accessible historic materials. SNED enables controlled investigation and validation of analytical methodologies used to interrogate the chemical, isotopic, radiological, and microstructural signatures preserved in nuclear explosion debris. This review presents an integrated assessment of destructive and non-destructive analytical techniques commonly employed within decision-driven nuclear forensic workflows. Each technique is discussed individually while highlighting how it contributes to different stages of post-detonation analysis. Core methods – including gamma and alpha spectrometry, ICP-MS, TIMS, SIMS, SEM-EDS, XRF, LIBS, vibrational spectroscopy, and X-ray absorption spectroscopy – are critically evaluated with respect to forensic maturity, information content, and matrix limitations. Emphasis is placed on the role of SNED in benchmarking multi-modal workflows and identifying gaps in reproducing heterogeneity, fractionation, and radiation-driven evolution relevant to forensic attribution.

X-ray spectroscopic methods↗

NASA NTRS2003

Resource Management for Real-Time Adaptive Agents

Increased autonomy and automation in onboard flight systems offer numerous potential benefits, including cost reduction and greater flexibility. The existence of generic mechanisms for automation is critical for handling unanticipated science events and anomalies where limitations in traditional control software with fixed, predetermined algorithms can mean loss of science data and missed opportunities for observing important terrestrial events. We have developed such a mechanism by adding a Hierarchical Agent-based ReaLTime technology (HART) extension to our Dynamic Resource Management (DRM) middleware. Traditional DRM provides mechanisms to monitor the realtime performance of distributed applications and to move applications among processors to improve real-time performance. In the HART project we have designed and implemented a performance adaptation mechanism to improve reaktime performance. To use this mechanism, applications are developed that can run at various levels of quality. The DRM can choose a setting for the quality level of an application dynamically at run-time in order to manage satellite resource usage more effectively. A groundbased prototype of a satellite system that captures and processes images has also been developed as part of this project to be used as a benchmark for evaluating the resource management framework A significant enhancement of this generic mission-independent framework allows scientists to specify the utility, or "scientific benefit," of science observations under various conditions like cloud cover and compression method. The resource manager then uses these benefit tables to determine in redtime how to set the quality levels for applications to maximize overall system utility as defined by the scientists running the mission. We also show how maintenance functions llke health and safety data can be integrated into the utility framework. Once thls framework has been certified for missions and successfully flight tested it can be reused with little development overhead for other missions. In contrast, current space missions llke Swift manage similar types of resource trade -off completely with the scientific application code itself, and such code must be re-certified and tested for each mission even if a large portion of the code base is shared. This final report discusses some of the major issues motivating this research effort, provides a literature review of the related work, discusses the resource management framework and ground-based satellite system prototype that has been developed, indicates what work is yet to be performed, and provides a list of publications resulting from this work.

Welch, Lonnie↗

DOE OSTI2023

Polymer Compositions in Critical Experiments: Possibly Not What You Think

The Chlorine Worth Study (CWS) Experiment was performed at the National Criticality Experiments Research Center (NCERC) in December 2021. Its goal was to provide a validation benchmark experiment with chlorine and plutonium in the thermal neutron spectrum. This purpose is necessary to reduce the margin of subcriticality in aqueous chloride operations at the Los Alamos National Laboratory Plutonium Facility (PF-4). Reducing the margin of subcriticality will enable higher throughput, required to meet NNSA mission needs. The experiment used layers of plutonium plates, polyethylene (HDPE), aluminum, and polyvinyl chloride (PVC) or chlorinated polyvinyl chloride (CPVC). They were optimized to match plutonium-chloride solutions of 30 g/L, 300 g/L, and 600 g/L. Upon completion of the experiment, a International Criticality Safety Benchmark Evaluation Program (ICSBEP) report was immediately started. The compositions of all materials were assumed pure unless additional information was known (such as for the plutonium plates). During the benchmark analysis, the assumed CPVC composition was questioned. The follow-on work led to lessons learned on compositions in benchmarks. Materials, and specifically polymers, are often much more complex than a basic chemical formula. The CWS experiment is used as an example in this paper to document the lessons learned.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2018

Path integral Monte Carlo simulations of warm dense aluminum

Here, we perform first-principles path integral Monte Carlo (PIMC) and density functional theory molecular dynamics (DFT-MD) calculations to explore warm dense matter states of aluminum. Our equation of state (EOS) simulations cover a wide density-temperature range of $0.1-32.4$~g$\,$cm$^{-3}$ and $10^4-10^8$~K. Since PIMC and DFT-MD accurately treat effects of the atomic shell structure, we find two compression maxima along the principal Hugoniot curve attributed to K-shell and L-shell ionization. The results provide a benchmark for widely used EOS tables, such as SESAME, QEOS, and models based on Thomas-Fermi and average-atom techniques. A subsequent multi-shock analysis provides a quantitative assessment for how much heating occurs relative to an isentrope in multi-shock experiments. Finally, we compute heat capacity, pair-correlation functions, the electronic density of states, and Z to reveal the evolution of the plasma structure and ionization behavior.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2016

Diagnosing the Dynamics of Observed and Simulated Ecosystem Gross Primary Productivity with Time Causal Information Theory Quantifiers

Data analysis and model-data comparisons in the environmental sciences require diagnostic measures that quantify time series dynamics and structure, and are robust to noise in observational data. This paper investigates the temporal dynamics of environmental time series using measures quantifying their information content and complexity. The measures are used to classify natural processes on one hand, and to compare models with observations on the other. The present analysis focuses on the global carbon cycle as an area of research in which model-data integration and comparisons are key to improving our understanding of natural phenomena. We investigate the dynamics of observed and simulated time series of Gross Primary Productivity (GPP), a key variable in terrestrial ecosystems that quantifies ecosystem carbon uptake. However, the dynamics, patterns and magnitudes of GPP time series, both observed and simulated, vary substantially on different temporal and spatial scales. Here we demonstrate that information content and complexity, or Information Theory Quantifiers (ITQ) for short, serve as robust and efficient data-analytical and model benchmarking tools for evaluating the temporal structure and dynamical properties of simulated or observed time series at various spatial scales. At continental scale, we compare GPP time series simulated with two models and an observations-based product. This analysis reveals qualitative differences between model evaluation based on ITQ compared to traditional model performance metrics, indicating that good model performance in terms of absolute or relative error does not imply that the dynamics of the observations is captured well. Furthermore, we show, using an ensemble of site-scale measurements obtained from the FLUXNET archive in the Mediterranean, that model-data or model-model mismatches as indicated by ITQ can be attributed to and interpreted as differences in the temporal structure of the respective ecological time series. At global scale, our understanding of C fluxes relies on the use of consistently applied land models. Here, we use ITQ to evaluate model structure: The measures are largely insensitive to climatic scenarios, land use and atmospheric gas concentrations used to drive them, but clearly separate the structure of 13 different land models taken from the CMIP5 archive and an observations-based product. In conclusion, diagnostic measures of this kind provide data-analytical tools that distinguish different types of natural processes based solely on their dynamics, and are thus highly suitable for environmental science applications such as model structural diagnostics.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Performance Improvements of the Griffin Solvers in FY24

The Griffin code is a MOOSE-based reactor physics application jointly developed by Idaho National Laboratory and Argonne National Laboratory under the Department of Energy Office of Nuclear Energy Nuclear Energy Advanced Modeling and Simulation Program. This fiscal year, we have made significant efforts to improve the performance of transport solver options and cross-section generation for the efficient use of Griffin in advanced reactor applications. For the HFEM-PN solver, the residual evaluations of HFEM kernels were optimized by utilizing the pre- computed averaged cross sections for individual elements. Numerical integration involving the evaluation of basis functions at quadrature points was bypassed by facilitating precomputed element mass matrices for response matrices. Red-black iterations were improved by introducing a new generalized minimum residual based solver. The memory usage of response matrix storage was significantly reduced by applying basis function rotations on interfaces and calculating volumetric odd-parity moments on the fly. Additionally, the adjoint flux and transient calculation capabilities of the HFEM-PN solver were successfully implemented and verified using the TWIGL benchmark problem. For the DFEM-SN solver, memory footprint and computation time were significantly reduced by not treating angular flux vectors as the MOOSE nonlinear system vectors. Specifically for IQS, scalar adjoint weighting was introduced to further eliminate angular adjoint flux storage in the MOOSE auxiliary system. It was demonstrated through the three-dimensional Advanced Burner Test Reactor core problem that the memory usage for transient calculations with the IQS method was reduced by over 7.5× compared to before the optimizations. For the self-shielding application programming interface, a new double-heterogeneity treatment method, named the Bell Function-Based Analytic Two-Region Slowing Down Method, was developed to efficiently flux-volume homogenize TRISO particles with the matrix. Additionally, optimizations were made to hyper- fine group (HFG) slowing down calculations by pretabulating collision probability coefficients and grouping isotopes, significantly reducing the computational time for calculating scattering sources per HFG. Lastly, the pin power reconstruction module was extended to account for temporal behavior in a microreactor analysis problem, specifically for a control drum transient. Verification tests for each of these improvements demonstrated significant performance enhancements and memory reduction.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS2000

Benchmark Testing of the Largest Titanium Aluminide Sheet Subelement Conducted

To evaluate wrought titanium aluminide (gamma TiAl) as a viable candidate material for the High-Speed Civil Transport (HSCT) exhaust nozzle, an international team led by the NASA Glenn Research Center at Lewis Field successfully fabricated and tested the largest gamma TiAl sheet structure ever manufactured. The gamma TiAl sheet structure, a 56-percent subscale divergent flap subelement, was fabricated for benchmark testing in three-point bending. Overall, the subelement was 84-cm (33-in.) long by 13-cm (5-in.) wide by 8-cm (3-in.) deep. Incorporated into the subelement were features that might be used in the fabrication of a full-scale divergent flap. These features include the use of: (1) gamma TiAl shear clips to join together sections of corrugations, (2) multiple gamma TiAl face sheets, (3) double hot-formed gamma TiAl corrugations, and (4) brazed joints. The structural integrity of the gamma TiAl sheet subelement was evaluated by conducting a room-temperature three-point static bend test.

Bartolotta, Paul A.↗

NASA NTRS2005

Implementation of Programmatic Quality and the Impact on Safety

The purpose of this paper is to discuss the implementation of a programmatic quality assurance discipline within the International Space Station Program and the resulting impact on safety. NASA culture has continued to stress safety at the expense of quality when both are extremely important and both can equally influence the success or failure of a Program or Mission. Although safety was heavily criticized in the media after Col~imbiaa, strong case can be made that it was the failure of quality processes and quality assurance in all processes that eventually led to the Columbia accident. Consequently, it is possible to have good quality processes without safety, but it is impossible to have good safety processes without quality. The ISS Program quality assurance function was analyzed as representative of the long-term manned missions that are consistent with the President s Vision for Space Exploration. Background topics are as follows: The quality assurance organizational structure within the ISS Program and the interrelationships between various internal and external organizations. ISS Program quality roles and responsibilities with respect to internal Program Offices and other external organizations such as the Shuttle Program, JSC Directorates, NASA Headquarters, NASA Contractors, other NASA Centers, and International Partner/participants will be addressed. A detailed analysis of implemented quality assurance responsibilities and functions with respect to NASA Headquarters, the JSC S&MA Directorate, and the ISS Program will be presented. Discussions topics are as follows: A comparison of quality and safety resources in terms of staffing, training, experience, and certifications. A benchmark assessment of the lessons learned from the Columbia Accident Investigation (CAB) Report (and follow-up reports and assessments), NASA Benchmarking, and traditional quality assurance activities against ISS quality procedures and practices. The lack of a coherent operational and sustaining quality assurance strategy for long-term manned space flight. An analysis of the ISS waiver processes and the Problem Reporting and Corrective Action (PRACA) process implemented as quality functions. Impact of current ISS Program procedures and practices with regards to operational safety and risk A discussion regarding a "defense-in-depth" approach to quality functions will be provided to address the issue of "integration vs independence" with respect to the roles of Programs, NASA Centers, and NASA Headquarters. Generic recommendations are offered to address the inadequacies identified in the implementation of ISS quality assurance. A reassessment by the NASA community regarding the importance of a "quality culture" as a component within a larger "safety culture" will generate a more effective and value-added functionality that will ultimately enhance safety.

Huls, Dale Thomas↗

NASA NTRS2015

Human Research Program Advanced Exercise Concepts (AEC) Overview

Exercise countermeasures provide benefits that are crucial for successful human spaceflight, to mitigate the spaceflight physiological deconditioning which occurs during exposure to microgravity. The NASA Human Research Program (HRP) within the Human Exploration and Operations Mission Directorate (HEOMD) is managing next generation Advanced Exercise Concepts (AEC) requirements development and candidate technology maturation to Technology Readiness Level (TRL) 7 (ground prototyping and flight demonstration) for all exploration mission profiles from Multi Purpose Crew Vehicle (MPCV) Exploration Missions (up to 21 day duration) to Mars Transit (up to 1000 day duration) missions. These validated and optimized exercise countermeasures systems will be provided to the ISS Program and MPCV Program for subsequent flight development and operations. The International Space Station (ISS) currently has three major pieces of operational exercise countermeasures hardware: the Advanced Resistive Exercise Device (ARED), the second-generation (T2) treadmill, and the cycle ergometer with vibration isolation system (CEVIS). This suite of exercise countermeasures hardware serves as a benchmark and is a vast improvement over previous generations of countermeasures hardware, providing both aerobic and resistive exercise for the crew. However, vehicle and resource constraints for future exploration missions beyond low Earth orbit will require that the exercise countermeasures hardware mass, volume, and power be minimized, while preserving the current ISS capabilities or even enhancing these exercise capabilities directed at mission specific physiological functional performance and medical standards requirements. Further, mission-specific considerations such as preservation of sensorimotor function, autonomous and adaptable operation, integration with medical data systems, rehabilitation, and in-flight monitoring and feedback are being developed for integration with the exercise countermeasures systems. Numerous technologies have been considered and evaluated against HRP-approved functional device requirements for these extreme mission profiles, and include wearable sensors, exoskeletons, flywheel, pneumatic, and closed-loop microprocessor controlled motor driven systems. Each technology has unique advantages and disadvantages. The Advanced Exercise Concepts project oversees development of candidate next generation exercise countermeasures hardware, performs trade studies of current and state of the art exercise technologies, manages and supports candidate systems physiological evaluations with human test subjects on the ground, in flight analogs and flight. The near term goal is evaluation of candidate systems in flight, culminating in an integrated candidate next generation exercise countermeasures suite on the ISS which coalesces research findings from HRP disciplines in the areas of exercise performance for muscle, bone, cardiovascular, sensorimotor, behavioral health, and nutrition for optimal benefit to the crew.

Countermeasures↗

NASA NTRS2021

A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion

Dense time-series remote sensing data with detailed spatial information are highly desired for the monitoring of dynamic earth systems. Due to the sensor tradeoff, most remote sensing systems cannot provide images with both high spatial and temporal resolutions. Spatiotemporal image fusion models provide a feasible solution to generate such a type of satellite imagery, yet existing fusion methods are limited in predicting rapid and/or transient phenological changes. Additionally, a systematic approach to assessing and understanding how varying levels of temporal phenological changes affect fusion results is lacking in spatiotemporal fusion research. The objective of this study is to develop an innovative hybrid deep learning model that can effectively and robustly fuse the satellite imagery of various spatial and temporal resolutions. The proposed model integrates two types of network models: super-resolution convolutional neural network (SRCNN) and long short-term memory (LSTM). SRCNN can enhance the coarse images by restoring degraded spatial details, while LSTM can learn and extract the temporal changing patterns from the time-series images. To systematically assess the effects of varying levels of phenological changes, we identify image phenological transition dates and design three temporal phenological change scenarios representing rapid, moderate, and minimal phenological changes. The hybrid deep learning model, alongside three benchmark fusion models, is assessed in different scenarios of phenological changes. Results indicate the hybrid deep learning model yields significantly better results when rapid or moderate phenological changes are present. It holds great potential in generating high-quality time-series datasets of both high spatial and temporal resolutions, which can further benefit terrestrial system dynamic studies. The innovative approach to understanding phenological changes’ effect will help us better comprehend the strengths and weaknesses of current and future fusion models.

spatiotemporal fusion↗

DOE OSTI2022

Proposed Methodology for Evaluating and Validating TSLs [Slides]

This presentation begins by providing an overview of the research and discusses recent efforts in validating thermal neutron scattering cross sections, including differential cross section measurements at ORNL for evaluation and validation, total cross section measurements at RPI, pulsed-neutron die-away experiments at LLNL, and integral criticality experiments by LLNL. The presentation also proposes a methodology not only for validating thermal scattering files that utilizes all available experimental data, but also for evaluating new libraries as demonstrated on polystyrene. In conclusion, polystyrene evaluation was conducted using multiple experiments, including differential and integral. The proposed methodology has been shown to improve neutron transport and files have been submitted to NNDC for inclusion in ENDF/B-VIII.1 release. MCT-012 & PCM-002 has been relatively insensitive to changes in $\mathcal{S}(α,β)$. The research found that differences in differential results don’t always propagate to differences in integral results, hence why evaluation of differential data important. Future work on the project is also discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Why Lady Godiva Should be Replaced as the Default Validation Experiment for U-235 Nuclear Data

For the past seven decades the Lady Godiva benchmark (HEU-MET-FAST-001) has been the primary experiment used for 235 U nuclear data validation. The papers associated with large nuclear data library releases such as ENDF/B-VII.1 and ENDF/B-VIII.0 refer to it frequently. The reasons why HEU-MET-FAST-001 has been used as the primary validation experiment for 235 U nuclear data validation will be discussed in the following section. However, as discussed among the benchmark community, the standards associated with International Criticality Safety Benchmark Evaluation Project (ICSBEP) evaluations have changed throughout time; this is subject of the OECD/NEA WPEC (Working Party on Nuclear Criticality Safety) Subgroup 8. HEU-MET-FAST-001 is an older benchmark (issued during the inaugural year of ICSBEP in 1996 with only minor revisions occurring since then) and (along with many of the other benchmarks from this era) does not meet the standards for a modern benchmark. This work explores why HEU-MET-FAST-001 is useful for 235 U nuclear data validation and discusses other alternative validation experiments.

07 ISOTOPE AND RADIATION SOURCES↗

NASA NTRSDate not supplied

Objective Structured Clinical Evaluation (OSCE) of an Artificial Intelligence (AI) Clinical Decision Support System (CDSS) Tool

BACKGROUND Objective Structured Clinical Evaluations (OSCEs) have long been established as a robust methodology for summative assessment of clinical skills and decision-making during medical education. The recent integration of Artificial Intelligence (AI) into clinical decision-making processes has prompted the need for novel evaluation frameworks to assess the efficacy and reliability of AI clinical decision support system (CDSS) tools. This abstract outlines the process of quantitatively evaluating a novel CDSS (“Doc in a Box” Google 2024) trained on curated medical spaceflight data in the psychomotor domain as it interfaces with a human volunteer acting as the crew medical officer (CMO). PURPOSE The AI CDSS under review was developed as part of the Lunar Command and Control Interoperability (LuCCI) project, which is intended to address a gap in how Lunar Surface Systems (LSS) would interoperate across multiple programs, commercial partners, and international partners. The project objective is to define, prototype, integrate, and evaluate an interoperable lunar command, control, data, and software reference architecture to enable autonomy and informatics capability through common standards across LSS. A multi-modal AI-based CDSS compatible with Federated LSS will assist clinicians in diagnosing and managing complex medical conditions by providing evidence-based recommendations through predictive analytics. Given the critical role of decision-support as NASA continues to evolve its Earth-independent medical operations (EIMO), it is imperative to ensure that such AI tools perform reliably and align with clinical standards during progressive lunar and Martian exploration class missions. METHODS The OSCE framework, traditionally used for evaluating human clinicians, was adapted to assess the AI tool's decision-making capabilities in simulated clinical scenarios. In this adapted OSCE, the AI CDSS was tested across a series of structured clinical scenarios designed to mimic real-life spaceflight patient cases. These scenarios included a range of conditions and complexities, allowing for comprehensive assessment of the tool's performance. Key evaluation metrics included accuracy of diagnosis, timeliness of decision-making, and appropriate recommendations for therapies. The OSCE was scored by human physician evaluators who assessed the AI's recommendations in comparison with expert clinicians' medical decision making to ensure alignment with best practices and the standard of care. RESULTS Preliminary results indicate that the AI CDSS demonstrated high accuracy in diagnostic recommendations and decision support across various scenarios. However, certain limitations were noted, such as occasional discrepancies in handling complex or nuanced cases that required a more contextual understanding. Additionally, the tool scored higher on the diagnostic portion of the rubric, with lower scores in the therapeutic recommendations. These findings highlight the importance of continuous refinement and validation of AI tools through rigorous evaluation frameworks like the OSCE. The adaptation of OSCEs for AI tools presents several advantages, including a structured and reproducible approach to evaluation, the ability to test AI systems in diverse clinical scenarios, and the opportunity to benchmark AI performance against established clinical standards to permit charting of future progress as aerospace medicine evolves as a discipline. Remaining challenges include ensuring that these evaluations capture the full spectrum of clinical decision-making scenarios that will be confronted by CMOs during missions and adequately reflecting real-world variability of the austere spaceflight environment. CONCLUSION Employing OSCEs to evaluate AI clinical decision support tools offers a promising approach to validating their clinical utility and efficacy. This methodology not only provides insights into the tool's performance but also fosters ongoing improvement and alignment with standard of care practices. Future research should focus on refining these evaluation processes and addressing limitations to enhance the integration of AI tools in clinical spaceflight settings. REFERENCES Scott S, Hearns V, Barker MA. Testing Clinical Skills: A Look at the OSCE and USMLE Clinical Skills Exams. S D Med. 2019 Oct;72(10):451-453. Majumder MAA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Adv Med Educ Pract. 2019 Jun 5;10:387-397. Karam VY, Park YS, Tekian A, Youssef N. Evaluating the validity evidence of an OSCE: results from a new medical school. BMC Med Educ. 2018 Dec 20;18(1):313.

Ariana M Nelson↗