SEARCH · Search NASA
Results for “Benchmark data”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Thermodynamic Modeling of Intrinsic Defects in MnBi₂Te₄
This repository contains the computational data supporting the manuscript titled “The critical role of intrinsic defects and many-body interactions on the stability of MnBi₂Te₄.” It includes: 1. DFT data generated using VASP, used for training and benchmarking electronic structure models. 2. Quantum Monte Carlo (QMC) data produced with QMCPACK, used to apply many-body corrections and validate the electronic and magnetic properties of MnBi₂Te₄. 3. Relevant scripts used to run, analyze, and process the calculations, enabling reproducibility and transparency of the workflows.
Benchmark of the Chlorine Worth Study Experiments in Support of Chlorine Nuclear Data Validation for Nuclear Criticality Safety
The Chlorine Worth Study (CWS) was a critical experiment to address an urgent need for thermal chlorine nuclear data validation in plutonium systems. This urgent need is tied directly to plutonium recycle and recovery operations in the plutonium facility at Los Alamos National Laboratory, where exceptionally conservative criticality safety limits are used because no credit is taken for the neutron capture by chlorine. The experiment used weapons-grade plutonium metal plates clad in stainless steel, known as the PANN (plutonium aluminum no nickel) ZPPR (zero power physics reactor) plates. The plutonium was reflected and moderated by high-density polyethylene and included combinations of polyvinyl chloride (PVC) and chlorinated polyvinyl chloride (CPVC) as absorbers. The experiment and benchmark included three configurations mimicking 30 g 239 Pu/L plutonium, 300 g 239 Pu/L plutonium, and 600 g 239 Pu/L plutonium in an aqueous chloride solution. Uncertainties in the benchmark included five broad categories: (1) criticality measurement, (2) mass and density, (3) dimensions, (4) material compositions, and (5) positioning. The largest contribution to the overall uncertainties for all three cases came from the material compositions, in particular the PVC and CPVC absorber compositions. A detailed model was created to be a near match (that is within expectations of transport code users) and a simplified model was created to minimize offset dimensions and expedite modeling for code validation. Sample calculations were completed in MCNP6.3 with ENDF/B-VIII.0 and ENDF/B-VII.1 nuclear data. For the detailed and simplified models, the average difference between the computed and experimental k eff was 951 pcm. CWS will serve as the key validation experiment for nuclear criticality safety in support of aqueous chloride operations. The sensitivity to the chlorine capture cross section is orders of magnitude greater than other existing benchmarks. The current limits, as defined by nuclear criticality safety, are 520 g Pu per batch, i.e. the minimum critical mass of the Pu solution infinitely reflected by water [Criticality Handbook: Volume II, (1969)]. This extremely conservative critical mass limit does not credit any neutron capture by chlorine (in particular neutron capture by 35 Cl) and greatly impedes the throughput required for current and future operations.
Development of an Improved RELAP5-3D Model for the High Temperature Test Facility
High-temperature gas-cooled reactors (HTGRs) are rapidly approaching deployment. Confidence in transient analysis of these systems for design, optimization, and licensing calculations requires modeling and simulation tools that have been validated against data relevant to HTGR conditions. The High Temperature Test Facility (HTTF) is an integral effects thermal hydraulics test facility for prismatic HTGRs. In spring and summer of 2019, HTTF was used for a series of experiments that now serve as the basis for the OECD/NEA Thermal Hydraulic Code Validation Benchmark for High Temperature Gas-Cooled Reactors using HTTF Data (HTGR T/H Benchmark). This benchmark contains problems for systems code, computational fluid dynamics (CFD), and coupled systems code/CFD modeling representing lower plenum mixing and both the depressurized and pressurized conduction cooldown (DCC and PCC respectively) transients. Benchmark problems include exercises for code-to-code and code-to-data comparisons as well as an exercise for error scaling between HTTF and the Modular High Temperature Gas-Cooled Reactor, which serves as the basis for the HTTF design. Previous analysis as part of the HTGR T/H benchmark used a RELAP5-3D model developed at Idaho National Laboratory (INL) and demonstrated an ability to reproduce trends in the measured data but difficulties reproducing experimental values within their uncertainty. These difficulties were largely attributed to assumptions made during the development of the initial RELAP5-3D model, which predated the HTTF experiments. A significant cause of difficulty reproducing the measured temperatures may be the radial nodalization of the previous RELAP5-3D model. The new model provides a finer nodalization to assess the impact of radial nodalization and allows for asymmetric heating within the core, which was a feature of multiple HTTF experiments. In this paper, we present the new RELAP5-3D model of HTTF. In addition to describing the new model, this paper compares the new and old models and provides results for a full-power steady state, a DCC, and a PCC in HTTF. These analyses are based on the code-to-code comparison exercises for the DCC and PCC problems of the HTGR T/H benchmark. We present the results of these exercises from the new model and compare them to the results of the old model.
Integrated Energy-Water Data for Cross-Sector Resilience
This white paper focuses on the “energy-for-water” domain, addressing the urgent need for integrated, empirical data to support regional management, benchmarking, and research on improving efficiency and developing technologies for water and wastewater management systems. The costs and energy required for the supply, treatment, and distribution of water and wastewater lack a standard data collection mechanism and centralized database or storage infrastructure, limiting data-driven decision-making across interdependent infrastructure systems.
Correlated Electromagnetic Levitation Actuator: A Reaction Sphere Based Attitude Control System
To address problems experienced by current reaction wheels and control moment gyroscopebased attitude control systems (ACS), researchers at NASA’s Marshall Space Flight Center have begun developing a reaction sphere actuator based on correlated electromagnetic levitation that will be immune to destructive bearing friction, momentum saturation, and gimbal lock. The Correlated Electromagnetic Levitation Actuator (CELA) advances the state of the art of reaction sphere ACSs by employing the concept of correlated magnetics. It is a frictionless, direct-drive reaction sphere that harnesses a unique technology with an array of applications across multiple disciplines. Correlated electromagnets function in a manner that is analogous to a matched filter; the convolution of two signals is peaked at the index representing the greatest match. For CELA, the signals are the patterns of magnetic flux density as a function of position. The magnitude of the convolution equates to an attractive or repulsive force, and these forces can be azimuthal or radial. The development of CELA is based in four distinct disciplines: Advanced Manufacturing, Prototype Development, Electromagnetic Modeling, and Controls. We are developing novel manufacturing techniques required to build arrays of permanent and electromagnet dipoles on curved surfaces. To print the permanent magnetic array, we have developed a probe with pyramidal magnets that will reside on a robotic arm to induce localized magnetic fields on a surface. The probe also includes the ability to erase dipole patterns from a permanent magnet by heating the surface to its Curie temperature. A number of test articles and prototypes have been developed using additive manufacturing methods. These prototypes have included hemispherical motors to test the drive algorithm, and a levitation test bed that demonstrates a magnetic bearing method based on attractive magnetic forces and ratiometric Hall effect sensors. We developed an array of electromagnetic dipoles on a printed circuit board (PCB) with individual H-bridges controlling each coil. This device created various flux density patterns and we measured their magnetic fields using a custom Hall effect 3-D probe and a LabVIEW virtual instrument. These data will serve as a benchmark for characterizing the accuracy of future models. Current work is focused on modeling the magnetic fields of our prototype arrays using COMSOL Finite Element Analysis and verifying the model against our test data. Accurate modeling will allow us to quickly test new patterns of electromagnets and their macro behavior. Eventually, the magnetic field models will be implemented in our controls simulations to facilitate precise control of the reaction sphere. Initial model results agree with field measurements to within 1 G (5% of measured flux density). Currently, we are testing different material properties of the electromagnets and their magnetic fields and thermal effects. These results will be used to refine the design of the electromagnetic dipoles. Our control efforts have centered on developing commutation, levitation, and field pattern shaping hardware in the form of breadboards and PCBs with software running on a local microcontroller. In addition, our partners developed MATLAB Simulink models to demonstrate a PID controller thatmitigates disturbance forces resulting from the interaction of drive and levitation magnetics. Finally, we have designed a three-axis test stand that will be used in future work to demonstrate CELA’s orientation control capability.
USM3D Predictions of Supersonic Nozzle Flow
This study focused on the NASA Tetrahedral Unstructured Software System CFD code (USM3D) capability to predict supersonic plume flow. Previous studies, published in 2004 and 2009, investigated USM3D's results versus historical experimental data. This current study continued that comparison however focusing on the use of the volume souring to capture the shear layers and internal shock structure of the plume. This study was conducted using two benchmark axisymmetric supersonic jet experimental data sets. The study showed that with the use of volume sourcing, USM3D was able to capture and model a jet plume's shear layer and internal shock structure.
Correlated Electromagnetic Levitation Actuator: A Reaction Sphere-Based Attitude Control System
To address problems experienced by current reaction wheels and control moment gyroscopebased attitude control systems (ACS), researchers at National Aeronautics and Space Administration’s (NASA’s) Marshall Space Flight Center (MSFC) have begun developing a reaction sphere actuator based on correlated electromagnetic levitation that will be immune to destructive bearing friction, momentum saturation, and gimbal lock. The Correlated Electromagnetic Levitation Actuator (CELA) advances the state of the art of reaction sphere ACSs by employing the concept of correlated magnetics. It is a frictionless, direct-drive reaction sphere that harnesses a unique technology with an array of applications across multiple disciplines. Correlated electromagnets function in a manner that is analogous to a matched filter; the convolution of two signals is peaked at the index representing the greatest match. For CELA, the signals are the patterns of magnetic flux density as a function of position. The magnitude of the convolution equates to an attractive or repulsive force, and these forces can be azimuthal or radial. The development of CELA is based in four distinct disciplines: Advanced Manufacturing, Prototype Development, Electromagnetic Modeling, and Controls. We are developing novel manufacturing techniques required to build arrays of permanent and electromagnet dipoles on curved surfaces. To print the permanent magnetic array, we have developed a probe with pyramidal magnets that will reside on a robotic arm to induce localized magnetic fields on a surface. We have also used high temperature ovens to erase dipole patterns from a permanent magnet by heating the surface to its Curie temperature. A number of test articles and prototypes have been developed using additive manufacturing methods. These prototypes have included hemispherical motors to test the drive algorithm, and a levitation test bed that demonstrates a magnetic bearing method based on attractive magnetic forces and ratiometric Hall effect sensors. We developed an array of electromagnetic dipoles on a printed circuit board (PCB) with individual H-bridges controlling each coil. This device created various flux density patterns and we measured their magnetic fields using a custom Hall effect 3-D probe and a LabVIEW virtual instrument. These data will serve as a benchmark for characterizing the accuracy of future models. Current work is focused on modeling the magnetic fields of our prototype arrays using COMSOL finite element analysis (FEA) and verifying the model against our test data. Accurate modeling will allow us to quickly test new patterns of electromagnets and their macro behavior. Eventually, the magnetic field models will be implemented in our controls simulations to facilitate precise control of the reaction sphere. Initial model results agree with field measurements to within 1 G (5% of measured flux density). Currently, we are testing different material properties of the electromagnets and their magnetic fields and thermal effects. These results will be used to refine the design of the electromagnetic dipoles. Our control efforts have centered on developing commutation, levitation, and field pattern shaping hardware in the form of breadboards and PCBs with software running on a local microcontroller. In addition, our partners developed MATLAB Simulink models to demonstrate a PID controller that mitigates disturbance forces resulting from the interaction of drive and levitation magnetics. Finally, we have designed a three-axis test stand that will be used in future work to demonstrate CELA’s orientation control capability.
Unsupervised Anomaly Detection in High-Dimensional Flight Data Using Convolutional Variational Auto-Encoder
The modern National Airspace System (NAS) is an extremely safe system. The industry has experienced a steady decrease in fatalities over the years. This can be contributed to both improved flight critical systems with redundant hardware and software protections as well as an increased focus on active monitoring and response to real time and historically identified vulnerabilities by implementing more resilient procedures and protocols. The main practice for identifying vulnerabilities in operations leverages domain expertise using knowledge about how the system should behave with the expected tolerances to known safety margins. This approach works well when the system has a well-defined operating condition. However, the operations in the NAS can be highly complex with various nuances that render it difficult to clearly pre-define all known safety vulnerabilities. With the advancement of data science and machine learning techniques, the potential to automatically identify emerging vulnerabilities in the observed operations has become more practical in recent years. The state-of-the-art anomaly detection approaches in aerospace data usually rely on supervised or semi-supervised learning. However, in many real-world problems such as flight safety creating labels for the data requires huge amount of efforts and is largely expensive. As a result, in this article, we develop a Convolutional Variational Auto-Encoder (CVAE), an unsupervised learning approach for anomaly detection in high-dimensional heterogeneous time-series data. We validate performance of CVAE compared to the state-of-the-art supervised learning approach (as an upper bound) as well as an supervised clustering based on K-Means (as a lower bound) on Yahoo!'s benchmark time series anomaly detection data. Finally, we showcase performance of CVAE on a case study of identifying anomalies in the first 60 seconds of commercial flights' take-offs using Flight Operational Quality Assurance (FOQA) data.
JARVIS-Leaderboard: a large scale benchmark of materials design methods
Abstract Lack of rigorous reproducibility and validation are significant hurdles for scientific development across many fields. Materials science, in particular, encompasses a variety of experimental and theoretical approaches that require careful benchmarking. Leaderboard efforts have been developed previously to mitigate these issues. However, a comprehensive comparison and benchmarking on an integrated platform with multiple data modalities with perfect and defect materials data is still lacking. This work introduces JARVIS-Leaderboard, an open-source and community-driven platform that facilitates benchmarking and enhances reproducibility. The platform allows users to set up benchmarks with custom tasks and enables contributions in the form of dataset, code, and meta-data submissions. We cover the following materials design categories: Artificial Intelligence (AI), Electronic Structure (ES), Force-fields (FF), Quantum Computation (QC), and Experiments (EXP). For AI, we cover several types of input data, including atomic structures, atomistic images, spectra, and text. For ES, we consider multiple ES approaches, software packages, pseudopotentials, materials, and properties, comparing results to experiment. For FF, we compare multiple approaches for material property predictions. For QC, we benchmark Hamiltonian simulations using various quantum algorithms and circuits. Finally, for experiments, we use the inter-laboratory approach to establish benchmarks. There are 1281 contributions to 274 benchmarks using 152 methods with more than 8 million data points, and the leaderboard is continuously expanding. The JARVIS-Leaderboard is available at the website: https://pages.nist.gov/jarvis_leaderboard/
Demography, dynamics and data: building confidence for simulating changes in the world's forests
Vegetation demographic models (VDMs) are advanced tools for simulating forest responses to climate and land-use changes, and are essential for projecting carbon cycling and large-scale forest management strategies. Despite their increasing incorporation into Earth System Models, VDMs differ in their demographic assumptions, with no prior quantitative comparison of their performance. We benchmarked nine VDMs against observational data from boreal, temperate and tropical sites, assessing their accuracy in predicting tree growth, carbon turnover, biomass stocks and size distributions. Models were simulated under consistent climate conditions with postdisturbance recovery monitored for at least 420 yr. Postdisturbance carbon recovery trajectories showed significant variability while remaining within observational ranges. Initial regrowth rates varied substantially (0.03-0.60, 0.18-0.70 and 0.35-1.10 kgCm-2 yr-1 for boreal, temperate and tropical sites, respectively), influenced by each model's initial forest state. Models captured mature forest carbon content but showed compensating effects between overestimated growth and underestimated mortality rates. This first multi-model benchmarking identifies growth and mortality rates as critical calibration targets and highlights the need to refine postdisturbance establishment conditions for model development. We outline specific benchmarking variables needed to improve predictions of forest responses to environmental change.
FY25 Mid-Year Report: FNCL Enhancements Implementation
During the first half of FY25 the FNCL team has made consistent progress toward the completion of our project goals. The FNCL prototype panel design has been successfully applied to a fully instrumented 3-panel system which is actively under construction. The FNCL Demonstrator System contains solid scintillators instrumented with SiPMs, which operate on an updated CAEN digitizer, requires no high-voltage, and has a smaller overall footprint. The onboard software will include the LLNL-developed GMM-PSD signal processing. Later this year the system will be experimentally tested alongside the baseline FNCL instrument at LLNLs ISSA facility. In addition to a full systems test, the performance of a DD generator for active interrogation measurements compared to the standard AmLi source will be established for both systems. The data collected at the ISSA facility will be used to experimentally validate the FNCL-Fast Isotopic Fuel Assay’s (FIFA) capability to measure U-235 loading and to predict gadolinium poison content with passive interrogation. The FNCL-FIFA modal was benchmarked with simulation-based data and a user-friendly GUI was added earlier this year. Three separate codes have been submitted to the LLNL ESW system for review prior to their transfers. These include the Predictive Modeling Response toolkit, GMM-PSD firmware beta version, and the FNCL-FIFA analysis package with GUI and user documentation.
Overview for the reanalysis of Mariner 9 UV spectrometer data for ozone, cloud, and dust abundances, and their interaction over climate timescales
Mariner 9 UV spectrometer data were reinverted for the ozone abundance, cloud abundance, dust abundance, and polar-cap albedo. The original reduction of the spectra ignored the presence of atmospheric dust and clouds, even though their abundance is substantial and can mask appreciable amounts of ozone if not accounted for (Lindner, 1988). The Mariner 9 ozone data has been used as a benchmark in all theoretical models of atmospheric composition, escape, and photochemistry. A second objective is to examine the data for the interrelationship of the ozone cycle, dust cycle, and cloud cycle, on an annual, inter-annual, and climatic basis, testing predictions by Lindner (1988). This also has implications for many terrestrial ozone studies, such as the ozone hole, acid rain, and ozone-smog. A third objective is to evaluate the efficacy of the reflectance spectroscopy technique at retrieving the ozone abundance on Mars. This would be useful for planning ozone observations on future Mars missions or the terrestrial troposphere.
Unsupervised Anomaly Detection in High-Dimensional Flight Data Using Convolutional Variational Auto-Encoder
The modern National Airspace System (NAS) is an extremely safe system and the aviation industry has experienced a steady decrease in fatalities over the years. This can be attributed to both improved flight critical systems with redundant hardware and software protections, as well as an increased focus on active monitoring and response to real time and historically identified vulnerabilities by implementing more resilient procedures and protocols. The main approach for identifying vulnerabilities in operations leverages domain expertise using knowledge about how the system should behave within the expected tolerances to known safety margins. This approach works well when the system has a well-defined operating condition. However, the operations in the NAS can be highly complex with various nuances that render it difficult to clearly pre-define all known safety vulnerabilities. With the advancement of data science and machine learning techniques, the potential to automatically identify emerging vulnerabilities in the observed operations has become more practical in recent years. The state-of-the-art anomaly detection approaches in aerospace data usually rely on supervised or semi-supervised learning. However, in many real-world problems such as flight safety, creating labels for the data requires huge amount of effort and is largely impractical. To address this challenge, we developed a Convolutional Variational Auto-Encoder (CVAE), which is an unsupervised learning approach for anomaly detection in high-dimensional heterogeneous time-series data. We validate performance of CVAE compared to the state-of-the-art supervised learning approach as well as unsupervised clustering-based approach using KMeans++ and kernel-based approach using One-Class Support Vector Machine (OC-SVM) on Yahoo!'s benchmark time series anomaly detection data. Finally, we showcase performance of CVAE on a case study of identifying anomalies in the first 60 seconds of commercial flights' take-offs using Flight Operational Quality Assurance (FOQA) data.
Experimental Covariance Determination for Critical Integral Experiments
Integral benchmarks for criticality safety and nuclear data validation require expensive uncertainty quantification studies. In general, uncertainty quantification techniques ignore correlations between experiments and shared parts. Experiments, such as the TEX (Thermal/Epithermal eXperiments) campaigns, consist of many shared parts, such as the ‘Jemima’ HEU fuel plates, which create a strong correlation in their uncertainties. While these correlations are known to exist, they are often not estimated due to the complexity of such calculations. This paper describes an intuitive method of determining the covariance for each of the experimental components, providing a correlation for each family of parts across the multiple cases examined within a benchmark. A proof-of-principle study using the TEX-HEU experimental campaign was performed and verified that the correlations can be calculated with information commonly found in the ICSBEP (International Criticality Safety Benchmark Evaluation Project) benchmarks. This study showed that the introduction of model and experimental covariances reduces the χ 2 per degree of freedom from 2.203 to 1.179, indicating that the omission causes overly pessimistic bias quantifications. This technique can be seamlessly integrated to current benchmark evaluations as well as reevaluations of legacy benchmarks.
Distribution of gas in the inner comae of comets
In order to understand the physical and chemical processes which produce the observed spatial morphology of the cometary coma, it is necessary to analyze observational data with physically meaningful models. Thus, a coupled program of theoretical modeling and complementary observational data analysis was undertaken regarding the spatial distributions of neutral gases in the coma. More, specifically, the particular topics of interest are: (1) the theoretical modeling of the nonequilibrium dynamics of the inner coma with emphasis on the region of the coma from the transition from collisional fluid flow out to the free-flow region and on observable conditions in the coma (i.e., density, outflow speed, and temperature); and (2) the model analysis of an important set of long-slit CCD spectra of comets. The side-by-side development of models along with the observation and analysis of data is an important and integral part of this project. The scientific community has in hand valuable observational and in situ data regarding one comet, Halley. It is important to use Halley as the benchmark by which other remotely observed comet data can be understood. Therefore, the self-consistant analysis of data with appropriate models is of the utmost importance. The data analysis work includes the analysis of the spatial profiles of (OI), NH2, CN, and C2.
Azimuthal correlation anisotropies in p + p collisions simulated using Pythia
Stimulated by a keen interest in possible collective behavior in high-energy proton-proton and proton-nucleus collisions, we study two-particle angular correlations in pseudorapidity and azimuthal differences in simulated p + p interactions using the Pythia 8 event generator. Multi-parton interactions and color connection are included in these simulations, which have been perceived to produce collectivity in final-state particles. Meanwhile, contributions from genuine few-body nonflow correlations, not of collective flow behavior, are known to be severe in these small-system collisions. We present our Pythia correlation studies pedagogically and report azimuthal harmonic anisotropies analyzed using several methods. We observe anisotropies in these Pythia simulated events qualitatively and semi-quantitatively, similar to experimental data. Furthermore, our findings highlight the delicate nature of azimuthal anisotropies in small-system collisions and provide a benchmark that can aid in improving data analysis and interpreting experimental measurements in small-system collisions.
Constraining the impact of chlorine as a neutron absorber in next-gen fast reactor designs
The role of chlorine as a neutron poison and as a seed for producing radioactive waste in nuclear systems has driven a renewed interest to improve its nuclear data uncertainties. Additionally, basic and applied science programs that use CLYC (Cs 2 LiYCl 6 :Ce) detectors for neutron spectroscopy and monitoring are also very sensitive to any change in chlorine nuclear data for simulations of the detector response. In this work, sensitivities relevant for these different applications are addressed through simulations of the efficiency of CLYC detectors in a fast fission spectrum when applying new chlorine nuclear data as input. These simulations are validated by an experimental measurement using CLYC detectors coupled to an ionization chamber loaded with a 252 Cf spontaneous fission source. The results are then used to obtain the first reliable direct measurement of the 35 Cl(n,p 0 ) and summed Cl(n,p+n,α) fission spectrum average cross sections, found to be 54.7(32) and 105.0(98) mb, respectively. The results are within uncertainty of calculated fission spectrum averaged cross sections based on recently re-evaluated chlorine nuclear data, which confirm recent impact studies performed for the Molten Chloride Reactor Experiment. Meanwhile, there currently exists only one published criticality benchmark experiment that is sufficiently sensitive to chlorine nuclear data. Discrepancies are found with this set of criticality safety benchmarks, which are more sensitive to thermal and epithermal neutron energies than the energies, above 100 keV, tested in this current work. Hence, there is still a need to re-evaluate the chlorine nuclear data at lower energies to assess these discrepancies. Interpretation of the data from future “faster” criticality benchmarks, which are needed for next-gen fast reactor designs, benefit from the improved constraints on the chlorine nuclear data validated in this work.