Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

Verification and Validation of CFD Simulations involving Twin Rectangular Jets using Steady RANS in Star-CCM+

Analyses such as component design and safety analysis can be conducted through careful application of computational fluid dynamics (CFD) codes. Before these codes can be considered ready for specific analysis, benchmarking (validation) against applicable datasets is needed. The twin jet water facility (TJWF) developed originally at the University of Tennessee, Knoxville has been purposed for aiding in this type of work. The TJWF allows for large data sets to be created using state of the art non-invasive measurement techniques within a mostly transparent testing volume. Utilizing velocity data collected in the TJWF by Wang et. al., the Star-CCM+ code was benchmarked a using steady Reynolds averaged Navier Stokes (RANS) framework for a combining two jet case. Careful efforts to properly conduct verification of the numerical work was undertaken with a sample reported in this summary. The computational results were validated against the experimental work and found to be within acceptable limits. The parameters of interest were the merging and combining points (MP and CP) of the twin jets, and the velocity profiles at increasing heights after injection. Fig.1 provides a graphical depiction of the twin jets combining within a free shear volume with key geometric factors included. The MP and velocity profiles were predicted with reasonable agreement. Whereas, the combining point of the twin jets was found to be over-predicted by {approx}24% difference. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

A multi-dataset data-collection strategy produces better diffraction data

Theoretical analysis and experimental validation prove that a multi-dataset data-collection strategy produces better diffraction data. The readiness test is a simple and sensitive method for X-ray data-collection system evaluation and a benchmark. A multi-dataset (MDS) data-collection strategy is proposed and analyzed for macromolecular crystal diffraction data acquisition. The theoretical analysis indicated that the MDS strategy can reduce the standard deviation (background noise) of diffraction data compared with the commonly used single-dataset strategy for a fixed X-ray dose. In order to validate the hypothesis experimentally, a data-quality evaluation process, termed a readiness test of the X-ray data-collection system, was developed. The anomalous signals of sulfur atoms in zinc-free insulin crystals were used as the probe to differentiate the quality of data collected using different data-collection strategies. The data-collection results using home-laboratory-based rotating-anode X-ray and synchrotron X-ray systems indicate that the diffraction data collected with the MDS strategy contain more accurate anomalous signals from sulfur atoms than the data collected with a regular data-collection strategy. In addition, the MDS strategy offered more advantages with respect to radiation-damage-sensitive crystals and better usage of rotating-anode as well as synchrotron X-rays.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

On the flow induced vibration of an externally excited nuclear reactor experiment

An in-pile, drop-in experiment design is presently being designed and studied for the near-term deployment within the Advanced Test Reactor (ATR) located at the Idaho National Laboratory (INL); this experiment is termed the Miniplate-1 Large-B (MP-1 LB) Experiment. A number of explicit studies are performed during the design- and safety-related stage. Traditionally, a clear and logical methodology has been developed and utilized for analyses such as hydraulics, thermal-loads, mechanical loads, and others for these experiments. Recently a small component from a different experiment assembly mechanically separated while in the reactor’s core. While this experiment didn’t compromise the safety of the reactor, it led to a higher-level question which centered on whether the appropriate level of consideration was being made toward the fluid–structure-interactions of these experiments. The outcome yielded separate flow test experiments of like-for-like geometry in an experimental loop located at Oregon State University which produces experimental data compliant with applicable parts and requirements to ASME’s NQA-1 2008, 2009a standard – suitable for benchmark evaluation. Here, the objectives of this study are to (1) develop a process for handling and interpreting the mechanical response of the test elements during hydraulic testing, to (2) characterize the motion of a specific test element during a flow test which imposes a wide range of hydraulic conditions, and (3) provide objective observations toward the potential safety related implications that are tied to the synthesized data. The outcome of this study has led to a confident process in inspecting the experimental data, synthesized it for interpretation, identified several unique hydraulic characteristics of the experiment design which were previously unknown, and demonstrated that the likelihood for mechanical failure resulting from fluid-structure-interactions in the reactor is far below any criterion for concern of the element’s safety.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Contemporary Trends in Radiation Oncology Resident Research

Purpose: To test the hypothesis that recent resident research productivity might be different than a decade ago, and to provide contemporary information about resident scholarly activity. Methods and Materials: We compiled a list of radiation oncology residents from the 2 most recent graduating classes (June 2014 and 2015) using the Association of Residents in Radiation Oncology annual directories. We queried the PubMed database for each resident's first-authored publications from postgraduate years (PGY) 2 through 5, plus a 3-month period after residency completion. We abstracted corresponding historical data for 2002 to 2007 from the benchmark publication by Morgan and colleagues (Int J Radiat Oncol Biol Phys 2009;74:1567-1572). We tested the null hypothesis that these 2 samples had the same distribution for number of publications using the Wilcoxon rank-sum test. We explored the association of demographic factors and publication number using multivariable zero-inflated Poisson regression. Results: There were 334 residents publishing 659 eligible first-author publications during residency (range 0-17; interquartile range 0-3; mean 2.0; median 1). The contemporary and historical distributions were significantly different (P<.001); contemporary publication rates were higher. Publications accrued late in residency (27% in PGY-4, 59% in PGY-5), and most were original research (75%). In the historical cohort, half of all articles were published in 3 journals; in contrast, the top half of contemporary publications were spread over 10 journals—most commonly International Journal of Radiation Oncology • Biology • Physics (17%), Practical Radiation Oncology (7%), and Radiation Oncology (4%). Male gender, non-PhD status, and larger residency size were associated with higher number of publications in the multivariable analysis. Conclusion: We observed an increase in first-author publications during training compared with historical data from the mid-2000s. These contemporary figures may be useful to medical students considering radiation oncology, current residents, training programs, and prospective employers.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Nuclear fragmentation measurements for hadrontherapy and space radiation protection

Nuclear fragmentation measurements are necessary in hadrontherapy and space radiation protection, to predict the effects of the ion nuclear interactions within the human body. Nowadays, a very limited set of carbon fragmentation cross sections has been measured and in particular, to our knowledge, no double differential fragmentation cross sections at intermediate energies are available in literature. We have measured the double differential cross sections and the angular distributions of the secondary fragments produced in the {sup 12}C fragmentation at 62 AMeV on a thin carbon target. The experimental data have been also used to benchmark the prediction capability of the Geant4 Monte Carlo code at intermediate energies, where it was never tested before.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real- world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5- based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

azimuth↗

SystemBurn: Principles of Design and Operation Release 2.1

As high performance computing technology progresses toward the progressively more extreme scales required to address critical computational problems of both national and global interest, power and cooling for these extreme scale systems is becoming a growing concern. A standardized methodology for testing system requirements under maximal system load and validating system environmental capability to meet those requirements is critical to maintaining system stability and minimizing power and cooling risks for high end data centers. Moreover, accurate testing permits the high end data center to avoid issues of under- or over-provisioning power and cooling capacity saving resources and mitigating hazards. Previous approaches to such testing have employed an ad hoc collection of tools, which have been anecdotally perceived to produce a heavy system load. In this report, we present SystemBurn, a software tool engineered to allow a system user to methodically create a maximal system load on large scale systems for the purposes of testing and validation.

99 GENERAL AND MISCELLANEOUS↗

Linearised Fokker–Planck collision model for gyrokinetic simulations

We introduce a gyrokinetic, linearised Fokker–Planck collision model that satisfies conservation laws and is accurate at arbitrary collisionalities. The differential test-particle component of the operator is exact; the integral field-particle component is approximated using a spherical harmonic and a modified Laguerre polynomial expansion developed by Hirshman and Sigmar (1976 Phys. Fluids 19 1532). The numerical methods of the implementation in the δf-gyrokinetic code stella (Barnes et al 2019 J. Comput. Phys. 391 365–80) are discussed, and conservation properties of the operator are demonstrated. The collision model is then benchmarked against the collision model of the gyrokinetic solver GS2 in the limiting cases of a reduced test-particle collision operator and energy- and momentum-conserving operator. The accuracy of the full collision model is investigated by solving the parallel Spitzer-Härm problem for the transport coefficients. It is shown that retaining collisional energy flux and higher-order terms in the field-particle operator reduces errors in the transport coefficients from 10%–25% for a simple momentum- and energy-conserving model to under 1%.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Reproducible Validation of Algorithms for Estimating Array Tilt and Azimuth from Photovoltaic Power Time Series

In this research, we assess the viability of four different, publicly available algorithms for estimating the azimuth and tilt parameters of solar photovoltaic systems using only the associated AC power time series data and site latitude-longitude coordinates. In this work, we curated a benchmarking data set of 44 fixed-tilt systems, comprising 275 measured AC power inverter data streams, with known azimuth and tilt parameters. Additionally, we isolated test cases in the data set with real-world issues, including shading and clipping, to determine how algorithm performance varies based on the presence of these phenomena. Using this data set for benchmarking, we evaluated the estimated vs. actual system characteristics for each algorithm, as well as the associated algorithm execution time using a standardized benchmarking process. The two highest performing algorithms were the Solar Data Tools and the PVWatts 5-based methods, which both achieved a median absolute error of approximately 5 and 1 degrees for azimuth and tilt, respectively. During run time analysis, the SDT method was approximately 5 times faster than the PVWatts 5-based method, with the median execution time for a stream varying between 6 and 8 seconds vs. a median run time of 31 seconds for the PVWatts 5-based method.

algorithm validation↗

AFIP-7 Post-irradiation Examination Summary Report

The following report contains the results and conclusions from post irradiation examinations performed on the fuel plates irradiated as part of the AFIP-7 experiment, the first test of base monolithic fuel in a fuel assembly consisting of curved, constrained fuel plates. These examinations include visual examination; eddy-current testing to measure oxide growth; profilometry to measure dimensional changes of the fuel plates; neutron radiography for evaluating fuel condition prior to sectioning; gamma scanning to provide relative measurements of burn-up and indication of fission-product relocation; analytical chemistry to benchmark the physics burn-up calculations; metallographic examination to identify microstructural changes in the fuel, interlayer and cladding; and blister threshold testing. These characterization activities were conducted to demonstrate that the fuel meets irradiation performance requirements for mechanical integrity, geometric stability, and stable and predictable behavior. Revision 1 being reviewed for share publicly.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

High Temperature Test Facility Preliminary RELAP5-3D Input Model Description

A RELAP5-3D input model is being developed for the High Temperature Test Facility at Oregon State University. The current model is described in detail. Further refinements will be made to the model as final as-built drawings are released and when system characterization data are available for benchmarking the input model.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Benchmarking machine learning interatomic potentials via phonon anharmonicity

Abstract Machine learning approaches have recently emerged as powerful tools to probe structure-property relationships in crystals and molecules. Specifically, machine learning interatomic potentials (MLIPs) can accurately reproduce first-principles data at a cost similar to that of conventional interatomic potential approaches. While MLIPs have been extensively tested across various classes of materials and molecules, a clear characterization of the anharmonic terms encoded in the MLIPs is lacking. Here, we benchmark popular MLIPs using the anharmonic vibrational Hamiltonian of ThO 2 in the fluorite crystal structure, which was constructed from density functional theory (DFT) using our highly accurate and efficient irreducible derivative methods. The anharmonic Hamiltonian was used to generate molecular dynamics (MD) trajectories, which were used to train three classes of MLIPs: Gaussian approximation potentials, artificial neural networks (ANN), and graph neural networks (GNN). The results were assessed by directly comparing phonons and their interactions, as well as phonon linewidths, phonon lineshifts, and thermal conductivity. The models were also trained on a DFT MD dataset, demonstrating good agreement up to fifth-order for the ANN and GNN. Our analysis demonstrates that MLIPs have great potential for accurately characterizing anharmonicity in materials systems at a fraction of the cost of conventional first principles-based approaches.

interatomic potentials↗

Concentration and Recovery of Rare Earth Elements from Eastern US Coal Refuse

Recent studies funded by the US Department of energy have shown that coal and coal byproducts contain elevated contents of Rare Earth Elements (REEs), making them a potential resource for these critical materials. The approach employed in this research focused on the concentration and extraction of REEs from fine coal refuse derived from various preparation plants in the Appalachian coal basin of the United States. Initial efforts in this research focused on the identification and characterization of REEs in various fine coal refuse streams from nine distinct industrial preparation plants in Appalachia. The average REE content in these materials was determined to be approximately 200 ppm, but the REE content showed a strong correlation to the aluminum content, suggesting that the REEs are closely associated with the clay minerals present in the refuse. Given the relatively low REE concentrations, initial efforts sought to concentrate the REEs through decarbonization and dispersive liberation steps. In these tests, high-shear agitation in the presence of a polyelectrolyte, followed by sedimentation was able to isolate the REE-enriched fine clay particles from siliceous gangue minerals. Following the dispersive liberation step, all samples were found to have an REE content greater than 300 ppm, a benchmark used for many initial exploratory studies. In one case, the REE content was increased by more than 125%. Subsequent extraction tests initially utilized a direct ion-exchange leaching approach with ammonium sulfate as lixiviant. In all cases, the simple ion-exchange leaching process failed to recover significant quantities of rare earth elements, ultimately suggesting that the REEs in fine coal waste may be passivated or bound in a colloidal phase. To access this colloidal phase, several alternative approaches were evaluated, including leaching with alternative ion-exchange lixiviants, reductive leaching, gas-purged leaching, and others. The approach that showed the most promise was strong alkaline pretreatment, followed by ion-exchange leaching with ammonium sulfate at pH 4. A combination of strong alkali and high-temperatures treatment successfully liberated the REEs, converting them to a form amenable to ion-exchange leaching. The highest REE recovery achieved with this method was determined to be 39%. Lastly, bench-scale solvent extraction tests were used to further concentrate REEs in the leach solution and demonstrate that mixed rare earth concentrates can be successfully produced from fine coal refuse.

01 COAL, LIGNITE, AND PEAT↗

TH-E-BRE-01: A 3D Solver of Linear Boltzmann Transport Equation Based On a New Angular Discretization Method with Positivity for Photon Dose Calculation Benchmarked with Geant4

Purpose: The Linear Boltzmann Transport Equation (LBTE) solved through statistical Monte Carlo (MC) method provides the accurate dose calculation in radiotherapy. This work is to investigate the alternative way for accurately solving LBTE using deterministic numerical method due to its possible advantage in computational speed from MC. Methods: Instead of using traditional spherical harmonics to approximate angular scattering kernel, our deterministic numerical method directly computes angular scattering weights, based on a new angular discretization method that utilizes linear finite element method on the local triangulation of unit angular sphere. As a Result, our angular discretization method has the unique advantage in positivity, i.e., to maintain all scattering weights nonnegative all the time, which is physically correct. Moreover, our method is local in angular space, and therefore handles the anisotropic scattering well, such as the forward-peaking scattering. To be compatible with image-guided radiotherapy, the spatial variables are discretized on the structured grid with the standard diamond scheme. After discretization, the improved sourceiteration method is utilized for solving the linear system without saving the linear system to memory. The accuracy of our 3D solver is validated using analytic solutions and benchmarked with Geant4, a popular MC solver. Results: The differences between Geant4 solutions and our solutions were less than 1.5% for various testing cases that mimic the practical cases. More details are available in the supporting document. Conclusion: We have developed a 3D LBTE solver based on a new angular discretization method that guarantees the positivity of scattering weights for physical correctness, and it has been benchmarked with Geant4 for photon dose calculation.

97 MATHEMATICS AND COMPUTING↗

Comparative verification of hydro-mechanical fracture behavior: Task G of international research project DECOVALEX–2023

Numerical simulations become a necessity when experimental approaches cannot cover the required physical and time scale of interest. One of such area is a simulation of long-term host rock behaviors for nuclear waste disposal and simulation tools involved in the assessment must go through rigorous validation tests. The DECOVALEX project (Development of COupled models and their VAlidation against EXperiments) is dedicated to this purpose by international participants. This work is part of the ongoing phase DECOVALEX–2023 (D–2023, Task G) particularly aiming to simulate fracture behaviors under various conditions. Here, in this study, we cross-verified a variety of numerical methods including continuous and discontinuous approaches against four benchmark exercises with emphasis on numerical accuracy and parameterization of the various numerical approaches. The systematic inter-comparisons of test cases highlight advantages and disadvantages of the different numerical models. Numerical details on discretization effects (e.g. mesh density and orientation) and domain size were investigated in detail for practical applications. It became evident that meticulous attention to mesh resolution and domain size is imperative for achieving accurate numerical simulations, even for static cracks. Moreover, when comparing numerical methods to closed-form solutions for static cracks, all models successfully reproduced the maximum crack opening but encountered challenges near the crack tips. Finally, the paper discusses how to convert between and therefore compare parameters of various numerical approaches. Our benchmark studies reveal that each model necessitates a distinct number of parameters, even in simple scenarios like static crack aperture benchmarks. It is generally more practical to employ fewer parameters to mitigate model over-parameterization and enhance experimental feasibility.

58 GEOSCIENCES↗

National Security Science and Technology Initiative: Air Cargo Screening

The non-intrusive inspection (NII) of consolidated air cargo carried on commercial passenger aircraft continues to be a technically challenging, high-priority requirement of the Department of Homeland Security's Science and Technology Directorate (DHS S&T), the Transportation Security Agency and the Federal Aviation Administration. The goal of deploying a screening system that can reliably and cost-effectively detect explosive threats in consolidated cargo without adversely affecting the flow of commerce will require significant technical advances that will take years to develop. To address this critical National Security need, the Battelle Memorial Institute (Battelle), under a Cooperative Research and Development Agreement (CRADA) with four of its associated US Department of Energy (DOE) National Laboratories (Oak Ridge, Pacific Northwest, Idaho, and Brookhaven), conducted a research and development initiative focused on identifying, evaluating, and integrating technologies for screening consolidated air cargo for the presence of explosive threats. Battelle invested $8.5M of internal research and development funds during fiscal years 2007 through 2009. The primary results of this effort are described in this document and can be summarized as follows: (1) Completed a gap analysis that identified threat signatures and observables, candidate technologies for detection, their current state of development, and provided recommendations for improvements to meet air cargo screening requirements. (2) Defined a Commodity/Threat/Detection matrix that focuses modeling and experimental efforts, identifies technology gaps and game-changing opportunities, and provides a means of summarizing current and emerging capabilities. (3) Defined key properties (e.g., elemental composition, average density, effective atomic weight) for basic commodity and explosive benchmarks, developed virtual models of the physical distributions (pallets) of three commodity types and three explosive benchmarks combinations, and conducted modeling and simulation studies to begin populating the matrix of commodities, threats, and detection technologies. (4) Designed and fabricated basic (homogeneous) commodity test pallets and fabricated inert stimulants to support experiments and to validate modeling/simulation results. (5) Developed/expanded the team's capabilities to conduct full-scale imaging (neutron and x-ray) experiments of air cargo commodities and explosive benchmarks. (6) Conducted experiments to improve the collection of trace particles of explosives from a variety of surfaces representative of air cargo materials by means of mechanical (air/vibration/pressure), thermal, and electrostatic methods. Air cargo screening is a difficult challenge that will require significant investment in both research and development to find a suitable solution to ensure the safety of passengers without significantly hindering the flow of commodities. The initiative funded by Battelle has positioned this group to make major contributions in meeting the air cargo challenge by developing collaborations, developing laboratory test systems, improving knowledge of the challenges (both technical and business) for air cargo screening, and increasing the understanding of the capabilities for current inspection methods (x-ray radiography, x-ray backscatter, etc.) and potential future inspection methods (neutron radiography, fusion of detector modalities, advanced trace detection, etc.). Lastly, air cargo screening is still an issue that will benefit from collaboration between Department of Energy Laboratories and Battelle. On January 7, 2010, DHS Secretary Napolitano joined White House Press Secretary Robert Gibbs and Assistant to the President for Counterterrorism and Homeland Security John Brennan to announce several recommendations DHS has made to the President for improving the technology and procedures used to protect air travel from acts of terrorism. (This announcement followed the 25 Dec'09 Delta/Northwest Airlines Flight 253 terror attack.) Secretary Napolitano outlined five recommendations DHS will pursue to enhance the safety of the traveling public. One of the five recommendations, read as follows: 'Establish a partnership on aviation security between DHS and the Department of Energy and its National Laboratories in order to develop new and more effective technologies to deter and disrupt known threats and proactively anticipate and protect against new ways by which terrorists could seek to board an aircraft.' In conclusion, it appears very timely that Battelle and its DOE lab partners initiated a serious collaboration on the air cargo topic, and that we should continue to work toward future collaboration in response to the government's needs.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

Current Reactor Physics Benchmark Activities at the Idaho National Laboratory

The International Reactor Physics Experiment Evaluation Project (IRPhEP) and the International Criticality Safety Benchmark Evaluation Project (ICSBEP) were established to preserve integral reactor physics and criticality experiment data for present and future research. These valuable assets provide the basis for recording, developing, and validating our integral nuclear data, and experimental and computational methods. These projects are managed through the Idaho National Laboratory (INL) and the Organisation for Economic Co-operation and Development Nuclear Energy Agency (OECD-NEA). Staff and students at the Department of Energy - Idaho (DOE-ID) and INL are engaged in the development of benchmarks to support ongoing research activities. These benchmarks include reactors or assemblies that support Next Generation Nuclear Plant (NGNP) research, space nuclear Fission Surface Power System (FSPS) design validation, and currently operational facilities in Southeastern Idaho.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗