Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

Statistical change detection of building energy consumption: Applications to savings estimation

The surge in interval meter data availability and associated activity in energy data analytics has inspired new interest in advanced methods for building efficiency savings estimation. Statistical and machine learning approaches are being explored to improve the energy baseline models used to measure and verify savings. One outstanding challenge is the ability to identify and account for operational changes that may confound savings estimates. In the measurement and verification (M&V) context, ‘non-routine events’ (NREs) cause changes in building energy use that are not attributable to installed efficiency measures, and not accounted for in the baseline model’s independent variables. In the M&V process NREs must be accounted for as ‘adjustments’ to appropriately attribute the estimated energy savings to the specific efficiency interventions that were implemented. Currently this is a manual and custom process, conducted using professional judgment and engineering expertise. As such it remains a barrier in scaling and standardizing meter-based savings estimation. Here in this work, a data driven methodology was developed to (partially) automate, and therefore streamline the process of detecting NREs in the post-retrofit period and making associated savings adjustments. The proposed NRE detection algorithm is based on a statistical change point detection method and a dissimilarity metric. The dissimilarity metric measures the proximity between the actual time series of the post-retrofit energy consumption and the projected baseline, which is generated using a statistical baseline model. The suggested approach for NRE adjustment involves the NRE detection algorithm, the M&V practitioner, and a regression modeling algorithm. The performance of the detection and adjustment algorithm was evaluated using a simulation-generated test data set, and two benchmark algorithms. Results show a high true positive detection rate (75%-100% across the test cases), higher than ideal false positive detection rates (20%-70%), and low errors in energy adjustment (<0.7%). These results indicate that the algorithm holds for helping M&V practitioners to streamline the process of handling NREs. Moreover, the change point algorithm and underlying statistical principles could prove valuable for other building analytics applications such as anomaly detection and fault diagnostics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

GEN-IV Benchmarking of Triso Fuel Performance Models under accident conditions modeling input data

This document presents the benchmark plan for the calculation of particle fuel performance on safety testing experiments that are representative of operational accidental transients. The benchmark is dedicated to the modeling of fission product release under accident conditions by fuel performance codes from around the world, and the subsequent comparison to post-irradiation experiment (PIE) data from the modeled heating tests. The accident condition benchmark is divided into three parts: • The modeling of a simplified benchmark problem to assess potential numerical calculation issues at low fission product release. • The modeling of the AGR-1 and HFR-EU1bis safety testing experiments. • The comparison of the AGR-1 and HFR-EU1bis modeling results with PIE data. The simplified benchmark case, thereafter named NCC (Numerical Calculation Case), is derived from “Case 5” of the International Atomic Energy Agency (IAEA) Coordinated Research Program (CRP) on coated particle fuel technology [IAEA 2012]. It is included so participants can evaluate their codes at low fission product release. “Case 5” of the IAEA CRP-6 showed large code-to-code discrepancies in the release of fission products, which were attributed to “effects of the numerical calculation method rather than the physical model” [IAEA 2012]. The NCC is therefore intended to check if these numerical effects subsist. The first two steps imply the involvement of the benchmark participants with a modeling effort following the guidelines and recommendations provided by this document. The third step involves the collection of the modeling results by Idaho National Laboratory (INL) and the comparison of these results with the available PIE data. The objective of this document is to provide all necessary input data to model the benchmark cases, and to give some methodology guidelines and recommendations in order to make all results suitable for comparison with each other. The participants should read this document thoroughly to make sure all the data needed for their calculations is provided in the document. Missing data will be added to a revision of the document if necessary. 09/2016: Tables 6 and 8 updated. AGR-2 input data added

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Development and testing of the VITAMIN-B7/BUGLE-B7 coupled neutron-gamma multigroup cross-section libraries

The U.S. Nuclear Regulatory Commission's Regulatory Guide 1.190 states that calculational methods used to estimate reactor pressure vessel (RPV) fluence should use the latest version of the evaluated nuclear data file (ENDF). The VITAMIN-B6 fine-group library and BUGLE-96 broad-group library, which are widely used for RPV fluence calculations, were generated using ENDF/B-VI.3 data, which was the most current data when Regulatory Guide 1.190 was issued. We have developed new fine-group (VITAMIN-B7) and broad-group (BUGLE-B7) libraries based on ENDF/B-VII.0. These new libraries, which were processed using the AMPX code system, maintain the same group structures as the VITAMIN-B6 and BUGLE-96 libraries. Verification and validation of the new libraries were accomplished using diagnostic checks in AMPX, 'unit tests' for each element in VITAMIN-B7, and a diverse set of benchmark experiments including critical evaluations for fast and thermal systems, a set of experimental benchmarks that are used for SCALE regression tests, and three RPV fluence benchmarks. The benchmark evaluation results demonstrate that VITAMIN-B7 and BUGLE-B7 are appropriate for use in RPV fluence calculations and meet the calculational uncertainty criterion in Regulatory Guide 1.190. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Benchmark and Framework for Encouraging Research on Multi-Threaded Testing Tools

A problem that has been getting prominence in testing is that of looking for intermittent bugs. Multi-threaded code is becoming very common, mostly on the server side. As there is no silver bullet solution, research focuses on a variety of partial solutions. In this paper (invited by PADTAD 2003) we outline a proposed project to facilitate research. The project goals are as follows. The first goal is to create a benchmark that can be used to evaluate different solutions. The benchmark, apart from containing programs with documented bugs, will include other artifacts, such as traces, that are useful for evaluating some of the technologies. The second goal is to create a set of tools with open API s that can be used to check ideas without building a large system. For example an instrumentor will be available, that could be used to test temporal noise making heuristics. The third goal is to create a focus for the research in this area around which a community of people who try to solve similar problems with different techniques, could congregate.

Havelund, Klaus↗

Investigation of thermal hydraulic behavior of the High Temperature Test Facility's lower plenum via large eddy simulation

A high-fidelity computational fluid dynamics (CFD) analysis was performed using the Large Eddy Simulation (LES) model for the lower plenum of the High–Temperature Test Facility (HTTF), a ¼ scale test facility of the modular high temperature gas-cooled reactor (MHTGR) managed by Oregon State University. In most next–generation nuclear reactors, thermal stress due to thermal striping is one of the risks to be curiously considered. This is also true for HTGRs, especially since the exhaust helium gas temperature is high. In order to evaluate these risks and performance, organizations in the United States led by the OECD NEA are conducting a thermal hydraulic code benchmark for HTGR, and the test facility used for this benchmark is HTTF. HTTF can perform experiments in both normal and accident situations and provide high-quality experimental data. However, it is difficult to provide sufficient data for benchmarking through experiments, and there is a problem with the reliability of CFD analysis results based on Reynolds–averaged Navier–Stokes to analyze thermal hydraulic behavior without verification. To solve this problem, high-fidelity 3-D CFD analysis was performed using the LES model for HTTF. It was also verified that the LES model can properly simulate this jet mixing phenomenon via a unit cell test that provides experimental information. As a result of CFD analysis, the lower the dependency of the sub-grid scale model, the closer to the actual analysis result. In the case of unit cell test CFD analysis and HTTF CFD analysis, the volume-averaged sub-grid scale model dependency was calculated to be 13.0% and 9.16%, respectively. As a result of HTTF analysis, quantitative data of the fluid inside the HTTF lower plenum was provided in this paper. As a result of qualitative analysis, the temperature was highest at the center of the lower plenum, while the temperature fluctuation was highest near the edge of the lower plenum wall. The power spectral density of temperature was analyzed via fast Fourier transform (FFT) for specific points on the center and side of the lower plenum. FFT results did not reveal specific frequency-dominant temperature fluctuations in the center part. It was confirmed that the temperature power spectral density (PSD) at the top increased from the center to the wake. The vortex was visualized using the well-known scalar Q-criterion, and as a result, the closer to the outlet duct, the greater the influence of the mainstream, so that the inflow jet vortex was dissipated and mixed at the top of the lower plenum. Additionally, FFT analysis was performed on the support structure near the corner of the lower plenum with large temperature fluctuations, and as a result, it was confirmed that the temperature fluctuation of the flow did not have a significant effect near the corner wall. In addition, the vortices generated from the lower plenum to the outlet duct were identified in this paper. It is considered that the quantitative and qualitative results presented in this paper will serve as reference data for the benchmark.

97 MATHEMATICS AND COMPUTING↗

Metal hydride composition-derived parameters as machine learning features for material design and H 2 storage

Though hydrogen is a promising energy carrier for a green future, many challenges persist. One is the difficulty in engineering storage solutions, with metal hydrides being a leading contender among solid-state strategies. To facilitate efficient searching of candidate materials, ridge regression, simple decision trees, random forest ensembles, and gradient boosting ensembles were employed to predict the energy of formation, with the random forest ensemble resulting in the lowest test set error. First, two public databases, Materials Project and HydPark, were searched for metal hydrides. Feature engineering was performed before the models were developed, resulting in electronegativity, density, atomic density, d-character, f-character, band gap, hydrogen weight fraction, magnetization, temperature, and pressure being retained. The models were then benchmarked by the lowest test error before a random forest ensemble was used to populate entries missing energy of formation. Furthermore, all were then scored by hydrogen storage capacity and energy of formation suitability. Readily available features including several derived from only the chemical formula which were found to be highly predictive. and so are promising for high-throughput screening of arbitrary novel hydride formulations and blends for thermodynamic feasibility.

25 ENERGY STORAGE↗

Third and Final Shuttle Mission of the Isothermal Dendritic Growth Experiment Conducted: Highest Supercooling Ever Recorded Achieved

Dendrites describe the treelike crystal morphology commonly assumed in metals and alloys that freeze from supercooled or supersaturated melts. There remains a high level of engineering interest in dendritic solidification because the size, shape, and orientation of the dendrites determine the final microstructure of a material. It is the microstructure that then determines the physical properties of cast or welded products. Although it is well known that dendritic growth is controlled by the transport of latent heat from the moving solid-liquid interface, an accurate and predictive model has not yet been developed. The effects of gravity-induced convection on the transfer of heat from the interface have prevented adequate testing, under terrestrial conditions, of solidification models. The Isothermal Dendritic Growth Experiment (IDGE) constituted a series of three microgravity experiments flown aboard the Space Shuttle Columbia. The apparatus was used to grow and record dendrite solidification in the absence of gravity-induced convective heat transfer, thereby producing a wealth of benchmark-quality data for testing solidification models and theories.

Glicksman, Martin E.↗

Secure Retrieval of FFTF Testing, Design, and Operating Information

One of the goals of the Advanced Fuel Cycle Initiative (AFCI) is to preserve the knowledge that has been gained in the United States on Liquid Metal Reactors (LMR). In addition, preserving LMR information and knowledge is part of a larger international collaborative activity conducted under the auspices of the International Atomic Energy Agency (IAEA). A similar program is being conducted for EBR-II at the Idaho Nuclear Laboratory (INL) and international programs are also in progress. Knowledge preservation at the FFTF is focused on the areas of design, construction, startup, and operation of the reactor. As the primary function of the FFTF was testing, the focus is also on preserving information obtained from irradiation testing of fuels and materials. This information will be invaluable when, at a later date, international decisions are made to pursue new LMRs. In the interim, this information may be of potential use for international exchanges with other LMR programs around the world. At least as important in the United States, which is emphasizing large-scale computer simulation and modeling, this information provides the basis for creating benchmarks for validating and testing these large scale computer programs. Although the preservation activity with respect to FFTF information as discussed below is still underway, the team of authors above is currently retrieving and providing experimental and design information to the LMR modeling and simulation efforts for use in validating their computer models. On the Hanford Site, the FFTF reactor plant is one of the facilities intended for decontamination and decommissioning consistent with the cleanup mission on this site. The reactor facility has been deactivated and is being maintained in a cold and dark minimal surveillance and maintenance mode until final decommissioning is pursued. In order to ensure protection of information at risk, the program to date has focused on sequestering and secure retrieval. Accomplishments include secure retrieval of: more than 400 boxes of FFTF information, several hundred microfilm reels including Clinch River Breeder Reactor (CRBR) information, and 40 boxes of information on the Fuels and Materials Examination Facility (FMEF). All information preserved to date is now being stored and categorized consistent with the IAEA international standardized taxonomy. Earlier information largely related to irradiation testing is likewise being categorized. The fuel test results information exists in several different formats depending upon the final stage of the test evaluation. In some cases there is information from both non-destructive and destructive examination while in other cases only non-destructive results are available. Non-destructive information would include disassembly records, dimensional profilometry, gamma spectrometry, and neutron radiography. Information from destructive examinations would include fission gas analysis, metallography, and photomicrographs. Archiving of FFTF data, including both the reactor plant and the fuel test information, is being performed in coordination with other data archiving efforts underway under the aegis of the AFCI program. In addition to the FFTF efforts, archiving of data from the EBR-II reactor is being carried out by INL. All material at risk associated with FFTF documentation has been secured in a timely manner consistent with the stated plan. This documentation is now being categorized consistent with internationally agreed upon IAEA standards. Documents are being converted to electronic format for transfer to a large searchable electronic database being developed by INL. In addition, selected FFTF information is being used to generate test cases for large-scale simulation modeling efforts and for providing Design Data Need (DDN) packages as requested by the AFCI program.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Intercode comparison of gyrokinetic global electromagnetic modes

Aiming to fill a corresponding lack of sophisticated test cases for global electromagnetic gyrokinetic codes, a new hierarchical benchmark is proposed. Starting from established test sets with adiabatic electrons, fully gyrokinetic electrons, and electrostatic fluctuations are taken into account before finally studying the global electromagnetic micro-instabilities. Results from up to five codes involving representatives from different numerical approaches as particle-in-cell methods, Eulerian and Semi-Lagrangian are shown. By means of spectrally resolved growth rates and frequencies and mode structure comparisons, agreement can be confirmed on ion-gyro-radius scales, thus providing confidence in the correct implementation of the underlying equations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Latency and Bandwidth Microbenchmarks of US Department of Energy Systems in the June 2023 Top 500 List

As a rule, Top 500 class supercomputers are extensively benchmarked as part of their acceptance testing process. However, barring publicly posted LINPACK / HPCG results, most benchmark results are often inaccessible outside the hosting institution. Moreover, these higher level benchmarks do not provide easy answers to common questions such as “What is the realizable memory bandwidth?” or “What is the launch latency on the accelerator?” To partially address these issues, we executed selected single-node micro-benchmarks — focused on latencies and memory bandwidth — on every US Department of Energy system above rank 150 of the June 2023 Top 500 list, excepting NERSC’s Cori and ORNL’s Frontier TDS (now decommissioned or repurposed). We hope to provide an easy “first stop” reference for users of current Top 500 systems and inspire users and administrators of other Top 500 systems to similarly compile and make available benchmark results for their systems.

Siefert, Christopher M.↗

Verification of Unstructured Mesh Capabilities in MCNP6 for Reactor Physics Problems

New unstructured mesh capabilities in MCNP6 (developmental version during summer 2012) show potential for conducting multi-physics analyses by coupling MCNP to a finite element solver such as Abaqus/CAE. Before these new capabilities can be utilized, the ability of MCNP to accurately estimate eigenvalues and pin powers using an unstructured mesh must first be verified. Previous work to verify the unstructured mesh capabilities in MCNP was accomplished using the Godiva sphere, and this work attempts to build on that. To accomplish this, a criticality benchmark and a fuel assembly benchmark were used for calculations in MCNP using both the Constructive Solid Geometry (CSG) native to MCNP and the unstructured mesh geometry generated using Abaqus/CAE. The Big Ten criticality benchmark was modeled due to its geometry being similar to that of a reactor fuel pin. The C5G7 3-D Mixed Oxide (MOX) Fuel Assembly Benchmark was modeled to test the unstructured mesh capabilities on a reactor-type problem.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

FY2012 Annual Progress Report for Vehicle and Systems Simulation and Testing

Annual progress report that evaluates the technologies and performance characteristics of advanced automotive powertrain components and subsystems in an integrated vehicle systems context. These evaluations address light-, medium-, and heavy-duty vehicle platforms. This work is directed toward evaluating and verifying the targets of the VTO R&D teams and to providing guidance in establishing roadmaps for achievement of these goals.

33 ADVANCED PROPULSION SYSTEMS↗

Space network scheduling benchmark: A proof-of-concept process for technology transfer

This paper describes a detailed proof-of-concept activity to evaluate flexible scheduling technology as implemented in the Request Oriented Scheduling Engine (ROSE) and applied to Space Network (SN) scheduling. The criteria developed for an operational evaluation of a reusable scheduling system is addressed including a methodology to prove that the proposed system performs at least as well as the current system in function and performance. The improvement of the new technology must be demonstrated and evaluated against the cost of making changes. Finally, there is a need to show significant improvement in SN operational procedures. Successful completion of a proof-of-concept would eventually lead to an operational concept and implementation transition plan, which is outside the scope of this paper. However, a high-fidelity benchmark using actual SN scheduling requests has been designed to test the ROSE scheduling tool. The benchmark evaluation methodology, scheduling data, and preliminary results are described.

Moe, Karen↗

Improving the Effectiveness of Context-based Prefetching with Multi-order Analysis

Data prefetching is an effective way to accelerate data access in high-end computing systems and to bridge the increasing performance gap between processor and memory. In recent years, the context-based data prefetching has received intensive attention because of its general applicability. In this study, we provide a preliminary analysis of the impact of orders on the effectiveness of the context-based prefetching. Motivated by the observations from the analytical results, we propose a new context-based prefetching method named Multi-Order Context-based (MOC) prefetching to adopt multi-order context analysis to increase the context-based prefetching effectiveness. We have carried out simulation testing with the SPEC-CPU2006 benchmarks via an enhanced CMP$im simulator. The simulation results show that the proposed MOC prefetching method outperforms the existing single-order prefetching and reduces the data-access latency effectively.

99 GENERAL AND MISCELLANEOUS↗

The Isothermal Dendritic Growth Experiment (IDGE): USMP-4 One-Year-Report

Dendrites describe the tree-like crystal morphology commonly assumed in many material systems--particularly in metals and alloys that freeze from supercooled or supersaturated melts. There remains a high level of engineering interest in dendritic solidification because of the role of dendrites in the determination of cast alloy microstructures. Microstructure plays a key role in determining the physical properties of cast or welded products. In addition, dendritic solidification provides an example of non-equilibrium physics and is one of the simplest non-trivial examples of dynamic pattern formation, where an amorphous melt, under simple starting conditions, evolves into a complex ramified microstructure. Although it is well-known that dendritic growth is controlled by the transport of latent heat from the moving solid-melt interface as the dendrite advances into a supercooled melt, an accurate, and predictive model has not been developed. Current theories consider: 1) the transfer of heat or solute from the solid-liquid interface into the melt, and 2) the interfacial crystal growth and growth selection physics for the interface. However, the effects of gravity-induced convection on the transfer of heat from the interface prevent either element from being adequately tested solely under terrestrial conditions. The Isothermal Dendritic Growth Experiment (IDGE) constituted a series of three NASA-supported microgravity experiments, all of which flew aboard the space shuttle, Columbia. This experimental space flight series was designed and operated to grow and record dendrite solidification in the absence of gravity-induced convective heat transfer, and thereby produce a wealth of benchmark-quality data for testing solidification scaling laws. The data collection from the on-orbit phase of the IDGE flight series is now complete. We are currently completing analyses and moving towards final data archiving.

Glicksman, M. E.↗

Translating an AI application from Lisp to Ada: A case study

A set of benchmarks was developed to test the performance of a newly designed computer executing both Lisp and Ada. Among these was AutoClassII -- a large Artificial Intelligence (AI) application written in Common Lisp. The extraction of a representative subset of this complex application was aided by a Lisp Code Analyzer (LCA). The LCA enabled rapid analysis of the code, putting it in a concise and functionally readable form. An equivalent benchmark was created in Ada through manual translation of the Lisp version. A comparison of the execution results of both programs across a variety of compiler-machine combinations indicate that line-by-line translation coupled with analysis of the initial code can produce relatively efficient and reusable target code.

Davis, Gloria J.↗