Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50

ChatPORT: Fine-Tuned LLM for Easy Code {PORT}ing

Fine-tuning existing LLMs for specialized tasks has become a very attractive alternative due to its low cost and quick development cycle. With many pre-trained LLMs available, it is an increasingly complex task to choose the correct model as the starting point or base model. In this work we discuss ChatPORT - a specialized fine-tuned LLM geared towards providing correctly translated codes from one programming model to another. We evaluate a number of base models and compare and contrast their features and characteristics that make them a viable starting point. In this paper, we focus on the OpenMP offload porting capabilities of ChatPORT. We build our training data using kernels from the Heterogeneous Computing Benchmarks (HeCBench) [12] and the OpenMP Validation and Verification suite [5] to fine-tune the base models. We then test the model using unseen kernels extracted from the HeCBench benchmark suite. Our results show that: (1) not all open LLMs geared towards HPC are aware of programming models like OpenMP, (2) although all base models benefit from fine-tuning they learn differently and produce different correctness rates, (3) depending on the memory size and compute resource available, different base models can be used for fine-tuning without significantly affecting the quality of transpiled code they generate, (4) fine-tuning improved the correctness rate of the LLM by an average of 43.2%, and (5) feedback-based training data further increased the correctness rate by an average of 6% over the LLMs tested.

Pophale, Swaroop [ORNL] (ORCID:0000000185446367)↗

7X performance results - final report : ASCI Red vs Red Storm.

The goal of the 7X performance testing was to assure Sandia National Laboratories, Cray Inc., and the Department of Energy that Red Storm would achieve its performance requirements which were defined as a comparison between ASCI Red and Red Storm. Our approach was to identify one or more problems for each application in the 7X suite, run those problems at multiple processor sizes in the capability computing range, and compare the results between ASCI Red and Red Storm. The first part of this report describes the two computer systems, the applications in the 7X suite, the test problems, and the results of the performance tests on ASCI Red and Red Storm. During the course of the testing on Red Storm, we had the opportunity to run the test problems in both single-core mode and dual-core mode and the second part of this report describes those results. Finally, we reflect on lessons learned in undertaking a major head-to-head benchmark comparison.

99 GENERAL AND MISCELLANEOUS↗

Application-level regression testing framework using Jenkins

Monitoring and testing for regression of large-scale systems such as the NCSA's Blue Waters supercomputer are challenging tasks. In this paper, we describe the solution we came up with to perform those tasks. The goal was to find an automated solution for running user-level regression tests to evaluate system usability and performance. Jenkins, an automation server software, was chosen for its versatility, large user base, and multitude of plugins including collecting data and plotting test results over time. We also describe our Jenkins deployment to launch and monitor jobs on remote HPC system, perform authentication with one-time password, and integrate with our LDAP server for its authorization. We show some use cases and describe our best practices for successfully using Jenkins as a user-level system-wide regression testing and monitoring framework for large supercomputer systems.

97 MATHEMATICS AND COMPUTING↗

Improved Gamma Yield and Interaction Cross-Section Libraries of MC2-3

The MC{sup 2}-3 code developed by Argonne National Laboratory (ANL) has been widely used for multigroup cross section generation for fast reactor design and analysis. The neutron cross sections generated with MC{sup 2}-3 have been validated with various fast critical experiments. The gamma production data and interaction cross sections were also verified with the EBR-II benchmark problem. The current gamma calculation process is based on the 21-group gamma libraries generated with the NJOY code, which does not include the delayed gamma production and does not account for the self-shielding effect on the KERMA (Kinetic Energy Released in Material) factor. In order to improve gamma prediction accuracy, the gamma libraries of MC{sup 2}-3 have recently been extended from 21 to 94 groups, including delayed gamma data and kinetic energy release per reaction data. The latter is used to take into account the self-shielding effect on KERMA factors. Gamma transport calculation capabilities have also been added to MC{sup 2}-3. In this paper, we present the updated data and gamma transport capabilities along with the verification tests with ABTR and EBR-II benchmark problems. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

The ESIF-HPC-2 Benchmark Suite

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of "dimensions" that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The Nature of Scatter at the DARHT Facility and Suggestions for Improved Modeling of DARHT Facility

The U.S. Stockpile Stewardship Program [1] is designed to sustain and evaluate the nuclear weapons stockpile while foregoing underground nuclear tests. The maintenance of a smaller, aging U.S. nuclear weapons stockpile without underground testing requires complex computer calculations [14]. These calculations in turn need to be verified and benchmarked [14]. A wide range of research facilities have been used to test and evaluate nuclear weapons while respecting the Comprehensive Nuclear Test-Ban Treaty (CTBT) [2]. Some of these facilities include the National Ignition Facility (NIF) at Lawrence Livermore National Laboratory, the Z machine at Sandia National Laboratories, and the Dual Axis Radiographic Hydrodynamic Test (DARHT) facility at Los Alamos National Laboratory. This research will focus largely on DARHT (although some information from Cygnus and the Los Alamos Microtron may be used in this research) by modeling it and comparing to experimental data. DARHT is an electron accelerator that employs high-energy flash x-ray sources for imaging hydro-tests. This research proposes to address some of the issues crucial to understanding DARHT Axis II and the analysis of the radiographic images produced. Primarily, the nature of scatter at DARHT will be modeled and verified with experimental data. It will then be shown that certain design decisions can be made to optimize the scatter field for hydrotest experiments. Spectral effects will be briefly explored to determine if there is any considerable effect on the density reconstruction caused by changes in the energy spectrum caused by target changes. Finally, a generalized scatter model will be made using results from MCNP that can be convolved with the direct transmission of an object to simulate the scatter of that object at the detector plane. The region in which with this scatter model is appropriate will be explored.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

The ESIF-HPC-2 Benchmark Suite: Preprint

We describe the development of the ESIF-HPC-2 benchmark suite, a collection of kernel and application benchmark codes for measuring computational and I/O performance from single nodes to full HPC systems that was used for acceptance testing in our recent HPC procurement. The configurations of the benchmarks used for our system is presented. We also describe a set of 'dimensions' that can be used to classify benchmarks and assess coverage of a suite systematically. The collection is offered cost-free as a GitHub repository for general usage and further development.

97 MATHEMATICS AND COMPUTING↗

Model year 2010 Ford Fusion Level-1 testing report.

As a part of the US Department of Energy's Advanced Vehicle Testing Activity (AVTA), a model year 2010 Ford Fusion was procured by eTec (Phoenix, AZ) and sent to ANL's Advanced Powertrain Research Facility for the purposes of vehicle-level testing in support of the Advanced Vehicle Testing Activity. Data was acquired during testing using non-intrusive sensors, vehicle network information, and facilities equipment (emissions and dynamometer). Standard drive cycles, performance cycles, steady-state cycles, and A/C usage cycles were conducted. Much of this data is openly available for download in ANL's Downloadable Dynamometer Database. The major results are shown in this report. Given the benchmark nature of this assessment, the majority of the testing was done over standard regulatory cycles and sought to obtain a general overview of how the vehicle performs. These cycles include the US FTP cycle (Urban) and Highway Fuel Economy Test cycle as well as the US06, a more aggressive supplemental regulatory cycle. Data collection for this testing was kept at a fairly high level and includes emissions and fuel measurements from an exhaust emissions bench, high-voltage and accessory current/voltage from a DC power analyzer, and CAN bus data such as engine speed, engine load, and electric machine operation. The following sections will seek to explain some of the basic operating characteristics of the MY2010 Fusion and provide insight into unique features of its operation and design.

33 ADVANCED PROPULSION SYSTEMS↗

Kinetics of heterogeneous nucleation in supersaturated vapor: Fundamental limits to neutral particle detection revisited

We examine the nucleated (with barrier) activation of perfectly wetting (zero contact angle) particles ranging from essentially bulk size down to approximately 1-nm mass diameter. While similar studies trace back to the pioneering work of Fletcher, we present here a novel approach to the analysis based on general area constructions that enable key thermodynamic properties, including surface and bulk contributions to nucleation work, to be interpreted geometrically with reference to the Kelvin curve. The kinetics of activation are described in more detail in terms of the mean first passage time (MFPT) for barrier crossing. MFPT theory and benchmark calculations are used to develop and test a new approximate-but-simpler-to-use analytic expression for the barrier crossing rate. The present study is motivated by recent condensation particle counter (CPC) studies that appear to finally establish the long-predicted detection of 'sub-Kelvin' particles in the nano-size regime. Corresponding states thermodynamic and kinetic scaling approaches are used to facilitate the correlation and selection of optimal CPC working fluids and operating conditions based on a new metric for heterogeneous nucleation, the signal-to-noise ratio, and physical and chemical properties.

54 ENVIRONMENTAL SCIENCES↗

America Saves! Energizing Main Street's Small Businesses

The America Saves! Energizing Main Street Small Businesses project engaged the 1,200-member National Main Street Center (NMSC) network of downtown organizations and other local, regional, and national partners to test a methodology for sharing customized energy efficiency information with owners of commercial buildings smaller than 50,000 square feet. Led by the National Trust for Historic Preservation’s Preservation Green Lab, the project marshalled local staff and volunteers to gather voluntarily-disclosed energy use information from participating businesses. This information was analyzed using a remote auditing tool (validated by the National Renewable Energy Lab) to assess energy savings opportunities and design retrofit strategies targeting seven building types (food service and sales, attached mixed-use, strip mall, retail, office, lodging, and schools). The original project design contemplated extensive leveraging of the Green Button protocol for sharing annualized utility data at a district scale. Due the lack of adoption of Green Button, the project partners developed customized approaches to data collection in each of twelve pilot communities. The project team encountered considerable challenges in gathering standardized annual utility data from local partners. After overcoming these issues, the data was uploaded to a data storehouse. Over 450 properties were benchmarked and the remote auditing tool was tested using full building profiles and utility records for more than 100 commercial properties in three of the pilot communities. The audit tool demonstrated potential for quickly capturing, analyzing, and communicating energy efficiency opportunities in small commercial buildings. However, the project team found that the unique physical characteristics and use patterns (partial vacancy, periodic intensive uses) of small commercial buildings required more trouble-shooting and data correction than was anticipated. In addition, the project revealed that remote technology alone (such as audits) is not sufficient to convince most owners of commercial buildings or businesses to invest in energy efficiency. Additional, one-on-one personal communication is critical. A combination of technology and well-planned direct contact is likely to produce the highest rate of energy efficiency implementation in the small commercial building market sector. Note that only two of the three planned phases of this project were completed. As a result, research and testing were not fully implemented and thus all results and conclusions from the America Saves! Energizing Main Street Small Businesses project should be considered preliminary. In addition to the National Main Street Center, local organizations, and regional utilities, the America Saves! project partners included the National Renewable Energy Laboratory, Energy Center of Wisconsin (Seventh Wave), Lend Lease, Building Energy, and Energy RM.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

MCNP6 Developments: A 2022-23 Year in Review [Slides]

A separate MCNP6.3 V&V document reports on all the default calculations for all test suites is touched on. This report does not include the subcritical multiplication benchmark suite. After some additional clean-up and finalizing the post-processing and documentation steps, the subcritical multiplication benchmark suite will be released in the next version of our vnvstats repository. We tested the new HDF5 PTRAC feature in MCNP6.3 and found encouraging outcomes. Identical results coming out of the simulation with respect to the legacy PTRAC results. The overall runtime for all simulations is reduced by ~20% with the new HDF5 PTRAC capability. We consider giving the new HDF5 PTRAC features a try and using it for all subcritical multiplication and any other relevant (PTRAC) calculations.

97 MATHEMATICS AND COMPUTING↗

A SEARCH FOR L/T TRANSITION DWARFS WITH PAN-STARRS1 AND WISE. II. L/T TRANSITION ATMOSPHERES AND YOUNG DISCOVERIES

The evolution of brown dwarfs from L to T spectral types is one of the least understood aspects of the ultracool population, partly for lack of a large, well-defined, and well-characterized sample in the L/T transition. To improve the existing census, we have searched ≈28,000 deg{sup 2} using the Pan-STARRS1 and Wide-field Infrared Survey Explorer surveys for L/T transition dwarfs within 25 pc. We present 130 ultracool dwarf discoveries with estimated distances ≈9–130 pc, including 21 that were independently discovered by other authors and 3 that were previously identified as photometric candidates. Seventy-nine of our objects have near-IR spectral types of L6–T4.5, the most L/T transition dwarfs from any search to date, and we have increased the census of L9–T1.5 objects within 25 pc by over 50%. The color distribution of our discoveries provides further evidence for the “L/T gap,” a deficit of objects with (J − K){sub MKO} ≈ 0.0–0.5 mag in the L/T transition, and thus reinforces the idea that the transition from cloudy to clear photospheres occurs rapidly. Among our discoveries are 31 candidate binaries based on their low-resolution spectral features. Two of these candidates are common proper motion companions to nearby main sequence stars; if confirmed as binaries, these would be rare benchmark systems with the potential to stringently test ultracool evolutionary models. Our search also serendipitously identified 23 late-M and L dwarfs with spectroscopic signs of low gravity implying youth, including 10 with vl-g or int-g gravity classifications and another 13 with indications of low gravity whose spectral types or modest spectral signal-to-noise ratio do not allow us to assign formal classifications. Finally, we identify 10 candidate members of nearby young moving groups (YMG) with spectral types L7–T4.5, including three showing spectroscopic signs of low gravity. If confirmed, any of these would be among the coolest known YMG members and would help to determine the effective temperature at which young brown dwarfs cross the L/T transition.

79 ASTRONOMY AND ASTROPHYSICS↗

Development of Ionic Liquid-Additized, GF-5/6 Compatible Low-Viscosity Oils for Automotive Engine and Rear Axle Lubrication for 4% Improved Fuel Economy

This joint project among ORNL, GM, and DRO successfully developed ionic liquid-additized low viscosity lubricants, SAE 0W-12 and 70W-80, for both IC engine and rear axle lubrication, respectively, and demonstrated combined vehicle fuel economy improvement (FEI). Phosphonium-alkylphosphate ILs have been found to be the most effective among the candidates in protecting surface damage from both sliding wear and rolling contact fatigue, and optimal IL concentrations have been determined. The IL’s impact on three-way catalyst seems to be significantly less adverse than the conventional ZDDP. Oil formulations were optimized for both energy efficiency and wear protection. Tribological bench testing of the IL-additized low-viscosity oils showed at least 40% lower boundary friction, more than 60% lower elastohydrodynamic friction, and greater than 50% less wear compared with commercial baselines. IL-additized lubricants have demonstrated FEI by 9.9% in engine dynamometer tests under extreme conditions and 6.9% increased power output in gear rig tests. Prototype IL-additized engine oil and rear axle fluid exhibited combined FEI of 1.39% and 3.24% benchmarked against the state-of-the-art baselines in vehicle testing of fresh oils under the Federal Test Procedure (FTP) city cycle and Highway Fuel Economy Driving Schedule (HWFET), respectively. Used oil analysis suggested that the IL-additized low-viscosity oils produced less wear than the commercial baselines. The gained fundamental understanding and demonstrated FEI in this work pave the way for future development and implementation of the ionic liquid technology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Performance tests of boron-coated straw detectors with thermal and cold neutron beams

Prototypes of in-house developed boron-coated straw (BCS) detectors have been tested in the thermal and cold neutron energy ranges. Their neutron detection performance has been benchmarked against the industry standard (detector tubes filled with 3 He gas). The tests show that the BCS detectors perform near their theoretical limit regarding the detection efficiency, which is adequate for scientific instruments in the cold neutron energy range. Finally, the BCS detectors perform on par with 3 He tubes in terms of signal to noise and timing resolution, and superior regarding longitudinal spatial resolution.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Ionic Liquids as Novel Lubricants and /or Lubricant Additives

This ORNL-GM CRADA developed ionic liquids (ILs) as novel lubricants or oil additives for engine lubrication. A new group of oil-miscible ILs have been designed and synthesized with high thermal stability, non-corrosiveness, excellent wettability, and most importantly effective anti-scuffing/anti-wear and friction reduction characteristics. Mechanistic analysis attributes the superior lubricating performance of IL additives to their physical and chemical interactions with metallic surfaces. Working with a leading lubricant formulation company, the team has successfully developed a prototype low-viscosity engine oil using a phosphonium-phosphate IL as an anti-wear additive. Tribological bench tests of the IL-additized formulated oil showed 20-33% lower friction in mixed and elastohydrodynamic lubrication and 38-92% lower wear in boundary lubrication when compared with commercial Mobil 1 and Mobil Clean 5W-30 engine oils. High-temperature, high load (HTHL) full-size engine tests confirmed the excellent anti-wear performance for the IL-additized engine oil. Sequence VID engine dynamometer tests demonstrated an improved fuel economy by >2% for this IL-additized engine oil benchmarked against the Mobil 1 5W-30 oil. In addition, accelerated catalyst aging tests suggest that the IL additive may potentially have less adverse impact on three-way catalysts compared to the conventional ZDDP. Follow-on research is needed for further development and optimization of IL chemistry and oil formulation to fully meet ILSAC GF-5 specifications and further enhance the automotive engine efficiency and durability.

33 ADVANCED PROPULSION SYSTEMS↗

Finite Volume Method Based Neutronic Solvers for Steady and Transient Analysis of Molten Salt Reactors

In recent years, the coupled neutronics and thermal hydraulics analysis becomes more and more important, because it helps to identify the most relevant safety issues without conservative assumptions. In addition, such coupled simulations are essential for the generation IV advanced nuclear power systems such as molten salt reactor (MSR), because the presence of strong coupling between different phenomena, like the transport of delayed neutron precursors by the fluid flow?. Due to the different discretization methods used in reactor physics analysis and thermal-hydraulics analysis, it is hard to couple different kinds of codes. This work presents an attempt to solve the multigroup neutron diffusion equation with the finite volume method which is quite widely used for the numerical simulation of a variety of applications involving fluid flow and heat and mass transfer. The unstructured finite volume method (FVM) based on open source C++ library OpenFOAM was chosen as the development platform?. The objective of this paper is to concisely present the main features of the developed codes in terms of the theoretical background and numerical approach and as can be seen in the following sections, the calculation results obtained by the NDSFoam which solves the steady-state neutron diffusion equation and NDTFoam which solves the transient-state neutron diffusion equation are discussed and compared to the benchmark problems. In this paper, two new neutronic solvers using finite volume method have been developed to perform the neutron physics analysis. The solvers are based on OpenFOAM, and have been tested with several steady state and transient-state benchmark problems. By verifications of the codes, it is proved that the development of NDSFoam and NDTFoam are correct. In our future work, both of these two solvers will be coupled with the precompiled CFD solvers in OpenFOAM to analyze the molten salt fast reactors. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

A mass conservative well-balanced reconstruction at wet/dry interfaces for the Godunov-type shallow water model

This paper presents a novel mass conservative, positivity preserving wetting and drying treatment for Godunov-type shallow water models with second-order bed elevation discretization. The novel method allows to compute water depths equal to machine accuracy without any restrictions on the time step or any threshold that defines whether the finite volume cell is considered to be wet or dry. The resulting scheme is second-order accurate in space and keeps the C-property condition at fully flooded area and also at the wet/dry interface. For the time integration, a second-order accurate Runge–Kutta method is used. The method is tested in two well-known computational benchmarks for which an analytical solution can be derived, a C-property benchmark and in an additional example where the experimental results are reproduced. Overall, the presented scheme shows very good agreement with the reference solutions. The method can also be used in the discontinuous Galerkin method.

47 OTHER INSTRUMENTATION↗