Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

SU-E-T-53: Benchmarking a Monte Carlo Model for Patient Plane Leakage Calculations of Low Energy 6MV Unique Linacs

Purpose: To validate the physics parameters of a Monte Carlo model for patient plane leakage calculations on the 6MV Unique linac by comparing the simulations against IEC patient plane leakage measurements. The benchmarked model can further be used for shielding design optimization, to predict leakage in the proximity of intended treatment fields, reduce the system weight and cost, and improve components reliability. Methods: The treatment head geometry of the Unique linac was simulated in Geant4 (v9.4.p02 with “Opt3” standard electromagnetic physics list) based on CAD drawings of all collimation and shielding components projected from the target to the area within 2m from isocenter. A 4×4m2 scorer was inserted 1m from the target in the patient plane and multiple phase space files were recorded by performing a 40-node computing cluster simulation on the EC2 cloud. The photon energy fluence was calculated relative to the value at isocenter for a 10×10cm2 field using 10×10mm2 bins. Tungsten blocks were parked accordingly to represent MLC120. The secondary particle contamination to patient plane was eliminated by “killing” those particles prior to the primary collimator entrance using a “kill-plane”, which represented the upper head shielding components not being modeled. Both IEC patient-plane leakage and X/Y-jaws transmission were simulated. Results: The contribution of photons to energy fluence was 0.064% on average, in excellent agreement with the experimental data available at 0.5, 1.0, and 1.5m from isocenter, characterized by an average leakage of 0.045% and a maximum leakage of 0.085%. X- and Y-jaws transmissions of 0.43% and 0.44% were found in good agreement with measurements of 0.48% and 0.43%, respectively. Conclusion: A Geant4 model based on energy fluence calculations for the 6MV Unique linac was created and validated using IEC patient plane leakage measurements. The “kill-plane” has effectively eliminated electron contamination to patient plane in these simulations.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2015

Quality assurance for online adapted treatment plans: Benchmarking and delivery monitoring simulation

Purpose: An important challenge facing online adaptive radiation therapy is the development of feasible and efficient quality assurance (QA). This project aimed to validate the deliverability of online adapted plans and develop a proof-of-concept online delivery monitoring system for online adaptive radiation therapy QA. Methods: The first part of this project benchmarked automatically online adapted prostate treatment plans using traditional portal dosimetry IMRT QA. The portal dosimetry QA results of online adapted plans were compared to original (unadapted) plans as well as randomly selected prostate IMRT plans from our clinic. In the second part, an online delivery monitoring system was designed and validated via a simulated treatment with intentional multileaf collimator (MLC) errors. This system was based on inputs from the dynamic machine information (DMI), which continuously reports actual MLC positions and machine monitor units (MUs) at intervals of 50 ms or less during delivery. Based on the DMI, the system performed two levels of monitoring/verification during the delivery: (1) dynamic monitoring of cumulative fluence errors resulting from leaf position deviations and visualization using fluence error maps (FEMs); and (2) verification of MLC positions against the treatment plan for potential errors in MLC motion and data transfer at each control point. Validation of the online delivery monitoring system was performed by introducing intentional systematic MLC errors (ranging from 0.5 to 2 mm) to the DMI files for both leaf banks. These DMI files were analyzed by the proposed system to evaluate the system’s performance in quantifying errors and revealing the source of errors, as well as to understand patterns in the FEMs. In addition, FEMs from 210 actual prostate IMRT beams were analyzed using the proposed system to further validate its ability to catch and identify errors, as well as establish error magnitude baselines for prostate IMRT delivery. Results: Online adapted plans were found to have similar delivery accuracy in comparison to clinical IMRT plans when validated with portal dosimetry IMRT QA. FEMs for the simulated deliveries with intentional MLC errors exhibited distinct patterns for different MLC error magnitudes and directions, indicating that the proposed delivery monitoring system is highly specific in detecting the source of errors. Implementing the proposed QA system for online adapted plans revealed excellent delivery accuracy: over 99% of leaf position differences were within 0.5 mm, and >99% of pixels in the FEMs had fluence errors within 0.5 MU. Patterns present in the FEMs and MLC control point analysis for actual patient cases agreed with the error pattern analysis results, further validating the system’s ability to reveal and differentiate MLC deviations. Calculation of the fluence map based on the DMI was performed within 2 ms after receiving each DMI input. Conclusions: The proposed online delivery monitoring system requires minimal additional resources and time commitment to the current clinical workflow while still maintaining high sensitivity to leaf position errors and specificity to error types. The presented online delivery monitoring system therefore represents a promising QA system candidate for online adaptive radiation therapy.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2019

Benchmark analytical solutions for steady state high frequency broadband sound fields in three rectangular enclosures

Analytical solutions are presented for broadband sound fields in three rectangular enclosures with absorption applied on the floor and ceiling, rigid sidewalls, and a vertically oriented dipole source. The solutions are intended to serve as benchmarks that can be used to assess the performance of broadband techniques, particularly energy-based methods, in a relatively straightforward configuration with precisely specified boundary conditions. A broadband Helmholtz solution is developed using a frequency-by-frequency modal approach to determine the exact band averaged mean-square pressures along spatial trajectories within each enclosure. Due to the specific choice of enclosure configuration and absorption distribution, an approximate specular solution can be obtained through a summation of uncorrelated image sources. Comparisons between the band averaged Helmholtz solution and the uncorrelated image solution reveal excellent agreement for a wide range of absorption levels and improve the understanding of correlation effects in broadband sound fields. In conclusion, a boundary element solution with diffuse boundaries is also presented, which produces consistently higher mean-square pressures in comparison with the specular solution, emphasizing the careful attention that must be placed on correctly modeling reflecting boundary conditions and demonstrating the errors that can result from assuming a Lambertian surface.

42 ENGINEERING↗
DOE OSTI2014

Benchmarking of Methods for Genomic Taxonomy

One of the first issues that emerges when a prokaryotic organism of interest is encountered is the question of what it is—that is, which species it is. The 16S rRNA gene formed the basis of the first method for sequence-based taxonomy and has had a tremendous impact on the field of microbiology. Nevertheless, the method has been found to have a number of shortcomings. In this paper, we trained and benchmarked five methods for whole-genome sequence-based prokaryotic species identification on a common data set of complete genomes: (i) SpeciesFinder, which is based on the complete 16S rRNA gene; (ii) Reads2Type that searches for species-specific 50-mers in either the 16S rRNA gene or the gyrB gene (for the Enterobacteraceae family); (iii) the ribosomal multilocus sequence typing (rMLST) method that samples up to 53 ribosomal genes; (iv) TaxonomyFinder, which is based on species-specific functional protein domain profiles; and finally (v) KmerFinder, which examines the number of cooccurring k-mers (substrings of k nucleotides in DNA sequence data). The performances of the methods were subsequently evaluated on three data sets of short sequence reads or draft genomes from public databases. In total, the evaluation sets constituted sequence data from more than 11,000 isolates covering 159 genera and 243 species. Our results indicate that methods that sample only chromosomal, core genes have difficulties in distinguishing closely related species which only recently diverged. Finally, the KmerFinder method had the overall highest accuracy and correctly identified from 93% to 97% of the isolates in the evaluations sets.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2015

New LHC benchmarks for the $\mathcal{CP}$-conserving two-Higgs-doublet model

We introduce a strategy to study the parameter space of the general, CP-conserving, two-Higgs-doublet Model (2HDM) with a softly broken $\mathbb Z_{2}$-symmetry by means of a new “hybrid” basis. In this basis the input parameters are the measured values of the mass of the observed Standard Model (SM)-like Higgs boson and its coupling strength to vector boson pairs, the mass of the second CP-even Higgs boson, the ratio of neutral Higgs vacuum expectation values, and three additional dimensionless parameters. Here, using the hybrid basis, we present numerical scans of the 2HDM parameter space where we survey available parameter regions and analyze model constraints. From these results, we define a number of benchmark scenarios that capture different aspects of non-standard Higgs phenomenology that are of interest for future LHC Higgs searches.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Benchmarking a neutral-atom quantum computer

In this study, we simulated the algorithmic performance of a small neutral atom quantum computer and compared its performance when operating with all-to-all versus nearest-neighbor connectivity. This comparison was made using a suite of algorithmic benchmarks developed by the Quantum Economic Development Consortium. Circuits were simulated with a noise model consistent with experimental data from [Nature 604, 457 (2022)]. We find that all-to-all connectivity improves simulated circuit fidelity by [Formula: see text]–[Formula: see text], compared to nearest-neighbor connectivity.

Computer Science↗
DOE OSTI2023

Implementing and Benchmarking the Locally Competitive Algorithm on the Loihi 2 Neuromorphic Processor

Neuromorphic processors have garnered considerable interest in recent years for their potential in enabling energy-efficient and high-speed computing. The Locally Competative Algorithm (LCA) has been utilized for power efficient sparse coding on neuromophic processors, including the first Loihi processor \cite{appletospikes, loihi1}. With the Loihi 2 processor enabling custom neuron models and graded spike communication, more complex implementations of LCA are possible \cite{loihi2}. We present a new implementation of LCA designed for the Loihi 2 processor and perform an initial set of benchmarks comparing it to LCA on CPU and GPU devices. In these experiments LCA on Loihi 2 is faster and orders of magnitude more efficient, while maintaining similar reconstruction quality. We find this performance improvement increases as the LCA parameters are tuned towards greater representation sparsity. Our study highlights the potential of neuromorphic processors, particularly Loihi 2, in enabling intelligent,autonomous, real-time processing on small robots, satellite where there are strict SWaP (small, lightweighr, and low-power) requirement. By demonstrating the superior performance of LCA on Loihi 2 compared to conventional computing device, our study suggests that Loihi 2 could be a valuable tool in advancing these types of applications. Overall, our study highlights the potential of neuromorphic processors for efficient and accurate data processing on resource-constrained devices.

Parpart, Gavin G.↗
DOE OSTI2026

Did You Win the GPU Cloud Lottery? Benchmarking from TFLOPS to Tokens/$

Cloud GPUs are commonly assumed to deliver consistent performance for a given GPU model. This assumption does not always hold: cloud providers employ diverse system configurations and virtualization mechanisms, and GPUs themselves exhibit non-negligible manufacturing variability (the silicon lottery). In this work, we present a large-scale measurement study of GPU performance variability across 11 cloud providers, covering over 3,500 physical GPUs and 6,800 benchmark runs. Our hierarchical analysis shows that while execution-level variation stays below 9%, performance varies by up to 38% across devices and providers for the same GPU model. Regression analysis indicates that driver- and OS-related software factors contribute less than 1% of the variance; instead, silicon lottery effects dominate observed performance variation, and cloud providers further amplify them through persistent, systematic second-order effects.

Slynko, Platon [Silicon Data, New York, USA] (ORCI↗
DOE OSTI2022

Benchmarking Solid-State Batteries Containing Sulfide Separators: Effects of Electrode Composition and Stack Pressure

Integrating sulfide separators into solid-state batteries (SSBs) containing high energy cathodes typically requires one or more materials and engineering solutions including: (i) applying interfacial coatings to mitigate electrolyte decomposition, (ii) applying high stack pressures to form robust solid-solid contacts, and (iii) using alloying anodes to avoid Li dendrite formation. Despite the promise of these approaches, a lack of standardized testing protocols makes it difficult to directly compare results among different studies. To address this problem, the present work benchmarks the performance of SSBs containing β-Li 3 PS 4 (LPS) separators and composite cathodes. By systematically varying the anode/cathode composition and stack pressure, this work demonstrates that cathode design is a major bottleneck for solid-state cells cycled at low rates (<100 µA cm -2 ). Operando stack pressure measurements show that, while mechanical confinement generally promotes higher active material utilization and cycling stability, this strategy alone does not address interfacial reactivity between LPS and high voltage cathodes. Here, these results also demonstrate that stress evolution during cycling is dominated by volume changes at the Li metal anode. Finally, we show that FeS 2 cathodes with moderate operating voltages (<3 V vs Li/Li + ) exhibit superior cycling performance compared to high voltage cathodes by facilitating formation of stable cathode/electrolyte interfaces.

25 ENERGY STORAGE↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗
DOE OSTI2014

Python/Lua Benchmarks

This is an adaptation of the pre-existing Scimark benchmark code to a variety of Python and Lua implementations. It also measures performance of the Fparser expression parser and C and C++ code on a variety of simple scientific expressions.

Busby, L.↗
DOE OSTI2016

Processor Emulator with Benchmark Applications

A processor emulator and a suite of benchmark applications have been developed to assist in characterizing the performance of data-centric workloads on current and future computer architectures. Some of the applications have been collected from other open source projects. For more details on the emulator and an example of its usage, see reference [1].

Lloyd, G. Scott↗
DOE OSTI2018

RMA-MT Benchmarks

The RMA-MT benchmarks are a suite of performance tests for multi-threaded Message Passing Interface (MPI) Remote Memory Access (RMA) implementations. The suite includes tests for bandwidth and latency with any number of threads. This suite was originally developed and release by Sandia National Laboratory. LANL modifications to the suite include a complete re-write of much of the source code, bi-directional bandwidth, and message rate tests.

Hjelm, Nathan↗
DOE OSTI2018

OpenOA™, FKA: Wind Plant Performance Project (WP3) Benchmarking

This library provides a framework for working with large timeseries data from wind plants, such as SCADA. Its development has been motivated by the WP3 Benchmarking (PRUF) project, which aims to provide a reference implementation for plant-level performance assessment.

Perr-Sauer, Jordan↗
DOE OSTI2019

Benchmark Algorithm for RadioNuclide Identification v1.0

Benchmark Algorithm for Radio-Nuclide Identification (BARNI) is an automatic radionuclide identification tool for use with gamma-ray spectra. BARNI is designed to serve as a comparative tool for commercial vendor algorithms, such as those found in RadioIsotope Identification Devices (RIID). The tool incorporates an automated training routine allowing BARNI's classifier to be trained for a variety of detectors. The identification algorithm uses a peak search, the results of which are used to build features that are subsequently used for training a classifier. The user provided spectral templates allow BARNI to be trained for any detector of the user's choice. Is

NELSON, KARLE.↗
DOE OSTI2014

Benchmarking and Energy Saving Tool for Low Carbon Cities (BEST Cities)

BEST-Cities is designed to provide city authorities with strategies they can follow to reduce city-wide carbon dioxide (CO2) and methane (CH4) emissions. The tool quickly assesses local energy use and energy-related CO2 emissions across nine sectors (i.e., industry, public and commercial buildings, residential buildings, transportation, power and heat, street lighting, water & wastewater, solid waste, and urban green space), giving officials a comprehensive perspective on their local carbon performance. Cities can also use the tool to benchmark their energy and emissions performance to other cities inside and outside China, and identify those sectors with the greatest energy saving and emissions reduction potential.

Ede, Ben↗