SEARCH · Search NASA

Results for “Performance Benchmarking”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,081 records · Page 60

All sources NASA NTRS DOE OSTI

DOE OSTI2019

Workshop on Advanced Computing for Connected and Automated Vehicles

To safely and reliably operate without a human driver, connected and automated vehicles (CAVs) require more advanced computing hardware and software solutions than are implemented today in vehicles that provide driver-assistance features. A workshop was held to discuss advanced microelectronics and computing approaches that can help meet future energy and computational requirements for CAVs. Workshop questions were posed as follows: will highly automated vehicles be viable with conventional computing approaches or will they require a step-change in computing; what are the energy requirements to support on-board sensing and computing; and what advanced computing approaches could reduce the energy requirements while meeting their computational requirements? At present, there is no clear convergence in the computing architecture for highly automated vehicles. However, workshop participants generally agreed that there is a need to improve the computing performance per watt by at least 10x to advance the degree of automation. Participants suggested that DOE and the national laboratories could play a near-term role by developing benchmarks for determining and comparing CAV computing performance, developing public data sets to support algorithm and software development, and contributing precompetitive advancements in energy efficient computing.

33 ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2023

Engineering Redox Flow Battery Electrodes with Spatially Varying Porosity Using Non‐Solvent‐Induced Phase Separation

Redox flow batteries (RFBs) are a promising electrochemical platform for efficiently and reliably delivering electricity to the grid. Within the RFB, porous carbonaceous electrodes facilitate electrochemical reactions and distribute the flowing electrolyte. Tailoring electrode microstructure and surface area can improve RFB performance, lowering costs. Electrodes with spatially varying porosity may increase electrode utilization and provide surface area in reaction‐limited zones; however, the efficacy of such designs remains an open area of research. Herein, a non‐solvent‐induced phase‐separation (NIPS) technique that enables the reproducible synthesis of macrovoid‐free electrodes with well‐defined across‐thickness porosity gradients is described. The monotonically varying porosity profile is quantified and the physical properties and surface chemistries of porosity‐gradient electrodes are compared with macrovoid‐containing electrode, also synthesized by NIPS. Then, the electrochemical and fluid dynamic performance of the porosity‐gradient electrodes is evaluated, exploring the effect of changing the direction of the porosity gradient and benchmarking against the macrovoid‐containing electrode. Lastly, the performance is examined in a vanadium RFB, finding that the porosity‐gradient electrode outperforms the macrovoid electrode, is independent of gradient direction, and performs favorably compared to advanced electrodes in the contemporary literature. It is anticipated that the approach motivates further exploration of microstructurally tailored electrodes in electrochemical systems.

25 ENERGY STORAGE↗

DOE OSTI2016

Identification of fuel cycle simulator functionalities for analysis of transition to a new fuel cycle

Dynamic fuel cycle simulation tools are intended to model holistic transient nuclear fuel cycle scenarios. As with all simulation tools, fuel cycle simulators require verification through unit tests, benchmark cases, and integral tests. Model validation is a vital aspect as well. Although compara-tive studies have been performed, there is no comprehensive unit test and benchmark library for fuel cycle simulator tools. The objective of this paper is to identify the must test functionalities of a fuel cycle simulator tool within the context of specific problems of interest to the Fuel Cycle Options Campaign within the U.S. Department of Energy s Office of Nuclear Energy. The approach in this paper identifies the features needed to cover the range of promising fuel cycle options identified in the DOE-NE Fuel Cycle Evaluation and Screening (E&S) and categorizes these features to facilitate prioritization. Features were categorized as essential functions, integrating features, and exemplary capabilities. One objective of this paper is to propose a library of unit tests applicable to each of the essential functions. Another underlying motivation for this paper is to encourage an international dialog on the functionalities and standard test methods for fuel cycle simulator tools.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2012

Advanced Computational Methods for Security Constrained Financial Transmission Rights: Structure and Parallelism

Financial Transmission Rights (FTRs) help power market participants reduce price risks associated with transmission congestion. FTRs are issued based on a process of solving a constrained optimization problem with the objective to maximize the FTR social welfare under power flow security constraints. Security constraints for different FTR categories (monthly, seasonal or annual) are usually coupled and the number of constraints increases exponentially with the number of categories. Commercial software for FTR calculation can only provide limited categories of FTRs due to the inherent computational challenges mentioned above. In this paper, a novel non-linear dynamical system (NDS) approach is proposed to solve the optimization problem. The new formulation and performance of the NDS solver is benchmarked against widely used linear programming (LP) solvers like CPLEX™ and tested on large-scale systems using data from the Western Electricity Coordinating Council (WECC). The NDS is demonstrated to outperform the widely used CPLEX algorithms while exhibiting superior scalability. Furthermore, the NDS based solver can be easily parallelized which results in significant computational improvement.

Environmental Molecular Sciences Laboratory↗

DOE OSTI2020

Pulsed Thermal Tomography Nondestructive Examination of Additively Manufactured Reactor Materials (Second Annual Progress Report)

Additive manufacturing (AM) is an emerging method for cost-efficient fabrication of nuclear reactor parts. AM of metallic structures for nuclear energy applications is currently based on laser powder bed fusion (LPBF) process, which can introduce internal material flaws, such as pores and anisotropy. Integrity of AM structures needs to be evaluated nondestructively because material flaws could lead to premature failures due to exposure to high temperature, radiation and corrosive environment in a nuclear reactor. Thermal tomography (TT) provides a capability for non-destructive evaluation of sub-surface defects in arbitrary size structures. Thermal tomography is a computational method for heat diffusion-based imaging of solids, which provides 3D visualization of data from flash thermography measurements. We investigate thermal tomography imaging and nondestructive evaluation of stainless steel and nickel super alloy metallic structures produced with laser powder bed fusion (LPBF) additive manufacturing (AM) process. Metallic structures produced with LPBF contain defects, and there are limited capabilities to evaluate these structures non-destructively. Thermal tomography reconstruction of 3D apparent spatial effusivity provides information about AM structure geometry and internal material flaws. We study performance of thermal tomography in imaging of metallic structures through COMSOL computer simulations of transient heat transfer, and through reconstruction of data obtained from experimental measurements. Reconstruction of internal defects is investigated using a stainless steel 316L specimen with flat bottom hole (FBH) indentations, and Inconel 718 plate produced with laser powder bed fusion (LPBF) method, which contains imprinted hemispherical shape low density regions containing non-sintered metallic powder. The FBH’s have the same sizes as the imprinted defects in the LPBF specimens, but offer better imaging contrast. Thermal tomography reconstructions provide visualizations of internal defects, and allow for estimation of their sizes and locations. Detection sensitivity of TT is limited by noises. We investigate separation of signal from noise in thermography images using several machine learning (ML) methods, including new spatio-temporal blind source separation (STBSS) and spatio-temporal sparse dictionary learning (STSDL) methods. Performance of the ML methods is benchmarked using thermography data obtained from imaging stainless steel 316L and Inconel 718 specimens produced LPBF method with imprinted calibrated porosity defects. The ML methods are ranked by F-score and execution runtime. Finally, we investigate TT of AM stainless steel 316L specimen with imprinted internal porosity defects using relatively low-cost, small form factor infrared (IR) camera based on uncooled micro bolometer detector. Sparse coding related K-means singular value decomposition (SVD) machine learning, image processing algorithms are developed to improve quality of TT images through removal of Additive white Gaussian noise without blurring the images. Following initial qualification of an AM component for deployment in a nuclear reactor, a compact TT can also be used for in-service nondestructive evaluation (NDE). With capability to perform in-service NDE of the AM component lifecycle, TT data can be used for development of a component digital twin.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2015

SU-E-T-148: Benchmarks and Pre-Treatment Reviews: A Study of Quality Assurance Effectiveness

Purpose: To determine the impact benchmarks and pre-treatment reviews have on improving the quality of submitted clinical trial data. Methods: Benchmarks are used to evaluate a site’s ability to develop a treatment that meets a specific protocol’s treatment guidelines prior to placing their first patient on the protocol. A pre-treatment review is an actual patient placed on the protocol in which the dosimetry and contour volumes are evaluated to be per protocol guidelines prior to allowing the beginning of the treatment. A key component of these QA mechanisms is that sites are provided timely feedback to educate them on how to plan per the protocol and prevent protocol deviations on patients accrued to a protocol. For both benchmarks and pre-treatment reviews a dose volume analysis (DVA) was performed using MIM softwareTM. For pre-treatment reviews a volume contour evaluation was also performed. Results: IROC Houston performed a QA effectiveness analysis of a protocol which required both benchmarks and pre-treatment reviews. In 70 percent of the patient cases submitted, the benchmark played an effective role in assuring that the pre-treatment review of the cases met protocol requirements. The 35 percent of sites failing the benchmark subsequently modified there planning technique to pass the benchmark before being allowed to submit a patient for pre-treatment review. However, in 30 percent of the submitted cases the pre-treatment review failed where the majority (71 percent) failed the DVA. 20 percent of sites submitting patients failed to correct their dose volume discrepancies indicated by the benchmark case. Conclusion: Benchmark cases and pre-treatment reviews can be an effective QA tool to educate sites on protocol guidelines and to minimize deviations. Without the benchmark cases it is possible that 65 percent of the cases undergoing a pre-treatment review would have failed to meet the protocols requirements.Support: U24-CA-180803.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2012

Benchmark specifications for EBR-II shutdown heat removal tests

Argonne National Laboratory (ANL) is hosting an IAEA-coordinated research project on benchmark analyses of sodium-cooled fast reactor passive safety tests performed at the Experimental Breeder Reactor-II (EBR-II). The benchmark project involves analysis of a protected and an unprotected loss of flow tests conducted during an extensive testing program within the framework of the U.S. Integral Fast Reactor program to demonstrate the inherently safety features of EBR-II as a pool-type, sodium-cooled fast reactor prototype. The project is intended to improve the participants' design and safety analysis capabilities for sodium-cooled fast reactors through validation and qualification of safety analysis codes and methods. This paper provides a description of the EBR-II tests included in the program, and outlines the benchmark specifications being prepared to support the IAEA-coordinated research project. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

On the predictability of land surface fluxes from meteorological variables

Previous research has demonstrated that land surface models (LSMs) are performing poorly when compared with relatively simple empirical models over a wide range of metrics and environments. Atmospheric driving data appear toprovide information about land surface fluxes that LSMs are not fully utilising. Here, we further quantify the information available in the meteorological forcing data that are used by LSMs for predicting land surface fluxes, by interrogating FLUXNET data, and extending the benchmarking methodology used in previous experiments. We show that substantial performance improvement is possible for empirical models using meteorological data alone, with no explicit vegetation or soil properties, thus setting lower bounds on a priori expectations on LSM performance. The process also identifies key meteorological variables that provide predictive power. We provide an ensemble of empirical benchmarks that are simple to reproduce and provide a range of behaviours and predictive performance, acting as a baseline benchmark set for future studies. We reanalyse previously published LSM simulations and show that there is more diversity between LSMs than previously indicated, although it remains unclear why LSMs are broadly performing so much worse than simple empirical models.

58 GEOSCIENCES↗

DOE OSTI2017

Active control of bright electron beams with RF optics for femtosecond microscopy

A frontier challenge in implementing femtosecond electron microscopy is to gain precise optical control of intense beams to mitigate collective space charge effects for significantly improving the throughput. Here, we explore the flexible uses of an RF cavity as a longitudinal lens in a high-intensity beam column for condensing the electron beams both temporally and spectrally, relevant to the design of ultrafast electron microscopy. Through the introduction of a novel atomic grating approach for characterization of electron bunch phase space and control optics, we elucidate the principles for predicting and controlling the phase space dynamics to reach optimal compressions at various electron densities and generating conditions. We provide strategies to identify high-brightness modes, achieving ∼100 fs and ∼1 eV resolutions with 106 electrons per bunch, and establish the scaling of performance for different bunch charges. These results benchmark the sensitivity and resolution from the fundamental beam brightness perspective and also validate the adaptive optics concept to enable delicate control of the density-dependent phase space structures to optimize the performance, including delivering ultrashort, monochromatic, high-dose, or coherent electron bunches.

43 PARTICLE ACCELERATORS↗

DOE OSTI2021

Nuclear Data Sensitivity Study for the EBR-II Fast Reactor Benchmark Using SCALE with ENDF/B-VII.1 and ENDF/B-VIII.0

The EBR-II benchmark, which was recently included in the International Handbook of Evaluated Reactor Physics Benchmark Experiments, served as a basis for assessing the performance of the SCALE code system for fast reactor analyses. A reference SCALE model was developed based on the benchmark specifications. Great agreement was observed between the eigenvalue calculated with this SCALE model and the benchmark eigenvalue. To identify potential gaps and uncertainties of nuclear data for the simulation of various quantities of interest in fast spectrum systems, sensitivity and uncertainty analyses were performed for the eigenvalue, reactivity effects, and the radial power profile of EBR-II using the two most recent ENDF/B nuclear data library releases. While the nominal results are consistent between the calculations with the different libraries, the uncertainties due to nuclear data vary significantly. The major driver of observed uncertainties is the uncertainty of the 235 U ( n,γ ) reaction. Since the uncertainty of this reaction is significantly reduced in the ENDF/B-VIII.0 library compared to ENDF/B-VII.1, the obtained output uncertainties tend to be smaller in ENDF/B-VIII.0 calculations, although the decrease is partially compensated by increased uncertainties in 235 U fission and ν ¯ .

42 ENGINEERING↗

DOE OSTI2017

Enabling Diverse Software Stacks on Supercomputers using High Performance Virtual Clusters.

While large-scale simulations have been the hallmark of the High Performance Computing (HPC) community for decades, Large Scale Data Analytics (LSDA) workloads are gaining attention within the scientific community not only as a processing component to large HPC simulations, but also as standalone scientific tools for knowledge discovery. With the path towards Exascale, new HPC runtime systems are also emerging in a way that differs from classical distributed com- puting models. However, system software for such capabilities on the latest extreme-scale DOE supercomputing needs to be enhanced to more appropriately support these types of emerging soft- ware ecosystems. In this paper, we propose the use of Virtual Clusters on advanced supercomputing resources to enable systems to support not only HPC workloads, but also emerging big data stacks. Specifi- cally, we have deployed the KVM hypervisor within Cray's Compute Node Linux on a XC-series supercomputer testbed. We also use libvirt and QEMU to manage and provision VMs directly on compute nodes, leveraging Ethernet-over-Aries network emulation. To our knowledge, this is the first known use of KVM on a true MPP supercomputer. We investigate the overhead our solution using HPC benchmarks, both evaluating single-node performance as well as weak scaling of a 32-node virtual cluster. Overall, we find single node performance of our solution using KVM on a Cray is very efficient with near-native performance. However overhead increases by up to 20% as virtual cluster size increases, due to limitations of the Ethernet-over-Aries bridged network. Furthermore, we deploy Apache Spark with large data analysis workloads in a Virtual Cluster, ef- fectively demonstrating how diverse software ecosystems can be supported by High Performance Virtual Clusters.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

Are we witnessing the spectre of an HPC meltdown?: Are We Witnessing the Spectre of an HPC Meltdown?

In this study, we measure and analyze the performance observed when running applications and benchmarks before and after the Meltdown and Spectre fixes have been applied to the Cray supercomputers and supporting systems at the Oak Ridge Leadership Computing Facility (OLCF). Of particular interest is the effect of these fixes on applications selected from the OLCF portfolio when running at scale. This comprehensive study presents results from experiments run on Titan, Eos, Cumulus, and Percival supercomputers at the OLCF. Lastly, the results from this study are useful for HPC users running on Cray supercomputers and serve to better understand the impact that these two vulnerabilities have on diverse HPC workloads at scale.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Curation of Ground-Truth Validated Benchmarking Datasets for Fault Detection & Diagnostics Tools

Fault detection and diagnostics (FDD) analytical tools for heating, ventilation and air conditioning (HVAC) systems represent one of the most active areas of smart building technology development. A diversity of techniques is used for FDD analytics, spanning physical models, black box, and rule-based approaches, and researchers continuously strive to develop improved algorithms. With FDD algorithm numbers now in the hundreds, there is a need for performance evaluation of these algorithms in order to assess improvements, improve costeffectiveness, and to prioritize investment in the further development of these technologies. A persistent challenge of FDD advance has been the lack of common datasets to benchmark the performance accuracy of FDD algorithms. This paper summarizes the successful curation of HVAC operational data, paired with validated ground-truth information regarding the presence and absence of faults. The current dataset, consisting of both simulation and experimental data, will evolve to include a larger set of HVAC systems with the objective of creating the largest publicly available dataset to be used by FDD developers, users, and researchers to compare and contrast performance accuracy across FDD algorithms, helping to drive improvements that will spur greater market adoption of FDD tools. Furthermore, in order to avoid previously observed issues with contributed datasets and ensure high quality and consistency of future submissions, the development of data validation and ground-truth assessment protocol is detailed in this study.

Casillas, Armando↗

DOE OSTI2017

Complex relationship between seasonal streamflow forecast skill and value in reservoir operations

Considerable research effort has recently been directed at improving and operationalising ensemble seasonal streamflow forecasts. Whilst this creates new opportunities for improving the performance of water resources systems, there may also be associated risks. Here, we explore these potential risks by examining the sensitivity of forecast value (improvement in system performance brought about by adopting forecasts) to changes in the forecast skill for a range of hypothetical reservoir designs with contrasting operating objectives. Forecast-informed operations are simulated using rolling horizon, adaptive control and then benchmarked against optimised control rules to assess performance improvements. Results show that there exists a strong relationship between forecast skill and value for systems operated to maintain a target water level. But this relationship breaks down when the reservoir is operated to satisfy a target demand for water; good forecast accuracy does not necessarily translate into performance improvement. We show that the primary cause of this behaviour is the buffering role played by storage in water supply reservoirs, which renders the forecast superfluous for long periods of the operation. System performance depends primarily on forecast accuracy when critical decisions are made – namely during severe drought. As it is not possible to know in advance if a forecast will perform well at such moments, we advocate measuring the consistency of forecast performance, through bootstrap resampling, to indicate potential usefulness in storage operations. Our results highlight the need for sensitivity assessment in value-of-forecast studies involving reservoirs with supply objectives.

42 ENGINEERING↗

DOE OSTI2022

Generating Models of the Flattop Critical Assembly for Benchmark Experiments with Python

Los Alamos National Laboratory has been performing nuclear criticality experiments since 1946 at the Pajarito site, starting the Los Alamos Critical Experiments Facility in 1948. A transition period occurred between 2004 and 2011 as operations moved to the National Criticality Experiments Research Center (NCERC), where criticality experiments are now performed. Criticality experiments are essential for determination and verification of nuclear data used in calculations and modeling—such as radiation transport codes—throughout the industry, enhancing nuclear criticality safety. In addition to nuclear data validation and benchmarking, the remotely operated critical assemblies at NCERC are used for a variety of experiments and training classes supporting criticality safety.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2022

Effects of Chlorine Capture and a Proposed Density Law on the Reactivity of Plutonium Solution Systems

During fissionable material processing, all normal and credible abnormal conditions must remain safely subcritical. Nuclear Criticality Safety (NCS) uses a number of methods to determine subcriticality, one of which is the use of neutron transport codes such as MCNP6. In order to create models for use with MCNP6, both the geometry and materials in fissionable material processes must be known, or assumptions must be made and quantified for the impact to bias. One of the systems with a significant amount of bias due to material modeling assumptions is in the area of aqueous plutonium processing. These solutions are typically plutonium nitrate solutions or plutonium chloride solutions, which are modeled as fictitious plutonium metal-water mixtures because little is known about the actual density of the solution and there is no current predictive capability approved for use at Los Alamos National Laboratory (LANL) for modeling them. This research is currently underway to fill the gap and develop an algorithm for use with MCNP6 to model the density of plutonium chloride solutions. The method is to be validated with experimental data for density, and also validated with critical experiments using MCNP6. Note that the Chlorine Worth Study (CWS) was performed in December 2021 to help bridge the gap in chlorine data for critical experiments, and is currently awaiting International Criticality Safety Benchmark Evaluation Project (ICSBEP) review. This study was performed by LANL at the National Criticality Experiments Research Center (NCERC) at the Nevada National Security Site (NNSS). Additional information regarding this experiment may currently be found in LA-UR- 22-29180. Additionally, the Chemistry-Actinide Analytical Chemistry (C-AAC) at LANL has performed a number of solution density measurements for PuCl 3 -HC 1 -H 2 O, allowing for such data be used to create a semi-empirical density via the Pitzer method. The published dataset for the measurements is documented in LA-UR-22-25454. This method has already been tested successfully for aqueous plutonium nitrate solutions in SCALE. Current solution density measurements exist of plutonium concentrations of 0-~142g/L, all at 2M HC1, at temperatures 20-40°C. Additional data was taken for HC1-corrected density values, which essentially mimics the data for a pure PuCl x -water solution. The calculations in this report aim to support the current research by demonstrating the difference in system reactivity for the current modeling method when compared to the new proposed modeling with a density law implementation, which is being written as a Python tool to be used with MCNP6.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗