Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmarking Software”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Gaussian Process Optimization of Sensitivity-Based Similarity Metrics between New Nuclear Applications and New/Existing Benchmarks

Confidently designing safe, new nuclear criticality experiments requires expert judgement, which could take years of experience. Sensitivity/uncertainty (S/U) analysis can be utilized by less experienced individuals to conservatively estimate uncertainties in important parameters, such as k eff , in newly proposed nuclear applications. This type of analysis relies on matching new nuclear applications with existing benchmark experiments. The Whisper-1.1 software package included in MCNP6.2 ®1 contains more than 1,100 International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmarks. These benchmarks however rarely match new nuclear applications. The number of benchmarks available to match a given set of materials or geometric configurations varies significantly. Furthermore, recently performed benchmark experiments may not have had enough time to be properly documented and published. Benchmarks are vital for determining the accuracy of nuclear data and can assist nuclear physics and evaluators in improving nuclear data libraries. Exhaustively exploring the parameter space using simulations with software such as MCNP is too computationally expensive. In this work, Gaussian process optimization was implemented to reduce the number of simulations needed for optimization over multiple parameters. This optimization scheme was designed to selectively generate new benchmarks with high sensitivity-based similarity metrics to user-defined nuclear applications. Two benchmark models of spherically nested shells containing plutonium, uranium, tantalum, and water were used in the optimization to match an application containing plutonium plates, stacked in a tantalum reflector, surrounded by water.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Feature review of photovoltaic modeling software utilizing blind performance assessment

While confidence in photovoltaic (PV) modeling software has always been essential, the rapid pace of new PV plant developments makes accuracy and credibility more critical than ever. Independent assessments, particularly through blind modeling comparisons, are therefore necessary to ensure unbiased benchmarking across PV modeling software. Previous studies have been limited by a narrow range of models compared, anonymized results, or system size. This study presents results from the first-ever onymous blind modeling comparison, evaluated using both lab- and utility-scale fixed-tilt, monofacial, south-facing systems at sub-hourly time intervals. Seven commercially used PV software tools were compared: 3E SynaptiQ, PlantPredict, PVsyst, RatedPower, SAM, SolarFarmer, and Solargis Evaluate. Predictions were submitted directly by software representatives, providing unique insights into each software’s implementation and resulting prediction behavior. Notable features, including plane-of-array (POA) transposition model, module temperature model, shading model, and performance model were analyzed and compared. Four summary tables compile these features of the software, serving as a resource to help users understand the methodological differences and select the most suitable software for their applications. The software tools show deviations from mean error in annual yield up to 2.5 % in the lab-scale system, increasing to 6.0 % for the utility-scale system. These differences arise from a combination of user decisions and the inherent behavior of the software, indicating the need for continuous and rigorous validation of modeling methods using these software tools against complex, real-world systems.

14 SOLAR ENERGY↗

DFT-based QM/MM with Particle-Mesh Ewald for Direct, Long-Range Electrostatic Embedding

In this work, we present a DFT-based, QM/MM implementation with long-range electrostatic embedding achieved by direct real-space integration of the particle mesh Ewald (PME) computed electrostatic potential. The key transformation is the interpolation of the electrostatic potential from the PME grid to the DFT quadrature grid, from which integrals are easily evaluated utilizing standard DFT machinery. We provide benchmarks of the numerical accuracy with choice of grid size and real-space corrections, and demonstrate that good convergence is achieved while introducing nominal computational overhead. Furthermore, the approach requires only small modification to existing software packages, as is demonstrated with our implementation in the OpenMM and Psi 4 software. After presenting convergence benchmarks, we evaluate the importance of long-range electrostatic embedding in three solute/solvent systems modeled with QM/MM. Water and BMIM/BF 4 ionic liquid were considered as "simple" and "complex" solvents respectively, with water and p-phenylenediamine (PPD) solute molecules treated at QM level of theory. While electrostatic embedding with standard real-space truncation may introduce negligible error for simple systems such as water solute in water solvent, errors become more significant when QM/MM is applied to complex solvents such as ionic liquids. An extreme example is the electrostatic embedding energy for oxidized PPD in BMIM/BF 4 for which real-space truncation produces severe error even at 2-3 nm cutoff distances. This latter example illustrates that utilization of QM/MM to compute redox potentials within concentrated electrolytes/ionic media requires carefully chosen long-range electrostatic embedding algorithms, with our presented algorithm providing a general and robust approach.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A flux-matched gyrokinetic analysis of DIII-D L-mode turbulence

Previous nonlinear gyrokinetic simulations of specific DIII-D L-mode cases have been found to significantly underpredict the ion heat transport and associated density and temperature fluctuation levels by up to almost one of order of magnitude in the outer-core domain, i.e., roughly in the last third of the minor radius. Since then, this so-called shortfall issue has been subject to various speculations on possible reasons and furthermore motivation for a number of dedicated comparisons for L-mode plasmas in comparable machines. However, only a rather limited number of simulations and gyrokinetic codes has been applied to the original scenario, thus calling for further dedicated investigations in order to broaden the scientific basis. The present work contributes along these lines by employing another well-established gyrokinetic code in a numerically and physically comprehensive manner. Contrary to the previous studies, only a mild underprediction is observed at the outer radial positions which can furthermore be overcome by varying the ion temperature gradient within the error bars associated with the experimental measurement. The significance and reliability of these simulations are demonstrated by benchmarks, numerical convergence tests, and furthermore by extensive validation studies. The latter involve cross-phase and cross-power spectra analyses of various fluctuating quantities and confirm a high degree of realism. The code discrepancies come as a surprise since the involved software packages had been benchmarked repeatedly and very successfully in the past. Further collaborative effort in identifying the underlying difference is hence required.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A Software Developer’s Guide to Informal Evaluation of Visual Analytics Environments Using VAST Challenge Information

The VAST Challenge has been a popular venue for academic and industry participants for over ten years. Many participants comment that the majority of their time in preparing VAST Challenge entries is discovering elements in their software environments that need to be redesigned in order to solve the given task. Fortunately, there is no need to wait until the VAST Challenge is announced to test out software systems. The Visual Analytics Benchmark Repository contains all past VAST Challenge tasks, data, solutions and submissions. This paper details the various types of evaluations that may be conducted using the Repository information. In this paper we describe how developers can do informal evaluations of various aspects of their visual analytics environments using VAST Challenge information. Aspects that can be evaluated include the appropriateness of the software for various tasks, the various data types and formats that can be accommodated, the effectiveness and efficiency of the process supported by the software, and the intuitiveness of the visualizations and interactions. Researchers can compare their visualizations and interactions to those submitted to determine novelty. In addition, the paper provides pointers to various guidelines that software teams can use to evaluate the usability of their software. While these evaluations are not a replacement for formal evaluation methods, this information can be extremely useful during the development of visual analytics environments.

evaluation↗

Annual Reports in Computational Chemistry, Chapter 3: GronOR Nonorthogonal Configuration Interaction Calculations at Exascale

This chapter describes the implementation and benchmarking of a new software code for nonorthogonal configuration interaction calculations. GronOR is named after the University of Groningen and Oak Ridge National Laboratory, the two institutions that are collaborating on the development of this code for the largest massively parallel supercomputers for open science. The basic implementation approach is described and preliminary benchmarking results are given as obtained from early access to Summit, the next Oak Ridge Leadership Computing Facility's supercomputer with 200 PFlop.

Chemistry↗

Chapter Three - GronOR Nonorthogonal Configuration Interaction Calculations at Exascale

This chapter describes the implementation and benchmarking of a new software code for nonorthogonal configuration interaction calculations. GronOR is named after the University of Groningen and Oak Ridge National Laboratory, the two institutions that are collaborating on the development of this code for the largest massively parallel supercomputers for open science. The basic implementation approach is described and preliminary benchmarking results are given as obtained from early access to Summit, the next Oak Ridge Leadership Computing Facility's supercomputer with 200 PFlop.

Straatsma, T.P.↗

NEAMS Thermal-Fluids Test Stand for Fluoride-Slat Cooled, High Temperature Reactor Development (Final CRADA Report)

The work performed in this CRADA resulted in significant progress toward the development of a Thermal-fluid test-stand based on NEAMS tools. Significant work has been achieved in this area with simulations performed in Nek5000 of the heat and fluid flow in twisted tube heat exchangers and various software and modeling improvements in SAM geared toward heat exchanger simulations. In particular a set of simulations of flow and heat transfer in twisted elliptical tube geometries has been performed with Nek5000. The method was validated by simulating a case at a moderate Reynolds number and relatively high Froude number. This case was demonstrated to predict a Nusselt number in excellent agreement with the applicable correlation found in literature. It was further shown that at lower Froude numbers, this agreement becomes much worse. Additional cases were then simulated with low Froude numbers for both a square and a triangular unit cell to test the effect of pitch-to-maximum diameter ratio for the purposes of future CFD modeling. The results have proven invaluable to understand the flow physics in twisted tubes and provide reference data for design. Results have been summarized in a conference article. Moreover, work in SAM has led to significant improvement of software quality, the creation of a set of benchmarks for natural circulation and other significant software improvement.

97 MATHEMATICS AND COMPUTING↗

EQ_phase_detection

The EQ_phase_detection software is designed to scan continuous daily waveforms to detect earthquake phase arrivals from local to regional (150 km) events. The detections are made with a deep learning encoder-decoder model. When the model detects an earthquake in the waveforms, a second model is implemented to classify the first arriving motions. Both deep learning models are trained with the Tensorflow package using publicly available benchmark data sets. The software input is a path to a directory that contains waveforms in mseed format and the associated response files in xml format. The output is a data table of time stamped detections, signal amplitude, signal-to-noise ratio, and softmax probability of the detection in a generic format applicable to post-processing association algorithms for event locations. Additionally, the p-wave and s-wave waveforms are saved in a data table for rapid access when producing improved locations using correlation-based techniques. The software is designed for multiprocessing with multiple GPU’s for rapid processing of large data sets. The configuration file provides flexibility in the trained models implemented and allows access to multiple models trained for different sampling rates or input dimensions. This is particularly useful for regions with multiple networks that do not have the same data parameters.

Johnson, Christopher↗

2011 EnergyValue Housing Award Report

The EnergyValue Housing Award (EVHA), an integral part of the U.S. Department of Energy’s Building America (BA) program, seeks to improve and expand the energy efficiency of residential construction nationwide by honoring builders and remodelers and awarding highly efficient homes. Receiving EVHA recognition helps builders and remodelers communicate the value of energy efficiency to prospective customers and distinguish their homes from those built by more traditional residential construction companies. It also provides opportunities for EVHA winning builders and remodelers to set an example for improving the quality and performance of homes throughout the country. Each year, the project scope of the EVHA includes the application and judging processes, developing the EVHA publication, updating the program and application for the next year, hosting the EVHA banquet, and outreach and technical support. Following the award banquet, the NAHB Research Center evaluates the energy performance of that year’s EVHA winning houses. This work is primarily comprised of estimating the energy savings of each home/ or housing unit over the BA Benchmark using energy simulation software and procedures developed by the BA program. The energy evaluations serve as a quantitative measure of the energy efficiency of the units which the judges decided merited an award. As part of this work, the NAHB Research Center Industry Partnership (NAHBRC-IP) for High Performing Homes strives to make new and existing homes more energy efficient, durable, and affordable through research and development as one of the U.S. Department of Energy’s BA teams. Each year, through the EVHA, builder and remodeler partners work with industry partners in a variety of climates endeavor to reach the energy efficiency targets of the BA program. (Department of Energy 2012) This report details the simulation tool(s) and energy modeling methodology followed in making the energy efficiency estimates and documents the estimated performance of the EVHA award winning houses in comparison with the BA Benchmark and the associated House Simulation Protocols (HSPs). A summary of each building and its features is included with a brief description of the project and the judges’ comments. The purpose of this report is to assess the energy performance of the 2011 EVHA winners and to align the EVHA program with the BA program.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Preliminary Computational Analysis of the (HIRENASD) Configuration in Preparation for the Aeroelastic Prediction Workshop

This paper presents preliminary computational aeroelastic analysis results generated in preparation for the first Aeroelastic Prediction Workshop (AePW). These results were produced using FUN3D software developed at NASA Langley and are compared against the experimental data generated during the HIgh REynolds Number Aero- Structural Dynamics (HIRENASD) Project. The HIRENASD wind-tunnel model was tested in the European Transonic Windtunnel in 2006 by Aachen University0s Department of Mechanics with funding from the German Research Foundation. The computational effort discussed here was performed (1) to obtain a preliminary assessment of the ability of the FUN3D code to accurately compute physical quantities experimentally measured on the HIRENASD model and (2) to translate the lessons learned from the FUN3D analysis of HIRENASD into a set of initial guidelines for the first AePW, which includes test cases for the HIRENASD model and its experimental data set. This paper compares the computational and experimental results obtained at Mach 0.8 for a Reynolds number of 7 million based on chord, corresponding to the HIRENASD test conditions No. 132 and No. 159. Aerodynamic loads and static aeroelastic displacements are compared at two levels of the grid resolution. Harmonic perturbation numerical results are compared with the experimental data using the magnitude and phase relationship between pressure coefficients and displacement. A dynamic aeroelastic numerical calculation is presented at one wind-tunnel condition in the form of the time history of the generalized displacements. Additional FUN3D validation results are also presented for the AGARD 445.6 wing data set. This wing was tested in the Transonic Dynamics Tunnel and is commonly used in the preliminary benchmarking of computational aeroelastic software.

Chwalowski, Pawel↗

Best Practices and Lessons Learned from Deploying and Operating Large-Scale Data-Centric Parallel File Systems, In: SC '14: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

The Oak Ridge Leadership Computing Facility (OLCF) has deployed multiple large-scale parallel file systems (PFS) to support its operations. During this process, OLCF acquired significant expertise in large-scale storage system design, file system software development, technology evaluation, benchmarking, procurement, deployment, and operational practices. Based on the lessons learned from each new PFS deployment, OLCF improved its operating procedures, and strategies. This paper provides an account of our experience and lessons learned in acquiring, deploying, and operating large-scale parallel file systems. We believe that these lessons will be useful to the wider HPC community.

Computer Science↗

Resonant ultrasound spectroscopy probe for in-situ neutron scattering measurements

Resonant ultrasound spectroscopy (RUS) is an efficient, nondestructive technique to study the elastic properties of solids. A low-temperature (2-300 K) probe has been assembled and tested at the NOMAD beamline at the Spallation Neutron Source at Oak Ridge National Laboratory to assess the probe's neutronic properties, data acquisition system, and compatibility with existing sample environment. A case study on a bulk metallic glass, La 65 Cu 20 Al 10 Co 5, served to benchmark both hardware and software developments. The elastic constants of the metallic glass were determined as a function of temperature between 4 and 300 K and were used to guide neutron diffraction measurements at NOMAD. Tracking of a specific RUS peak center frequency and width enabled live monitoring of the sample temperature evolution that lagged thermometry by upwards of 40 K near room temperature. Assembly of a high-temperature (300-875 K) probe is underway and both probes are scheduled to be available to users by late-2021. Our aim is to provide users with live monitoring of an intrinsic variable at the neutron scattering beamlines, in addition to existing controls, to monitor the state of their samples and its elastic moduli and make informed decisions in real time.

Torres, James↗

pnnl/ZebrafishBMD

ZebrafishBMD: A software tool to perform benchmark dose calculations using toxicity screening data of high-throughput zebrafish assays. High-content imaging of larval zebrafish behavior can be used as a screening approach to rapidly evaluate the relative potential for chemicals to cause toxicity. However, most statistical methods applied to these data transform movement values to incidence-based “hits” and calculate lowest effect levels (LELs), which loses individual fish resolution of behavior and defies hazard ranking due to reliance on applied dose levels. We developed a parallelizable workflow to calculate benchmark dose (BMD) values from dynamic, high-content zebrafish behavior data that scales for high throughput chemical screening. To capture the zebrafish movement response from light to dark stimulus, we summarized time-dependent data using both area under the curve and the immediate change at the transition point into two novel metrics that characterized abnormal behavior as a function of chemical concentration. The BMD workflow was applied to calculate BMD10 values of 1,060 ToxCast chemicals for 24 zebrafish endpoints, including behavior, mortality and morphology. ZebrafishBMD implements the BMD workflow and provides Matlab scripts to read the data (stored in a specific structure) from a .mat file, to calculate the behavioral endpoint values from movement versus time data, to construct concentrationresponse curves, to perform curve-fitting, and to calculate BMD10 values.

Thomas, Dennis↗

VERA BWR progression problems

During the first phase of the Consortium for Advanced Simulation of Light Water Reactors (CASL) program, the Virtual Environment for Reactor Applications (VERA) was developed with a focus on capabilities for high-fidelity, multiphysics simulation of pressurized water reactors (PWRs). During this development effort, a set of progression problems was created ranging from smaller pin cell calculations to larger 3D full-core calculations. These progression problems helped to guide the development of the software and served as benchmarks against which to test VERA. Since 2019, efforts have been made to extend the capabilities of VERA to model boiling water reactors (BWRs). Because BWR simulations come with many unique challenges, a set of BWR progression problems was developed to aid in this new effort. The BWR progression problems range from 2D lattice calculations to 3D mini-core problems, and reference neutronic solutions were computed using continuous-energy Monte Carlo codes. MPACT, one of the neutronics code in VERA, was benchmarked using the BWR progression problems. The code is capable of computing solutions to all problems. The eigenvalues computed by MPACT agree well with the Monte Carlo reference solutions. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

CAE for thermal management of aerospace electronic boards using the BETAsoft program

Aerospace electronic boards require special attention to thermal management due to constraints such as their need to be light, small, and maintain high power densities. Also, cooling is mainly through conductive and radiative modes with minor or negligible convective cooling. Due to these particular requirements, thermal design has become an integrated part of the electronic design process in order to avoid expensive repeat prototyping and to ensure high reliability. To achieve high speed simulations, the BETAsoft code uses semi-empirical formulations and an advanced finite difference scheme that incorporates local adaptive grids. Detailed conduction, convection and radiation heat transfer is considered. Various benchmark verifications of the software simulation compared to infrared images typically prove to be within 10% of each other. The thermal analysis of a sample avionic card in a natural convection environment is shown. Then, the individual effects of attaching metal screws to the casing, increasing radiative emissivities of the casing, increasing the conductance of the wedge lock, adding an aluminum core to the board, adding metal strips in board layers, inserting conduction pads under components, and adding heat sinks to components are demonstrated.

Bobish, Kimberly↗

Energy economic evaluation of hydrogen production by HTGR and solar tower - 18554

Changing the high temperature heat supply of energy intensive industries from today's mostly fossil sources to nuclear or renewable sources offers the opportunity of massive reductions of greenhouse gas emissions. Even though the HTGR as well as the solar tower are technically capable to supply heat at the required high temperature level, it is uncertain whether these technologies can compete with the fossil energy supply. Therefore, the aim of this paper is the assessment of the economic competitiveness of process heat supply by HTGR or solar tower to fossil fired processes using steam methane reforming as industrial example process. Applying a self-developed optimization model, energy economic analyzes are conducted for nuclear, solar and fossil energy supply systems. The results are benchmarked with the IAEA software HEEP and compared to literature values. The analyzes show that hydrogen generation costs of the solar tower system are higher than those of the HTGR system. Both technologies benefit from the support of a fossil heater, as the hydrogen generation costs can be significantly reduced even with a small amount of burned natural gas. Nevertheless, a steam methane reforming process run by HTGR or by solar tower is not yet competitive to the natural gas fired hydrogen production. (authors)

08 HYDROGEN↗

Storage-Intensive Supercomputing Benchmark Study

Critical data science applications requiring frequent access to storage perform poorly on today's computing architectures. This project addresses efficient computation of data-intensive problems in national security and basic science by exploring, advancing, and applying a new form of computing called storage-intensive supercomputing (SISC). Our goal is to enable applications that simply cannot run on current systems, and, for a broad range of data-intensive problems, to deliver an order of magnitude improvement in price/performance over today's data-intensive architectures. This technical report documents much of the work done under LDRD 07-ERD-063 Storage Intensive Supercomputing during the period 05/07-09/07. The following chapters describe: (1) a new file I/O monitoring tool iotrace developed to capture the dynamic I/O profiles of Linux processes; (2) an out-of-core graph benchmark for level-set expansion of scale-free graphs; (3) an entity extraction benchmark consisting of a pipeline of eight components; and (4) an image resampling benchmark drawn from the SWarp program in the LSST data processing pipeline. The performance of the graph and entity extraction benchmarks was measured in three different scenarios: data sets residing on the NFS file server and accessed over the network; data sets stored on local disk; and data sets stored on the Fusion I/O parallel NAND Flash array. The image resampling benchmark compared performance of software-only to GPU-accelerated. In addition to the work reported here, an additional text processing application was developed that used an FPGA to accelerate n-gram profiling for language classification. The n-gram application will be presented at SC07 at the High Performance Reconfigurable Computing Technologies and Applications Workshop. The graph and entity extraction benchmarks were run on a Supermicro server housing the NAND Flash 40GB parallel disk array, the Fusion-io. The Fusion system specs are as follows: SuperMicro X7DBE Xeon Dual Socket Blackford Server Motherboard; 2 Intel Xeon Dual-Core 2.66 GHz processors; 1 GB DDR2 PC2-5300 RAM (2 x 512); 80GB Hard Drive (Seagate SATA II Barracuda). The Fusion board is presently capable of 4X in a PCIe slot. The image resampling benchmark was run on a dual Xeon workstation with NVIDIA graphics card (see Chapter 5 for full specification). An XtremeData Opteron+FPGA was used for the language classification application. We observed that these benchmarks are not uniformly I/O intensive. The only benchmark that showed greater that 50% of the time in I/O was the graph algorithm when it accessed data files over NFS. When local disk was used, the graph benchmark spent at most 40% of its time in I/O. The other benchmarks were CPU dominated. The image resampling benchmark and language classification showed order of magnitude speedup over software by using co-processor technology to offload the CPU-intensive kernels. Our experiments to date suggest that emerging hardware technologies offer significant benefit to boosting the performance of data-intensive algorithms. Using GPU and FPGA co-processors, we were able to improve performance by more than an order of magnitude on the benchmark algorithms, eliminating the processor bottleneck of CPU-bound tasks. Experiments with a prototype solid state nonvolative memory available today show 10X better throughput on random reads than disk, with a 2X speedup on a graph processing benchmark when compared to the use of local SATA disk.

99 GENERAL AND MISCELLANEOUS↗