Search NASA⌕ Search

SEARCH · Search NASA

Results for “benchmark exercise”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Evaluation of Opportunistic Contact Graph Routing in Random Mobility Environments

Routing in networks where nodes move randomly is particularly challenging due their potentially unpredictable, and rapidly changing topology. Several routing algorithms have been presented in the literature to address the needs of such networks, most of them implementing variants of controlled network flooding in the hope of successful data delivery. In this note, we compare the results of previous routing algorithms with Opportunistic Contact Graph Routing (OCGR), an enhanced version of Contact Graph Routing (CGR) that is suitable for networks where contacts cannot always be scheduled ahead of time. To perform the benchmark, we simulate a network of nodes moving in a certain space according to the Random Waypoint Mobility Model, and then take measurements of bundle delivery probabilty and overhead ratio as metrics of performance and cost respectively. Through this exercise, we demonstrate that the performance of OCGR is highly dependent on the type of network under consideration (e.g. very sparse vs. densely connected) and the assumed mobility model.

Burleigh, Scott↗
DOE OSTI2025

First multi-institutional systematic comparison of the neutron ambient dose equivalent produced by proton therapy systems

Objective. Isochronous cyclotrons, synchrocyclotrons, and synchrotrons are used to accelerate protons for proton therapy. An accurate measurement of neutron doses generated by these accelerators and associated delivery systems and its clinical relevance requires systematic protocols and proper neutron dosimetry for a meaningful assessment. We present the first comprehensive comparison of neutron ambient dose equivalent (H*(10)) produced by clinically operational proton therapy systems. Approach. Treatment plans with 10 cm modulation-depth and ranges of 10 cm (R10M10) and 25 cm (R25M10) were created to cover a 10 × 10 × 10 cm 3 water target. The pencil beam scanning proton therapy machines studied were: two gantry-mounted synchrocyclotrons (Hyperscan, Mevion, half-gantry), two isochronous cyclotrons (ProBeam, Varian, full-gantry), one isochronous cyclotron (Proteus, IBA, full-gantry), and two synchrotrons (PROBEAT, Hitachi, full- and half-gantry). Proton beams were delivered to 30 × 30 × 40 cm 3 plastic water phantoms. WENDI-II and LUPIN-BF3-NP neutron rem-meters were positioned at three angles (0°, 45°, 90°) relative to the beam direction to measure the neutron H*(10) at distances between 50–300 cm from the isocenter. Main results. H*(10) showed dependence on beam energy, machine type, and measurement location. The highest reading was for the gantry-mounted synchrocyclotron, whereas other systems produced approximately comparable neutron doses. In all cases, the H*(10) reduced with distance from the isocenter. The H*(10) drop at 2 m distance compared to that at 0.5 m was a factor of ∼5 for the gantry-mounted synchrocyclotron whereas in other systems the decrease was a factor of 10. The WENDI-II device suffered from dead-time-associated under-estimation of the dose by a factor of ∼2–3 under the synchrocyclotron beam due to its high dose-per-pulse. However, WENDI-II and LUPIN-BF3-NP results were within reasonable agreement in isochronous cyclotron and synchrotron beams, indicating that both devices are suitable for those systems. Significance. Neutron H*(10) is dependent on various parameters including beam energy, measurement location, as well as machine design. Caution must be exercised in choosing the appropriate neutron-dose-measurement device to be used for low-duty-factor, particularly in high-instantaneous-rate proton delivery systems. By delivering the same volumetric proton dose across different machines, this work provides a benchmark for inter-system comparisons and serves as a foundation for future studies.

LUPIN↗
NASA NTRS1992

Reanalysis of Mariner 9 UV spectrometer data for ozone, cloud, and dust abundances, and their interaction over climate timescales

Research activities to date are discussed. Selected Mariner 9 UV spectra were obtained. Radiative transfer models were updated and then exercised to simulate spectra. Simulated and observed spectra compare favorably. It is noted that large amounts of ozone are currently not retrieved with reflectance spectroscopy, raising large doubts about earlier published ozone abundances. As these published abundances have been used as a benchmark for all theoretical photochemical models of Mars, this deserves further exploration. Three manuscripts were published, and one is in review. Papers were presented and published at three conferences, and are planned for five more conferences in the next six months. The research plan for the next reporting period is discussed and involves continuing studies of reflectance spectroscopy, further examination of Mariner 9 data, and climate change studies of ozone.

Lindner, Bernhard Lee↗
DOE OSTI2022

Improved Verification and Validation Testing and Tools including Nuclear Criticality Safety Applications with the MCNP6.3® Code [Abstract]

A new Python-based framework has been developed to enable a more consistent layout with automatable setup, execution, and documentation of all verification and validation (V&V) test suites previously established for use with the MCNP code. In this paper, the new general framework for the V&V test suites is discussed, including information on all of the current capabilities and plans for future capabilities. For nuclear criticality safety applications, the existing V&V benchmark problems within the criticality, extended criticality, and analytic k-effective test suites have been ported into this new framework. In addition, the status and updates to the Rossi-alpha and subcritical multiplicationtest suites will be discussed. Some V&V results exercising new MCNP6.3 capabilities will be demonstrated.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Verification and Performance Impact of the New Parallel MCNP6.3 Particle Track Output Capability for Subcritical Multiplication Simulations

The MCNP6® code, version 6.3, has several new features that are intended to ultimately replace legacy features that are now marked for deprecation. One of these features is the new particle track output (PTRAC) format and capability, where the legacy PTRAC capability still exists alongside the modern PTRAC capability in MCNP6.3. While the MCNP6.3 code has been extensively verified and validated for many applications, the PTRAC feature is not exercised in any of the typical verification and validation (V&V) applications studied during the course of a typical MCNP code release. The primary goal of this paper is to verify that the legacy and modern PTRAC feature produces equivalent results for subcritical multiplication benchmarks previously studied. In the process of verifying that the simulated benchmark results are equivalent, the computational performance is compared between the legacy and modern PTRAC uses. In addition to verification of the update, which is important to the community as a whole, this effort also supports advances in the simulation of recent subcritical neutron noise measurements that require higher computational effort per second of real-time measurement than that of systems typically measured.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2022

Plasma Composition Measurements in an Active Region from Solar Orbiter/SPICE and Hinode/EIS

A key goal of the Solar Orbiter mission is to connect elemental abundance measurements of the solar wind enveloping the spacecraft with extreme-UV (EUV) spectroscopic observations of their solar sources, but this is not an easy exercise. Observations from previous missions have revealed a highly complex picture of spatial and temporal variations of elemental abundances in the solar corona. We have used coordinated observations from Hinode and Solar Orbiter to attempt new abundance measurements with the Spectral Imaging of the Coronal Environment (SPICE) instrument, and benchmark them against standard analyses from the EUV Imaging Spectrometer (EIS). We use observations of several solar features in active region (AR) 12781 taken from an Earth-facing view by EIS on 2020 November 10, and SPICE data obtained one week later on 2020 November 17, when the AR had rotated into the Solar Orbiter field of view. We identify a range of spectral lines that are useful for determining the transition region and low-coronal-temperature structure with SPICE, and demonstrate that SPICE measurements are able to differentiate between photospheric and coronal magnesium/neon abundances. The combination of SPICE and EIS is able to establish the atmospheric composition structure of a fan loop/outflow area at the AR edge. We also discuss the problem of resolving the degree of elemental fractionation with SPICE, which is more challenging without further constraints on the temperature structure, and comment on what that can tell us about the sources of the solar wind and solar energetic particles.

Solar physics↗
NASA NTRS2013

Overview of the Aeroelastic Prediction Workshop

The Aeroelastic Prediction Workshop brought together an international community of computational fluid dynamicists as a step in defining the state of the art in computational aeroelasticity. This workshop's technical focus was prediction of unsteady pressure distributions resulting from forced motion, benchmarking the results first using unforced system data. The most challenging aspects of the physics were identified as capturing oscillatory shock behavior, dynamic shock-induced separated flow and tunnel wall boundary layer influences. The majority of the participants used unsteady Reynolds-averaged Navier Stokes codes. These codes were exercised at transonic Mach numbers for three configurations and comparisons were made with existing experimental data. Substantial variations were observed among the computational solutions as well as differences relative to the experimental data. Contributing issues to these differences include wall effects and wall modeling, non-standardized convergence criteria, inclusion of static aeroelastic deflection, methodology for oscillatory solutions, post-processing methods. Contributing issues pertaining principally to the experimental data sets include the position of the model relative to the tunnel wall, splitter plate size, wind tunnel expansion slot configuration, spacing and location of pressure instrumentation, and data processing methods.

Heeg, Jennifer↗
DOE OSTI2026

The 0E2 benchmarks for PWR UO 2 decay heat: an analysis from the NEA WPNCS

This paper presents the work performed in the subgroup 16 of the Working Party for Nuclear Criticality Safety (WPNCS) of the OECD Nuclear Energy Agency. The main goal was to define two decay heat benchmarks for Spent Nuclear Fuel (one pincell and one assembly), perform calculations and compare and analyze the results in light of existing calorimetric measurements. The selected case is the PWR UO2 assembly 0E2, irradiated at the Ringhals-3 reactor and measured at the Clab facility in Sweden. In total, 21 institutes worldwide participated to the exercise, leading to 55 calculated results (named C). It was found that the measured decay heat values (E) can be satisfactorily reproduced with two-dimensional assembly calculations, leading to an average C/E value of 0.99, with an uncertainty (or one standard deviation) of ±0.01.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influences the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as a case study, we perform a large-scale study across over 200 DA scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision (CV) techniques as well as pretrained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As demonstrated by a “cherry-picking” exercise, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. Furthermore, this study intends to motivate the community toward more consistent evaluation protocols while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

42 ENGINEERING↗
NASA NTRS2004

Initial characterization of the microgravity environment of the international space station: increments 2 through 4

The primary objective of the International Space Station (ISS) is to provide a long-term quiescent environment for the conduct of scientific research for a variety of microgravity science disciplines. This paper reports to the microgravity scientific community the results of an initial characterization of the microgravity environment on the International Space Station for increments 2 through 4. During that period almost 70,000 hours of station operations and scientific experiments were conducted. 720 hours of crew research time were logged aboard the orbiting laboratory and over half a terabyte of acceleration data were recorded and much of that was analyzed. The results discussed in this paper cover both the quasi-steady and vibratory acceleration environment of the station during its first year of scientific operation. For the quasi-steady environment, results are presented and discussed for the following: the space station attitudes Torque Equilibrium Attitude and the X-Axis Perpendicular to the Orbital Plane; station docking attitude maneuvers; Space Shuttle joint operation with the station; cabin de-pressurizations and the station water dumps. For the vibratory environment, results are presented for the following: crew exercise, docking events, and the activation/de-activation of both station life support system hardware and experiment hardware. Finally, a grand summary of all the data collected aboard the station during the 1-year period is presented showing where the overall quasi-steady and vibratory acceleration magnitude levels fall over that period of time using a 95th percentile benchmark. Published by Elsevier Ltd.

Flight Experiment↗
DOE OSTI2021

A multimodal and integrated approach to interrogate human kidney biopsies with rigor and reproducibility: guidelines from the Kidney Precision Medicine Project

Comprehensive and spatially mapped molecular atlases of organs at a cellular level are a critical resource to gain insights into pathogenic mechanisms and personalized therapies for diseases. The Kidney Precision Medicine Project (KPMP) is an endeavor to generate three-dimensional (3-D) molecular atlases of healthy and diseased kidney biopsies by using multiple state-of-the-art omics and imaging technologies across several institutions. Obtaining rigorous and reproducible results from disparate methods and at different sites to interrogate biomolecules at a single-cell level or in 3-D space is a significant challenge that can be a futile exercise if not well controlled. Here we describe a “follow the tissue” pipeline for generating a reliable and authentic single-cell/region 3-D molecular atlas of human adult kidney. Our approach emphasizes quality assurance, quality control, validation, and harmonization across different omics and imaging technologies from sample procurement, processing, storage, shipping to data generation, analysis, and sharing. We established benchmarks for quality control, rigor, reproducibility, and feasibility across multiple technologies through a pilot experiment using common source tissue that was processed and analyzed at different institutions and different technologies. A peer review system was established to critically review quality control measures and the reproducibility of data generated by each technology before their being approved to interrogate clinical biopsy specimens. The process established economizes the use of valuable biopsy tissue for multiomics and imaging analysis with stringent quality control to ensure rigor and reproducibility of results and serves as a model for precision medicine projects across laboratories, institutions and consortia.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Incorporating Full Elastodynamic Effects and Dipping Fault Geometries in Community Code Verification Exercises for Simulations of Earthquake Sequences and Aseismic Slip (SEAS)

ABSTRACT Numerical modeling of earthquake dynamics and derived insight for seismic hazard relies on credible, reproducible model results. The sequences of earthquakes and aseismic slip (SEAS) initiative has set out to facilitate community code comparisons, and verify and advance the next generation of physics-based earthquake models that reproduce all phases of the seismic cycle. With the goal of advancing SEAS models to robustly incorporate physical and geometrical complexities, here we present code comparison results from two new benchmark problems: BP1-FD considers full elastodynamic effects, and BP3-QD considers dipping fault geometries. Seven and eight modeling groups participated in BP1-FD and BP3-QD, respectively, allowing us to explore these physical ingredients across multiple codes and better understand associated numerical considerations. With new comparison metrics, we find that numerical resolution and computational domain size are critical parameters to obtain matching results. Codes for BP1-FD implement different criteria for switching between quasi-static and dynamic solvers, which require tuning to obtain matching results. In BP3-QD, proper remote boundary conditions consistent with specified rigid body translation are required to obtain matching surface displacements. With these numerical and mathematical issues resolved, we obtain excellent quantitative agreements among codes in earthquake interevent times, event moments, and coseismic slip, with reasonable agreements made in peak slip rates and rupture arrival time. We find that including full inertial effects generates events with larger slip rates and rupture speeds compared to the quasi-dynamic counterpart. For BP3-QD, both dip angle and sense of motion (thrust versus normal faulting) alter ground motion on the hanging and foot walls, and influence event patterns, with some sequences exhibiting similar-size characteristic earthquakes, and others exhibiting different-size events. These findings underscore the importance of considering full elastodynamics and nonvertical dip angles in SEAS models, as both influence short- and long-term earthquake behavior and are relevant to seismic hazard.

Geochemistry & Geophysics↗
DOE OSTI2021

The simulation of 3D hypervelocity spallation using a hydrocode PAGOSA with FLIP+MPM

n this work, a hydrocode PAGOSA with FLIP+MPM is presented and exercised to investigate the fracture in ductile material. The merit of PAGOSA with FLIP+MPM to solve the advection problem is first illustrated by a solid piston periodically moving in a sealed tube with air. Furthermore, the ability of PAGOSA with FLIP+MPM to capture the fracture in material is shown by a simple stretching fracture in ductile material. In both of two benchmark problems, the PAGOSA results and analytical solutions are also presented for comparison. Then PAGOSA with FLIP+MPM is used to model complex spall in ductile material, which is a challenging problem in engineering applications. The convergences of PAGOSA with FLIP+MPM—based on both the mesh size and the marker density—are investigated by monitoring free surface velocity. To further show the ability of PAGOSA with FLIP+MPM to predict fracture in ductile material, the numerical results are compared with the experimental results and other published numerical results. Moreover, the effect of the spall parameter in PAGOSA with FLIP+MPM on the numerical simulation is also analyzed by investigating the free surface velocity. Finally, the effects of the peak compressive stress, the tensile strain rate, and the loading rate on the spallation are further investigated. The numerical results show that PAGOSA with FLIP+MPM can improve PAGOSA's performance when applied to predicting fracture in ductile materials, and is robust enough to accurately predict spall fracture in ductile materials.

42 ENGINEERING↗
NASA NTRS2007

Observations on Cost Modeling and Performance Measurement of Long Term Archives

This paper describes a prototype suite of Excel-based tools that could be used for estimating lifecycle costs for newly planned or modified long-term archival facilities. These tools may also prove valuable for monitoring the long-term performance of such facilities once operational. Cost estimation is by analogy, using statistical curve-fitting techniques across a database of comparable data activities. The database currently includes 29 operational data centers ranging from small (2 FTEs) to large (66 FTEs), and is readily expandable to include additional activities specifically involving data preservation and added value. Each comparable data center is described in terms of its staffing, throughput workload, archival and distribution requirements, levels of user service, overall complexity, degree of automation, and other data, comprising 94 distinct descriptors in all. The descriptors were developed by normalizing heterogeneous data from the various centers and mapping them into an Excel framework consistent with the OAIS reference model. A user-friendly tool is provided for generating input to and updating the comparables database. This tool can also be used to benchmark the performance (in terms of cost versus throughput) of an operational data center, and to update the staffing, cost and workload data on a periodic basis. The comparables database could thus provide a history of staffing and throughput over time, as a means of performance monitoring and providing feedback for continuous improvement. Ancillary tools are also provided for performing "what-if' cost exercises for planning purposes, and for graphical display of data and results. We provide a high-level description of the tools; present our experiences and observations on gathering the information and maintaining the database; and discuss how this tool set might be applied to long term archives.

Fontaine, Kathy↗
NASA NTRS2024

NASA Engineering and Safety Center Technical Bulletin No. 24-04: 6DOF Check Cases

In 2015, the NESC released benchmark Earth-based check-cases for well specified, rigid-body, six-degree-of-freedom (6DOF) aero/spacecraft models to promote consistent and accurate flight simulations across multiple Agency tools and facilities. Recently, the NESC expanded upon that effort to add Lunar-based check-cases to support new lunar exploration initiatives. This study produced a smaller, focused set of cases that exercise new and unique features of missions in the lunar environment in comparison with 8 high-fidelity NASA simulation tools and provides a measure of validation for simulations supporting Human Landing Systems.

Flight Mechanics↗
DOE OSTI2023

Tidal Turbine Benchmarking Project: Stage I - Steady Flow Blind Predictions: Preprint

This paper presents the first blind prediction stage of the Tidal Turbine Benchmarking Project being conducted and funded by the UK's EPSRC and Supergen ORE Hub. In this first stage, only steady flow conditions, at low and elevated turbulence (3.1%) levels, were considered. Prior to the blind prediction stage, a large laboratory scale experiment was conducted in which a highly instrumented 1.6m diameter tidal rotor was towed through a large towing tank in well-defined flow conditions with and without an upstream turbulence grid. Details of the test campaign and rotor design were released as part of this community blind prediction exercise. Participants were invited to use a range of engineering modelling approaches to simulate the performance and loads of the turbine. 26 submissions were received from 12 groups from across academia and industry using solution techniques ranging from blade resolved computational fluid dynamics through actuator line, boundary integral element methods, vortex methods to engineering Blade Element Momentum methods. The comparisons between experiments and blind predictions were extremely positive helping to provide validation and uncertainty estimates for the models, but also validating the experimental tests themselves. The exercise demonstrated that the experimental turbine data provides a robust data set against which researchers and design engineers can test their models and implementations to ensure robustness in their processes, helping to reduce uncertainty and provide increased confidence in engineering processes. Furthermore, the data set provides the basis by which modellers can evaluate and refine approaches.

benchmarking↗
DOE OSTI2024

Implementation and Validation of a Generalized Actuator Disk Parameterization for Wind Turbine Simulations Within the FastEddy Model

ABSTRACT Fast and accurate large‐eddy simulation (LES) of the atmospheric boundary layer plays a crucial role in advancing wind energy research. Long‐duration wind farm studies at turbine‐resolving scales have become increasingly important to understand the intricate interactions between large wind farms and the atmospheric boundary layer. However, the prohibitive computational cost of these turbulence‐ and turbine‐resolving simulations has precluded such modeling to be exercised on a regular basis. To that end, we implement and validate the generalized actuator disk (GAD) model in the computationally efficient, graphics processing unit (GPU)–resident, LES model FastEddy. We perform single‐turbine simulations under three atmospheric stabilities (neutral, unstable, and stable) and compare them against observations from the Scaled Wind Farm Technology (SWiFT) facility and other LES codes from the recent Wakebench turbine wake model benchmark. Our idealized LES results agree well with observed wake velocity deficit and downstream recovery across stability regimes. Turbine response in terms of rotational speed, generated power, torque, and thrust coefficient are well predicted across stability regimes and are consistent with the LES results from the benchmark. The FastEddy simulations are found to be at least two orders of magnitude more efficient than the traditional CPU‐based LES models, opening the door for realistic LES simulations of full wind plants as a viable standard practice.

17 WIND ENERGY↗
DOE OSTI2021

Critical Experiment Design Phase 2 Report for Integral Request 304

This report documents the second phase of the critical experiment design process (CED-2) conducted as part of integral experiment request (IER) 304. The purpose of IER-304 is to develop a temperature-dependent critical experiment capability at the Sandia National Laboratories critical experiment facility using low-enriched uranium oxide fuel. Only a few benchmark quality critical experiments are currently available to validate criticality safety computational methods at temperatures other than room temperature. Recent advancements in computational techniques such as implementation of on-the-fly Doppler broadening techniques and advanced treatment of thermal scattering data necessitate the development of additional experimental capabilities in this area. The work described in this report developed representative configurations using 7uPCX and BUCCX fuels, analyzed the physics exercised by the different arrays, and analyzed the estimated experimental uncertainties. This report also scopes the information necessary for performing modifications to the reactor to accommodate operation at elevated and reduced temperatures.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗