Search NASA⌕ Search

SEARCH · Search NASA

Results for “code verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,567 records · Page 87

SU-E-J-203: Reconstruction of the Treatment Area by Use of Sinogram in Helical Tomotherapy

Purpose: TomoTherapy (Accuray Co.) has an image-guided radiotherapy system with megavoltage (MV) X-ray source and the on-board imaging device. With the MV computed tomography (MVCT), it became feasible to perform the efficient daily-3D registration of the patient position before each treatment delivery. This system also allows one to acquire the delivery sinogram during the actual treatment, which partly includes the information of the irradiated object. In this study, we try to develop the image reconstruction during treatment in helical Tomotherapy. Methods: Sinogram data were acquired during helical Tomotherapy delivery using an arc-shaped detector array that consists of 738 xenon-gas filled detector cells. In preprocessing, these were normalized by full air-scan data. A software program was developed that reconstructs 3D images during treatment with corrections as; (1) the regions outside the field were masked not to be added in the backprojection (a masking correction), and (2) each voxel of the reconstructed image was divided by the number of the X-ray passing through its voxel (a ray-passing correction). Results: Without masking and ray-passing corrections, the image reconstruction was failed. The masking correction made the image clear, however, the streak artifact was accompanied. The ray-passing correction reduced this artifact. Although the SNR (the ratio of mean to standard deviation in homogeneous region) and the contrast of the reconstructed image were slightly improved with the ray-passing correction, the masking correction only is enough for the visualization purpose. Conclusion: The visualization of the treatment area was feasible by use of the sinogram in helical Tomotherapy. This proposed method can be utilized in the treatment verification. This work was partly supported by JSPS KAKENHI 24234567. No COI, but the data in this paper were prepared by collaborators in Accuray.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Phenomena, gaps, and issues for neutronics modeling and simulation of FHRs

On the path to deployment of any reactor, modeling tool verification and validation is a key step. Fluoride-Salt-Cooled High Temperature Reactors (FHR) pose challenges to neutronics modeling and simulation tools due to several design features of the reactor type. This paper presents a categorized list of phenomena that pose challenges to FHR modeling using current neutronics tools. These phenomena are presented in four categories: Fundamental Cross Section Data, Material Composition, Computational Methodology, and General Depletion. Here, a short path forward for these phenomena is also presented. In addition, a discussion of the resulting gaps in current codes is presented.

42 ENGINEERING↗

Initial Atmospheric Transport of Particles

A new version of the Initial Atmospheric Transport (IAT) model has been developed. This report fully characterizes the new model. IAT is designed to model heat release into the atmosphere that contains contaminated particles. The heat release forms a buoyant puff or plume. Buoyancy is quickly dissipated in 10's to 100's of seconds through drag and entrainment of ambient air. The final location of contaminant particles in the atmosphere after dissipation of abnormal heat is an important input to long term transportation codes such as the National Oceanic and Atmospheric Administration's (NOAA) Hybrid Single Particle Lagrangian Integrated Trajectory (HYSPLIT) code. Ultimately the settling of contaminants after long term transport provides a basis for estimating health consequences. In comparison to the old version, the new model includes explicit simulation of particles, newly formulated equations of motion for a rising puff, and a new plume model. The plume model releases many instances of a single puff along a single trajectory in a way that balances energy released over a longer period than would be appropriate for a single puff. The puff model has additions for vorticity, turbulence, virtual mass, drag, and entrainment. The particle model leads to direct interfacing between IAT and the long-term transport code HYSPLIT. Previously, particle locations were given a pattern based on IAT's puff trajectory with no particle simulation. Now, particles are explicitly simulated, and 3-D spatial distributions of particles are output based on final positions. Ballistic particles are thrown outward at high velocity from explosive events and tend to hit the ground before the end of the simulation. Non-ballistic particles tend to get pulled into the rising puff s ring of vorticity with some escaping to ambient conditions as buoyant energy dissipates. A new model verification and validation effort was made for the new plume model. The Wallops Island Solid Rocket Propellant (SRP) fire plume was extracted to a 3-D transient dataset. After calibration, the model performs better than previous single puff attempts. Even so, error levels in the process used to estimate location and ambient weather conditions approach 30%. This leads to the conclusion that unplanned photogrammetry evaluations of the IAT model are useful but not sufficiently accurate to quantify model parameters. Like previous attempts, the vertical motion entrainment parameters come out much too high in comparison to values for other studies in the literature. It is suspected that a new model of shear entrainment would alleviate this problem. Unfortunately, IAT already has more calibration parameters than the data comparisons thus far can resolve with statistical certainty. This underscores the need to verify and validate the IAT model using detailed CFD studies that allow ideal statistical comparison with the IAT model for hundreds of cases. Even so, validation by comparison to carefully planned experiments (i.e. with acceptable error levels) is also necessary because the dynamics of entrainment are at a sub-grid level for CFD. Though better validation and verification are most important, many new enhancements are still envisioned for future IAT development. These include particle rain out due to condensation and agglomeration of water, direct connection to the North American Mesoscale (NAM) weather dataset, dynamic weather conditions, including ambient particle entrainment into the puff, and detrainment at the puff to ambient mixing interface. In general, including detailed aerosol dynamics is the greatest coming challenge which will require an entire new sub-model in IAT. This is very important if contaminant particles change in such a way that they increase or decrease in their potential harm to the environment.

54 ENVIRONMENTAL SCIENCES↗

Integrated modeling applications for tokamak experiments with OMFIT

We developed one modeling framework for integrated tasks (OMFIT) is a comprehensive integrated modeling framework which will enable physics codes to interact in complicated workflows, and support scientists at all stages of the modeling cycle. The OMFIT development follows a unique bottom-up approach, where the framework design and capabilities organically evolve to support progressive integration of the components that are required to accomplish physics goals of increasing complexity. OMFIT provides a workflow for easily generating full kinetic equilibrium reconstructions that are constrained by magnetic and motional Stark effect measurements, and kinetic profile information that includes fast-ion pressure modeled by a transport code. It was found that magnetic measurements can be used to quantify the amount of anomalous fast-ion diffusion that is present in DIII-D discharges, and provide an estimate that is consistent with what would be needed for transport simulations to match the measured neutron rates. OMFIT was used to streamline edge-stability analyses, and evaluate the effect of resonant magnetic perturbation (RMP) on the pedestal stability, which have been found to be consistent with the experimental observations. Moreover, the development of a five-dimensional numerical fluid model for estimating the effects of the interaction between magnetohydrodynamic (MHD) and microturbulence, and its systematic verification against analytic models was also supported by the framework. OMFIT was used for optimizing an innovative high-harmonic fast wave system proposed for DIII-D. For a parallel refractive index n(parallel to) > 3, the conditions for strong electron-Landau damping were found to be independent of launched n(parallel to) and poloidal angle. OMFIT has been the platform of choice for developing a neural-network based approach to efficiently perform a non-linear multivariate regression of local transport fluxes as a function of local dimensionless parameters. Transport predictions for thousands of DIII-D discharges showed excellent agreement with the power balance calculations across the whole plasma radius and over a broad range of operating regimes. Concerning predictive transport simulations, the framework made possible the design and automation of a workflow that enables self-consistent predictions of kinetic profiles and the plasma equilibrium. It is found that the feedback between the transport fluxes and plasma equilibrium can significantly affect the kinetic profiles predictions. Finally, such a rich set of results provide tangible evidence of how bottom-up approaches can potentially provide a fast track to integrated modeling solutions that are functional, cost-effective, and in sync with the research effort of the community.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

SYSTEMATIC MULTI-INDEX VALIDATIONS OF SIDI ENGINE FLOW FIELD LES COMPUTATIONS USING CRANK ANGLE-RESOLVED PIV MEASUREMENTS

In-cylinder flow fields make significant impacts on the fuel atomization, fuel mixture formation, and combustion process in spark ignition direct injection (SIDI) engines. In recent years, model-based simulation approaches are preferred in regard to investigating the transient in-cylinder flow field characteristics. Most commonly, the simulation models are validated using single representative flow field at a typical crank angle measured by particle image velocimetry (PIV). However, it provides only limited knowledge about the flow field which is highly three-dimensional and of transient nature. In this study, crank angle-resolved Ply measurements are conducted on three distinct planes inside the cylinder to capture the transient process of flow field characteristics which vary with the crank angle. These three planes consist of one tumble plane through the spark plug tip, one tumble plane along two intake ports, and one swirl plane at 30 mm below the cylinder head. Large eddy simulation (LES) is employed for the numerical computations using the CONVERGE codes. On the basis of large datasets for both temporal and spatial domains, a multi-index systematic validation approach is conducted. Crank angle-resolved calculations of global indices and local indices are implemented using the flow fields velocity data obtained from both PIV and LES on select planes. Global indices reveal the trends in similarities of different crank angle degrees and locations, while local indices give the detail comparison results. In summary, with the systematic multi-index validation approach, the crucial crank angle degrees and locations for model verification will be detected. Furthermore, the corresponding critical flow features are analyzed. Practical guideline of flow field validation is proposed.

Crank angle-resolved PIV↗

Program Correctness, Verification and Testing for Exascale (Corvette)

The goal of this project is to provide tools to assess the correctness of parallel programs written using hybrid parallelism. There is a dire lack of both theoretical and engineering know-how in the area of finding bugs in hybrid or large scale parallel programs, which our research aims to change. In the project we have demonstrated novel approaches in several areas: 1. Low overhead automated and precise detection of concurrency bugs at scale. 2. Using low overhead bug detection tools to guide speculative program transformations for performance. 3. Techniques to reduce the concurrency required to reproduce a bug using partial program restart/replay. 4. Techniques to provide reproducible execution of floating point programs. 5. Techniques for tuning the floating point precision used in codes.

97 MATHEMATICS AND COMPUTING↗

Exploring Explicit Uncertainty for Binary Analysis (EUBA)

Reverse engineering (RE) analysts struggle to address critical questions about the safety of binary code accurately and promptly, and their supporting program analysis tools are simply wrong sometimes. The analysis tools have to approximate in order to provide any information at all, but this means that they introduce uncertainty into their results. And those uncertainties chain from analysis to analysis. We hypothesize that exposing sources, impacts, and control of uncertainty to human binary analysts will allow the analysts to approach their hardest problems with high-powered analytic techniques that they know when to trust. Combining expertise in binary analysis algorithms, human cognition, uncertainty quantification, verification and validation, and visualization, we pursue research that should benefit binary software analysis efforts across the board. We find a strong analogy between RE and exploratory data analysis (EDA); we begin to characterize sources and types of uncertainty found in practice in RE (both in the process and in supporting analyses); we explore a domain-specific focus on uncertainty in pointer analysis, showing that more precise models do help analysts answer small information flow questions faster and more accurately; and we test a general population with domain-general sudoku problems, showing that adding "knobs" to an analysis does not significantly slow down performance. This document describes our explorations in uncertainty in binary analysis.

97 MATHEMATICS AND COMPUTING↗

PWR Cross Section Libraries for ORIGEN-ARP

New pressurized water reactor (PWR) cross-section libraries were generated for use with the ORIGEN-ARP depletion sequence in the SCALE nuclear analysis code system. These libraries are based on ENDF/B-VII nuclear data and were generated using the two-dimensional depletion sequence, TRITON/NEWT, in SCALE 6.1. The libraries contain multiple burnup-dependent cross-sections for seven PWR fuel designs, with enrichments ranging from 1.5 to 6 wt% 235U. The burnup range has been extended from the 72 GWd/MTU used in previous versions of the libraries to 90 GWd/MTU. Validation of the libraries using radiochemical assay measurements and decay heat measurements for PWR spent fuel showed good agreement between calculated and experimental data. Verification against detailed TRITON simulations for the considered assembly designs showed that depletion calculations performed in ORIGEN-ARP with the pre-generated libraries provide similar results as obtained with direct TRITON depletion, while greatly reducing the computation time.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Modeling and Simulation of Fuel Dispersal During the Loss-of-Coolant Accident

This document is the compilation of the milestone portion to a larger end of project NEUP report. The executive summary of the modeling portion is provided below: In the event of cladding rupture during a postulated LOCA in a pressurized water reactor, fuel particles, along with fission gases, can be expelled into the reactor core from the fractured fuel rod, a phenomenon referred to as fuel dispersal. The initial stage of fuel dispersal is strongly influenced by the high-pressure ejection of fuel fragments, the size and geometry of the ruptured cladding, and the depressurization history of the fuel rod during the postulated LOCA transient. Depending on the location of the burst orifice relative to the quench front, the dispersal event represents an intricate three-phase flow and heat transfer phenomenon, where high-temperature fuel particles carried by the fission gases interact with the coolant within the narrow subchannels of the fuel assemblies, inducing localized phase change. Given the unique multiphysics nature of this phenomena, the current study develops a dedicated computational framework to predict the mass distribution and cooling of dispersing fuel particles, facilitating post-accident assessment and management of the fuel assemblies. Considering the scale of nuclear reactor applications, a continuum three-fluid model is proposed for simulating the transport of solids within the reactor core. With high-temperature fuel fragments within the liquid media, nucleation sites inducing phase changes are dispersed within the flow domain. Coupled with the fact that the transient dispersal event occurs on different time scales than other three-phase flow applications, this study derives a time-averaged three-fluid flow model without losing generality. The assumptions regarding the continuum treatment of the solid phase and the modeling of fuel dispersal behavior are incorporated to simplify the governing equations and derive applicable closure relations. The computational validation of the model was conducted using adiabatic experimental results obtained from ongoing research at Oregon State University, focusing on characterizing fuel dispersal behavior during simulated LOCA conditions. Settlement characteristics of the solids, quantified by the probability distribution of equivalent particles, closely matched the probability density functions reported in experimental studies. The transport of fuel particles within a scaled 5 × 5 lattice of a pressurized-water reactor rod bundle geometry was modeled through a two-fluid Eulerian framework. The required boundary conditions were evaluated from the fuel performance code BISON in a postulated large-break LOCA scenario. The modeling framework considered solid fuel particles as granular matter, interacting with the gaseous dry steam phase and fission gases through the governing interfacial momentum exchange between the participating fluids. The simulation results provided the volume fraction of the solids obtained at the bottom surface of the enclosing tank geometry. Postulated LOCA leading to fuel dispersal phenomena involves the strong coupling between fuel thermomechanics, cladding deformation, thermal-hydraulics, and fuel particle transport. Incorporation of such a strong coupling in numerical simulation is performed by coupling the multiphysics solvers. In the case of fuel dispersal, a strong coupled simulation can be performed by coupling the BISON code for fuel performance, the TRACE code for system-level thermal hydraulics, and fuel particle transport in Multiphysics Object-Oriented Simulation Environment (MOOSE). For such intricate infrastructure, the MOOSE Framework eases the data transfer between codes. The recent version of MOOSE has incorporated the Navier-Stokes module for the fluid flow. An exploratory exercise was done to gain familiarity with finite volume capabilities in the MOOSE framework to incorporate the Spalart-Allmaras (SA) turbulence model. New finite-volume and auxiliary kernels were introduced to assemble the SA transport equation, compute turbulent viscosity, and evaluate wall distance and diagnostic turbulence terms, fully integrated with existing Navier-Stokes modules. A turbulent lid-driven cavity at a Reynolds number of approximately 10,000 is used for verification. MOOSE shows the robust solver convergence and produces the turbulent features. But it underpredicts the velocity profile and turbulent quantities, emphasizing the need to develop improved SA near-wall treatments (e.g., low-Re corrections or wall functions) as a key direction for future work.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Summary of FY2024 Experimental Results to Support Development of New Inelastic Material Models and Validation of Section III, Division 5, Class A Design Rules

To address the data gap crucial for updating existing viscoplastic constitutive material models for the Class A materials in the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code, Section III, Division 5, High Temperature Reactors (ASME, 2023) for inelastic design analysis, Oak Ridge National Laboratory (ORNL) conducted experimental studies on Alloy 800H and Alloy 617 across a range of temperatures up to their maximum temperature limits in Section III, Division 5. The studies aim to characterize the materials' deformation behavior under both strain-controlled mechanical cyclic loading and thermal cycling conditions. This report summarizes ORNL's FY 2024 experimental findings on Alloy 800H and Alloy 617, focusing on pure fatigue tests, cyclic stress-strain curves, thermomechanical fatigue experiments, and verification of the high temperature cyclic damage summation design rules.

36 MATERIALS SCIENCE↗

PWR ENDF/B-VII cross-section libraries for ORIGEN-ARP

New pressurized water reactor (PWR) cross-section libraries were generated for use with the ORIGEN-ARP depletion sequence in the SCALE nuclear analysis code system. These libraries are based on ENDF/B-VII nuclear data and were generated using the two-dimensional depletion sequence, TRITON/NEWT, in SCALE 6.1. The libraries contain multiple burnup-dependent cross-sections for seven PWR fuel designs, with enrichments ranging from 1.5 to 6 wt% {sup 235}U. The burnup range has been extended from the 72 GWd/MTU used in previous versions of the libraries to 90 GWd/MTU. Validation of the libraries using radiochemical assay measurements and decay heat measurements for PWR spent fuel showed good agreement between calculated and experimental data. Verification against detailed TRITON simulations for the considered assembly designs showed that depletion calculations performed in ORIGEN-ARP with the pre-generated libraries provide similar results as obtained with direct TRITON depletion, while greatly reducing the computation time. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Multiphysics Analysis of the MSRE Experiment Using Griffin-SAM Coupled Code System

Molten salt reactors (MSRs) with flowing fuel have the unique feature of utilizing the fuel salt for heat generation and extraction at the same time since the fuel salt is circulating through the whole primary loop of the reactor. This movement of the fuel salt results in a partial decay of the delayed neutron precursors (DNPs) outside the core and corresponding redistribution in the active core region. To capture this phenomenon accurately, neutronics and thermal hydraulics computational tools need to be modified to handle the movement of the DNPs and their decay. Idaho and Argonne National Laboratories are actively working on developing neutronics and thermal hydraulics tools to model MSRs with flowing fuel. The neutronics code Griffin and the thermal hydraulics code SAM were extended to handle flowing fuel with the drift of the DNPs under the MOOSE framework. In this work, a simplified Multiphysics model of the Molten Salt Reactor Experiment (MSRE) is developed and utilized to perform steady-state and transient analyses. The MSRE experiments were designed to show the impact of the DNPs losses on core reactivity. The following section provides a description of the MSRE experiment with the core main parameters used to develop the model. Then, the developed Multiphysics model of the MSRE is presented. Finally, steady-state verification tests and unprotected loss of flow transient tests are discussed followed by a summary and conclusions.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Serpent - Bison - THM Preliminary Multiphysics Modeling of a Nuclear Thermal Propulsion System Fuel Assembly

This work demonstrates the Monte Carlo neutronic and Thermo-Hydraulic coupling scheme using the Serpent code and MOOSE application Bison and Thermo Hydraulic Module. The coupling scheme is then applied to the reference BWX Technologies Nuclear Ther- mal Propulsion system at he fuel assembly level where it’s used to perform an analysis of the isothermal material coefficients and potential material reactivity worth. A method is developed to isolate which feedback effects should be considered for proceeding with reduced order deterministic neutronic modeling where branch off analysis must be con- ducted. The convergence behavior of the coupling scheme is demonstrated where it fol- lows the standard Picard iteration approach. Verification studies for the method of deduc- ing relevant feedback effects is also demonstrated.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Particulate Fuel Modeling of MC 2 -3 using Iterative Local Spatial Self-shielding Method

We report a new spatial self-shielding method for particulate fuels has been developed based on disadvantage factors and implemented in the MC 2 -3 code. This method named the iterative local spatial self-shielding (ILSS) method considers the shadowing effect of randomly distributed particles on spatial self-shielding in particles through a homogenized composition region added outside the particle of interest at the center. The self-shielded cross sections of the central particle are determined iteratively since they are used in determining the cross sections of the homogenized composition region. The ILSS method was verified for infinite stochastic medium problems of single and multiple types of particles, VHTR unit cell problems, and HTTR assembly problems. The verification test results show that the ILSS method accurately predicts the stochastic particle shadowing effect and reaction rates in particles, whereas the regular array model and the stochastic collision probability method underpredict the particle shadowing effect and overestimate reaction rates in particles.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Ocean Energy Systems Wave Energy Modelling Task: Modelling, Verification and Validation of Wave Energy Converters

The International Energy Agency Technology Collaboration Programme for Ocean Energy Systems (OES) initiated the OES Wave Energy Conversion Modelling Task, which focused on the verification and validation of numerical models for simulating wave energy converters (WECs). The long-term goal is to assess the accuracy of and establish confidence in the use of numerical models used in design as well as power performance assessment of WECs. To establish this confidence, the authors used different existing computational modelling tools to simulate given tasks to identify uncertainties related to simulation methodologies: (i) linear potential flow methods; (ii) weakly nonlinear Froude–Krylov methods; and (iii) fully nonlinear methods (fully nonlinear potential flow and Navier–Stokes models). This article summarizes the code-to-code task and code-to-experiment task that have been performed so far in this project, with a focus on investigating the impact of different levels of nonlinearities in the numerical models. Two different WECs were studied and simulated. The first was a heaving semi-submerged sphere, where free-decay tests and both regular and irregular wave cases were investigated in a code-to-code comparison. The second case was a heaving float corresponding to a physical model tested in a wave tank. We considered radiation, diffraction, and regular wave cases and compared quantities, such as the WEC motion, power output and hydrodynamic loading.

16 TIDAL AND WAVE POWER↗

SAM Code Enhancements for Fission Product Tracking of Noble Gases and Metals in MSRs

This report documents fiscal year 2026 enhancements to the System Analysis Module (SAM) for modeling fission product transport in liquid-fueled molten salt reactors (MSRs). The work advances three principal areas: noble gas transport, noble metal deposition, and user interface improvements. The noble gas transport capability integrates drift-flux gas transport, Henry’s law two-film interphase mass transfer with pressure-based nucleation suppression, Knudsen-regime pore diffusion into porous graphite with a conjugate salt-graphite interface constraint, built-in material properties, five Sherwood-number mass transfer correlations including three derived from high-fidelity NekRS simulations, and xenon-135 reactivity feedback through SAM’s point-kinetics model. This work also presents a comprehensive verification test suite, including new analytically verified cases for pressure-dependent onset of interphase gas transfer in a stagnant vertical pipe, a postulated FLiBe-graphite Xe extraction permeator, a gravity riser with a fission-product source, and a descending pipe with gas redissolution driven by hydrostatic pressure. A machine learning framework for bubble rise velocity prediction in molten salt systems is developed and benchmarked on molten-salt and diverse aqueous bubble datasets. The best-performing fine-tuned transfer-learning networks achieve an 82% reduction in RMSE relative to the Clift correlation, and is implemented directly in SAM. The noble metal transport capability is developed, including a liquid-wall deposition model and a gas-surface flotation mechanism that transfers insoluble particles entrained by sparging gas to wetted structures. Verification tests and demonstration cases cover the surface deposition, flotation efflux, and flotation shedding. Finally, a new [SpeciesTransport] input structure replaces positional global vectors with selfcontained, order-independent, named species blocks, simplifies the specification of multiphase species and decay chains, and remains fully compatible with existing SAM input files. Together, these developments improve the physical fidelity, verification basis, and usability of SAM for system-level analyses of fissionproduct behavior in MSRs.

Mui, Travis (ORCID:0000000303736470)↗

Free-boundary MRxMHD equilibrium calculations using the stepped-pressure equilibrium code

Here the stepped-pressure equilibrium code (SPEC) (Hudson et al 2012 Phys. Plasmas 19 , 112 502) is extended to enable free-boundary multi-region relaxed magnetohydrodynamic (MRxMHD) equilibrium calculations. The vacuum field surrounding the plasma inside an arbitrary 'computational boundary', , is computed, and the virtual-casing principle is used iteratively to compute the normal field on so that the equilibrium is consistent with an externally produced magnetic field. Recent modifications to SPEC are described, such as the use of Chebyshev polynomials to describe the radial dependence of the magnetic vector potential, and a variety of free-boundary verification calculations are presented.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Efforts to enhance reproducibility in a human performance research project

Background: Ensuring the validity of results from funded programs is a critical concern for agencies that sponsor biological research. In recent years, the open science movement has sought to promote reproducibility by encouraging sharing not only of finished manuscripts but also of data and code supporting their findings. While these innovations have lent support to third-party efforts to replicate calculations underlying key results in the scientific literature, fields of inquiry where privacy considerations or other sensitivities preclude the broad distribution of raw data or analysis may require a more targeted approach to promote the quality of research output. Methods: We describe efforts oriented toward this goal that were implemented in one human performance research program, Measuring Biological Aptitude, organized by the Defense Advanced Research Project Agency's Biological Technologies Office. Our team implemented a four-pronged independent verification and validation (IV&V) strategy including 1) a centralized data storage and exchange platform, 2) quality assurance and quality control (QA/QC) of data collection, 3) test and evaluation of performer models, and 4) an archival software and data repository. Results: Our IV&V plan was carried out with assistance from both the funding agency and participating teams of researchers. QA/QC of data acquisition aided in process improvement and the flagging of experimental errors. Holdout validation set tests provided an independent gauge of model performance. Conclusions: In circumstances that do not support a fully open approach to scientific criticism, standing up independent teams to cross-check and validate the results generated by primary investigators can be an important tool to promote reproducibility of results.

59 BASIC BIOLOGICAL SCIENCES↗