Search NASA⌕ Search

SEARCH · Search NASA

Results for “Validation Methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A minimally invasive, efficient method for propagation of full-field uncertainty in solid dynamics

In this work, we present a minimally invasive method for forward propagation of material property uncertainty to full-field quantities of interest in solid dynamics. Full-field uncertainty quantification enables the design of complex systems where quantities of interest, such as failure points, are not known a priori. The method, motivated by the well-known probability density function (PDF) propagation method of turbulence modeling, uses an ensemble of solutions to provide the joint PDF of desired quantities at every point in the domain. A small subset of the ensemble is computed exactly, and the remainder of the samples are computed with approximation of the evolution equations based on those exact solutions. Although the proposed method has commonalities with traditional interpolatory stochastic collocation methods applied directly to quantities of interest, it is distinct and exploits the parameter dependence and smoothness of the driving term of the evolution equations. The implementation is model independent, storage and communication efficient, and straightforward. We demonstrate its efficiency, accuracy, scaling with dimension of the parameter space, and convergence in distribution with two problems: a quasi-one-dimensional bar impact, and a two material notched plate impact. For the bar impact problem, we provide an analytical solution to PDF of the solution fields for method validation. With the notched plate problem, we also demonstrate good parallel efficiency and scaling of the method.

42 ENGINEERING↗

Integral Nuclear Data and Benchmarking Needs for Fusion Energy Systems

Fusion energy systems are currently being designed and optimized using radiation transport codes. To deal with the unique environment inside a fusion-based system, many of these designs incorporate novel materials able to withstand the high radiation fields, ensure adequate cooling and thermal protection, and produce tritium. Validation plays a vital role in building trust in the predictive power of these models and computational methods. Validation of a code consists of modeling documented real-world experiments and comparing the code-predicted response to the measured response. Adequate validation requires measured responses from real-world experiments, also known as integral data, that mimic the system being designed, including materials, impinging radiation, and temperature, among other variables. The most trusted integral data are experimental responses that have been through a rigorous benchmarking process that develops a recommended computational model and evaluates all experimental uncertainties. Finally, there are a few research groups around the world that have been producing integral data for fusion applications, but a substantial investment is needed to address the unique validation needs of the fusion community.

Fusion↗

TPSAS-NF1676L-30290-DND

The Summer 2017 Ozone Water Land Environmental Transition Study (OWLETS) mission set out to compare the differences in ozone concentrations between inland measurements of air quality, specifically at Langley Research Center (LaRC), and measurements taken over water, specifically on the Chesapeake Bay Bridge Tunnel (CBBT), as well as the vertical profiles above each location. Current weather models often predict a gradient in ozone concentration between land and ocean and this campaign attempted to capture this difference using hand-held Personal Ozone Monitors (POMs) in correspondence to ozonesondes and LIDAR measurements. The Tropospheric Emissions: Monitoring of Pollution (TEMPO) is the first geostationary satellite that will take hourly measurements to monitor air pollutants across North America using solar backscatter. The small footprint allows for higher spatial resolution readings of many parameters including O3, NO2 , and aerosol. TEMPO’s higher resolution readings would benefit from validation techniques on the ground. Validation methods usually include comparison to air quality monitoring stations, but they could also incorporate other forms of validation such as comparison to small sensors. In cooperation with TEMPO, OWLETS aims to provide the user community high resolution temporal and spatial, both horizontal and vertical, variability of O3 simultaneously over the land and water during various air quality events to improve forecast models and air quality satellite retrievals. Together, these missions will improve spatial resolution and capture temporal variability of air quality over North America.

Emily Gargulinski↗

Advanced mass balance characterization and fractionation of algal biomass composition

Abstract Opportunities associated with biomass production and bioproduct isolation from algae-derived feedstocks are plentiful and promising; however, there are challenges associated with realizing these applications. One of the most important, and often overlooked, challenges is the lack of availability of a strong foundation of compositional analysis methods validated on microalgal biomass. Currently, compositional analysis in algae is dominated by the use of interference-prone methods, a lack of full mass balance accounting, and the use of top-down approaches that bin all unaccounted-for mass into a single category, such as carbohydrates. We present here an approach based on a bottom-up algal biomass characterization aimed at moving towards, and highlighting the importance of, full and accurate mass closure to achieve the maximum economic potential from a sustainable and renewable feedstock. Algal biomass representing three genera, Nannochloropsis , Scenedesmus , and Monoraphidium , was subjected to a cell rupture and fractionation process, followed by detailed characterization of each fraction to determine the partitioning of measured and unknown components. The goal of this work is to identify where the missing components partition, and develop a strategy to close the mass balance or identify the unknowns, while utilizing a rigorous characterization approach for characterizing algal biomass. Although only 75–80% of the biomass was accounted for, the fractionation approach utilized here provides key insight into possible chemical components for future investigations.

59 BASIC BIOLOGICAL SCIENCES↗

Statistical Validation of Multiple Related Data Sets—Case Study Using Interstellar Boundary Explorer Satellite Data

Abstract Space scientists often face the question of whether data collected by different instruments are measurements of the same source population. This paper proposes a statistical validation method for evaluating the agreement between such related data sets. It offers a detailed case study focused on validating a new data set from the Interstellar Boundary Explorer (IBEX) mission, which serves as a practical how-to guide for similar analyses. Since 2008, the IBEX satellite has been gathering data on heliospheric energetic neutral atoms (ENAs) while being exposed to various sources of background noise, such as cosmic rays and solar energetic particles. The IBEX mission initially released only a qualified triple-coincidence (qABC) data product, which was designed to provide observations of ENAs free of background contamination. Further measurements revealed that the qABC data were in fact susceptible to contamination, having relatively low ENA counts and high background rates. To mitigate this issue, the mission team recently considered releasing a certain qualified double-coincidence (qBC) data product, which has roughly twice the detection rate of the qABC data product. This paper presents a simulation-based validation of the new qBC data product against the already-released qABC data product. The results show that the qBCs can plausibly be said to be measuring the same source population as the qABCs up to an average absolute deviation of 3.6%. Visual diagnostics provide additional confirmation of source rate coherence across data products. The framework introduced here is general and can be applied to other validation problems both within and outside the field of space physics.

79 ASTRONOMY AND ASTROPHYSICS↗

Evaluating the feasibility of LA-ICP-TOF-MS for the analysis of environmental particle collections

Laser ablation-inductively coupled plasma-time-of-flight-mass spectrometry (LA-ICP-TOF-MS) was employed to rapidly analyze environmental particle samples collected using aerosol contaminate extractors (ACE). The ACE particle collectors were placed at various distances (0.5, 1.3, and 4.5 km) from a source that released Ru-bearing particles. Samples for measurement were then generated (as sub-samples) from the ACE collection plates via particle “lift off” with gunshot residue (GSR) tabs. The LA-ICP-TOF-MS method was employed such that 10+ samples could be analyzed in a single unattended analytical session. A 3 × 1 mm area of individual GSR tab samples were analyzed in less than 30 minutes. This provided spatially resolved elemental and isotopic measurements of the particulate content and confirmed the presence of Ru-bearing particles within the complex background environmental particle loading. As anticipated, measurements showed collectors closest to the source had the highest concentration of the released Ru-bearing particles, while all collectors, regardless of distance, contained similar levels of background particles (e.g., Fe and Sr). Sequential scanning electron microscopy – automated particle analysis (SEM-APA) and LA-ICP-TOF-MS analysis was employed for method validation and a demonstration of the multi-modal approach. The same 2-dimensional region was analyzed by both methods and the particles identified via SEM-APA were also detected using LA-ICP-TOF-MS, with 100% accuracy. Overall, LA-ICP-TOF-MS demonstrated its utility for rapid elemental and isotopic particle analysis from environmental air samples.

Manard, Benjamin T. [Oak Ridge National Laboratory↗

Utilization of Low Gravity Environment for Measuring Liquid Viscosity

The method of drop coalescence is used for determining the viscosity of highly viscous undercooled liquids. Low gravity environment is necessary in order to allow for examining large volumes affording much higher accuracy for the viscosity calculations than possible for smaller volumes available under 1 - g conditions. The drop coalescence method is preferred over the drop oscillation technique since the latter method can only be applied for liquids with vanishingly small viscosities. The technique developed relies on both the highly accurate solution of the Navier-Stokes equations as well as on data from experiments conducted in near zero gravity environment. Results are presented for method validation experiments recently performed on board the NASA/KC-135 aircraft. While the numerical solution was produced using the Boundary Element Method. In these tests the viscosity of a highly viscous liquid, glycerine at room temperature, was determined using the liquid coalescence method. The results from these experiments will be discussed.

Antar, Basil N.↗

TROPOMI Methane Validation in the GeoCarb Domain

Objective: GeoCarb validation •Purpose: •Do the four TCCON sites in the GeoCarb domain characterize satellite validation? •Method: •Assess the error estimates globally vs. in the GeoCarb domain •Assess the use of GML/DOE aircraft for CH4 validation •Systematic error estimate (from Kulawik, 2016): •+ regional bias (stdevof average bias versus validation at each station) •+ correlated error (stdev of daily average versus validation) •-co-location error (above quantities for model@satelliteminus model@valid) •-validation error (systematic error of validation estimates, e.g. model extension) Aircraft validation error •CH4 profile extension error estimate: 7 ppb •Extend aircraft observations to the top of the atmosphere with different model configurations •CAMS, scaling to stratosphere, then taking stratosphere •Extend by repeating top aircraft value to the troposphere, then taking CAMS •Extend with GEOS-Chem run (from 2010). •Difference CAMS-Scale vs. CAMS-Tropo: •3.6 +-5.5 ppb •Systematic error: 6.6 ppb •Extending aircraft to tropopause is blatantly wrong, but these two have the same stratosphere. •Difference CAMS-Scale vs. GEOS-Scale: •6.2 +-8.7 ppb •Do not consider bias since model is 10 years off. Systematic error ~8.7 ppb. •Estimate of systematic error from profile extension (possibly underestimated, resulting in possibly higher TROPOMI error estimate) •7 ppb •Would likely vary some by season, site, other conditions •Aircraft measurement error: 1 ppb •1 ppb from measurement error3

TROMPOMI↗

Validation of a High-Order Compact Code for Nonlinear Flows About Complex Geometric

The slides review computational requirements for nozzle exhaust flow and noise calculations and the current numerical method, validation of prefactored compact scheme on CAA benchmark problems, a curvilinear grid performance test of gust response of a Joukowski airfoil, airfoil surface RMS pressure distribution and far field noise radiation results for Joukowski airfoil in a vortical gust, boundary distance study for Joukowski airfoil problem, and performance of ICOMP parallel Macintosh cluster.

Hixon, Ray↗

An Investigation Into Transecting Satellites in Future Space Traffic Management Scenarios

The number of satellites expected to populate the near-Earth space environment is set to dramatically increase in the coming decade as new large constellations are approved and deployed. Current strategies for placing new batches of these satellites on orbit often involve launching into an initial orbit, and then performing apogee raising maneuvers to reach a target altitude. Similarly, end-of-life planning for these constellation satellites can consist of de-orbit burns that lower perigee to permit disposal via re-entry. Both the raising and de-orbiting maneuvers can result in the individual satellites traveling in high-eccentricity orbits that have the potential to transect other spacecraft trajectories. While individual large constellations may be able to coexist in separate altitude and inclination bands, having thousands of satellites moving between these bands as new satellites are replaced and old ones are removed could pose additional collision risks. Similar concerns have been raised regarding the impact that large numbers of university-class CubeSats might have in terms of their overall collision risk, especially as these satellites typically do not have propulsion systems for active maneuvering. To assess the impact that transecting satellites might have to future space traffic management strategies, this study explored a variety of future realistic scenarios using a high-fidelity simulation tool. The model can simulate the orbit of tens of thousands of resident space objects (RSOs) simultaneously, to include active satellites, debris, rocket bodies, or even future hypothetical scenarios, using a realistic force model that incorporates non-spherical gravity, atmospheric drag, solar radiation pressure, and more. As the model is run forward in time, various statistics and meta-data are gathered on any predicted conjunction event, providing insight into the nature and frequency of potential collisions, e.g., what size are the two satellites, who operates the satellites, are they active or passive objects, etc. Additional customization is available in terms of how probability of collision is computed, and how the probability ellipsoids and screening volumes are determined. The simulation tool also allows for rule-based maneuvers for active satellites, e.g, given an advance conjunction “warning,” one or both of the satellites can maneuver to a safe distance. A wide range of maneuvers can be implemented using impulsive or low-thrust methods, and the latencies can also be varied, e.g., using maneuver lead times of 48 hours, 24 hours, or 12 hours. Validation of the simulation results is performed against current and historical datasets available, to include comparisons to prior conjunction data messages (CDMs), object properties (mass, volume, etc.), and two-line-element records from both public and internal sources. Using the simulation environment, an assessment on the general risks that transecting satellites might pose for hypothetical future space object environments will be presented. This will involve the simulation of approximately 50,000 new large constellation satellites, in addition to the existing catalog of approximately 20,000 known resident space objects (RSOs), over propagation periods of one month to one year. A description of the simulation methodologies, scenarios evaluated, and validation methods will also be presented, as well as a preliminary assessment of the effectiveness of several candidate maneuver strategies that have the potential to reduce collision risk between active satellites.

Brian Gunter↗

Display format, highlight validity, and highlight method: Their effects on search performance

Display format and highlight validity were shown to affect visual display search performance; however, these studies were conducted on small, artificial displays of alphanumeric stimuli. A study manipulating these variables was conducted using realistic, complex Space Shuttle information displays. A 2x2x3 within-subjects analysis of variance found that search times were faster for items in reformatted displays than for current displays. Responses to valid applications of highlight were significantly faster than responses to non or invalidly highlighted applications. The significant format by highlight validity interaction showed that there was little difference in response time to both current and reformatted displays when the highlight validity was applied; however, under the non or invalid highlight conditions, search times were faster with reformatted displays. A separate within-subject analysis of variance of display format, highlight validity, and several highlight methods did not reveal a main effect of highlight method. In addition, observed display search times were compared to search time predicted by Tullis' Display Analysis Program. Benefits of highlighting and reformatting displays to enhance search and the necessity to consider highlight validity and format characteristics in tandem for predicting search performance are discussed.

Donner, Kimberly A.↗

Implications of Simulation Conceptual Model Development for Simulation Management and Uncertainty Assessment

A simulation conceptual model is a simulation developers way of translating modeling requirements (i. e., what is to be represented by the simulation or its modification) into a detailed design framework (i. e., how it is to be done), from which the software, hardware, networks (in the case of distributed simulation), and systems/equipment that will make up the simulation can be built or modified. A conceptual model is the collection of information which describes a simulation developers concept about the simulation and its pieces. That information consists of assumptions, algorithms, characteristics, relationships, and data. Taken together, these describe how the simulation developer understands what is to be represented by the simulation (entities, actions, tasks, processes, interactions, etc.) and how that representation will satisfy the requirements to which the simulation responds. Thus the conceptual model is the basis for judgment about simulation fidelity and validity for any condition that is not specifically tested. The more perspicuous and precise the conceptual model, the more likely it is that the simulation development will both fully satisfy requirements and allow demonstration that the requirements are satisfied (i. e., validation). Methods used in simulation conceptual model development have significant implications for simulation management and for assessment of simulation uncertainty. This paper suggests how to develop and document a simulation conceptual model so that the simulation fidelity and validity can be most effectively determined. These ideas for conceptual model development apply to all simulation varieties. The paper relates these ideas to uncertainty assessments as they relate to simulation fidelity and validity. The paper also explores implications for simulation management from conceptual model development methods, especially relative to reuse of simulation components.

Pace, Dale K.↗

Summary Report Of The FY25 Reactor Physics Verification And Validation Exercises In The Advanced Reactor Technologies - Gas-cooled Reactor Program

Valdiation and verification of numerical tools is critical for ensuring reasonable predictions for design scoping, licensing, and safety analsyis. In this report, two reactor physics verification and validation exercises are presented. The first of these exercises focuses on burnup analysis with data from the Advanced Gas Reactor (AGR) program. Simulations are performed with Monte Carlo N-Particle (MCNP) and are compared with the experimental measurements for the AGR 1 and 2 experiments that utilize both UCO and UO2 fuel. The second exercises utilizes data from the HTR-Proteus experiments to perform reactor physics validation. Specifications of the experimental facility are provdied, along with a demonstration of initial modeling efforts in Serpent for one of the determistic packing experiments. Both cases are part of the Generation-IV international forum (GIF) Very High-Temperature Reactor (VHTR) Computational Methods, Validation, and Benchmarking (CMVB) program, an international collaborative organization dedicated to the verification and validation of High-Temperature Gas-Cooled Reactor (HTGR) analysis. Participation in the CMVB allows the US Department of Energy (DOE) to leverage these existing validation activities to provide extra value through benchmarking activities with other CMVB members.

and Benchmarking (CMVB) program↗

Subcomponent Validation of Composite Joints for Marine Energy Structures

The Marine Energy Advanced Materials project addresses the barriers and uncertainties facing marine renewable energy developers in using composite materials for load-bearing structures. Sponsored by the U.S. Department of Energy's Water Power Technologies Office, the multiyear project comprises of collaborators from the National Renewable Energy Laboratory (NREL), Sandia National Laboratories, Pacific Northwest National Laboratory (PNNL), Montana State University (MSU), Florida Atlantic University (FAU), and industry stakeholders. As part of the Marine Energy Advanced Materials project, marine renewable energy industry surveys and assessments were conducted to identify key materials and knowledge gaps that hinder the adoption of composite materials in marine renewable energy structures. Specific knowledge gaps highlighted for composite materials were environmental effects, fatigue strengths, and bonded and bolted interconnects (composite/composite and composite/metal). It was concluded that many of these gaps could be addressed through subcomponent validation; consequently, a program was developed at NREL with the goals of developing subcomponent validation methods for appropriate marine energy materials, which would improve the understanding of design allowables for full-scale structural components and joints. Ultimately, the aim is to reduce timelines and costs associated with full-scale structural validation efforts while also providing near-net-scale static and fatigue data of composite/metal subcomponents for marine renewable energy systems. To approach these goals, a testing program was developed at NREL to investigate a variety of materials and structural design details at the subcomponent scale to understand (a) the effects of harsh and corrosive marine energy environments and (b) the static and fatigue strengths of the complex geometries. The recent study from this testing program is perhaps the largest that has ever been conducted with respect to specimen scale and geographic diversity of underwater environmental conditions that the specimens were subjected to. A variety of specimen geometries were designed by NREL to highlight key features of multimaterial (composite/composite or metal/composite) interconnects that may be used in marine renewable energy structural designs. The designs used several different composite matrices, adhesives, and marine-grade steels, which were highlighted in the surveys as being the most appropriate for harsh marine environments. Composite panels were then manufactured at MSU, and were subsequently manufactured into test specimens by NREL. The specimens were shipped to FAU and PNNL for conditioning in ocean water tanks at various temperatures for an extended period. The specimens were then returned to NREL for structural validation. This presentation will provide an overview of recent advances within the testing program in terms of specimen design and test methods, and will discuss results and key findings of the subcomponent testing program to date at NREL.

adhesives↗

Validation of Safety-Critical Systems for Aircraft Loss-of-Control Prevention and Recovery

Validation of technologies developed for loss of control (LOC) prevention and recovery poses significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the validation framework must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). This paper summarizes a proposed validation framework for safety-critical systems, provides an overview of validation methods and tools developed by NASA to date within the Vehicle Systems Safety Project, and develops a preliminary set of test scenarios for the validation of technologies for LOC prevention and recovery

Belcastro, Christine M.↗

Experimental validation of boundary element methods for noise prediction

Experimental validation of methods to predict radiated noise is presented. A combined finite element and boundary element model was used to predict the vibration and noise of a rectangular box excited by a mechanical shaker. The predicted noise was compared to sound power measured by the acoustic intensity method. Inaccuracies in the finite element model shifted the resonance frequencies by about 5 percent. The predicted and measured sound power levels agree within about 2.5 dB. In a second experiment, measured vibration data was used with a boundary element model to predict noise radiation from the top of an operating gearbox. The predicted and measured sound power for the gearbox agree within about 3 dB.

Seybert, A. F.↗

Validation of a fault-tolerant clock synchronization system

A validation method for the synchronization subsystem of a fault tolerant computer system is investigated. The method combines formal design verification with experimental testing. The design proof reduces the correctness of the clock synchronization system to the correctness of a set of axioms which are experimentally validated. Since the reliability requirements are often extreme, requiring the estimation of extremely large quantiles, an asymptotic approach to estimation in the tail of a distribution is employed.

Butler, R. W.↗

Validating the Autonomous Science Agent

This paper describes the validation process for the Autonomous Science Agent, a software agent that will fly onboard the EO-1 spacecraft from 2003-2004. This agent will recognize science events, retarget the spacecraft to respond to the science events, and reduce data downlink to only the highest value science data. The autonomous science agent has been designed using a layered architectural approach with specific redundant safeguards to reduce the risk of an agent malfunction to the EO-1 spacecraft. This 'safe' design is also in the process of being thoroughly validated by informal validation methods and extensive testing. This paper describes the analysis used to define agent safety, elements of the design that increase the safety of the agent, and the process being used to validate agent safety prior to the agent software controlling the spacecraft.

agent safety↗