Search NASA⌕ Search

SEARCH · Search NASA

Results for “Model validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
NASA NTRS2014

Learning the Task Management Space of an Aircraft Approach Model

Validating models of airspace operations is a particular challenge. These models are often aimed at finding and exploring safety violations, and aim to be accurate representations of real-world behavior. However, the rules governing the behavior are quite complex: nonlinear physics, operational modes, human behavior, and stochastic environmental concerns all determine the responses of the system. In this paper, we present a study on aircraft runway approaches as modeled in Georgia Tech's Work Models that Compute (WMC) simulation. We use a new learner, Genetic-Active Learning for Search-Based Software Engineering (GALE) to discover the Pareto frontiers defined by cognitive structures. These cognitive structures organize the prioritization and assignment of tasks of each pilot during approaches. We discuss the benefits of our approach, and also discuss future work necessary to enable uncertainty quantification.

Validation↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
NASA NTRS1993

Satellite data for diagnostics and for validation of model simulations

Two issues in the treatment of tropical convection in general circulation models are examined. First, several studies have found significant gradients in clear sky longwave fluxes near large convective systems. Increased upper tropospheric moisture associated with deep convection may explain the reduction in the longwave emission. Similar local gradients are not apparent in measurements from the Earth Radiation Budget Experiment (ERBE), an important data set for model validation. Thus the average cloud forcing and greenhouse effect derived from models and observations may differ systematically over warm tropical oceans. A comparison of ERBE fluxes with radiative calculations using coincident balloon-sonde atmospheric profiles indicates negligible systematic bias in the observations. The effect of convection on the clear sky fluxes may be localized to the edges of individual cloud systems. Second, the balance between shortwave and cloud forcing and longwave forcing is a persistent feature of tropical cloud systems in ERBE data. This cancellation effect has been used to diagnose problems in GCM (General Circulation Model) cloud fields on seasonal time scales. The daily record of net cloud radiative forcing is analyzed to determine the smallest spatial and temporal scales for the balance. The results show cancellation on periods as short as three days for regions smaller than 2.5 by 2.5 deg. The analysis indicates that the balance is primarily a local phenomenon characteristic of tropical convection. This is consistent with findings that the small cloud radiative forcing is due primarily to thick tropical cirrus. These results represent a particularly stringent test of convective parameterizations in GCM's with interactive ocean surfaces.

Collins, W. D.↗
DOE OSTI2021

Challenging Practices of Algebraic Battery Life Models through Statistical Validation and Model Identification via Machine-Learning

Various modeling techniques are used to predict the capacity fade of Li-ion batteries. Algebraic reduced-order models, which are inherently interpretable and computationally fast, are ideal for use in battery controllers, technoeconomic models, and multi-objective optimizations. For Li-ion batteries with graphite anodes, solid-electrolyte-interphase (SEI) growth on the graphite surface dominates fade. This fade is often modeled using physically informed equations, such as square-root of time for predicting solvent-diffusion limited SEI growth, and Arrhenius and Tafel-like equations predicting the temperature and state-of-charge rate dependencies. In some cases, completely empirical relationships are proposed. However, statistical validation is rarely conducted to evaluate model optimality, and only a handful of possible models are usually investigated. This article demonstrates a novel procedure for automatically identifying reduced-order degradation models from millions of algorithmically generated equations via bi-level optimization and symbolic regression. Identified models are statistically validated using cross-validation, sensitivity analysis, and uncertainty quantification via bootstrapping. On a LiFePO 4 /Graphite cell calendar aging data set, automatically identified models utilizing square-root, power law, stretched exponential, and sigmoidal functions result in greater accuracy and lower uncertainty than models identified by human experts, and demonstrate that previously known physical relationships can be empirically "rediscovered" using machine learning.

25 ENERGY STORAGE↗
DOE OSTI2019

Validating Performance Models for Hybrid Power Plant Control Assessment

The need for simple, but accurate performance models of wind turbine generators (WTGs), photovoltaic (PV) plants, and battery energy storage systems (BESS) for various hybrid power plant (HPP) studies motivates the present work. Particularly, the development and verification stage of HPP controls requires reduced-order models to minimize the complexity and computation effort of simulation platforms. In this paper, such models are proposed, and the most essential parts of the models are validated through field measurements. The models target power system integration studies involving active and reactive power, as well as frequency and voltage regulation where detailed models, as proposed in the standards, can be cumbersome. Field measurements of two Vestas WTGs, one 1-MW PV plant, and one 1-MW/1-MWh BESS are used for model validation. The results show that the WTG and PV performance models correctly estimate the power generation variability according to fluctuations in wind speed and solar irradiance. The BESS performance model provides satisfactory results related to grid-forming control performance and estimation of state-of-charge. The presented validation work enables using the proposed performance models for power system studies and HPP control design in all model-based design stages, that is, preliminary analysis, design, verification, and validation with a high level of confidence.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2018

Comparison of Bifacial Solar Irradiance Model Predictions With Field Validation

Models predicting rear irradiance for bifacial systems are critical to establish accurate estimates of energy yield. In this paper, we compare five published bifacial irradiance models, varying clearance, row spacing, tilt, and albedo to measure the sensitivity to these parameters. Bifacial energy gains (BG E ) as high as 20% are predicted for some configurations. Model agreement is generally good for low ground clearance (clearance heights lower than 0.75 times the collector width), but at higher clearances, finite system size and edge effects become a significant factor in simulations, stretching assumptions of infinite system extent made in some models. In addition, rear irradiance uniformity is improved at high ground clearance, as expected. A test-bed construction and results are described for comparison between modeled and measured data in Golden, CO, USA. Finally, the investigations indicate that model agreement for BG E calculation is better than 2% (absolute) when compared with measured results, depending on the system configuration.

14 SOLAR ENERGY↗
DOE OSTI2020

Calibrating Synchronous-Generator-Interfaced DG Models in Microgrids Using Multiple Event Data

Microgrids, consisting of distributed generators (DGs), loads, and energy storage systems, play an important role in future smart grids. In order to evaluate how microgrids operate and their dynamic response, it is necessary to develop accurate dynamic models of DGs in microgrids. This paper proposes a systematic method to calibrate synchronous-generator-interfaced DG models. Challenges associated with model validation in microgrids are analyzed. Underlying parameters are categorized into two groups, i.e., the steady-state parameters and time constants, which are estimated in two successive stages using multiple event data. This methodology ensures that the validated model with the estimated parameters are applicable for all events. A modified unscented Kalman filter (UKF) is proposed to deal with the cases in which measurement of power angle is unavailable. The effectiveness of the proposed method is validated using field test data of a 2.6 MVA diesel generator in a real microgrid.

Wang, Zhiwen↗
NASA NTRSDate not supplied

Behavior of Ullage Bubbles During Blowdown in Low-g Experiment (BUBBLE): Overview of a Cryogenic Tank Depressurization Test

Behavior of Ullage Bubbles during Blowdown in Low-g Experiment (BUBBLE) is a cryogenic tank depressurization drop tower experiment currently being designed at NASA Glenn Research Center. There is a need for liquid level rise data while venting a cryogenic propellant tank below the liquid saturation pressure in a reduced gravity environment. When a cryogenic tank is vented to a vapor pressure below the liquid saturation pressure, bubbles become entrained in the liquid causing the bulk liquid-vapor interface to rise. In reduced gravity, bubble rise velocity is diminished leading to a larger liquid level rise compared to a 1g environment. The purpose of the experiment is to gain further understanding of tank depressurization fluid physics and obtain high-fidelity data for model validation in 1g and reduced gravity environments. On-orbit cryogenic propellant tanks operating at high fill levels must efficiently manage venting operations to avoid the risk of liquid entrainment in the vent line, which could lead to asymmetric control thruster loads, freezing and clogging, and loss of liquid propellant. Validated models could be used to design settling and venting profiles to reduce risk and increase efficiency for cryogenic storage and transfer operations.

tank venting↗
NASA NTRS2016

Community-Wide Validation of Geospace Model Local K-Index Predictions to Support Model Transition to Operations

We present the latest result of a community-wide space weather model validation effort coordinated among the Community Coordinated Modeling Center (CCMC), NOAA Space Weather Prediction Center (SWPC), model developers, and the broader science community. Validation of geospace models is a critical activity for both building confidence in the science results produced by the models and in assessing the suitability of the models for transition to operations. Indeed, a primary motivation of this work is supporting NOAA/SWPCs effort to select a model or models to be transitioned into operations. Our validation efforts focus on the ability of the models to reproduce a regional index of geomagnetic disturbance, the local K-index. Our analysis includes six events representing a range of geomagnetic activity conditions and six geomagnetic observatories representing midlatitude and high-latitude locations. Contingency tables, skill scores, and distribution metrics are used for the quantitative analysis of model performance. We consider model performance on an event-by-event basis, aggregated over events, at specific station locations, and separated into high-latitude and midlatitude domains. A summary of results is presented in this report, and an online tool for detailed analysis is available at the CCMC.

Glocer, A.↗
DOE OSTI2021

An Application for Validation of Power Distribution System Models in an ADMS Environment

An accurate model of a power distribution system is the foundation for model-based applications that ensure efficient and reliable grid operation in an advanced distribution management system (ADMS) environment. However, these models are error-prone and comprehensive model validation is challenging due to lack of standards-based systems, data originating from disparate databases and other sources, and the constantly evolving nature of modern power distribution systems. In this paper, a novel framework for comprehensive model validation is described. The proposed application, the Model Validator, ensures that a model is both consistent and feasible by validating the derivative static and operational network model. A modular architecture for the application has been implemented and integrated with an open-source standards-based platform for ADMS application development, GridAPPS-D, allowing new validation capability to be added with minimal time and effort. The Model Validator application is demonstrated on the IEEE 13-bus, 123-bus, and 8500-node test cases over three validation scenarios.

Poudel, Shiva↗
NASA NTRS2020

Validation on the Europa Clipper Mission in the Formulation Phase

The Europa Clipper mission will perform a detailed investigation of Europa’s environment and habitability by placing a spacecraft into a looping orbit around Jupiter and performing repeated close Europa flybys. The Europa Clipper Project Verification and Validation (V&V) team is responsible for verifying that the Project System is compliant with the science and engineering requirements levied on it and validating that it is capable of meeting the mission objectives. Europa Clipper has developed a V&V framework to ensure that mission capabilities are realized at a reasonable confidence level prior to launch. The V&V team will coordinate the validation activities defined by this framework throughout the project lifecycle as the spacecraft evolves from design to implementation and operations. Validation of space missions is typically grouped into three categories: 1) requirements validation, 2) model validation, and 3) system validation. System validation, along with verification activities, are often the primary focus of V&V programs and these efforts peak in the implementation phase. However, the increasing reliance of exploration missions on models and simulations, and the complexity of the Europa Clipper mission, have dictated attention to requirements and model validation in the formulation phase. This paper discusses the development of the Europa Clipper V&V framework with emphasis on the validation of models and simulations, and of requirements. Approaches include the adaptation of institutional guidelines, such as those in the NASA-STD-7009 document, to create a Project Model V&V Plan and the use of tools such as DOORS NG to plan model validation and derive requirement quality metrics.

Srivastava, Priyanka↗
NASA NTRS2013

Community-wide Validation of Geospace Model Ground Magnetic Field Perturbation Predictions to Support Model Transition to Operations

In this paper we continue the community-wide rigorous modern space weather model validation efforts carried out within GEM, CEDAR and SHINE programs. In this particular effort, in coordination among the Community Coordinated Modeling Center (CCMC), NOAA Space Weather Prediction Center (SWPC), modelers, and science community, we focus on studying the models' capability to reproduce observed ground magnetic field fluctuations, which are closely related to geomagnetically induced current phenomenon. One of the primary motivations of the work is to support NOAA SWPC in their selection of the next numerical model that will be transitioned into operations. Six geomagnetic events and 12 geomagnetic observatories were selected for validation.While modeled and observed magnetic field time series are available for all 12 stations, the primary metrics analysis is based on six stations that were selected to represent the high-latitude and mid-latitude locations. Events-based analysis and the corresponding contingency tables were built for each event and each station. The elements in the contingency table were then used to calculate Probability of Detection (POD), Probability of False Detection (POFD) and Heidke Skill Score (HSS) for rigorous quantification of the models' performance. In this paper the summary results of the metrics analyses are reported in terms of POD, POFD and HSS. More detailed analyses can be carried out using the event by event contingency tables provided as an online appendix. An online interface built at CCMC and described in the supporting information is also available for more detailed time series analyses.

MAGNETIC↗
NASA NTRS1997

Importance of Sea Ice for Validating Global Climate Models

Reproduction of current day large-scale physical features and processes is a critical test of global climate model performance. Without this benchmark, prognoses of future climate conditions are at best speculation. A fundamental question relevant to this issue is, which processes and observations are both robust and sensitive enough to be used for model validation and furthermore are they also indicators of the problem at hand? In the case of global climate, one of the problems at hand is to distinguish between anthropogenic and naturally occuring climate responses. The polar regions provide an excellent testing ground to examine this problem because few humans make their livelihood there, such that anthropogenic influences in the polar regions usually spawn from global redistribution of a source originating elsewhere. Concomitantly, polar regions are one of the few places where responses to climate are non-anthropogenic. Thus, if an anthropogenic effect has reached the polar regions (e.g. the case of upper atmospheric ozone sensitivity to CFCs), it has most likely had an impact globally but is more difficult to sort out from local effects in areas where anthropogenic activity is high. Within this context, sea ice has served as both a monitoring platform and sensitivity parameter of polar climate response since the time of Fridtjof Nansen. Sea ice resides in the polar regions at the air-sea interface such that changes in either the global atmospheric or oceanic circulation set up complex non-linear responses in sea ice which are uniquely determined. Sea ice currently covers a maximum of about 7% of the earth's surface but was completely absent during the Jurassic Period and far more extensive during the various ice ages. It is also geophysically very thin (typically <10 m in Arctic, <3 m in Antarctic) compared to the troposphere (roughly 10 km) and deep ocean (roughly 3 to 4 km). Because of these unique conditions, polar researchers regard sea ice as one of the more important features to monitor in terms of heat, mass, and momentum transfer between the air and sea and furthermore, the impact of such responses to global climate.

Geiger, Cathleen A.↗
NASA NTRS2016

OVERFLOW Turbulence Modeling Resource Validation Results

Abstract:We exercise the computational fluid dynamics code OVERFLOW on sixteen turbulence model validation cases from the NASALangley Turbulence Model Resource web site. We give some information about the OVERFLOW options used to run these cases, and compare OVERFLOW results with results from other codes and with experiment. The goal is turbulence model validation for OVERFLOW.

computational fluid dynamics↗
DOE OSTI2026

A comprehensive framework for validating simulation models of power system equipment using terminal measurements

Accurate simulation of power-plants is essential to the planning and operation of modern power grids. The current methods used to periodically check power-plant simulation models have many open questions about their limitations and accuracy. The research in this project explored using Monte-Carlo Experimentation (MCE) as a means for answering these important questions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

DIII-D research advancing the physics basis for optimizing the tokamak approach to fusion energy

DIII-D physics research addresses critical challenges for the operation of ITER and the next generation of fusion energy devices. This is done through a focus on innovations to provide solutions for high performance long pulse operation, coupled with fundamental plasma physics understanding and model validation, to drive scenario development by integrating high performance core and boundary plasmas. Substantial increases in off-axis current drive efficiency from an innovative top launch system for EC power, and in pressure broadening for Alfven eigenmode control from a co-/counter-I p steerable off-axis neutral beam, all improve the prospects for optimization of future long pulse/steady state high performance tokamak operation. Fundamental studies into the modes that drive the evolution of the pedestal pressure profile and electron vs ion heat flux validate predictive models of pedestal recovery after ELMs. Understanding the physics mechanisms of ELM control and density pumpout by 3D magnetic perturbation fields leads to confident predictions for ITER and future devices. Validated modeling of high-Z shattered pellet injection for disruption mitigation, runaway electron dissipation, and techniques for disruption prediction and avoidance including machine learning, give confidence in handling disruptivity for future devices. For the non-nuclear phase of ITER, two actuators are identified to lower the L–H threshold power in hydrogen plasmas. With this physics understanding and suite of capabilities, a high poloidal beta optimized-core scenario with an internal transport barrier that projects nearly to Q = 10 in ITER at ~8 MA was coupled to a detached divertor, and a near super H-mode optimized-pedestal scenario with co-I p beam injection was coupled to a radiative divertor. The hybrid core scenario was achieved directly, without the need for anomalous current diffusion, using off-axis current drive actuators. Also, a controller to assess proximity to stability limits and regulate β N in the ITER baseline scenario, based on plasma response to probing 3D fields, was demonstrated. Finally, innovative tokamak operation using a negative triangularity shape showed many attractive features for future pilot plant operation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Diii-D Research Advancing The Physics Basis For Optimizing The Tokamak Approach To Fusion Energy

DIII-D physics research addresses critical challenges for operation of ITER and the next generation of fusion energy devices. This is done through a focus on innovations to provide solutions for high performance long pulse operation, coupled with fundamental plasma physics understanding and model validation, to drive scenario development by integrating high performance core and boundary plasmas. Substantial increases in off-axis current drive efficiency from an innovative top launch system for EC power, and in pressure broadening for Alfven eigenmode control from a co-/counter-Ip steerable off-axis neutral beam, all improve the prospects for optimization of future long pulse/steady state high performance tokamak operation. Fundamental studies into the modes that drive the evolution of the pedestal pressure profile and electron vs. ion heat flux validate predictive models of pedestal recovery after ELMs. Understanding the physics mechanisms of ELM control and density pumpout by 3D magnetic perturbation fields leads to confident predictions for ITER and future devices. Validated modeling of high-Z shattered pellet injection for disruption mitigation, runaway electron dissipation, and techniques for disruption prediction and avoidance including machine learning, give confidence in handling disruptivity for future devices. With this physics understanding and suite of actuators, a high poloidal beta optimized-core scenario with an internal transport barrier that projects nearly to Q=10 in ITER at 9 MA was coupled to a detached divertor, and a near Super H-mode optimized-pedestal scenario with co-Ip beam injection, was coupled to a radiative divertor. The Hybrid core scenario was achieved directly, without the need for internal flux pumping, using off-axis current drive actuators, and a controller to avoid mode locking and disruptions in the ITER baseline scenario, based on plasma response to probing 3D fields, was demonstrated. Finally innovative tokamak operation using a negative triangularity shape showed many attractive features for future pilot plant operation.

Fenstermacher, M.↗