Search NASASearch

SEARCH · Search NASA

Results for “Simulation Credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD

Pretest Simulations of a Supersonic Mixing and Combustion Validation Experiment to Assess Sensitivities

The reliance on CFD simulations to develop, design, and optimize scramjet systems (or components) has become commonplace. This reliance inevitably hinges on the ability of the computational analyst to quantify the level of confidence in their computational results. Unfortunately, nearly all the measured data available for this assessment comes from antiquated experimental datasets, or from tests that focused on the extraction of scramjet system (or component) performance. The objective of a CFD validation experiment is to quantify the predictive accuracy of one or more of the CFD physics submodels, implying that other uncertainties related to replicating the facility flow environment (e.g., knowledge of boundary conditions) must be minimized to the extent possible. This inevitably places stringent requirements on the quality and quantity of measurements taken to accurately specify inflow, outflow, and surface conditions for the CFD simulations; in addition to the measurements taken for the validation of physics submodels. This places additional demands on the experimental process above and beyond those for test article performance assessment. A recent high speed code credibility workshop series sponsored by AFRL identified a gap in existing validation data for fundamental assessments of turbulent mixing and combustion CFD closure models at scramjet engine relevant conditions. To address this gap, engineers at AFRL have designed a coaxial jet flame configuration that will be tested at two facilities (Research Cell 19 at the Air Force Research Lab, and at Purdue University). The effort described here documents pretest simulations of this validation experiment with the goal of fleshing out the extent of the facility flowpath that must be included to adequately reproduce the facility test section flow environment. The findings indicate that the flow around the support structure for the fuel injection centerbody upstream of the facility nozzle generates disturbances that persist throughout the nozzle expansion process; corrupting the azimuthal symmetry that was desired in the fuel/air mixing region of the test section. Simulations without this support structure maintained a high degree of azimuthal symmetry up until the fuel injection plane. However, even in this scenario the azimuthal symmetry was not maintained once the centerbody boundary layer transitioned to a wake flow downstream of the fuel injection plane.

CFD

Risk analysis simulation of rover operations for Mars surface exploration

Risk management advocates have long sought to directly influence the early stages of the systems engineering process through a more effective role in system design trade studies. The principal obstacle to this has been the lack of credible ways to represent and quantify mission risk—that is, a combination of the probability of mission success (“system safety”) and science value—for the project manager and the rest of the design team. If it were possible to quantify mission risk, then the effects of proposed mission and system design changes could be calculated, and along with life-cycle costs, could be used to explore the design space more extensively and select better designs. JPL has been working to build the capability to quantify the probability of mission success using a federation of diverse simulations and models, each of which contributes some vital piece of the puzzle. The initial institutional focus has been on Mars surface operations. This ensemble computing framework enables the diverse models and simulations to work together seamlessly. Recent work at JPL has demonstrated the capability to exercise this ensemble from end-to-end using an Oracle-based database to automatically move results from one model/simulation to the next stage in the analysis.

Shishko, Robert

Biopolymer-Templated Titania Film Formation for Nanostructured Coatings Revealed by Machine Learning-Supported Time-Resolved Analysis

This study presents a machine learning approach to derive the film formation of biopolymer-templated titania nanostructures during spray deposition, in combination with in situ grazing-incidence small-angle X-ray scattering (GISAXS). A neural network trained on synthetic GISAXS data directly predicts domain-size distributions from experimental two-dimensional scattering patterns, capturing the full kinetics of nanostructure evolution with high temporal resolution. The predictions reveal hierarchical size distributions and periodic growth features, consistent with layer-by-layer spray deposition and validated by complementary scanning electron microscopy (SEM) imaging. Quantitative comparison with conventional parametric GISAXS fits shows good qualitative agreement, with systematic differences explained by domain-shape assumptions and resolved by applying a geometric scaling factor. Simulated SEM-like surfaces derived from neural network outputs reproduce the porous, foam-like nanoscale morphology observed experimentally, reinforcing the method’s credibility. This integrated approach enables real-time, nondestructive, statistically averaged monitoring of bulk nanostructure development in functional coatings, offering a scalable methodology to accelerate the characterization and process control of sustainably manufactured nanostructured titania films for energy-related applications such as photocatalysis and photovoltaics.

Heger, JulianEliah

Verification Testing: Meet User Needs Figure of Merit

Verification is the process through which Modeling and Simulation(M&S) software goes to ensure that it has been rigorously tested and debugged for its intended use. Validation confirms that said software accurately models and represents the real world system. Credibility gives an assessment of the development and testing effort that the software has gone through as well as how accurate and reliable test results are. Together, these three components form Verification, Validation, and Credibility(VV&C), the process by which all NASA modeling software is to be tested to ensure that it is ready for implementation. NASA created this process following the CAIB (Columbia Accident Investigation Board) report seeking to understand the reasons the Columbia space shuttle failed during reentry. The reports conclusion was that the accident was fully avoidable, however, among other issues, the necessary data to make an informed decision was not there and the result was complete loss of the shuttle and crew. In an effort to mitigate this problem, NASA put out their Standard for Models and Simulations, currently in version NASA-STD-7009A, in which they detailed their recommendations, requirements and rationale for the different components of VV&C. They did this with the intention that it would allow for people receiving MS software to clearly understand and have data from the past development effort. This in turn would allow the people who had not worked with the MS software before to move forward with greater confidence and efficiency in their work. This particular project looks to perform Verification on several MATLAB (Registered Trademark)(The MathWorks, Inc.) scripts that will be later implemented in a website interface. It seeks to take note and define the limits of operation, the units and significance, and the expected datatype and format of the inputs and outputs of each of the scripts. This is intended to prevent the code from attempting to make incorrect or impossible calculations. Additionally, this project will look at the coding generally and note inconsistencies, redundancies, and other aspects that may become problematic or slow down the codes run time. Certain scripts lacking in documentation also will be commented and cataloged.

validation

Evaluation of GEOS-5 Sulfur Dioxide Simulations During the Frostburg, MD 2010 Field Campaign.

Sulfur dioxide (SO2) is a major atmospheric pollutant with a strong anthropogenic component mostly produced by the combustion of fossil fuel and other industrial activities. As a precursor of sulfate aerosols that affect climate, air quality, and human health, this gas needs to be monitored on a global scale. Global climate and chemistry models including aerosol processes along with their radiative effects are important tools for climate and air quality research. Validation of these models against in-situ and satellite measurements is essential to ascertain the credibility of these models and to guide model improvements. In this study the Goddard Chemistry, Aerosol, Radiation, and Transport (GOCART) module running on-line inside the Goddard Earth Observing System version 5 (GEOS-5) model is used to simulate aerosol and SO2 concentrations. Data taken in November 2010 over Frostburg, Maryland during an SO2 field campaign involving ground instrumentation and aircraft are used to evaluate GEOS-5 simulated SO2 concentrations. Preliminary data analysis indicated the model overestimated surface SO2 concentration, which motivated the examination of mixing processes in the model and the specification of SO2 anthropogenic emission rates. As a result of this analysis, a revision of anthropogenic emission inventories in GEOS-5 was implemented, and the vertical placement of SO2 sources was updated. Results show that these revisions improve the model agreement with observations locally and in regions outside the area of this field campaign. In particular, we use the ground-based measurements collected by the United States Environmental Protection Agency (US EPA) for the year 2010 to evaluate the revised model simulations over North America.

Air Quality Campaign

Magnetotactic bacteria on Earth and on Mars

Continued interest in the possibility of evidence for life in the ALH84001 Martian meteorite has focused on the magnetite crystals. This review is structured around three related questions: is the magnetite in ALH84001 of biological or non-biological origin, or a mixture of both? does magnetite on Earth provide insight to the plausibility of biogenic magnetite on Mars? could magnetotaxis have developed on Mars? There are credible arguments for both the biological and non-biological origin of the magnetite in ALH84001, and we suggest that more studies of ALH84001, extensive laboratory simulations of non-biological magnetite formation, as well as further studies of magnetotactic bacteria on Earth will be required to further address this question. Magnetite grains produced by bacteria could provide one of the few inorganic traces of past bacterial life on Mars that could be recovered from surface soils and sediments. If there was biogenic magnetite on Mars in sufficient abundance to leave fossil remains in the volcanic rocks of ALH84001, then it is likely that better-preserved magnetite will be found in sedimentary deposits on Mars. Deposits in ancient lakebeds could contain well-preserved chains of magnetite clearly indicating a biogenic origin.

Review, Tutorial

Simulator evaluation of displays for a revised takeoff performance monitoring system

Cockpit displays for a Takeoff Performance Monitoring System (TOPMS) to provide pilots with graphic and alphanumeric information pertinent to their decision to continue or abort a takeoff are evaluated. Revised head-down and newly developed head-up displays were implemented on electronic screens in the real-time Transport Systems Research Vehicle (TSRV) Simulator for the Boeing 737 airplane at the Langley Research Center and evaluated by 17 NASA, U.S. Air Force, airline, and industry pilots. Both types of displays were in color, but they were not dependent upon it. The TOPMS head-down display is composed of a runway graphic overlaid with symbolic status and advisory information related to both the expected takeoff point and the predicted stop point (in the event an abort becomes necessary). In addition, an overall Situation Advisory Flag indicates a preferred course of action based on analysis of the various elements of airplane performance and system status. A simpler head-up display conveys most of this same information and relates it to the visual scene. The evaluation pilots found the displays to be credible, easy to monitor, and appropriate for the task. In particular, the pilots said the head-up display was monitored with very little effort and did not obstruct or distract them from monitoring the simulated out-the-window runway scene. This report augments NASA TP-2908, 1989.

Middleton, David B.

Mission Scenario Development Workbench

The Mission Scenario Development Workbench (MSDW) is a multidisciplinary performance analysis software tool for planning and optimizing space missions. It provides a number of new capabilities that are particularly useful for planning the surface activities on other planets. MSDW enables rapid planning of a space mission and supports flight system and scientific-instrumentation trades. It also provides an estimate of the ability of flight, ground, and science systems to meet high-level mission goals and provides means of evaluating expected mission performance at an early stage of planning in the project life cycle. In MSDW, activity plans and equipment-list spreadsheets are integrated with validated parameterized simulation models of spacecraft systems. In contrast to traditional approaches involving worst-case estimates with large margins, the approach embodied in MSDW affords more flexibility and more credible results early in the lifecycle through the use of validated, variable- fidelity models of spacecraft systems. MSDW is expected to help maximize the scientific return on investment for space missions by understanding early the performance required to have a successful mission while reducing the risk of costly design changes made at late stages in the project life cycle.

Kordon, Mark

Resolving Low Cloud Feedbacks Globally With E3SM High‐Res MMF: Agreement With LES but Stronger Shortwave Effects

This study investigates low cloud feedback in a warmer climate using global simulations from the High-Resolution Multi-scale Modeling Framework (HR-MMF), which explicitly simulates small-scale eddies globally. Two 5-year simulations—one with present-day sea surface temperatures (SSTs) and a second with SSTs warmed uniformly by 4 K—reveal a positive global shortwave cloud radiative effect (SWCRE = 0.3 W/m 2 /K), comparable to estimates from CMIP models. As the climate warms, significant reductions in low cloud cover occur over stratocumulus regions. This study is the first attempt to compare HR-MMF results with predictions from idealized large-eddy simulations from the CGILS intercomparison. Despite different underlying assumptions, we find qualitative agreement in SWCRE and inversion height changes between HR-MMF and CGILS predictions. This suggests reasonable credibility for the CGILS framework in predicting cloud responses under the out-of-sample conditions found in HR-MMF. However, the HR-MMF exhibits stronger SWCRE changes than predicted by CGILS. We explore potential causes for this discrepancy, examining variations in cloud-controlling factors (CCFs) and cloud conditions. Our results show a fairly homogeneous SWCRE response, with little systematic variation tied to the variations in CCFs. This reveals a dominant role for SST forcing in modulating SWCRE.

boundary-layer clouds

Flight test of a synthetic aperture radar antenna using STEP

To establish confidence in its overall performance, credible information on the synthetic aperture radar antenna's mechanical properties in orbit must be obtained. However, the antenna's size, design, and operating environment make it difficult to simulate operating conditions under 1-g Earth conditions. The Space Technology Experiments Platform (STEP) offers a timely opportunity to mechanically qualify and characterize the antenna design in a representative environment. The proposed experimental configuration would employ a half-system of the full-scale RADARSAT antenna which would be mounted on the STEP platform in the orbiter cargo bay such that it could be deployed and retracted in orbit (as shown in this figure). The antenna would be subjected to typical environmental exposures while an array of targets and sensors on the antenna support structure and reflecting surface are observed and monitored. In particular, the typical environments would include deployment and retraction, dynamic response to vehicle thruster or base exciter inputs, and thermal soak and transient effects upon entering or exiting Earth eclipse. The proposed experiment would also provide generic information on the properties of large space structures in space and on techniques to obtain the desired information.

Zimcik, D. G.

Gateway Modeling and Simulation Plan

This plan institutes direction across the Gateway Program and the Element Projects to ensure that Cross Program M&S are produced in a manner that (1) generate the artifacts required for NASA-STD-7009 compliance, (2) ensures interoperability of M&S exchanged and integrated across the program and, (3) drives integrated development efforts to provide cross-domain integrated simulation of the Gateway elements, space environment, and operational scenarios. This direction is flowed down via contractual enforcement to prime contractors and includes both the GMS requirements specified in this plan and the NASASTD- 7009 derived requirements necessary for compliance. Grounding principles for management of Gateway Models and Simulations (M&S) are derived from the Columbia Accident Investigation Board (CAIB) report and the Diaz team report, “A Renewed Commitment to Excellence.” As an outcome of these reports, and in response to Action 4 of the Diaz team report, the NASA Standard for Models and Simulations, NASA-STD-7009 was developed. The standard establishes M&S requirements for development and use activities to ensure proper capture and communication of M&S pedigree and credibility information to Gateway program decision makers. Through the course of the Gateway program life cycle M&S will be heavily relied upon to conduct analysis, test products, support operations activities, enable informed decision making and ultimately to certify the Gateway with an acceptable level of risk to crew and mission. To reduce risk associated with M&S influenced decisions, this plan applies the NASA-STD-7009 requirements to produce the artifacts that support credibility assessments and ensure the information is communicated to program management.

NASA-STD-7009

DeepDiagnostics: A Software Package for Streamlined Posterior Evaluation

Automated prediction techniques like simulation-based inference (SBI) are important tasks for science experiments that produce large amounts of complex, raw data. However, their development remains in its early stages because the uncertainties of these techniques lack sufficient trustworthiness and interpretability. Packages for SBI provide a growing set of diagnostics; however, the software requirements are substantial, as they are tied to the inference technology itself, and the APIs lack adaptability. We introduce the DeepDiagnostics package for diagnosing posteriors from analytic likelihood-based methods and SBI methods, such as neural posterior estimation. DeepDiagnostics produces a comprehensive set of high-quality visualizations and metrics in a highly accessible, easy-to-use, and flexible package. We address all of these goals by providing a command-line inference tool and a Python API that is controlled through a configuration file. The package includes common diagnostics, such as parity plots, corner (covariance) plots, simulation-based calibration (SBC) diagnostics (including posterior coverage and rank histograms), Lemos et al. s PQMass and TARP, Masserano et al. s WALDO, Linhart et al. s LC2ST, as well as credible region diagnostics developed by our group.

Voetberg, Maggie [Fermilab]

An Overview of Recent Developments in Computational Aeroelasticity

The motivation for Computational Aeroelasticity (CA) and the elements of one type of the analysis or simulation process are briefly reviewed. The need for streamlining and improving the overall process to reduce elapsed time and improve overall accuracy is discussed. Further effort is needed to establish the credibility of the methodology, obtain experience, and to incorporate the experience base to simplify the method for future use. Experience with the application of a variety of Computational Aeroelasticity programs is summarized for the transonic flutter of two wings, the AGARD 445.6 wing and a typical business jet wing. There is a compelling need for a broad range of additional flutter test cases for further comparisons. Some existing data sets that may offer CA challenges are presented.

Bennett, Robert M.

Interdisciplinary technology

The 'computational test-cell' will enable the incorporation of new methodologies, such as concurrent engineering and probabilistic methods, into the propulsion design process. This will provide the capability to conduct credible, interdisciplinary analyses of new propulsion concepts and designs. Probabilistic methods can be used as the basis for reliability-based design. Recently methods have been devised that provide the capability of simulating the performance of propulsion systems at several levels of resolution. These methods make it possible to quantify uncertainty and to establish confidence bounds for the calculated values. The introduction of reliability-based design methodology along with probabilistic analyses will provide a tool to reduce the design space for new systems and to reduce our dependence on hardware testing for proof-of-concept and system integration demonstrations. The resulting simulations will reduce the need for testing and identify potential operational problems early in the design process. This capability will make it possible to compute the expected performance, stability, reliability, and life of propulsion components, subsystems, and systems at design and off-design conditions, to bring life cycle cost trade-offs early into the design process and to determine optimum designs to satisfy specified mission requirements.

Nichols, Lester D.

Conformal Hierarchical Simulation-Based Inference with Local Validity

Trustworthy and interpretable uncertainty quantification is a long-standing challenge in artificial intelligence. Simulation-based inference (SBI) comprises a broad swath of approaches for estimating latent parameters with uncertainties. Although flexible neural density estimators in SBI can be remark- ably expressive capturing highly structured, high-dimensional posteriors their credible regions can be badly mis-calibrated and are often only accompanied by heuristic coverage checks. We present the first SBI framework that delivers finite-sample local valid coverage guarantees that hold in the neighborhood of each observation. Our framework can couple any off-the-shelf hierarchical SBI engine with a confor- mal Bayesian post-processing step that operates on the posterior predictive density. A kernel-weighted conformity score adapts the conformal quantile to the local geometry of the data, yielding prediction sets that are simultaneously (i) marginally calibrated, (ii) locally valid, and (iii) hierarchical, handling global and observation-specific parameters in a single pass. Through experiments on synthetic data and benchmarks from neuroscience and physics, we show that our approach attains 1 − α coverage, where prior SBI methods under- or over-cover. Our approach also maintains a competitive, credible set size with minimal computational overhead. Finally, our approach can be used to make predictions on real data and give valid credible regions modulo weight-initialization-based model mis-specification.

Trivedi, Shubhendu [Fermilab]

Oblate-Earth Effects on the Calculation of Ec During Spacecraft Reentry

The bulge in the Earth at its equator has been shown to lead to a clustering of natural decays biased to occur towards the equator and away from the orbit's extreme latitudes. Such clustering must be considered when predicting the Expectation of Casualty (Ec) during a natural decay because of the clustering of the human population in the same lower latitudes. This study expands upon prior work, and formalizes the correction that must be made to the calculation of the average exposed population density as a result of this effect. Although a generic equation can be derived from this work to approximate the effects of gravitational and atmospheric perturbations on a final decay, such an equation averages certain important subtleties in achieving a best fit over all conditions. The authors recommend that direct simulation be used to calculate the true Ec for any specific entry as a more accurate method. A generic equation is provided, represented as a function of ballistic number and inclination of the entering spacecraft over the credible range of ballistic numbers.

Bacon, John B.

Taking Climate Model Evaluation to the Next Level

Earth system models are complex and represent a large number of processes, resulting in a persistent spread across climate projections for a given future scenario. Owing to different model performances against observations and the lack of independence among models, there is now evidence that giving equal weight to each available model projection is suboptimal. This Perspective discusses newly developed tools that facilitate a more rapid and comprehensive evaluation of model simulations with observations, process-based emergent constraints that are a promising way to focus evaluation on the observations most relevant to climate projections, and advanced methods for model weighting. These approaches are needed to distil the most credible information on regional climate changes, impacts, and risks for stakeholders and policy-makers.

Eyring, Veronika