Search NASA⌕ Search

SEARCH · Search NASA

Results for “model credibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 1: Workshop objectives and summary

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 2: Comparisons with global atmospheric measurements

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

The atmospheric effects of stratospheric aircraft. Report of the 1992 Models and Measurements Workshop. Volume 3: Special diagnostic studies

This Workshop on Stratospheric Models and Measurements (M&M) marks a significant expansion in the history of model intercomparisons. It provides a foundation for establishing the credibility of stratospheric models used in environmental assessments of chlorofluorocarbons, aircraft emissions, and climate-chemistry interactions. The core of the M&M comparisons involves the selection of observations of the current stratosphere (i.e., within the last 15 years): these data are believed to be accurate and representative of certain aspects of stratospheric chemistry and dynamics that the models should be able to simulate.

Prather, Michael J.↗

Simplified ISCCP Cloud Regimes for Evaluating Cloudiness in CMIP5 Models

We take advantage of the ISCCP simulator implemented in many CMIP5 models, in order to introduce a credible method to compare model cloud output with corresponding ISCCP observations based on the cloud regime (CR) concept. A novel simplified CR derivation method is introduced that exploits the co-variations of three variables, cloud optical thickness, cloud top pressure and cloud fraction (, pc, CF). Following evaluation criteria established in a previous paper of ours, we assess model cloud simulation performance based on how well the simplified CRs are simulated in terms of similarity of centroids, global values and map correlations of relative frequency-of-occurrence (RFO), and long term total clouds amounts (as opposed to average instantaneous cloud fractions). Mirroring prior results, modeled clouds tend to be too thick optically and not as extensive as in observations. Averaging across models does not help in this or other evaluation aspects, except for cloud geographical locations. A good model for one criterion is usually found to be also a good performer for other criteria despite plenty of exceptions for the criterion of centroid similarity. On the other hand, CRs that include convectively generated high-altitude clouds are not as well simulated here compared to the previous study, but other regimes containing low clouds with high CF improve. Models that have previously performed well distinguish themselves again here, but improvements from other models are also found. The simplified cloud evaluation method thus proves to be more forgiving while still good enough to reveal model weaknesses.

global climate models↗

Evaluation of GEOS-5 Sulfur Dioxide Simulations During the Frostburg, MD 2010 Field Campaign.

Sulfur dioxide (SO2) is a major atmospheric pollutant with a strong anthropogenic component mostly produced by the combustion of fossil fuel and other industrial activities. As a precursor of sulfate aerosols that affect climate, air quality, and human health, this gas needs to be monitored on a global scale. Global climate and chemistry models including aerosol processes along with their radiative effects are important tools for climate and air quality research. Validation of these models against in-situ and satellite measurements is essential to ascertain the credibility of these models and to guide model improvements. In this study the Goddard Chemistry, Aerosol, Radiation, and Transport (GOCART) module running on-line inside the Goddard Earth Observing System version 5 (GEOS-5) model is used to simulate aerosol and SO2 concentrations. Data taken in November 2010 over Frostburg, Maryland during an SO2 field campaign involving ground instrumentation and aircraft are used to evaluate GEOS-5 simulated SO2 concentrations. Preliminary data analysis indicated the model overestimated surface SO2 concentration, which motivated the examination of mixing processes in the model and the specification of SO2 anthropogenic emission rates. As a result of this analysis, a revision of anthropogenic emission inventories in GEOS-5 was implemented, and the vertical placement of SO2 sources was updated. Results show that these revisions improve the model agreement with observations locally and in regions outside the area of this field campaign. In particular, we use the ground-based measurements collected by the United States Environmental Protection Agency (US EPA) for the year 2010 to evaluate the revised model simulations over North America.

Air Quality Campaign↗

Building a Database for a Quantitative Model

A database can greatly benefit a quantitative analysis. The defining characteristic of a quantitative risk, or reliability, model is the use of failure estimate data. Models can easily contain a thousand Basic Events, relying on hundreds of individual data sources. Obviously, entering so much data by hand will eventually lead to errors. Not so obviously entering data this way does not aid linking the Basic Events to the data sources. The best way to organize large amounts of data on a computer is with a database. But a model does not require a large, enterprise-level database with dedicated developers and administrators. A database built in Excel can be quite sufficient. A simple spreadsheet database can link every Basic Event to the individual data source selected for them. This database can also contain the manipulations appropriate for how the data is used in the model. These manipulations include stressing factors based on use and maintenance cycles, dormancy, unique failure modes, the modeling of multiple items as a single "Super component" Basic Event, and Bayesian Updating based on flight and testing experience. A simple, unique metadata field in both the model and database provides a link from any Basic Event in the model to its data source and all relevant calculations. The credibility for the entire model often rests on the credibility and traceability of the data.

Kahn, C. Joseph↗

HRP's Healthcare Spin-Offs Through Computational Modeling and Simulation Practice Methodologies

Spaceflight missions expose astronauts to novel operational and environmental conditions that pose health risks that are currently not well understood, and perhaps unanticipated. Furthermore, given the limited number of humans that have flown in long duration missions and beyond low Earth-orbit, the amount of research and clinical data necessary to predict and mitigate these health and performance risks are limited. Consequently, NASA's Human Research Program (HRP) conducts research and develops advanced methods and tools to predict, assess, and mitigate potential hazards to the health of astronauts. In this light, NASA has explored the possibility of leveraging computational modeling since the 1970s as a means to elucidate the physiologic risks of spaceflight and develop countermeasures. Since that time, substantial progress has been realized in this arena through a number of HRP funded activates such as the Digital Astronaut Project (DAP) and the Integrated Medical Model (IMM). Much of this success can be attributed to HRP's endeavor to establish rigorous verification, validation, and credibility (VV&C) processes that ensure computational models and simulations (M&S) are sufficiently credible to address issues within their intended scope. This presentation summarizes HRP's activities in credibility of modeling and simulation, in particular through its outreach to the community of modeling and simulation practitioners. METHODS: The HRP requires all M&S that can have moderate to high impact on crew health or mission success must be vetted in accordance to NASA Standard for Models and Simulations, NASA-STD-7009 (7009) [5]. As this standard mostly focuses on engineering systems, the IMM and DAP have invested substantial efforts to adapt the processes established in this standard for their application to biological M&S, which is more prevalent in human health and performance (HHP) and space biomedical research and operations [6,7]. These methods have also generated substantial interest by the broader medical community though institutions like the National Institutes of Health (NIH) and the Food and Drug Administration (FDA) to develop similar standards and guidelines applicable to the larger medical operations and research community. DISCUSSION: Similar to NASA, many leading government agencies, health institutions and medical product developers around the world are recognizing the potential of computational M&S to support clinical research and decision making. In this light, substantial investments are being made in computational medicine and notable discoveries are being realized [8]. However, there is a lack of broadly applicable practice guidance for the development and implementation of M&S in clinical care and research in a manner that instills confidence among medical practitioners and biological researchers [9,10]. In this presentation, we will give an overview on how HRP is working with the NIH's Interagency Modeling and Analysis Group (IMAG), the FDA and the American Society of Mechanical Engineers (ASME) to leverage NASA's biomedical VV&C processes to establish a new regulatory standard for Verification and Validation in Computational Modeling of Medical Devices, and Guidelines for Credible Practice of Computational Modeling and Simulation in Healthcare.

Mulugeta, Lealem↗

An Overview of NASA's IM&S Verification and Validation Process Plan and Specification for Space Exploration

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model's fidelity, credibility, and quality. This information will allow the decision-maker to understand the risks involved in using a model's results in the decision-making process. This presentation will discuss NASA's approach for verification and validation (V&V) of its models or simulations supporting space exploration. This presentation will describe NASA's V&V process and the associated M&S verification and validation (V&V) activities required to support the decision-making process. The M&S V&V Plan and V&V Report templates for ESMD will also be illustrated.

Gravitz, Robert M.↗

NASA's M and S Accreditation Process Plan and Specification for Space Exploration

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model's fidelity, credibility, and quality. This information will allow the decision-maker to understand the risks involved in using a model's results in the decision-making process. This presentation will discuss NASA's overall approach to achieving formal accreditation of its models or simulations supporting space exploration. The development of a formal Accreditation Plan is a key component in the preliminary activities for modeling and simulation (M&S) assessment. This presentation will describe NASA's process for identifying risks associated with M&S use and the associated M&S assessments that will dictate the level of data certification and M&S verification and validation (V&V) activities required to support the decision-making process. The M&S Accreditation Plan and Report templates for ESMD will also be illustrated.

O'Neil, David↗

NASA Handbook for Models and Simulations: An Implementation Guide for NASA-STD-7009

The purpose of this Handbook is to provide technical information, clarification, examples, processes, and techniques to help institute good modeling and simulation practices in the National Aeronautics and Space Administration (NASA). As a companion guide to NASA-STD- 7009, Standard for Models and Simulations, this Handbook provides a broader scope of information than may be included in a Standard and promotes good practices in the production, use, and consumption of NASA modeling and simulation products. NASA-STD-7009 specifies what a modeling and simulation activity shall or should do (in the requirements) but does not prescribe how the requirements are to be met, which varies with the specific engineering discipline, or who is responsible for complying with the requirements, which depends on the size and type of project. A guidance document, which is not constrained by the requirements of a Standard, is better suited to address these additional aspects and provide necessary clarification. This Handbook stems from the Space Shuttle Columbia Accident Investigation (2003), which called for Agency-wide improvements in the "development, documentation, and operation of models and simulations"' that subsequently elicited additional guidance from the NASA Office of the Chief Engineer to include "a standard method to assess the credibility of the models and simulations."2 General methods applicable across the broad spectrum of model and simulation (M&S) disciplines were sought to help guide the modeling and simulation processes within NASA and to provide for consistent reporting ofM&S activities and analysis results. From this, the standardized process for the M&S activity was developed. The major contents of this Handbook are the implementation details of the general M&S requirements ofNASA-STD-7009, including explanations, examples, and suggestions for improving the credibility assessment of an M&S-based analysis.

Steele, Martin J.↗

On the modelling of scalar and mass transport in combustor flows

Results are presented of a numerical study of swirling and nonswirling combustor flows with and without density variations. Constant-density arguments are used to justify closure assumptions invoked for the transport equations for turbulent momentum and scalar fluxes, which are written in terms of density-weighted variables. Comparisons are carried out with measurements obtained from three different axisymmetric model combustor experiments covering recirculating flow, swirling flow, and variable-density swirling flow inside the model combustors. Results show that the Reynolds stress/flux models do a credible job of predicting constant-density swirling and nonswirling combustor flows with passive scalar transport. However, their improvements over algebraic stress/flux models are marginal. The extension of the constant-density models to variable-density flow calculations shows that the models are equally valid for such flows.

Nikjooy, M.↗

An analysis of possible applications of fuzzy set theory to the actuarial credibility theory

In this work, we review the basic concepts of actuarial credibility theory from the point of view of introducing applications of the fuzzy set-theoretic method. We show how the concept of actuarial credibility can be modeled through the fuzzy set membership functions and how fuzzy set methods, especially fuzzy pattern recognition, can provide an alternative tool for estimating credibility.

Ostaszewski, Krzysztof↗

Modeling Personnel Turnover in the Parametric Organization

A primary issue in organizing a new parametric cost analysis function is to determine the skill mix and number of personnel required. The skill mix can be obtained by a functional decomposition of the tasks required within the organization and a matrixed correlation with educational or experience backgrounds. The number of personnel is a function of the skills required to cover all tasks, personnel skill background and cross training, the intensity of the workload for each task, migration through various tasks by personnel along a career path, personnel hiring limitations imposed by management and the applicant marketplace, personnel training limitations imposed by management and personnel capability, and the rate at which personnel leave the organization for whatever reason. Faced with the task of relating all of these organizational facets in order to grow a parametric cost analysis (PCA) organization from scratch, it was decided that a dynamic model was required in order to account for the obvious dynamics of the forming organization. The challenge was to create such a simple model which would be credible during all phases of organizational development. The model development process was broken down into the activities of determining the tasks required for PCA, determining the skills required for each PCA task, determining the skills available in the applicant marketplace, determining the structure of the dynamic model, implementing the dynamic model, and testing the dynamic model.

Dean, Edwin B.↗

Thermal spectrum of Uranus - Implications for large helium abundance

An analysis of the infrared spectrum of Uranus' disk between 7 micrometers and 3 millimeters suggests a volume mixing ratio for helium in the atmosphere of 40 + or - 20 percent, more than for the sun, Jupiter, or Saturn. Alternative explanations require even more extreme assumptions regarding gas abundances or aerosol vertical distribution and spectral properties. The most serious difficulty with a model containing large amounts of helium is devising a credible evolutionary or chemical model explaining the absence or segregation of so much hydrogen.

Orton, G. S.↗

Reconstructing PM 2.5 Data Record for the Kathmandu Valley Using a Machine Learning Model

This paper presents a method for reconstructing the historical hourly concentrations of particulate matter 2.5 (PM2.5) over the Kathmandu Valley from 1980 to the present. The method uses a machine learning model that is trained using PM2.5 readings from US Embassy (Phora Durbar) as a ground truth, and the meteorological data from Modern-Era Retrospective Analysis for Research and Applications v2 (MERRA2) as input. The Extreme Gradient Boosting (XGBoost) model acquires a credible 10-fold cross-validation (CV) score of ~83.4%, an r2-score of ~84%, a Root Mean Square Error (RMSE) of ~15.82 µg/m3, and a Mean Absolute Error (MAE) of ~10.27 µg/m3. Further demonstrating the model's applicability to years other than those for which truth values are unavailable, the multiple cross-test with an unseen data set offered r2-scores for 2018, 2019, and 2020 ranging from 56% to 67%. The model-predicted data agrees with true values and indicates that MERRA2 underestimates PM2.5 over the region. It strongly agrees with ground-based evidence showing substantially higher mass concentrations in the dry pre- and post-monsoon seasons than in the monsoon months. It also shows a strong anti-correlation between PM2.5 concentration and humidity. The results also demonstrate that none of the years fulfilled the annual mean air quality index (AQI) standards set by the World Health Organization (WHO).

machine learning↗

Evaluating the Credibility of Transport Processes in the Global Modeling Initiative 3D Model Simulations of Ozone Recovery

The Global Modeling Initiative has integrated two 35-year simulations of an ozone recovery scenario with an offline chemistry and transport model using two different meteorological inputs. Physically based diagnostics, derived from satellite and aircraft data sets, are described and then used to evaluate the realism of temperature and transport processes in the simulations. Processes evaluated include barrier formation in the subtropics and polar regions, and extratropical wave-driven transport. Some diagnostics are especially relevant to simulation of lower stratospheric ozone, but most are applicable to any stratospheric simulation. The temperature evaluation, which is relevant to gas phase chemical reactions, showed that both sets of meteorological fields have near climatological values at all latitudes and seasons at 30 hPa and below. Both simulations showed weakness in upper stratospheric wave driving. The simulation using input from a general circulation model (GMI(sub GCM)) showed a very good residual circulation in the tropics and northern hemisphere. The simulation with input from a data assimilation system (GMI(sub DAS)) performed better in the midlatitudes than at high latitudes. Neither simulation forms a realistic barrier at the vortex edge, leading to uncertainty in the fate of ozone-depleted vortex air. Overall, tracer transport in the offline GMI(sub GCM) has greater fidelity throughout the stratosphere than the GMI(sub DAS).

Strahan, Susan E.↗

Model Experiments and Model Descriptions

The Second Workshop on Stratospheric Models and Measurements Workshop (M&M II) is the continuation of the effort previously started in the first Workshop (M&M I, Prather and Remsberg [1993]) held in 1992. As originally stated, the aim of M&M is to provide a foundation for establishing the credibility of stratospheric models used in environmental assessments of the ozone response to chlorofluorocarbons, aircraft emissions, and other climate-chemistry interactions. To accomplish this, a set of measurements of the present day atmosphere was selected. The intent was that successful simulations of the set of measurements should become the prerequisite for the acceptance of these models as having a reliable prediction for future ozone behavior. This section is divided into two: model experiment and model descriptions. In the model experiment, participant were given the charge to design a number of experiments that would use observations to test whether models are using the correct mechanisms to simulate the distributions of ozone and other trace gases in the atmosphere. The purpose is closely tied to the needs to reduce the uncertainties in the model predicted responses of stratospheric ozone to perturbations. The specifications for the experiments were sent out to the modeling community in June 1997. Twenty eight modeling groups responded to the requests for input. The first part of this section discusses the different modeling group, along with the experiments performed. Part two of this section, gives brief descriptions of each model as provided by the individual modeling groups.

Jackman, Charles H.↗