Search NASA⌕ Search

SEARCH · Search NASA

Results for “integral probability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

27 records · Page 2

Figures of Merit for Control Verification

This paper proposes a methodology for evaluating a controller's ability to satisfy a set of closed-loop specifications when the plant has an arbitrary functional dependency on uncertain parameters. Control verification metrics applicable to deterministic and probabilistic uncertainty models are proposed. These metrics, which result from sizing the largest uncertainty set of a given class for which the specifications are satisfied, enable systematic assessment of competing control alternatives regardless of the methods used to derive them. A particularly attractive feature of the tools derived is that their efficiency and accuracy do not depend on the robustness of the controller. This is in sharp contrast to Monte Carlo based methods where the number of simulations required to accurately approximate the failure probability grows exponentially with its closeness to zero. This framework allows for the integration of complex, high-fidelity simulations of the integrated system and only requires standard optimization algorithms for its implementation.

Crespo, Luis G.↗

Convective Weather Forecast Quality Metrics for Air Traffic Management Decision-Making

Since numerical weather prediction models are unable to accurately forecast the severity and the location of the storm cells several hours into the future when compared with observation data, there has been a growing interest in probabilistic description of convective weather. The classical approach for generating uncertainty bounds consists of integrating the state equations and covariance propagation equations forward in time. This step is readily recognized as the process update step of the Kalman Filter algorithm. The second well known method, known as the Monte Carlo method, consists of generating output samples by driving the forecast algorithm with input samples selected from distributions. The statistical properties of the distributions of the output samples are then used for defining the uncertainty bounds of the output variables. This method is computationally expensive for a complex model compared to the covariance propagation method. The main advantage of the Monte Carlo method is that a complex non-linear model can be easily handled. Recently, a few different methods for probabilistic forecasting have appeared in the literature. A method for computing probability of convection in a region using forecast data is described in Ref. 5. Probability at a grid location is computed as the fraction of grid points, within a box of specified dimensions around the grid location, with forecast convection precipitation exceeding a specified threshold. The main limitation of this method is that the results are dependent on the chosen dimensions of the box. The examples presented Ref. 5 show that this process is equivalent to low-pass filtering of the forecast data with a finite support spatial filter. References 6 and 7 describe the technique for computing percentage coverage within a 92 x 92 square-kilometer box and assigning the value to the center 4 x 4 square-kilometer box. This technique is same as that described in Ref. 5. Characterizing the forecast, following the process described in Refs. 5 through 7, in terms of percentage coverage or confidence level is notionally sound compared to characterizing in terms of probabilities because the probability of the forecast being correct can only be determined using actual observations. References 5 through 7 only use the forecast data and not the observations. The method for computing the probability of detection, false alarm ratio and several forecast quality metrics (Skill Scores) using both the forecast and observation data are given in Ref. 2. This paper extends the statistical verification method in Ref. 2 to determine co-occurrence probabilities. The method consists of computing the probability that a severe weather cell (grid location) is detected in the observation data in the neighborhood of the severe weather cell in the forecast data. Probabilities of occurrence at the grid location and in its neighborhood with higher severity, and with lower severity in the observation data compared to that in the forecast data are examined. The method proposed in Refs. 5 through 7 is used for computing the probability that a certain number of cells in the neighborhood of severe weather cells in the forecast data are seen as severe weather cells in the observation data. Finally, the probability of existence of gaps in the observation data in the neighborhood of severe weather cells in forecast data is computed. Gaps are defined as openings between severe weather cells through which an aircraft can safely fly to its intended destination. The rest of the paper is organized as follows. Section II summarizes the statistical verification method described in Ref. 2. The extension of this method for computing the co-occurrence probabilities in discussed in Section HI. Numerical examples using NCWF forecast data and NCWD observation data are presented in Section III to elucidate the characteristics of the co-occurrence probabilities. This section also discusses the procedure for computing throbabilities that the severity of convection in the observation data will be higher or lower in the neighborhood of grid locations compared to that indicated at the grid locations in the forecast data. The probability of coverage of neighborhood grid cells is also described via examples in this section. Section IV discusses the gap detection algorithm and presents a numerical example to illustrate the method. The locations of the detected gaps in the observation data are used along with the locations of convective weather cells in the forecast data to determine the probability of existence of gaps in the neighborhood of these cells. Finally, the paper is concluded in Section V.

Chatterji, Gano B.↗

Automated Storm Tracking and the Lightning Jump Algorithm Using GOES-R Geostationary Lightning Mapper (GLM) Proxy Data

This study develops a fully automated lightning jump system encompassing objective storm tracking, Geostationary Lightning Mapper proxy data, and the lightning jump algorithm (LJA), which are important elements in the transition of the LJA concept from a research to an operational based algorithm. Storm cluster tracking is based on a product created from the combination of a radar parameter (vertically integrated liquid, VIL), and lightning information (flash rate density). Evaluations showed that the spatial scale of tracked features or storm clusters had a large impact on the lightning jump system performance, where increasing spatial scale size resulted in decreased dynamic range of the system's performance. This framework will also serve as a means to refine the LJA itself to enhance its operational applicability. Parameters within the system are isolated and the system's performance is evaluated with adjustments to parameter sensitivity. The system's performance is evaluated using the probability of detection (POD) and false alarm ratio (FAR) statistics. Of the algorithm parameters tested, sigma-level (metric of lightning jump strength) and flash rate threshold influenced the system's performance the most. Finally, verification methodologies are investigated. It is discovered that minor changes in verification methodology can dramatically impact the evaluation of the lightning jump system.

lightning jump↗

Soil Moisture Active and Passive (SMAP) White-Painted Expanded Polystyrene (EPS) Radome Survivability Test

NASA's SMAP Mission launched in January 2015 into a 685 km near-polar, sun-synchronous orbit. The SMAP instrument architecture incorporates an L-band radar and radiometer which share a common feedhorn and mesh reflector. The instrument rotates about the nadir axis at approximately 15 rpm, thereby providing a conically scanning wide swath antenna beam that is capable of achieving global coverage within three days. The radiometer and its associated electronics have tight thermal stability requirements in order to meet the required surface emittance measurement precision from space. Maintaining the thermal stabilities is quite challenging because the radiometer is located on a spinning platform that can either be in full sunlight or eclipse, and thus exposed to a highly transient environment. Stability requirements were met by integrating a light-weight Expanded Polystyrene (EPS) radome into the design to prevent solar illumination of the feed horn interior. The radome was painted white since the thermo-optical properties of bare sunlit EPS degrade rapidly over the three-year mission. Milling of the EPS and solvent within the white paint created cavities on the EPS surface which may introduce localized hot spots possibly violating the EPS glass transition temperature of 96degC and leading to structural integrity concerns. A three-day thermal test was conducted in a vacuum chamber to verify survivability of the radome during a simulated non-spin fault condition at end of mission. A portable solar simulator illuminated the test article and the beam irradiance was kept nearly constant during the entire 50 hour test, except during the first hour which simulated the expected 79degC on-orbit surface temperature of the radome. The test article survived based on the established pass criteria for three separate metrics: dimensional, optical property, and color. If any hot spots exist locally, they did not cause any observable permanent deformation when compared to pre- and post-test images. The test results increase confidence that there is a high probability that the radome will survive the worst-case scenario of a no-spin fault condition at the end of mission.

Mikhaylov, Rebecca↗

Demonstration of a Probabilistic Technique for the Determination of Economic Viability of Very Large Transport Configurations

Over the past few years, modem aircraft design has experienced a paradigm shift from designing for performance to designing for affordability. This report contains a probabilistic approach that will allow traditional deterministic design methods to be extended to account for disciplinary, economic, and technological uncertainty. The probabilistic approach was facilitated by the Fast Probability Integration (FPI) technique; a technique which allows the designer to gather valuable information about the vehicle's behavior in the design space. This technique is efficient for assessing multi-attribute, multi-constraint problems in a more realistic fashion. For implementation purposes, this technique is applied to illustrate how both economic and technological uncertainty associated with a Very Large Transport aircraft concept may be assessed. The assessment is evaluated with the FPI technique to determine the cumulative probability distributions of the design space, as bound by economic objectives and performance constraints. These distributions were compared to established targets for a comparable large capacity aircraft, similar in size to the Boeing 747-400. The conventional baseline configuration design space was determined to be unfeasible and marginally viable, motivating the infusion of advanced technologies, including reductions in drag, specific fuel consumption, wing weight, and Research, Development, Testing, and Evaluation costs. The resulting system design space was qualitatively assessed with technology metric "k" factors. The infusion of technologies shifted the VLT design into regions of feasibility and greater viability. The study also demonstrated a method and relationship by which the impact of new technologies may be assessed in a more system focused approach.

Mavris, Dimitri N.↗

Quantitative Validation of the Integrated Medical Model (IMM) for ISS Missions

Lifetime Surveillance of Astronaut Health (LSAH) provided observed medical event data on 33 ISS and 111 STS person-missions for use in further improving and validating the Integrated Medical Model (IMM). Using only the crew characteristics from these observed missions, the newest development version, IMM v4.0, will simulate these missions to predict medical events and outcomes. Comparing IMM predictions to the actual observed medical event counts will provide external validation and identify areas of possible improvement. In an effort to improve the power of detecting differences in this validation study, the total over each program ISS and STS will serve as the main quantitative comparison objective, specifically the following parameters: total medical events (TME), probability of loss of crew life (LOCL), and probability of evacuation (EVAC). Scatter plots of observed versus median predicted TMEs (with error bars reflecting the simulation intervals) will graphically display comparisons while linear regression will serve as the statistical test of agreement. Two scatter plots will be analyzed 1) where each point reflects a mission and 2) where each point reflects a condition-specific total number of occurrences. The coefficient of determination (R2) resulting from a linear regression with no intercept bias (intercept fixed at zero) will serve as an overall metric of agreement between IMM and the real world system (RWS). In an effort to identify as many possible discrepancies as possible for further inspection, the -level for all statistical tests comparing IMM predictions to observed data will be set to 0.1. This less stringent criterion, along with the multiple testing being conducted, should detect all perceived differences including many false positive signals resulting from random variation. The results of these analyses will reveal areas of the model requiring adjustment to improve overall IMM output, which will thereby provide better decision support for mission critical applications.

validation↗

The Integrated Medical Model - Optimizing In-flight Space Medical Systems to Reduce Crew Health Risk and Mission Impacts

The Integrated Medical Model (IMM) is a decision support tool used by medical system planners and designers as they prepare for exploration planning activities of the Constellation program (CxP). IMM provides an evidence-based approach to help optimize the allocation of in-flight medical resources for a specified level of risk within spacecraft operational constraints. Eighty medical conditions and associated resources are represented in IMM. Nine conditions are due to Space Adaptation Syndrome. The IMM helps answer fundamental medical mission planning questions such as What medical conditions can be expected? What type and quantity of medical resources are most likely to be used?", and "What is the probability of crew death or evacuation due to medical events?" For a specified mission and crew profile, the IMM effectively characterizes the sequence of events that could potentially occur should a medical condition happen. The mathematical relationships among mission and crew attributes, medical conditions and incidence data, in-flight medical resources, potential clinical and crew health end states are established to generate end state probabilities. A Monte Carlo computational method is used to determine the probable outcomes and requires up to 25,000 mission trials to reach convergence. For each mission trial, the pharmaceuticals and supplies required to diagnose and treat prevalent medical conditions are tracked and decremented. The uncertainty of patient response to treatment is bounded via a best-case, worst-case, untreated case algorithm. A Crew Health Index (CHI) metric, developed to account for functional impairment due to a medical condition, provides a quantified measure of risk and enables risk comparisons across mission scenarios. The use of historical in-flight medical data, terrestrial surrogate data as appropriate, and space medicine subject matter expertise has enabled the development of a probabilistic, stochastic decision support tool capable of optimizing in-flight medical systems based on crew and mission parameters. This presentation will illustrate how to apply quantitative risk assessment methods to optimize the mass and volume of space-based medical systems for a space flight mission given the level of crew health and mission risk.

Kerstman, Eric↗

Long-Term Health Metric Development Effort

INTRODUCTION Probabilistic Risk Assessment (PRA) is a methodology applied when high-stakes decisions need to be made about complex systems. PRA often uses risk minimization to aid decision making, such as when establishing vehicle requirements or resource allocations. The NASA Human System Risk Board (HSRB) maintains the human spaceflight risk postures, many of which have in-mission medical outcomes, but the full risk set also includes performance and Long-Term Health (LTH) outcomes. An effort is underway to determine the dependencies between LTH risk outcomes and mission characteristics and to identify metrics for quantifying LTH outcomes. The LTH metrics will be incorporated into future PRA tools currently under development to support crew health and performance decision making. LONG-TERM HEALTH (LTH) RISK LTH risk encompasses the timeframe from immediately post-flight, through the rest of an astronaut’s career, through retirement, and until death. LTH outcomes include the time and interventions needed for the astronaut to return to preflight physiological states after experiencing spaceflight hazards and recovery from any in-mission medical events that persist into the post-flight timeframe. It includes chronic complications that may arise due to experiencing in-flight medical conditions or injuries and medical conditions that occur later in life with a higher probability of occurrence or with more severity because of their spaceflight exposure. Finally, LTH outcomes can also include a reduction in life expectancy due to spaceflight exposures. Eighteen of the HSRB risks contain an LTH component, each with a unique incidence rate and consequence severity. ON-GOING LTH METRIC DEVELOPMENT EFFORTS Based on guidance from a stakeholder workshop held on June 1, 2023, only LTH outcomes with causes tied to spaceflight experience that are significantly different from normal healthy aging and which can be affected by changes in in-mission resources are targeted for inclusion within the PRA modeling efforts. Current on-going efforts include a sensitivity analysis of potential in-flight medical conditions to determine which conditions may persist into post-flight and how the consequences may be affected by in-mission medical resources. Also underway is determination of differences in the time for a return to preflight physiological baselines when in-flight countermeasures are available versus when they are not. Analyses performed with publicly available astronaut data and collaborations with the Lifetime Surveillance of Astronaut Health team are being designed to understand which LTH outcomes differ significantly from natural healthy aging. Information exchanges are also occurring with LTH risk custodians to understand how they quantify outcomes. FUTURE PLANS Risk characteristics will be determined for each LTH risk that meets our inclusion criteria and the dependencies of their outcomes to in-mission resources and mission characteristics will be established. The individual risks will be integrated together into one or more high level LTH metrics for inclusion in the PRA tool, so that LTH risks can be considered along with in-mission risks during crew health and performance system decision support trade studies.

B. E. Lewandowski↗

Probabilistic Modeling of a Three-Stage Human Landing System Architecture

Space Policy Directive-1 has led to NASA partnerships with commercial entities on procurement which includes the development of the Human Landing System (HLS) [1]. With the goal of delivering human crew to the lunar surface by 2024, system uncertainties become an important obstacle to the maturation of multiple new, driving technologies and mission concepts of the HLS program. As unmitigated uncertainties have previously led to failed development programs, these risks and their impacts must be understood and handled to ensure program success [2]. Sources of uncertainty include novel engine designs and configurations, increased reliance on cryogenic fluid management(CFM), and refueling technologies—which propagate as high-level performance metrics such as overall propellant mass and engine performance. Also, the occurrence of operational uncertainties—e.g. launch conditions or need to abort during the mission—can cause cascading effects on the rest of the mission that are difficult to definitively quantify, and are outside the scope of control. These concrete examples and other occurrences can be categorized as either epistemic or aleatory uncertainties.Epistemic uncertainty arises due to a lack of knowledge and can be alleviated with design and program maturation. Aleatory uncertainty is due to the inherent randomness of the system and cannot be directly reduced, unlike epistemic uncertainty. Robust design and probabilistic methods can compensate for aleatory effects. A taxonomy of uncertainty is referred to for this work [3]. In this paper, a probabilistic methodology to handle uncertainties has been demonstrated on a three-element HLS concept [1, 4], which allows tracking of current best estimates of the concept and assessment of concept design robustness against uncertainties. A sample case has been completed for this abstract, and an expansion on the methodology will be included in the final paper. This methodology has two key parts: first, the creation of a dynamic architecture model of a three-element HLS concept; and second, its use with surrogate modeling and range estimating techniques to capture and propagate uncertainties. This abstract will cover the basics of the approach used, and further details and justifications will be in the final paper.The mission profile associated with this three-element concept (Fig 1) was modeled as a set of mission events that facilitated mass changes, idles, or spacecraft maneuvers. The mission profile scope starts with each element’s NRHO orbit insertion and aggregation and ends at post-sortie rendezvous with Orion. More detail on the mission profile will be in the final paper. The DYnamic Rocket EQuation Tool (DYREQT), a space systems synthesis and sizing framework used by NASA, was used as the physics framework to model the HLS architecture for applying the probabilistic methodology [5, 6]. Specifically, a parametric representation of the lander, ascent, and transfer elements and the mission profile of each element was established, with vehicle and mission parameters available as inputs to allow for a dynamic model. Each vehicle stage was modeled with high-level performance metrics, using Isp and propellant mass fraction (PMF) to remain parametric. For the probabilistic analysis, uncertainties of interest within the HLS concept were enumerated and represented as parameters within the DYREQT model as inputs for vehicle stages or mission profile events. These parameters were frozen at their nominal values for the purposes of baselining architecture performance and sizing the vehicle appropriately based on reference documentation [1]. Range estimating—a probabilistic method that combines Monte Carlo sampling, focus on critical parameters, and heuristics to assess risk and opportunities—is traditionally used with Mass Equipment Lists (MELs), but has been adapted with operational parameters as well as vehicle parameters in theDYREQT model to capture mission uncertainty alongside vehicle uncertainty [7, 3]. This method was selected due to its application and insight on a system from a bottom-up perspective, independence from historical rules of thumb, and ability to generate sensitivities based on design decisions and uncertainties. As a sample case for the abstract, the boiloff rates of the vehicle elements and the loiter times during the mission (simulating launch time variations and changing window of opportunities) were used with range estimating to provide preliminary results. To perform the range estimation portion of this methodology (depicted in Fig. 3, further details in final paper), the DYREQT model was sampled using a Design of Experiments (DoE) to efficiently explore the architecture design space with respect to the sample set of uncertainty parameters; 5,000 cases via Latin Hypercube Sampling were computed on the DYREQT architecture model. Then, the results were used to create surrogate models, multivariate regressions that can visualize hypercube trends in the design space, of the architecture with respect to the uncertainty parameters. Range estimating was applied to the surrogates instead of the actual models, which saves computational expense due to the bulk of cases needed for the Monte Carlo simulation as part of range estimating. Uncertainty parameters were sampled independently from triangular distributions using the DoE ranges as ‘min’ and ‘max’, and the nominal value as ‘most likely’. Based engineering intuition, some uncertainty parameters are correlated—e.g. if the main propellant has a high boil-off rate, the oxidizer should follow suit as both are related to CFM technology.While a Monte Carlo simulation samples all inputs as independent, the results would show model correlations; thus, it is efficient to sample the inputs as correlated. Using a correlation matrix constructed for the uncertainty parameters, previously independent samples were transformed to perform a Correlated Monte Carlo. A table for the DoE ranges and probability distribution parameters is shown in Table 1, and more details on Correlated Monte Carlo Simulations will be discussed in the final paper. The model’s resulting DoE showed that multivariate polynomial equations fit via least squares method captured its behavior accurately for the sample case. For the Correlated Monte Carlo Simulation, a positive correlation between fuel and oxidizer boiloff rates was used as a demonstration. 10,000 cases were computed with the surrogates and the launched masses for each vehicle element was collated. The results can be displayed in a probability density function (PDF), showing the impact of the uncertainty parameters chosen. Integrating the PDFs will yield a cumulative distribution function (CDF) that shows the cumulative probability of a given value on the x-axis. For the sample case, the elements’ launch mass margin was calculated and represented in as CDFs, as a demonstrated representation of figures of merit for the HLS concept. For the lander and ascent elements, the NRHO mass insertion limit is 16t; the transfer element has a limit of 30t [1]. It can be seen with Figure 2 that this probabilistic methodology can provide insight into mass margin with respect to the uncertainties being modeled. Currently, the results show that the lander (descent) vehicle element has the most restrictive design space; it is the only element to show a 10% probability of negative margin. Further analysis on the Monte Carlo results will show sensitivities for driving constraints and parameters for architecture feasibility, which can lead to establishing potential mission rules.The combination of range estimating with a parametric architecture model for HLS demonstrated the capability of this probabilistic methodology in a sample case. As the HLS development progresses, this methodology has the potential for keeping current best estimates of architecture performance for awarded concepts due to the flexibility in DYREQT’s modeling framework and its parametric nature. Concept maturation and increased epistemic knowledge can be injected into the model probabilistic modeling, and thus continue to track probability of mission success.

Stephanie Y Zhu↗