SEARCH · Search NASA

Results for “metric categorization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

NASA NTRS2020

Potential of GPM IMERG Precipitation Estimates to Monitor Natural Disaster Triggers in Urban Areas: the Case of Rio deJaneiro, Brazil

Extreme rainfall can be a catastrophic trigger for natural disaster events at urban scales. However, there remains large uncertainties as to how satellite precipitation can identify these triggers at a city scale. The objective of this study is to evaluate the potential of satellite-based rainfall estimates to monitor natural disaster triggers in urban areas. Rainfall estimates from the Global Precipitation Measurement (GPM) mission are evaluated over the city of Rio de Janeiro, Brazil, where urban floods and landslides occur periodically as a result of extreme rainfall events. Two rainfall products derived from the Integrated Multi-satellite Retrievals for GPM (IMERG), the IMERG Early and IMERG Final products, are integrated into the Noah Multi-Parameterization (Noah-MP) land surface model in order to simulate the spatial and temporal dynamics of two key hydrometeorological disaster triggers across the city over the wet seasons during 2001-2019. Here, total runoff (TR) and rootzone soil moisture (RZSM) are considered as flood and landslide triggers, respectively. Ground-based observations at 33 pluviometric stations are interpolated, and the resulting rainfall fields are used in an in-situ precipitation-based simulation, considered as the reference for evaluating the IMERG-driven simulations. The evaluation is performed during the wet seasons (November-April), when average rainfall over the city is 4.4mm/day. Results show that IMERG products show low spatial variability at the city scale, generally overestimate rainfall rates by 12-35%, and impacts on TR and RZSM vary spatially mostly as a function of land cover and soil types. Results based on statistical and categorical metrics show that IMERG skill in detecting extreme events is moderate, with IMERG Final performing slightly better for most metrics. By analyzing two recent storms, we observe that IMERG detects mostly hourly extreme events, but underestimates rainfall rates, resulting in underestimated TR and RZSM. An evaluation of normalized time series using percentiles shows that both satellite products have significantly improved skill in detecting extreme events when compared to the evaluation using absolute values, indicating that IMERG precipitation could be potentially used as a predictor for natural disasters in urban areas.

IMERG↗

NASA NTRS2022

Validation of IMERG Oceanic Precipitation over Kwajalein

The integrated Multi-satellitE Retrievals for GPM (IMERG) Version V05B and V06B precipitation products from the Global Precipitation Measurement (GPM) mission are validated against ground-based observations from the Kwajalein Polarimetric S-band Weather Radar (KPOL) deployed at Kwajalein Atoll in the central Pacific Ocean. Such a validation is particularly important as comprehensive surface measurements over the oceans are practically infeasible, which hampers the identification of possible errors, and improvement of future versions of IMERG and other satellitebased retrieval algorithms. The V05B and V06B IMERG products are validated at their native 0.1°, 30 min resolution from 2014 to 2018 based on both volumetric and categorical metrics. This validation study indicates that precipitation rates from both IMERG V05B and V06B are underestimated with respect to radar surface estimates, but the underestimation is much reduced from V05B IMERG V06B outperforms V05B with reduced systematic bias and improved precipitation detectability. The IMERG performance is further traced back to its individual sensors and morphing-based algorithms. The overall underestimation in V05B is mainly driven by the negative relative biases from morphing-based algorithms which are largely corrected in V06B. Imagers perform generally better than sounders because of the usage of low-frequency channels in imagers which can better detect emission signals by the hydrometeors. Among imagers, the GPM Microwave Imager (GMI) and Advanced Microwave Scanning Radiometer Version 2 (AMSR2) are the best, followed by Special Sensor Microwave Imager/Sounder (SSMIS). Among sounders, the Microwave Humidity Sounder (MHS) is the best, followed by Advanced Technology Microwave Sounder (ATMS) and the Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) for V06B. Among all categories, morph-only and IR+morph only perform better than SAPHIR. SAPHIR shows the worst performance among all categories, likely due to its limited channel selection. It is envisaged that these results will improve our understanding of IMERG performance over oceans and aid in the improvement of future versions of IMERG.

KPOL↗

NASA NTRS2021

Validation of Satellite-Based Precipitation Products from TRMM to GPM

The global precipitation measurement mission (GPM) has been in operation for seven years and continues to provide a vast quantity of global precipitation data at finer temporospatial resolutions with improved accuracy and coverage. GPM’s signature algorithm, the integrated multisatellite retrievals for GPM (IMERG) is a next-generation of precipitation product expected for wide variety of research and operational applications. This study evaluates the latest version (V06B) of IMERG and its predecessor, the tropical rainfall measuring mission (TRMM) multisatellite precipitation (TMPA) 3B42 (V7) using ground-based and gauge-corrected multiradar multisensor system (MRMS) precipitation products over the conterminous United States (CONUS). The spatial distributions of all products are analyzed. The error characteristics are further examined for 3B42 and IMERG in winter and summer by an error decomposition approach, which partitions total bias into hit bias, biases due to missed precipitation and false precipitation. The volumetric and categorical statistical metrics are used to quantitatively evaluate the performance of the two satellite-based products. All products show a similar precipitation climatology with some regional differences. The two satellite-based products perform better in the eastern CONUS than in the mountainous Western CONUS. The evaluation demonstrates the clear improvement in IMERG precipitation product in comparison with its predecessor 3B42, especially in reducing missed precipitation in winter and summer, and hit bias in winter, resulting in better performance in capturing lighter and heavier precipitation.

MRMS↗

NASA NTRS2024

Assessing the Convective Environment over Irrigated and Non-Irrigated Land Use with Land-Atmosphere Coupling Metrics: Results from GRAINEX

Land use land cover change affects weather and climate. This paper quantifies land-atmosphere interactions over irrigated and non-irrigated land uses during the Great Plains Irrigation Experiment (GRAINEX). Three coupling metrics were used to quantify some land-atmosphere interactions as it relates to convection. They include: the Convective Triggering Potential (CTP) and Low-Level Humidity Index (HI low ), and the Lifting Condensation Level (LCL) Deficit. These metrics were calculated from the rawinsonde data obtained from the Integrated Sounding Systems (ISS) for Rogers Farm and York Airport along with soundings launched from the Doppler on Wheels (DOW) sites. Each metric was categorized by Intensive Observation Period (IOP), cloud cover, and time of day.

Daniel Whitesel↗

NASA NTRS2010

Assessment of Scheduling and Plan Execution of Apollo 14 Lunar Surface Operations

Although over forty years have passed since first landing on the Moon, there is not yet a comprehensive, quantitative assessment of Apollo extravehicular activities (EVAs). Quantitatively evaluating lunar EVAs will provide a better understanding of the challenges involved with surface operations. This first evaluation of a surface EVA centers on comparing the planned and the as-ran timeline, specifically collecting data on discrepancies between durations that were estimated versus executed. Differences were summarized by task categories in order to gain insight as to the type of surface operation activities that were most challenging. One Apollo 14 EVA was assessed utilizing the described methodology. Selected metrics and task categorizations were effective, and limitations to this process were identified.

Marquez, Jessica J.↗

NASA NTRS2015

Are We There Yet? Determining the Adequacy of Formalized Requirements and Test Suites

Structural coverage metrics have traditionally categorized code as either covered or uncovered. Recent work presents a stronger notion of coverage, checked coverage, which counts only statements whose execution contributes to an outcome checked by an oracle. While this notion of coverage addresses the adequacy of the oracle, for Model-Based Development of safety critical systems, it is still not enough; we are also interested in how much of the oracle is covered, and whether the values of program variables are masked when the oracle is evaluated. Such information can help system engineers identify missing requirements as well as missing test cases. In this work, we add to the idea of checked coverage an analysis of requirements coverage to help provide insight to engineers as to whether the requirements or the test suite need to be improved. We implement our approach based on a dynamic backward slicing technique and evaluate it on several systems developed in Simulink. The results of our preliminary study show that even for systems with comprehensive test suites and good sets of requirements, our approach can identify cases where more tests or more requirements are needed to improve coverage numbers.

Dynamic Slicing↗

NASA NTRS2013

Vehicle Integrated Prognostic Reasoner (VIPR) Metric Report

This document outlines a set of metrics for evaluating the diagnostic and prognostic schemes developed for the Vehicle Integrated Prognostic Reasoner (VIPR), a system-level reasoner that encompasses the multiple levels of large, complex systems such as those for aircraft and spacecraft. VIPR health managers are organized hierarchically and operate together to derive diagnostic and prognostic inferences from symptoms and conditions reported by a set of diagnostic and prognostic monitors. For layered reasoners such as VIPR, the overall performance cannot be evaluated by metrics solely directed toward timely detection and accuracy of estimation of the faults in individual components. Among other factors, overall vehicle reasoner performance is governed by the effectiveness of the communication schemes between monitors and reasoners in the architecture, and the ability to propagate and fuse relevant information to make accurate, consistent, and timely predictions at different levels of the reasoner hierarchy. We outline an extended set of diagnostic and prognostics metrics that can be broadly categorized as evaluation measures for diagnostic coverage, prognostic coverage, accuracy of inferences, latency in making inferences, computational cost, and sensitivity to different fault and degradation conditions. We report metrics from Monte Carlo experiments using two variations of an aircraft reference model that supported both flat and hierarchical reasoning.

Cornhill, Dennis↗

NASA NTRS2013

A Statistical Theory of Bidirectionality

Original concepts related to the quantification and assessment of bidirectionality in strain-gage balances were introduced by Ulbrich in 2012. These concepts are extended here in three ways: 1) the metric originally proposed by Ulbrich is normalized, 2) a categorical variable is introduced in the regression analysis to account for load polarity, and 3) the uncertainty in both normalized and non-normalized bidirectionality metrics is quantified. These extensions are applied to four representative balances to assess the bidirectionality characteristics of each. The paper is tutorial in nature, featuring reviews of certain elements of regression and formal inference. Principal findings are that bidirectionality appears to be a common characteristic of most balance outputs and that unless it is taken into account, it is likely to consume the entire error budget of a typical balance calibration experiment. Data volume and correlation among calibration loads are shown to have a significant impact on the precision with which bidirectionality metrics can be assessed.

DeLoach, Richard↗

NASA NTRS2019

Onboard Decision-Making for Nominal and Contingency sUAS Flight

This study presents an onboard decision-making architecture for small unmanned aerial systems (sUAS). The decision-maker is part of NASA's SAFE50 project that is working under the UAS Traffic Management (UTM) Technical Capability Level (TCL) 4 to provide autonomous point-to-point UAV flight in BVLOS, high-density urban environments. The decision-maker monitors various metrics to determine the safety and feasibility of the mission and categorizes flight states as Nominal, Off-Nominal, Alternate Land, and Land Now in a finite state machine. Changes in the monitored metrics serve as transitions in the state machine and trigger replanning. Navigation degradation and communication failure are simulated to show the feasibility of the decision-maker framework in appropriately switching the flight state.

Baculi, Joshua↗

NASA NTRS2013

Determining Component Probability using Problem Report Data for Ground Systems used in Manned Space Flight

During the shuttle era NASA utilized a failure reporting system called the Problem Reporting and Corrective Action (PRACA) it purpose was to identify and track system non-conformance. The PRACA system over the years evolved from a relatively nominal way to identify system problems to a very complex tracking and report generating data base. The PRACA system became the primary method to categorize any and all anomalies from corrosion to catastrophic failure. The systems documented in the PRACA system range from flight hardware to ground or facility support equipment. While the PRACA system is complex, it does possess all the failure modes, times of occurrence, length of system delay, parts repaired or replaced, and corrective action performed. The difficulty is mining the data then to utilize that data in order to estimate component, Line Replaceable Unit (LRU), and system reliability analysis metrics. In this paper, we identify a methodology to categorize qualitative data from the ground system PRACA data base for common ground or facility support equipment. Then utilizing a heuristic developed for review of the PRACA data determine what reports identify a credible failure. These data are the used to determine inter-arrival times to perform an estimation of a metric for repairable component-or LRU reliability. This analysis is used to determine failure modes of the equipment, determine the probability of the component failure mode, and support various quantitative differing techniques for performing repairable system analysis. The result is that an effective and concise estimate of components used in manned space flight operations. The advantage is the components or LRU's are evaluated in the same environment and condition that occurs during the launch process.

Monaghan, Mark W.↗

NASA NTRS2002

Warpage of Large Curved Composite Panels due to Manufacturing Anomalies

This paper discusses the influences of a misaligned layer, a resin-rich slightly thicker layer, and a small thermal gradient on the thermally-induced deformations of large curved composite panels during cooldown from their cure temperature. The deformations represent warpage of the panels due to anomalies that occur during layup, consolidation, and cure. Two-dimensional finite element analyses are used The deformations are categorized as to their impact on circumferential and twist warpage metrics. The results are intended to highlight the sensitivity of manufactured panel shape to the various unwanted effects that can occur during manufacturing.

Starnes, James H., Jr.↗

NASA NTRS2005

Using Qualitative Hazard Analysis to Guide Quantitative Safety Analysis

Quantitative methods can be beneficial in many types of safety investigations. However, there are many difficulties in using quantitative m ethods. Far example, there may be little relevant data available. This paper proposes a framework for using quantitative hazard analysis to prioritize hazard scenarios most suitable for quantitative mziysis. The framework first categorizes hazard scenarios by severity and likelihood. We then propose another metric "modeling difficulty" that desc ribes the complexity in modeling a given hazard scenario quantitatively. The combined metrics of severity, likelihood, and modeling difficu lty help to prioritize hazard scenarios for which quantitative analys is should be applied. We have applied this methodology to proposed concepts of operations for reduced wake separation for airplane operatio ns at closely spaced parallel runways.

Shortle, J. F.↗

NASA NTRS2019

Satellite Conjunction "Probability," "Possibility," and "Plausibility": A Categorization of Competing Conjunction Assessment Risk Assessment Paradigms

A number of different conjunction assessment (CA) risk analysis methods and metrics have been proposed in the critical literature, and they vary widely in purport and form. However, they tend to be proposed individually and episodically, so that it is difficult for a CA practitioner to take stock of the possibilities, under- stand their fundamental differences, and make informed choices for their particular CA risk assessment enterprise. The present study seeks to collect the major proposals for risk assessment methods and parameters and organize them categorically, under the proposed divisions of “probability,” “plausibility,” and “possibility,” as well as formulate what appears for each to be its fundamental question and, where applicable, null hypothesis. This activity can, through a bottom-up approach, provide some of the building blocks for an overarching CA philosophy, as well as establish concepts and terminology potentially useful to the broader discussion of these topics.

Hejduk, M. D.↗

NASA NTRS2019

Satellite Conjunction “Probability,” “Plausibility,” and “Possibility”: A Categorization of Competing Satellite Conjunction Assessment Risk Analysis Paradigms

Hejduk, M. D.↗

NASA NTRSDate not supplied

Improving the Fidelity of Capability & Resource Weighting in A Probalistic Risk Assessment Model for Spaceflight

INTRODUCTION NASA’s Informing Mission Planning via Analysis of Complex Tradespaces (IMPACT) tool uses Probabilistic Risk Assessment (PRA) to provide an evidence-based, data-driven estimate of how medical system capabilities affect mission outcomes. IMPACT maps condition incidence to available resources thereby facilitating the calculation of outcome metrics that allow the estimation of mission medical risk. Conditions can be nominally categorized as either treated or untreated depending on the availability of necessary diagnostic and therapeutic capabilities. This categorization enables IMPACT to estimate the effect of various medical system configurations on mission outcomes such as crew mortality, disability, crew member down-time, return to duty/recovery, and need for evacuation. IMPACT currently employs an equal weighting, “partial credit” approach to define treatment in which each of the capabilities associated with a given condition contributes an equal amount to management of the condition. This feature enables IMPACT to report values in between “fully untreated” and “fully treated” based on the proportion of capabilities available within the model. However, as is normal in medical/clinical practice, not all individual capabilities contribute equally to medical care. For example, the ability to provide intramuscular epinephrine during an anaphylactic episode contributes more likelihood of overall management success than does the administration of oral diphenhydramine. We hypothesize that weighting the relative contribution of each capability to each specific condition will improve outcome prediction and therefore will better provide mission planners with more nuanced and accurate options when designing space medical systems. METHODS Using a five-point Fibonacci scaling sequence (1, 2, 3, 5, 8) subject matter experts from NASA’s Exploration Medical Capabilities (ExMC) element assigned relative contribution weighting values to each identified capability within IMPACT. Since the relative importance of each capability varies depending on the specific condition, the resulting “partial” weighting was completed for more than 1,600 individual weighting assignments for 666 capabilities across 121 conditions. Each assignment required three-physician concurrence based on the overall importance of the capability to the diagnosis and management of the condition being considered and the difficulty with which it could be improvised by the crew. Once complete, 100,000 IMPACT simulations were run for a 6-month Lunar mission with a 30-day surface stay to evaluate the effect of this modification of the model. RESULTS Partial weighting significantly decreased predicted task time loss (TTL), evacuation, and loss of crew life without causing significant changes to the recommended medical system design. CONCLUSIONS The paucity of real-world referent data to support long-duration space missions of this type limits the ability to judge one predictive analytics method as superior to another. However, since the proposed method significantly reduces and optimizes outcome risks—without changing the medical system design—incorporating a partial weighting methodology is likely to provide a more accurate and operationally-relevant representation of medical risk without compromising IMPACTs ability to inform overarching medical system requirements.

Steller JG↗

NASA NTRSDate not supplied

TriTruss Packaging and Deployment Trade Study

An architecture and feasibility study of next generation In-Space Assembled Telescopes (iSAT) concluded that robotic in-space assembly of modular components is necessary to enable large (>15-meter diameter) primary apertures. The iSAT study recommended that the foundational structure be assembled from modular TriTruss modules that are packaged for launch, deployed on-orbit and robotically assembled into the final configuration. This paper will describe and summarize the results of a trade study that investigated viable concept of operations (ConOps) for the packaging and deployment (P&D) of individual planar TriTruss modules that could be assembled to form a large aperture iSAT. The ideal TriTruss P&D concept is defined as one that: allows for efficient packaging; has sufficient geometric versatility to be launch vehicle independent; provides a stiff and lightweight structure; has low mechanical complexity; and has component modularity. The P&D concept should allow for prelaunch subsystem or utility integration if required. The concept would be kinematically simple and be robotically deployed using a minimum number of specialized tools. In this first phase of an ongoing more comprehensive trade study, concepts were proposed and then evaluated based on initial metrics representing features of an ideal P&D concept. The P&D concepts evaluated are categorized as: core collapse, face collapse, and erectable structures. Sub-scale models were constructed to help understand the kinematics and mechanical complexity required to enable P&D. Based on a weighting scale, the most promising candidate P&D concepts have selected and will undergo more rigorous structural design, analysis, and testing in the study’s next phase. The ultimate goal of the comprehensive trade study will be to recommend a single TriTruss design and associated P&D concept that will be built and evaluated at NASA Langley Research Center’s In-Space Assembly Laboratory.

in-space assembly (ISA)↗

NASA NTRS2015

Probability of Loss of Crew Achievability Studies for NASA's Exploration Systems Development

Over the last few years, NASA has been evaluating various vehicle designs for multiple proposed design reference missions (DRM) beyond low Earth orbit in support of its Exploration Systems Development (ESD) programs. This paper addresses several of the proposed missions and the analysis techniques used to assess the key risk metric, probability of loss of crew (LOC). Probability of LOC is a metric used to assess the safety risk as well as a design requirement. These assessments or studies were categorized as LOC achievability studies to help inform NASA management as to what "ball park" estimates of probability of LOC could be achieved for each DRM and were eventually used to establish the corresponding LOC requirements. Given that details of the vehicles and mission are not well known at this time, the ground rules, assumptions, and consistency across the programs become the important basis of the assessments as well as for the decision makers to understand.

Boyer, Roger L.↗

NASA NTRS2014

Complexity and Pilot Workload Metrics for the Evaluation of Adaptive Flight Controls on a Full Scale Piloted Aircraft

Flight research has shown the effectiveness of adaptive flight controls for improving aircraft safety and performance in the presence of uncertainties. The National Aeronautics and Space Administration's (NASA)'s Integrated Resilient Aircraft Control (IRAC) project designed and conducted a series of flight experiments to study the impact of variations in adaptive controller design complexity on performance and handling qualities. A novel complexity metric was devised to compare the degrees of simplicity achieved in three variations of a model reference adaptive controller (MRAC) for NASA's F-18 (McDonnell Douglas, now The Boeing Company, Chicago, Illinois) Full-Scale Advanced Systems Testbed (Gen-2A) aircraft. The complexity measures of these controllers are also compared to that of an earlier MRAC design for NASA's Intelligent Flight Control System (IFCS) project and flown on a highly modified F-15 aircraft (McDonnell Douglas, now The Boeing Company, Chicago, Illinois). Pilot comments during the IRAC research flights pointed to the importance of workload on handling qualities ratings for failure and damage scenarios. Modifications to existing pilot aggressiveness and duty cycle metrics are presented and applied to the IRAC controllers. Finally, while adaptive controllers may alleviate the effects of failures or damage on an aircraft's handling qualities, they also have the potential to introduce annoying changes to the flight dynamics or to the operation of aircraft systems. A nuisance rating scale is presented for the categorization of nuisance side-effects of adaptive controllers.

pilot workload metrics↗