Search NASA⌕ Search

SEARCH · Search NASA

Results for “Coverage Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Runway Sign Classifier: A DAL C Certifiable Machine Learning System

In recent years, the remarkable progress of Machine Learning (ML) technologies within the domain of Artificial Intelligence (AI) systems has presented unprecedented opportunities for the aviation industry, paving the way for further advancements in automation, including the potential for single pilot or fully autonomous operation of large commercial airplanes. However, ML technology faces major incompatibilities with existing airborne certification standards, such as ML model traceability and explainability issues or the inadequacy of traditional coverage metrics. Certification of ML-based airborne systems using current standards is problematic due to these challenges. This paper presents a case study of an airborne system utilizing a Deep Neural Network (DNN) for airport sign detection and classification. Building upon our previous work, which demonstrates compliance with Design Assurance Level (DAL) ”D”, we upgrade the system to meet the more stringent requirements of Design Assurance Level ”C”. To achieve DAL C, we employ an established architectural mitigation technique involving two redundant and dissimilar Deep Neural Networks. The application of novel ML-specific data management techniques further enhances this approach. This work is intended to illustrate how the certification challenges of ML-based systems can be addressed for medium criticality airborne applications.

Flight Software↗
DOE OSTI2025

An entropy-based debiasing approach to quantifying experimental coverage for novel applications of interest in the nuclear community

This manuscript proposes a novel information-theoretic approach to the quantification of experimental relevance, i.e., coverage, to achieve optimal data assimilation results for nuclear engineering applications. Specifically, this work posits the need for a new metric, called coverage (q C ) of an application’s quantity of interest, i.e., eigenvalue or power peaking for an advanced reactor concept, defined herein as the theoretically maximum achievable reduction in the quantity’s uncertainty given measurements from a pool of experiments in a manner that is independent of the data assimilation procedure employed. Currently, reduction in a quantity’s uncertainty is strongly biased by the underlying assumptions of the assimilation procedure to account for the under-determined nature of such problems and the similarity criterion employed to identify relevant experiments. To address this challenge, this work has developed a coverage metric, q C , based on mutual information, which establishes a new conceptual framework for assessing coverage, one that is independent of the model parameters and responses degree of variations in both the experimental and application domains, i.e., linear vs non-linear, and their prior uncertainty distributions, i.e., Gaussian vs. non-Gaussian. The q C is an entropic measure capable of addressing coverage for general nonlinear problems with non-Gaussian uncertainties and inclusive of the measurement uncertainties from multiple experiments. Numerical experiments from manufactured analytical problems as well as a set of benchmarks from the ICSBEP handbook are employed to demonstrate its theoretical and practical performance as compared to the c k -based experiment selection methodology, commonly employed in the neutronic community. The manuscript then employs other well-known adaptations to existing data assimilation methodologies for nonlinear and non-Gaussian problems capable of achieving the coverage posited by q C .

Bayesian data assimilation↗
NASA NTRS2016

TAT-C: A Trade-Space Analysis Tool for Constellations

Under a changing technological and economic environment, there is growing interest in implementing future NASA Earth Science missions as Distributed Spacecraft Missions (DSM). The objective of our project is to provide a framework that facilitates DSM Pre-Phase A investigations and optimizes DSM designs with respect to a-priori Science goals. In this first version of our Trade-space Analysis Tool for Constellations (TAT-C), we are investigating questions such as: Which type of constellations should be chosen? How many spacecraft should be included in the constellation? Which design has the best costrisk value? This paper describes the overall architecture of TAT-C including: a User Interface (UI) interacting with multiple users - scientists, missions designers or program managers; an Executive Driver gathering requirements from UI and formulating Trade-space Search Requests for the Trade-space Search Iterator, which in collaboration with the Orbit Coverage, Reduction Metrics, and Cost Risk modules generates multiple potential architectures and their associated characteristics. UI will include Graphical, Command Line and Application Programmer Interfaces to respond to the demands of various levels of users expertise. Science inputs are grouped into various mission concepts, satellite specifications, and payload specifications, while science outputs are grouped into several types of metrics - spatial, temporal, angular and radiometric. Orbit Coverage leverages the use of the Goddard Mission Analysis Tool (GMAT) to compute coverage and ancillary data that are passed to Reduction Metrics. Then, for each architecture design, Cost Risk will provide estimates of the cost and life cycle cost as well as technical and cost risk of the proposed mission. Additionally, the Knowledge Base module is a centralized store of structured data readable by humans and machines. It will support both TAT-C analysis when composing new mission concepts from existing model inputs, and TAT-C exploration when discovering new mission concepts by querying previous results.

Science Data Processing↗
NASA NTRS2017

End-to-End Trade-Space Analysis for Designing Constellation

Multipoint measurement missions can provide a significant advancement in science return and this science interest coupled with as many recent technological advances are driving a growing trend in exploring distributed architectures for future NASA missions. Distributed Spacecraft Missions (DSMs) leverage multiple spacecraft to achieve one or more common goals. In particular, a constellation is the most general form of DSM with two or more spacecraft placed into specific orbit(s) for the purpose of serving a common objective (e.g., CYGNSS). Because a DSM architectural trade-space includes both monolithic and distributed design variables, DSM optimization is a large and complex problem with multiple conflicting objectives. Over the last two years, our team has been developing a Trade-space Analysis Tool for Constellations (TAT-C), implemented in common programming languages for pre-Phase A constellation mission analysis. By evaluating alternative mission architectures, TAT-C seeks to minimize cost and maximize performance for pre-defined science goals. This presentation will describe the overall architecture of TAT-C including: a User Interface (UI) at several levels of details and user expertise; Trade-space Search Requests that are created from the Science requirements gathered by the UI and validated by a Knowledge Base; a Knowledge Base to compare the current requests to prior mission concepts to potentially prune the trade-space; a Trade-space Search Iterator which, with inputs from the Knowledge Base, and, in collaboration with the Orbit & Coverage, Reduction & Metrics, and Cost& Risk modules, generates multiple potential architectures and their associated characteristics. TAT-C leverages the use of the Goddard Mission Analysis Tool (GMAT) to compute coverage and ancillary data, modeling orbits to balance accuracy and performance. The current version includes uniform and non-uniform Walker constellations as well as Ad-Hoc and precessing constellations, and its cost model represents an aggregate model consisting of Cost Estimating Relationships (CERs) from widely accepted models. The current GUI automatically generates graphics representing metrics such as average revisit time or coverage as a function of cost. The end-to-end system will be demonstrated as part of the presentation.

Mission Desig↗
NASA NTRS2018

Evaluating Expected Performance and Graceful Degradation in Distributed Spacecraft Missions

Distributed Spacecraft Missions (DSMs) have largely garnered interest due to lower cost and new technical capabilities. However, another important attribute in DSMs is the potential of graceful degradation where a failure of one or more spacecraft (or instruments) in the DSM can reduce total performance, but the remaining functional spacecraft can maintain a limited science return. The potential for retaining some mission value in the event of partial failures leads to enhanced lifetime value. Here, an approach is developed to analyze graceful degradation and performance of DSM used for Earth observation. The system performance is defined through metrics of coverage and average revisit time of points of interest. Graceful degradation is quantified through total change and rate of change in performance metrics with increasing sub-system failures over the lifetime of a mission. An Expected Performance measure is formulated for conducting comparison of DSM architectures for early stage conceptual mission design and trade studies.

Distributed Spacecraft Missions↗
NASA NTRS2018

Evaluating Expected Performance and Graceful Degradation in Distributed Spacecraft Missions

Distributed Spacecraft Missions (DSMs) have largely garnered interest due to lower cost and new technical capabilities. However, another important attribute in DSMs is the potential of graceful degradation where a failure of one or more spacecraft (or instruments) in the DSM can reduce total performance, but the remaining functional spacecraft can maintain a limited science return. The potential for retaining some mission value in the event of partial failures leads to enhanced lifetime value. Here, an approach is developed to analyze graceful degradation and performance of DSM used for Earth observation. The system performance is defined through metrics of coverage and average revisit time of points of interest. Graceful degradation is quantified through total change and rate of change in performance metrics with increasing sub-system failures over the lifetime of a mission. An Expected Performance measure is formulated for conducting comparison of DSM architectures for early stage conceptual mission design and trade studies.

Distributed Spacecraft Missions; constellations; p↗
NASA NTRS2016

Trade-Space Analysis Tool for Constellations (TAT-C)

Traditionally, space missions have relied on relatively large and monolithic satellites, but in the past few years, under a changing technological and economic environment, including instrument and spacecraft miniaturization, scalable launchers, secondary launches as well as hosted payloads, there is growing interest in implementing future NASA missions as Distributed Spacecraft Missions (DSM). The objective of our project is to provide a framework that facilitates DSM Pre-Phase A investigations and optimizes DSM designs with respect to a-priori Science goals. In this first version of our Trade-space Analysis Tool for Constellations (TAT-C), we are investigating questions such as: How many spacecraft should be included in the constellation? Which design has the best costrisk value? The main goals of TAT-C are to: Handle multiple spacecraft sharing a mission objective, from SmallSats up through flagships, Explore the variables trade space for pre-defined science, cost and risk goals, and pre-defined metrics Optimize cost and performance across multiple instruments and platforms vs. one at a time.This paper describes the overall architecture of TAT-C including: a User Interface (UI) interacting with multiple users - scientists, missions designers or program managers; an Executive Driver gathering requirements from UI, then formulating Trade-space Search Requests for the Trade-space Search Iterator first with inputs from the Knowledge Base, then, in collaboration with the Orbit Coverage, Reduction Metrics, and Cost Risk modules, generating multiple potential architectures and their associated characteristics. TAT-C leverages the use of the Goddard Mission Analysis Tool (GMAT) to compute coverage and ancillary data, streamlining the computations by modeling orbits in a way that balances accuracy and performance.TAT-C current version includes uniform Walker constellations as well as Ad-Hoc constellations, and its cost model represents an aggregate model consisting of Cost Estimating Relationships (CERs) from widely accepted models. The Knowledge Base supports both analysis and exploration, and the current GUI prototype automatically generates graphics representing metrics such as average revisit time or coverage as a function of cost.

Science Data Processing↗
DOE OSTI2022

Selective Dynamical Imaging of Interferometric Data

Recent developments in very long baseline interferometry (VLBI) have made it possible for the Event Horizon Telescope (EHT) to resolve the innermost accretion flows of the largest supermassive black holes on the sky. The sparse nature of the EHT’s (u, v)-coverage presents a challenge when attempting to resolve highly time-variable sources. We demonstrate that the changing (u, v)-coverage of the EHT can contain regions of time over the course of a single observation that facilitate dynamical imaging. These optimal time regions typically have projected baseline distributions that are approximately angularly isotropic and radially homogeneous. We derive a metric of coverage quality based on baseline isotropy and density that is capable of ranking array configurations by their ability to produce accurate dynamical reconstructions. We compare this metric to existing metrics in the literature and investigate their utility by performing dynamical reconstructions on synthetic data from simulated EHT observations of sources with simple orbital variability. We then use these results to make recommendations for imaging the 2017 EHT Sgr A* data set.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Updates to the Regional Seismic Travel Time (RSTT) Model: 2. Path-dependent Travel-time Uncertainty

Abstract The regional seismic travel time (RSTT) model and software were developed to improve travel-time prediction accuracy by accounting for three-dimensional crust and upper mantle structure. Travel-time uncertainty estimates are used in the process of associating seismic phases to events and to accurately calculate location uncertainty bounds (i.e. event location error ellipses). We improve on the current distance-dependent uncertainty parameterization for RSTT using a random effects model to estimate slowness (inverse velocity) uncertainty as a mean squared error for each model parameter. The random effects model separates the error between observed slowness and model predicted slowness into bias and random components. The path-specific travel-time uncertainty is calculated by integrating these mean squared errors along a seismic-phase ray path. We demonstrate that event location error ellipses computed for a 90% coverage ellipse metric (used by the Comprehensive Nuclear-Test-Ban Treaty Organization International Data Centre (IDC)), and using the path-specific travel-time uncertainty approach, are more representative (median 82.5% ellipse percentage) of true location error than error ellipses computed using distance-dependent travel-time uncertainties (median 70.1%). We also demonstrate measurable improvement in location uncertainties using the RSTT method compared to the current station correction approach used at the IDC (median 74.3% coverage ellipse).

58 GEOSCIENCES↗
DOE OSTI2026

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY↗
NASA NTRS1989

Multi-version software reliability through fault-avoidance and fault-tolerance

A number of experimental and theoretical issues associated with the practical use of multi-version software to provide run-time tolerance to software faults were investigated. A specialized tool was developed and evaluated for measuring testing coverage for a variety of metrics. The tool was used to collect information on the relationships between software faults and coverage provided by the testing process as measured by different metrics (including data flow metrics). Considerable correlation was found between coverage provided by some higher metrics and the elimination of faults in the code. Back-to-back testing was continued as an efficient mechanism for removal of un-correlated faults, and common-cause faults of variable span. Software reliability estimation methods was also continued based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. New fault tolerance models were formulated. Simulation studies of the Acceptance Voting and Multi-stage Voting algorithms were finished and it was found that these two schemes for software fault tolerance are superior in many respects to some commonly used schemes. Particularly encouraging are the safety properties of the Acceptance testing scheme.

Vouk, Mladen A.↗
DOE OSTI2023

Outdoor annual algae productivity improvements at the pre-pilot scale through crop rotation and pond operational management strategies

The Development of Integrated Screening, Cultivar Optimization, and Verification Research (DISCOVR) collaborative consortium operated pre-pilot scale outdoor ponds to deliver much-needed multi-year, long-term and consistent, algae cultivation data relevant to understanding the current state of technology in terms of expected seasonal algae biomass productivity. Over the course of four years from 2018 to 2021, twelve identical 4.2 m 2 mini-ponds were run in triplicate sets to test strains and operational strategies demonstrated in small-, indoor photobioreactors, in pursuit of increasing overall algae areal productivity and projected farm yield. Fourteen different cultivars derived from a strain screening pipeline were tested. Through deliberate seasonal crop rotation and improvements in operational strategies, annual biomass productivity increased from 11.6 to 17.6 g m -2 day -1 , a > 50% increase over the 2018 baseline. Both brackish and marine strains were included and four out of the fourteen strains consistently yielded high productivity across multiple years; brackish strains Monoraphidium minutum (26BAM) and Scenedesmus obliquus (UTEX393), and marine strains Tetraselmis striata (LANL1001) and Picochlorum celeri (TG2). These freely available datasets, which represent nearly complete annual daily coverage of cultivation metrics including weather, pond temperature and pH, nutrients, and productivity, are unique in the public domain and seek to fill agronomic and operational knowledge gaps to help in the eventual commercialization of algal biofuels and bioproducts.

09 BIOMASS FUELS↗
NASA NTRS2015

A Framework for Orbital Performance Evaluation in Distributed Space Missions for Earth Observation

Distributed Space Missions (DSMs) are gaining momentum in their application to earth science missions owing to their unique ability to increase observation sampling in spatial, spectral and temporal dimensions simultaneously. DSM architectures have a large number of design variables and since they are expected to increase mission flexibility, scalability, evolvability and robustness, their design is a complex problem with many variables and objectives affecting performance. There are very few open-access tools available to explore the tradespace of variables which allow performance assessment and are easy to plug into science goals, and therefore select the most optimal design. This paper presents a software tool developed on the MATLAB engine interfacing with STK, for DSM orbit design and selection. It is capable of generating thousands of homogeneous constellation or formation flight architectures based on pre-defined design variable ranges and sizing those architectures in terms of predefined performance metrics. The metrics can be input into observing system simulation experiments, as available from the science teams, allowing dynamic coupling of science and engineering designs. Design variables include but are not restricted to constellation type, formation flight type, FOV of instrument, altitude and inclination of chief orbits, differential orbital elements, leader satellites, latitudes or regions of interest, planes and satellite numbers. Intermediate performance metrics include angular coverage, number of accesses, revisit coverage, access deterioration over time at every point of the Earth's grid. The orbit design process can be streamlined and variables more bounded along the way, owing to the availability of low fidelity and low complexity models such as corrected HCW equations up to high precision STK models with J2 and drag. The tool can thus help any scientist or program manager select pre-Phase A, Pareto optimal DSM designs for a variety of science goals without having to delve into the details of the engineering design process.

Earth Science↗
NASA NTRS2010

Orion Flight Performance Design Trades

A significant portion of the Orion pre-PDR design effort has focused on balancing mass with performance. High level performance metrics include abort success rates, lunar surface coverage, landing accuracy and touchdown loads. These metrics may be converted to parameters that affect mass, such as ballast for stabilizing the abort vehicle, propellant to achieve increased lunar coverage or extended missions, or ballast to increase the lift-to-drag ratio to improve entry and landing performance. The Orion Flight Dynamics team was tasked to perform analyses to evaluate many of these trades. These analyses not only provide insight into the physics of each particular trade but, in aggregate, they illustrate the processes used by Orion to balance performance and mass margins, and thereby make design decisions. Lessons learned can be gleaned from a review of these studies which will be useful to other spacecraft system designers. These lessons fall into several categories, including: appropriate application of Monte Carlo analysis in design trades, managing margin in a highly mass-constrained environment, and the use of requirements to balance margin between subsystems and components. This paper provides a review of some of the trades and analyses conducted by the Flight Dynamics team, as well as systems engineering lessons learned.

Jackson, Mark C.↗
NASA NTRSDate not supplied

A New Statistical Estimate of the Radar Coverage of the Low Earth Orbit Debris Environment

For over three decades, the NASA Orbital Debris Program Office (ODPO) has used the Goldstone Orbital Debris Radar, Haystack Ultrawideband Satellite Imaging Radar (HUSIR), and Haystack Auxiliary (HAX) radar assets to collect data on the low Earth orbit (LEO) debris environment. Each radar, with its unique beamwidth, altitude and inclination coverage, and limiting size threshold, operates in a beam park mode to statistically sample the orbital debris population in LEO. Provided that these assets are shared with other users, the orbital debris data collection is not continuous; rather, intermittent data collects are acquired and sent to the NASA ODPO. To understand the sampling process conducted by each radar over time and any related observational biases, a Statistical Confirmation of Radar Uniformity or Bias (SCRUB) code has been implemented to model the coverage of these assets for informing future operations, as well as usage of the data collected from these ground-based sensors. For this analysis, Right Ascension of the Ascending Node (RAAN) is used as a metric to measure statistical coverage. A complete survey of the LEO environment is understood to be measurements that sufficiently sample all values of RAAN for each altitude-inclination pair visible from the radar asset. Regions of incompleteness or statistical bias can help inform future observation campaigns. The SCRUB tool evaluates the coverage of the LEO environment, not by examining individual objects that may pass through a sensor’s field of view (FOV), but by determining which orbit planes pass through the FOV. Once a pointing geometry for a radar site, observation time, and range extent are configured by the user, SCRUB computes the inclinations and altitudes that are visible by the sensor. For each inclination-altitude pair, there is a distinct pair of possible RAAN values, corresponding to the ascending and descending orbit passing through that point in the sensor cone. Repeating this process for all points in the beam, and for multiple time periods during an observation window, creates a matrix of all inclination-altitude-RAAN combinations that are visible during a sensor run. This process can then be repeated for all observations within a year (for an annual survey), propagating all the RAAN values to a common epoch, typically the start of a calendar year, and combining the observations to assemble a full estimate of the RAAN coverage of the LEO environment. This RAAN coverage can then be analyzed for uniformity of sampling, within a certain inclination-altitude pair, or between larger regions of the space environment. This paper provides a general overview of the radar assets utilized by ODPO, typical analysis data products assuming circular orbits, and a discussion of the algorithms that feed between modeling and measurement operations. Following the description of the algorithm, estimates of coverage using HUSIR radar data collected from multiple years are developed and compared.

Chris L Ostrom↗
DOE OSTI2022

Sea ice breakup and freeze-up indicators for users of the Arctic coastal environment

Abstract. The timing of sea ice retreat and advance in Arctic coastal waters varies substantially from year to year. Various activities, ranging from marine transport to the use of sea ice as a platform for industrial activity or winter travel, are affected by variations in the timing of breakup and freeze-up, resulting in a need for indicators to document the regional and temporal variations in coastal areas. The primary objective of this study is to use locally based metrics to construct indicators of breakup and freeze-up in the Arctic and subarctic coastal environment. The indicators developed here are based on daily sea ice concentrations derived from satellite passive-microwave measurements. The “day of year” indicators are designed to optimize value for users while building on past studies characterizing breakup and freeze-up dates in the open pack ice. Relative to indicators for broader adjacent seas, the coastal indicators generally show later breakup at sites known to have landfast ice. The coastal indicators also show earlier freeze-up at some sites in comparison with freeze-up for broader offshore regions, likely tied to earlier freezing of shallow-water regions and areas affected by freshwater input from nearby streams and rivers. A factor analysis performed to synthesize the local indicator variations shows that the local breakup and freeze-up indicators have greater spatial variability than corresponding metrics based on regional ice coverage. However, the trends towards earlier breakup and later freeze-up are unmistakable over the post-1979 period in the synthesized metrics of coastal breakup and freeze-up and the corresponding regional ice coverage. The findings imply that locally defined indicators can serve as key links between pan-Arctic or global indicators such as sea ice extent or volume and local uses of sea ice, with the potential to inform community-scale adaptation and response.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2015

Quantifying Mapping Orbit Performance in the Vicinity of Primitive Bodies

Predicting and quantifying the capability of mapping orbits in the vicinity of primitive bodies is challenging given the complex orbit geometries that exist and the irregular shape of the bodies themselves. This paper employs various quantitative metrics to characterize the performance and relative effectiveness of various types of mapping orbits including terminator, quasi-terminator, hovering, pingpong, and conic-like trajectories. Metrics of interest include surface area coverage, lighting conditions, and the variety of viewing angles achieved. The metrics discussed in this investigation are intended to enable mission designers and project stakeholders to better characterize candidate mapping orbits during preliminary mission formulation activities.The goal of this investigation is to understand the trade space associated with carrying out remotesensing campaigns at small primitive bodies in the context of a robotic space mission. Specifically,this study seeks to understand the surface viewing geometries, ranges, etc. that are available fromseveral commonly proposed mapping orbits architectures.

Pavlak, Thomas A.↗