SEARCH · Search NASA

Results for “data quality metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

NASA NTRS2008

Experiences in Bridging the Gap between Science and Decision Making at NASA's GSFC Earth Science Data and Information Services Center (GES DISC)

Recognizing the significance of NASA remote sensing Earth science data in monitoring and better understanding our planet s natural environment, NASA has implemented the Decision Support Through Earth Science Research Results program (NASA ROSES solicitations). a) This successful program has yielded several monitoring, surveillance, and decision support systems through collaborations with benefiting organizations. b) The Goddard Space Flight Center (GSFC) Earth Sciences Data and Information Services Center (GES DISC) has participated in this program on two projects (one complete, one ongoing), and has had opportune ad hoc collaborations gaining much experience in the formulation, management, development, and implementation of decision support systems utilizing NASA Earth science data. c) In addition, GES DISC s understanding of Earth science missions and resulting data and information, including data structures, data usability and interpretation, data interoperability, and information management systems, enables the GES DISC to identify challenges that come with bringing science data to decision makers. d) The purpose of this presentation is to share GES DISC decision support system project experiences in regards to system sustainability, required data quality (versus timeliness), data provider understanding of how decisions are made, and the data receivers willingness to use new types of information to make decisions, as well as other topics. In addition, defining metrics that really evaluate success will be exemplified.

Kempler, Steven↗

NASA NTRS1986

Maximum normalized acceleration as a flying qualities parameter

In 1984, Maximum Normalized Rate (MNR) was presented as a Flying Qualities parameter. Subsequent analysis of data from ground based simulation and flight test revealed the utility of a companion parameter, Maximum Normalized Acceleration (MNA). MNR and MNA profiles reveal the presence of both continuous and pulsed compensation strategies during discrete attitude tracking. In addition, MNR appears to be a suitable metric for pilot opinion in the LATHOS data base, while the MNR/MNA relationship is sensitive to pilot-induced-oscillation (PIO) and roll ratcheting problems. As part of an investigation of this problem, Northrop has developed an analysis technique known as the Step Target Method. The Step Target method is essentially a one degree-of-freedom simulation, where an attitude command in the form of a step function is presented to a closed-loop pilot/aircraft model. The two parameters MNR and MNA were shown to be useful in Flying Qualities analysis. MNR was shown to correlate with Pilot Opinion Rating in the LATHOS data base, while MNA reflects PIO and roll ratcheting. Profiles of MNR versus MNA reveal the presence of pulsed compensation strategies in both ground based and in-flight simulation. Furthermore, comparison of continuous and discrete attitude tracking simulation data reveals that these two tracking tasks exhibit independent sensitivities to aircraft characteristics.

Warner, J. S.↗

NASA NTRSDate not supplied

Lessons Learned From Smallsat Microwave Sounders Data Assimilation in the NASA Goddard Earth Observing System (GEOS)

The NASA Time-Resolved Observations of Precipitation structure and storm Intensity with a Constellation of Smallsats (TROPICS) mission is a constellation of small satellites carrying state-of-art microwave temperature and humidity sounders with 12 channels between 91 GHz and 205 GHz frequency. Including TROPICS-pathfinder, launched on 30 June 2021, five TROPICS satellites operate and provide temperature and humidity data to NWP and atmospheric retrieval communities. This study evaluates the quality of radiance data from these five satellites. It assesses data impacts on NASA Goddard Earth Observing System (GEOS) global NWP analysis and forecasts while seeking answers to the following key questions: 1. What information do TROPICS satellites bring to NWP analysis? 2. How will TROPICS data assimilation affect NWP forecasts, including tropical cyclone analysis and forecasts? 3. What are the benefits of having multiple TROPICS satellites in NWP forecasts? 4. What are the weaknesses regarding instrument stability, consistency, and data quality of these CubeSats? 5. Can multiple TROPICS make similar impacts as conventional MW sensors such as ATMS or GPM Microwave Imager (GMI)? The sensitivities of each TROPICS channel to temperature, water vapor, cloud, and precipitation are examined in various weather conditions. Results from observing system experiments assimilating TROPICS radiance are used for evaluation metrics, including forecast skills and fit to other observations such as radiosondes and microwave and infrared sounders. Finally, the lessons learned from using these cube satellite radiance data in the GEOS atmospheric data assimilation system are shared.

Min-Jeong Kim↗

NASA NTRSDate not supplied

Linear Regression Model for Predictive Service Provider Selection

The increasing number of satellites in orbit has led to a growing reliance on third-party service providers for data transfer between Earth and space. Traditional approaches to managing satellite communications require human intervention, which becomes more burdensome with the escalating number of satellites. This research addresses the need for an efficient and automated system to optimize service provider selection for NASA space communication. Previous research has utilized human-operated approaches for service provider management. Our study fills a gap by developing a cognitive algorithm that automates and optimizes the selection process based on various parameters, such as data volume, priority, quality of service and cost. This novel solution reduces user burden, facilitates service management, and contributes to the development of cognitive spaceflight missions, ultimately supporting NASA’s research into Cognitive Communications technology. The algorithm design consists of three major steps: modeling data, developing a Link Selection Algorithm (LSA) based on a grading system, and applying machine learning using linear regression. The LSA evaluates providers based on user-defined constraints, considering factors such as delivery time, cost, and quality of service. We define a suitability metric which allows our algorithm to make a recommendation to a user regarding which commercial service providers to select. The addition of Linear Regression predicts the future suitability value. Our main findings demonstrate that the resulting algorithm can autonomously manage connections between satellites and providers, maximizing communication channel efficiency. This research has significant implications, as it not only addresses a pressing issue in satellite communication management but also advances the field of cognitive spaceflight missions.

Linear regression↗

NASA NTRS2015

Integrated Medical Model Project - Overview and Summary of Historical Application

Introduction: The Integrated Medical Model (IMM) Project represents one aspect of NASA's Human Research Program (HRP) to quantitatively assess medical risks to astronauts for existing operational missions as well as missions associated with future exploration and commercial space flight ventures. The IMM takes a probabilistic approach to assessing the likelihood and specific outcomes of one hundred medical conditions within the envelope of accepted space flight standards of care over a selectable range of mission capabilities. A specially developed Integrated Medical Evidence Database (iMED) maintains evidence-based, organizational knowledge across a variety of data sources. Since becoming operational in 2011, version 3.0 of the IMM, the supporting iMED, and the expertise of the IMM project team have contributed to a wide range of decision and informational processes for the space medical and human research community. This presentation provides an overview of the IMM conceptual architecture and range of application through examples of actual space flight community questions posed to the IMM project. Methods: Figure 1 [see document] illustrates the IMM modeling system and scenario process. As illustrated, the IMM computational architecture is based on Probabilistic Risk Assessment techniques. Nineteen assumptions and limitations define the IMM application domain. Scenario definitions include crew medical attributes and mission specific details. The IMM forecasts probabilities of loss of crew life (LOCL), evacuation (EVAC), quality time lost during the mission, number of medical resources utilized and the number and type of medical events by combining scenario information with in-flight, analog, and terrestrial medical information stored in the iMED. In addition, the metrics provide the integrated information necessary to estimate optimized in-flight medical kit contents under constraints of mass and volume or acceptable level of mission risk. Results and Conclusions: Historically, IMM simulations support Science and Technology planning, Exploration mission planning, and ISS program operations by supplying simulation support, iMED data information, and subject matter expertise to Crew Health and Safety and the HRP. Upcoming release of IMM version 4.0 seeks to provide enhanced functionality to increase the quality of risk decisions made using the IMM through a more accurate representation of the real world system.

Myers, J.↗

NASA NTRS1995

Metrics and Benchmarks for Visualization

What is a "good" visualization? How can the quality of a visualization be measured? How can one tell whether one visualization is "better" than another? I claim that the true quality of a visualization can only be measured in the context of a particular purpose. The same image generated from the same data may be excellent for one purpose and abysmal for another. A good measure of visualization quality will correspond to the performance of users in accomplishing the intended purpose, so the "gold standard" is user testing. As a user of visualization software (or at least a consultant to such users) I don't expect visualization software to have been tested in this way for every possible use. In fact, scientific visualization (as distinct from more "production oriented" uses of visualization) will continually encounter new data, new questions and new purposes; user testing can never keep up. User need software they can trust, and advice on appropriate visualizations of particular purposes. Considering the following four processes, and their impact on visualization trustworthiness, reveals important work needed to create worthwhile metrics and benchmarks for visualization. These four processes are (1) complete system testing (user-in-loop), (2) software testing, (3) software design and (4) information dissemination. Additional information is contained in the original extended abstract.

Uselton, Samuel P.↗

NASA NTRS1997

A Validation of Object-Oriented Design Metrics as Quality Indicators

This paper presents the results of a study in which we empirically investigated the suits of object-oriented (00) design metrics introduced in another work. More specifically, our goal is to assess these metrics as predictors of fault-prone classes and, therefore, determine whether they can be used as early quality indicators. This study is complementary to the work described where the same suite of metrics had been used to assess frequencies of maintenance changes to classes. To perform our validation accurately, we collected data on the development of eight medium-sized information management systems based on identical requirements. All eight projects were developed using a sequential life cycle model, a well-known 00 analysis/design method and the C++ programming language. Based on empirical and quantitative analysis, the advantages and drawbacks of these 00 metrics are discussed. Several of Chidamber and Kamerer's 00 metrics appear to be useful to predict class fault-proneness during the early phases of the life-cycle. Also, on our data set, they are better predictors than 'traditional' code metrics, which can only be collected at a later phase of the software development processes.

Basili, Victor R.↗

NASA NTRS2004

The coverage of elliptical orbits using Ergodic theory

One of the key performance metrics for satellite constellations is the statistics of the visibility periods between the satellites and poins on the ground. Associated with this are other desirable communications statistis such as data through-put, link qualities, etc. Typically, the computation of coverage statistics requires the propagation of the trajectories. In this paper a new algorithm using differential geometry enables us to extend the Ergodic theory to eliptical orbits.

Lo, Martin W.↗

NASA NTRS2007

A Comparison of Two Balance Calibration Model Building Methods

Simulated strain-gage balance calibration data is used to compare the accuracy of two balance calibration model building methods for different noise environments and calibration experiment designs. The first building method obtains a math model for the analysis of balance calibration data after applying a candidate math model search algorithm to the calibration data set. The second building method uses stepwise regression analysis in order to construct a model for the analysis. Four balance calibration data sets were simulated in order to compare the accuracy of the two math model building methods. The simulated data sets were prepared using the traditional One Factor At a Time (OFAT) technique and the Modern Design of Experiments (MDOE) approach. Random and systematic errors were introduced in the simulated calibration data sets in order to study their influence on the math model building methods. Residuals of the fitted calibration responses and other statistical metrics were compared in order to evaluate the calibration models developed with different combinations of noise environment, experiment design, and model building method. Overall, predicted math models and residuals of both math model building methods show very good agreement. Significant differences in model quality were attributable to noise environment, experiment design, and their interaction. Generally, the addition of systematic error significantly degraded the quality of calibration models developed from OFAT data by either method, but MDOE experiment designs were more robust with respect to the introduction of a systematic component of the unexplained variance.

DeLoach, Richard↗

NASA NTRSDate not supplied

Use of Spectral Analysis of Singular Values as a Test Metric for IMMAT Trials

One of many challenges in the implementation of multiple exciter testing is establishing a reasonable set of test metrics to measure the quality of testing. This is especially true in the application of Impedance Matched Multi-Axis Testing; in that it is possible to have very large spectral density matrices that serve as reference criteria. While there exist plotting schemes to view a spectral density matrix, it is often necessary to break the overlay of reference and test results into subsections of the matrices to get sufficient resolution to interpret the data. In addition, as one attempts to control multiple locations on a structure, implementation of classical single degree-of freedom test tolerances across all channels and associated cross spectra is simply not feasible. Hence it is challenging to evaluate overall test quality. The use of spectral views of the dominant singular values from the singular value decomposition of the spectral density matrices and metrics based upon them is proposed for establishing a set of compact metrics for evaluating test quality. A laboratory experiment will be included to demonstrate this proposed technique.

Impedance Matched Multi-Axis Testing↗

NASA NTRSDate not supplied

Use of Spectral Analysis of Singular Values as a Test Metric for Impedance Matched Multi-Axis Test Trials

Vibration Testing↗

NASA NTRS2016

Kepler Planet Detection Metrics: Statistical Bootstrap Test

This document describes the data produced by the Statistical Bootstrap Test over the final three Threshold Crossing Event (TCE) deliveries to NExScI: SOC 9.1 (Q1Q16)1 (Tenenbaum et al. 2014), SOC 9.2 (Q1Q17) aka DR242 (Seader et al. 2015), and SOC 9.3 (Q1Q17) aka DR253 (Twicken et al. 2016). The last few years have seen significant improvements in the SOC science data processing pipeline, leading to higher quality light curves and more sensitive transit searches. The statistical bootstrap analysis results presented here and the numerical results archived at NASAs Exoplanet Science Institute (NExScI) bear witness to these software improvements. This document attempts to introduce and describe the main features and differences between these three data sets as a consequence of the software changes.

Bootstrap↗

NASA NTRS2011

The 2010 Desert Rats Science Operations Test: Outcomes and Lessons Learned

The Desert RATS 2010 Team tested a variety of science operations management techniques, applying experience gained during the manned Apollo missions and the robotic Mars missions. This test assessed integrated science operations management of human planetary exploration using real-time, tactical science operations to oversee daily crew science activities, and a night shift strategic science operations team to conduct strategic level assessment of science data and daily traverse results. In addition, an attempt was made to collect numerical metric data on the outcome of the science operations to assist test evaluation. The two most important outcomes were 1) the production of significant (almost overwhelming) volume of data produced during daily traverse operations with two rovers, advanced imaging systems and well trained, scientifically proficient crew-members, and 2) the degree to which the tactical team s interaction with the surface crew enhanced science return. This interaction depended on continuous real-time voice and data communications, and the quality of science return from any human planetary exploration mission will be based strongly on the aggregate interaction between a well trained surface crew and a dedicated science operations support team using voice and imaging data from a planet s surface. In addition, the scientific insight developed by both the science operations team and the crews could not be measurable by simple numerical quantities, and its value will be missed by a purely metric-based evaluation of test outcome. In particular, failure to recognize the critical importance of this qualitative type interaction may result in mission architecture choices that will reduce science return.

Eppler, D. B.↗

NASA NTRS1986

Validation of a new flying quality criterion for the landing task

A strong correlation has been found to exist between flight path angle peak overshoot and pilot ratings for the landing task. The use of flightpath overshoot as a flying quality metric for landing is validated by correlation with four different in-flight simulation programs and a ground simulation study. Configurations tested were primarily medium-weight generic transports. As a result of good correlation with this extensive data base, criterion boundaries are proposed for landing based on the flight path peak overshoot metric.

Berry, D. T.↗

NASA NTRS1986

Validation of a new flying quality criterion for the landing task

A strong correlation has been found to exist between flight path angle peak overshoot and pilot ratings for the landing task. The use of flight path overshoot as a flying quality metric for landing is validated by correlation with four different in-flight simulation programs and a ground simulation study. Configurations tested were primarily medium-weight generic transports. As a result of good correlation with this extensive data base, criterion boundaries are proposed for landing based on the flight path peak overshoot metric.

Berry, D. T.↗

NASA NTRSDate not supplied

In-Depth Analysis of Subject Functional Performance and Subjective Feedback Data for Evaluating Argos Simulation Quality

Mechanical offloading systems such as the Active Response Gravity Offload System (ARGOS) at NASA Johnson Space Center (JSC) are used to simulate various partial gravity environments. ARGOS offloads pressurized suited subjects through a gimbaling pivot point system attached to a volumetric Portable Life Support System (PLSS). The pivot point can be configured to multiple locations with respect to the human-suit system center of gravity (CG). Previous work has tested and documented the interaction between different pivot point locations and measured CG. Observations indicated that small changes in pivot point location substantially affect the stability and functional performance (simulation quality) of a subject. This led to the development of a standardized gimbal assessment protocol to assess the functional performance of a pressurized suited subject as a function of the pivot point location, using the data obtained from a variety of planetary suit mobility tasks. As a result, an expansive repository of CG-related task metrics was generated from numerous ARGOS spacesuit test events, for different gimbal configurations and different test subjects. Based on the evaluation metrics and observed trends, several pivot point locations were iteratively identified to determine an “appropriate/optimal” configuration. This study thus describes the in-depth analysis of the CG evaluation metrics to better quantify trends and determine if specific factors are strongly associated with appropriate/optimal configurations. Quantitative and qualitative variables relating to subject task performance, subjective feedback, and anthropometry will be evaluated via probabilistic methods. The results from this study are expected to improve our understanding of optimal ARGOS gimbal settings, which will better inform the gimbal configuration identification process and improve simulation quality for extravehicular activity (EVA) training.

Joseph Yao↗

NASA NTRS2017

Kepler Planet Detection Metrics: Per-Target Detection Contours for Data Release 25

A necessary input to planet occurrence calculations is an accurate model for the pipeline completeness (Burke et al., 2015). This document describes the use of the Kepler planet occurrence rate products in order to calculate a per-target detection contour for the measured Data Release 25 (DR25) pipeline performance. A per-target detection contour measures for a given combination of orbital period, Porb, and planet radius, Rp, what fraction of transit signals are recoverable by the Kepler pipeline (Twicken et al., 2016; Jenkins et al., 2017). The steps for calculating a detection contour follow the procedure outlined in Burke et al. (2015), but have been updated to provide improved accuracy enabled by the substantially larger database of transit injection and recovery tests that were performed on the final version (i.e., SOC 9.3) of the Kepler pipeline (Christiansen, 2017; Burke Catanzarite, 2017a). In the following sections, we describe the main inputs to the per-target detection contour and provide a worked example of the python software released with this document (Kepler Planet Occurrence Rate Tools KeplerPORTs)1 that illustrates the generation of a detection contour in practice. As background material for this document and its nomenclature, we recommend the reader be familiar with the previous method of calculating a detection contour (Section 2 of Burke et al.,2015), input parameters relevant for describing the data quantity and quality of Kepler targets (Burke Catanzarite, 2017b), and the extensive new transit injection and recovery tests of the Kepler pipeline (Christiansen et al., 2016; Burke Catanzarite, 2017a; Christiansen, 2017).

Planet Detection Metrics↗

NASA NTRS2021

Florida Transportation and Infrastructure: Monitoring Water Quality Along Southern Florida Seaports to Assess Impact on Coral Reef Tracts from Harbor Deepening Projects

The U.S. Army Corps of Engineers (USACE) and National Oceanic and Atmospheric Administration (NOAA) National Marine Fisheries Service (NMFS) will be supervising a harbor deepening project in Port Everglades, Florida. The project raises concerns about potential impacts on the nearby Florida reef tract through increased turbidity and sediment from the dredging. To better understand these potential impacts, the NASA DEVELOP team created an interactive Google Earth Engine tool to help establish a historical baseline of water quality parameters and assist monitoring these parameters more frequently than traditional sampling. This Seaport & Harbor Area Resource Quality (SHARQ) tool incorporates remotely sensed data from Sentinel-2 Multispectral Instrument, Landsat 5 Thematic Mapper, Landsat 7 Enhanced Thematic Mapper+, Landsat 8 Operational Land Imager, and Aqua Moderate Resolution Imaging Spectroradiometer. It allows users to view true color images and calculate water quality parameters, like turbidity and chlorophyll-a, for any given study area and time period from 1984 onward. The SHARQ tool also generate time series charts, allowing users to interpret changes in water quality over a given time range. The accuracy of the remotely sensed water quality parameter algorithms was determined using in situ data to calculate percent difference and root mean square error values (RMSE), which ranged from 0.32 to 0.58 error between sites. Using the SHARQ tool’s time series analysis feature, a baseline average turbidity metric of ~6.8 FNUs provides a historical baseline average for turbidity between September 2000 and 2020 and can assist in future decision-making for determining thresholds for turbidity.

Benjamin Rocha↗