SEARCH · Search NASA

Results for “data quality metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2013

Hidden Connections between Regression Models of Strain-Gage Balance Calibration Data

Hidden connections between regression models of wind tunnel strain-gage balance calibration data are investigated. These connections become visible whenever balance calibration data is supplied in its design format and both the Iterative and Non-Iterative Method are used to process the data. First, it is shown how the regression coefficients of the fitted balance loads of a force balance can be approximated by using the corresponding regression coefficients of the fitted strain-gage outputs. Then, data from the manual calibration of the Ames MK40 six-component force balance is chosen to illustrate how estimates of the regression coefficients of the fitted balance loads can be obtained from the regression coefficients of the fitted strain-gage outputs. The study illustrates that load predictions obtained by applying the Iterative or the Non-Iterative Method originate from two related regression solutions of the balance calibration data as long as balance loads are given in the design format of the balance, gage outputs behave highly linear, strict statistical quality metrics are used to assess regression models of the data, and regression model term combinations of the fitted loads and gage outputs can be obtained by a simple variable exchange.

Ulbrich, Norbert↗

NASA NTRSDate not supplied

A Framework for Evaluating Climate Model Performance Metrics

The CMIP5 archive contains future climate projections from over 50 models provided by dozens of modeling centers from around the world. Individual model projections, however, are subject to biases created by structural model uncertainties. As a result, ensemble averaging of multiple models is often used to add value to model projections: consensus projections have been shown to consistently outperform individual models. Previous reports for the IPCC establish climate change projections based on an equal-weighted average of all model projections. However, certain models reproduce climate processes better than other models. Should models be weighted based on performance? Unequal ensemble averages have previously been constructed using a variety of mean state metrics. What metrics are most relevant for constraining future climate projections? This project develops a framework for systematically testing metrics in models to identify optimal metrics for unequal weighting multi-model ensembles. A unique aspect of this project is the construction and testing of climate process-based model evaluation metrics. A climate process-based metric is defined as a metric based on the relationship between two physically related climate variables?e.g., outgoing longwave radiation and surface temperature. Metrics are constructed using high-quality Earth radiation budget data from NASA's Clouds and Earth's Radiant Energy System (CERES) instrument and surface temperature data sets. It is found that regional values of tested quantities can vary significantly when comparing weighted and unweighted model ensembles. For example, one tested metric weights the ensemble by how well models reproduce the time-series probability distribution of the cloud forcing component of reflected shortwave radiation. The weighted ensemble for this metric indicates lower simulated precipitation (up to .7 mm/day) in tropical regions than the unweighted ensemble: since CMIP5 models have been shown to overproduce precipitation, this result could indicate that the metric is effective in identifying models which simulate more realistic precipitation. Ultimately, the goal of the framework is to identify performance metrics for advising better methods for ensemble averaging models and create better climate predictions.

Noel C Baker↗

NASA NTRS2006

Convective Weather Forecast Quality Metrics for Air Traffic Management Decision-Making

Since numerical weather prediction models are unable to accurately forecast the severity and the location of the storm cells several hours into the future when compared with observation data, there has been a growing interest in probabilistic description of convective weather. The classical approach for generating uncertainty bounds consists of integrating the state equations and covariance propagation equations forward in time. This step is readily recognized as the process update step of the Kalman Filter algorithm. The second well known method, known as the Monte Carlo method, consists of generating output samples by driving the forecast algorithm with input samples selected from distributions. The statistical properties of the distributions of the output samples are then used for defining the uncertainty bounds of the output variables. This method is computationally expensive for a complex model compared to the covariance propagation method. The main advantage of the Monte Carlo method is that a complex non-linear model can be easily handled. Recently, a few different methods for probabilistic forecasting have appeared in the literature. A method for computing probability of convection in a region using forecast data is described in Ref. 5. Probability at a grid location is computed as the fraction of grid points, within a box of specified dimensions around the grid location, with forecast convection precipitation exceeding a specified threshold. The main limitation of this method is that the results are dependent on the chosen dimensions of the box. The examples presented Ref. 5 show that this process is equivalent to low-pass filtering of the forecast data with a finite support spatial filter. References 6 and 7 describe the technique for computing percentage coverage within a 92 x 92 square-kilometer box and assigning the value to the center 4 x 4 square-kilometer box. This technique is same as that described in Ref. 5. Characterizing the forecast, following the process described in Refs. 5 through 7, in terms of percentage coverage or confidence level is notionally sound compared to characterizing in terms of probabilities because the probability of the forecast being correct can only be determined using actual observations. References 5 through 7 only use the forecast data and not the observations. The method for computing the probability of detection, false alarm ratio and several forecast quality metrics (Skill Scores) using both the forecast and observation data are given in Ref. 2. This paper extends the statistical verification method in Ref. 2 to determine co-occurrence probabilities. The method consists of computing the probability that a severe weather cell (grid location) is detected in the observation data in the neighborhood of the severe weather cell in the forecast data. Probabilities of occurrence at the grid location and in its neighborhood with higher severity, and with lower severity in the observation data compared to that in the forecast data are examined. The method proposed in Refs. 5 through 7 is used for computing the probability that a certain number of cells in the neighborhood of severe weather cells in the forecast data are seen as severe weather cells in the observation data. Finally, the probability of existence of gaps in the observation data in the neighborhood of severe weather cells in forecast data is computed. Gaps are defined as openings between severe weather cells through which an aircraft can safely fly to its intended destination. The rest of the paper is organized as follows. Section II summarizes the statistical verification method described in Ref. 2. The extension of this method for computing the co-occurrence probabilities in discussed in Section HI. Numerical examples using NCWF forecast data and NCWD observation data are presented in Section III to elucidate the characteristics of the co-occurrence probabilities. This section also discusses the procedure for computing throbabilities that the severity of convection in the observation data will be higher or lower in the neighborhood of grid locations compared to that indicated at the grid locations in the forecast data. The probability of coverage of neighborhood grid cells is also described via examples in this section. Section IV discusses the gap detection algorithm and presents a numerical example to illustrate the method. The locations of the detected gaps in the observation data are used along with the locations of convective weather cells in the forecast data to determine the probability of existence of gaps in the neighborhood of these cells. Finally, the paper is concluded in Section V.

Chatterji, Gano B.↗

NASA NTRS2009

Analysis of Sting Balance Calibration Data Using Optimized Regression Models

Calibration data of a wind tunnel sting balance was processed using a search algorithm that identifies an optimized regression model for the data analysis. The selected sting balance had two moment gages that were mounted forward and aft of the balance moment center. The difference and the sum of the two gage outputs were fitted in the least squares sense using the normal force and the pitching moment at the balance moment center as independent variables. The regression model search algorithm predicted that the difference of the gage outputs should be modeled using the intercept and the normal force. The sum of the two gage outputs, on the other hand, should be modeled using the intercept, the pitching moment, and the square of the pitching moment. Equations of the deflection of a cantilever beam are used to show that the search algorithm s two recommended math models can also be obtained after performing a rigorous theoretical analysis of the deflection of the sting balance under load. The analysis of the sting balance calibration data set is a rare example of a situation when regression models of balance calibration data can directly be derived from first principles of physics and engineering. In addition, it is interesting to see that the search algorithm recommended the same regression models for the data analysis using only a set of statistical quality metrics.

Ulbrich, Norbert↗

NASA NTRS1993

Software quality: Process or people

This paper will present data related to software development processes and personnel involvement from the perspective of software quality assurance. We examine eight years of data collected from six projects. Data collected varied by project but usually included defect and fault density with limited use of code metrics, schedule adherence, and budget growth information. The data are a blend of AFSCP 800-14 and suggested productivity measures in Software Metrics: A Practioner's Guide to Improved Product Development. A software quality assurance database tool, SQUID, was used to store and tabulate the data.

Palmer, Regina↗

NASA NTRS2005

A New Approach to Image Fusion Based on Cokriging

We consider the image fusion problem involving remotely sensed data. We introduce cokriging as a method to perform fusion. We investigate the advantages of fusing Hyperion with ALI. The evaluation is performed by comparing the classification of the fused data with that of input images and by calculating well-chosen quantitative fusion quality metrics. We consider the Invasive Species Forecasting System (ISFS) project as our fusion application. The fusion of ALI with Hyperion data is studies using PCA and wavelet-based fusion. We then propose utilizing a geostatistical based interpolation method called cokriging as a new approach for image fusion.

Memarsadeghi, Nargess↗

NASA NTRS1999

Validation of Surface Skin Temperature and Moisture Profiles Using Satellite Data

New validation techniques and metrics using satellite data have been developed to evaluate the quality of model-based estimates of surface skin temperature (Tg) and moisture profiles (q). The satellite data consist of clear sky outgoing long-wave radiation (CLR), broadband radiances from 8 to 12 mu (RadWn), brightness temperature centered around 10.8 mu (Tbb), and total precipitable water (TPW) from microwave radiometry. We show that CLR can be used to diagnose Tg. Furthermore, by using a combination of CLR and RadWn from CERES-TRMM measurements and TPW from SSM/I, we are able to identify errors in the moisture profile. Finally, three-hourly Tbb from the International Satellite Cloud Climatology Project can be used to evaluate the amplitude and diurnal variation of Tg. For purpose of illustration, Tg and q are evaluated from runs with an early version of the Goddard Earth Observing System Data Assimilation System (GEOS-2). It is found that, in general, Tg is too cold in the winter hemisphere and q is too wet in the upper atmosphere. In order to address these deficiencies, several improvements have been implemented into GEOS-2, including a Land-Surface-Model, a Moist Turbulence Scheme, and the assimilation of new TOVS retrievals. Preliminary results indicate positive impacts from each of these implementations.

Wu, Man Li C.↗

NASA NTRS2015

An Empirical Jet-Surface Interaction Noise Model with Temperature and Nozzle Aspect Ratio Effects

An empirical model for jet-surface interaction (JSI) noise produced by a round jet near a flat plate is described and the resulting model evaluated. The model covers unheated and hot jet conditions (1 less than or equal to jet total temperature ratio less than or equal to 2.7) in the subsonic range (0.5 less than or equal to M(sub a) less than or equal to 0.9), surface lengths 0.6 less than or equal to (axial distance from jet exit to surface trailing edge (inches)/nozzle exit diameter) less than or equal to 10, and surface standoff distances (0 less than or equal to (radial distance from jet lipline to surface (inches)/axial distance from jet exit to surface trailing edge (inches)) less than or equal to 1) using only second-order polynomials to provide predictable behavior. The JSI noise model is combined with an existing jet mixing noise model to produce exhaust noise predictions. Fit quality metrics and comparisons to between the predicted and experimental data indicate that the model is suitable for many system level studies. A first-order correction to the JSI source model that accounts for the effect of nozzle aspect ratio is also explored. This correction is based on changes to the potential core length and frequency scaling associated with rectangular nozzles up to 8:1 aspect ratio. However, more work is needed to refine these findings into a formal model.

Noise↗

NASA NTRS2010

Implications of Microwave Holography Using Minimum Required Frequency Samples for Weakly- and Strongly-Scattering Indications

Microwave imaging techniques, an integral component of nondestructive testing and evaluation (NDTE), have received significant attention in the past decade. These techniques have included the implementation of synthetic aperture focusing (SAF) algorithms for obtaining high spatial resolution images. The next important step in these developments is the implementation of 3-D holographic imaging algorithms. These are well-known wideband imaging technique requiring a swept-frequency (i.e., wideband), which unlike SAF that is a single frequency technique, are not easily performed on a real-time basis. This is due to the fact that a significant number of data points (in the frequency domain) must be obtained within the frequency band of interest. This not only makes for a complex imaging system design, it also significantly increases the image-production time. Consequently in an attempt to reduce the measurement time and system complexity, an investigation was conducted to determine the minimum required number of frequency samples needed to image a specific object while preserving a desired maximum measurement range and range resolution. To this end the 3-D holographic algorithm was modified to use properlyinterpolated frequency data. Measurements of the complex reflection coefficient for several samples were conducted using a swept-frequency approach. Subsequently, holographical images were generated using data containing a relatively large number of frequency samples and were compared with images generated by the reduced data set data. Quantitative metrics such as average, contrast, and signal-to-noise ratio were used to evaluate the quality of images generated using reduced data sets. Furthermore, this approach was applied to both weakly- and strongly-scattering indications. This paper presents the methods used and the results of this investigation.

Fallahpour, M.↗

NASA NTRS2006

Towards Designing an Integrated Earth Observation System for the Provision of Solar Energy Resource and Assessment

The GEOSS strategic plan specifically targets the area of improved energy resource management due to the importance of these to the economic and social viability of every nation of the world. With the world s increasing demand for energy resources, the need for new alternative energy resources grows. This paper overviews a new initiative within the International Energy Agency that addresses needs to better manage and develop solar energy resources worldwide. The goal is to provide the solar energy industry, the electricity sector, governments, and renewable energy organizations and institutions with the most suitable and accurate information of the solar radiation resources at the Earth's surface in easily-accessible formats and understandable quality metrics. The scope of solar resource assessment information includes historic data sets and currently derived data products using satellite imagery and other means. Thus, this new task will address the needs of the solar energy sector while at the same time will serve as a model that satisfies GEOSS objectives and goals.

Stackouse, Paul W., Jr.↗

NASA NTRS2010

Robust Mosaicking of Stereo Digital Elevation Models from the Ames Stereo Pipeline

Robust estimation method is proposed to combine multiple observations and create consistent, accurate, dense Digital Elevation Models (DEMs) from lunar orbital imagery. The NASA Ames Intelligent Robotics Group (IRG) aims to produce higher-quality terrain reconstructions of the Moon from Apollo Metric Camera (AMC) data than is currently possible. In particular, IRG makes use of a stereo vision process, the Ames Stereo Pipeline (ASP), to automatically generate DEMs from consecutive AMC image pairs. However, the DEMs currently produced by the ASP often contain errors and inconsistencies due to image noise, shadows, etc. The proposed method addresses this problem by making use of multiple observations and by considering their goodness of fit to improve both the accuracy and robustness of the estimate. The stepwise regression method is applied to estimate the relaxed weight of each observation.

Kim, Tae Min↗

NASA NTRS2011

Orthographic Stereo Correlator on the Terrain Model for Apollo Metric Images

A stereo correlation method on the object domain is proposed to generate the accurate and dense Digital Elevation Models (DEMs) from lunar orbital imagery. The NASA Ames Intelligent Robotics Group (IRG) aims to produce high-quality terrain reconstructions of the Moon from Apollo Metric Camera (AMC) data. In particular, IRG makes use of a stereo vision process, the Ames Stereo Pipeline (ASP), to automatically generate DEMs from consecutive AMC image pairs. Given camera parameters of an image pair from bundle adjustment in ASP, a correlation window is defined on the terrain with the predefined surface normal of a post rather than image domain. The squared error of back-projected images on the local terrain is minimized with respect to the post elevation. This single dimensional optimization is solved efficiently and improves the accuracy of the elevation estimate.

Terrain Model↗

NASA NTRS2019

Learning from GES DISC's MLS and OMI Data Users: Metrics Matter

It has been over 15 years since Aura research satellite launched in 2004 to observe the Earth's ozone layer, air quality, and climate from four different instruments - the High Resolution Dynamics Limb Sounder (HIRDLS), the Microwave Limb Sounder (MLS), the Ozone Monitoring Instrument (OMI), and the Tropospheric Emission Spectrometer (TES). Observations from the Aura mission have established a concrete understanding of the changing chemistry of our atmosphere.The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is the official archive and distribution center for the HIRDLS, MLS, and OMI instruments. This presentation will report metrics of data usage and services on these instruments. Key to GES DISC's mission to provide better data support is gaining a better understanding of our users' needs and behaviors as they discover, access and utilize these data. We will summarize the users' needs from these instruments based on user inquiry information collected over the Aura mission lifetime and present findings from this ensemble metrics.

Metrics↗

NASA NTRS1990

Analyzing the flared landing task with pitch-rate flight control systems

A closed-loop handling qualities methodology is applied to an analysis of the flared landing task with pitch-rate flight control systems. A model of pilot behavior throughout approach and flare is developed which postulates the manner in which the pilot may move from pitch attitude to flight path angle control. Twenty-five configurations flight tested on the NC-131H Total In-flight Simulator aircraft are analyzed using a structural pilot model ad a handling qualities methodology previously reported in the literature. Closed-loop simulation of the simplified landing task is undertaken using the structural model. The pilot ratings from flight test extended the data base supporting the utility of a model-based handling qualities metric. A handling qualities sensitivity function is introduced which may have potential as a design tool.

Hess, Ronald A.↗

NASA NTRS2004

Strain-Gage Loads Calibration Parametric Study

This paper documents a parametric study of various aircraft wing-load test features that affect the quality of the resultant derived shear, bending-moment, and torque strain-gage load equations. The effect of the following on derived strain-gage equation accuracy are compared: single-point loading compared with distributed loading, variation in applied test load magnitude, number of applied load cases, and wing-box-only compared with control-surface loading. The subject of this study is an extensive wing-load calibration test of the Active Aeroelastic Wing F/A-18 airplane. Selected subsets of the available test data were used to derive load equations using the linear regression method. Results show the benefit of distributed loading and the diminishing-return benefits of test load magnitudes and number of load cases. The use of independent check cases as a quality metric for the derived load equations is shown to overcome blind extrapolating beyond the load data used to derive the load equations.

Lokos, William A.↗

NASA NTRS1990

Getting started on metrics - Jet Propulsion Laboratory productivity and quality

A review is presented to describe the effort and difficulties of reconstructing fifteen years of JPL software history. In 1987 the collection and analysis of project data were started with the objective of creating laboratory-wide measures of quality and productivity for software development. As a result of this two-year Software Product Assurance metrics study, a rough measurement foundation for software productivity and software quality, and an order-of-magnitude quantitative baseline for software systems and subsystems are now available.

Bush, M. W.↗

NASA NTRS2021

The Salinity Pilot-Mission Exploitation Platform (Pi-MEP): A Hub for Validation and Exploitation of Satellite Sea Surface Salinity Data

The Pilot-Mission Exploitation Platform (Pi-MEP) for salinity is an ESA initiative originally meant to support and widen the uptake of Soil Moisture and Ocean Salinity (SMOS) mission data over the ocean. Starting in 2017, the project aims at setting up a computational web-based platform focusing on satellite sea surface salinity data, supporting studies on enhanced validation and scientific process over the ocean. It has been designed in close collaboration with a dedicated science advisory group in order to achieve three main objectives: gathering all the data required to exploit satellite sea surface salinity data, systematically producing a wide range of metrics for comparing and monitoring sea surface salinity products’ quality, and providing user-friendly tools to explore, visualize and exploit both the collected products and the results of the automated analyses. The Salinity Pi-MEP is becoming a reference hub for the validation of satellite sea surface salinity missions by providing valuable information on satellite products (SMOS, Aquarius, SMAP), an extensive in situ database (e.g., Argo, thermosalinographs, moorings, drifters) and additional thematic datasets (precipitation, evaporation, currents, sea level anomalies, sea surface temperature, etc.). Co-localized databases between satellite products and in situ datasets are systematically generated together with validation analysis reports for 30 predefined regions. The data and reports are made fully accessible through the web interface of the platform. The datasets, validation metrics and tools (automatic, user-driven) of the platform are described in detail in this paper. Several dedicated scientific case studies involving satellite SSS data are also systematically monitored by the platform, including major river plumes, mesoscale signatures in boundary currents, high latitudes, semi-enclosed seas, and the high-precipitation region of the eastern tropical Pacific. Since 2019, a partnership in the Salinity Pi-MEP project has been agreed between ESA and NASA to enlarge focus to encompass the entire set of satellite salinity sensors. The two agencies are now working together to widen the platform features on several technical aspects, such as triple-collocation software implementation, additional match-up collocation criteria and sustained exploitation of data from the SPURS campaigns

ocean↗

NASA NTRS2006

A Predictive Approach to Eliminating Errors in Software Code

NASA s Metrics Data Program Data Repository is a database that stores problem, product, and metrics data. The primary goal of this data repository is to provide project data to the software community. In doing so, the Metrics Data Program collects artifacts from a large NASA dataset, generates metrics on the artifacts, and then generates reports that are made available to the public at no cost. The data that are made available to general users have been sanitized and authorized for publication through the Metrics Data Program Web site by officials representing the projects from which the data originated. The data repository is operated by NASA s Independent Verification and Validation (IV&V) Facility, which is located in Fairmont, West Virginia, a high-tech hub for emerging innovation in the Mountain State. The IV&V Facility was founded in 1993, under the NASA Office of Safety and Mission Assurance, as a direct result of recommendations made by the National Research Council and the Report of the Presidential Commission on the Space Shuttle Challenger Accident. Today, under the direction of Goddard Space Flight Center, the IV&V Facility continues its mission to provide the highest achievable levels of safety and cost-effectiveness for mission-critical software. By extending its data to public users, the facility has helped improve the safety, reliability, and quality of complex software systems throughout private industry and other government agencies. Integrated Software Metrics, Inc., is one of the organizations that has benefited from studying the metrics data. As a result, the company has evolved into a leading developer of innovative software-error prediction tools that help organizations deliver better software, on time and on budget.

Source record↗