Search NASA⌕ Search

SEARCH · Search NASA

Results for “Error Metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2000

Accelerating Time-Varying Hardware Volume Rendering Using TSP Trees and Color-Based Error Metrics

This paper describes a new hardware volume rendering algorithm for time-varying data. The algorithm uses the Time-Space Partitioning (TSP) tree data structure to identify regions within the data that have spatial or temporal coherence. By using this coherence, the rendering algorithm can improve performance when the volume data is larger than the texture memory capacity by decreasing the amount of textures required. This coherence can also allow improved speed by appropriately rendering flat-shaded polygons instead of textured polygons, and by not rendering transparent regions. To reduce the polygonization overhead caused by the use of the hierarchical data structure, we introduce an optimization method using polygon templates. The paper also introduces new color-based error metrics, which more accurately identify coherent regions compared to the earlier scalar-based metrics. By showing experimental results from runs using different data sets and error metrics, we demonstrate that the new methods give substantial improvements in volume rendering performance.

Ellsworth, David↗
NASA NTRS2016

Performance Metrics, Error Modeling, and Uncertainty Quantification

A common set of statistical metrics has been used to summarize the performance of models or measurements-­ the most widely used ones being bias, mean square error, and linear correlation coefficient. They assume linear, additive, Gaussian errors, and they are interdependent, incomplete, and incapable of directly quantifying un­certainty. The authors demonstrate that these metrics can be directly derived from the parameters of the simple linear error model. Since a correct error model captures the full error information, it is argued that the specification of a parametric error model should be an alternative to the metrics-based approach. The error-modeling meth­odology is applicable to both linear and nonlinear errors, while the metrics are only meaningful for linear errors. In addition, the error model expresses the error structure more naturally, and directly quantifies uncertainty. This argument is further explained by highlighting the intrinsic connections between the performance metrics, the error model, and the joint distribution between the data and the reference.

Quantification↗
NASA NTRS1993

Radio metric errors due to mismatch and offset between a DSN antenna beam and the beam of a troposphere calibration instrument

Two components of the error of a troposphere calibration measurement were quantified by theoretical calculations. The first component is a beam mismatch error, which occurs when the calibration instrument senses a conical volume different from the cylindrical volume sampled by a Deep Space Network (DSN) antenna. The second component is a beam offset error, which occurs if the calibration instrument is not mounted on the axis of the DSN antenna. These two error sources were calculated for both delay (e.g., VLBI) and delay rate (e.g., Doppler) measurements. The beam mismatch error for both delay and delay rate drops rapidly as the beamwidth of the troposphere calibration instrument (e.g., a water vapor radiometer or an infrared Fourier transform spectrometer) is reduced. At a 10-deg elevation angle, the instantaneous beam mismatch error is 1.0 mm for a 6-deg beamwidth and 0.09 mm for a 0.5-deg beam (these are the full angular widths of a circular beam with uniform gain out to a sharp cutoff). Time averaging for 60-100 sec will reduce these errors by factors of 1.2-2.2. At a 20-deg elevation angle, the lower limit for current Doppler observations, the beam-mismatch delay rate error is an Allan standard deviation over 100 sec of 1.1 x 10(exp -14) with a 4-deg beam and 1.3 x 10(exp -l5) for a 0.5-deg beam. A 50-m beam offset would result in a fairly modest (compared to other expected error sources) delay error (less than or equal to 0.3 mm for 60-sec integrations at any elevation angle is greater than or equal to 6 deg). However, the same offset would cause a large error in delay rate measurements (e.g., an Allan standard deviation of 1.2 x 10(exp -14) over 100 sec at a 20-deg elevation angle), which would dominate over other known error sources if the beamwidth is 2 deg or smaller. An on-axis location is essential for accurate troposphere calibration of delay rate measurements. A half-power beamwidth (for a beam with a tapered gain profile) of 1.2 deg or smaller is desired for calibration of all types of radio metrics. A water-vapor radiometer calibration beam of this size with very low sidelobes would require a clear aperture antenna with a diameter of at least 1.5 m if the primary water vapor sensing channel were in the 20-22 GHz range.

R P Linfield↗
NASA NTRS2001

Response Surface Modeling Using Multivariate Orthogonal Functions

A nonlinear modeling technique was used to characterize response surfaces for non-dimensional longitudinal aerodynamic force and moment coefficients, based on wind tunnel data from a commercial jet transport model. Data were collected using two experimental procedures - one based on modem design of experiments (MDOE), and one using a classical one factor at a time (OFAT) approach. The nonlinear modeling technique used multivariate orthogonal functions generated from the independent variable data as modeling functions in a least squares context to characterize the response surfaces. Model terms were selected automatically using a prediction error metric. Prediction error bounds computed from the modeling data alone were found to be- a good measure of actual prediction error for prediction points within the inference space. Root-mean-square model fit error and prediction error were less than 4 percent of the mean response value in all cases. Efficacy and prediction performance of the response surface models identified from both MDOE and OFAT experiments were investigated.

Morelli, Eugene A.↗
NASA NTRSDate not supplied

Commercial Supersonic Fan Noise Modeling

This presentation describes the development of an empirical noise model for a two-stage fan designed for a supersonic commercial aircraft. The existing fan noise models are reviewed and compared with noise data from a wind tunnel experiment where a model scale two-stage fan was tested. An error metric was developed to quantify how well the measurements and empirical model match. This error metric was then used to determine new coefficients and parameters for a version of the fan model that is a good match for the experimental data.

supersonics, acoustics, turbofan↗
NASA NTRSDate not supplied

NESC GN&C TDT Workshop on 2D Image Motion Optical Transfer Functions, Pointing Performance Analysis, and Requirements

What You Will Learn: The focus is on payload imaging performance due to pointing motion. Some historical background on pointing performance analysis is given. The Optical Transfer Function (OTF) and Modulation Transfer Function (MTF) are defined. The imaging performance due to pointing motion is measured by image motion optical transfer functions (IM OTF). IM OTFs are defined for displacement, smear, and jitter motions, which are all rigorously defined. Deterministic and Statistical IM OTFs are briefly derived and graphically illustrated and compared. The IM OTFs are parameterized by pointing error metrics(PEM), which are means and covariances of displacement, smear, and jitter. Emphasis is on procedures and algorithms to evaluate the image motion optical transfer functions and pointing error metrics. Three procedures are covered, which depend on whether the pointing error data is from time-domain simulation, frequency-domain analysis, or stochastic modeling. A method to evaluate the relative contribution of disturbance sources and to identify the most significant contributors is presented. The presentation includes pertinent discussion of flexible structures and control-structure interaction. No single book can adequately cover this subject, so a book is not required for the course. A list of selected articles, reports, documents, and books is provided for reference and further study. Mathis kept to the minimum necessary to convey principles; lengthy derivations are left to the reference material. Graphics are used to illustrate concepts. As with any such learning endeavor, the knowledge gained will be retained and strengthened through actual practice.c©2019–2021 Mark E. Pittelkau— 5

NASA Engineering and Safety Center (NESC)↗
NASA NTRS1999

"Tools For Analysis and Visualization of Large Time- Varying CFD Data Sets"

During the four years of this grant (including the one year extension), we have explored many aspects of the visualization of large CFD (Computational Fluid Dynamics) datasets. These have included new direct volume rendering approaches, hierarchical methods, volume decimation, error metrics, parallelization, hardware texture mapping, and methods for analyzing and comparing images. First, we implemented an extremely general direct volume rendering approach that can be used to render rectilinear, curvilinear, or tetrahedral grids, including overlapping multiple zone grids, and time-varying grids. Next, we developed techniques for associating the sample data with a k-d tree, a simple hierarchial data model to approximate samples in the regions covered by each node of the tree, and an error metric for the accuracy of the model. We also explored a new method for determining the accuracy of approximate models based on the light field method described at ACM SIGGRAPH (Association for Computing Machinery Special Interest Group on Computer Graphics) '96. In our initial implementation, we automatically image the volume from 32 approximately evenly distributed positions on the surface of an enclosing tessellated sphere. We then calculate differences between these images under different conditions of volume approximation or decimation.

Wilhelms, Jane↗
NASA NTRS2012

Evaluation of Two Crew Module Boilerplate Tests Using Newly Developed Calibration Metrics

The paper discusses a application of multi-dimensional calibration metrics to evaluate pressure data from water drop tests of the Max Launch Abort System (MLAS) crew module boilerplate. Specifically, three metrics are discussed: 1) a metric to assess the probability of enveloping the measured data with the model, 2) a multi-dimensional orthogonality metric to assess model adequacy between test and analysis, and 3) a prediction error metric to conduct sensor placement to minimize pressure prediction errors. Data from similar (nearly repeated) capsule drop tests shows significant variability in the measured pressure responses. When compared to expected variability using model predictions, it is demonstrated that the measured variability cannot be explained by the model under the current uncertainty assumptions.

Horta, Lucas G.↗
NASA NTRS2020

Ground Validation of TRMM 3B43 V7 Precipitation Estimates Over Colombia. Part I: Monthly and Seasonal Timescales

In this study, we validate precipitation estimates remotely sensed by the Tropical Rainfall Measuring Mission (TRMM) at monthly and seasonal timescales, during the period 1998–2015, by calculating and analyzing diverse error metrics between the 3B43 V7 product and in situ measurements from 1,180 rain gauges over Colombia, of which at least 987 are fully independent of TRMM. We explore the existence of spatiotemporal patterns to assess the performance of 3B43 V7 over the five major natural regions of Colombia: Caribbean, Pacific, Andes, Orinoco and Amazon. The results show that 3B43 V7 product is able to capture the phase of the annual cycle of monthly mean precipitation, but the performance is not good for the amplitude, in particular over the Andes and Pacific regions owing to complex climatic and topographic conditions. In general, 3B43 V7 exhibits good performance in the low‐lying and plain Amazon, Orinoco and Caribbean regions. Over the Andes region, characterized by complex topography, overestimation errors are identified [root mean squared error (RMSE) ≥83.59 mm·month−1 and relative bias (BIAS) ≥4.69%], whereas the extremely wet rainfall regime of the Pacific region is largely underestimated (RMSE ≥253.52 mm ·month−1 and BIAS ≤−11.75%). These errors are greater during the wet seasons when the metrics reach worse scores than those reported in similar studies worldwide. Occurrence analyses showed that 3B43 V7 misses very frequent light rainfall events and less frequent but very heavy storms, which contribute to the overall underestimation (overestimation) observed over the Pacific (Andes) region. The error characteristics identified and quantified in this study confirm the well‐documented limitations of remote precipitation sensing and constitute a warning about major challenges that complex climatic and physiographic features can impose on satellite rainfall missions.

Columbia↗
NASA NTRS2011

Assessment of Computational Fluid Dynamics (CFD) Models for Shock Boundary-Layer Interaction

A workshop on the computational fluid dynamics (CFD) prediction of shock boundary-layer interactions (SBLIs) was held at the 48th AIAA Aerospace Sciences Meeting. As part of the workshop numerous CFD analysts submitted solutions to four experimentally measured SBLIs. This paper describes the assessment of the CFD predictions. The assessment includes an uncertainty analysis of the experimental data, the definition of an error metric and the application of that metric to the CFD solutions. The CFD solutions provided very similar levels of error and in general it was difficult to discern clear trends in the data. For the Reynolds Averaged Navier-Stokes methods the choice of turbulence model appeared to be the largest factor in solution accuracy. Large-eddy simulation methods produced error levels similar to RANS methods but provided superior predictions of normal stresses.

DeBonis, James R.↗
NASA NTRS1993

DCT quantization matrices visually optimized for individual images

This presentation describes how a vision model incorporating contrast sensitivity, contrast masking, and light adaptation is used to design visually optimal quantization matrices for Discrete Cosine Transform image compression. The Discrete Cosine Transform (DCT) underlies several image compression standards (JPEG, MPEG, H.261). The DCT is applied to 8x8 pixel blocks, and the resulting coefficients are quantized by division and rounding. The 8x8 'quantization matrix' of divisors determines the visual quality of the reconstructed image; the design of this matrix is left to the user. Since each DCT coefficient corresponds to a particular spatial frequency in a particular image region, each quantization error consists of a local increment or decrement in a particular frequency. After adjustments for contrast sensitivity, local light adaptation, and local contrast masking, this coefficient error can be converted to a just-noticeable-difference (jnd). The jnd's for different frequencies and image blocks can be pooled to yield a global perceptual error metric. With this metric, we can compute for each image the quantization matrix that minimizes the bit-rate for a given perceptual error, or perceptual error for a given bit-rate. Implementation of this system demonstrates its advantages over existing techniques. A unique feature of this scheme is that the quantization matrix is optimized for each individual image. This is compatible with the JPEG standard, which requires transmission of the quantization matrix.

Watson, Andrew B.↗
NASA NTRS2013

An Approach for the Assessment of System Upset Resilience

This report describes an approach for the assessment of upset resilience that is applicable to systems in general, including safety-critical, real-time systems. For this work, resilience is defined as the ability to preserve and restore service availability and integrity under stated conditions of configuration, functional inputs and environmental conditions. To enable a quantitative approach, we define novel system service degradation metrics and propose a new mathematical definition of resilience. These behavioral-level metrics are based on the fundamental service classification criteria of correctness, detectability, symmetry and persistence. This approach consists of a Monte-Carlo-based stimulus injection experiment, on a physical implementation or an error-propagation model of a system, to generate a system response set that can be characterized in terms of dimensional error metrics and integrated to form an overall measure of resilience. We expect this approach to be helpful in gaining insight into the error containment and repair capabilities of systems for a wide range of conditions.

Torres-Pomales, Wilfredo↗
NASA NTRS1975

Automatic documentation system extension to multi-manufacturers' computers and to measure, improve, and predict software reliability

The DOMONIC system has been modified to run on the Univac 1108 and the CDC 6600 as well as the IBM 370 computer system. The DOMONIC monitor system has been implemented to gather data which can be used to optimize the DOMONIC system and to predict the reliability of software developed using DOMONIC. The areas of quality metrics, error characterization, program complexity, program testing, validation and verification are analyzed. A software reliability model for estimating program completion levels and one on which to base system acceptance have been developed. The DAVE system which performs flow analysis and error detection has been converted from the University of Colorado CDC 6400/6600 computer to the IBM 360/370 computer system for use with the DOMONIC system.

Simmons, D. B.↗
NASA NTRS1992

Robust Method For Robotic Mapping

Robot constructs map from experience. Topological model consists of nodes and arcs corresponding to distinctive places and local travel edges linking nearby distinctive places. Model created by linking places and edges. Enables accumulation of metrical information with reduced vulnerability to metrical errors. Applications include robotic sentires, robotic delivery trucks, robotic floor cleaners, and robotic lawnmowers.

Kuipers, Benjamin J.↗
NASA NTRS1993

Phase-retrieval algorithms for a complicated optical system

Phase-retrieval algorithms have been developed that handle a complicated optical system that requires multiple Fresnellike transforms to propagate from one end of the system to the other including the absorption by apertures in more than one plane and allowance for bad detector pixels. Gradient-search algorithms and generalizations of the iterative-transform phase-retrieval algorithms are derived. Analytic expressions for the gradient of an error metric, with respect to polynomial coefficients and with respect to point-by-point phase descriptions, are given. The entire gradient can be computed with the number of transforms required to propagate a wave front from one end of the optical system to the other and back again, independent of the number of coefficients or phase points. This greatly speeds the computation. The reconstruction of pupil amplitude is also given. A convergence proof of the generalized iterative transform algorithm is given. These improved algorithms permit a more accurate characterization of complicated optical systems from their point spread functions.

Fienup, J. R.↗
NASA NTRS1993

Nonlinear aerodynamic modeling using multivariate orthogonal functions

A technique was developed for global modeling of nonlinear aerodynamic coefficients using multivariate orthogonal functions based on the data. Each orthogonal function retained in the model was decomposed into an expansion of ordinary polynomials in the independent variables, so that the final model could be interpreted as selectively retained terms from a multivariable power series expansion. A predicted squared-error metric was used to determine the orthogonal functions to be retained in the model; analytical derivatives were easily computed. The approach was demonstrated on the Z-body axis aerodynamic force coefficient (Cz) wind tunnel data for an F-18 research vehicle which came from a tabular wind tunnel and covered the entire subsonic flight envelope. For a realistic case, the analytical model predicted experimental values of Cz very well. The modeling technique is shown to be capable of generating a compact, global analytical representation of nonlinear aerodynamics. The polynomial model has good predictive capability, global validity, and analytical differentiability.

Morelli, Eugene A.↗
NASA NTRS2009

Comparison of Image Restoration Methods for Lunar Epithermal Neutron Emission Mapping

Orbital measurements of neutrons by the Lunar Exploring Neutron Detector (LEND) onboard the Lunar Reconnaissance Orbiter are being used to quantify the spatial distribution of near surface hydrogen (H). Inferred H concentration maps have low signal-to-noise (SN) and image restoration (IR) techniques are being studied to enhance results. A single-blind. two-phase study is described in which four teams of researchers independently developed image restoration techniques optimized for LEND data. Synthetic lunar epithermal neutron emission maps were derived from LEND simulations. These data were used as ground truth to determine the relative quantitative performance of the IR methods vs. a default denoising (smoothing) technique. We review and used factors influencing orbital remote sensing of neutrons emitted from the lunar surface to develop a database of synthetic "true" maps for performance evaluation. A prior independent training phase was implemented for each technique to assure methods were optimized before the blind trial. Method performance was determined using several regional root-mean-square error metrics specific to epithermal signals of interest. Results indicate unbiased IR methods realize only small signal gains in most of the tested metrics. This suggests other physically based modeling assumptions are required to produce appreciable signal gains in similar low SN IR applications.

McClanahan, T. P.↗