Search NASA⌕ Search

SEARCH · Search NASA

Results for “error impact analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

Man-Machine Integration Design and Analysis System (MIDAS) v5: Augmentations, Motivations, and Directions for Aeronautics Applications

As automation and advanced technologies are introduced into transport systems ranging from the Next Generation Air Transportation System termed NextGen, to the advanced surface transportation systems as exemplified by the Intelligent Transportations Systems, to future systems designed for space exploration, there is an increased need to validly predict how the future systems will be vulnerable to error given the demands imposed by the assistive technologies. One formalized approach to study the impact of assistive technologies on the human operator in a safe and non-obtrusive manner is through the use of human performance models (HPMs). HPMs play an integral role when complex human-system designs are proposed, developed, and tested. One HPM tool termed the Man-machine Integration Design and Analysis System (MIDAS) is a NASA Ames Research Center HPM software tool that has been applied to predict human-system performance in various domains since 1986. MIDAS is a dynamic, integrated HPM and simulation environment that facilitates the design, visualization, and computational evaluation of complex man-machine system concepts in simulated operational environments. The paper will discuss a range of aviation specific applications including an approach used to model human error for NASA s Aviation Safety Program, and what-if analyses to evaluate flight deck technologies for NextGen operations. This chapter will culminate by raising two challenges for the field of predictive HPMs for complex human-system designs that evaluate assistive technologies: that of (1) model transparency and (2) model validation.

Gore, Brian F.↗

Subjective evaluation of the combined influence of satellite temperature sounding data and increased model resolution on numerical weather forecasting

The present evaluation is concerned with (1) the significance of prognostic differences resulting from the inclusion of satellite-derived temperature soundings, (2) how specific differences between the SAT and NOSAT prognoses evolve, and (3) comparison of two experiments using the Goddard Laboratory for Atmospheric Sciences general circulation model. The subjective evaluation indicates that the beneficial impact of sounding data is enhanced with increased resolution. It is suggested that satellite sounding data posses valuable information content which at times can correct gross analysis errors in data sparse regions.

Atlas, R.↗

An Overview of Atmospheric Composition OSSE Activities at NASA's Global Modeling and Assimilation Office

A model-based Observing System Simulation Experiment (OSSE) is a framework for numerical experimentation in which observables are simulated from fields generated by an earth system model, including a parameterized description of observational error characteristics. Simulated observations can be used for sampling studies, quantifying errors in analysis or retrieval algorithms, and ultimately being a planning tool for designing new observing missions. While this framework has traditionally been used to assess the impact of observations on numerical weather prediction, it has a much broader applicability, in particular to aerosols and chemical constituents. In this talk we will give a general overview of Observing System Simulation Experiments (OSSE) activities at NASA's Global Modeling and Assimilation Office, with focus on its emerging atmospheric composition component.

daSilva, Arlinda↗

Sources of error in detonation calorimeters and error analysis for neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB)

Here, a calorimeter for measuring heats of detonation at Lawrence Livermore National Laboratory is described. A calibration precision of 0.2 % at the 95 % confidence interval (CI) is reported. Sources of uncertainty are discussed, including nonequivalent sources, which are those arising from differences between calibration and experimental tests. The systematic error due to nonequivalent sources is bounded to 0.19–0.22 % of the measured heat for a standard detonation test where the confinement material is gold. The recommendation is to correct for the systematic error by adding 0.19 % to the reported value and adding 0.03 % to the uncertainty. It is demonstrated that the precision of a detonation test is variable with testing duration because a source of uncertainty resides in the thermodynamic correction factor k6, which accounts for the contribution to heat by stirring; the generated power is additive and therefore highly impacted by total test time. The relative proportion also varies with the magnitude of heat release and sample size, adding variance to the weight of the error arising from the correction factor. A full error analysis based on the described sources of uncertainty is developed. The methodology is applied to a test series on neat 2,4,6-triamino-1,3,5- trinitrobenzene (TATB), demonstrating an ultimate precision of 0.7 % (single test) for materials testing and a relative standard deviation of 1.8 %.

47 OTHER INSTRUMENTATION↗

System design impact of guidance and navigation analysis for a SEPS 1979 Encke flyby

Baseline guidance and navigation strategy for an ecliptic projection of the Encke flyby mission consider solar electric propulsion stage parameters in generating optimized and targeted trajectory control. Results show the Encke relative approach error to be dominated by ephemerical uncertainties, particularly the velocity components.

Hong, P.↗

Differing Air Traffic Controller Responses to Similar Trajectory Prediction Errors: An Interrupted Time-Series Analysis of Controller Behavior

A Human-In-The-Loop simulation was conducted in January of 2013 in the Airspace Operations Laboratory at NASAs Ames Research Center. The simulation airspace included two en route sectors feeding the northwest corner of Atlantas Terminal Radar Approach Control. The focus of this paper is on how uncertainties in the studys trajectory predictions impacted the controllers ability to perform their duties. Of particular interest is how the controllers interacted with the delay information displayed in the meter list and data block while managing the arrival flows. Due to wind forecasts with 20-knot over-predictions and 20-knot under-predictions, delay value computations included errors of similar magnitude, albeit in opposite directions. However, when performing their duties in the presence of these errors, did the controllers issue clearances of similar magnitude, albeit in opposite directions?

human-automation interaction↗

Analysis of Mars 2020 Perseverance Entry, Descent, and Landing Attitude Initialization Performance

The Mars 2020 Perseverance rover successfully landed at Jezero Crater on February 18, 2021. To perform this feat, the entry, descent, and landing navigation filter must be initialized with an estimate of the spacecraft state, which includes attitude from the cruise attitude control subsystem. Accuracy of this initial attitude estimate has an impact on important metrics such as touchdown velocity. This paper presents a post-landing reconstruction of the attitude initialization error budget, showing that the requirement of 0.15 deg ($3\sigma$, per axis) was met. Each error source is described and analyzed, including flight telemetry where possible. Analysis of the error budget shows that it is driven by systematic sun sensor errors, star-scanner-to-sun-sensor alignment stability, and inertial-measurement-unit-to-sun-sensor alignment stability. Finally, a contingency plan to initialize the navigation filter in the event of a star scanner failure is presented. While this plan was not needed in flight, results indicate that the attitude initialization requirement could have still been met in this off-nominal scenario.

Lo, Kevin D.↗

Benchmarking machine learning strategies for phase-field problems

Abstract We present a comprehensive benchmarking framework for evaluating machine-learning approaches applied to phase-field problems. This framework focuses on four key analysis areas crucial for assessing the performance of such approaches in a systematic and structured way. Firstly, interpolation tasks are examined to identify trends in prediction accuracy and accumulation of error over simulation time. Secondly, extrapolation tasks are also evaluated according to the same metrics. Thirdly, the relationship between model performance and data requirements is investigated to understand the impact on predictions and robustness of these approaches. Finally, systematic errors are analyzed to identify specific events or inadvertent rare events triggering high errors. Quantitative metrics evaluating the local and global description of the microstructure evolution, along with other scalar metrics representative of phase-field problems, are used across these four analysis areas. This benchmarking framework provides a path to evaluate the effectiveness and limitations of machine-learning strategies applied to phase-field problems, ultimately facilitating their practical application.

36 MATERIALS SCIENCE↗

Failure analysis and modeling of a VAXcluster system

This paper discusses the results of a measurement-based analysis of real error data collected from a DEC VAXcluster multicomputer system. In addition to evaluating basic system dependability characteristics such as error and failure distributions and hazard rates for both individual machines and for the VAXcluster, reward models were developed to analyze the impact of failures on the system as a whole. The results show that more than 46 percent of all failures were due to errors in shared resources. This is despite the fact that these errors have a recovery probability greater than 0.99. The hazard rate calculations show that not only errors, but also failures occur in bursts. Approximately 40 percent of all failures occur in bursts and involved multiple machines. This result indicates that correlated failures are significant. Analysis of rewards shows that software errors have the lowest reward (0.05 vs 0.74 for disk errors). The expected reward rate (reliability measure) of the VAXcluster drops to 0.5 in 18 hours for the 7-out-of-7 model and in 80 days for the 3-out-of-7 model.

Tang, Dong↗

Impact of uncertainties in the cosmological parameters on the measurement of primordial non-Gaussianity

We study the impact of cosmological parameters' uncertainties on estimates of the primordial non-Gaussianity (NG) parameter f{sub NL} in local and equilateral models of non-Gaussianity. We show that propagating these errors increases the f{sub NL} 1{sigma} uncertainty by a term {delta}f{sub NL}{sup local}/f{sub NL}{sup local}{approx_equal}16% for WMAP and {delta}f{sub NL}{sup local}/f{sub NL}{sup local}{approx_equal}5% for Planck in the local case, whereas for equilateral configurations the correction terms are {delta}f{sub NL}{sup equil}/f{sub NL}{sup equil}{approx_equal}14% and {delta}f{sub NL}{sup equil}/f{sub NL}{sup equil}{approx_equal}4%, respectively. If we assume for f{sub NL}{sup local} a central value <f{sub NL}{sup local}>{approx_equal}60, according to recent WMAP 5-year estimates, we obtain for Planck a final correction {delta}f{sub NL}{sup local}{approx_equal}3. Although not dramatic, this correction is at the level of the expected estimator uncertainty for Planck, and should then be taken into account when quoting the significance of an eventual future detection. In current estimates of f{sub NL} the cosmological parameters are held fixed at their best-fit values. We finally note that the impact of uncertainties in the cosmological parameters on the final f{sub NL} error bar would become totally negligible if the parameters were allowed to vary in the analysis and then be marginalized over.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Scaling Automatic Vector Data Alignment to Satellite Imagery

Given the tremendous volume of accessible Earth Observation (EO) data, there is a need to develop scalable Geospatial Artificial Intelligence (GeoAI) solutions for time-sensitive applications. Scalability in this context refers to rapidly processing large-scale EO data using high performance computing resources. Accurate mapping of the built environment from remote sensing (RS) imagery has been one of the crucial components in GeoAI workflows for a wide spectrum of humanitarian applications. Derived vector data of built environment is often leveraged for disaster preparedness and response activities. However, factors such as differences in ortho-rectification, atmospheric conditions and human error, results in spatial misalignment between vector data and the timely available RS imagery. Model training for downstream tasks such as object detection, change analysis, etc., is negatively impacted due to such spatial misalignment. Although there has been progress towards automatic alignment of vector data, the lack of scalability remains an open research challenge. This paper proposes to leverage parallel computing to optimize an automatic vector data alignment workflow. It further employs CPU-level multi-core parallelism for improving the performance of the workflow for scalable built environment mapping. We report observations and discuss findings from the preliminary experiments performed on the Summit Supercomputer.

Potnis, Abhishek↗

Enhancing ZFP: A Statistical Approach to Understanding and Reducing Error Bias in a Lossy Floating-Point Compression Algorithm

The amount of data generated and gathered in scientific simulations and data collection applications is continuously growing, putting mounting pressure on storage and bandwidth concerns. A means of reducing such issues is data compression; but, lossless data compression is typically ineffective when applied to floating-point data. Thus, users tend to apply a lossy data compressor, which allows for small deviations from the original data. It is essential to understand how the error from lossy compression impacts the accuracy of the data analytics. Thus, we must analyze not only the compression properties but the error as well. In this paper, we provide a statistical analysis of the error caused by ZFP compression, a state-of-the-art, lossy compression algorithm explicitly designed for floating-point data. We show that the error is indeed biased and propose simple modifications to the algorithm to neutralize the bias and further reduce the resulting error.

97 MATHEMATICS AND COMPUTING↗

Using TRMM Field Campaign Data for Assessing GEOS Forecast and Assimilation Products

The wealth of in-situ measurements gathered during Tropical Rain Measuring Mission (TRMM) field campaigns over a wide range of tropical conditions constitute an important data source for evaluating the quality of global model forecasts and assimilated datasets. In this study we use selected observations of cloud microphysics and atmospheric sounding from TEFLUN-1998, SCEMEX-1998, and TRMMLBA-1999 to examine the assimilation and forecast fields produced by the operational GEOS-3 (Goddard Earth Observing System - version 3) global data assimilation system (DAS) and a new finite-volume DAS under development at the Data Assimilation Office. Additionally, TRMM field campaign measurements are used to verify the impact of assimilating rainfall and moisture data derived from TRMM Microwave Imager and Special Sensor Microwave/Imager instruments on the GEOS analysis. We will also explore issues concerning the 'error of representativeness' in using in-situ observations of quantities with large spatial and temporal variability such as precipitation for validating gridded global data products.

Hou, Arthur Y.↗

Improving Boundary Layer Data Assimilation Using Observation Data from Multiple Observing Systems in the NASA GEOS System

The Planetary Boundary Layer (PBL) is a complex interface that mediates energy and moisture exchanges between the Earth’s surface and atmosphere. Accurate simulation and observation of PBL characteristics, such as PBL height and thermodynamic structure, have proven to be a challenge. In our latest efforts, we have focused on improving PBL thermodynamic structure using data from multiple observing systems in the Goddard Earth Observing System (GEOS), developed by the NASA Global Modeling and Assimilation Office (GMAO). We present strategies and results from assimilating PBL height data derived from radiosondes, GNSS radio occultation, space-based lidar (CALIPSO, CATS, IceSat-2), ground-based lidar (MPLNET), and radar wind profilers, including a novel global PBL height analysis dataset. We also discuss the impacts of better representing capping inversions by using PBL height data together with other observations in GEOS through adjustments to the background error covariance. Long-term statistics of the impact of assimilating and utilizing PBL height data in GEOS are presented. In addition, we explore an innovative approach to assimilate GNSS-RO refractivity data in the lower troposphere.

Eun-Gyeong Yang↗

Assessing the Impact of Observations in a Multi-Year Reanalysis

Operational and quasi-operational weather prediction centers have been routinely assessing the contribution from various observing systems to reducing errors in short-range forecasts for a number of years now. The original technique, Forecast Sensitivity Observation Impact (FSOI), involves definition of a forecast error measure and evaluation of sensitivities with respect to changes in the observing system that require adjoint operators of both the underlying tangent linear model and corresponding analysis technique. The present work applies FSOI to Reanalysis and aims at providing an expanded view of the contribution of various observing systems over nearly 40 years of assimilation. Specifically, this study uses MERRA-2 given that its supporting software includes all ingredients necessary to calculate FSOI. Part of this work shows how the quality of forecasts improves over the course of the reanalysis, and examines forecast sensitivities relevant to FSOI. The assessment here finds, for example, that: conventional observations are a major player in reducing forecast error throughout the 40 years of reanalysis, even when their volume reduces from 45% in the earlier periods to about 5% in the modern era; satellite radiances, especially microwave instruments are major contributors to error reduction from the early single platform TIROS-N days to the current multi-platform scenario; infrared instruments play a secondary role to microwave but are significant still, with the peculiar result of fractional impacts contribution from modern hyperspectral instruments being roughly similar to those from early infrared instruments. The dependence of results on the chosen error measure is emphasized throughout.

Fabio L R Diniz↗

Assessing the Impact of Observations in a Multi-Year Reanalysis (MERRA-2)

Operational and quasi-operational weather prediction centers have been routinely assessing the contribution from various observing systems to reducing errors in short-range forecasts for a number of years now. The original technique, Forecast Sensitivity-based Observation Impact (FSOI), involves definition of a forecast error measure and evaluation of sensitivities with respect to changes in the observations that require adjoint operators of both the underlying tangent linear model and corresponding analysis technique. The present work applies FSOI to reanalysis and aims at providing an expanded view of the contribution of various observing systems over nearly 40 years of assimilation. Specifically, this study uses MERRA-2 given that its supporting software includes all ingredients necessary to calculate FSOI. Part of this work shows how the quality of forecasts improves over the course of the reanalysis, and examines forecast sensitivities relevant to FSOI. The assessment here finds, for example, that: conventional observations are a major player in reducing forecast error throughout the 40 years of reanalysis, even when their volume reduces from 45\% in the earlier periods to about 5% in the modern era; satellite radiances, especially microwave instruments are major contributors to error reduction from the early single platform TIROS-N days to the current multi-platform scenario, though their fractional contribution reduces slightly from the early 2000's onward after the increased availability of wind observation from aircraft and atmospheric motion vectors, and the introduction of GPSRO; infrared instruments play a secondary role to microwave but are significant still, with the peculiar result of fractional impacts contribution from modern hyperspectral instruments being roughly similar to those from early infrared instruments. The dependence of results on the chosen error measure is emphasized throughout.

Todling, Ricardo↗

Building a standardized Observing System Simulation Experiment (OSSE) framework for Mars

We advocate that the Decadal Survey recommends the NASA Science Mission Directorate to develop a rigorous Observing System Simulation Experiment (OSSE) framework for Mars, to optimize future atmospheric observations. Atmospheric conditions on Mars are a potential hazard source for landing missions. Errors in the estimates of atmospheric density profiles, inadequate knowledge of wind vertical structure and dust concentration as a function of height are likely causes of uncertainty at the landing site on the order of kilometers. An operational real-time weather forecasting capability for Mars would reduce such uncertainties, carrying enormous benefits to future robotic missions, and would be an invaluable prerequisite for human missions.A real-time forecasting capability relies upon three fundamental components: a critical mass of observing systems, a data assimilation system (DAS), and a global forecast model. The DAS allows the model to ingest the data effectively, optimizing the observational information content,and transforming them into a gridded representation of the atmosphere at a given time, called an ‘analysis’. The analysis is the best estimate of the atmospheric state for that time, and also represents a set of ‘initial conditions’ from which a global model can be initialized, to predict a future state of the atmosphere. The connection between analysis and forecast represents the foundation of modern weather forecasting. However, from the point of view of a forecast system,not all observations are equally impactful, partially because of the problem of “observational error correlation”, one important research topic in data assimilation development. For the Earth, partly due to the spontaneous and deregulated development of observations and forecast capabilities worldwide for more than half a century,the use of observations in contemporary operational forecast systems is suboptimal, with many potentially useful data being underutilized. On the contrary, Mars atmospheric scientists are in the unique situation of designing the next-generation observing systems by learning from the experience gathered on the Earth, so as to assure that the future instruments are specifically optimized to give the maximum benefit to a future weather forecast capability.An immensely powerful tool that has been firmly established by atmospheric scientists on the Earth is represented by a properly designed OSSE framework. A realistic OSSE framework cannot only quantify the benefit of future data types, be them surface based or space borne, but can also help design and optimize an entire observational network. Furthermore, OSSEs can provide deep insights into an atmosphere’s behavior, by addressing conceptual problems of its intrinsic predictability and delineating the regions or features of the atmosphere which are more sensitive to additional data and would benefit from a denser sampling. The difficulties posed by OSSEs are fundamentally different for Earth and Mars. For Earth, the enormous data volume imposes a tremendous constraint on any innovation in the observing systems: it is very hard for a single sensor to impact the skill. For Mars, the problem is the opposite: almost any additional instrument will exert some impact. However, OSSEs can help to evaluate the cost/benefit for every sensor and suggest optimal data configuration and density.The purpose of this white paper is to provide an introduction to a rigorously designed OSSE framework, explain the underlying problems and challenges, and engage the Mars community to collaborate with Earth Atmospheric scientists in order to develop a joint-OSSE framework for Mars with the largest consensual basis possible. An OSSE infrastructure would increase the understanding of the Martian atmosphere, would help NASA to optimize instrument specifications and orbit choice, providing the maximium benefit for a given expenditure of resources, and could even help establishing a roadmap for a future real-time weather forecasting capability.

Oreste Reale↗

Quality Assurance With Plan Veto: Reincarnation of a Record and Verify System and Its Potential Value

Purpose: To quantify the potential impact of the Integrating the Healthcare Enterprise–Radiation Oncology Quality Assurance with Plan Veto (QAPV) on patient safety of external beam radiation therapy (RT) operations. Methods and Materials: An institutional database of events (errors and near-misses) was used to evaluate the ability of QAPV to prevent clinically observed events. We analyzed reported events that were related to Digital Imaging and Communications in Medicine RT plan parameter inconsistencies between the intended treatment (on the treatment planning system) and the delivered treatment (on the treatment machine). Critical Digital Imaging and Communications in Medicine RT plan parameters were identified. Each event was scored for importance using the Failure Mode and Effects Analysis methodology. Potential error occurrence (frequency) was derived according to the collected event data, along with the potential event severity, and the probability of detection with and without the theoretical implementation of the QAPV plan comparison check. Failure Mode and Effects Analysis Risk Priority Numbers (RPNs) with and without QAPV were compared to quantify the potential benefit of clinical implementation of QAPV. Results: The implementation of QAPV could reduce the RPN values for 15 of 22 (71%) of evaluated parameters, with an overall average reduction in RPN of 68 (range, 0-216). For the 6 high-risk parameters (>200), the average reduction in RPN value was 163 (range, 108-216). The RPN value reduction for the intermediate-risk (200 > RPN > 100) parameters was (0-140). With QAPV, the largest RPN value for “Beam Meterset” was reduced from 324 to 108. The maximum reduction in RPN value was for Beam Meterset (216, 66.7%), whereas the maximum percentage reduction was for Cumulative Meterset Weight (80, 88.9%). Conclusion: This analysis quantifies the value of the Integrating the Healthcare Enterprise–Radiation Oncology QAPV implementation in clinical workflow. We demonstrate that although QAPV does not provide a comprehensive solution for error prevention in RT, it can have a significant impact on a subset of the most severe clinically observed events.

62 RADIOLOGY AND NUCLEAR MEDICINE↗