Search NASA⌕ Search

SEARCH · Search NASA

Results for “key performance metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

NASA Engineering and Safety Center Lunar Rover Design Concepts Assessments

Mass is a significant risk to programs and projects as they transition from formulation to implementation, especially in larger human space systems where delivery mass or volume can be a constrained commodity. • Concepts developed without an adequate systems-engineering basis, including design and operations experience, may miss significant functionality and subsequent mass required for integration or operations. • Further uncertainty can be associated with not fully understanding design best practices and standards that drive mass, such as design for minimum risk or fault tolerance.1 • Finally, the appropriate systems engineering mass properties management rigor and technical discipline insight are required to set the mass baseline, including appropriate mass growth allowance (MGA) to ensure program success. Mass is a key quantity that should be constantly monitored by the systems engineer and the program/project management stakeholder to ensure mission compatibility throughout the project life cycle. In addition, mass is a key programmatic performance metric monitored by the NASA Chief Financial Officer for current and future program cost estimating. NASA Exploration Systems sought an assessment of reasonableness with respect to different potential rover concepts that balances mass needs and human-class cargo lander capabilities. Appropriate systems engineering mass properties management rigor and technical discipline insight were used, and are required to set the mass baseline, including appropriate MGA to ensure program success. Two independent Lunar rover concepts were evaluated, with a goal to understand concept credibility and the Lunar rover designs potential extensibility for Mars surface operations. A notional generic rover concept is shown in Figure 1.

Systems Engineering↗

Wind Prediction Accuracy for Air Traffic Management Decision Support Tools

The performance of Air Traffic Management and flight deck decision support tools depends in large part on the accuracy of the supporting 4D trajectory predictions. This is particularly relevant to conflict prediction and active advisories for the resolution of conflicts and the conformance with of traffic-flow management flow-rate constraints (e.g., arrival metering / required time of arrival). Flight test results have indicated that wind prediction errors may represent the largest source of trajectory prediction error. The tests also discovered relatively large errors (e.g., greater than 20 knots), existing in pockets of space and time critical to ATM DST performance (one or more sectors, greater than 20 minutes), are inadequately represented by the classic RMS aggregate prediction-accuracy studies of the past. To facilitate the identification and reduction of DST-critical wind-prediction errors, NASA has lead a collaborative research and development activity with MIT Lincoln Laboratories and the Forecast Systems Lab of the National Oceanographic and Atmospheric Administration (NOAA). This activity, begun in 1996, has focussed on the development of key metrics for ATM DST performance, assessment of wind-prediction skill for state of the art systems, and development/validation of system enhancements to improve skill. A 13 month study was conducted for the Denver Center airspace in 1997. Two complementary wind-prediction systems were analyzed and compared to the forecast performance of the then standard 60 km Rapid Update Cycle - version 1 (RUC-1). One system, developed by NOAA, was the prototype 40-km RUC-2 that became operational at NCEP in 1999. RUC-2 introduced a faster cycle (1 hr vs. 3 hr) and improved mesoscale physics. The second system, Augmented Winds (AW), is a prototype en route wind application developed by MITLL based on the Integrated Terminal Wind System (ITWS). AW is run at a local facility (Center) level, and updates RUC predictions based on an optimal interpolation of the latest ACARS reports since the RUC run. This paper presents an overview of the study's results including the identification and use of new large mor wind-prediction accuracy metrics that are key to ATM DST performance.

Cole, Rod↗

Initial Performance of the Aspect System on the Chandra Observatory: Post-Facto Aspect Reconstruction

The aspect system of the Chandra Observatory plays a key role in realizing the full potential of Chandra's x-ray optics and detectors. To achieve the highest spatial and spectral resolution (for grating observations), an accurate post-facto time history of the spacecraft attitude and internal alignment is needed. The CXC has developed a suite of tools which process sensor data from the aspect camera assembly and gyroscopes, and produce the spacecraft aspect solution. In this poster, the design of the aspect pipeline software is briefly described, followed by details of aspect system performance during the first eight months of flight. The two key metrics of aspect performance are: image reconstruction accuracy, which measures the x-ray image blurring introduced by aspect; and celestial location, which is the accuracy of detected source positions in absolute sky coordinates.

Aldcroft, T.↗

Job Superscheduler Architecture and Performance in Computational Grid Environments

Computational grids hold great promise in utilizing geographically separated heterogeneous resources to solve large-scale complex scientific problems. However, a number of major technical hurdles, including distributed resource management and effective job scheduling, stand in the way of realizing these gains. In this paper, we propose a novel grid superscheduler architecture and three distributed job migration algorithms. We also model the critical interaction between the superscheduler and autonomous local schedulers. Extensive performance comparisons with ideal, central, and local schemes using real workloads from leading computational centers are conducted in a simulation environment. Additionally, synthetic workloads are used to perform a detailed sensitivity analysis of our superscheduler. Several key metrics demonstrate that substantial performance gains can be achieved via smart superscheduling in distributed computational grids.

Shan, Hongzhang↗

Impact of Airborne Radar Uncertainties on Detect-and-Avoid Systems’ Performance

Impact of sensor uncertainty on Detect-and-Avoid (DAA) systems’ performance is investigated. Key metrics analyzed are the loss of DAA well clear ratio, the near-mid-air-collision risk ratio, the alert ratio, and the number of maneuvers per loss of DAA well clear. Sensitivity of these metrics to the magnitude of sensor uncertainty, pilots’ selection of maneuver, and surveillance range is investigated. Benefits of a dynamic buffer around the DAA well clear separation boundary, computed based on track accuracies, are also analyzed. These metrics are computed from open- and closed-loop simulations of a large number of representative encounters between an unmanned aircraft system and a manned aircraft. Results show that sensor uncertainty degrades safety metrics considerably but has only a minor effect on the number of maneuvers per loss of DAA well clear. The number of maneuvers, nonetheless, can be reduced by a 5◦ buffer away from the edge of the range of conflict-resulting heading in pilots’ selection of maneuver.

UAS↗

Job Scheduling in a Heterogeneous Grid Environment

Computational grids have the potential for solving large-scale scientific problems using heterogeneous and geographically distributed resources. However, a number of major technical hurdles must be overcome before this potential can be realized. One problem that is critical to effective utilization of computational grids is the efficient scheduling of jobs. This work addresses this problem by describing and evaluating a grid scheduling architecture and three job migration algorithms. The architecture is scalable and does not assume control of local site resources. The job migration policies use the availability and performance of computer systems, the network bandwidth available between systems, and the volume of input and output data associated with each job. An extensive performance comparison is presented using real workloads from leading computational centers. The results, based on several key metrics, demonstrate that the performance of our distributed migration algorithms is significantly greater than that of a local scheduling framework and comparable to a non-scalable global scheduling approach.

Shan, Hong-Zhang↗

Distributed Space Mission Design for Earth Observation Using Model-Based Performance Evaluation

Distributed Space Missions (DSMs) are gaining momentum in their application to earth observation missions owing to their unique ability to increase observation sampling in multiple dimensions. DSM design is a complex problem with many design variables, multiple objectives determining performance and cost and emergent, often unexpected, behaviors. There are very few open-access tools available to explore the tradespace of variables, minimize cost and maximize performance for pre-defined science goals, and therefore select the most optimal design. This paper presents a software tool that can multiple DSM architectures based on pre-defined design variable ranges and size those architectures in terms of predefined science and cost metrics. The tool will help a user select Pareto optimal DSM designs based on design of experiments techniques. The tool will be applied to some earth observation examples to demonstrate its applicability in making some key decisions between different performance metrics and cost metrics early in the design lifecycle.

Constellation↗

I/O Performance Characterization of Lustre and NASA Applications on Pleiades

In this paper we study the performance of the Lustre file system using five scientific and engineering applications representative of NASA workload on large-scale supercomputing systems such as NASA s Pleiades. In order to facilitate the collection of Lustre performance metrics, we have developed a software tool that exports a wide variety of client and server-side metrics using SGI's Performance Co-Pilot (PCP), and generates a human readable report on key metrics at the end of a batch job. These performance metrics are (a) amount of data read and written, (b) number of files opened and closed, and (c) remote procedure call (RPC) size distribution (4 KB to 1024 KB, in powers of 2) for I/O operations. RPC size distribution measures the efficiency of the Lustre client and can pinpoint problems such as small write sizes, disk fragmentation, etc. These extracted statistics are useful in determining the I/O pattern of the application and can assist in identifying possible improvements for users applications. Information on the number of file operations enables a scientist to optimize the I/O performance of their applications. Amount of I/O data helps users choose the optimal stripe size and stripe count to enhance I/O performance. In this paper, we demonstrate the usefulness of this tool on Pleiades for five production quality NASA scientific and engineering applications. We compare the latency of read and write operations under Lustre to that with NFS by tracing system calls and signals. We also investigate the read and write policies and study the effect of page cache size on I/O operations. We examine the performance impact of Lustre stripe size and stripe count along with performance evaluation of file per process and single shared file accessed by all the processes for NASA workload using parameterized IOR benchmark.

Saini, Subhash↗

Shifting the Intertial Navigation Paradigm with the MEMS Technology

"Why don't you use MEMS?" is of the most common questions posed to navigation systems engineers designing inertial navigation solutions in the modern era. The question stems from a general understanding that great strides have been made in terrestrial MEMS accelerometers and attitude rate sensors in terms of accuracy, mass, and power. Yet, when compared on a unit-to-unit basis, MEMS devices do not provide comparable performance (accuracy) to navigation grade sensors in several key metrics. This paper will propose a paradigm shift where the comparison in performance is between multiple MEMS devices and a single navigation grade sensor. The concept is that systematically, a sufficient number of MEMS sensors may mathematically provide comparable performance to a single navigation grade device and be competitive in terms power and mass allocations when viewed on a systems level. The implication is that both inertial navigation system design and fault detection, identification, and recovery could benefit from a system of MEMS devices in the same way that swarm sensing has benefited Earth observation and astronomy. A survey of the state of the art in inertial sensor accuracy scaled by mass and power will be provided to show the scaled error in MEMS and navigation graded devices, a mathematical comparison of multi-unit to single-unit sensor errors will be developed, and preliminary application to an Orion lunar skip atmospheric entry trajectory will be explored.

Crain, Timothy P., II↗

Feasibility of Turing-Style Tests for Autonomous Aerial Vehicle "Intelligence"

A new approach is suggested to define and evaluate key metrics as to autonomous aerial vehicle performance. This approach entails the conceptual definition of a "Turing Test" for UAVs. Such a "UAV Turing test" would be conducted by means of mission simulations and/or tailored flight demonstrations of vehicles under the guidance of their autonomous system software. These autonomous vehicle mission simulations and flight demonstrations would also have to be benchmarked against missions "flown" with pilots/human-operators in the loop. In turn, scoring criteria for such testing could be based upon both quantitative mission success metrics (unique to each mission) and by turning to analog "handling quality" metrics similar to the well-known Cooper-Harper pilot ratings used for manned aircraft. Autonomous aerial vehicles would be considered to have successfully passed this "UAV Turing Test" if the aggregate mission success metrics and handling qualities for the autonomous aerial vehicle matched or exceeded the equivalent metrics for missions conducted with pilots/human-operators in the loop. Alternatively, an independent, knowledgeable observer could provide the "UAV Turing Test" ratings of whether a vehicle is autonomous or "piloted." This observer ideally would, in the more sophisticated mission simulations, also have the enhanced capability of being able to override the scripted mission scenario and instigate failure modes and change of flight profile/plans. If a majority of mission tasks are rated as "piloted" by the observer, when in reality the vehicle/simulation is fully- or semi- autonomously controlled, then the vehicle/simulation "passes" the "UAV Turing Test." In this regards, this second "UAV Turing Test" approach is more consistent with Turing s original "imitation game" proposal. The overall feasibility, and important considerations and limitations, of such an approach for judging/evaluating autonomous aerial vehicle "intelligence" will be discussed from a theoretical perspective.

Young, Larry A.↗

Performance Evaluation of the Approaches and Algorithms for Hamburg Airport Operations

In this work, fast-time simulations have been conducted using SARDA tools at Hamburg airport by NASA and real-time simulations using CADEO and TRACC with the NLR ATM Research Simulator (NARSIM) by DLR. The outputs are analyzed using a set of common metrics collaborated between DLR and NASA. The proposed metrics are derived from International Civil Aviation Organization (ICAO)s Key Performance Areas (KPAs) in capability, efficiency, predictability and environment, and adapted to simulation studies. The results are examined to explore and compare the merits and shortcomings of the two approaches using the common performance metrics. Particular attention is paid to the concept of the close-loop, trajectory-based taxi as well as the application of US concept to the European airport. Both teams consider the trajectory-based surface operation concept a critical technology advance in not only addressing the current surface traffic management problems, but also having potential application in unmanned vehicle maneuver on airport surface, such as autonomous towing or TaxiBot [6][7] and even Remote Piloted Aircraft (RPA). Based on this work, a future integration of TRACC and SOSS is described aiming at bringing conflict-free trajectory-based operation concept to US airport.

air traffic management↗

Combined Analysis of NASA's High Efficiency Megawatt Motor and Its Converter

NASA’s High Efficiency Megawatt Motor (HEMM) is being developed to achieve the performance needed by single aisle class electrified aircraft. It is a 1.4 MW electric machine designed as a generator for NASA’s STARC-ABL concept vehicle. It has performance objectives of greater than 16 kW/kg electromagnetic specific power and efficiency of greater than 98%. The key enabling technology for HEMM’s high performance metrics is a superconducting rotor cooled by an integrated cryocooler. HEMM’s integrated cryocooler is being designed to only lift 50 W of heat at its target 50 K cold tip operating temperature. HEMM’s superconducting rotor therefore needs to have essentially zero electromagnetic losses in order to stay below the 50 W heat rejection limit of the cryocooler. This paper presents the combined analysis of HEMM and a preliminarily designed HEMM converter to analyze the rotor loss impact of several potential converter filter topologies. The analysis results point to interleaving and series inductance filters not being sufficient to suppress switching harmonic caused rotor loss. Paths forward to develop a converter to eliminate rotor loss are discussed.

Matthew Granger↗

Combined Analysis of NASA's High Efficiency Megawatt Motor and Its Converter

NASA’s High Efficiency Megawatt Motor (HEMM) is being developed to achieve the performance needed by single aisle class electrified aircraft. It is a 1.4 MW electric machine designed as a generator for NASA’s STARC-ABL concept vehicle. It has performance objectives of greater than 16 kW/kg electromagnetic specific power and efficiency of greater than 98%. The key enabling technology for HEMM’s high performance metrics is a superconducting rotor cooled by an integrated cryocooler. HEMM’s integrated cryocooler is being designed to only lift 50 W of heat at its target 50 K cold tip operating temperature. HEMM’s superconducting rotor therefore needs to have essentially zero electromagnetic losses in order to stay below the 50 W heat rejection limit of the cryocooler. This paper presents the combined analysis of HEMM and a preliminarily designed HEMM converter to analyze the rotor loss impact of several potential converter filter topologies. The analysis results point to interleaving and series inductance filters not being sufficient to suppress switching harmonic caused rotor loss. Paths forward to develop a converter to eliminate rotor loss are discussed.

Matthew Granger↗

Alternative Metrics for Evaluating the Resilence of Advanced Life Support Systems

Ensuring the safety of the crew is a key performance requirement of a life support system. However, a number of conceptual and practical difficulties arise when devising metrics to concretely measure the ability of a life support system to maintain critical functions in the presence of anticipated and unanticipated faults. Resilience is a dynamic property of a life support system that depends on the complex interactions between faults, controls and system hardware. We review some of the approaches to understanding the robustness or resilience of complex systems being developed in diverse fields such as ecology, software engineering and cell biology and discuss their applicability to regenerative life support systems. We also consider how approaches to measuring resilience vary depending on system design choices such as the definition and choice of the nominal operating regime. Finally, we explore data collection and implementation issues such as the key differences between the instantaneous or conditional and average or overall measures of resilience. Extensive simulation of a hybrid computational model of a water revitalization subsystem (WRS) with probabilistic, component-level faults provides data about off-nominal behavior of the system. The data are used to consider alternative measures of resilience as predictors of the system's ability to recover from component-level faults.

Bell, Ann Maria↗

Environmentally Responsible Aviation (ERA) Project - N+2 Advanced Vehicle Concepts Study and Conceptual Design of Subscale Test Vehicle (STV) Final Report

NASA has set demanding goals for technology developments to meet national needs to improve fuel efficiency concurrent with improving the environment to enable air transportation growth. A figure shows NASA's subsonic transport system metrics. The results of Boeing ERA N+2 Advanced Vehicle Concept Study show that the Blended Wing Body (BWB) vehicle, with ultra high bypass propulsion systems have the potential to meet the combined NASA ERA N+2 goals. This study had 3 main activities. 1) The development of an advanced vehicle concepts that can meet the NASA system level metrics. 2) Identification of key enabling technologies and the development of technology roadmaps and maturation plans. 3) The development of a subscale test vehicle that can demonstrate and mature the key enabling technologies needed to meet the NASA system level metrics. Technology maturation plans are presented and include key performance parameters and technical performance measures. The plans describe the risks that will be reduced with technology development and the expected progression of technical maturity.

Aircraft Engines↗

Evaluating Delta-V Dispersions Using Linear Covariance Techniques with Applications to Rendezvous and Docking

One of the attractive advantages of using linear covariance analysis (LinCov) is that it can accurately produce both navigation errors and trajectory dispersions in a fraction of the time comparable to the standard Monte Carlo analysis approach that is traditionally adopted. It also has the capability of generating delta-v dispersions which is perhaps the most influential performance metric since many aspects of a mission design revolve around the anticipated propellant usage. Accurately capturing this key performance criteria using linear covariance techniques has several subtle caveats raising reasonable doubts and a cautious sense of skepticism. This paper outlines multiple approaches of modeling delta-v usage in LinCov, highlights their limitations and advantages, and ultimately compares their corresponding results to the actual Monte Carlo performance. The theory is also extended to reliably account for an arbitrary thruster configuration and the resultant propellant consumptions due to both translational burns and attitude maneuvers. This provides near instant insight into the impacts of thruster layout design and different vehicle mass properties. These derived techniques are applied and verified using an NRHO rendezvous and docking scenario consistent with upcoming NASA Artemis missions for Orion.

Linear Covariance Analysis↗

Multiple symbol partially coherent detection of MPSK

It is shown that by using the known (or estimated) value of carrier tracking loop signal to noise ratio (SNR) in the decision metric, it is possible to improve the error probability performance of a partially coherent multiple phase-shift-keying (MPSK) system relative to that corresponding to the commonly used ideal coherent decision rule. Using a maximum-likeihood approach, an optimum decision metric is derived and shown to take the form of a weighted sum of the ideal coherent decision metric (i.e., correlation) and the noncoherent decision metric which is optimum for differential detection of MPSK. The performance of a receiver based on this optimum decision rule is derived and shown to provide continued improvement with increasing length of observation interval (data symbol sequence length). Unfortunately, increasing the observation length does not eliminate the error floor associated with the finite loop SNR. Nevertheless, in the limit of infinite observation length, the average error probability performance approaches the algebraic sum of the error floor and the performance of ideal coherent detection, i.e., at any error probability above the error floor, there is no degradation due to the partial coherence. It is shown that this limiting behavior is virtually achievable with practical size observation lengths. Furthermore, the performance is quite insensitive to mismatch between the estimate of loop SNR (e.g., obtained from measurement) fed to the decision metric and its true value. These results may be of use in low-cost Earth-orbiting or deep-space missions employing coded modulations.

Simon, M. K.↗

Evaluating Algorithm Performance Metrics Tailored for Prognostics

Prognostics has taken a center stage in Condition Based Maintenance (CBM) where it is desired to estimate Remaining Useful Life (RUL) of the system so that remedial measures may be taken in advance to avoid catastrophic events or unwanted downtimes. Validation of such predictions is an important but difficult proposition and a lack of appropriate evaluation methods renders prognostics meaningless. Evaluation methods currently used in the research community are not standardized and in many cases do not sufficiently assess key performance aspects expected out of a prognostics algorithm. In this paper we introduce several new evaluation metrics tailored for prognostics and show that they can effectively evaluate various algorithms as compared to other conventional metrics. Specifically four algorithms namely; Relevance Vector Machine (RVM), Gaussian Process Regression (GPR), Artificial Neural Network (ANN), and Polynomial Regression (PR) are compared. These algorithms vary in complexity and their ability to manage uncertainty around predicted estimates. Results show that the new metrics rank these algorithms in different manner and depending on the requirements and constraints suitable metrics may be chosen. Beyond these results, these metrics offer ideas about how metrics suitable to prognostics may be designed so that the evaluation procedure can be standardized. 1

Saxena, Abhinav↗