Search NASA⌕ Search

SEARCH · Search NASA

Results for “Testing metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

The International Space Station, Optical Communications, and Delay Tolerant Networking: Towards A Solar System Internet Architecture

As Delay Tolerant Networking (DTN) matures as a software product its use cases have extended to infrastructural and architectural studies, bringing DTN closer to a widespread, operational technology. While it is known that a DTN can be configured, given complete information about a system of nodes, to provision a system with a complete network capability, what remains is to tackle practical considerations such as scalability, networking best practices, and how to establish service providers. In this paper, we document progress towards a scalable DTN architecture that was tested across multiple organizational and project boundaries; our approach was tested across nodes flying on the International Space Station (ISS) connected to a ground network. The network consists of four main network areas: the ISS, the White Sands Complex (WSC), the Mission Cloud Platform (MCP), and NASA’s Huntsville Operations Support Center (HOSC). The connections on the ground (WSC, MCP, and HOSC) are straightforward technically, but represent administrative and non-technical challenges that had to be overcome. The connection from the ISS to WSC was realized through a hybrid optical/RF means. In particular, NASA’s Integrated Laser Communications Relay Demonstration (LCRD) can form a relay from lowEarth-orbit (LEO) to the Earth, and was used in conjunction with the LCRD LEO User Modem and Amplifier Terminal (ILLUMA-T) on the ISS to form a link to one of three geographically diverse ground stations. A major goal was to achieve communications while keeping the ISS nodes unaware of ground station choices and scheduling. These disparate links were networked using High-rate DTN (HDTN), which enabled the desired architecture. The network architecture, tests, HDTN performance metrics, and general observations made are all discussed in detail, including a discussion on the successful utilization of the DTN security standard known as DTN Bundle Protocol Security (BPSec). We conclude with suggestions for next steps, and in particular focus on extending this architecture to the Near Space Network (NSN) and the upcoming LunaNet.

Alan Hylton↗
NASA NTRSDate not supplied

Rapid Aero Modeling for Urban Air Mobility Aircraft in Computational Experiments

Rapid Aero Modeling (RAM) applied to computational testing, RAM-C, is an approach to efficiently and automatically obtain aerodynamic models during computational investigations. RAM-C is designed to estimate models appropriate for flight dynamics studies and simulations. The approach responds to a demand for experimental efficiency and model fidelity that has increased with growing aircraft complexity and aerodynamic nonlinearities associated with hybrid and electric vertical takeoff and landing (eVTOL) aircraft. In an Urban Air Mobility (UAM) transportation system, it is expected that aircraft will embrace many features from both airplanes and rotorcraft. These vehicles present many more factors than conventional aircraft which can lead to increased computational costs and missed key factor interactions when applying traditional testing and modeling methods. RAM-C provides feedback loops around computational codes to rapidly guide testing toward aerodynamic models meeting user-defined fidelity goals. It combines and extends concepts from design of experiment theory and aircraft system identification theory that allow the user the freedom to choose, in advance of the test, a specific level of fidelity in terms of prediction error. RAM-C only collects enough data required to meet the user-specified prediction error requirements thus saving computational time and resources. The overall achievable fidelity of the final model also depends on the accuracy of the test facility, or in this case, the computational modeling approach. Previous studies to support development of the RAM-T process were conducted in wind tunnel tests to assess potential metrics, algorithms, and procedures. This paper presents results from the next steps taken and tests conducted for the development of RAM-C technology and highlights some of the unique features of RAM applied eVTOL configurations in a computational study.

Aerodynamics↗
NASA NTRS1985

Preliminary evaluation of a micro-based repeated measures testing system

A need exists for an automated performance test system to study the effects of various treatments which are of interest to the aerospace medical community, i.e., the effects of drugs and environmental stress. The ethics and pragmatics of such assessment demand that repeated measures in small groups of subjects be the customary research paradigm. Test stability, reliability-efficiency and factor structure take on extreme significance; in a program of study by the U.S. Navy, 80 percent of 150 tests failed to meet minimum metric requirements. The best is being programmed on a portable microprocessor and administered along with tests in their original formats in order to examine their metric properties in the computerized mode. Twenty subjects have been tested over four replications on a 6.0 minute computerized battery (six tests) and which compared with five paper and pencil marker tests. All tests achieved stability within the four test sessions, reliability-efficiencies were high (r greater than .707 for three minutes testing), and the computerized tests were largely comparable to the paper and pencil version from which they were derived. This computerized performance test system is portable, inexpensive and rugged.

Kennedy, Robert S.↗
NASA NTRS2017

Kepler Planet Detection Metrics: Per-Target Flux-Level Transit Injection Tests of TPS for Data Release 25

Quantifying the ability of a transiting planet survey to recover transit signals has commonly been accomplished through Monte-Carlo injection of transit signals into the observed data and subsequent running of the signal search algorithm (Gilliland et al., 2000; Weldrake et al., 2005; Burke et al., 2006). In order to characterize the performance of the Kepler pipeline (Twicken et al., 2016; Jenkins et al., 2017) on a sample of over 200,000 stars, two complementary injection and recovery tests are utilized:1. Injection of a single transit signal per target into the image or pixel-level data, hereafter referred to as pixel-level transit injection (PLTI), with subsequent processing through the Photometric Analysis (PA), Presearch Data Conditioning (PDC), Transiting Planet Search (TPS), and Data Validation (DV) modules of the Kepler pipeline. The PLTI quantification of the Kepler pipeline's completeness has been described previously by Christiansen et al. (2015, 2016); the completeness of the final SOC 9.3 Kepler pipeline acting on the Data Release 25 (DR25) light curves is described by Christiansen (2017).2. Injection of multiple transit signals per target into the normalized flux time series data with a subsequent transit search using a stream-lined version of the Transiting Planet Search (TPS) module. This test, hereafter referred to as flux-level transit injection (FLTI), is the subject of this document. By running a heavily modified version of TPS, FLTI is able to perform many injections on selected targets and determine in some detail which injected signals are recoverable. Significant numerical efficiency gains are enabled by precomputing the data conditioning steps at the onset of TPS and limiting the search parameter space (i.e., orbital period, transit duration, and ephemeris zero-point) to a small region around each injected transit signal.The PLTI test has the advantage that it follows transit signals through all processing steps of the Kepler pipeline, and the recovered signals can be further classified as planet candidates or false positives in the exact same manner as detections from the nominal (i.e., observed) pipeline run (Twicken et al., 2016, Thompson et al., in preparation). To date, the PLTI test has been the standard means of measuring pipeline completeness averaged over large samples of targets (Christiansen et al., 2015, 2016; Christiansen, 2017). However, since the PLTI test uses only one injection per target, it does not elucidate individual-target variations in pipeline completeness due to differences in stellar properties or astrophysical variability. Thus, we developed the FLTI test to provide a numerically efficient way to fully map individual targets and explore the performance of the pipeline in greater detail. The FLTI tests thereby allow a thorough validation of the pipeline completeness models (such as window function (Burke and Catanzarite, 2017a), detection efficiency (Burke Catanzarite, 2017b), etc.) across the spectrum of Kepler targets (i.e., various astrophysical phenomena and differences in instrumental noise). Tests during development of the FLTI capability revealed that there are significant target-to-target variations in the detection efficiency.

DR25↗
NASA NTRSDate not supplied

High-Altitude ADS-B/GPS LPV Flight Tests on a NASA ER-2 Research Airplane

The research presented in this paper describes the conceptual design of a system architecture that integrates Automatic Dependent Surveillance-Broadcast (ADS-B) and Global Positioning System (GPS) Localizer Performance Vertical (LPV) guidance technology onto a unique high-altitude research airplane: a United States Air Force (USAF) / Lockheed Martin (Bethesda, Maryland) Aeronautics U-2S airplane. The design features modern display capabilities to provide air-to-air surveillance and precision navigation, to adhere to Federal Aviation Administration (FAA) certification standards for operations in upper Class E airspace. The National Aeronautics and Space Administration (NASA) variant of the U-2S, now called the Earth Resources (ER-2) airplane, remains unrivaled in the art of sustained high-altitude flight for scientific expeditions. Capable of routinely cruising above flight level (FL) 650 that had been considered, at inception, the domain of only the most elite experimental research aircraft types. Nicknamed the Dragon Lady, this U-2S research testbed is still one of the most advanced aircraft in the world. The exceptional military design of the vehicle, security guidelines, and the performance envelope of the ER-2 posed unique challenges to the integration of modern civilian avionics. ADS-B epitomizes the next generation of surveillance technology, incorporating both air and ground aspects. ADS-B provides air traffic control (ATC) with a more accurate picture of the three-dimensional positioning of aircraft in various phases of flight, including en route, terminal, approach, and ground operations. The airborne surveillance system broadcasts its identification, position, altitude, velocity, and other information. GPS LPV represents a significant advancement in aviation technology, emphasizing the pivotal role that GPS and Performance-Based Navigation concepts will play in the foreseeable future. This technology represents a shift from sensor-based navigation to performance-based navigation, allowing for more flexible and efficient use of airspace. This research described herein is structured as follows: Section II, “Systems Background,” provides a systems background and description of an ADS-B and GPS LPV system equipped on the high-altitude ER-2 research airplane to satisfy the FAA airworthiness requirements for high-altitude flight operations. Section III, “Flight Test System,” describes the flight-test airplane systems. Section IV, “Analysis of GPS SBAS, Safety, and Ground Tests,” provides an overview of the GPS Satellite-Based Augmentation System (SBAS) and an analysis of the GPS LPV metrics, safety, and ground tests. Section V, “High-Altitude Flight Tests,” describes the high-altitude flight tests, including 3 flight-test results, analysis, human factors, and lessons learned. Section VI, the conclusion, draws insights from the lessons learned, discusses the design challenges associated with ADS-B and GPS LPV, and showcases the paramount significance of these pivotal technologies in aircraft surveillance and navigation.

Ricardo A. Arteaga↗
NASA NTRS1995

Rotorcraft flight control design using quantitative feedback theory and dynamic crossfeeds

A multi-input, multi-output controls design with robust crossfeeds is presented for a rotorcraft in near-hovering flight using quantitative feedback theory (QFT). Decoupling criteria are developed for dynamic crossfeed design and implementation. Frequency dependent performance metrics focusing on piloted flight are developed and tested on 23 flight configurations. The metrics show that the resulting design is superior to alternative control system designs using conventional fixed-gain crossfeeds and to feedback-only designs which rely on high gains to suppress undesired off-axis responses. The use of dynamic, robust crossfeeds prior to the QFT design reduces the magnitude of required feedback gain and results in performance that meets current handling qualities specifications relative to the decoupling of off-axis responses. The combined effect of the QFT feedback design following the implementation of low-order, dynamic crossfeed compensator successfully decouples ten of twelve off-axis channels. For the other two channels it was not possible to find a single, low-order crossfeed that was effective.

Cheng, Rendy P.↗
NASA NTRS2001

The Application of Time-Frequency Methods to HUMS

This paper reports the study of four time-frequency transforms applied to vibration signals and presents a new metric for comparing them for fault detection. The four methods to be described and compared are the Short Time Frequency Transform (STFT), the Choi-Williams Distribution (WV-CW), the Continuous Wavelet Transform (CWT) and the Discrete Wavelet Transform (DWT). Vibration data of bevel gear tooth fatigue cracks, under a variety of operating load levels, are analyzed using these methods. The new metric for automatic fault detection is developed and can be produced from any systematic numerical representation of the vibration signals. This new metric reveals indications of gear damage with all of the methods on this data set. Analysis with the CWT detects mechanical problems with the test rig not found with the other transforms. The WV-CW and CWT use considerably more resources than the STFT and the DWT. More testing of the new metric is needed to determine its value for automatic fault detection and to develop methods of setting the threshold for the metric.

Pryor, Anna H.↗
NASA NTRS2020

Urban Air Mobility Noise: Current Practice, Gaps, and Recommendations

In 2018, NASA formed an Urban Air Mobility Noise Working Group to assemble noise experts from industry, universities and government agencies to identify, discuss, and address urban air mobility (UAM) noise issues. This paper presents a set of high-level goals intended to address barriers associated with UAM noise that may hamper their entry into service. It summarizes the current practice, identifies gaps in the current practice, and makes recommendations to address the gaps to achieve the high-level goals in four areas of interest: Tools and Technologies, Ground and Flight Testing, Human Response and Metrics, and Regulation and Policy.

urban air mobility↗
NASA NTRS2004

Optical design for the Laser Astrometric Test of Relativity

This paper discusses the Laser Astrometric Test of Relativity (LATOR) mission. LATOR is a Michelson-Morley-type experiment designed to test the pure tensor metric nature of gravitation the fundamental postulate of Einstein's theory of general relativity. With its focus on gravity's action on light propagation it complements other tests which rely on the gravitational dynamics of bodies.

Laser Astrometric Test of Relativity (LATOR)↗
NASA NTRSDate not supplied

Thermal Desktop Modeling of 2016 CRYOTE-2 Tank Chill and Fill Testing

The storing and transfer of cryogenic propellants is an enabling technology for NASA and industry as civilization moves to expand future missions into low Earth orbit and beyond. Several ground propellant transfer tests have been conducted in past decades examining various tank chilldown and fill methods. Creating accurate models using these historical datasets is a vital step toward developing appropriate modeling tools to form pre-test predictions for subsequent ground and flight propellant transfers. This paper presents model validation of Thermal Desktop using data from the 2016 CRYOTE-2 liquid nitrogen transfer experiments whose purpose was to demonstrate the chill and fill process in a spherical receiver tank. Tank pressure, fill level, and wall temperature were all modeled and compared with 8 test cases using two modeling methods, Mass Controlled, and Pressure Controlled. This test series is the hardest Thermal Desktop tank chill and fill validation case to-date, yet the model was able to accurately predict each of these metrics for the 2016 tests within 28%, 10%, 20% respectively, for pressure, fill level, and wall temperature using the Mass Controlled model, and 29%, 79%, and 12%, respectively for the Pressure Controlled model.

Cryogenic Tank Chilldown↗
NASA NTRS2004

COTS Ceramic Chip Capacitors: An Evaluation of the Parts and Assurance Methodologies

Commercial-Off-The-Shelf (COTS) multilayer ceramic chip capacitors (MLCCs) are continually evolving to reduce physical size and increase volumetric efficiency. Designers of high reliability aerospace and military systems are attracted to these attributes of COTS MLCCs and would like to take advantage of them while maintaining the high standards for long-term reliable operation they are accustomed io when selecting military qualified established reliability (MIL-ER) MLCCs. However, MIL-ER MLCCs are not available in the full range of small chip sizes with high capacitance as found in today's COTS MLCCs. The objectives for this evaluation were to assess the long-term performance of small case size COTS MLCCs and to identify effective, lower-cost product assurance methodologies. Fifteen (15) lots of COTS X7R dielectric MLCCs from four (4) different manufacturers and two (2) MIL-ER BX dielectric MLCCs from two (2) of the same manufacturers were evaluated. Both 0805 and 0402 chip sizes were included. Several voltage ratings were tested ranging from a high of 50 volts to a low of 6.3 volts. The evaluation consisted of a comprehensive screening and qualification test program based upon MIL-PRF-55681 (i.e., voltage conditioning, thermal shock, moisture resistance, 2000-hour life test, etc.). In addition, several lot characterization tests were performed including Destructive Physical Analysis (DPA), Highly Accelerated Life Test (HALT) and Dielectric Voltage Breakdown Strength. The data analysis included a comparison of the 2000-hour life test results (used as a metric for long-term performance) relative to the screening and characterization test results. Results of this analysis indicate that the long-term life performance of COTS MLCCs is variable -- some lots perform well, some lots perform poorly. DPA and HALT were found to be promising lot characterization tests to identify substandard COTS MLCC lots prior to conducting more expensive screening and qualification tests. The results indicate that lot- specific screening and qualification are still recommended for high reliability applications. One significant and concerning observation is that MIL- type voltage conditioning (100 hours at twice rated voltage, 125 C) was not an effective screen in removing infant mortality parts for the particular lots of COTS MLCCs evaluated.

Brusse, Jay A.↗
NASA NTRS1989

Multi-version software reliability through fault-avoidance and fault-tolerance

A number of experimental and theoretical issues associated with the practical use of multi-version software to provide run-time tolerance to software faults were investigated. A specialized tool was developed and evaluated for measuring testing coverage for a variety of metrics. The tool was used to collect information on the relationships between software faults and coverage provided by the testing process as measured by different metrics (including data flow metrics). Considerable correlation was found between coverage provided by some higher metrics and the elimination of faults in the code. Back-to-back testing was continued as an efficient mechanism for removal of un-correlated faults, and common-cause faults of variable span. Software reliability estimation methods was also continued based on non-random sampling, and the relationship between software reliability and code coverage provided through testing. New fault tolerance models were formulated. Simulation studies of the Acceptance Voting and Multi-stage Voting algorithms were finished and it was found that these two schemes for software fault tolerance are superior in many respects to some commonly used schemes. Particularly encouraging are the safety properties of the Acceptance testing scheme.

Vouk, Mladen A.↗
NASA NTRSDate not supplied

SLS Integrated Modal Test Uncertainty Quantification using the Hybrid Parametric Variation Method

Uncertainty in structural loading during launch is a significant concern in the development of spacecraft and launch vehicles. Small variations in launch vehicle and payload mode shapes and their interaction can result in significant variation in system loads. In many cases involving large aerospace systems it is difficult, not economical, or impossible to perform a system modal test. However, it is still vital to obtain test results that can be compared with analytical predictions to validate models. Instead, the “Building Block Approach” is used in which system components are tested individually. Component models are correlated and updated to agree as best they can with test results. The Space Launch System consists of a number of components that are assembled into a launch vehicle. Finite element models of the components are developed, reduced to Hurty/Craig-Bampton models and assembled to represent different phases of flight. The only opportunity to obtain modal test data from an assembled Space Launch System will be during the Integrated Modal Test. There is always uncertainty in every model, which flows into uncertainty in predicted system results. Uncertainty Quantification is used to determine statistical bounds on prediction accuracy based on model uncertainty. For the Space Launch System, model uncertainty is at the Hurty/Craig-Bampton component level. Uncertainty in the Hurty/Craig-Bampton components is quantified using the hybrid parametric variation approach that combines parametric and nonparametric uncertainty. Uncertainty in model form is one of the biggest contributors to uncertainty in complex built-up structures. This type of uncertainty cannot be represented by variations infinite element model input parameters and thus cannot be included in a parametric approach. However, model-form uncertainty can be modeled using a nonparametric approach based on random matrix theory. The hybrid parametric variation method requires the selection of dispersion values for the Hurty/Craig-Bampton fixed-interface eigenvalues, and the Hurty/Craig-Bampton stiffness matrices. Component test/analysis frequency error is used to identify the fixed-interface eigenvalue dispersions, while test/analysis cross-orthogonality is used to identify stiffness dispersion values. The hybrid parametric variation uncertainty quantification approach is applied to the Space Launch System Integrated Modal Test configuration. Monte Carlo analysis is performed, and statistics are determined for modal correlation metrics, frequency response from Integrated Modal Test shakers to selected accelerometers, as well as other metrics for determining how well target modes are excited and identified. If the predicted uncertainty envelopes future Integrated Modal Test results, then there will be increased confidence in the utility of the component-based hybrid parametric variation uncertainty quantification approach.

Uncertainty Quantification↗
NASA NTRS1986

Software unit testing in Ada environment

A validation procedure for the Ada binding of the Graphical Kernel System (GKS) is being developed. PRIOR Data Sciences is also producing a version of the GKS written in Ada. These major software engineering projects will provide an opportunity to demonstrate a sound approach for software testing in an Ada environment. The GKS/Ada validation capability will be a collection of test programs and data, and test management guidelines. These products will be used to assess the correctness, completeness, and efficiency of any GKS/Ada implementation. The GKS/Ada developers will be able to obtain the validation software for their own use. It is anticipated that this validation software will eventually be taken over by an independent standards body to provide objective assessments of GKS/Ada implementations, using an approach similar to the validation testing currently applied to Ada compilers. In the meantime, if requested, this validation software will be used to assess GKS/Ada products. The second project, implementation of GKS using the Ada language, is a conventional software engineering tasks. It represents a large body of Ada code and has some interesting testing problems associated with automatic testing of graphics routines. Here the normal test practices which include automated regression testing, independent quality assistance, test configuration management, and the application of software quality metrics will be employed. The software testing methods emphasize quality enhancement and automated procedures. Ada makes some aspects of testing easier, and introduces some concerns. These issues are addressed.

Warnock, Glenn↗
NASA NTRS1989

Experimental constraints on metric and non-metric theories of gravity

Experimental constraints on metric and non-metric theories of gravitation are reviewed. Tests of the Einstein Equivalence Principle indicate that only metric theories of gravity are likely to be viable. Solar system experiments constrain the parameters of the weak field, post-Newtonian limit to be close to the values predicted by general relativity. Future space experiments will provide further constraints on post-Newtonian gravity.

Will, Clifford M.↗
NASA NTRS1983

Software development predictors, error analysis, reliability models and software metric analysis

The use of dynamic characteristics as predictors for software development was studied. It was found that there are some significant factors that could be useful as predictors. From a study on software errors and complexity, it was shown that meaningful results can be obtained which allow insight into software traits and the environment in which it is developed. Reliability models were studied. The research included the field of program testing because the validity of some reliability models depends on the answers to some unanswered questions about testing. In studying software metrics, data collected from seven software engineering laboratory (FORTRAN) projects were examined and three effort reporting accuracy checks were applied to demonstrate the need to validate a data base. Results are discussed.

Basili, Victor↗
NASA NTRSDate not supplied

Advances in Integrally Formed Metallic Structures

NASA Langley Research Center (LaRC) is applying flow forming technology to aerospace aluminum alloys for the manufacture of integrally stiffened cylinders that meet aircraft fuselage or launch vehicle specifications. High-strength 2xxx series alloys, in various temper/product form conditions, are being compared with medium-strength, very formable AA 6061 as the benchmark. Screening of candidate materials traditionally relies on empirical forming trials, but more recently on metrics derived from constant nominal strain rate tensile testing. In this work, the established hierarchy of formability metrics is adopted as the baseline; tensile area reduction, work hardening exponent, and modulus of resilience. The suite of metrics can be expanded based on phenomenological and mechanistic considerations of plastic flow instabilities to incorporate anisotropy ratio, rate of work hardening, and dynamic strain aging effects. Separate research thrusts at LaRC are aimed at formulating a protocol for scalable production of fully formed, defect-free components: 1) quantifying the extended set of formability metrics from constant and jump strain rate tensile tests with digital image correlation (DIC); 2) using statistically designed experiments on a vertical spin/flow forming facility (WF VUD-600®) to isolate the effect of convenient process variables, such as axial feed rate and thickness reduction; and 3) predicting the effect of feed ratio (roller translation relative to mandrel rotation speeds) on material response via process modeling using the DEFORM® simulation software. Current results will be discussed in terms of correlating intrinsic material properties with the bulk workability characteristics to successfully fabricate cylinders with complex cross-sections.

Aluminum↗