SEARCH · Search NASA

Results for “test frameworks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRS2009

Model Based Analysis and Test Generation for Flight Software

We describe a framework for model-based analysis and test case generation in the context of a heterogeneous model-based development paradigm that uses and combines Math- Works and UML 2.0 models and the associated code generation tools. This paradigm poses novel challenges to analysis and test case generation that, to the best of our knowledge, have not been addressed before. The framework is based on a common intermediate representation for different modeling formalisms and leverages and extends model checking and symbolic execution tools for model analysis and test case generation, respectively. We discuss the application of our framework to software models for a NASA flight mission.

Pasareanu, Corina S.↗

NASA NTRSDate not supplied

Component Level Regression Testing in a Hierarchical Architecture

The Goddard Earth Observing System (GEOS) is an Earth system model consisting of a large suite of individual model components that can be coupled in a flexible manner to investigate a variety of Earth science issues. Specific GEOS model configurations are composed as a hierarchical collection of components based on the Earth System Modeling Framework (ESMF). Regression testing of GEOS is currently limited to (1) full system tests that are poor at isolating specific defects and (2) a suite of unit tests which have very limited coverage. As part of our approach to improve upon the current testing situation, we have prototyped the capability to perform regression tests on individual GEOS components by leveraging and extending existing checkpoint/restart capabilities. In our implementation, each ESMF component has 3 states: Import (what it needs to run), Export (which it needs to provide to other components), and Internal (the component state proper). By capturing, Import, Export and Internal states for a given component during a ull run of GEOS, a generic driver can then rerun the component offline and compare expected exports with those that have been saved. The hierarchical structure of GEOS introduces an interesting wrinkle when trying to test components that in turn drive interacting child components. To fully isolate a parent component, we use the approach of software mocks, in which the exports of children are also saved during the initial capture run of GEOS. Then when testing the parent component, the children components are replaced by a generic mock component that produces exports from the previously saved data and ensures that that all interdependencies among children components are satisfied.

Thomas Clune↗

NASA NTRSDate not supplied

Component Level Testing in a Hierarchical Architecture

Tom Clune↗

NASA NTRS1990

Users manual for the Automated Performance Test System (APTS)

The characteristics of and the user information for the Essex Automated Performance Test System (APTS) computer-based portable performance assessment battery are given. The battery was developed to provide a menu of performance test tapping the widest possible variety of human cognitive and motor functions, implemented on a portable computer system suitable for use in both laboratory and field settings for studying the effects of toxic agents and other stressors. The manual gives guidance in selecting, administering and scoring tests from the battery, and reviews the data and studies underlying the development of the battery. Its main emphasis is on the users of the battery - the scientists, researchers and technicians who wish to examine changes in human performance across time or as a function of changes in the conditions under which test data are obtained. First the how to information needed to make decisions about where and how to use the battery is given, followed by the research background supporting the battery development. Further, the development history of the battery focuses largely on the logical framework within which tests were evaluated.

Lane, N. E.↗

NASA NTRS2009

Proceedings of the First NASA Formal Methods Symposium

Topics covered include: Model Checking - My 27-Year Quest to Overcome the State Explosion Problem; Applying Formal Methods to NASA Projects: Transition from Research to Practice; TLA+: Whence, Wherefore, and Whither; Formal Methods Applications in Air Transportation; Theorem Proving in Intel Hardware Design; Building a Formal Model of a Human-Interactive System: Insights into the Integration of Formal Methods and Human Factors Engineering; Model Checking for Autonomic Systems Specified with ASSL; A Game-Theoretic Approach to Branching Time Abstract-Check-Refine Process; Software Model Checking Without Source Code; Generalized Abstract Symbolic Summaries; A Comparative Study of Randomized Constraint Solvers for Random-Symbolic Testing; Component-Oriented Behavior Extraction for Autonomic System Design; Automated Verification of Design Patterns with LePUS3; A Module Language for Typing by Contracts; From Goal-Oriented Requirements to Event-B Specifications; Introduction of Virtualization Technology to Multi-Process Model Checking; Comparing Techniques for Certified Static Analysis; Towards a Framework for Generating Tests to Satisfy Complex Code Coverage in Java Pathfinder; jFuzz: A Concolic Whitebox Fuzzer for Java; Machine-Checkable Timed CSP; Stochastic Formal Correctness of Numerical Algorithms; Deductive Verification of Cryptographic Software; Coloured Petri Net Refinement Specification and Correctness Proof with Coq; Modeling Guidelines for Code Generation in the Railway Signaling Context; Tactical Synthesis Of Efficient Global Search Algorithms; Towards Co-Engineering Communicating Autonomous Cyber-Physical Systems; and Formal Methods for Automated Diagnosis of Autosub 6000.

Denney, Ewen↗

NASA NTRS2021

Lunar Asset Messaging and On Orbit Navigation

NASA has titled its 2020 thrust for the Moon, Artemis. The increased focus on the Moon as a destination for future human and robotic expeditions necessitates general purpose navigational and communications infrastructure reducing their complexity to help establish a sustained presence. A framework through which Lunar missions can relay communications and localize their positions shifts the burden from the individual mission and enables resource allocation tailored to mission-specific goals. During the summer of 2020, student interns under the Innovation to Flight (i2F) program at the National Aeronautics and Space Administration’s (NASA) Jet Propulsion Laboratory (JPL) in collaboration with the University of Colorado Boulder designed, built, and tested a prototype framework capable of providing surface assets with communication and positioning services. The team utilized the existing i2F CubeSat bus in addition to developing several CubeSat engineering development units (EDUs), a ground vehicle, and a ground station to simulate a scenario in which a lunar surface mission is supported by these services. A primary goal of the summer was to develop a method for localizing the ground vehicle through trilateration. Distances are inferred from the round-trip time of flight (ToF) of radio signals between an asset and several elements. Signals were sent and received using LimeSDR software defined radios on-board both the ground vehicle and the EDUs; ToF and trilateration were calculated on a Qualcomm Snapdragon development board located within the LA MOON payload data system. The ModalAI chipset on the Qualcomm was instrumental in executing visual based position estimation. Communications was facilitated through a bent-pipe approach addressing the NASA requirement to provide solutions for in communication denied locations. The ground vehicle relayed information to other surface assets in addition to its ground station through the supporting constellation. This project demonstrates the feasibility of a lunar CubeSat constellation for the support of surface assets and explores packaging and operations of the components critical to trilateration and bent-pipe communication into a standard CubeSat form factor. When implemented, this framework will open a door for new surface missions designed with lower power requirements and increased operational access.

Huang, Calvin↗

NASA NTRSDate not supplied

A Framework for Evaluating Climate Model Performance Metrics

The CMIP5 archive contains future climate projections from over 50 models provided by dozens of modeling centers from around the world. Individual model projections, however, are subject to biases created by structural model uncertainties. As a result, ensemble averaging of multiple models is often used to add value to model projections: consensus projections have been shown to consistently outperform individual models. Previous reports for the IPCC establish climate change projections based on an equal-weighted average of all model projections. However, certain models reproduce climate processes better than other models. Should models be weighted based on performance? Unequal ensemble averages have previously been constructed using a variety of mean state metrics. What metrics are most relevant for constraining future climate projections? This project develops a framework for systematically testing metrics in models to identify optimal metrics for unequal weighting multi-model ensembles. A unique aspect of this project is the construction and testing of climate process-based model evaluation metrics. A climate process-based metric is defined as a metric based on the relationship between two physically related climate variables?e.g., outgoing longwave radiation and surface temperature. Metrics are constructed using high-quality Earth radiation budget data from NASA's Clouds and Earth's Radiant Energy System (CERES) instrument and surface temperature data sets. It is found that regional values of tested quantities can vary significantly when comparing weighted and unweighted model ensembles. For example, one tested metric weights the ensemble by how well models reproduce the time-series probability distribution of the cloud forcing component of reflected shortwave radiation. The weighted ensemble for this metric indicates lower simulated precipitation (up to .7 mm/day) in tropical regions than the unweighted ensemble: since CMIP5 models have been shown to overproduce precipitation, this result could indicate that the metric is effective in identifying models which simulate more realistic precipitation. Ultimately, the goal of the framework is to identify performance metrics for advising better methods for ensemble averaging models and create better climate predictions.

Noel C Baker↗

NASA NTRS1993

Systems aspects of COBE science data compression

A general approach to compression of diverse data from large scientific projects has been developed and this paper addresses the appropriate system and scientific constraints together with the algorithm development and test strategy. This framework has been implemented for the COsmic Background Explorer spacecraft (COBE) by retrofitting the existing VAS-based data management system with high-performance compression software permitting random access to the data. Algorithms which incorporate scientific knowledge and consume relatively few system resources are preferred over ad hoc methods. COBE exceeded its planned storage by a large and growing factor and the retrieval of data significantly affects the processing, delaying the availability of data for scientific usage and software test. Embedded compression software is planned to make the project tractable by reducing the data storage volume to an acceptable level during normal processing.

Freedman, I.↗

NASA NTRS2013

Assessment Environment for Complex Systems Software Guide

This Software Guide (SG) describes the software developed to test the Assessment Environment for Complex Systems (AECS) by the West Virginia High Technology Consortium (WVHTC) Foundation's Mission Systems Group (MSG) for the National Aeronautics and Space Administration (NASA) Aeronautics Research Mission Directorate (ARMD). This software is referred to as the AECS Test Project throughout the remainder of this document. AECS provides a framework for developing, simulating, testing, and analyzing modern avionics systems within an Integrated Modular Avionics (IMA) architecture. The purpose of the AECS Test Project is twofold. First, it provides a means to test the AECS hardware and system developed by MSG. Second, it provides an example project upon which future AECS research may be based. This Software Guide fully describes building, installing, and executing the AECS Test Project as well as its architecture and design. The design of the AECS hardware is described in the AECS Hardware Guide. Instructions on how to configure, build and use the AECS are described in the User's Guide. Sample AECS software, developed by the WVHTC Foundation, is presented in the AECS Software Guide. The AECS Hardware Guide, AECS User's Guide, and AECS Software Guide are authored by MSG. The requirements set forth for AECS are presented in the Statement of Work for the Assessment Environment for Complex Systems authored by NASA Dryden Flight Research Center (DFRC). The intended audience for this document includes software engineers, hardware engineers, project managers, and quality assurance personnel from WVHTC Foundation (the suppliers of the software), NASA (the customer), and future researchers (users of the software). Readers are assumed to have general knowledge in the field of real-time, embedded computer software development.

tests↗

NASA NTRS1971

Multipath/RFI/modulation study for DRSS-RFI problem: Voice coding and intelligibility testing for a satellite-based air traffic control system

Analog and digital voice coding techniques for application to an L-band satellite-basedair traffic control (ATC) system for over ocean deployment are examined. In addition to performance, the techniques are compared on the basis of cost, size, weight, power consumption, availability, reliability, and multiplexing features. Candidate systems are chosen on the bases of minimum required RF bandwidth and received carrier-to-noise density ratios. A detailed survey of automated and nonautomated intelligibility testing methods and devices is presented and comparisons given. Subjective evaluation of speech system by preference tests is considered. Conclusion and recommendations are developed regarding the selection of the voice system. Likewise, conclusions and recommendations are developed for the appropriate use of intelligibility tests, speech quality measurements, and preference tests with the framework of the proposed ATC system.

Birch, J. N.↗

NASA NTRS1991

Pre-main-sequence evolution

The primary uncertainty that remains concerning the premain sequence (PMS) stage of stellar evolution is the possible role of accretion in the hydrodynamic stage. Possible PMS evolution scenarios are presented with a view to extant or potentially obtainable observational data that may unravel the details of PMS evolution. Attention is given to recent evolution calculations which encompass the role of rotation; it is concluded that the rotational feature of PMS can be tested within the framework of the chemical abundance of trace elements in the solar system, on the sun, and on solar analogs of various ages.

Sofia, S.↗

NASA NTRS1993

Thermal-mechanical behavior of high precision composite mirrors

Composite mirror panels were designed, constructed, analyzed, and tested in the framework of a NASA precision segmented reflector task. The deformations of the reflector surface during the exposure to space enviroments were predicted using a finite element model. The composite mirror panels have graphite-epoxy or graphite-cyanate facesheets, separated by an aluminum or a composite honeycomb core. It is pointed out that in order to carry out detailed modeling of composite mirrors with high accuracy, it is necessary to have temperature dependent properties of the materials involved and the type and magnitude of manufacturing errors and material nonuniformities. The structural modeling and analysis efforts addressed the impact of key design and materials parameters on the performance of mirrors.

Kuo, C. P.↗

NASA NTRS2001

Advances in Global Water Cycle Science Made Possible by Global Precipitation Mission (GPM)

Within this decade the internationally sponsored Global Precipitation Mission (GPM) will take an important step in creating a global precipitation observing system from space. One perspective for understanding the nature of GPM is that it will be a hierarchical system of datastreams from very high caliber combined dual frequency radar/passive microwave (PMW) rain-radiometer retrievals, to high caliber PMW rain-radiometer only retrievals, and on to blends of the former datastreams with other less-high caliber PMW-based and IR-based rain retrievals. Within the context of NASA's role in global water cycle science and its own Global Water & Energy Cycle (GWEC) program, GPM is the centerpiece mission for improving our understanding of the global water cycle from a space-based measurement perspective. One of the salient problems within our current understanding of the global water and energy cycle is determining whether a change in the rate of the water cycle is accompanying changes in global temperature. As there are a number of ways in which to define a rate-change of the global water cycle, it is not entirely clear as to what constitutes such a determination, This paper presents an overview of the Global Precipitation Mission and how its datasets can be used in a set of quantitative tests within the framework of the oceanic and continental water budget equations to determine comprehensively whether substantive rate changes do accompany perturbations in global temperatures and how such rate changes manifest themselves in both water storage and water flux transport processes.

Smith, Eric A.↗

NASA NTRS2005

Line asymmetry in the Seyfert Galaxy NGC 3783

We have reanalyzed the 900 ks Chandra X-ray spectrum of NGC 3783, finding evidence on the asymmetry of the spectral absorption lines. The lines are fitted with a parametric expression that results from an analytical treatment of radiatively driven winds. The line asymmetry distribution derived from the spectrum is consistent with a non-spherical outflow with a finite optical depth. Within this scenario, our model explains the observed correlations between the line velocity shifts and the ionization parameter and between the line velocity shift and the line asymmetry. The present results may provide a framework for detailed testing of models for the dynamic and physical properties of warm absorber in Seyfert galaxies.

Ramirez, J. M.↗

NASA NTRS2005

Apex Reference Manual 3.0 Beta

Apex is a toolkit for constructing software that behaves intelligently and responsively in demanding task environments. Reflecting its origin at NASA where Apex continues to be developed, current applications include: a) Providing autonomous mission management and tactical control capabilities for unmanned aerial vehicles including an autonomous surveillance helicopter and a simulation prototype of an unmanned fixed-wing aircraft to be used for wildfire mapping; b) Simulating human air traffic controllers, pilots and astronauts to help predict how people might respond to changes in equipment or procedures; and c) Predicting the precise duration and sequence of routine human behaviors based on a human-computer interaction engineering technique called CPM-GOMS. Among Apex s components are a set of implemented reasoning services, such as those for reactive planning and temporal pattern recognition; a software architecture that embeds and integrates these services and allows additional reasoning elements to be added as extensions; a formal language for specifying agent knowledge; a simulation environment to facilitate prototyping and analysis; and Sherpa, a set of tools for visualizing autonomy logic and runtime behavior. In combination, these are meant to provide a flexible and usable framework for creating, testing, and deploying intelligent agent software. Overall, our goal in developing Apex is to lower economic barriers to developing intelligent software agents. New ideas about how to extend or modify the system are evaluated in terms of their impact in reducing the time, expertise, and inventiveness required to build and maintain applications. For example, potential enhancements to the AI reasoning capabilities in the system are reviewed not only for usefulness and distinctiveness, but also for their impact on the readability and general usability of Apex s behavior representation language (PDL) and on the transparency of resulting behavior. A second central part of our approach is to iteratively refine Apex based on lessons learned from as diverse a set of applications as possible. Many applications have been developed by users outside the core development team including engineers, researchers, and students. Usability is thus a central concern for every aspect of Apex visible to a user, including PDL, Sherpa, the Apex installation process, APIs, and user documentation. Apex users vary in their areas of expertise and in their familiarity with autonomy technology. Focusing on usability, a development philosophy summarized by the project motto "Usable Autonomy," has been important part of enabling diverse users to employ Apex successfully and to provide feedback needed to guide iterative, user-centered refinement.

Freed, Michael A.↗

NASA NTRS2012

Space Launch System Complex Decision-Making Process

The Space Shuttle program has ended and elements of the Constellation Program have either been cancelled or transitioned to new NASA exploration endeavors. The National Aeronautics and Space Administration (NASA) has worked diligently to select an optimum configuration for the Space Launch System (SLS), a heavy lift vehicle that will provide the foundation for future beyond low earth orbit (LEO) large-scale missions for the next several decades. From Fall 2010 until Spring 2011, an SLS decision-making framework was formulated, tested, fully documented, and applied to multiple SLS vehicle concepts at NASA from previous exploration architecture studies. This was a multistep process that involved performing figure of merit (FOM)-based assessments, creating Pass/Fail gates based on draft threshold requirements, performing a margin-based assessment with supporting statistical analyses, and performing sensitivity analysis on each. This paper focuses on the various steps and methods of this process (rather than specific data) that allowed for competing concepts to be compared across a variety of launch vehicle metrics in support of the successful completion of the SLS Mission Concept Review (MCR) milestone.

Lyles, Garry↗

NASA NTRS2019

MARGInS Model-Based Analysis of Realizable Goals in Systems

The high complexity of modern aircraft and spacecraft requires elaborate Verification and Validation (V&V) approaches to make sure that such complex systems work properly and reliably. MARGInS is a framework for the analysis, understanding, and prediction of the behavior of a complex, hybrid system. MARGInS contains a set of machine learning and statistical algorithms for multivariate clustering, treatment learning, critical factor determination, time-series analysis, event prediction, and safety-boundary detection and characterization. The framework supports system testing and can be configured to find novel features in test suites, determine classes of behavior, propose new experiments that can efficiently explore and characterize the boundaries between classes of system behavior, and to create visualizations and reports.

He, Yuning↗

NASA NTRSDate not supplied

Simulated Multipath Using Software Generated GPS Signals

Depending on the environment, multipath can be one of the largest error sources contributing to degradation in Global Navigation Satellite System (GNSS) (e.g., GPS) performance. Multipath is a phenomenon that occurs as radio signals reflect off of surfaces, such as buildings, producing multiple copies of the original signal. When this occurs with GPS signals, it results in one or more delayed signals arriving at the receiver with or without the on-time/direct GPS signal. The receiver measures the composite of these signals which, depending on the severity of the multipath, can substantially degrade the accuracy of the receiver's calculated position. Multipath is commonly experienced in cities due to tall buildings and its mitigation is an ongoing area of study. This research demonstrates a novel approach for simulating GPS multipath through the modification of an open-source tool, GPS-SDR-SIM. The resulting additional testing capability could allow for improved development of multipath mitigating technologies. Currently, open-source tools for simulating GPS signals are available and can be used in the testing and evaluation of GPS receiver equipment. These tools can generate GPS signals that, when used by a GPS receiver, result in computation of a position solution that was pre-determined at the time of signal generation. That is, the signals produced are properly formed for the pre-determined location and result in the receiver reporting that position. This allows for a GPS receiver under test to be exposed to various simulated locations and conditions without having to be physically subjected to them. Additionally, while these signals are generated by a software simulation, they can be processed by real or software defined GPS receivers. This work utilizes the GPS-SDR-SIM software tool for GPS signal generation and while this tool does implement some sources of error that are inherent to GPS, it cannot inject multipath. GPS-SDR-SIM was modified in this effort to produce additional copies of signals with pre-determined delays. These additional delayed signals mimic multipath and represent what happens to GPS signals in the real world as they reflect off of surfaces and arrive at a receiver in place of or alongside the direct GPS signal. A successful proof of concept was prototyped and demonstrated using this modified version of GPS-SDR-SIM to produce simulated GPS signals as well as additional simulated multipath signals. The generated data was processed using a software defined GPS receiver and it was found that the introduction of simulated multipath signals successfully produced the expected characteristics of a composite multipath signal. Further maturation of this work could allow for the development of a GPS receiver testing and evaluation framework and aid in the development of multipath mitigating technologies.

GPS↗