Search NASA⌕ Search

SEARCH · Search NASA

Results for “Software Test Tool”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16

Inspection and Verification of Domain Models with PlanWorks and Aver

When developing a domain model, it seems natural to bring the traditional informal tools of inspection and verification, debuggers and automated test suites, to bear upon the problems that will inevitably arise. Debuggers that allow inspection of registers and memory and stepwise execution have been a staple of software development of all sorts from the very beginning. Automated testing has repeatedly proven its considerable worth, to the extent that an entire design philosophy (Test Driven Development) has been developed around the writing of tests. Unfortunately, while not entirely without their uses, the limitations of these tools and the nature of the complexity of models and the underlying planning systems make the diagnosis of certain classes of problems and the verification of their solutions difficult or impossible. Debuggers provide a good local view of executing code, allowing a fine-grained look at algorithms and data. This view is, however, usually only at the level of the current scope in the implementation language, and the data-inspection capabilities of most debuggers usually consist of on-line print statements. More modem graphical debuggers offer a sort of tree view of data structures, but even this is too low-level and is often inappropriate for the kinds of structures created by planning systems. For instance, god or constraint networks are at best awkward when visualized as trees. Any any non-structural link between data structures, as through a lookup table, isn't captured at all. Further, while debuggers have powerful breakpointing facilities that are suitable for finding specific algorithmic errors, they have little use in the diagnosis of modeling errors.

Bedrax-Weiss, Tania↗

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗

In-Flight Evaluation of the Traffic Aware Planner on the NASA HU-25A Guardian Aircraft

NASA’s Traffic Aware Planner (TAP) software is a research-prototype decision support tool that provides pilots with time- and fuel-saving route recommendations that optimize their current trajectory. The software runs on a first-of-a-kind system architecture onboard three aircraft in revenue service conducting operational evaluations with a major domestic airline. Therefore, significant NASA-internal testing is required prior to releasing the software to the partner airline. This paper describes a flight test plan that exercises the functionality of the TAP software in a representative operational environment, describes the system architecture developed and implemented for the NASA Langley HU-25A Guardian aircraft to support the test objectives, presents outcomes of the flight test campaign, and discusses use cases that demonstrate the value of flight testing for this activity.Research into flight path optimization of transport aircraft conducted by the National Aeronautics and SpaceAdministration (NASA) has produced an operational concept known as Traffic Aware Strategic Aircrew Requests(TASAR) [1, 2]. This near-term concept [3] provides the aircrew with a flight deck decision support tool known asthe Traffic Aware Planner (TAP). The TAP software leverages a growing number of information sources on the flightdeck to make time- and fuel-saving route optimization recommendations to the aircrew while en route. The aircrewcan then use the suggestions provided by the tool to make route change requests with a greater likelihood of acceptanceby air traffic control (ATC). Since TASAR is a concept intended for the current operational environment, it isintentionally designed to have no safety-critical impact or require any changes to current Federal AviationAdministration (FAA) rules and procedures [4, 5].The research prototype TAP system [6–8], explained further in Section III.C, continually incorporates up-to-dateaircraft state data from onboard avionics, as well as the latest position of surrounding traffic, the most recent windforecast, and the most recent convective weather forecast, in order to calculate candidate trajectory modifications thatimprove upon the current active route. These trajectories account for user-selectable objective functions [3] of reducedfuel burn, reduced flight time, or an airline-derived combination of factors known as trip cost. Previous analyses andsimulations have estimated substantial savings for airlines employing this technique within the U.S. National AirspaceSystem (NAS) [9–11]. Operational evaluations with Alaska Airlines seek to validate these projected benefits usingmeasured data while simultaneously providing benefits to the airline [12, 13].The TAP software has undergone a number of human-in-the-loop simulations [14] and flight test activities[15–17] in order to validate the operational concept, evaluate human factors considerations (e.g., workload, usability,distraction, etc.), and to assess the ability of the software to function in a representative operational environment (e.g.,connected to live avionics data, using in-flight internet connectivity, etc.). However, these simulations and flight testcampaigns did not account for the hardware architecture implemented on the three aircraft for Alaska Airlines’operational evaluations of the TAP software. Therefore, a need was identified to thoroughly test the functionality ofthe software in a similar hardware architecture to that of the partner airline’s aircraft. Information regarding testapparatus and environments used to evaluate TAP prior to testing on the HU-25A can be found in reference [18].A campaign of flight trials on a NASA aircraft, the HU-25A Guardian, was conducted to ensure that the researchprototype TAP system functions well in a configuration similar to the Alaska Airlines aircraft prior to deployment.This airborne, networked environment enables an assessment of the operational factors unique to the flight environment. Additionally, this activity evaluated the effectiveness and benefit of new TAP functionality andoperation in a relevant flight environment while allowing the rapid prototyping of new concepts and features.This paper is organized as follows: Section II discusses the details of the flight test plan, flight profiles, and theduties of personnel involved with conducting flight operations. Section III describes the test platform, avionicsequipage, and system architecture. Section IV presents a discussion of results, and Section V contains concludingremarks.

Underwood, Matthew C.↗

Software System for the Mars 2020 Mission Sampling and Caching Testbeds

The development of the Sampling and Caching Subsystem (SCS) of the Mars 2020 Rover Mission is highly dependent on testing of prototype hardware and software operating in explicit conditions as part of integrated testbeds. To achieve relevant integration of hardware and software while maintaining rapid algorithm development capabilities and high testing throughput, the Controls and Autonomy for Sample Acquisition and Handling (CASAH) software system was developed. CASAH is an implementation of the Intelligent Robotics System Architecture (IRSA),which mimics JPL Flight Software (FSW) in that it is divided into hierarchical modules that run separate processes that communicate via message passing, each module is assigned an owner that is a single developer, and the operator initiates requests via a text-based interface that interprets sequences of commands.IRSA enables a modular breakdown of CASAH that follows that of 2020 Flight Software,so developers can take an algorithm from a module in CASAH and re-code it into the same module in FSW. As deployment of CASAH has grown to ten testbeds - each with different hardware and objectives - bottom-up design decisions have been intentionally made to keep the system lightweight and maintainable by a very small team. To date, CASAH has been used to run 1393 different tests. This work describes CASAH, the testbeds and functionality it supports, the tools used to manage the development and sharing of code, and the features of the software. Lessons learned over the past three years of development and deployment are provided.

Vieira, Peter↗

Pyomo: Accidentally outrunning the bear

Pyomo is an open-source optimization modeling software that has undergone significant evolution since its inception in 2008. Pyomo has evolved to enhance flexibility, solver integration, and community engagement. Modern collaborative tools for open-source software have facilitated the development of new Pyomo functionality and improved our development process through automated testing and performance-tracking pipelines. However, Pyomo faces challenges typical of research software, including resource limitations and knowledge retention. The Pyomo team’s commitment to better development practices and community engagement reflects a proactive approach to these issues. We describe Pyomo’s development journey, highlighting both successes and failures, in the hopes that other open-source research software packages may benefit from our experiences.

automation↗

Applications of Formal Methods to Specification and Safety of Avionics Software

This report treats several topics in applications of formal methods to avionics software development. Most of these topics concern decision tables, an orderly, easy-to-understand format for formally specifying complex choices among alternative courses of action. The topics relating to decision tables include: generalizations fo decision tables that are more concise and support the use of decision tables in a refinement-based formal software development process; a formalism for systems of decision tables with behaviors; an exposition of Parnas tables for users of decision tables; and test coverage criteria and decision tables. We outline features of a revised version of ORA's decision table tool, Tablewise, which will support many of the new ideas described in this report. We also survey formal safety analysis of specifications and software.

Hoover, D. N.↗

Development of a Genetic Algorithm to Automate Clustering of a Dependency Structure Matrix

Much technology assessment and organization design data exists in Microsoft Excel spreadsheets. Tools are needed to put this data into a form that can be used by design managers to make design decisions. One need is to cluster data that is highly coupled. Tools such as the Dependency Structure Matrix (DSM) and a Genetic Algorithm (GA) can be of great benefit. However, no tool currently combines the DSM and a GA to solve the clustering problem. This paper describes a new software tool that interfaces a GA written as an Excel macro with a DSM in spreadsheet format. The results of several test cases are included to demonstrate how well this new tool works.

Rogers, James L.↗

Tool Use Within NASA Software Quality Assurance

As space mission software systems become larger and more complex, it is increasingly important for the software assurance effort to have the ability to effectively assess both the artifacts produced during software system development and the development process itself. Conceptually, assurance is a straightforward idea - it is the result of activities carried out by an organization independent of the software developers to better inform project management of potential technical and programmatic risks, and thus increase management's confidence in the decisions they ultimately make. In practice, effective assurance for large, complex systems often entails assessing large, complex software artifacts (e.g., requirements specifications, architectural descriptions) as well as substantial amounts of unstructured information (e.g., anomaly reports resulting from testing activities during development). In such an environment, assurance engineers can benefit greatly from appropriate tool support. In order to do so, an assurance organization will need accurate and timely information on the tool support available for various types of assurance activities. In this paper, we investigate the current use of tool support for assurance organizations within NASA, and describe on-going work at JPL for providing assurance organizations with the information about tools they need to use them effectively.

software assurance↗

Simulated Multipath Using Software Generated GPS Signals

Depending on the environment, multipath can be one of the largest error sources contributing to degradation in Global Navigation Satellite System (GNSS) (e.g., GPS) performance. Multipath is a phenomenon that occurs as radio signals reflect off of surfaces, such as buildings, producing multiple copies of the original signal. When this occurs with GPS signals, it results in one or more delayed signals arriving at the receiver with or without the on-time/direct GPS signal. The receiver measures the composite of these signals which, depending on the severity of the multipath, can substantially degrade the accuracy of the receiver's calculated position. Multipath is commonly experienced in cities due to tall buildings and its mitigation is an ongoing area of study. This research demonstrates a novel approach for simulating GPS multipath through the modification of an open-source tool, GPS-SDR-SIM. The resulting additional testing capability could allow for improved development of multipath mitigating technologies. Currently, open-source tools for simulating GPS signals are available and can be used in the testing and evaluation of GPS receiver equipment. These tools can generate GPS signals that, when used by a GPS receiver, result in computation of a position solution that was pre-determined at the time of signal generation. That is, the signals produced are properly formed for the pre-determined location and result in the receiver reporting that position. This allows for a GPS receiver under test to be exposed to various simulated locations and conditions without having to be physically subjected to them. Additionally, while these signals are generated by a software simulation, they can be processed by real or software defined GPS receivers. This work utilizes the GPS-SDR-SIM software tool for GPS signal generation and while this tool does implement some sources of error that are inherent to GPS, it cannot inject multipath. GPS-SDR-SIM was modified in this effort to produce additional copies of signals with pre-determined delays. These additional delayed signals mimic multipath and represent what happens to GPS signals in the real world as they reflect off of surfaces and arrive at a receiver in place of or alongside the direct GPS signal. A successful proof of concept was prototyped and demonstrated using this modified version of GPS-SDR-SIM to produce simulated GPS signals as well as additional simulated multipath signals. The generated data was processed using a software defined GPS receiver and it was found that the introduction of simulated multipath signals successfully produced the expected characteristics of a composite multipath signal. Further maturation of this work could allow for the development of a GPS receiver testing and evaluation framework and aid in the development of multipath mitigating technologies.

GPS↗

Testing and Application of Commercial Thermodynamics Software for Solid-Gas Equilibria in Planetary Science Applications

The ability to calculate chemical equilibria from thermochemical constants (e.g., H, S, Cp) or other experimental data is a tool widely employed by scientists. However, phase equilibria for planetary science problems is limited by easy access to libraries of thermochemical data, as well as the software to invert that data. Many academic software packages are focused on specific types of problems (e.g., aqueous solutions or melting and crystallization of magmas). This focus makes them excellent tools for specific uses, but poorer tools for other scenarios, especially those involving elements or chemical species that are less well-studied in that specific scientific field (e.g. NaCl vapor). Amongst the more general thermodynamic software tools, there are closed academic software packages, open-source options, as well as commercial software. We have explored the extent to which commercial chemistry/chemical engineering software may be able to satisfy the need for thermochemical modeling packages that can accommodate the diverse species and conditions of planetary science problems. Here we compare the output of software package HSC Chemistry™ (Metso Outotec) to published vapor-solid phase diagrams for a variety of major and minor elements. HSC Chemistry™ has been used in several recent geochemical studies of terrestrial volcanic systems [2- 4]. However, it has not to our knowledge been benchmarked against peer-reviewed model results for the conditions and compositions relevant to planetary science. We present one such comparison here, the low- pressure environment of the solar nebula, using literature data. In short, we find excellent agreement between HSC Chemistry™ and the well-regarded academic software package CONDOR. These preliminary results suggest that at least some commercial software packages are capable of robust thermochemical calculations for planetary science.

planetary science↗

Electric Drive Technologies Research: ELT223 Component Modeling, Co-Optimization, and Trade-Space Evaluation Annual Report

This project is intended to support the development of new traction drive systems that meet the targets of 100 kW/L for power electronics and 50 kW/L for electric machines with reliable operation to 300,000 miles. To meet these goals, new designs must be identified that make use of state-of-the-art and next-generation electronic materials and design methods. Designs must exploit synergies between components, for example converters designed for high-frequency switching using wide band gap (WBG) devices and ceramic capacitors. This project included: (1) a survey of available technologies; (2) investigating new technologies, that for example, reduce volume of thermal management or magnetic components; (3) the development of computer aided design tools that consider the converter volume, reliability, and electrical performance; (4) exercising the design software to evaluate performance gaps and predict the impact of certain technologies and design approaches, i.e. GaN semiconductors, ceramic capacitors, ceramic thermal management components, and select topologies; (5) building and testing hardware prototypes to validate models and concepts. The design tools enable co-optimization of the power module and passive elements and provide some design guidance. At the end of the project, new advanced computing methods, such as machine learning approaches, were considered.

33 ADVANCED PROPULSION SYSTEMS↗

Software Users Manual (SUM): Extended Testability Analysis (ETA) Tool

This software user manual describes the implementation and use the Extended Testability Analysis (ETA) Tool. The ETA Tool is a software program that augments the analysis and reporting capabilities of a commercial-off-the-shelf (COTS) testability analysis software package called the Testability Engineering And Maintenance System (TEAMS) Designer. An initial diagnostic assessment is performed by the TEAMS Designer software using a qualitative, directed-graph model of the system being analyzed. The ETA Tool utilizes system design information captured within the diagnostic model and testability analysis output from the TEAMS Designer software to create a series of six reports for various system engineering needs. The ETA Tool allows the user to perform additional studies on the testability analysis results by determining the detection sensitivity to the loss of certain sensors or tests. The ETA Tool was developed to support design and development of the NASA Ares I Crew Launch Vehicle. The diagnostic analysis provided by the ETA Tool was proven to be valuable system engineering output that provided consistency in the verification of system engineering requirements. This software user manual provides a description of each output report generated by the ETA Tool. The manual also describes the example diagnostic model and supporting documentation - also provided with the ETA Tool software release package - that were used to generate the reports presented in the manual

Maul, William A.↗

Autonomous Mission Operations

NASA's Advanced Exploration Systems Autonomous Mission Operations (AMO) project conducted an empirical investigation of the impact of time-delay on todays mission operations, and of the effect of processes and mission support tools designed to mitigate time-delay related impacts. Mission operation scenarios were designed for NASA's Deep Space Habitat (DSH), an analog spacecraft habitat, covering a range of activities including nominal objectives, DSH system failures, and crew medical emergencies. The scenarios were simulated at time-delay values representative of Lunar (1.2-5 sec), Near Earth Object (NEO) (50 sec) and Mars (300 sec) missions. Each combination of operational scenario and time-delay was tested in a Baseline configuration, designed to reflect present-day operations of the International Space Station, and a Mitigation configuration in which a variety of software tools, information displays, and crew-ground communications protocols were employed to assist both crews and Flight Control Team (FCT) members with the long-delay conditions. Preliminary findings indicate: 1) Workload of both crew members and FCT members generally increased along with increasing time delay. 2) Advanced procedure execution viewers, caution and warning tools, and communications protocols such as text messaging decreased the workload of both flight controllers and crew, and decreased the difficulty of coordinating activities. 3) Whereas crew workload ratings increased between 50 sec and 300 sec of time-delay in the Baseline configuration, workload ratings decreased (or remained flat) in the Mitigation configuration.

Mission Operations↗

Man-rated flight software for the F-8 DFBW program

The design, implementation, and verification of the flight control software used in the F-8 DFBW program are discussed. Since the DFBW utilizes an Apollo computer and hardware, the procedures, controls, and basic management techniques employed are based on those developed for the Apollo software system. Program Assembly Control, simulator configuration control, erasable-memory load generation, change procedures and anomaly reporting are discussed. The primary verification tools--the all-digital simulator, the hybrid simulator, and the Iron Bird simulator--are described, as well as the program test plans and their implementation on the various simulators. Failure-effects analysis and the creation of special failure-generating software for testing purposes are described. The quality of the end product is evidenced by the F-8 DFBW flight test program in which 42 flights, totaling 58 hours of flight time, were successfully made without any DFCS inflight software, or hardware, failures.

Bairnsfather, R. R.↗

Providing Experimental Infrastructure for Accelerating Advanced Reactor Demonstrations through the National Reactor Innovation Center

A suite of experimental infrastructure projects has been developed by the National Reactor Innovation Center to accelerate advanced reactor demonstrations and facilitate their development, addressing crucial gaps in data, materials characterization, and modeling. First, the Molten Salt Thermophysical Examination Capability (MSTEC) provides a specialized platform for post-irradiation characterization of molten salt reactor fuel, coolant salts, and structural materials, essential for supporting the design and operation of advanced reactors and future commercial molten salt reactor development and licensing. The Virtual Test Bed (VTB) complements these efforts by leveraging advanced modeling and simulation tools to evaluate reactor performance and safety. Serving as a library of reference models, the VTB offers a database of multiphysics reactor models, facilitating rapid safety evaluations and includes continuous software quality assurance, crucial for accelerating deployment while maintaining reliability. Additionally, the Helium Component Test Facility (HeCTF) addresses the need for high-temperature helium-cooled reactor component testing. As the first-of-its-kind facility in the United States, HeCTF emulates high-temperature gas reactor conditions, reducing time and cost associated with component validation, thereby accelerating reactor development. Finally, In-cell Thermal Creep Frames provide a unique solution for obtaining thermal creep data from irradiated materials, critical for materials qualification and licensing. Developed by the National Reactor Innovation Center, these compact frames enable the examination of previously irradiated materials, overcoming traditional limitations and enhancing the understanding of mechanical properties crucial for reactor development. Collectively, these experimental infrastructure projects form a comprehensive framework aimed at expediting advanced reactor demonstrations, fostering innovation, and ensuring the viability of next-generation nuclear energy solutions.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗

Multiple IMU system development, volume 1

A redundant gimballed inertial system is described. System requirements and mechanization methods are defined and hardware and software development is described. Failure detection and isolation algorithms are presented and technology achievements described. Application of the system as a test tool for shuttle avionics concepts is outlined.

Landey, M.↗

A new method for hardware/software integration of strategic systems - Case study of the Space Shuttle

An advanced system integrated self-test has been developed to provide dynamic checkout of all critical subsystems and hardware/software interfaces of the Space Shuttle during pre-launch ground testing. The system modifies hardware sensor data to represent a real flight scenario. This modified data then drives the flight software. The system was sucessfully utilized for three phases of Space Shuttle testing, and will be expanded for use as a maintenance tool.

Haque, S. I.↗

Proceedings of the 3rd Annual Conference on Aerospace Computational Control, volume 1

Conference topics included definition of tool requirements, advanced multibody component representation descriptions, model reduction, parallel computation, real time simulation, control design and analysis software, user interface issues, testing and verification, and applications to spacecraft, robotics, and aircraft.

Bernard, Douglas E.↗