Results for “testing complex systems” — Page 4

NASA NTRS2012

Nemesis Autonomous Test System

A generalized framework has been developed for systems validation that can be applied to both traditional and autonomous systems. The framework consists of an automated test case generation and execution system called Nemesis that rapidly and thoroughly identifies flaws or vulnerabilities within a system. By applying genetic optimization and goal-seeking algorithms on the test equipment side, a "war game" is conducted between a system and its complementary nemesis. The end result of the war games is a collection of scenarios that reveals any undesirable behaviors of the system under test. The software provides a reusable framework to evolve test scenarios using genetic algorithms using an operation model of the system under test. It can automatically generate and execute test cases that reveal flaws in behaviorally complex systems. Genetic algorithms focus the exploration of tests on the set of test cases that most effectively reveals the flaws and vulnerabilities of the system under test. It leverages advances in state- and model-based engineering, which are essential in defining the behavior of autonomous systems. It also uses goal networks to describe test scenarios.

Barltrop, Kevin J.↗

NASA NTRS1990

Realtime multi-plot graphics system

The increased complexity of test operations and customer requirements at Langley Research Center's National Transonic Facility (NTF) surpassed the capabilities of the initial realtime graphics system. The analysis of existing hardware and software and the enhancements made to develop a new realtime graphics system are described. The result of this effort is a cost effective system, based on hardware already in place, that support high speed, high resolution, generation and display of multiple realtime plots. The enhanced graphics system (EGS) meets the current and foreseeable future realtime graphics requirements of the NTF. While this system was developed to support wind tunnel operations, the overall design and capability of the system is applicable to other realtime data acquisition systems that have realtime plot requirements.

Shipkowski, Michael S.↗

NASA NTRS1979

Test-configuration identifiers

Distributed computer system, which allows great deal of interaction within totally synchronized environment, comprises test system that presents systematic approach for identifying test configurations for large complex systems such as submarines, aircraft, or air traffic controllers.

Sumrall, W. D.↗

DOE OSTI2023

Dataset for SIAM MPI23 Project "Model inversion for complex physical systems using low-dimensional surrogates"

Training and testing data for the SIAM MPI23 workshop "Model inversion for complex physical systems using low-dimensional surrogates". The data consists of ensembles of input and output pairs corresponding to queries of a 2D saturated groundwater flow model. The inputs consist of random vectors which map to discretized model parameter fields via a Kosambi-Karhunen-Loève expansion. Output corresponds to discretized pressure fields.

basis adaptation, conditional Karhunen-Loève expan↗

NASA NTRS1985

Electronics systems test laboratory testing of shuttle communications systems

Shuttle communications and tracking systems space to space and space to ground compatibility and performance evaluations are conducted in the NASA Johnson Space Center Electronics Systems Test Laboratory (ESTL). This evaluation is accomplished through systems verification/certification tests using orbiter communications hardware in conjunction with other shuttle communications and tracking external elements to evaluate end to end system compatibility and to verify/certify that overall system performance meets program requirements before manned flight usage. In this role, the ESTL serves as a multielement major ground test facility. The ESTL capability and program concept are discussed. The system test philosophy for the complex communications channels is described in terms of the major phases. Results of space to space and space to ground systems tests are presented. Several examples of the ESTL's unique capabilities to locate and help resolve potential problems are discussed in detail.

Stoker, C. J.↗

NASA NTRSDate not supplied

Certification Considerations for Adaptive Stress Testing of Airborne Software

eduAdaptive Stress Testing (AST) has shown promise in identifying errant corner cases in complex software used in aerospace applications including Flight Management Systems (FMS). The strength of AST is performing test-based verification of complex aerospace software intensive systems at scale in simulated operational environments.Simulating and capturing the realistic operational complexities in integrated verification environments may exposeflaws in the softwareprior to field deployment, whereas the software may perform just fine to traditional requirements-basedunit and component level testing.AST can be used to test the whole system.Individual components may behave safely, but together can result in complex interactions and emergent failures, so it is important to test at the integrated system level.Motivated by the observed benefitsat the prototype proof of concept scale, this paper considers how AST may be integrated into a production workflow and used to generate objective evidence in a processthat delivers certified aerospace software.The research includes evaluation of alignment with both DO-178C and Overarching Properties(OP). The paper addresses questions such as “where should AST fit in the Plan for Software Aspects of Certification (PSAC) and Software Verification Plan (SVP), what aspects of AST do not fit, and what objectives does it satisfy?” The paper concludes that AST is in fact useful at locating errors in complex airborne application software and in doing so provides benefits to suppliers and end users. Furthermore, AST appears appropriate to add value in both DO-178Cbased and Overarching Properties based certification approaches.

certification↗

NASA NTRS1985

In-flight testing of the space shuttle orbiter thermal control system

In-flight thermal control system testing of a complex manned spacecraft such as the space shuttle orbiter and the considerations attendant to the definition of the tests are described. Design concerns, design mission requirements, flight test objectives, crew vehicle and mission risk considerations, instrumentation, data requirements, and real-time mission monitoring are discussed. An overview of the tests results is presented.

Taylor, J. T.↗

NASA NTRS2009

Combinatorial Generation of Test Suites

Testgen is a computer program that generates suites of input and configuration vectors for testing other software or software/hardware systems. As systems become ever more complex, often, there is not enough time to test systems against all possible combinations of inputs and configurations, so test engineers need to be selective in formulating test plans. Testgen helps to satisfy this need: In response to a test-suite-requirement-specification model, it generates a minimal set of test vectors that satisfies all the requirements.

Dvorak, Daniel L.↗

NASA NTRS2003

The Data Acquisition and Controls Systems (DACS) of the E-Complex at the John C. Stennis Space Center, MS: A General Overview

The John C. Stennis Space Center (SSC) provides test operations services to a variety of customers including NASA, DoD, commercial enterprises, and others for the development of current next-generation rocket propulsion systems. Many of these test operations services are provided in the E-Complex series of test facilities. The E-Complex is composed of three active test stands (E1, E2, & E3), each with two or more test positions. Each test position is comprised of unique sets of data acquisition and controls hardware and software that record both facility and test article data and safely operate the test facility. The E-Complex data acquisition system (DAS) is actually composed of two separate systems, one for static data and the other for dynamic. The static DAS, otherwise known as the Low-Speed DAS (LSDAS), samples 16 bit data at 250 samples-per-second (SPS), although an aggregate sample rate of 200,000 SPS is possible. The dynamic data acquisition system, otherwise known as the high-speed DAS (HSDAS), samples 16 bit data at 100K SPS with a 45 KHz bandwidth.

Hughes, Mark S.↗

DOE OSTI2022

The Ground Truth Program: Simulations as Test Beds for Social Science Research Methods.

Social systems are uniquely complex and difficult to study, but understanding them is vital to solving the world’s problems. The Ground Truth program developed a new way of testing the research methods that attempt to understand and leverage the Human Domain and its associated complexities. The program developed simulations of social systems as virtual world test beds. Not only were these simulations able to produce data on future states of the system under various circumstances and scenarios, but their causal ground truth was also explicitly known. Research teams studied these virtual worlds, facilitating deep validation of causal inference, prediction, and prescription methods. The Ground Truth program model provides a way to test and validate research methods to an extent previously impossible, and to study the intricacies and interactions of different components of research.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

ATTRACTOR: Toward Trustworthy and Trusted Autonomous Systems

The question of what it means and what it takes for an autonomous system to consider another autonomous system justifiably trustworthy must be addressed by all who seek to integrate intelligent machine agents into real-world operations. A satisfactory answer to this question is an essential component in accepting autonomous machine decision-making in safety-critical and time-critical environments, such as aviation. Historically, simulation platforms for test and evaluation of complex systems have proven to be effective in assessing performance and contributing to decisions on the fitness of systems to operate in current general and commercial aviation airspace. Moreover, simulations have informed the definition of safety-critical constraints. However, as machine systems progressively take on responsibilities for decision-making traditionally supplied by humans, simulations require enhancement. Mixed reality simulation that integrates real-world platforms and data or high-fidelity simulation data in a sim-to-flight paradigm provides insight into agent interaction and the rationale behind autonomous agent decision-making as well as the capacity for seamless integrated implementation, testing, and operation of systems. Strong simulation capabilities are especially important in the presence of algorithms that hold great promise in decision-making yet increase the uncertainty in the system. Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) is a subproject of NASA’s Convergent Aeronautics Solutions (CAS) Project. ATTRACTOR’s objective is to build a basis for understanding trust and trustworthiness in multi-agent autonomous teams, and thus to inform future certification of safety-critical and time-critical autonomous systems in aviation. Because the concepts of trust and trustworthiness must be addressed in a context, ATTRACTOR has chosen Search and Rescue (SAR) in dynamic and unstructured environments, with emphasis on search, as its design reference mission (DRM). During dynamic planning and execution of trajectory-based operations, autonomous agents determine their trajectories given an assigned mission or missions and call for assistance from an appropriate teammate when needed. This experience along with the attendant human-machine and machine-machine interactions, serve as a platform for developing approaches to identifying and measuring trustworthiness and increasing trust. In this paper, we give an overview of some of ATTRACTOR’s research and development activities, findings, and ongoing work.

ATTRACTOR↗

NASA NTRS2000

Signal Processing and Fault Detection with Application to CH-46 Helicopter Data

Central to Qualtech Systems' mission is its testability and maintenance software (TEAMS) and derivatives. Many systems comprise components equipped with self-testing capability; but, if the system is complex (and involves feedback and if the self-testing itself may occasionally be faulty) tracing faults to a single or multiple causes is difficult. However, even for systems involving many thousands of components the PC-based TEAMS provides essentially real-time system-state diagnosis. Until recently TEAMS operation was passive: its diagnoses were based on whatever data sensors could provide. Now, however, a signal-processing (SP) "frontend" matched to inference needs is available. Many standard signal processing primitives, such as filtering, spectrum analysis and multi-resolution decomposition are available; the SP toolbox is also equipped with a (supervised) classification capability based on a number of decision-making paradigms. This paper is about the SP toolbox. We show its capabilities, and demonstrate its performance on the CH-46 "Westland" data set.

Wen, Fang↗

NASA NTRS2004

Modular, Adaptive, Reconfigurable Systems: Technology for Sustainable, Reliable, Effective, and Affordable Space Exploration

In order to execute the Vision for Space Exploration, we must find ways to reduce cost, system complexity, design, build, and test times, and at the same time increase flexibility to satisfy multiple functions. Modular, Adaptive, Reconfigurable System (MARS) technologies promise to set the stage for the delivery of system elements that form the building blocks of increasingly ambitious missions involving humans and robots. Today, space systems are largely specialized and built on a case-by-case basis. The notion of modularity however, is nothing new to NASA. The 1970's saw the development of the Multi-Mission Modular spacecraft (MMS). From 1980 to 1992 at least six satellites were built under this paradigm, and included such Goddard Space Flight Center missions as SSM, EUVE, UARS, and Landsat 4 and 5. Earlier versions consisted of standard subsystem "module" or "box" components that could be replaced within a structure based on predefined form factors. Although the primary motivation for MMS was faster/cheaper integration and test, standardization of interfaces, and ease of incorporating new subsystem technology, it lacked the technology maturity and programmatic "upgrade infrastructure" needed to satisfy varied mission requirements, and ultimately it lacked user buy-in. Consequently, it never evolved and was phased out. Such concepts as the Rapid Spacecraft Development Office (RSDO) with its regularly updated catalogue of prequalified busses became the preferred method for acquiring satellites. Notwithstanding, over the past 30 years since MMS inception, technology has advanced considerably and now modularity can be extended beyond the traditional MMS module or box to cover levels of integration, from the chip, card, box, subsystem, to the space system and to the system-of-systems. This paper will present the MARS architecture, cast within the historical context of MMS. Its application will be highlighted by comparing a state-of-the-art point design vs. a MARS-enabled lunar mission, as a representative robotic case design.

Esper, Jaime↗

NASA NTRS1989

Comparison of sine dwell and broadband methods for modal testing

The objectives of modal tests for large complex spacecraft structural systems are outlined. The comparison criteria for the modal test methods, namely, the broadband excitation and the sine dwell methods, are established. Using the Galileo spacecraft modal test and the Centaur G Prime upper stage vehicle modal test as examples, the relative advantage or disadvantage of each method is examined. The usefulness or shortcomings of the methods are given from a practical engineering viewpoint.

Chen, Jay-Chung↗

NASA NTRS2013

Spacecraft Parachute Recovery System Testing from a Failure Rate Perspective

Spacecraft parachute recovery systems, especially those with a parachute cluster, require testing to identify and reduce failures. This is especially important when the spacecraft in question is human-rated. Due to the recent effort to make spaceflight affordable, the importance of determining a minimum requirement for testing has increased. The number of tests required to achieve a mature design, with a relatively constant failure rate, can be estimated from a review of previous complex spacecraft recovery systems. Examination of the Apollo parachute testing and the Shuttle Solid Rocket Booster recovery chute system operation will clarify at which point in those programs the system reached maturity. This examination will also clarify the risks inherent in not performing a sufficient number of tests prior to operation with humans on-board. When looking at complex parachute systems used in spaceflight landing systems, a pattern begins to emerge regarding the need for a minimum amount of testing required to wring out the failure modes and reduce the failure rate of the parachute system to an acceptable level for human spaceflight. Not only a sufficient number of system level testing, but also the ability to update the design as failure modes are found is required to drive the failure rate of the system down to an acceptable level. In addition, sufficient data and images are necessary to identify incipient failure modes or to identify failure causes when a system failure occurs. In order to demonstrate the need for sufficient system level testing prior to an acceptable failure rate, the Apollo Earth Landing System (ELS) test program and the Shuttle Solid Rocket Booster Recovery System failure history will be examined, as well as some experiences in the Orion Capsule Parachute Assembly System will be noted.

Stewart, Christine E.↗

NASA NTRS2012

Supersonic Retropropulsion Technology Development in NASA's Entry, Descent, and Landing Project

NASA's Entry, Descent, and Landing (EDL) space technology roadmap calls for new technologies to achieve human exploration of Mars in the coming decades [1]. One of those technologies, termed Supersonic Retropropulsion (SRP), involves initiation of propulsive deceleration at supersonic Mach numbers. The potential benefits afforded by SRP to improve payload mass and landing precision make the technology attractive for future EDL missions. NASA's EDL project spent two years advancing the technological maturity of SRP for Mars exploration [2-15]. This paper summarizes the technical accomplishments from the project and highlights challenges and recommendations for future SRP technology development programs. These challenges include: developing sufficiently large SRP engines for use on human-scale entry systems; testing and computationally modelling complex and unsteady SRP fluid dynamics; understanding the effects of SRP on entry vehicle stability and controllability; and demonstrating sub-scale SRP entry systems in Earth's atmosphere.

Edquist, Karl T.↗

NASA NTRSDate not supplied

CLINICAL DECISION SUPPORT: PATH TO FUNCTIONAL REQUIREMENTS

Long-duration, deep-space exploration missions present significant challenges to crew health and performance. These challenges include the individual and combined effects of microgravity, radiation exposure, isolation, limited resources (mass, volume, power, data and crew time), limited options for evacuation and those associated with delayed or constrained communications, all of which demand greater crew autonomy. Specifically, as the communication delays intensify the further we explore space, the unqualified need for Earth-independent medical operations focused on autonomous diagnosis, treatment and prevention will be key to mission continuation and success. To augment the requisite knowledge, skills and abilities (KSAs) of a time-constrained crew operating under stressful conditions, combatting fatigue, and facing a potential medical crisis, a robust clinical decision support system (CDSS) is a probable solution that would facilitate, guide and inform Earth-independent medical operations, while assisting crewmembers through various clinical presentations. The Exploration Medical Capability (ExMC) Element of the Human Research Program (HRP) is expanding the boundaries of space medical systems to advance the care of astronauts on future exploration missions beyond low Earth orbit. ExMC is actively identifying and testing next-generation medical care and crew health maintenance technologies. The Clinical Decision Support (CDS) project addresses gap Medical-701 within the Inflight Medical Conditions risk: “Enhance medical capabilities within an exploration medical system.” Though mass, volume, and power will face increasing constraints, the projected computational capabilities of spacecraft systems will increase exponentially as information technology continues to advance this decade and beyond. Hence, data, software and computational resources will play an essential and synergistic role in maintaining crew health, wellness and performance in deep space missions. The focus of the CDS project is to develop recommended requirements for an in-vehicle CDSS that acts as a ‘virtual assistant’ for delivering optimal health, performance and medical care during exploration missions. The CDSS is envisioned as an integrated, software-based tool deployed on a laptop computer or handheld device. The CDSS will assist the crew and ground support when interacting with knowledge/databases (e.g. records, pharmacy, schedule), instrumentation (e.g. imaging, physiological monitoring devices), and habitat (e.g. wellness system, task performance system) and vehicle systems (e.g. environmental system, communication system). In addition, the human interface will employ a context-based approach that accounts for the crew’s situation. Thus, extraneous and clinically/operationally non-relevant information are reduced to avoid an increase in cognitive load. The framework of an ideal spaceflight CDSS is to include core and advanced analytical features that incorporate work from collaborators yet maintain a flexible platform for integrating new technology in the future. In fiscal year 2021 (FY21), the CDS project identified requirements through two primary mechanisms: (i) the development of software implementation prototypes and (ii) the application of systems engineering processes. The CDS project developed and tested a series of increasingly complex system prototypes that were based on use cases derived from the CDSS concept of operations (ConOps). These software implementations yielded insights on CDSS functionality as well as lessons learned that provided the initial requirements for CDSS capability. By applying a systems engineering (SE) approach, medical scenarios provided in the ConOps and the use cases for software implementation underwent functional decomposition to identify CDSS functionality. Also, systems-based modeling language (SysML) tools such as activity diagrams were developed from the same ConOps and use cases to identify CDSS functionality. The lessons learned from software implementation defined both specific requirements and broad areas of requirements. Within these defined broad requirement areas, further analysis of the SE products identified specific capability that resulted in the final functional requirements. In summary, the software prototypes, functional decomposition of the ConOps and use cases, and SysML diagrams provided the basis for the CDSS requirements developed in FY21. In the upcoming year, these requirements will be refined for their final ExMC baseline review in latter FY22.

clinical decision support↗

NASA NTRSDate not supplied

Clinical Decision Support: Path to Functional Requirements

Long-duration, deep-space exploration missions present significant challenges to crew health and performance. These challenges include the individual and combined effects of microgravity, radiation exposure, isolation, limited resources (mass, volume, power, data and crew time), limited options for evacuation and those associated with delayed or constrained communications, all of which demand greater crew autonomy. Specifically, as the communication delays intensify the further we explore space, the unqualified need for Earth-independent medical operations focused on autonomous diagnosis, treatment and prevention will be key to mission continuation and success. To augment the requisite knowledge, skills and abilities (KSAs) of a time-constrained crew operating under stressful conditions, combatting fatigue, and facing a potential medical crisis, a robust clinical decision support system (CDSS) is a probable solution that would facilitate, guide and inform Earth-independent medical operations, while assisting crewmembers through various clinical presentations. The Exploration Medical Capability (ExMC) Element of the Human Research Program (HRP) is expanding the boundaries of space medical systems to advance the care of astronauts on future exploration missions beyond low Earth orbit. ExMC is actively identifying and testing next-generation medical care and crew health maintenance technologies. The Clinical Decision Support (CDS) project addresses gap Medical-701 within the Inflight Medical Conditions risk: “Enhance medical capabilities within an exploration medical system.” Though mass, volume, and power will face increasing constraints, the projected computational capabilities of spacecraft systems will increase exponentially as information technology continues to advance this decade and beyond. Hence, data, software and computational resources will play an essential and synergistic role in maintaining crew health, wellness and performance in deep space missions. The focus of the CDS project is to develop recommended requirements for an in-vehicle CDSS that acts as a ‘virtual assistant’ for delivering optimal health, performance and medical care during exploration missions. The CDSS is envisioned as an integrated, software-based tool deployed on a laptop computer or handheld device. The CDSS will assist the crew and ground support when interacting with knowledge/databases (e.g. records, pharmacy, schedule), instrumentation (e.g. imaging, physiological monitoring devices), and habitat (e.g. wellness system, task performance system) and vehicle systems (e.g. environmental system, communication system). In addition, the human interface will employ a context-based approach that accounts for the crew’s situation. Thus, extraneous and clinically/operationally non-relevant information are reduced to avoid an increase in cognitive load. The framework of an ideal spaceflight CDSS is to include core and advanced analytical features that incorporate work from collaborators yet maintain a flexible platform for integrating new technology in the future. In fiscal year 2021 (FY21), the CDS project identified requirements through two primary mechanisms: (i) the development of software implementation prototypes and (ii) the application of systems engineering processes. The CDS project developed and tested a series of increasingly complex system prototypes that were based on use cases derived from the CDSS concept of operations (ConOps). These software implementations yielded insights on CDSS functionality as well as lessons learned that provided the initial requirements for CDSS capability. By applying a systems engineering (SE) approach, medical scenarios provided in the ConOps and the use cases for software implementation underwent functional decomposition to identify CDSS functionality. Also, systems-based modeling language (SysML) tools such as activity diagrams were developed from the same ConOps and use cases to identify CDSS functionality. The lessons learned from software implementation defined both specific requirements and broad areas of requirements. Within these defined broad requirement areas, further analysis of the SE products identified specific capability that resulted in the final functional requirements. In summary, the software prototypes, functional decomposition of the ConOps and use cases, and SysML diagrams provided the basis for the CDSS requirements developed in FY21. In the upcoming year, these requirements will be refined for their final ExMC baseline review in latter FY22.

Clinical decision support↗