SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
This is a presentation of the preliminary results on final V and V (Verification and Validation) activity of [RTCA (Radio Technical Commission for Aeronautics)] SC (Special Committee)-228 DAA (Detect and Avoid) HMI (Human-Machine Interface) requirements for display alerting and guidance.
As demands increase to use unmanned aircraft systems (UAS) for a broad spectrum of commercial applications, regulatory authorities are examining how to safely integrate them without loss of safety or major disruption to existing airspace operations. This work addresses the development of the Safeguard system as an assured safety net technology for UAS. The Safeguard system monitors and enforces conformance to a set of rules defined prior to flight (e.g., geospatial stay-out or stay-in regions, speed limits, altitude limits). Safeguard operates independently of the UAS autopilot and is strategically designed in a way that can be realized by a small set of verifiable functions to simplify compliance with regulatory standards for commercial aircraft. A framework is described that decouples the system from any other devices on the UAS as well as introduces complementary positioning source(s) for applications that require integrity and availability beyond what the Global Positioning System (GPS) can provide. Additionally, the high level logic embedded within the software is presented, as well as the steps being taken toward verification and validation (V&V) of proper functionality. Next, an initial prototype implementation of the described system is disclosed. Lastly, future work including development, testing, and system V&V is summarized.
As spaceflight systems increase in complexity, Fault Management (FM) systems are ranked high in risk-based assessment of software criticality, emphasizing the importance of establishing highly competent domain expertise to provide assurance. Adverse conditions (ACs) and specific vulnerabilities encountered by safety- and mission-critical software systems have been identified through efforts to reduce the risk posture of software-intensive NASA missions. Acknowledgement of potential off-nominal conditions and analysis to determine software system resiliency are important aspects of hazard analysis and FM. A key component of assuring FM is an assessment of how well software addresses susceptibility to failure through consideration of ACs. Focus on significant risk predicted through experienced analysis conducted at the NASA Independent Verification & Validation (IV&V) Program enables the scoping of effective assurance strategies with regard to overall asset protection of complex spaceflight as well as ground systems. Research efforts sponsored by NASAs Office of Safety and Mission Assurance (OSMA) defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs and allowing queries based on project, mission type, domain/component, causal fault, and other key characteristics. Vulnerability in off-nominal situations, architectural design weaknesses, and unexpected or undesirable system behaviors in reaction to faults are curtailed with the awareness of ACs and risk-significant scenarios modeled for analysts through this database. Integration within the Enterprise Architecture at NASA IV&V enables interfacing with other tools and datasets, technical support, and accessibility across the Agency. This paper discusses the development of an improved workflow process utilizing this database for adaptive, risk-informed FM assurance that critical software systems will safely and securely protect against faults and respond to ACs in order to achieve successful missions.
This paper describes the processes and results of Verification and Validation (V&V) efforts for the Collocation Stand Alone Library and Toolkit (CSALT). We describe the test program and environments, the tools used for independent test data, and comparison results. The V&V effort employs classical problems with known analytic solutions, solutions from other available software tools, and comparisons to benchmarking data available in the public literature. Presenting all test results are beyond the scope of a single paper. Here we present high-level test results for a broad range of problems, and detailed comparisons for selected problems.
Reducing the timeline for development and certification for composite structures has been a long standing objective of the aerospace industry. This timeline can be further exacerbated when attempting to integrate new fiber-reinforced composite materials due to the large number of testing required at every level of design. computational progressive damage and failure analysis (PDFA) attempts to mitigate this effect; however, new PDFA methods have been slow to be adopted in industry since material model evaluation techniques have not been fully defined. This study presents an efficient evaluation framework which uses a piecewise verification and validation (V&V) approach for PDFA methods. Specifically, the framework is applied to evaluate PDFA research codes within the context of intralaminar damage. Methods are incrementally taken through various V&V exercises specifically tailored to study PDFA intralaminar damage modeling capability. Finally, methods are evaluated against a defined set of success criteria to highlight successes and limitations.
The NASA has determined that a multi-functional exercise device will be developed for use as an exercise device during exploration missions. The device will allow for full body resistance and metabolic exercise necessary to minimize physiological losses during space flight and to maintain fitness necessary to perform critical mission tasks. Prior to implementation as an exercise device on an Exploration vehicle, there will be verification and validation testing completed to determine device efficacy at providing the necessary training stimuli to achieve desired goals. Because the exploration device will be new device that has yet be specified, specific Verification and Validation (V&V) protocols have yet to be developed. Upon delivery of an exploration exercise device training unit, stakeholders throughout NASA will develop V&V plans that include ground-based testing and testing on the International Space Station (ISS). Stakeholders will develop test protocols that include success criterion for the device. Ground tests will occur at NASA Johnson Space Station prior to flight testing. The intents of the ground tests are to allow crew, spaceflight medicine, science, engineering, Astronaut Strength, Conditioning, and Reconditioning staff, and others to gain experience in the best utilization of the device. The goal is to obtain an evidence base for recommending use of the device on the ISS. The developed protocol will be created to achieve multiple objectives, including determining if the device provides an adequate training stimulus for 5th - 95th percentile males and females, allows for exercise modalities that protect functional capability, and is robust and can withstand extensive human use. Although protocols are yet to be determined, current expectations include use of the device by test subjects and current crew in order to obtain quantitative and qualitative feedback. Information obtained during the ground tests may be used to influence device modifications during design iterations. Assuming successful ground tests, the device will be installed on the ISS for testing during space flight. Spaceflight testing is envisioned to include an activation and checkout (ACO) phase and a V&V phase. During the ACO phase, 1-2 crewmembers will exercise with the device to ensure proper function. ACO is expected to last multiple months because of the many modes and methods of exercise that need to be assessed. However, the goal is to complete the ACO as quickly as possible. Once successful ACO occurs, the crew will be free to use the device for normal exercise pending concurrence from stakeholders. V&V tests on the ISS will ideally consist of crew using the device for all of their exercise for an entire mission. Exercise prescriptions will be supplied that replicate expected prescriptions during exploration missions. Crew that are not enrolled in the V&V studies would be also free to use the device as their schedule permits. As experience is gained by users, exercise protocols could change. The intent of all V&V testing is to ensure that all have thorough understanding of experience at optimizing device capability
NASA space-qualification of reusable off-the-shelf real-time operating systems (RTOSs) remains elusive due to several factors notably (1) The diverse nature of RTOSs utilized across NASA, (2) No single NASA space-qualification criteria, lack of verification and validation (V&V) analysis, or test beds, and (3) different RTOS heritages, specifically open-source RTOSs and closed vendor-provided RTOSs. As a leader in simulation test beds, the NASA IV&V Program is poised to help jump-start and lead the space-qualification effort of the open source Real-Time Executive for Multiprocessor Systems (RTEMS) RTOS. RTEMS, as a case-study, can be utilized as an example of how to qualify all RTOSs, particularly the reusable non-commercial (open-source) ones that are gaining usage and popularity across NASA. Qualification will improve the overall safety and mission assurance of RTOSs for NASA-agency wide usage. NASA's involvement in space-qualification of an open-source RTOS such as RTEMS will drive the RTOS industry toward a more qualified and mature open-source RTOS product.
With the ever-growing desire for mankind to reach destinations whose distances had been deemed impossible to transit, the largest rocket known to man was designed and is being developed. The Space Launch System (SLS), National Aeronautics and Space Administration’s (NASA) solution for deep space travel, will begin its missions with the launch of Exploration Mission 1 (EM-1) and Exploration Mission 2 (EM-2). In order to accommodate the larger rocket, Kennedy Space Center made crucial upgrades to its existing facilities. At Launch Complex 39B, an entirely new Environmental Control System (ECS) was developed to supply the vehicle with the appropriate air or nitrogen gas mixture for launch. The new ECS displays must undergo Validation and Verification (V&V) using testing procedures developed to meet this requirement.
The Space Launch System (SLS) is the next premier launch vehicle for NASA. It is the next stage of manned space exploration from American soil, and will be the platform in which we push further beyond Earth orbit. In preparation of the SLS maiden voyage on Exploration Mission 1 (EM-1), the existing ground support architecture at Kennedy Space Center required significant overhaul and updating. A comprehensive upgrade of controls systems was necessary, including programmable logic controller software, as well as Launch Control Center (LCC) firing room and local launch pad displays for technician use. Environmental control acts as an integral component in these systems, being the foremost system for conditioning the pad and extremely sensitive launch vehicle until T-0. The Environmental Controls Subsystem (ECS) required testing and modification to meet the requirements of the designed system, as well as the human factors requirements of NASA software for Validation and Verification (V&V). This term saw significant strides in the progress and functionality of the human-machine interfaces used at the launch pad, and improved integration with the controller code.
The NASA Operational Simulator for Small Satellites (NOS3) is a suite of open-source software tools to aid in areas such as software development, integration & test (I&T), mission operations/training, verification and validation (V&V), and software systems check-out. NOS3 provides a software development environment, a multi-target build system, operational interface/ground software, dynamics and environment simulations, and software-based hardware models. NOS3 has just recently been open-sourced by NASA and is available for immediate use. It enables the development of flight software (FSW) early in the project life cycle when hardware availability is limited. Small satellite development suffers from extensive lead times on many of the commercial-off-the-shelf (COTS) components as well as limited funding for engineering test units (ETUs). To alleviate the need to provide a hardware test-bed for each developer/tester, NOS3 hardware models are based upon characteristic data or manufacturer's data sheets for each individual component. The NOS3 hardware models' fidelity is such that FSW executes unaware that physical hardware is not present. This allows FSW binaries to be compiled for both the simulation environment and the flight computer without changing the FSW source code. For hardware models that provide data which is dependent upon the environment and spacecraft dynamics, such as a GPS receiver or magnetometer, an open-source tool from NASA GSFC (42 Spacecraft Simulator) is used to provide the necessary data. The underlying infrastructure used to transfer messages between FSW and the hardware models can also be used to monitor, intercept, and inject messages, which has proven to be beneficial for V&V of larger missions such as James Webb Space Telescope (JWST). As hardware is selected and becomes available, drivers can be added to the NOS3 environment to enable hardware-in-the-loop (HWIL) testing. When strict time synchronization is not vital, any number of combinations of hardware components and software-based models can be tested. NOS3 was actively used for FSW development and component testing of the Simulation-to-Flight 1 (STF-1) CubeSat and the Lunar IceCube CubeSat. As NOS3 matures, hardware models have been added for common small satellite components such as GPS receivers, electrical power systems and batteries, and antenna systems.
The Geostationary Operational Environmental Satellite, R-Series (GOES-R) represents a dramatic improvement in GEO (Geostationary Earth Orbit) weather observation capabilities over the previous generation. To provide these new capabilities, GOES-R incorporates a number of new technologies flying for the first time. As with any new spacecraft design, extensive ground testing was performed to validate the vehicle performance. In this paper, we present several successes and several lessons-learned from the GOES-R verification and validation (V&V) efforts. Included are the Dynamic Interaction Test (DIT) results for jitter assessment, and comparison to flight results. Also included are the effects of thermally-induced alignment perturbations, along with post-launch mitigations. Finally, we discuss unexpected GOES-17 gyro performance, which caused a Safe Mode entry shortly after launch. V&V mitigations are presented, which will be used for the next two GOES-R vehicles.
The high complexity of modern aircraft and spacecraft requires elaborate Verification and Validation (V&V) approaches to make sure that such complex systems work properly and reliably. MARGInS is a framework for the analysis, understanding, and prediction of the behavior of a complex, hybrid system. MARGInS contains a set of machine learning and statistical algorithms for multivariate clustering, treatment learning, critical factor determination, time-series analysis, event prediction, and safety-boundary detection and characterization. The framework supports system testing and can be configured to find novel features in test suites, determine classes of behavior, propose new experiments that can efficiently explore and characterize the boundaries between classes of system behavior, and to create visualizations and reports.
In recent years, capabilties of Deep Neural Networks (DNN) and Artificial Intelligence (AI) systems have grown tremendously. They are now applied in many areas ranging from game playing, social media, science, to robotics, automotive, and aerospace applications.Based upon requirements for safety of DNN and AI in high assurance automotive and aerospace applications, I will discuss the necessity to ensure that AI technqiues for the analysis of Earth observation data and reasoning are working correctly and reliably.In this talk I will present modern techniques for the verification and validation (V&V) of DNN and other AI components as well as approaches for interpretable AI. I will discuss how these techniques can help to ensure quality of the AI results, improve confidence in their application, and facilitate human-AI interaction and collaboration.
Thermocouples (TCs) are a critical diagnostic sensor for high enthalpy environments including thermal protection systems (TPS) for atmospheric re-entry, hypersonic wind tunnels, and arc jet test facilities. They are used for both evaluating facility environmental conditions (enthalpy, heat flux) as well as for measuring in-situ material thermal response under test conditions (thermal conductivity, heat capacity, B’, etc.). Consequently, accurate TC measurements are fundamental for both material and facility verification and validation (V&V). However, a number of TC anomalies have been observed in both ground tests and flight instrumentation. These anomalies are caused via various coupling mechanisms: electrostatic gradients, temperature-dependent electrical shorts, radio pickup, magnetic induction, and pressure-dependent leaks. All of these mechanisms can, if unaddressed, result in obviously compromised results or worse, non-physical measurements which appear reasonable. This work will survey known anomaly mechanisms and discuss ongoing efforts to quantify and mitigate anomaly mechanisms including anomaly modeling, diagnostics, and new TC designs/development.
Thermocouples (TCs) are a critical diagnostic sensor for high enthalpy environments including Thermal Protection Systems (TPS) for atmospheric entry, hypersonic wind tunnels, and arc jet test facilities. They are used for both evaluating facility environmental conditions (enthalpy, and heat flux) as well as for measuring in-situ material thermal response under test conditions (thermal conductivity, heat capacity, etc.). Consequently, accurate TC measurements are fundamental for both material and facility Verification and Validation (V&V). However, a number of electromagnetic TC anomalies have been observed in both ground tests and flight instrumentations. These anomalies are caused via various coupling mechanisms: surface voltage coupling, Hall effect currents, and RF rectification. All of these mechanisms can, if unaddressed, result in obviously compromised results or worse, non-physical measurements which appear reasonable. This talk will provide an overview of each of these anomaly mechanisms and discuss ongoing efforts to quantify and mitigate them.
The NASA InSight Mars lander is a roboticspacecraft which is investigating the interior structure of Marsthrough a series of seismological measurements andexperiments. Much of the commanding of activities performedby the instruments was done using ‘blocks’ – reusable functionswritten in Virtual Machine Language stored onboard thespacecraft– and sequences (series of commands) that werecreated by teams to operate their instruments. It was vital thatthese blocks and sequences (collectively called commandproducts) be thoroughly examined prior to onboard executionusing a comprehensive test program to reduce risk to thespacecraft and instruments and ensure smooth operations onthe surface of Mars. The Verification and Validation (V&V)process that was designed and implemented during the monthsleading up to InSight’s launch and landing successfullyuncovered a number of issues that could have causedoperational delays or impacted hardware. The process alsohelped adapt ground tools to better model onboard activities,create operational principles to avoid unintended consequencesduring flight, and generate a reference database for commandproduct use during operations. This allowed InSight to meet itssurface operations goals in the desired timeframe. This paperdescribes the development, implementation and results ofInSight’s command product V&V process, its contribution tomission success, and challenges faced.
This paper summarizes key findings related to methods for the risk considerations of autonomy software. Existing methods for Verification and Validation (V&V) of autonomy software are summarized and a method for the assurance of autonomy software is suggested and demonstrated on a command execution use case. Risk and reliability are defined in the context of autonomy and an approach for risk assessment of autonomy is presented using an example use case. Key insights regarding areas of uncertainty for autonomy are provided, along with a suggested architecture for the systematic consideration of reliability within the context of a given autonomous planner.
Thermocouples (TCs) are a critical diagnostic sensor for high enthalpy environments including thermal protection systems (TPS) for atmospheric re-entry, hypersonic wind tunnels, and arc jet test facilities. They are used for both evaluating facility environmental conditions (enthalpy, heat flux) as well as for measuring in-situ material thermal response under test conditions (thermal conductivity, heat capacity, B’, etc.). Consequently, accurate TC measurements are fundamental for both material and facility verification and validation (V&V). However, a number of TC anomalies have been observed in both ground tests and flight instrumentation. These anomalies are caused via various coupling mechanisms: electrostatic gradients, temperature-dependent electrical shorts, radio pickup, magnetic induction, and pressure-dependent leaks. All of these mechanisms can, if unaddressed, result in obviously compromised results or worse, non-physical measurements which appear reasonable. This work will survey known anomaly mechanisms and discuss ongoing efforts to quantify and mitigate anomaly mechanisms including anomaly modeling, diagnostics, and new TC designs/development.