Search NASA⌕ Search

SEARCH · Search NASA

Results for “Verification Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Safe Physics-Informed Machine Learning for Dynamics and Control

This tutorial paper focuses on safe physics-informed machine learning in the context of dynamics and control, providing a comprehensive overview of how to integrate physical models and safety guarantees. As machine learning techniques enhance the modeling and control of complex dynamical systems, ensuring safety and stability remains a critical challenge, especially in safety-critical applications like autonomous vehicles, robotics, medical decision-making, and energy systems. We explore various approaches for embedding and ensuring safety constraints, including structural priors, Lyapunov and Control Barrier Functions, predictive control, projections, and robust optimization techniques. Additionally, we delve into methods for uncertainty quantification and safety verification, including reachability analysis and neural network verification tools, which help validate that control policies remain within safe operating bounds even in uncertain environments. The paper includes illustrative examples demonstrating the implementation aspects of safe learning frameworks that combine the strengths of data-driven approaches with the rigor of physical principles, offering a path toward the safe control of complex dynamical systems.

Drgona, Jan↗

Thermal Analysis of a Carbon Fiber Rope Barrier for Use in the Reusable Solid Rocket Motor Nozzle Joint-2

This study provides development and verification of analysis methods used to assess performance of a carbon fiber rope (CFR) thermal barrier system that is currently being qualified for use in Reusable Solid Rocket Motor (RSRM) nozzle joint-2. Modeled geometry for flow calculations considers the joint to be vented with the porous CFR barriers placed in the 'open' assembly gap. Model development is based on a 1-D volume filling approach where flow resistances (assembly gap and CFRs) are defined by serially connected internal flow and the porous media 'Darcy' relationships. Combustion gas flow rates are computed using the volume filling code by assuming a lumped distribution total joint fill volume on a per linear circumferential inch basis. Gas compressibility, friction and heat transfer are included in the modeling. Gas-to-wall heat transfer is simulated by concurrent solution of the compressible flow equations and a large thermal 2-D finite element (FE) conduction grid. The derived numerical technique loosely couples the FE conduction matrix with the compressible gas flow equations. Free constants that appear in the governing equations are calibrated by parametric model comparison to hot fire subscale test results. The calibrated model is then used to make full-scale motor predictions using RSRM aft dome environments. Model results indicate that CFR thermal barrier systems will provide a thermally benign and controlled pressurization environment for the RSRM nozzle joint-2 primary seal activation.

Clayton, J. Louie↗

Thermal Analysis of a Carbon Fiber Rope Barrier for Use in the Reusable Solid Rocket Motor Nozzle Joint-2

This study provides for development and verification of analysis methods used to assess performance of a carbon fiber rope (CFR) thermal barrier system that is currently being qualified for use in Reusable Solid Rocket Motor (RSRM) nozzle joint-2. Modeled geometry for flow calculations considers the joint to be vented with the porous CFR barriers placed in the "open' assembly gap. Model development is based on a 1-D volume filling approach where flow resistances (assembly gap and CFRs) are defined by serially connected internal flow and the porous media "Darcy" relationships. Combustion gas flow rates are computed using the volume filling code by assuming a lumped distribution total joint fill volume on a per linear circumferential inch basis. Gas compressibility, friction and heat transfer are included in the modeling. Gas-to-wall heat transfer is simulated by concurrent solution of the compressible flow equations and a large thermal 2-D finite element (FE) conduction grid. The derived numerical technique loosely couples the FE conduction matrix with the compressible gas flow equations, Free constants that appear in the governing equations are calibrated by parametric model comparison to hot fire subscale test results. The calibrated model is then used to make full-scale motor predictions using RSRM aft dome environments. Model results indicate that CFR thermal barrier systems will provide a thermally benign and controlled pressurization environment for the RSRM nozzle joint-2 primary seal activation.

Clayton, J. Louie↗

LH2 on-orbit storage tank support trunnion design and verification

A detailed fatigue analysis was conducted to provide verification of the trunnion design in the reusable Cryogenic Fluid Management Facility for Shuttle flights and to assess the performance capability of the trunnion E-glass/S-glass epoxy composite material. Basic material property data at ambient and liquid hydrogen temperatures support the adequacy of the epoxy composite for seven-mission requirement. Testing of trunnions fabricated to the flight design has verified adequate strength and fatigue properties of the design to meet the requirements of seven Shuttle flights.

Bailey, W. J.↗

Spectral Analysis of Forecast Error Investigated with an Observing System Simulation Experiment

The spectra of analysis and forecast error are examined using the observing system simulation experiment (OSSE) framework developed at the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASAGMAO). A global numerical weather prediction model, the Global Earth Observing System version 5 (GEOS-5) with Gridpoint Statistical Interpolation (GSI) data assimilation, is cycled for two months with once-daily forecasts to 336 hours to generate a control case. Verification of forecast errors using the Nature Run as truth is compared with verification of forecast errors using self-analysis; significant underestimation of forecast errors is seen using self-analysis verification for up to 48 hours. Likewise, self analysis verification significantly overestimates the error growth rates of the early forecast, as well as mischaracterizing the spatial scales at which the strongest growth occurs. The Nature Run-verified error variances exhibit a complicated progression of growth, particularly for low wave number errors. In a second experiment, cycling of the model and data assimilation over the same period is repeated, but using synthetic observations with different explicitly added observation errors having the same error variances as the control experiment, thus creating a different realization of the control. The forecast errors of the two experiments become more correlated during the early forecast period, with correlations increasing for up to 72 hours before beginning to decrease.

OSSE↗

MFANS 2024 - Formally Proving Characteristics of Cyber-Physical Systems

Cyber-physical systems (CPS) are engineered systems that rely on the smooth integration of computational algorithms and physical elements. This integration presents new challenges for verifying that systems will behave as expected. The goal of this presentation is to present current challenges and potential solutions for the formal verification of cyber-physical systems. For cyber systems, formal methods refer to systematically rigorous mathematical techniques employed in the specification, development, analysis, and verification of both software and hardware systems. Recent advancements in computer science have yielded sophisticated tools specifically designed to address challenges associated with formal methods in complex systems. These tools leverage various foundational concepts such as logic, formal languages, program semantics, type systems, type theory, and automata theory. A notable achievement in the application of formal methods is the seL4 microkernel, claimed to be the first general-purpose operating-system kernel to be verified. Its proof implies the absence of bugs and guarantees that the kernel meets specifications. For physical systems, dynamic and control theory has a history of using rigorous analytic techniques to prove functional correctness. Lyapunov, optimal, classical, modern, and robust control theories all provide rigorous mathematical methods both to analyze system performance and to design controller that can be guaranteed to meet certain objectives. Recent computational techniques like level set theory and reachability analysis provide assertions that a system's state will avoid unsafe regions. Even though success has been independently achieved for cyber systems and physical systems, the integration of such systems creates new challenges. In particular, there is an obvious discrepancy between finite-state machines and infinite-state systems, resulting in different approaches for modeling and analyzing these system. While it is possible to simulate hybrid systems, this provides only a demonstration of a performance and not proof. For hybrid systems, current formal methods and system analysis approaches typically require a workarounds to work on hybrid systems like CPS. This paper will outline the state of the art and limits of current practice for formally verifying CPS and will identify possible research directions that require attention.

97 MATHEMATICS AND COMPUTING↗

Europa Clipper Payload Verification and Validation: Test and Analysis Program Design

NASA’s Europa Clipper mission – which is planned to launch in the mid-2020s – is designed to investigate the habitability of the jovian moon Europa with a state-of-the-art suite of over 10 sensors and instruments. Now past its project critical design review, Europa Clipper is starting the execution of its verification and validation (V&V) program. The payload V&V function in particular is responsible for designing and coordinating a thorough test and analysis program that will confirm the proper functioning and performance of our payload instruments. Given the large number of instruments to manage, we have developed a suite of standardized tests and analyses that are designed to cover all of the common interfaces across instruments, while allowing for flexibility to support the unique needs of individual instruments. This suite of activities, coordinated and/or performed by the payload team, forms the bulk of our payload verification and validation campaign. This paper provides a detailed summary of the full set of V&V activities in that suite, covering the four categories of activities: analysis of lower-level evidence, analysis via simulation, testing in the system testbeds, and testing in assembly, test, and launch operations (ATLO). We will describe the purpose of each activity type, how the activities are developed, and the review process for each type of activity. We then define the complete list of activities written for each instrument and how subtleties specific to certain instruments are managed. The discussion will provide a comprehensive understanding of the activities in the payload V&V purview and how various interfaces with different levels of the project are managed. This paper also describes key metrics the payload team uses to track V&V progress and some insights into the teaming design and division of labor we use to ensure consistency and completeness across the entire payload suite. Finally, the paper will offer a snapshot of the status of the Payload V&V program prior to instrument delivery and offer insights into the upcoming phase of instrument delivery and the start of ATLO.

Lee, Dennis↗

Crash Certification by Analysis - Are We There Yet?

This paper addresses the issue of crash certification by analysis. This broad topic encompasses many ancillary issues including model validation procedures, uncertainty in test data and analysis models, probabilistic techniques for test-analysis correlation, verification of the mathematical formulation, and establishment of appropriate qualification requirements. This paper will focus on certification requirements for crashworthiness of military helicopters; capabilities of the current analysis codes used for crash modeling and simulation, including some examples of simulations from the literature to illustrate the current approach to model validation; and future directions needed to achieve "crash certification by analysis."

Jackson, Karen E.↗

The HTR-Proteus Benchmark: Analysis and Use as a Verification and Validation Case

This presentation outlines the evaluation and application of the HTR-Proteus benchmark as a verification and validation (V&V) case for advanced reactor modeling tools. The work supports the U.S. Department of Energy’s HALEU Availability Program (HAP) and the joint DOE/NRC DNCSH project, which aims to reduce criticality safety uncertainties in commercial-scale HALEU fuel cycle and transportation systems. The HTR-Proteus experiments, conducted at the Paul Scherrer Institute, provide high-fidelity data for TRISO-fueled, graphite-moderated pebble bed reactors with high neutron leakage—conditions relevant to HALEU transport scenarios. This study focuses on Cores 4.2 and 4.3 of the HTR-Proteus benchmark, analyzing key sources of uncertainty including pebble packing, TRISO particle positioning, and core height. Using Project Chrono for realistic pebble geometries and Serpent, SHIFT, and MCNP for neutronics simulations, the study quantifies the impact of these uncertainties on the effective multiplication factor (keff). Results show that a sample size of 110 pebble configurations is sufficient to converge keff, with ±30 pcm uncertainty due to packing randomness. TRISO positioning and core height variations also significantly influence keff, highlighting the importance of detailed modeling in V&V efforts. The benchmark serves as a valuable test case for validating the Griffin reactor physics code and improving confidence in HALEU system simulations.

73 - NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Analysis and design technology for high-speed aircraft structures

Recent high-speed aircraft structures research activities at NASA Langley Research Center are described. The following topics are covered: the development of analytical and numerical solutions to global and local thermal and structural problems, experimental verification of analysis methods, identification of failure mechanisms, and the incorporation of analysis methods into design and optimization strategies. The paper describes recent NASA Langley advances in analysis and design methods, structural and thermal concepts, and test methods.

Starnes, James H., Jr.↗

Mechanical verification of a schematic Byzantine clock synchronization algorithm

Schneider generalizes a number of protocols for Byzantine fault tolerant clock synchronization and presents a uniform proof for their correctness. The authors present a machine checked proof of this schematic protocol that revises some of the details in Schneider's original analysis. The verification was carried out with the EHDM system developed at the SRI Computer Science Laboratory. The mechanically checked proofs include the verification that the egocentric mean function used in Lamport and Melliar-Smith's Interactive Convergence Algorithm satisfies the requirements of Schneider's protocol.

Shankar, Natarajan↗

Validating an Air Traffic Management Concept of Operation Using Statistical Modeling

Validating a concept of operation for a complex, safety-critical system (like the National Airspace System) is challenging because of the high dimensionality of the controllable parameters and the infinite number of states of the system. In this paper, we use statistical modeling techniques to explore the behavior of a conflict detection and resolution algorithm designed for the terminal airspace. These techniques predict the robustness of the system simulation to both nominal and off-nominal behaviors within the overall airspace. They also can be used to evaluate the output of the simulation against recorded airspace data. Additionally, the techniques carry with them a mathematical value of the worth of each prediction-a statistical uncertainty for any robustness estimate. Uncertainty Quantification (UQ) is the process of quantitative characterization and ultimately a reduction of uncertainties in complex systems. UQ is important for understanding the influence of uncertainties on the behavior of a system and therefore is valuable for design, analysis, and verification and validation. In this paper, we apply advanced statistical modeling methodologies and techniques on an advanced air traffic management system, namely the Terminal Tactical Separation Assured Flight Environment (T-TSAFE). We show initial results for a parameter analysis and safety boundary (envelope) detection in the high-dimensional parameter space. For our boundary analysis, we developed a new sequential approach based upon the design of computer experiments, allowing us to incorporate knowledge from domain experts into our modeling and to determine the most likely boundary shapes and its parameters. We carried out the analysis on system parameters and describe an initial approach that will allow us to include time-series inputs, such as the radar track data, into the analysis

Statistical emulation↗

Thermal Fault Tolerance Analysis of Carbon Fiber Rope Barrier Systems for Use in the Reusable Solid Rocket Motor ( RSRM) Nozzle Joints

Carbon Fiber Rope (CFR) thermal barrier systems are being considered for use in several RSRM (Reusable Solid Rocket Motor) nozzle joints as a replacement for the current assembly gap close-out process/design. This study provides for development and test verification of analysis methods used for flow-thermal modeling of a CFR thermal barrier subject to fault conditions such as rope combustion gas blow-by and CFR splice failure. Global model development is based on a 1-D (one dimensional) transient volume filling approach where the flow conditions are calculated as a function of internal 'pipe' and porous media 'Darcy' flow correlations. Combustion gas flow rates are calculated for the CFR on a per-linear inch basis and solved simultaneously with a detailed thermal-gas dynamic model of a local region of gas blow by (or splice fault). Effects of gas compressibility, friction and heat transfer are accounted for the model. Computational Fluid Dynamic (CFD) solutions of the fault regions are used to characterize the local flow field, quantify the amount of free jet spreading and assist in the determination of impingement film coefficients on the nozzle housings. Gas to wall heat transfer is simulated by a large thermal finite element grid of the local structure. The employed numerical technique loosely couples the FE (Finite Element) solution with the gas dynamics solution of the faulted region. All free constants that appear in the governing equations are calibrated by hot fire sub-scale test. The calibrated model is used to make flight predictions using motor aft end environments and timelines. Model results indicate that CFR barrier systems provide a near 'vented joint' style of pressurization. Hypothetical fault conditions considered in this study (blow by, splice defect) are relatively benign in terms of overall heating to nozzle metal housing structures.

Clayton, J. Louie↗

Space Shuttle Cargo Integration Coupled Loads Analysis Lessons Learned

When a system experiences a loading environment characterized by rapidly varying forces, such as a rocket launch, a transient analysis is used to analyze the response of the system. The most common transient analysis methodology is the Coupled Loads Analysis(CLA). CLAs are used by the automotive and aerospace industry to analyze cars, trucks,planes, helicopters, spacecraft, etc. The Space Shuttle program used the CLA methodology to assess the compatibility of the payload with the Orbiter and the flight environment. The Space Shuttle Verification Loads Analysis (VLA) was a standardized CLA process that started between ten and thirteen months prior to launch, and included several meetings as well as analysis by both the Space Shuttle Program and the payload developers to verify that the payloads were compatible with the flight loads environment and would not interact negatively with the vehicle. Over the course of the Space Shuttle Program, many improvements were made to the process, which reduced cycle time and improved manifest flexibility. There were several issues which were never fully addressed, but work-arounds were developed to keep the process flowing. The lessons learned included automation of some processes and standardization of others, early assessments, improved documentation and better coordination with all stakeholders in the process. Lessons learned also included the limitations in the current process, and what to do to avoid the same issues.

ERICA E. BRUNO↗

Space Shuttle Cargo Integration Coupled Loads Analysis Lessons Learned

When a system experiences a loading environment characterized by rapidly varying forces, such as a rocket launch, a transient analysis is used to analyze the response of the system. The most common transient analysis methodology is the Coupled Loads Analysis (CLA). CLAs are used by the automotive and aerospace industry to analyze cars, trucks, planes, helicopters, spacecraft, etc. The Space Shuttle program used the CLA methodology to assess the compatibility of the payload with the Orbiter and the flight environment. The Space Shuttle Verification Loads Analysis (VLA) was a standardized CLA process that started between ten and thirteen months prior to launch, and included several meetings as well as analysis by both the Space Shuttle Program and the payload developers to verify that the payloads were compatible with the flight loads environment and would not interact negatively with the vehicle. Over the course of the Space Shuttle Program, many improvements were made to the process, which reduced cycle time and improved manifest flexibility. There were several issues which were never fully addressed, but workarounds were developed to keep the process flowing. The lessons learned included automation of some processes and standardization of others, early assessments, improved documentation and better coordination with all stakeholders in the process. Lessons learned also included the limitations in the current process, and what to do to avoid the same issues in the future.

Erica E. Bruno↗

Pattern Recognition for a Flight Dynamics Monte Carlo Simulation

The design, analysis, and verification and validation of a spacecraft relies heavily on Monte Carlo simulations. Modern computational techniques are able to generate large amounts of Monte Carlo data but flight dynamics engineers lack the time and resources to analyze it all. The growing amounts of data combined with the diminished available time of engineers motivates the need to automate the analysis process. Pattern recognition algorithms are an innovative way of analyzing flight dynamics data efficiently. They can search large data sets for specific patterns and highlight critical variables so analysts can focus their analysis efforts. This work combines a few tractable pattern recognition algorithms with basic flight dynamics concepts to build a practical analysis tool for Monte Carlo simulations. Current results show that this tool can quickly and automatically identify individual design parameters, and most importantly, specific combinations of parameters that should be avoided in order to prevent specific system failures. The current version uses a kernel density estimation algorithm and a sequential feature selection algorithm combined with a k-nearest neighbor classifier to find and rank important design parameters. This provides an increased level of confidence in the analysis and saves a significant amount of time.

Restrepo, Carolina↗

Material Property Estimation in Thin Battery Components Using Guided Wave Measurement, Experimental Dispersion Curve Extraction and Finite Element Modeling

At NASA, we are investigating nondestructive evaluation (NDE) and structural health monitoring (SHM) techniques to detect precursors of thermal runaway failure in lithium metal based lithium ion batteries (LIB). The approach is centered on computational simulation models to guide inspection and aid in interpretation of results. Since lithium metal LIBs have combinations of solid and fluid-filled porous materials, obtaining accurate material properties is both challenging and critical for successful simulation of battery inspection. To this end, we investigated a multi-verification approach for material characterization of thin battery components. First, a laser Doppler vibrometer (LDV) was used to measure guided wave fields in thin (microns-thick) battery components subject to broadband excitation. The time-space wavefield data was converted to frequency-wave number data to extract guided wave dispersion curves. A data visualization and post processing graphics user interface (GUI) was developed at NASA to aid the data exploration and analysis. Due to the thinness of the samples, low frequency-thickness-product plate wave approximations allowed for the calculation of closed-form solutions for material elastic property estimation. These approximations were then verified by calculating the Lamb wave solutions using the previously obtained material properties. Finally, the estimated elastic material properties were implemented in a COMSOL simulation model, and dispersion curves were extracted from simulation results. The dispersion curves and derived material properties were then compared to the analysis results from the experimental data. These comparisons informed on the accuracy of the measured material properties and helped demonstrate the accuracy of the finite element analysis (FEA) computational models. This assessment will prove vital when we start simulating more complex multi-layer components and poroelastic media. This paper gives a brief background of the problem space, outlines the workflow for data analysis and verification, shows results from the workflow, and gives an overview of future plans for simulation of lithium metal LIB inspection.

Peter Juarez↗

Verification and Validation of the General Mission Analysis Tool (GMAT)

This paper describes the processes and results of Verification and Validation (V&V) efforts for the General Mission Analysis Tool (GMAT). We describe the test program and environments, the tools used for independent test data, and comparison results. The V&V effort produced approximately 13,000 test scripts that are run as part of the nightly buildtest process. In addition, we created approximately 3000 automated GUI tests that are run every two weeks. Presenting all test results are beyond the scope of a single paper. Here we present high-level test results in most areas, and detailed test results for key areas. The final product of the V&V effort presented in this paper was GMAT version R2013a, the first Gold release of the software with completely updated documentation and greatly improved quality. Release R2013a was the staging release for flight qualification performed at Goddard Space Flight Center (GSFC) ultimately resulting in GMAT version R2013b.

GMAT↗