Results for “Validation Methods” — Page 4

NASA NTRS1992

Experimental validation of boundary element methods for noise prediction

Experimental validation of methods to predict radiated noise is presented. A combined finite element and boundary element model was used to predict the vibration and noise of a rectangular box excited by a mechanical shaker. The predicted noise was compared to sound power measured by the acoustic intensity method. Inaccuracies in the finite element model shifted the resonance frequencies by about 5 percent. The predicted and measured sound power levels agree within about 2.5 dB. In a second experiment, measured vibration data was used with a boundary element model to predict noise radiation from the top of an operating gearbox. The predicted and measured sound power for the gearbox agree within about 3 dB.

Seybert, A. F.↗

NASA NTRS1984

Validation of a fault-tolerant clock synchronization system

A validation method for the synchronization subsystem of a fault tolerant computer system is investigated. The method combines formal design verification with experimental testing. The design proof reduces the correctness of the clock synchronization system to the correctness of a set of axioms which are experimentally validated. Since the reliability requirements are often extreme, requiring the estimation of extremely large quantiles, an asymptotic approach to estimation in the tail of a distribution is employed.

Butler, R. W.↗

NASA NTRS2003

Validating the Autonomous Science Agent

This paper describes the validation process for the Autonomous Science Agent, a software agent that will fly onboard the EO-1 spacecraft from 2003-2004. This agent will recognize science events, retarget the spacecraft to respond to the science events, and reduce data downlink to only the highest value science data. The autonomous science agent has been designed using a layered architectural approach with specific redundant safeguards to reduce the risk of an agent malfunction to the EO-1 spacecraft. This 'safe' design is also in the process of being thoroughly validated by informal validation methods and extensive testing. This paper describes the analysis used to define agent safety, elements of the design that increase the safety of the agent, and the process being used to validate agent safety prior to the agent software controlling the spacecraft.

agent safety↗

NASA NTRSDate not supplied

Porosity Prediction and Detection During Composite Cure Using Simulation and Ultrasonic In-Situ Inspection Inside and Autoclave

Composite materials offer unique benefits in aerospace applications such as increased strength-to-weight ratio and improved fatigue properties. They are increasingly being used in major commercial aircraft programs. However, current processing methods can lead to defects in composite parts, which are currently identified using post-manufacturing inspection methods. A cure defects process model has been developed to predict the formation of manufacturing defects (e.g., porosity and fiber waviness) in composites based on the cure parameters and part geometry. However, a capability to directly validate porosity during cure did not exist. Validation methods included comparing resin pressure measured during cure with predictions by the process model and inspection/microscopy after cure. This study developed a high-temperature ultrasonic inspection system to detect porosity defects in composites during autoclave cure and experimentally verify the predictions of the model. The system operated inside an autoclave within an enclosure cooled by intermittent liquid nitrogen(LN2) injections. A high-temperature 2.25 MHz ultrasonic transducer was utilized to transmit ultrasonic waves through the tool plate and into the composite material and to measure the amplitude and time of flight of the reflected waves with a 1 mm × 1 mm resolution. Porosity was observed via the ultrasonic reflections, which experienced increased attenuation in regions of high porosity. Distinct regions of increased porosity were present due to uneven pressure across the panel, which was driven by an intentional misfit between the flat caul plate and the tapered composite panel with ply drops. These observations matched the predictions of the model using the inputs from the experiment (e.g., part material and geometry, cure cycle) and were validated by post-cure ultrasonic inspection and micrographs. The in-situ inspection system was able to successfully provide defect detection and localization and can be applied to future manufacturing of composite structure for aerospace applications.

Porosity↗

NASA NTRS2021

A Study into Validating a Coupled Method of Characteristics and Direct Simulation Monte Carlo Method Against Empirical Data

The following will outline the methodology and results of validating a coupled Method of Characteristics (MOC) and Direct Simulation Monte Carlo (DSMC) method. This research focused specifically on modeling plume impingement, induced by Reaction Control System (RCS) thrusters that flew on the National Aeronautics and Space Administration’s (NASA’s) space shuttle Discovery. For each simulation, the continuum portion of the RCS thruster was simulated using MOC for solving hyperbolic Partial Differential Equations (PDEs) and computed with the NASA code, Reacting and Multi-phase Program (RAMP). The solution was then implemented as a starting condition into the NASA DSMC code, Direct Simulation and Monte Carlo Analysis Code (DAC). Typically, DSMC models rely on code-to-code validation for fidelity. The significance of this research is in its ability to validate its models against empirical data. Prior to computing solutions for these simulations, the mesh size and structure were optimized and many variants of DSMC input parameters were iterated on in order to acquire a reliable, mesh-independent, fully optimized numerical solution. This research will discuss the mathematical formulation of MOC for nozzle flow and DSMC for rarefied gases. Additionally, it will provide an explanation of how to implement these mathematical concepts into the two solvers: RAMP and DAC. Ultimately, this research will demonstrate that the overall process illustrated produces results in good agreement with empirical data. As a consequence, the methodology presented is granted an increased level of confidence and will greatly contribute to the aerospace industry and its effort in understanding and predicting rarefied flow fields.

Direct Simulation Monte Carlo Analysis Code↗

NASA NTRS2021

A Study into Validating A Coupled Method of Characteristics And Direct Simulation Monte Carlo Method Against Empirical Data

The following will outline the methodology and results of validating a coupled Method of Characteristics (MOC) and Direct Simulation Monte Carlo (DSMC) method. This research focused specifically on modeling plume impingement, induced by Reaction Control System (RCS) thrusters that flew on the National Aeronautics and Space Administration’s (NASA’s) space shuttle Discovery. For each simulation, the continuum portion of the RCS thruster was simulated using MOC for solving hyperbolic Partial Differential Equations (PDEs) and computed with the NASA code, Reacting and Multi-phase Program (RAMP). The solution was then implemented as a starting condition into the NASA DSMC code, Direct Simulation and Monte Carlo Analysis Code (DAC). Typically, DSMC models rely on code-to-code validation for fidelity. The significance of this research is in its ability to validate its models against empirical data. Prior to computing solutions for these simulations, the mesh size and structure were optimized and many variants of DSMC input parameters were iterated on in order to acquire a reliable, mesh-independent, fully optimized numerical solution. This research will discuss the mathematical formulation of MOC for nozzle flow and DSMC for rarefied gases. Additionally, it will provide an explanation of how to implement these mathematical concepts into the two solvers: RAMP and DAC. Ultimately, this research will demonstrate that the overall process illustrated produces results in good agreement with empirical data. As a consequence, the methodology presented is granted an increased level of confidence and will greatly contribute to the aerospace industry and its effort in understanding and predicting rarefied flow fields.

DAC↗

NASA NTRS2004

Validating the autonomous EO-1 science agent

This paper describes the validation process for the Autonomous Science Agent, a software agent that is currently flying onboard NASA EO-1 spacecraft. The agent autonomously collects, analyzes, and reacts to onboard science data. The Autonomous Science Agent has been designed using a layered architectural approach with specific redundant safeguards to reduce the risk of an agent malfunction to the EO-1 spacecraft. This safe design has been thoroughly validated by informal validation methods supplemented by sub-system and system-level testing. This paper describes the analysis used to define agent safety, elements of the design that increase the safety of the agent, and the process used to validate agent safety.

software↗

NASA NTRS1990

Knowledge-based system verification and validation

The objective of this task is to develop and evaluate a methodology for verification and validation (V&V) of knowledge-based systems (KBS) for space station applications with high reliability requirements. The approach consists of three interrelated tasks. The first task is to evaluate the effectiveness of various validation methods for space station applications. The second task is to recommend requirements for KBS V&V for Space Station Freedom (SSF). The third task is to recommend modifications to the SSF to support the development of KBS using effectiveness software engineering and validation techniques. To accomplish the first task, three complementary techniques will be evaluated: (1) Sensitivity Analysis (Worchester Polytechnic Institute); (2) Formal Verification of Safety Properties (SRI International); and (3) Consistency and Completeness Checking (Lockheed AI Center). During FY89 and FY90, each contractor will independently demonstrate the user of his technique on the fault detection, isolation, and reconfiguration (FDIR) KBS or the manned maneuvering unit (MMU), a rule-based system implemented in LISP. During FY91, the application of each of the techniques to other knowledge representations and KBS architectures will be addressed. After evaluation of the results of the first task and examination of Space Station Freedom V&V requirements for conventional software, a comprehensive KBS V&V methodology will be developed and documented. Development of highly reliable KBS's cannot be accomplished without effective software engineering methods. Using the results of current in-house research to develop and assess software engineering methods for KBS's as well as assessment of techniques being developed elsewhere, an effective software engineering methodology for space station KBS's will be developed, and modification of the SSF to support these tools and methods will be addressed.

Johnson, Sally C.↗

NASA NTRSDate not supplied

Tailoring the Stiffness of Large Mirrors to Reduce Specific Optical Aberrations

Presented is a method that tailors the geometry of a mirror’s core structure as well as the mounting interface such that a desired set of aberrations is reduced. It will be shown that targeting specific aberrations as opposed to reducing overall surface figure error (SFE) is a valid method to reduce weight or improve performance. Analyzed first is the self-weight deflection of a thin circular plate of uniform thickness continuously supported along its outer edge. The Kirchhoff-Love plate bending equations are related to the radially symmetric Zernike polynomials of piston, defocus, and primary spherical aberration. Next, the mirror’s radial thickness is optimized to eliminate primary and secondary spherical aberrations when the outer edge is simply supported. As an example, the method is demonstrated using a 2-meter diameter aluminum mirror where surface deflections can have an appreciable effect at mid-to-near infrared wavelengths

mirror stiffness optimization↗

NASA NTRS2005

IEEE/NASA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation

This volume contains the Preliminary Proceedings of the 2005 IEEE ISoLA Workshop on Leveraging Applications of Formal Methods, Verification, and Validation, with a special track on the theme of Formal Methods in Human and Robotic Space Exploration. The workshop was held on 23-24 September 2005 at the Loyola College Graduate Center, Columbia, MD, USA. The idea behind the Workshop arose from the experience and feedback of ISoLA 2004, the 1st International Symposium on Leveraging Applications of Formal Methods held in Paphos (Cyprus) last October-November. ISoLA 2004 served the need of providing a forum for developers, users, and researchers to discuss issues related to the adoption and use of rigorous tools and methods for the specification, analysis, verification, certification, construction, test, and maintenance of systems from the point of view of their different application domains.

Margaria, Tiziana↗

NASA NTRS1977

A generalized vortex lattice method for subsonic and supersonic flow applications

If the discrete vortex lattice is considered as an approximation to the surface-distributed vorticity, then the concept of the generalized principal part of an integral yields a residual term to the vorticity-induced velocity field. The proper incorporation of this term to the velocity field generated by the discrete vortex lines renders the present vortex lattice method valid for supersonic flow. Special techniques for simulating nonzero thickness lifting surfaces and fusiform bodies with vortex lattice elements are included. Thickness effects of wing-like components are simulated by a double (biplanar) vortex lattice layer, and fusiform bodies are represented by a vortex grid arranged on a series of concentrical cylindrical surfaces. The analysis of sideslip effects by the subject method is described. Numerical considerations peculiar to the application of these techniques are also discussed. The method has been implemented in a digital computer code. A users manual is included along with a complete FORTRAN compilation, an executed case, and conversion programs for transforming input for the NASA wave drag program.

Miranda, L. R.↗

NASA NTRS2001

Follow That Satellite: EO-1 Maneuvers into Closed Formation With Landsat-7

As the Landsat-7 spacecraft continued NASA's historic program of earth imaging, begun over three decades ago, NASA launched the Earth Observing-1 (EO-1) spacecraft carrying examples of the next generation of Landsat-7 instruments. The validation method for these instruments was to have EO-1 fly in a close formation behind Landsat-7 on the same World Reference System path. From that formation hundreds of near coincident images would be taken by each spacecraft and compared to evaluate improvements in the EO-1 instruments. This paper will address the mission analysis required to launch and maneuver EO-1 into the formation with Landsat-7 where instrument validation was to occur plus a summary of completing the formation acquisition. EO-1 is required to operate one minute +/- 6 seconds behind Landsat-7 during the period of co-fly imaging with a cross track separation of within + 3 kilometers. This separation time can also be stated as a one minute +/- 6 seconds time difference in the Mean Local Time (MLT) at the descending nodes. Achieving the required MLT is heavily dependent on the time of launch. The EO-1 launch window, which had to accommodate the dual payloads of EO-1 and SAC-C, was very limited ranging from 0 to 22 seconds over the 16 day Landsat-7 WRS repeat cycle during which EO-1 was launched. Each EO-1 launch opportunity that occurred on a different day of a Landsat-7 16 day repeat cycle required a separate and distinct maneuver profile. These profiles varied significantly in duration and amount of onboard propellant required to achieve them. EO-1 launched on a day judged to have "medium" resource requirements for achieving the formation with Landsat-7. To phase EO-1 one minute behind Landsat-7 in the along track direction, a series of altitude adjusts separated by specific drift intervals were executed. Additional maneuvers slightly changed the EO-1 inclination to maintain the MLT requirements. Orbit maneuvers were planned and executed within errors of less than 1.5 percent and propellant usage was near nominal consuming 3.4 kilograms out of a launch and early orbit budget of 11 kilograms. The pre-launch 3 -sigma propellant budget allowed for 1+ years of EO-1 mission life. The success of the EO-1 launch and early orbit operations provided sufficient propellant for nearly 4 years of on orbit operations. Special action taken during the EO-1 maneuver period involved some maneuver re-planning to reduce concerns about a potential close approach between EO-1 and Landsat-7. The result of this re-planning was a safer close approach and improvement in future formation acquisition planning.

DeFazio, Robert↗

NASA NTRS2001

Follow That Satellite: EO-1 Maneuvers Into Close Formation With Landsat-7

As the Landsat-7 (LS-7) spacecraft continued NASA's historic program of earth imaging begun over three decades ago, NASA launched the Earth Observing-1 (EO-1) spacecraft carrying examples of the next generation of LS instruments. The validation method for these instruments was to have EO-1 fly in a close formation behind LS-7 on the same World Reference System (WRS) path. From that formation hundreds of near-coincident images would be taken by each spacecraft and compared to evaluate improvements in the EO-1 instruments. This paper will address the mission analysis required to launch and maneuver EO-1 into the formation with LS-7 where instrument validation was to occur plus a summary of completing the formation acquisition. Each EO-1 launch opportunity that occurred on a different day of a LS-7 16-day repeat cycle required a separate and distinct maneuver profile.

DeFazio, Robert L.↗

NASA NTRSDate not supplied

A Physics-Based Spatial Wake Interactional Model of Fixed-Wing Aircraft and Rotorcraft for Urban Air Mobility

This paper presents a study of a physics-based spatial wake interactional model for urban air mobility. A modeling method for spatial wake interaction of fixed-wing aircraft and rotorcraft is presented. The method validates an existing wake vortex model with flight data from the existing literature. The model is then adopted for both fixed-wing aircraft and rotorcraft. A lift circulation model is developed for rotorcraft to account for unsteady aerodynamics and the compressibility effect. This lift circulation model provides more realistic estimate of lift circulation for rotorcraft for used in wake vortex modeling. A vehicle-building wake interaction is studied using the method of image. The effects of wake vortex encounters on aerodynamic loads, stability, and control are examined. A flight control simulation is performed to demonstrate recovery during a simulated wake vortex encounter.

Wake Vortex Encounter↗

NASA NTRS2013

Modeling Weather Impact on Airport Arrival Miles-in-Trail Restrictions

When the demand for either a region of airspace or an airport approaches or exceeds the available capacity, miles-in-trail (MIT) restrictions are the most frequently issued traffic management initiatives (TMIs) that are used to mitigate these imbalances. Miles-intrail operations require aircraft in a traffic stream to meet a specific inter-aircraft separation in exchange for maintaining a safe and orderly flow within the stream. This stream of aircraft can be departing an airport, over a common fix, through a sector, on a specific route or arriving at an airport. This study begins by providing a high-level overview of the distribution and causes of arrival MIT restrictions for the top ten airports in the United States. This is followed by an in-depth analysis of the frequency, duration and cause of MIT restrictions impacting the Hartsfield-Jackson Atlanta International Airport (ATL) from 2009 through 2011. Then, machine-learning methods for predicting (1) situations in which MIT restrictions for ATL arrivals are implemented under low demand scenarios, and (2) days in which a large number of MIT restrictions are required to properly manage and control ATL arrivals are presented. More specifically, these predictions were accomplished by using an ensemble of decision trees with Bootstrap aggregation (BDT) and supervised machine learning was used to train the BDT binary classification models. The models were subsequently validated using data cross validation methods. When predicting the occurrence of arrival MIT restrictions under low demand situations, the model was able to achieve over all accuracy rates ranging from 84% to 90%, with false alarm ratios ranging from 10% to 15%. In the second set of studies designed to predict days on which a high number of MIT restrictions were required, overall accuracy rates of 80% were achieved with false alarm ratios of 20%. Overall, the predictions proposed by the model give better MIT usage information than what has been currently provided under current day operations. Traffic flow managers can use these predictions to identify potential MIT restrictions to eliminate (e.g., those occurring during low arrival demand periods), and to determine the days in which a significant number of restrictions may be required

Operation↗

NASA NTRS1989

Real-time application of knowledge-based systems

The Rapid Prototyping Facility (RPF) was developed to meet a need for a facility which allows flight systems concepts to be prototyped in a manner which allows for real-time flight test experience with a prototype system. This need was focused during the development and demonstration of the expert system flight status monitor (ESFSM). The ESFSM was a prototype system developed on a LISP machine, but lack of a method for progressive testing and problem identification led to an impractical system. The RPF concept was developed, and the ATMS designed to exercise its capabilities. The ATMS Phase 1 demonstration provided a practical vehicle for testing the RPF, as well as a useful tool. ATMS Phase 2 development continues. A dedicated F-18 is expected to be assigned for facility use in late 1988, with RAV modifications. A knowledge-based autopilot is being developed using the RPF. This is a system which provides elementary autopilot functions and is intended as a vehicle for testing expert system verification and validation methods. An expert system propulsion monitor is being prototyped. This system provides real-time assistance to an engineer monitoring a propulsion system during a flight.

Brumbaugh, Randal W.↗

NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗

NASA NTRSDate not supplied

Discovery and Analysis of Rare High-Impact Failure Modes using Adversarial RL-Informed Sampling

Adaptive learning agents have tremendous potential to handle critical tasks currently performed by humans. Unfortunately, due to their complexity, it can be difficult to verify that these learning agents do not have critical failure modes. Standard verification and validation methods often do not apply directly to learning agents and Monte Carlo methods have difficulty covering even a small fraction of the state space, especially in multiagent systems or over long time horizons. To overcome this difficulty, we demonstrate an adaptive stress-testing method based on reinforcement learning of correlations that raise the probability of failure. This approach has three key properties: (1) it is able to find rare failure modes with far greater sample efficiency than Monte Carlo methods, (2) it can estimate the true probability of a failure mode despite the inherent bias in the learning method, and (3) it is capable of learning and resampling compact representations of multimodal failure spaces. These properties are important in practice as we need to find disparate failure modes while accounting for their actual relevance. This is a significant advantage over traditional adaptive stress testing methods that give abstract likelihoods of particular failure instances, but cannot estimate the probability of a broader failure mode. We test our algorithm on a simple problem from the aviation domain where an autonomous aircraft lands in gusty wind conditions. The results suggest that we can find failure modes with far fewer samples than the Monte Carlo approach and simultaneously estimate the probability of failure.

Validation↗