Open Source Core Flight System (cFS) Flight Software (FSW) Verification & Validation (V&V) Final Summary: IV&V Analysis Technical Report
Explore the source record for details and available documents.
CFS↗
SEARCH · Search NASA
Results for “verification and validation (V&V)”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Bison Verification and Validation Activities for TRISO
Numerical modeling and simulation (M&S) tools play a key role in the research, development, and overall safety assessments of next-generation nuclear energy systems. One such tool, Bison, is a nuclear fuel performance code that is applicable to many fuel forms (e.g., light-water reactor fuel, oxide and metallic fuel for fast reactors, tri-structural isotropic (TRISO) fuel, and plate fuel), and it uses the finite element method to model the thermo- mechanical response of nuclear fuels. One fuel form widely utilized in Generation-IV high-temperature gas-cooled and fluoride- salt-cooled nuclear reactor concepts is TRISO fuel. Recently, Bison’s capabilities were significantly expanded to enable it to model the performance of TRISO particles and compacts. It is important that Bison’s computational results be reliable and predictive, since this code is used to inform high-consequence decisions. The various processes developed to address this issue generally entail two fundamental steps: verification and validation (V&V). Verification ensures that the code functions correctly and is reliable. Code/solution verification, code benchmark, and software quality assurance exercises are examples of verification activities. On the other hand, validation is the process of assessing a code’s capability to accurately model physical problems. Comparisons between code results and experiments quantify the validation level. Application of V&V procedures is crucial to the development of computational tools that are free of coding mistakes and can accurately represent reality. The current study presents an overview of Bison V&V activities relevant to the TRISO fuel concept, which include code/solution verification exercises, CRP-6 Benchmark—a Coordinated Research Program through the International Atomic Energy Agency (IAEA)—exercises, and validation exercises with the Advanced Gas Reactor (AGR)- 1/2/3/4 experiment series.
22 GENERAL STUDIES OF NUCLEAR REACTORS↗
Validation and Verification (V&V) of Safety-Critical Systems Operating Under Off-Nominal Conditions
Loss of control (LOC) remains one of the largest contributors to aircraft fatal accidents worldwide. Aircraft LOC accidents are highly complex in that they can result from numerous causal and contributing factors acting alone or more often in combination. Hence, there is no single intervention strategy to prevent these accidents. Research is underway at the National Aeronautics and Space Administration (NASA) in the development of advanced onboard system technologies for preventing or recovering from loss of vehicle control and for assuring safe operation under off-nominal conditions associated with aircraft LOC accidents. The transition of these technologies into the commercial fleet will require their extensive validation and verification (V&V) and ultimate certification. The V&V of complex integrated systems poses highly significant technical challenges and is the subject of a parallel research effort at NASA. This chapter summarizes the V&V problem and presents a proposed process that could be applied to complex integrated safety-critical systems developed for preventing aircraft LOC accidents. A summary of recent research accomplishments in this effort is referenced.
Belcastro, Christine M.↗
Integrated Medical Model (IMM) 4.0 Verification and Validation (V & V) Testing
No abstract available
Walton, M.↗
A Survey of ISS and Visiting Vehicle Returned Surfaces for Environmental Characterization and Computer Model Development
The Orbital Debris Engineering Model (ORDEM) developed by the NASA Orbital Debris Program Office (ODPO) is a data-driven model — extensive radar, optical, laboratory, and in situ measurement data sets have been used to build the model since its earliest versions. A salient aspect of professional software development is the verification and validation (V&V) process. Verification answers the question “Is the model built correctly?” while validation addresses the question “Did we build the correct model?” Less extensive, reserved, or independent data sets serve the validation requirement. Due to the dynamic nature of the orbital debris environment, it is critical to use contemporaneous data sources that represent the current environment to support ORDEM development and validation. ORDEM has utilized in situ data collected from Space Shuttle and Hubble Space Telescope surface inspections, now over a decade old. This historical dataset is fundamental for providing baseline in situ measurement data for sizes between 10 to 300 microns, but new data sources are being evaluated using returned surfaces from or near the International Space Station (ISS). This paper reviews a general microscopic survey of ISS soft goods, the Pressurized Mating Adapter 2 (PMA-2) blanket, and a limited-scope feasibility study conducted on the Space Exploration Technologies Corporation (SpaceX) Dragon capsule’s Thermal Protection System (TPS) material. The PMA-2 blanket, exposed to the space environment between 09 July 2013 and 25 February 2015, is an approximately 3.7 m2-area blanket composed of a betacloth outer layer and multiple ballistic fabric inner layers. The SpaceX Cargo Dragon capsule regularly visited the ISS from 2012 through 2020 and potentially provides a timely and well-characterized source of data for modeling purposes. The capsule’s lateral surfaces use SpaceX Proprietary Ablative Material (SPAM) TPS material, a syntactic foam, for thermal management during all mission phases. Seven SPAM extracted samples have been analyzed to date. This paper will provide an overview of the characterization completed for impact features by size, depth, impactor diameter, and the impactor residues chemical analyses, allowing a differentiation between micrometeoroids and orbital debris and a categorization by mass density and density class. Impactor diameter is estimated using damage equations generated from ground-based hypervelocity impact testing. The orbital debris impactors are compared to the current ORDEM 3.2 model of the environment at ISS altitudes. We briefly discuss the meteoroid impactors, including constituents and mass densities, in the general context of current models.
Phillip Anz-meador↗
A Survey of ISS and Visiting Vehicle Returned Surfaces for Environmental Characterization and Computer Model Development
The Orbital Debris Engineering Model (ORDEM) developed by the NASA Orbital Debris Program Office (ODPO) is a data-driven model — extensive radar, optical, laboratory, and in situ measurement data sets have been used to build the model since its earliest versions. A salient aspect of professional software development is the verification and validation (V&V) process. Verification answers the question “Is the model built correctly?” while validation addresses the question “Did we build the correct model?” Less extensive, reserved, or independent data sets serve the validation requirement. Due to the dynamic nature of the orbital debris environment, it is critical to use contemporaneous data sources that represent the current environment to support ORDEM development and validation. ORDEM has utilized in situ data collected from Space Shuttle and Hubble Space Telescope surface inspections, now over a decade old. This historical dataset is fundamental for providing baseline in situ measurement data for sizes between 10 to 300 microns, but new data sources are being evaluated using returned surfaces from or near the International Space Station (ISS). This paper reviews a general microscopic survey of ISS soft goods, the Pressurized Mating Adapter 2 (PMA-2) blanket, and a limited-scope feasibility study conducted on the Space Exploration Technologies Corporation (SpaceX) Dragon capsule’s Thermal Protection System (TPS) material. The PMA-2 blanket, exposed to the space environment between 09 July 2013 and 25 February 2015, is an approximately 3.7 m2-area blanket composed of a betacloth outer layer and multiple ballistic fabric inner layers. The SpaceX Cargo Dragon capsule regularly visited the ISS from 2012 through 2020 and potentially provides a timely and well-characterized source of data for modeling purposes. The capsule’s lateral surfaces use SpaceX Proprietary Ablative Material (SPAM) TPS material, a syntactic foam, for thermal management during all mission phases. Seven SPAM extracted samples have been analyzed to date. This paper will provide an overview of the characterization completed for impact features by size, depth, impactor diameter, and the impactor residues chemical analyses, allowing a differentiation between micrometeoroids and orbital debris and a categorization by mass density and density class. Impactor diameter is estimated using damage equations generated from ground-based hypervelocity impact testing. The orbital debris impactors are compared to the current ORDEM 3.2 model of the environment at ISS altitudes. We briefly discuss the meteoroid impactors, including constituents and mass densities, in the general context of current models.
Phillip Anz-Meador↗
Gemma V&V/UQ/Credibility Activities: FY2020 Progress
This report describes the credibility activities undertaken in support of Gemma code development in FY20, which include Verification & Validation (V&V), Uncertainty Quantification (UQ), and Credibility process application. The main goal of these activities is to establish capabilities and process frameworks that can be more broadly applied to new and more advanced problems as the Gemma code development effort matures. This will provide Gemma developers and analysts with the tools needed to generate credibility evidence in support of Gemma predictions for future use cases. The FY20 Gemma V&V/UQ/Credibility activities described in this report include experimental uncertainty analysis, the development and use of methods for optimal design of computer experiments, and the development of a framework for validation. These initial activities supported the development of broader credibility planning for Gemma that continued into FY21.
97 MATHEMATICS AND COMPUTING↗
Recent Advances in Discrete Crack Modeling Applied to Laminated Composites with Emphasis on: Floating Node Method, VCCT and Cohesive Zone Modeling
The present talk will provide an overview of the work performed during the Advanced Composites Project (ACP) on the development, and verification and validation of the Floating Node Method (FNM) as well as the Virtual Crack Closure Technique (VCCT) and cohesive zone modeling (CZM). The FNM is a finite element-based technique to represent crack networks. The complex nature of matrix crack-delamination interactions observed in unidirectional (UD) tape laminates suggests that a methodology such as the FNM method may be required to be able to accurately simulate damage progression in these laminates. Simulating crack onset and growth within the context of the FNM relies on techniques such as VCCT and CZM. The talk is organized to provide, via select examples, an overview of the breadth of the Verification & Validation (V&V) exercises performed during the ACP, and how these challenged the state-of-the art and guided further developments in discrete crack modeling, while helping to establish confidence in the progress made and map the challenges ahead. The performance of the VCCT and CZM individually, and in combination with the FNM can be assessed through verification exercises. These exercises typically consist of a comparison of simulation results to known numerical or analytical solutions. Verification is key to identify implementation issues and limitations that, otherwise, may remain undetected and cloud any subsequent validation efforts. Indeed, a subset of these numerical exercises led to further developments of the VCCT and the FNM method as will be illustrated. Before embarking on the subsequent validation of the framework, it is critical to have adequate characterization data. However, the testing campaign conducted revealed material responses that challenged the state-of-the-art and required further developments. The developments in CZM technology associated with the modeling of the responses of hybrid interfaces (fabric/UD) will be given as an example. Finally, the talk will conclude with a summary of the validation exercises performed under quasi-static and fatigue loadings, highlighting some of the key achievements, outstanding challenges and lessons learned.
finite elements↗
Calculation of the First Moment of Energy Using D-T Reactivity Formalisms Under the Maxwell-Boltzmann Distribution (Pt. 1)
Nuclear fusion science is an example of a scientific field with a rich history of expert involvement and scientific publications, which together, form an expert-knowledge base. One example of this history is the utilization of published reaction rates from a variety of authors. Investigators for Deuterium-Tritium (D-T) ion fusion can choose from using frequently cited methods: the Bosch and Hale reactivity, thermonuclear reaction rates from Caughlan and Fowler, and the reactivity evaluation from Miley, Towner & Ivich which forms the basis of the Naval Research Lab (NRL) formulary. There are other choices available. Each of the reactivity formulations considered here, are based upon the Maxwell- Boltzmann velocity distribution for D-T fusion ion reactants. Numerical methods for computer codes simulating hot, energetic plasmas, include tabulations of the reactivity, and the first moment of energy. The purpose of this study is to answer the question: what is the first moment of energy, and how has it been formulated? This report is part of a series of reports by the authors on D-T fusion-reaction formalism. The present focus is on defining the mathematical relationship of the first moment of D-T fusion ion kinetic energy, $\langle$E$\rangle$, with the fusion crosssection, fusion reactivity and its derivative with respect to ion-temperature. Three variants of the first moment $\langle$E$\rangle$ are analytically developed and explored: 1) constant cross-section, 2) a normalized first moment, and 3) a particular function of the first moment. Much of the mathematical development is relegated to the Appendices, for a concise presentation of topics and results. A research component of this discussion is our comparison of some of the reactivity and its derivative results between evaluations (1) versus (2) and (3), which the authors have not found in any other publication to date. This investigation is useful for validation, verification (V&V) and uncertainty quantification (UQ) as it compares the work of several authors.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
The Sandmeier Field Goniometer: A Measurement Tool for Bi-Directional Reflectance
Recent activities at the Remote Sensing Program at Stennis Space Center have identified the need to properly verify and validate data provided by the remote sensing community. One important variable, which effects remote sensing data is bi-directional reflectance distribution (BRDF). In order to quantify the effects of BRDF on man-made and natural ground targets, the Stennis Verification & Validation (V&V) team commissioned the Systems Engineering Division at NASA Ames Research Center to develop a Field Goniometer for use at the V&V Large Target Range and for various ground truthing missions. The Swiss Field Goniometer (FIGOS) was used as a benchmark instrument to design the new state of the art Sandmeier Field Goniometer (SGF), named after Stefan Sandmeier, developer of FIGOS. After establishing requirements for the SFG, design efforts began in early May 1998. The design of the SFG was completed in September 1998. Manufacturing, construction, and testing was completed in May 1999. The SFG was shipped to NASA SSC and fully operational by June 1999.
Turner, Mark↗
Measurement Sets and Sites Commonly used for Characterizations
Scientists with NASA's Earth Science Applications Directorate are creating a well-characterized Verification & Validation (V&V) site at the Stennis Space Center (SSC). This site enables the in-flight characterization of remote sensing systems and the data that they require. The data are predominantly acquired by commercial, high-spatial resolution satellite systems, such as IKONOS and QuickBird 2, and airborne systems. The smaller scale of these newer high-resolution remote sensing systems allows scientists to characterize the geometric, spatial, and radiometric data properties using a single V&V site. The targets and techniques used to characterize data from these newer systems can differ significantly from the earlier, coarser spatial resolution systems. Scientists are also using the SSC V&V site to characterize thermal infrared systems and active Light Detection and Ranging (LIDAR) systems. SSC employs geodetic targets, edge targets, radiometric tarps, and thermal calibration ponds to characterize remote sensing data products. This paper presents a proposed set of required measurements for visible-through-longwave infrared remote sensing systems, and a description of the Stennis characterization. Other topics discussed inslude: 1) use of ancillary atmospheric and solar measurements taken at SSC that support various characterizations, 2) other sites used for radiometric, geometric, and spatial characterization in the continental United States,a nd 3) the need for a standardized technique to be adopted by the Committee on Earth Observation Satellites (CEOS) and other organizations.
Pagnutti, Mary↗
Measurement Sets and Sites Commonly Used for Characterization
Scientists at NASA's Earth Science Applications Directorate are creating a well-characterized Verification & Validation (V&V) site at the Stennis Space Center. This site enables the in-flight characterization of remote sensing systems and the data they acquire. The data are predominantly acquired by commercial, high spatial resolution satellite systems, such as IKONOS and QuickBird 2, and airborne systems. The smaller scale of these newer high resolution remote sensing systems allows scientists to characterize the geometric, spatial, and radiometric data properties using a single V&V site. The targets and techniques used to characterize data from these newer systems can differ significantly from the techniques used to characterize data from the earlier, coarser spatial resolution systems. Scientists are also using the SSC V&V site to characterize thermal infrared systems and active LIDAR systems. SSC employs geodetic targets, edge targets, radiometric tarps, and thermal calibration ponds to characterize remote sensing data products. This paper presents a proposed set of required measurements for visible through long-wave infrared remote sensing systems and a description of the Stennis characterization. Other topics discussed include: 1) The use of ancillary atmospheric and solar measurements taken at SSC that support various characterizations; 2) Additional sites used for radiometric, geometric, and spatial characterization in the continental United States; 3) The need for a standardized technique to be adopted by CEOS and other organizations.
Pagnutti, Mary↗
Performance Monitoring and Assessment of Neuro-Adaptive Controllers for Aerospace Applications Using a Bayesian Approach
Modern exploration missions require modern control systems-control systems that can handle catastrophic changes in the system's behavior, compensate for slow deterioration in sustained operations, and support fast system ID. Adaptive controllers, based upon Neural Networks have these capabilities, but they can only be used safely if proper verification & validation (V&V) can be done. In this paper we present our V & V approach and simulation result within NASA's Intelligent Flight Control Systems (IFCS).
Gupta, Pramod↗
Measurement Sets and Sites Commonly Used for High Spatial Resolution Image Product Characterization
Scientists within NASA's Applied Sciences Directorate have developed a well-characterized remote sensing Verification & Validation (V&V) site at the John C. Stennis Space Center (SSC). This site has enabled the in-flight characterization of satellite high spatial resolution remote sensing system products form Space Imaging IKONOS, Digital Globe QuickBird, and ORBIMAGE OrbView, as well as advanced multispectral airborne digital camera products. SSC utilizes engineered geodetic targets, edge targets, radiometric tarps, atmospheric monitoring equipment and their Instrument Validation Laboratory to characterize high spatial resolution remote sensing data products. This presentation describes the SSC characterization capabilities and techniques in the visible through near infrared spectrum and examples of calibration results.
Pagnutti, Mary↗
Stennis Space Center Verification & Validation Capabilities
Scientists within NASA s Applied Sciences Directorate have developed a well-characterized remote sensing Verification & Validation (V&V) site at the John C. Stennis Space Center (SSC). This site enables the in-flight characterization of satellite and airborne high spatial and moderate resolution remote sensing systems and their products. The smaller scale of the newer high resolution remote sensing systems allows scientists to characterize geometric, spatial, and radiometric data properties using a single V&V site. The targets and techniques used to characterize data from these newer systems can differ significantly from the techniques used to characterize data from the earlier, coarser spatial resolution systems. Scientists are also using the SSC V&V site to characterize thermal infrared systems and active lidar systems. SSC employs geodetic targets, edge targets, radiometric tarps, atmospheric monitoring equipment, and thermal calibration ponds to characterize remote sensing data products. The SSC Instrument Validation Lab is a key component of the V&V capability and is used to calibrate field instrumentation and to provide National Institute of Standards and Technology traceability. This poster presents a description of the SSC characterization capabilities and examples of calibration data.
Pagnutti, Mary↗
Stennis Space Center Verification & Validation Capabilities
Scientists within NASA#s Applied Research & Technology Project Office (formerly the Applied Sciences Directorate) have developed a well-characterized remote sensing Verification & Validation (V&V) site at the John C. Stennis Space Center (SSC). This site enables the in-flight characterization of satellite and airborne high spatial resolution remote sensing systems and their products. The smaller scale of the newer high resolution remote sensing systems allows scientists to characterize geometric, spatial, and radiometric data properties using a single V&V site. The targets and techniques used to characterize data from these newer systems can differ significantly from the techniques used to characterize data from the earlier, coarser spatial resolution systems. Scientists have used the SSC V&V site to characterize thermal infrared systems. Enhancements are being considered to characterize active lidar systems. SSC employs geodetic targets, edge targets, radiometric tarps, atmospheric monitoring equipment, and thermal calibration ponds to characterize remote sensing data products. Similar techniques are used to characterize moderate spatial resolution sensing systems at selected nearby locations. The SSC Instrument Validation Lab is a key component of the V&V capability and is used to calibrate field instrumentation and to provide National Institute of Standards and Technology traceability. This poster presents a description of the SSC characterization capabilities and examples of calibration data.
Pagnutti, Mary↗
Internal NASA Study: NASAs Protoflight Research Initiative
The NASA Protoflight Research Initiative is an internal NASA study conducted within the Office of the Chief Engineer to better understand the use of Protoflight within NASA. Extensive literature reviews and interviews with key NASA members with experience in both robotic and human spaceflight missions has resulted in three main conclusions and two observations. The first conclusion is that NASA's Protoflight method is not considered to be "prescriptive." The current policies and guidance allows each Program/Project to tailor the Protoflight approach to better meet their needs, goals and objectives. Second, Risk Management plays a key role in implementation of the Protoflight approach. Any deviations from full qualification will be based on the level of acceptable risk with guidance found in NPR 8705.4. Finally, over the past decade (2004 - 2014) only 6% of NASA's Protoflight missions and 6% of NASA's Full qualification missions experienced a publicly disclosed mission failure. In other words, the data indicates that the Protoflight approach, in and of it itself, does not increase the mission risk of in-flight failure. The first observation is that it would be beneficial to document the decision making process on the implementation and use of Protoflight. The second observation is that If a Project/Program chooses to use the Protoflight approach with relevant heritage, it is extremely important that the Program/Project Manager ensures that the current project's requirements falls within the heritage design, component, instrument and/or subsystem's requirements for both the planned and operational use, and that the documentation of the relevant heritage is comprehensive, sufficient and the decision well documented. To further benefit/inform this study, a recommendation to perform a deep dive into 30 missions with accessible data on their testing/verification methodology and decision process to research the differences between Protoflight and Full Qualification missions' Design Requirements and Verification & Validation (V&V) (without any impact or special request directly to the project).
Protoflight↗