Search NASA⌕ Search

SEARCH · Search NASA

Results for “V&V process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Assessing V and V Processes for Automation with Respect to Vulnerabilities to Loss of Airplane State Awareness

Automation has contributed substantially to the sustained improvement of aviation safety by minimizing the physical workload of the pilot and increasing operational efficiency. Nevertheless, in complex and highly automated aircraft, automation also has unintended consequences. As systems become more complex and the authority and autonomy (A&A) of the automation increases, human operators become relegated to the role of a system supervisor or administrator, a passive role not conducive to maintaining engagement and airplane state awareness (ASA). The consequence is that flight crews can often come to over rely on the automation, become less engaged in the human-machine interaction, and lose awareness of the automation mode under which the aircraft is operating. Likewise, the complexity of the system and automation modes may lead to poor understanding of the interaction between a mode of automation and a particular system configuration or phase of flight. These and other examples of mode confusion often lead to mismanaging the aircraftâ€"TM"s energy state or the aircraft deviating from the intended flight path. This report examines methods for assessing whether, and how, operational constructs properly assign authority and autonomy in a safe and coordinated manner, with particular emphasis on assuring adequate airplane state awareness by the flight crew and air traffic controllers in off-nominal and/or complex situations.

Whitlow, Stephen↗

A Spectrum of IV and V Modeling Techniques

The aerospace industry in general and NASA in particular is using more (semi-formal) model-based software development. Model-based development produces a collection of artifacts, for example, state diagrams, module diagrams (such as class diagrams), control-block diagrams, etc. These artifacts may than be used as a basis for auto code generation for production use. Therefore, these models must be properly evaluated in the IV and V process. IV and V practitioners know how assess standard procedural systems. But what can we du about IV and V of model-based systems? The goal of the work outlined in this proposal is to use cost effective automated techniques to the largest extent possible during the IV and V process. Our working hypotheses are: 1. There exists a range of validation techniques that can assess models built using a range of modeling techniques of increasing cost and complexity. Specifically, we hypotesize that the "cheaper" techniques can find faults cheaply and early in a project. These early results are then used to predict if this is a problem system and if a more elaborate and expensive IV and V effort is justified. 2. There exists a set of migration procedures that let us seamlessly move from simple models using cheaper techniques into more elaborate models suitable for a more expensive and detailed analysis. 3. We further hypothesize that this migration process is much cheaper than simply remodeling the system under investigation from scratch when moving to models needed for the more detailed and expensive IV and V assessments.

Heimdahl, Mats↗

Verification and Validation of the Stakeholder Tool for Assessing Radioactive Transportation (START) – 22323

The U.S. Department of Energy (DOE) Office of Integrated Waste Management is planning for the eventual transportation, storage, and disposal of spent nuclear fuel (SNF) and high-level radioactive waste (HLW) from nuclear power plant and DOE sites. The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based, geospatial decision-support tool developed for evaluating routing options and other aspects of transporting SNF and HLW, covering rail, truck, barge, and intermodal infrastructure and operations in the continental United States. The verification and validation (V&V) process is intended to independently assess START to provide confidence in the ability of START to accurately provide intended results. The V&V process checks the START tool using a variety of methods, ranging from independent hand calculations to comparison of START performance and results to those of other codes. The V&V activity was conducted independently from the START development team with opportunities to provide feedback and collaborate throughout the process. The V&V analyzed attributes of transportation routes produced by START, including route distance and both population and population density captured within buffer zones around routes. Population in the buffer zone, population density in the buffer zone, and route distance were all identified as crucial outputs of the START code and were subject to V&V tasks. Some of the improvements identified through the V&V process were standardizing the underlying population data in START, changing the projection of the population raster data, and changes to the methodology used for population density to improve its applicability for expected users. This collaboration also led to suggested improvements to some of the underlying shape file segments within START. A formal series of test routes went through the V&V process from START version 3.2.1 following the implementation of the recommended improvements to START. The V&V of the buffer zone population and distance reported in START were achieved by utilizing a custom workflow developed in QGIS. This platform was chosen because it is fully independent of START. The buffer zone population and route distances in START showed excellent agreement to independent V&V test results. Over 200 route test cases were run; in all cases, the percent difference in the population within the buffer zone was less than +/- 5%, and the majority of cases were below +/- 1%. These cases were also duplicated in ArcMap, which is Environmental Systems Research Institute’s (ESRI) GIS desktop application. The ArcMap results also show good agreement; however, they cannot be considered independent of START because START also uses ESRI software. The V&V tasks are currently addressing population density and estimated radiation dose in START as well as implementing further automation of the V&V process to allow larger suites of test routes to be run more easily and quickly by less experienced GIS users. These improvements to the V&V process will make it a valuable tool for the development team and the stakeholders so that each new iteration of START can easily be checked for quality and consistency. This paper and presentation will describe the V&V process, findings and implications, and ongoing V&V activities.

START, DOE, Transporation, dose, GIS↗

Verification and Validation of START: A Spent Nuclear Fuel Routing and Decision Support Tool

The Stakeholder Tool for Assessing Radioactive Transportation (START) is a web-based geospatial decision-support tool being developed by the US Department of Energy’s Office of Integrated Waste Management (IWM) to support federal interim storage for spent nuclear fuel (SNF) and associated transportation. START provides many functions for the IWM program including: serving as a communications tool for conveying geospatial data and information, an options analysis tool for exploring potential transport modes and routes for transporting SNF from nuclear power plants to future federal interim storage facilities, an emergency response planning tool for Tribes and States to identify training needs along potential SNF transport corridors, an environmental analysis tool for estimating potential radiation dose exposure from incident-free and incident-case SNF transport conditions, and a systems analysis support tool providing route-related inputs for system throughput analysis. As part of the START development process, a verification and validation (V&V) effort is being undertaken. In the initial V&V phase, several outputs of the START tool were checked such as the total distance, population and population densities within the buffer zone, and incident free dose. The V&V process is fluid as it will be utilized after each version change to ensure that the core functionalities of the tool are maintained and the results are consistent with the previous versions. Efforts have also been put into developing scripts to aid in the process of automating certain sections of the V&V work. As some of the V&V efforts use Environmental Systems Research Institute’s (ESRI) tools upon which the START framework is built, the START data were compared against outputs from tools like Quantum Geographic Information System (QGIS) for buffer zone populations and route lengths to ensure independence of the V&V process. Good agreement was observed between the START results and the independent V&V studies with the majority of the differences falling between 1% and 5% for populations within the buffer zone and route distance. This presentation describes the development and design of the START tool, V&V methods employed for various metrics of interest and their respective results, and future plans.

START, transportation, GIS, V&V↗

A Greenhouse-Gas Information System: Monitoring and Validating Emissions Reporting and Mitigation

Current GHG-mitigating regimes, whether internationally agreed or self-imposed, rely on the aggregation of self-reported data, with limited checks for consistency and accuracy, for monitoring. As nations commit to more stringent GHG emissions-mitigation actions and as economic rewards or penalties are attached to emission levels, self-reported data will require independent confirmation that they are accurate and reliable, if they are to provide the basis for critical choices and actions that may be required. Supporting emissions-mitigation efforts and agreements, as well as monitoring energy- and fossil-fuel intensive national and global activities would be best achieved by a process of: (1) monitoring of emissions and emission-mitigation actions, based, in part, on, (2) (self-) reporting of pertinent bottom-up inventory data, (3) verification that reported data derive from and are consistent with agreed-upon processes and procedures, and (4) validation that reported emissions and emissions-mitigation action data are correct, based on independent measurements (top-down) derived from a suite of sensors in space, air, land, and, possibly, sea, used to deduce and attribute anthropogenic emissions. These data would be assessed and used to deduce and attribute measured GHG concentrations to anthropogenic emissions, attributed geographically and, to the extent possible, by economic sector. The validation element is needed to provide independent assurance that emissions are in accord with reported values, and should be considered as an important addition to the accepted MRV process, leading to a MRV&V process. This study and report focus on attributes of a greenhouse-gas information system (GHGIS) needed to support MRV&V needs. These needs set the function of such a system apart from scientific/research monitoring of GHGs and carbon-cycle systems, and include (not exclusively): the need for a GHGIS that is operational, as required for decision-support; the need for a system that meets specifications derived from imposed requirements; the need for rigorous calibration, verification, and validation (CV&V) standards, processes, and records for all measurement and modeling/data-inversion data; the need to develop and adopt an uncertainty-quantification (UQ) regimen for all measurement and modeling data; and the requirement that GHGIS products can be subjected to third-party questioning and scientific scrutiny. This report examines and assesses presently available capabilities that could contribute to a future GHGIS. These capabilities include sensors and measurement technologies; data analysis and data uncertainty quantification (UQ) practices and methods; and model-based data-inversion practices, methods, and their associated UQ. The report further examines the need for traceable calibration, verification, and validation processes and attached metadata; differences between present science-/research-oriented needs and those that would be required for an operational GHGIS; the development, operation, and maintenance of a GHGIS missions-operations center (GMOC); and the complex systems engineering and integration that would be required to develop, operate, and evolve a future GHGIS. Present monitoring systems would be heavily relied on in any GHGIS implementation at the outset and would likely continue to provide valuable future contributions to GHGIS. However, present monitoring systems were developed to serve science/research purposes. This study concludes that no component or capability presently available is at the level of technological maturity and readiness required for implementation in an operational GHGIS today. However, purpose-designed and -built components could be developed and implemented in support of a future GHGIS. The study concludes that it is possible to develop and provide a capability-driven prototype GHGIS, as part of a Phase-1 effort, within three years from project-funding start, that would make use of and integrate existing sensing and system capabilities. As part of a Phase-2 effort, a requirements-driven, operational GHGIS could be developed, within ten years from project funding start. That schedule is driven by the development and long lead-times for some system components. The two efforts would be focused on different deliverables but could commence concurrently, to save time, if that was deemed desirable. We note that, developing and supporting an operational GHGIS will require a new approach and management, sustained funding and other support, as well as technical advances and development of purpose-built components that meet the requisite specifications. A functioning GHGIS will provide the basis for reasoned choices on how best to respond to rising GHG levels, especially when proposed U.S. actions are compared with or conditioned on the actions of other nations.

54 ENVIRONMENTAL SCIENCES↗

ASC CSSE Level 2 Milestone #6362: Resilient Asynchronous Many Task Programming Model

This report is an outcome of the ASC CSSE Level 2 Milestone 6362: Analysis of Re- silient Asynchronous Many-Task (AMT) Programming Model. It comprises a summary and in-depth analysis of resilience schemes adapted to the AMT programming model. Herein, performance trade-offs of a resilient-AMT prograrnming model are assessed through two ap- proaches: (1) an analytical model realized by discrete event simulations and (2) empirical evaluation of benchmark programs representing regular and irregular workloads of explicit partial differential equation solvers. As part of this effort, an AMT execution simulator and a prototype resilient-AMT programming framework have been developed. The former permits us to hypothesize the performance behavior of a resilient-AMT model, and has undergone a verification and validation (V&V) process. The latter allows empirical evaluation of the perfor- mance of resilience schemes under emulated program failures and enabled the aforementioned V&V process. The outcome indicates that (1) resilience techniques implemented within an AMT framework allow efficient and scalable recovery under frequent failures, that (2) the abstraction of task and data instances in the AMT programming model enables readily us- able Application Program Interfaces (APIs) for resilience, and that (3) this abstraction enables predicting the performance of resilient-AMT applications with a simple simulation infrastruc- ture. This outcome will provide guidance for the design of the AMT programming model and runtime systems, user-level resilience support, and application development for ASC's next generation platforms (NGPs).

97 MATHEMATICS AND COMPUTING↗

Evaluation of the Relative Merits of Herbaceous and Woody Crops for Use in Tunable Thermochemical Processing

This report summarizes the work and findings of the grant work conducted from January 2009 until September 2011 under the collaboration between Ceres, Inc. and Choren USA, LLC. This DOE-funded project involves a head-to-head comparison of two types of dedicated energy crops in the context of a commercial gasification conversion process. The main goal of the project was to gain a better understanding of the differences in feedstock composition between herbaceous and woody species, and how these differences may impact a commercial gasification process. In this work, switchgrass was employed as a model herbaceous energy crop, and willow as a model short-rotation woody crop. Both crops are species native to the U.S. with significant potential to contribute to U.S. goals for renewable liquid fuel production, as outlined in the DOE Billion Ton Update (http://www1.eere.energy.gov/biomass/billion_ton_update.html, 2011). In some areas of the U.S., switching between woody and herbaceous feedstocks or blending of the two may be necessary to keep a large-scale gasifier operating near capacity year round. Based on laboratory tests and process simulations it has been successfully shown that suitable high yielding switchgrass and willow varieties exist that meet the feedstock specifications for large scale entrained flow biomass gasification. This data provides the foundation for better understanding how to use both materials in thermochemical processes. It has been shown that both switchgrass and willow varieties have comparable ranges of higher heating value, BTU content and indistinguishable hydrogen/carbon ratios. Benefits of switchgrass, and other herbaceous feedstocks, include its low moisture content, which reduce energy inputs and costs for drying feedstock. Compared to the typical feedstock currently being used in the Carbo-V® process, switchgrass has a higher ash content, combined with a lower ash melting temperature. Whether or not this may cause inefficiencies in the process, needs to be verified by long term test runs. Currently, there are not sufficient operational test data available for the Carbo-V® process for the utilization of higher ash content feedstocks. The application of currently evolving biomass pretreatment technologies, such as pelletization and torrefaction, will be able to expand the portfolio of biomass varieties and species acceptable in gasification processes. Tests showed that 6 mm diameter pellets of switchgrass were superior to 8 mm diameter pellets produced in a flat dye press, and that torrefaction of switchgrass produced an excellent (but currently costly) feedstock that could be handled, crushed, and combusted in a manner compatible with any coal-fed gasification facility. Ceres will use this information in the development of high yielding, dedicated energy crops specifically tailored for thermochemical conversion. CHOREN will make use of the information for improvement or development of low cost, highly efficient biomass gasification processes that convert a wide variety of biomass feedstocks to fuels, chemicals, heat and power via the production of tar free green syngas on an industrial scale.

09 BIOMASS FUELS↗

Developing and Implementing a Process to Verify and Validate the NASA InSight Mission’s Instrument Command Products

The NASA InSight Mars lander is a roboticspacecraft which is investigating the interior structure of Marsthrough a series of seismological measurements andexperiments. Much of the commanding of activities performedby the instruments was done using ‘blocks’ – reusable functionswritten in Virtual Machine Language stored onboard thespacecraft– and sequences (series of commands) that werecreated by teams to operate their instruments. It was vital thatthese blocks and sequences (collectively called commandproducts) be thoroughly examined prior to onboard executionusing a comprehensive test program to reduce risk to thespacecraft and instruments and ensure smooth operations onthe surface of Mars. The Verification and Validation (V&V)process that was designed and implemented during the monthsleading up to InSight’s launch and landing successfullyuncovered a number of issues that could have causedoperational delays or impacted hardware. The process alsohelped adapt ground tools to better model onboard activities,create operational principles to avoid unintended consequencesduring flight, and generate a reference database for commandproduct use during operations. This allowed InSight to meet itssurface operations goals in the desired timeframe. This paperdescribes the development, implementation and results ofInSight’s command product V&V process, its contribution tomission success, and challenges faced.

Cloutier, Kyle↗

Computational solution verification and validation applied to a thermal model of a ruggedized instrumentation package

This study details a methodology for quantification of errors and uncertainties of a finite element heat transfer model applied to a Ruggedized Instrumentation Package (RIP). The proposed verification and validation (V&V) process includes solution verification to examine errors associated with the code's solution techniques, and model validation to assess the model's predictive capability for quantities of interest. The model was subjected to mesh resolution and numerical parameters sensitivity studies to determine reasonable parameter values and to understand how they change the overall model response and performance criteria. To facilitate quantification of the uncertainty associated with the mesh, automatic meshing and mesh refining/coarsening algorithms were created and implemented on the complex geometry of the RIP. Automated software to vary model inputs was also developed to determine the solution’s sensitivity to numerical and physical parameters. The model was compared with an experiment to demonstrate its accuracy and determine the importance of both modelled and unmodelled physics in quantifying the results' uncertainty. An emphasis is placed on automating the V&V process to enable uncertainty quantification within tight development schedules.

97 MATHEMATICS AND COMPUTING↗

Self-alignment features for III-V ridge process and angled facet die

A method of forming a laser including device is provided that in one embodiment includes providing a laser chip including at least one ridge structure that provides an alignment features. The method further includes bonding a type IV photonics chip to the laser chip, wherein a vertical alignment feature from the type IV photonics chip is inserted in a recess relative to the at least one ridge structure that provides the alignment features of the laser structure.

42 ENGINEERING↗

Self-alignment features for III-V ridge process and angled facet die

A method of forming a laser including device is provided that in one embodiment includes providing a laser chip including at least one ridge structure that provides an alignment features. The method further includes bonding a type IV photonics chip to the laser chip, wherein a vertical alignment feature from the type IV photonics chip is inserted in a recess relative to the at least one ridge structure that provides the alignment features of the laser structure.

Barwicz, Tymon↗

Applying Independent Verification and Validation to Automatic Test Equipment

This paper describes a general overview of applying Independent Verification and Validation (IV&V) to Automatic Test Equipment (ATE). The overview is not inclusive of all IV&V activities that can occur or of all development and maintenance items that can be validated and verified, during the IV&V process. A sampling of possible IV&V activities that can occur within each phase of the ATE life cycle are described.

Calhoun, Cynthia C.↗

An Overview of NASA's IM&S Verification and Validation Process Plan and Specification for Space Exploration

NASA's Exploration Systems Mission Directorate (ESMD) is implementing a management approach for modeling and simulation (M&S) that will provide decision-makers information on the model's fidelity, credibility, and quality. This information will allow the decision-maker to understand the risks involved in using a model's results in the decision-making process. This presentation will discuss NASA's approach for verification and validation (V&V) of its models or simulations supporting space exploration. This presentation will describe NASA's V&V process and the associated M&S verification and validation (V&V) activities required to support the decision-making process. The M&S V&V Plan and V&V Report templates for ESMD will also be illustrated.

Gravitz, Robert M.↗

Implementing the Mars Science Laboratory Terminal Descent Sensor Field Test Campaign

The Mars Science Laboratory (MSL) will deliver a 900 kg rover to the surface of Mars in August 2012. MSL will utilize a new pulse-Doppler landing radar, the Terminal Descent Sensor (TDS). The TDS employs six narrow-beam antennas to provide unprecedented slant range and velocity performance at Mars to enable soft touchdown of the MSL rover using a unique sky crane Entry, De-scent, and Landing (EDL) technique. Prior to use on MSL, the TDS was put through a rigorous verification and validation (V&V) process. A key element of this V&V was operating the TDS over a series of field tests, using flight-like profiles expected during the descent and landing of MSL over Mars-like terrain on Earth. Limits of TDS performance were characterized with additional testing meant to stress operational modes outside of the expected EDL flight profiles. The flight envelope over which the TDS must operate on Mars encompasses such a large range of altitudes and velocities that a variety of venues were neces-sary to cover the test space. These venues included an F/A-18 high performance aircraft, a Eurocopter AS350 AStar helicopter and 100-meter tall Echo Towers at the China Lake Naval Air Warfare Center. Testing was carried out over a five year period from July 2006 to June 2011. TDS performance was shown, in gen-eral, to be excellent over all venues. This paper describes the planning, design, and implementation of the field test campaign plus results and lessons learned.

Mars Science Laboratory (MSL)↗

Optimizing IV and V for Mature Organizations

NASA is intending for its future software development agencies to have at least a Level 3 rating in the Carnegie Mellon University Capability Maturity Model (CMM). The CMM has built-in Verification and Validation (V&V) processes that support higher software quality. Independent Verification and Validation (IV&V) of software developed by mature agencies can be therefore more effective than for software developed by less mature organizations. How is Independent V&V different with respect to the maturity of an organization? Knowing a priori the maturity of an organization's processes, how can IV&V planners better identify areas of need choose IV&V activities, etc? The objective of this research is to provide a complementary set of guidelines and criteria to assist the planning of IV&V activities on a project using a priori knowledge of the measurable levels of maturity of the organization developing the software.

Fuhman, Christopher↗

Technical Area V (TA-V) transformation project close-out report.

Sandia National Laboratories (SNL) Technical Area V (TA-V) has provided unique nuclear experimental environments for decades. The technologies tested in TA-V facilities have furthered the United States Nuclear Weapons program and has contributed to the national energy and homeland security mission. The importance of TA-V working efficiently to produce an attractive and effective platform for experiments should not be underestimated. Throughout its brief history, TA-V has evolved to address multiple and diverse sets of requirements. These requirements evolved over many years; however, the requirements had not been managed nor communicated comprehensively or effectively. A series of programmatic findings over several years of external audits was evidence of this downfall. Today, these same requirements flow down through a new TA-V management system that produces consistently applied and reproducible approaches to work practices. In 2008, the TA-V department managers assessed the state of TA-V services and work activities to understand how to improve customer interfaces, stakeholders perceptions, and workforce efficiencies. The TA-V management team initiated the TA-V Transformation Project after they deemed the pre-June 2008 operational model to be ineffective in managing work and in providing integrated, continuous improvement to TA-V processes. This report summarizes the TA-V Transformation Project goals, activities, and accomplishments.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗