SEARCH · Search NASA

Results for “probabilistic risk assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

Human reliability analysis (HRA) is a method for evaluating human errors in a variety of complex systems such as nuclear power plants, military systems, aircraft, and chemical plants. Most HRA methods currently used by regulatory institutes or utilities are called static HRA and are carried out by simple worksheets or simple calculators. To date, there are many unsolved or intrinsic challenges in static HRA. For example, existing static HRA does not realistically model and evaluate human actions as they would be performed at actual systems. There is no method with HRA to objectively estimate the time required for human actions despite being essential to HRA processes. In addition, many HRA methods still rely on a dataset generated prior to the 1980s, from unrelated industry experience or simply from expert judgment. Accordingly, this study attempted to research how to overcome the challenges of existing HRA via dynamic risk assessment (a.k.a., simulation-based or computation-based risk assessment) techniques. First, this study developed a dynamic HRA method, named as PRocedure-based Investigation Method of EMRALD Risk Assessment – HRA (PRIMERA-HRA). The PRIMERA-HRA mainly concentrates on providing HRA analysts with specific guidelines on how to reasonably model human actions, assign human reliability data and evaluate output of simulation within a dynamic probabilistic risk assessment tool, called as Event Modeling Risk Assessment using Linked Diagrams (EMRALD). Second, this study also developed a module for performance shaping factors (i.e., the key concept in HRA quantification) applicable to dynamic HRA, then implemented it based on PRIMERA-HRA within the EMRALD tool. Third, this study developed an HRA data collection framework to support dynamic HRA, called as Simplified Human Error Experimental Program (SHEEP). Originally, the SHEEP study aimed to support static HRA and its data collection, but recently extended the scope to the new technologies such as dynamic HRA or HRA for advanced reactors. SHEEP focuses on the use of data collected from simplified simulators to complement—but not replace—data collection studies using full-scope simulators and actual operators. To date, many experiments were conducted under the SHEEP framework. Multiple analyses, such as human performance analysis, human error analysis, task complexity analysis, learning effect analysis and time distribution analysis, were also carried out using the collected data. Then, based on the major insights, an approach to inferring full-scope data based on simplified simulator data was proposed. The PRIMERA-HRA and SHEEP research are expected to evaluate human actions more realistically than existing static HRA, provide an opportunity to collect more HRA data with reasonable cost and labor, then contribute to enhance the quality of HRA.

99 - GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

An Approach to Dynamic Human Reliability Analysis and Its Data Collection Framework

99 - GENERAL AND MISCELLANEOUS↗

NASA NTRS2010

The Importance of HRA in Human Space Flight: Understanding the Risks

Human performance is critical to crew safety during space missions. Humans interact with hardware and software during ground processing, normal flight, and in response to events. Human interactions with hardware and software can cause Loss of Crew and/or Vehicle (LOCV) through improper actions, or may prevent LOCV through recovery and control actions. Humans have the ability to deal with complex situations and system interactions beyond the capability of machines. Human Reliability Analysis (HRA) is a method used to qualitatively and quantitatively assess the occurrence of human failures that affect availability and reliability of complex systems. Modeling human actions with their corresponding failure probabilities in a Probabilistic Risk Assessment (PRA) provides a more complete picture of system risks and risk contributions. A high-quality HRA can provide valuable information on potential areas for improvement, including training, procedures, human interfaces design, and the need for automation. Modeling human error has always been a challenge in part because performance data is not always readily available. For spaceflight, the challenge is amplified not only because of the small number of participants and limited amount of performance data available, but also due to the lack of definition of the unique factors influencing human performance in space. These factors, called performance shaping factors in HRA terminology, are used in HRA techniques to modify basic human error probabilities in order to capture the context of an analyzed task. Many of the human error modeling techniques were developed within the context of nuclear power plants and therefore the methodologies do not address spaceflight factors such as the effects of microgravity and longer duration missions. This presentation will describe the types of human error risks which have shown up as risk drivers in the Shuttle PRA which may be applicable to commercial space flight. As with other large PRAs of complex machines, human error in the Shuttle PRA proved to be an important contributor (~12 percent) to LOCV. An existing HRA technique was adapted for use in the Shuttle PRA, but additional guidance and improvements are needed to make the HRA task in space-related PRAs easier and more accurate. Therefore, this presentation will also outline plans for expanding current HRA methodology to more explicitly cover spaceflight performance shaping factors.

Hamlin, Teri↗

NASA NTRS2012

Bayesian Analysis for Risk Assessment of Selected Medical Events in Support of the Integrated Medical Model Effort

The Exploration Medical Capability project is creating a catalog of risk assessments using the Integrated Medical Model (IMM). The IMM is a software-based system intended to assist mission planners in preparing for spaceflight missions by helping them to make informed decisions about medical preparations and supplies needed for combating and treating various medical events using Probabilistic Risk Assessment. The objective is to use statistical analyses to inform the IMM decision tool with estimated probabilities of medical events occurring during an exploration mission. Because data regarding astronaut health are limited, Bayesian statistical analysis is used. Bayesian inference combines prior knowledge, such as data from the general U.S. population, the U.S. Submarine Force, or the analog astronaut population located at the NASA Johnson Space Center, with observed data for the medical condition of interest. The posterior results reflect the best evidence for specific medical events occurring in flight. Bayes theorem provides a formal mechanism for combining available observed data with data from similar studies to support the quantification process. The IMM team performed Bayesian updates on the following medical events: angina, appendicitis, atrial fibrillation, atrial flutter, dental abscess, dental caries, dental periodontal disease, gallstone disease, herpes zoster, renal stones, seizure, and stroke.

Gilkey, Kelly M.↗

NASA NTRS2014

Quantitative Risk Modeling of Fire on the International Space Station

The International Space Station (ISS) Program has worked to prevent fire events and to mitigate their impacts should they occur. Hardware is designed to reduce sources of ignition, oxygen systems are designed to control leaking, flammable materials are prevented from flying to ISS whenever possible, the crew is trained in fire response, and fire response equipment improvements are sought out and funded. Fire prevention and mitigation are a top ISS Program priority - however, programmatic resources are limited; thus, risk trades are made to ensure an adequate level of safety is maintained onboard the ISS. In support of these risk trades, the ISS Probabilistic Risk Assessment (PRA) team has modeled the likelihood of fire occurring in the ISS pressurized cabin, a phenomenological event that has never before been probabilistically modeled in a microgravity environment. This paper will discuss the genesis of the ISS PRA fire model, its enhancement in collaboration with fire experts, and the results which have informed ISS programmatic decisions and will continue to be used throughout the life of the program.

Castillo, Theresa↗

NASA NTRS2015

Dynamic Simulation Probabalistic Risk Assessment Model for an Enceladus Sample Return Mission

Enceladus, a moon of Saturn, has geyser-like jets that spray plumes of material into orbit. These jets could enable a free-flying spacecraft to collect samples and return them to Earth for study to determine if they contain the building blocks of life. The Office of Planetary Protection at NASA requires containment of any unsterilized samples and prohibits destructive impact of the spacecraft upon return to Earth, with a sample release probability of less than 1 in 1,000,000 as a recommended goal. This paper describes a probabilistic risk assessment model that uses dynamic simulation techniques to capture the physics-based, time- and state-dependent interactions between the sample return system and the environment, which drive the risk of sample release. The dynamic approach uses a Monte Carlo-style simulation to integrate the many phases and sources of risk for a sample return mission. The model is used to assess the achievability of the planetary protection reliability goal. This is accomplished by performing sensitivity studies assessing the impact of modeling assumptions to identify where uncertainties drive the risk. These results, in turn, are used to examine the feasibility of meeting key design and performance parameters that are needed to achieve the reliability goal for a given architecture with existing technologies.

Dynamic Simulation↗

NASA NTRSDate not supplied

Assessing Crew Medical Officer Preparedness for International Space Station Missions

INTRODUCTION Astronaut crew medical officers (CMO) undergo additional, specialized training to provide medical support onboard the International Space Station (ISS). CMO training focuses on utilizing resources on-station and optimizing communication with ground-based expertise to manage both common minor conditions as well as emergency/trauma situations. As NASA moves towards future Artemis and exploration class missions, we sought CMO perspectives on past, present, and future CMO training and medical resourcing to help inform mission planners regarding medical risk mitigation considerations. We present an interview-based qualitative assessment of CMO preparedness for the International Space Station with the aim to help iterate future training flows for Low Earth Orbit, Artemis, and exploration-class missions. METHODS A semi-structured interview tool was developed to assess CMO experience and their recommendations for future mission training considerations. Participants were recruited from former or current US astronauts who were designated as CMO and subsequently flew to the International Space Station. 10 former CMOs took part in the study. Interview transcripts were deidentified and evaluated for qualitative thematic analysis by two independent coders with NVivo Software suite (Lumivero, Denver, CO), and emergent themes were compiled. RESULTS Former CMO crew expressed appropriate preparedness for current ISS medical capabilities and requirements. They identified the role of the CMO as serving as a healthcare extender to ground-based flight surgeons for ISS and other LEO missions. Individuals with prior medical experience were preferred though not felt to be necessary for LEO missions. For exploration class missions with limitations in ground communication, a physician-as-CMO and more rigorous training curricula for both the CMO and astronaut crew were desired. DISCUSSION Former NASA astronaut crew were interviewed to retrospectively assess their preparedness as ISS crew medical officers and hypothesize future medical knowledge, skill, and ability needs. Along with medical operations subject matter expert opinion and advanced probabilistic risk assessment modeling, past CMO experience can help inform future CMO training considerations and responsibilities for future Artemis and exploration class missions.

M A Labarbera↗

DOE OSTI2025

Success Path Method: Introduction to the Success Path Method Software Tool©

As part of its commitment to advancing safety and reliability assessment methodologies, Argonne National Laboratory pioneered the use of an evaluation method called the Success Path Method (SPM) to improve risk management for offshore oil and gas operations. The development of the SPM at Argonne has been driven by the need to improve existing risk assessment methodologies by focusing on the steps necessary for success rather than failure modes alone. This is particularly important for industrial environments like offshore facilities that perform multiple functions under a continuously evolving set of operational conditions – such as water depth and temperature, currents, and weather conditions. In these dynamic environments, the traditional Probabilistic Risk Assessment (PRA) approach is far too complex as it focuses on what can go wrong – which comprises an infinite failure space that must be fully explored and understood. By shifting the focus to a finite space of success paths, the SPM enables operators and decision makers to prioritize a manageable number of steps that must go right to ensure success. Building on its five decades of experience in safety assessments for the nuclear industry, Argonne made major adaptations to existing risk assessment methods utilizing features similar to fault trees that are traditionally used in PRA to map all pathways in which the system can malfunction. In contrast, SPM identifies the components and processes that must function correctly to achieve specific outcomes – such as preventing the uncontrolled release of hydrocarbons during drilling operations. The SPM framework integrates equipment, procedures, software, processes, and human actions to ensure that physical barriers meet critical safety functions in dynamic operational conditions. This approach helps identify failure modes and improve operational risk management by narrowing the focus to key success elements, which in turn reduces uncertainty and helps users understand, manage, and respond to failures.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2010

Probabilistic Design Analysis (PDA) Approach to Determine the Probability of Cross-System Failures for a Space Launch Vehicle

Quantifying the probability of significant launch vehicle failure scenarios for a given design, while still in the design process, is critical to mission success and to the safety of the astronauts. Probabilistic risk assessment (PRA) is chosen from many system safety and reliability tools to verify the loss of mission (LOM) and loss of crew (LOC) requirements set by the NASA Program Office. To support the integrated vehicle PRA, probabilistic design analysis (PDA) models are developed by using vehicle design and operation data to better quantify failure probabilities and to better understand the characteristics of a failure and its outcome. This PDA approach uses a physics-based model to describe the system behavior and response for a given failure scenario. Each driving parameter in the model is treated as a random variable with a distribution function. Monte Carlo simulation is used to perform probabilistic calculations to statistically obtain the failure probability. Sensitivity analyses are performed to show how input parameters affect the predicted failure probability, providing insight for potential design improvements to mitigate the risk. The paper discusses the application of the PDA approach in determining the probability of failure for two scenarios from the NASA Ares I project

Shih, Ann T.↗

DOE OSTI2026

Validating a Dynamic PWR Safety and Security Model?

Nuclear power plants (NPPs) are assessed for safety and security using separate models that cannot capture how an attacker's decisions and a plant's response unfold together in real time, leaving regulators and operators without a complete picture of true plant vulnerability. Traditional probabilistic risk assessment (PRA) methods treat adversarial events as fixed initiators with predetermined outcomes, and are structurally incapable of representing the time-dependent interplay between physical security events, safety system response, and operator mitigative actions. At Idaho National Laboratory (INL), I contributed to the development and validation of Modeling and Analysis for Safety and Security using the Dynamic EMRALD Framework (MASS-DEF). Where static PRA relies on event-tree logic that cannot evolve mid-scenario, MASS-DEF couples a time-dependent dynamic PRA tool EMRALD (Event Modeling Risk Assessment using Linked Diagrams) with attack simulation software, allowing attacker behavior, plant system states, and operator actions to interact across time. My work focused on validating a general Pressurized Water Reactor (PWR) model. I traced model logic against PWR plant to identified errors in logic and confirm accuracy. I then built and tested attack scenarios against a general PWR model to verify that the model produced expected outcomes across all logical pathways. I also contributed a section to a related technical paper applying the same EMRALD platform to radiation dose modeling. Results show that MASS-DEF can quantitatively demonstrate that many plants exceed their regulatory security thresholds. This demonstrated margin provides a technically defensible basis for reducing the number of guards without compromising regulatory compliance. Physical security costs represent roughly 10% of annual operating budgets, making such reductions directly meaningful to INL's mission of sustaining existing commercial NPPs. This internship strengthened my understanding of nuclear systems, probabilistic modeling, and technical writing, and has solidified my pursuit of a career at a national laboratory.

98 - NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL↗

NASA NTRS2018

Modeling the Risk of U.S. Offshore Oil & Gas Exploration-Well Drilling, Commercial Nuclear Plants, and Human Spaceflight

Nuclear power, offshore oil & gas exploration, and human spaceflight all have high consequence potential if something goes wrong. Each has had at least one major incident that pointed to a need for improved risk assessment. Careful risk analysis is very important for technologies having complexity, uncertainty, and high consequence potential. All rely on multiple barriers/controls/redundancy to minimize risk. NRC (Nuclear Regulatory Commission) has moved towards risk informed regulation with Probabilistic Risk Assessment (PRA) as a major input.

PRA↗

NASA NTRS2009

Bayesian Inference for NASA Probabilistic Risk and Reliability Analysis

This document, Bayesian Inference for NASA Probabilistic Risk and Reliability Analysis, is intended to provide guidelines for the collection and evaluation of risk and reliability-related data. It is aimed at scientists and engineers familiar with risk and reliability methods and provides a hands-on approach to the investigation and application of a variety of risk and reliability data assessment methods, tools, and techniques. This document provides both: A broad perspective on data analysis collection and evaluation issues. A narrow focus on the methods to implement a comprehensive information repository. The topics addressed herein cover the fundamentals of how data and information are to be used in risk and reliability analysis models and their potential role in decision making. Understanding these topics is essential to attaining a risk informed decision making environment that is being sought by NASA requirements and procedures such as 8000.4 (Agency Risk Management Procedural Requirements), NPR 8705.05 (Probabilistic Risk Assessment Procedures for NASA Programs and Projects), and the System Safety requirements of NPR 8715.3 (NASA General Safety Program Requirements).

Dezfuli, Homayoon↗

NASA NTRS2018

A Prognostic Launch Vehicle Probability of Failure Assessment Methodology for Conceptual Systems Predicated on Human Causal Factors

Lessons learned from past failures of launch vehicle developments and operations were used to create a new method to predict the probability of failure of conceptual systems. Existing methods such as Probabilistic Risk Assessments and Human Risk Assessments were considered but found to be too cumbersome for this type of system-wide application for yet-to-be-flown vehicles. The basis for this methodology were historic databases of past failures, where it was determined that various faulty human-interactions were the predominant root causes of failure rather than deficient component reliabilities evaluated through statistical analysis. This methodology contains an expert scoring part which can be used in either a qualitative or a quantitative mode. The method produces two products: a numerical score of the probability of failure and guidance to program management on critical areas in need of increased focus to improve the probability of success. In order to evaluate the effectiveness of this new method, data from a concluded vehicle program (USAF's Titan IV with the Centaur G-Prime upper stage) was used as a test case. The theoretical vs. actual probability of failure was found to be 4.46% vs. 6.67% respectively. Recommendations are made for future applications of this method to ongoing launch vehicle development programs.

Launch Vehicle↗

NASA NTRS2019

Prognostic Launch Vehicle Probability of Failure Assessment Methodology for Conceptual Systems Predicated on Human Causal Factors

Lessons learned from past failures of launch vehicle developments and operations were used to create a new method to predict the probability of failure of conceptual systems. Existing methods such as Probabilistic Risk Assessments and Human Risk Assessments were considered but found to be too cumbersome for this type of system-wide application for yet-to-be-flown vehicles. The basis for this methodology were historic databases of past failures, where it was determined that various faulty human-interactions were the predominant root causes of failure rather than deficient component reliabilities evaluated through statistical analysis. This methodology contains an expert scoring part which can be used in either a qualitative or a quantitative mode. The method produces two products: a numerical score of the probability of failure or guidance to program management on critical areas in need of increased focus to improve the probability of success. In order to evaluate the effectiveness of this new method, data from a concluded vehicle program (USAF's Titan IV with the Centaur G-Prime upper stage) was used as a test case. Although the theoretical vs. actual probability of failure was found to be in reasonable agreement (4.46% vs. 6.67% respectively) the underlying sub-root cause scoring had significant disparities attributable to significant organizational changes and acquisitions. Recommendations are made for future applications of this method to ongoing launch vehicle development programs.

Craig H Williams↗

NASA NTRS2016

Tailoring a Human Reliability Analysis to Your Industry Needs

Companies at risk of accidents caused by human error that result in catastrophic consequences include: airline industry mishaps, medical malpractice, medication mistakes, aerospace failures, major oil spills, transportation mishaps, power production failures and manufacturing facility incidents. Human Reliability Assessment (HRA) is used to analyze the inherent risk of human behavior or actions introducing errors into the operation of a system or process. These assessments can be used to identify where errors are most likely to arise and the potential risks involved if they do occur. Using the basic concepts of HRA, an evolving group of methodologies are used to meet various industry needs. Determining which methodology or combination of techniques will provide a quality human reliability assessment is a key element to developing effective strategies for understanding and dealing with risks caused by human errors. There are a number of concerns and difficulties in "tailoring" a Human Reliability Assessment (HRA) for different industries. Although a variety of HRA methodologies are available to analyze human error events, determining the most appropriate tools to provide the most useful results can depend on industry specific cultures and requirements. Methodology selection may be based on a variety of factors that include: 1) how people act and react in different industries, 2) expectations based on industry standards, 3) factors that influence how the human errors could occur such as tasks, tools, environment, workplace, support, training and procedure, 4) type and availability of data, 5) how the industry views risk & reliability, and 6) types of emergencies, contingencies and routine tasks. Other considerations for methodology selection should be based on what information is needed from the assessment. If the principal concern is determination of the primary risk factors contributing to the potential human error, a more detailed analysis method may be employed versus a requirement to provide a numerical value as part of a probabilistic risk assessment. Industries involved with humans operating large equipment or transport systems (ex. railroads or airlines) would have more need to address the man machine interface than medical workers administering medications. Human error occurs in every industry; in most cases the consequences are relatively benign and occasionally beneficial. In cases where the results can have disastrous consequences, the use of Human Reliability techniques to identify and classify the risk of human errors allows a company more opportunities to mitigate or eliminate these types of risks and prevent costly tragedies.

DeMott, D. L.↗

NASA NTRS2012

Space Radiation Cancer Risks and Uncertainities for Different Mission Time Periods

Space radiation consists of solar particle events (SPEs), comprised largely of medium energy protons (less than several hundred MeV); and galactic cosmic ray (GCR), which includes high energy protons and high charge and energy (HZE) nuclei. For long duration missions, space radiation presents significant health risks including cancer mortality. Probabilistic risk assessment (PRA) is essential for radiation protection of crews on long term space missions outside of the protection of the Earth s magnetic field and for optimization of mission planning and costs. For the assessment of organ dosimetric quantities and cancer risks, the particle spectra at each critical body organs must be characterized. In implementing a PRA approach, a statistical model of SPE fluence was developed, because the individual SPE occurrences themselves are random in nature while the frequency distribution of SPEs depends strongly upon the phase within the solar activity cycle. Spectral variability of SPEs was also examined, because the detailed energy spectra of protons are important especially at high energy levels for assessing the cancer risk associated with energetic particles for large events. An overall cumulative probability of a GCR environment for a specified mission period was estimated for the temporal characterization of the GCR environment represented by the deceleration potential (theta). Finally, this probabilistic approach to space radiation cancer risk was coupled with a model of the radiobiological factors and uncertainties in projecting cancer risks. Probabilities of fatal cancer risk and 95% confidence intervals will be reported for various periods of space missions.

Kim,Myung-Hee Y.↗

NASA NTRS2016

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. In order to determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules and operational requirements are developed and then finalized.

DeMott, Diana↗

NASA NTRS2017

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. To determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators, and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules, and operational requirements are developed and then finalized.

DeMott, Diana L.↗