Search NASA⌕ Search

SEARCH · Search NASA

Results for “Evaluation Model Development and Assessment Process”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
NASA NTRS2004

Simulation Based Acquisition for NASA's Office of Exploration Systems

In January 2004, President George W. Bush unveiled his vision for NASA to advance U.S. scientific, security, and economic interests through a robust space exploration program. This vision includes the goal to extend human presence across the solar system, starting with a human return to the Moon no later than 2020, in preparation for human exploration of Mars and other destinations. In response to this vision, NASA has created the Office of Exploration Systems (OExS) to develop the innovative technologies, knowledge, and infrastructures to explore and support decisions about human exploration destinations, including the development of a new Crew Exploration Vehicle (CEV). Within the OExS organization, NASA is implementing Simulation Based Acquisition (SBA), a robust Modeling & Simulation (M&S) environment integrated across all acquisition phases and programs/teams, to make the realization of the President s vision more certain. Executed properly, SBA will foster better informed, timelier, and more defensible decisions throughout the acquisition life cycle. By doing so, SBA will improve the quality of NASA systems and speed their development, at less cost and risk than would otherwise be the case. SBA is a comprehensive, Enterprise-wide endeavor that necessitates an evolved culture, a revised spiral acquisition process, and an infrastructure of advanced Information Technology (IT) capabilities. SBA encompasses all project phases (from requirements analysis and concept formulation through design, manufacture, training, and operations), professional disciplines, and activities that can benefit from employing SBA capabilities. SBA capabilities include: developing and assessing system concepts and designs; planning manufacturing, assembly, transport, and launch; training crews, maintainers, launch personnel, and controllers; planning and monitoring missions; responding to emergencies by evaluating effects and exploring solutions; and communicating across the OExS enterprise, within the Government, and with the general public. The SBA process features empowered collaborative teams (including industry partners) to integrate requirements, acquisition, training, operations, and sustainment. The SBA process also utilizes an increased reliance on and investment in M&S to reduce design risk. SBA originated as a joint Industry and Department of Defense (DoD) initiative to define and integrate an acquisition process that employs robust, collaborative use of M&S technology across acquisition phases and programs. The SBA process was successfully implemented in the Air Force s Joint Strike Fighter (JSF) Program.

Hale, Joe↗
NASA NTRS2021

Hybrid phenology matching model for robust crop phenological retrieval

Crop phenology regulates seasonal agroecosystem carbon, water, and energy exchanges, and is a key component in empirical and process-based crop models for simulating biogeochemical cycles of farmlands, assessing gross and net primary production, and forecasting the crop yield. The advances in phenology matching models provide a feasible means to monitor crop phenological progress using remote sensing observations, with a priori information of reference shapes and reference phenological transition dates. Yet the underlying geometrical scaling assumption of models, together with the challenge in defining phenological references, hinders the applicability of phenology matching in crop phenological studies. The objective of this study is to develop a novel hybrid phenology matching model to robustly retrieve a diverse spectrum of crop phenological stages using satellite time series. The devised hybrid model leverages the complementary strengths of phenometric extraction methods and phenology matching models. It relaxes the geometrical scaling assumption and can characterize key phenological stages of crop cycles, ranging from farming practice-relevant stages (e.g., planted and harvested) to crop development stages (e.g., emerged and mature). To systematically evaluate the influence of phenological references on phenology matching, four representative phenological reference scenarios under varying levels of phenological calibrations in terms of time and space are further designed with publicly accessible phenological information. The results indicate that the hybrid phenology matching model can achieve high accuracies for estimating corn and soybean phenological growth stages in Illinois, particularly with the year- and region-adjusted phenological reference (R-squared higher than 0.9 and RMSE less than 5 days for most phenological stages). The inter-annual and regional phenological patterns characterized by the hybrid model correspond well with those in the crop progress reports (CPRs) from the USDA National Agricultural Statistics Service (NASS). Compared to the benchmark phenology matching model, the hybrid model is more robust to the decreasing levels of phenological reference calibrations, and is particularly advantageous in retrieving crop early phenological stages (e.g., planted and emerged stages) when the phenological reference information is limited. This innovative hybrid phenology matching model, together with CPR-enabled phenological reference calibrations, holds 3 considerable promise in revealing spatio-temporal patterns of crop phenology over extended geographical regions.

Phenology↗
NASA NTRS2017

Integrating Growth Stage Deficit Irrigation into a Process Based Crop Model

Current rates of agricultural water use are unsustainable in many regions, creating an urgent need to identify improved irrigation strategies for water limited areas. Crop models can be used to quantify plant water requirements, predict the impact of water shortages on yield, and calculate water productivity (WP) to link water availability and crop yields for economic analyses. Many simulations of crop growth and development, especially in regional and global assessments, rely on automatic irrigation algorithms to estimate irrigation dates and amounts. However, these algorithms are not well suited for water limited regions because they have simplistic irrigation rules, such as a single soil-moisture based threshold, and assume unlimited water. To address this constraint, a new modeling framework to simulate agricultural production in water limited areas was developed. The framework consists of a new automatic irrigation algorithm for the simulation of growth stage based deficit irrigation under limited seasonal water availability; and optimization of growth stage specific parameters. The new automatic irrigation algorithm was used to simulate maize and soybean in Gainesville, Florida, and first used to evaluate the sensitivity of maize and soybean simulations to irrigation at different growth stages and then to test the hypothesis that water productivity calculated using simplistic irrigation rules underestimates WP. In the first experiment, the effect of irrigating at specific growth stages on yield and irrigation water use efficiency (IWUE) in maize and soybean was evaluated. In the reproductive stages, IWUE tended to be higher than in the vegetative stages (e.g. IWUE was 18% higher than the well watered treatment when irrigating only during R3 in soybean), and when rainfall events were less frequent. In the second experiment, water productivity (WP) was significantly greater with optimized irrigation schedules compared to non-optimized irrigation schedules in water restricted scenarios. For example, the mean WP across 38 years of maize production was 1.1 kg/cu m for non-optimized irrigation schedules with 50 mm of seasonal available water and 2.1 kg/cu m optimized ion schedules, a 91% improvement in WP with optimized irrigation schedules. The framework described in this work could be used to estimate WP for regional to global assessments, as well as derive location specific irrigation guidance.

crop model↗
NASA NTRS1996

Utilization of an agility assessment module in analysis and optimization of preliminary fighter configuration

A study has been conducted to develop and to analyze a FORTRAN computer code for performing agility analysis on fighter aircraft configurations. This program is one of the modules of the NASA Ames ACSYNT (AirCraft SYNThesis) design code. The background of the agility research in the aircraft industry and a survey of a few agility metrics are discussed. The methodology, techniques, and models developed for the code are presented. The validity of the existing code was evaluated by comparing with existing flight test data. A FORTRAN program was developed for a specific metric, PM (Pointing Margin), as part of the agility module. Example trade studies using the agility module along with ACSYNT were conducted using a McDonnell Douglas F/A-18 Hornet aircraft model. Tile sensitivity of thrust loading, wing loading, and thrust vectoring on agility criteria were investigated. The module can compare the agility potential between different configurations and has capability to optimize agility performance in the preliminary design process. This research provides a new and useful design tool for analyzing fighter performance during air combat engagements in the preliminary design.

Ngan, Angelen↗
NASA NTRS2023

Towards an Indian Land Data Assimilation System (ILDAS): A Coupled Hydrologic-Hydraulic System for Water Balance Assessments

Effective management of water resources requires reliable estimates of land surface states and fluxes, including water balance components. But most land surface models run in uncoupled mode and do not produce river discharge at catchment scales to be useful for water resources management applications. Such integrated systems are also rare over India where hydrometeorological extremes have wreaked havoc on the economy and people. So, an Indian Land Data Assimilation System (ILDAS) with a coupled land surface and a hydrodynamic model has been developed and driven by multiple meteorological forcings (0.1°, daily) to estimate land surface states, channel discharge, and floodplain inundation. ILDAS benefits from an integrated framework as well as the largest suite of observation records collected over India and has been used to produce a reanalysis product for 1981–2021 using four forcing datasets, namely, Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2), Climate Hazards Group InfraRed Precipitation with Station data (CHIRPS), ECMWF’s ERA-5, and Indian Meteorological Department (IMD) gridded precipitation. We assessed the uncertainty and bias in these precipitation datasets and validated all major components of the terrestrial water balance, i.e., surface runoff, soil moisture, terrestrial water storage anomalies, evapotranspiration, and streamflow, against a combination of satellite and in situ observation datasets. Our assessment shows that ILDAS can represent the hydrological processes reasonably well over the Indian landmass with IMD precipitation showing the best relative performance. Evaluation against ESA-CCI soil moisture shows that MERRA-2 based estimates outperform the others, whereas ERA-5 performs best in simulating evapotranspiration when evaluated against MODIS ET. Evaluations against observed records show that CHIRPS-based estimates have the highest performance in reconstructing surface runoff and streamflow. Once operational, this system will be useful for supporting transboundary water management decision making in the region.

Indian Land Data Assimilation System (ILDAS)↗
NASA NTRS1999

Ozone Climatology

The primary role of models in the assessment process is to predict changes to ozone. It is crucial therefore that the ability of the models to reproduce the actual distribution of ozone be tested. Historically, maps of the ozone column (latitude by month) have been used for this purpose. In MM I a climatology was developed for the vertical distribution of ozone for 15-60 km, based on SBUV data for 1979-80. SBUV profiles are reported with vertical resolution of approx. 5 km, but the true resolution is lower, approx. 8 km above the ozone maximum and approx. 15 km for 10-25 km. The climatology was considered valid to about 20-30% at 20 km and to 50% at 15 km. Comparisons were made with models in mixing ratio (ppm), which emphasizes the middle and upper stratosphere. A new ozone climatology was developed for the vertical distribution of ozone for MM II. Our goal was to develop a product that could be used to evaluate models in the lower stratosphere, the region where most of the ozone column resides and where most of the ozone loss is occurring, as well as the middle and upper stratosphere.

Logan, Jennifer A.↗
NASA NTRS2001

Physical and biological studies with protons and HZE particles in a NASA supported research center in radiation health

NASA has established and supports a specialized center for research and training (NSCORT) to specifically address the potential deleterious effects of HZE particles on human health. The NSCORT in radiation health is a joint effort between Lawrence Berkeley National Laboratory (LBNL) and Colorado State University (CSU). The overall scope of research encompasses a broad range of subjects from microdosimetric studies to cellular and tissue responses to initial damage produced by highly energetic protons and heavy charged particles of the type found in galactic cosmic rays (GCR) spectrum. The objectives of the microdosimetry studies are to determine the response of Tissue Equivalent Proportional Counter (TEPC) to cosmic rays using ground based accelerators. This includes evaluation of energy loss due to the escape of high-energy delta rays and increased energy deposition due to the enhanced delta ray production in the wall of the detector. In this report major results are presented for 56Fe at 1000, 740, 600 and 400 MeV/nucleon. An assessment of DNA repair and early development of related chromosomal changes is extremely important to our overall understanding of enhanced biological effectiveness of high LET particle radiation. Results are presented with respect to the fidelity of the rejoining of double strand breaks and the implications of misrejoining. The relationship between molecular and cytogenetic measurements is presented by studying damage processing in highly heterochromatic supernumerary (correction of sypernumerary) X chromosomes and the active X-chromosome. One of the important consequences of cell's inability to handle DNA damage can be evaluated through mutation studies. Part of our goal is the assessment of potential radioprotectors to reduce the mutation yield following HZE exposures, and some promising results are presented on one compound. A second goal is the integration of DNA repair and mutation studies. Results are presented on a direct comparison of initial double strand breaks induction, the time course and fidelity of double strand break rejoining, cell killing and mutation induction in the same human model system. In order to understand the carcinogenic potential of protons and HZE particles, the role of damaged microenvironment in this process must be understood. In this project it has been postulated that radiation affects the microenvironment, which then modifies cell interactions in a manner conducive to neoplastic progression. Both TGF-beta and FGF-2 are important components of microenvironment. A recent result on the assessment of the role of FGF-2 and its cross-talk with TGF-beta as a function of radiation quality is presented. Theoretical modeling has so far played a central role in analyzing and integrating experimental data on repair and mutation studies and predicting new phenomena. The integrated NSCORT program also provides a broad training experience for students and postdoctoral fellows in space radiation health.

NASA Discipline Radiation Health↗
NASA NTRS2013

Planetary Protection Bioburden Analysis Program

This program is a Microsoft Access program that performed statistical analysis of the colony counts from assays performed on the Mars Science Laboratory (MSL) spacecraft to determine the bioburden density, 3-sigma biodensity, and the total bioburdens required for the MSL prelaunch reports. It also contains numerous tools that report the data in various ways to simplify the reports required. The program performs all the calculations directly in the MS Access program. Prior to this development, the data was exported to large Excel files that had to be cut and pasted to provide the desired results. The program contains a main menu and a number of submenus. Analyses can be performed by using either all the assays, or only the accountable assays that will be used in the final analysis. There are three options on the first menu: either calculate using (1) the old MER (Mars Exploration Rover) statistics, (2) the MSL statistics for all the assays, or This software implements penetration limit equations for common micrometeoroid and orbital debris (MMOD) shield configurations, windows, and thermal protection systems. Allowable MMOD risk is formulated in terms of the probability of penetration (PNP) of the spacecraft pressure hull. For calculating the risk, spacecraft geometry models, mission profiles, debris environment models, and penetration limit equations for installed shielding configurations are required. Risk assessment software such as NASA's BUMPERII is used to calculate mission PNP; however, they are unsuitable for use in shield design and preliminary analysis studies. The software defines a single equation for the design and performance evaluation of common MMOD shielding configurations, windows, and thermal protection systems, along with a description of their validity range and guidelines for their application. Recommendations are based on preliminary reviews of fundamental assumptions, and accuracy in predicting experimental impact test results. The software is programmed in Visual Basic for Applications for installation as a simple add-in for Microsoft Excel. The user is directed to a graphical user interface (GUI) that requires user inputs and provides solutions directly in Microsoft Excel workbooks. This work was done by Shannon Ryan of the USRA Lunar and Planetary Institute for Johnson Space Center. Further information is contained in a TSP (see page 1). MSC- 24582-1 Micrometeoroid and Orbital Debris (MMOD) Shield Ballistic Limit Analysis Program Lyndon B. Johnson Space Center, Houston, Texas Commercially, because it is so generic, Enigma can be used for almost any project that requires engineering visualization, model building, or animation. Models in Enigma can be exported to many other formats for use in other applications as well. Educationally, Enigma is being used to allow university students to visualize robotic algorithms in a simulation mode before using them with actual hardware. This work was done by David Shores and Sharon P. Goza of Johnson Space Center; Cheyenne McKeegan, Rick Easley, Janet Way, and Shonn Everett of MEI Technologies; Mark Manning of PTI; and Mark Guerra, Ray Kraesig, and William Leu of Tietronix Software, Inc. For further information, contact the JSC Innovation Partnerships Office at (281) 483-3809. MSC-24211-1 Spitzer Telemetry Processing System NASA's Jet Propulsion Laboratory, Pasadena, California The Spitzer Telemetry Processing System (SirtfTlmProc) was designed to address objectives of JPL's Multi-mission Image Processing Lab (MIPL) in processing spacecraft telemetry and distributing the resulting data to the science community. To minimize costs and maximize operability, the software design focused on automated error recovery, performance, and information management. The system processes telemetry from the Spitzer spacecraft and delivers Level 0 products to the Spitzer Science Center. SirtfTlmProc is a unique system with automated error notification and recovery, with a real-time continuous service that can go quiescent after periods of inactivity. The software can process 2 GB of telemetry and deliver Level 0 science products to the end user in four hours. It provides analysis tools so the operator can manage the system and troubleshoot problems. It automates telemetry processing in order to reduce staffing costs. This work was done by Alice Stanboli, Elmain M. Martinez, and James M. McAuley of Caltech for NASA's Jet Propulsion Laboratory. For more information, contact iaoffice @jpl.nasa.gov. This software is available for commercial licensing. Please contact Dan Broderick at Daniel.F. Broderick@jpl.nasa.gov. Refer to NPO-47803. NASA Tech Briefs, September 2013 29 This rapid response computer program predicts Orbiter Wing Leading Edge (WLE) damage caused by ice or foam impact during a Space Shuttle launch (Program "IMPACT2"). The program was developed after the Columbia accident in order to assess quickly WLE damage due to ice, foam, or metal impact (if any) during a Shuttle launch. IMPACT2 simulates an impact event in a few minutes for foam impactors, and in seconds for ice and metal impactors. The damage criterion is derived from results obtained from one sophisticated commercial program, which requires hours to carry out simulations of the same impact events. The program was designed to run much faster than the commercial program with prediction of projectile threshold velocities within 10 to 15% of commercial-program values. The mathematical model involves coupling of Orbiter wing normal modes of vibration to nonlinear or linear springmass models. IMPACT2 solves nonlinear or linear impact problems using classical normal modes of vibration of a target, and nonlinear/ linear time-domain equations for the projectile. Impact loads and stresses developed in the target are computed as functions of time. This model is novel because of its speed of execution. A typical model of foam, or other projectile characterized by material nonlinearities, impacting an RCC panel is executed in minutes instead of hours needed by the commercial programs. Target damage due to impact can be assessed quickly, provided that target vibration modes and allowable stress are known. This work was done by Robert Clark, Jr., Paul Cotter, and Constantine Michalopoulos of The Boeing Company for Johnson Space Center. For further information, contact the JSC Innovation Partnerships Office at (281) 483-3809. MSC-24988-1 Wing Leading Edge RCC Rapid Response Damage Prediction Tool (IMPACT2) Lyndon B. Johnson Space Center, Houston, Texas (3) the MSL statistics for only the accountable assays. Other options on the main menu include a data editing form and utility programs that produce various reports requested by the microbiologists and the project, and tools to generate the groupings for the final analyses. The analyses can be carried out in three ways: Each assay can be treated separately, the assays can be collectively treated for the whole zone as a group, or the assays can be collected in groups designated by the JPL Planetary Protection Manager. The latter approach was used to generate the final report because assays on the same equipment or similar equipment can be assumed to have been exposed to the same environment and cleaning. Thus, the statistics are improved by having a larger population, thereby reducing the standard deviation by the square root of N. For each method mentioned above, three reports are available. The first is a detailed report including all the data. This version was very useful in verifying the calculations. The second is a brief report that is similar to the full detailed report, but does not print out the data. The third is a grand total and summary report in which each assay requires only one line. For the first and second reports, most of the calculations are performed in the report section itself. For the third, all the calculations are performed directly in the query bound to the report. All the numerical results were verified by comparing them with Excel templates, then exporting the data from the Planetary Protection Analysis program to Excel.

Beaudet, Robert A.↗
NASA NTRS2014

Assessing the Effect of Spaceflight on the Propensity for Astronauts to Develop Disk Herniation

BACKGROUND: A previous study [1] reported that the instantaneous risk of developing a Herniated Nucleus Pulposus (HNP) was higher in astronauts who had flown at least one mission, as compared with those in the corps who had not yet flown. However, the study only analyzed time to HNP after the first mission (if any) and did not account for the possible effects of multiple missions. While many HNP's occurred well into astronauts' careers or in some cases years after retirement, the higher incidence of HNPs relatively soon after completion of space missions appears to indicate that spaceflight may lead to an increased risk of HNP. The purpose of this study was to support the Human System Risk Board assessment of back pain, evaluate the risk of injury due to dynamic loads, and update the previous dataset which contained events up to December 31, 2006. METHODS: Data was queried from the electronic medical record and provided by the Lifetime Surveillance of Astronaut Health. The data included all 330 United States astronauts from 1959 through February 2014. Cases were confirmed by Magnetic Resonance Imaging, Computerized Tomography, Myelography, operative findings, or through clinical confirmation with a neurologist or neurosurgeon. In this analysis, astronauts who had an HNP at selection into the corps or had an HNP diagnosis prior to their first flight were excluded. The statistical challenges in using the available data to separate effects of spaceflight from those associated with general astronaut training and lifestyle on propensity to develop HNPs are many. The primary outcome is reported date of first HNP (if any), which at best is only an approximation to the actual time of occurrence. To properly analyze this data with a survival analysis model, one must also know the "exposure" time - i.e. how long each astronaut has been at risk for developing an HNP. If an HNP is reported soon after a mission, is it mission caused or general? If the former, exposure time should be counted from the time of landing (assuming the risk of HNP occurring during a mission is zero). If the latter, exposure time should be counted from the time of selection; however we can't directly know which one to use. In our analysis we take both of these possibilities into account with a competing risks model, wherein two distinct stochastic processes are going on: TG = time to HNP (general) and TS = time to HNP (spaceflight). Under this type of model, whichever of these occurs first is what we observe; in other words we don't observe TG or TS, only min(TG, TS). Here, we parameterized the model in terms of separate Weibull hazard functions for each process and estimated all parameters using maximum likelihood. In addition, we allowed for a "cured fraction" - i.e. the possibility that some astronauts may never develop an HNP. RESULTS: Results will include a depiction of the competing hazard functions as well as a probability curve for the relative likelihood that an HNP reported at a given time after a mission is actually mission caused. Other factors, such as dwell time in microgravity and vehicle landing environment will be explored. An overall assessment as to whether spaceflight truly exacerbates HNP risk will be made.

Feiveson, A. H.↗
NASA NTRS2021

A Chlorophyll-a Algorithm for Landsat-8 Based on Mixture Density Networks

Retrieval of aquatic biogeochemical variables, such as the near-surface concentration of chlorophyll-a (Chla) in inland and coastal waters via remote observations, has long been regarded as a challenging task. This manuscript applies Mixture Density Networks (MDN) that use the visible spectral bands available by the Operational Land Imager (OLI) aboard Landsat-8 to estimate Chla. We utilize a database of co-located in situ radiometric and Chla measurements (N = 4,354), referred to as Type A data, to train and test an MDN model (MDN(A)). This algorithm’s performance, having been proven for other satellite missions, is further evaluated against other widely used machine learning models (e.g., support vector machines), as well as other domain-specific solutions (OC3), and shown to offer significant advancements in the field. Our performance assessment using a held-out test data set suggests that a 49% (median) accuracy with near-zero bias can be achieved via the MDN(A) model, offering improvements of 20 to 100% in retrievals with respect to other models. The sensitivity of the MDN(A) model and benchmarking methods to uncertainties from atmospheric correction (AC) methods, is further quantified through a semi-global matchup dataset (N = 3,337), referred to as Type B data. To tackle the increased uncertainties, alternative MDN models (MDN(B)) are developed through various features of the Type B data (e.g., Rayleigh-corrected reflectance spectra ρ(s)). Using held-out data, along with spatial and temporal analyses, we demonstrate that these alternative models show promise in enhancing the retrieval accuracy adversely influenced by the AC process. Results lend support for the adoption of MDN(B) models for regional and potentially global processing of OLI imagery, until a more robust AC method is developed. Index Terms—Chlorophyll-a, coastal water, inland water, Landsat-8, machine learning, ocean color, aquatic remote sensing.

Brandon Smith↗
NASA NTRS2014

Force Limiting Vibration Tests Evaluated from both Ground Acoustic Tests and FEM Simulations of a Flight Like Vehicle System Assembly

Marshall Space Flight Center has conducted a series of ground acoustic tests with the dual goals of informing analytical judgment, and validating analytical methods when estimating vibroacoustic responses of launch vehicle subsystems. The process of repeatedly correlating finite element-simulated responses with test-measured responses has assisted in the development of best practices for modeling and post-processing. In recent work, force transducers were integrated to measure interface forces at the base of avionics box equipment. Other force data was indirectly measured using strain gauges. The combination of these direct and indirect force measurements has been used to support and illustrate the advantages of implementing the Force Limiting approach for equipment qualification tests. The comparison of force response from integrated system level tests to measurements at the same locations during component level vibration tests provides an excellent illustration. A second comparison of the measured response cases from the system level acoustic tests to finite element simulations has also produced some principles for assessing the suitability of Finite Element Models (FEMs) for making vibroacoustics estimates. The results indicate that when FEM models are employed to guide force limiting choices, they should include sufficient detail to represent the apparent mass of the system in the frequency range of interest.

Smith, Andrew↗
NASA NTRS1997

Application of Domain Knowledge to Software Quality Assurance

This work focused on capturing, using, and evolving a qualitative decision support structure across the life cycle of a project. The particular application of this study was towards business process reengineering and the representation of the business process in a set of Business Rules (BR). In this work, we defined a decision model which captured the qualitative decision deliberation process. It represented arguments both for and against proposed alternatives to a problem. It was felt that the subjective nature of many critical business policy decisions required a qualitative modeling approach similar to that of Lee and Mylopoulos. While previous work was limited almost exclusively to the decision capture phase, which occurs early in the project life cycle, we investigated the use of such a model during the later stages as well. One of our significant developments was the use of the decision model during the operational phase of a project. By operational phase, we mean the phase in which the system or set of policies which were earlier decided are deployed and put into practice. By making the decision model available to operational decision makers, they would have access to the arguments pro and con for a variety of actions and can thus make a more informed decision which balances the often conflicting criteria by which the value of action is measured. We also developed the concept of a 'monitored decision' in which metrics of performance were identified during the decision making process and used to evaluate the quality of that decision. It is important to monitor those decision which seem at highest risk of not meeting their stated objectives. Operational decisions are also potentially high risk decisions. Finally, we investigated the use of performance metrics for monitored decisions and audit logs of operational decisions in order to feed an evolutionary phase of the the life cycle. During evolution, decisions are revisisted, assumptions verified or refuted, and possible reassessments resulting in new policy are made. In this regard we implemented a machine learning algorithm which automatically defined business rules based on expert assessment of the quality of operational decisions as recorded during deployment.

Wild, Christian W.↗
NASA NTRS2018

A Prognostic Launch Vehicle Probability of Failure Assessment Methodology for Conceptual Systems Predicated on Human Causal Factors

Create an improved method to calculate reliability of a conceptual launch vehicle system prior to fabrication by using historic data of actual root causes of failures. While failures have unique "proximate causes", there are typically a finite amount of common "root causes". Heretofore launch vehicle reliability evaluation typically hardware-centric statistical analyses, while most root causes of failures are been shown to be human-centric. A method based on human-centric root causes can be used to quantify reliability assessments and focus proposed actions to mitigate problems. Existing methods have been optimistic in their projections of launch vehicle reliability compared to actuals. Hypothesis: reliability of a conceptual launch vehicle can be more accurately evaluated based on a rational, probabilistic approach using past failure assessment teams' findings predicated on human-centric causes."Human Reliability Analysis Methods Selection Guidance for NASA"Chandler F.T., et al., NASA HQ/OSMA study group, July 2006. Outside HRA experts from academia, other federal labs, and the private sector. 50 system reliability methods considered, fourteen selected for further study, four finally selected as best suited for human spaceflight. Probabilistic Risk Analysis (PRA) + Human Reliability Analysis (HRA) enabled incorporating effects and probabilities of human errors. While four down-selected methods deemed appropriate for failure assessment, it did not appear that these methods could be concisely applied to perform major system-wide assessment of probability of failure of a conceptual design without becoming unwieldy."Engineering a Safer World", Detailed, comprehensive study external to NASA Leveson N. G., MIT, 2011.Systems-Theoretic Accident Model and Processes (STAMP). All-encompassing accident model based on systems theory analyzed accidents after they occurred and created approaches to prevent occurrence in developing systems not focused on failure prevention per se, but rather reducing hazards by influencing human behavior through use of constraints, hierarchical control structures, and process models to improve system safetySystem Theoretic Process Analysis (STPA) addresses predictive part of problem (a "hazard analysis"). Includes all causal factors identified in STAMP: "...design errors, software flaws, component interaction accidents, cognitively complex human decision-making errors, and social organizational and management factors contributing to accidents" can guide design process rather than require it to exist before-hand did not appear capable of concise application for system-wide assessment of probability of failure of a conceptual design without becoming unwieldy.

Williams, Craig H.↗
NASA NTRS2022

Directed Acyclic Graph Guidance Documentation

For over a decade, the National Aeronautics and Space Administration (NASA) has tracked and configuration-managed approximately 30 risks to astronaut health and performance that occur before, during and after spaceflight. The Human System Risk Board (HSRB), a Health and Medical Technical Authority (HMTA) Board at NASA Johnson Space Center, is the entity responsible for identifying, assessing, analyzing, and monitoring the official understanding of the risk or risk posture for each of the Human System Risks and determining – based on evaluation of the available evidence – when that risk posture changes. The ultimate purpose of tracking and researching these risks is to find ways to reduce the risk that astronaut crews face during spaceflight. Historically, research, development and operations relevant to one risk have been conducted in isolation from other risks; these individual risk ‘silos’ enabled initial characterization of each specific risk. In spaceflight however, the impact of exposure to risk for astronaut crews is cumulative, and not independent of exposures or other risks, as all the adverse effects of the spaceflight environment begin at launch, continue throughout the duration of the mission and in some cases across the lifetime of the crews. In January of 2020, the HSRB at NASA embarked on a pilot project designed to assess the potential value of causal diagramming as a tool to facilitate understanding these cumulative and interdependent effects as applied within Human System Risk management. This process uses directed acyclic graphs as a means of formalizing a shared mental model of the causal flow of risk among Risk Board stakeholders. Initially this model was to improve communication among those stakeholders, but the potential value exceeds communication alone. Formalization of the process for creating these causal diagrams will enable the creation of a composite risk network that is vetted by members of the NASA community and configuration managed. The causal diagrams are formulated as directed acyclic graphs (DAGs) to function as a type of knowledge graph for reference for the board and its stakeholders. This document outlines the pilot process, the standardized approaches, and guidance for risk custodian teams when creating and updating DAGs as a part of the NASA Human System Risk Management process.

Risk↗
NASA NTRSDate not supplied

Artificial Intelligence (AI) Methods for Augmenting the IMPACT Tool Evidence Library

Development of the Evidence Library for use with the IMPACT probability risk assessment tool took several years and involved a staggering amount of effort from a multi-disciplinary team. A very significant amount of the labor effort to collect, assess and finalize the Clinical Finding Form (CliFF) for each of the 119 medical conditions was provided by physician subject matter experts from the Exploration Medical Capability (ExMC) Element Clinical and Science Team. Many AI tools such as ChatGPT are excellent at summarizing large amounts of information and the current project was initiated to determine how such tools might streamline laborious processes, e.g., review and summarization of many scientific research publications, to execute key steps more efficiently in the process of developing CliFFs. The process for collecting the evidence which is found in the CliFFs is well documented in the Evidence Library Methods document (ELM; HRP-48036*). Using ELM and the CliFF development instructions as a guideline, a team of developers is leveraging Microsoft Azure AI tools and services along with open-source frameworks, to construct an AI-assisted automated pipeline. This pipeline is designed to search, retrieve, and process the necessary data sources, and ultimately help generate the final version of a CliFF. Currently, the large language model evaluates the relevance of each source material to spaceflights, either as direct evidence or as an analog. Additionally, the model assists in extracting keywords and generating brief summaries to enhance augmented retrieval and search processes in later stages of CliFF development. Once the data is ready, the model can perform semantic search and retrieval, generating and extracting valuable information for the CliFF. For instance, it can handle epidemiological statistical data, such as incidence rates and the likelihood of best or worst-case scenarios. The steps that required reading and summarizing articles were viewed as providing the greatest return on investment since large language models are very efficient and accurate in summarizing large amounts of text. Since labor effort to complete the original CliFF was not recorded with sufficient granularity, comparisons with an AI tool-generated CliFF will provide merely an approximation of time saved. Upon completion of the process, the CliFF for the medical condition “appendicitis” generated with the support of AI-based methods will serve as a proof-of-concept and will be compared to the original appendicitis CliFF to determine if use of the tools resulted in content and conclusory similarity. Based upon the results from face validation of the two CliFFs, modifications to the process will be made if necessary and additional condition CliFFs will be evaluated. Ultimately, CliFFs for the entire set of medical conditions will be created with the assistance of AI tools. Depending on the cost savings realized, CliFFs for additional medical conditions can be created to expand the Evidence Library. Future direction includes specifying the characteristics of the reviewer (prompting the AI tools to generate output assuming the reviewer is a sub-specialist physician, or nurse or EMT/medic) to determine if the effects on AI-generated output are different based on knowledge, skills and abilities. *Exploration Medical Capability Evidence Library Methods, HRP-48036 Rev A, July 2022.

Ali Al↗
NASA NTRS2018

Sensor Data Quality and Angular Rate Down-Selection Algorithms on SLS EM-1

The NASA Space Launch System Block 1 launch vehicle is equipped with an Inertial Navigation System (INS) and multiple Rate Gyro Assemblies (RGA) that are used in the Guidance, Navigation, and Control (GN&C) algorithms. The INS provides the inertial position, velocity, and attitude of the vehicle along with both angular rate and specific force measurements. Additionally, multiple sets of co-located rate gyros supply angular rate data. The collection of angular rate data, taken along the launch vehicle, is used to separate out vehicle motion from flexible body dynamics. Since the system architecture uses redundant sensors, the capability was developed to evaluate the health (or validity) of the independent measurements. A suite of Sensor Data Quality (SDQ) algorithms is responsible for assessing the angular rate data from the redundant sensors. When failures are detected, SDQ will take the appropriate action and disqualify or remove faulted sensors from forward processing. Additionally, the SDQ algorithms contain logic for down-selecting the angular rate data used by the GNC software from the set of healthy measurements. This paper explores the trades and analyses that were performed in selecting a set of robust fault-detection algorithms included in the GN&C flight software. These trades included both an assessment of hardware-provided health and status data as well as an evaluation of different algorithms based on time-to-detection, type of failures detected, and probability of detecting false positives. We then provide an overview of the algorithms used for both fault-detection and measurement down selection. We next discuss the role of trajectory design, flexible-body models, and vehicle response to off-nominal conditions in setting the detection thresholds. Lastly, we present lessons learned from software integration and hardware-in-the-loop testing.

Park, Thomas↗
NASA NTRS2022

Directed Acyclic Graphs: A Tool for Understanding the NASA Human Spaceflight System Risks - Human System Risk Board

For over a decade, the National Aeronautics and Space Administration (NASA) has tracked and configuration-managed approximately 30 risks to astronaut health and performance that occur before, during and after spaceflight. The Human System Risk Board (HSRB), a Health and Medical Technical Authority (HMTA) Board at NASA Johnson Space Center, is the entity responsible for identifying, assessing, analyzing, and monitoring the official understanding of the risk or risk posture for each of the Human System Risks and determining – based on evaluation of the available evidence – when that risk posture changes. The ultimate purpose of tracking and researching these risks is to find ways to reduce the risk that astronaut crews face during spaceflight. Historically, research, development and operations relevant to one risk have been conducted in isolation from other risks; these individual risk ‘silos’ enabled initial characterization of each specific risk. In spaceflight however, the impact of exposure to risk for astronaut crews is cumulative, and not independent of exposures or other risks, as all the adverse effects of the spaceflight environment begin at launch, continue throughout the duration of the mission and in some cases across the lifetime of the crews. In January of 2020, the HSRB at NASA embarked on a pilot project designed to assess the potential value of causal diagramming as a tool to facilitate understanding of these cumulative and interdependent effects as applied within Human System Risk management. This process uses directed acyclic graphs as a means of formalizing a shared mental model of the causal flow of risk among Risk Board stakeholders. Initially this model was to improve communication among those stakeholders, but the potential value exceeds communication alone. The causal diagrams are formulated as directed acyclic graphs (DAGs) to function as a type of knowledge graph for reference for the board and its stakeholders. This document is a sister document to NASA/TM 20220006812 Directed Acyclic Graph Guidance Documentation (1). In that document, the basic guidance for creating and standardizing directed acyclic graphs as tools for cross-risk analysis is provided. This document contains the initial configuration managed DAGs that were created as a result of applying those principles. These initial versions were accepted by the HSRB in January of 2022. Each of the Human System Risks are represented by a DAG that has been reviewed by the larger Human Health and Performance community at NASA including life scientists, physical scientists, physicians, nurses, pharmacists, exercise specialists and more. These results show the starting point for Human System Risk DAGs as shared mental models and communication aids across the boundaries of the various expertise needed to understand and mitigate the human risks in spaceflight. Because they are a starting point, each of these DAGs can be expected to change over time as new or refined evidence becomes available. The process for updating these DAGs can be found in the JSC-66705 Human System Risk Management Plan (2) that is publicly available on the NASA Technical Reports Server.

Erik L. Antonsen↗
NASA NTRS2000

Salient Characteristics for Terrestrial Analog Engineering Test Sites

The use of terrestrial analog sites is an essential component of the development process for systems that are planned for lunar and planetary surfaces. These sites can also be valuable for training personnel who are expected to operate those systems. The fact that the site is an integral element, capable of influencing data and impacting test results, must not be overlooked. For tests performed in analog environments to be truly valid it is essential that the relevant characteristics of the employed sites be thoroughly understood. It is also critical that a comparative evaluation be made to assess the similarity of the analog sites to target planetary sites. Examples of relevant characteristics include terrain, soil properties, meteorology, geologic features, biological history, and remoteness. The importance of each of the characteristics varies with the type of extraterrestrial activity to be conducted. As a result, a site that is ideal for one purpose may be totally inadequate for another. It may also be appropriate to utilize multiple sites of increasing fidelity as the development process evolves. For example, early in the development process, the use of lower-fidelity sites may be elected - especially if they are available for a lower cost. Later, it may be necessary to employ higher-fidelity sites to capture greater realism, even if there is an associated increased cost. Finally, when interpreting the results of field tests, it is necessary to understand the sensitivity of the results to the relevant conditions. Knowledge of characteristics and sensitivity of results is particularly important when using field test results to validate analytical model predictions. This is especially true for activities that involve physical interaction with the site. Examples would include tests of rovers or other vehicles and associated trafficability modeling, spacesuit mobility tests, and tests associated with system deployment operations.

Watson, J.K.↗