Search NASA⌕ Search

SEARCH · Search NASA

Results for “sequential decision making”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Final Report, DOE Early Career Award: Predictive modeling of complex physical systems: new tools for statistical inference, uncertainty quantification, and experimental design

Predictive simulation of complex physical systems increasingly rests on the interplay of experimental observations with computational models. Key inputs, parameters, or structural aspects of models may be incomplete or unknown, and must be developed from indirect and limited observations. At the same time, quantified uncertainties are needed to qualify computational predictions in the support of design and decision-making. In this context, Bayesian statistics provides a foundation for inference from noisy and limited data, but at prohibitive computional expense. This project intends to make rigorous predictive modeling *feasible* in complex physical systems, via accelerated and scalable tools for uncertainty quantification, Bayesian inference, and experimental design. Specific objectives are as follows: 1. Develop adaptive posterior approximations and dimensionality reduction approaches for Bayesian inference in high-dimensional nonlinear systems. 2. Extend accelerated Bayesian methodologies to large-scale {\em sequential} data assimilation, fully treating nonlinear models and non-Gaussian state and parameter distributions. 3. Devise efficient surrogate-based methods for Bayesian model selection and the learning of model structure. 4. Develop scalable simulation/optimization approaches to nonlinear Bayesian experimental design, for both parameter inference and model selection. 5. Demonstrate these inferential tools on chemical kinetic models in reacting flow, constructing and refining thermochemical and electrochemical models from limited data. Demonstrate Bayesian filtering on canonical stochastic PDEs and in the dynamic estimation of inhomogeneous subsurface properties and flow fields.

97 MATHEMATICS AND COMPUTING↗

Revisiting Statistical Aspects of Nuclear Material Accounting

Nuclear material accounting (NMA) is the only safeguards system whose benefits are routinely quantified. Process monitoring (PM) is another safeguards system that is increasingly used, and one challenge is how to quantify its benefit. This paper considers PM in the role of enabling frequent NMA, which is referred to as near-real-time accounting (NRTA). We quantify NRTA benefits using period-driven and data-driven testing. Period-driven testing makes a decision to alarm or not at fixed periods. Data-driven testing decides as the data arrives whether to alarm or continue testing. The difference between period-driven and datad-riven viewpoints is illustrated by using one-year and two-year periods. For both one-year and two-year periods, period-driven NMA using once-per-year cumulative material unaccounted for (CUMUF) testing is compared to more frequent Shewhart and joint sequential cusum testing using either MUF or standardized, independently transformed MUF (SITMUF) data. We show that the data-driven viewpoint is appropriate for NRTA and that it can be used to compare safeguards effectiveness. In addition to providing period-driven and data-driven viewpoints, new features include assessing the impact of uncertainty in the estimated covariance matrix of the MUF sequence and the impact of both random and systematic measurement errors.

Burr, T.↗

Use of Inverse Reinforcement Learning for Identity Prediction

We adopt Markov Decision Processes (MDP) to model sequential decision problems, which have the characteristic that the current decision made by a human decision maker has an uncertain impact on future opportunity. We hypothesize that the individuality of decision makers can be modeled as differences in the reward function under a common MDP model. A machine learning technique, Inverse Reinforcement Learning (IRL), was used to learn an individual's reward function based on limited observation of his or her decision choices. This work serves as an initial investigation for using IRL to analyze decision making, conducted through a human experiment in a cyber shopping environment. Specifically, the ability to determine the demographic identity of users is conducted through prediction analysis and supervised learning. The results show that IRL can be used to correctly identify participants, at a rate of 68% for gender and 66% for one of three college major categories.

Hayes, Roy↗

Practical and Optimal Sequential Bayesian Experimental Design for Complex Systems Incorporating Human Experimenter Preferences (Final Scientific/Technical Report)

Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.

97 MATHEMATICS AND COMPUTING↗

Statistical Engineering in Air Traffic Management Research

NASA is working to develop an integrated set of advanced technologies to enable efficient arrival operations in high-density terminal airspace for the Next Generation Air Transportation System. This integrated arrival solution is being validated and verified in laboratories and transitioned to a field prototype for an operational demonstration at a major U.S. airport. Within NASA, this is a collaborative effort between Ames and Langley Research Centers involving a multi-year iterative experimentation process. Designing and analyzing a series of sequential batch computer simulations and human-in-the-loop experiments across multiple facilities and simulation environments involves a number of statistical challenges. Experiments conducted in separate laboratories typically have different limitations and constraints, and can take different approaches with respect to the fundamental principles of statistical design of experiments. This often makes it difficult to compare results from multiple experiments and incorporate findings into the next experiment in the series. A statistical engineering approach is being employed within this project to support risk-informed decision making and maximize the knowledge gained within the available resources. This presentation describes a statistical engineering case study from NASA, highlights statistical challenges, and discusses areas where existing statistical methodology is adapted and extended.

Wilson, Sara R.↗

Failure detection system design methodology

The design of a failure detection and identification system consists of designing a robust residual generation process and a high performance decision making process. The design of these two processes are examined separately. Residual generation is based on analytical redundancy. Redundancy relations that are insensitive to modelling errors and noise effects are important for designing robust residual generation processes. The characterization of the concept of analytical redundancy in terms of a generalized parity space provides a framework in which a systematic approach to the determination of robust redundancy relations are developed. The Bayesian approach is adopted for the design of high performance decision processes. The FDI decision problem is formulated as a Bayes sequential decision problem. Since the optimal decision rule is incomputable, a methodology for designing suboptimal rules is proposed. A numerical algorithm is developed to facilitate the design and performance evaluation of suboptimal rules.

Chow, E. Y.↗

Transformation of zinc-concentrate in surface and subsurface environments: Implications for assessing zinc mobility/toxicity and choosing an optimal remediation strategy

Zinc contamination in near- and sub-surface environments is a serious threat to many ecosystems and to public health. Sufficient understanding of Zn speciation and transport mechanisms is therefore critical to evaluating its risk to the environment and to developing remediation strategies. The geochemical and mineralogical characteristics of contaminated soils in the vicinity of a Zn ore transportation route were thoroughly investigated using a variety of analytical techniques (sequential extraction, XRF, XRD, SEM, and XAFS). Imported Zn-concentrate (ZnS) was deposited in a receiving facility and dispersed over time to the surrounding roadside areas and rice-paddy soils. Subsequent physical and chemical weathering resulted in dispersal into the subsurface. The species identified in the contaminated areas included Zn-sulfide, Zn-carbonate, other O-coordinated Zn-minerals, and Zn species bound to Fe/Mn oxides or clays, as confirmed by XAFS spectroscopy and sequential extraction. The observed transformation from S-coordinated Zn to O-coordinated Zn associated with minerals suggests that this contaminant can change into more soluble and labile forms as a result of weathering. For the purpose of developing a soil washing remediation process, the contaminated samples were extracted with dilute acids. The extraction efficiency increased with the increase of O-coordinated Zn relative to S-coordinated Zn in the sediment. Furthermore, this study demonstrates that improved understanding of Zn speciation in contaminated soils is essential for well-informed decision making regarding metal mobility and toxicity, as well as for choosing an appropriate remediation strategy using soil washing.

54 ENVIRONMENTAL SCIENCES↗

Decision Processes in Discrimination: Fundamental Misrepresentations of Signal Detection Theory

In the first part of this article, I describe a new approach to studying decision making in discrimination tasks that does not depend on the technical assumptions of signal detection theory (e.g., normality of the encoding distributions). Applying these new distribution-free tests to data from three experiments, I show that base rate and payoff manipulations had substantial effects on the participants' encoding distributions but no effect on their decision rules, which were uniformly unbiased in equal and unequal base rate conditions and in symmetric and asymmetric payoff conditions. In the second part of the article, I show that this seemingly paradoxical result is readily explained by the sequential sampling models of discrimination. I then propose a new, "model-free" test for response bias that seems to more properly identify both the nature and direction of the biases induced by the classical bias manipulations.

Balakrishnan, J. D.↗

Using a Simple Binomial Model to Assess Improvement in Predictive Capability: Sequential Bayesian Inference, Hypothesis Testing, and Power Analysis

We present a Bayesian statistical methodology for identifying improvement in predictive simulations, including an analysis of the number of (presumably expensive) simulations that will need to be made in order to establish with a given level of confidence that an improvement has been observed. Our analysis assumes the ability to predict (or postdict) the same experiments with legacy and new simulation codes and uses a simple binomial model for the probability, {theta}, that, in an experiment chosen at random, the new code will provide a better prediction than the old. This model makes it possible to do statistical analysis with an absolute minimum of assumptions about the statistics of the quantities involved, at the price of discarding some potentially important information in the data. In particular, the analysis depends only on whether or not the new code predicts better than the old in any given experiment, and not on the magnitude of the improvement. We show how the posterior distribution for {theta} may be used, in a kind of Bayesian hypothesis testing, both to decide if an improvement has been observed and to quantify our confidence in that decision. We quantify the predictive probability that should be assigned, prior to taking any data, to the possibility of achieving a given level of confidence, as a function of sample size. We show how this predictive probability depends on the true value of {theta} and, in particular, how there will always be a region around {theta} = 1/2 where it is highly improbable that we will be able to identify an improvement in predictive capability, although the width of this region will shrink to zero as the sample size goes to infinity. We show how the posterior standard deviation may be used, as a kind of 'plan B metric' in the case that the analysis shows that {theta} is close to 1/2 and argue that such a plan B should generally be part of hypothesis testing. All the analysis presented in the paper is done with a general beta-function prior for {theta}, enabling sequential analysis in which a small number of new simulations may be done and the resulting posterior for {theta} used as a prior to inform the next stage of power analysis.

42 ENGINEERING↗

Prognostic analysis of high-flow nasal cannula therapy and non-invasive ventilation in mild to moderate hypoxemia patients and construction of a machine learning model for 48-h intubation prediction—a retrospective analysis of the MIMIC database

Background This study aims to investigate the clinical outcome between high-flow nasal cannula (HFNC) and non-invasive ventilation (NIV) therapy in mild to moderate hypoxemic patients on the first ICU day and to develop a predictive model of 48-h intubation. Methods The study included adult patients from the MIMIC III and IV databases who first initiated HFNC or NIV therapy due to mild to moderate hypoxemia (100 < PaO2/FiO2 ≤ 300). The 48-h and 30-day intubation rates were compared using cross-sectional and survival analysis. Nine machine learning and six ensemble algorithms were deployed to construct the 48-h intubation predictive models, of which the optimal model was determined by its prediction accuracy. The top 10 risk and protective factors were identified using the Shapley interpretation algorithm. Result A total of 123,042 patients were screened, of which, 673 were from the MIMIC IV database for ventilation therapy comparison (HFNC n = 363, NIV n = 310) and 48-h intubation predictive model construction (training dataset n = 471, internal validation set n = 202) and 408 were from the MIMIC III database for external validation. The NIV group had a lower intubation rate (23.1% vs. 16.1%, p = 0.001), ICU 28-day mortality (18.5% vs. 11.6%, p = 0.014), and in-hospital mortality (19.6% vs. 11.9%, p = 0.007) compared to the HFNC group. Survival analysis showed that the total and 48-h intubation rates were not significantly different. The ensemble AdaBoost decision tree model (internal and external validation set AUROC 0.878, 0.726) had the best predictive accuracy performance. The model Shapley algorithm showed Sequential Organ Failure Assessment (SOFA), acute physiology scores (APSIII), the minimum and maximum lactate value as risk factors for early failure and age, the maximum PaCO 2 and PH value, Glasgow Coma Scale (GCS), the minimum PaO 2 /FiO 2 ratio, and PaO 2 value as protective factors. Conclusion NIV was associated with lower intubation rate and ICU 28-day and in-hospital mortality. Further survival analysis reinforced that the effect of NIV on the intubation rate might partly be attributed to the other impact factors. The ensemble AdaBoost decision tree model may assist clinicians in making clinical decisions, and early organ function support to improve patients’ SOFA, APSIII, GCS, PaCO 2 , PaO 2 , PH, PaO 2 /FiO 2 ratio, and lactate values can reduce the early failure rate and improve patient prognosis.

Fu, Wei↗

Speed in Information Processing with a Computer Driven Visual Display in a Real-time Digital Simulation

Information transfer between the operator and computer-generated display systems is an area where the human factors engineer discovers little useful design data relating human performance to system effectiveness. This study utilized a computer-driven, cathode-ray-tube graphic display to quantify human response speed in a sequential information processing task. The performance criteria was response time to sixteen cell elements of a square matrix display. A stimulus signal instruction specified selected cell locations by both row and column identification. An equal probable number code, from one to four, was assigned at random to the sixteen cells of the matrix and correspondingly required one of four, matched keyed-response alternatives. The display format corresponded to a sequence of diagnostic system maintenance events, that enable the operator to verify prime system status, engage backup redundancy for failed subsystem components, and exercise alternate decision-making judgements. The experimental task bypassed the skilled decision-making element and computer processing time, in order to determine a lower bound on the basic response speed for given stimulus/response hardware arrangement.

Kyle, R. G.↗

EHR-BERT: A BERT-based model for effective anomaly detection in electronic health records

Objective: Physicians and clinicians rely on data contained in electronic health records (EHRs), as recorded by health information technology (HIT), to make informed decisions about their patients. The reliability of HIT systems in this regard is critical to patient safety. Consequently, better tools are needed to monitor the performance of HIT systems for potential hazards that could compromise the collected EHRs, which in turn could affect patient safety. In this paper, we propose a new framework for detecting anomalies in EHRs using sequence of clinical events. This new framework, EHR-Bidirectional Encoder Representations from Transformers (BERT), is motivated by the gaps in the existing deep-learning related methods, including high false negatives, sub-optimal accuracy, higher computational cost, and the risk of information loss. EHR-BERT is an innovative framework rooted in the BERT architecture, meticulously tailored to navigate the hurdles in the contemporary BERT method; thus, enhancing anomaly detection in EHRs for healthcare applications.Methods: The EHR-BERT framework was designed using the Sequential Masked Token Prediction (SMTP) method. This approach treats EHRs as natural language sentences and iteratively masks input tokens during both training and prediction stages. This method facilitates the learning of EHR sequence patterns in both directions for each event and identifies anomalies based on deviations from the normal execution models trained on EHR sequences.Results: Extensive experiments on large EHR datasets across various medical domains demonstrate that EHR-BERT markedly improves upon existing models. It significantly reduces the number of false positives and enhances the detection rate, thus bolstering the reliability of anomaly detection in electronic health records. This improvement is attributed to the model’s ability to minimize information loss and maximize data utilization effectively.Conclusion: EHR-BERT showcases immense potential in decreasing medical errors related to anomalous clinical events, positioning itself as an indispensable asset for enhancing patient safety and the overall standard of healthcare services. The framework effectively overcomes the drawbacks of earlier models, making it a promising solution for healthcare professionals to ensure the reliability and quality of health data.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Modeling the Resilience of Nuclear Power Plant for Unexpected Reactor Trip

Resilience can be defined as the intrinsic ability of a system to adjust to its functioning prior to, during, or following changes and disturbances, so that it can sustain required operations under both expected and unexpected conditions. Resilience engineering is a relatively new paradigm for safety management that focuses on how to cope with complexity under pressure or disturbance to achieve success. A resilient system combines inherent human abilities with engineered safety features and organizational features in a controlled and flexible manner both in expected and in unexpected situations. The concept of resilience has been suggested to complement conventional safety analyses. Conventional safety analysis methods such as PSA have several limitations: 1) they primarily focus on technical dimension, 2) the analysis are linear and sequential, 3) they are dominated by static models, 4) they do not take a systemic view into account, and 5) they focus primarily on why accidents happen and not how success is achieved. The concept of resilience provides a more integrated view for the safety analysis in safety-critical systems, e.g., nuclear power plants. The concept of resilience in the organization of NPPs has been highlighted since the Fukushima Daiichi Nuclear Power Plant accident. An IAEA report addresses that a resilient organization is one that quickly realizes deviation from normal operations and has the ability to make even the toughest and least popular decisions and to manage the margins in which it can maneuver. While traditional strategy to safety in NPPs is to identify what could go or has gone wrong, the concept of resilience is focused on what the organization does well and what it does to maintain its successful operation and to capitalize on those processes in the event of an unexpected situation. This study attempts to model the resilience of NPP dynamically, based on the operating experiences in Korean NPPs. First, this study revised the characterization of NPP. Then, a survey on the literature was conducted to identify the influences between the elements in the model of resilience in NPPs. These works provide a qualitative model for the resilience in NPPs. Then, this study reviewed 168 licensee events reports submitted to the Korean regulatory body since 2004 and quantified the relation between the attributes in the model by performing a Pearson correlation analysis. Finally, a few simulations using the System Dynamics were conducted to discuss which factors are most influencing the resilience. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Gaming the beamlines—employing reinforcement learning to maximize scientific outcomes at large-scale user facilities

Abstract Beamline experiments at central facilities are increasingly demanding of remote, high-throughput, and adaptive operation conditions. To accommodate such needs, new approaches must be developed that enable on-the-fly decision making for data intensive challenges. Reinforcement learning (RL) is a domain of AI that holds the potential to enable autonomous operations in a feedback loop between beamline experiments and trained agents. Here, we outline the advanced data acquisition and control software of the Bluesky suite, and demonstrate its functionality with a canonical RL problem: cartpole. We then extend these methods to efficient use of beamline resources by using RL to develop an optimal measurement strategy for samples with different scattering characteristics. The RL agents converge on the empirically optimal policy when under-constrained with time. When resource limited, the agents outperform a naive or sequential measurement strategy, often by a factor of 100%. We interface these methods directly with the data storage and provenance technologies at the National Synchrotron Light Source II, thus demonstrating the potential for RL to increase the scientific output of beamlines, and layout the framework for how to achieve this impact.

36 MATERIALS SCIENCE↗

A Performance Model for Allocating the Parallelism in a Multigrid-in-Time Solver

The traditional way to numerically solve time-dependent problems is to sequentially march through time, solving for one time step and then the next. The parallelism in this approach is limited to the spatial dimension, which is quickly exhausted, causing the gain from using more processors to solve a problem to diminish. One approach to overcome this barrier is to use methods that are parallel in time. These methods have the potential to achieve dramatically better performance compared to time-stepping approaches, but achieving this performance requires carefully choosing the amount of parallelism devoted to space versus the amount devoted to time. Here, we present a performance model that, for a multigrid-in-time solver, makes the decision on when to switch to parallel-in-time and on how much parallelism to devote to space vs. time. In our experiments, the model selects the best parallel configuration in most of our test cases and a configuration close to the best one in all other cases.

97 MATHEMATICS AND COMPUTING↗

A Subjective Assessment of Alternative Mission Architecture Operations Concepts for the Human Exploration of Mars at NASA Using a Three-Dimensional Multi-Criteria Decision Making Model

The primary driver for developing missions to send humans to other planets is to generate significant scientific return. NASA plans human planetary explorations with an acceptable level of risk consistent with other manned operations. Space exploration risks can not be completely eliminated. Therefore, an acceptable level of cost, technical, safety, schedule, and political risks and benefits must be established for exploratory missions. This study uses a three-dimensional multi-criteria decision making model to identify the risks and benefits associated with three alternative mission architecture operations concepts for the human exploration of Mars identified by the Mission Operations Directorate at Johnson Space Center. The three alternatives considered in this study include split, combo lander, and dual scenarios. The model considers the seven phases of the mission including: 1) Earth Vicinity/Departure; 2) Mars Transfer; 3) Mars Arrival; 4) Planetary Surface; 5) Mars Vicinity/Departure; 6) Earth Transfer; and 7) Earth Arrival. Analytic Hierarchy Process (AHP) and subjective probability estimation are used to captures the experts belief concerning the risks and benefits of the three alternative scenarios through a series of sequential, rational, and analytical processes.

Tavana, Madjid↗

Using Flory–Huggins-informed human-in-the-loop Bayesian optimization to map the phase diagram of polymer blends

Mapping the phase diagram of polymer blends is an essential step in controlling the structure–property relationship of polymer-based materials. However, traditional grid-based approaches are inefficient and rely on subjective judgements for terminating the experimental campaign. Artificial intelligence-guided experimentation offers a compelling alternative, especially when data-driven decision-making is interfaced with established polymer thermodynamics to improve efficiency and interpretability. Here, we introduce a physics-informed Bayesian optimization approach to guide the mapping of the phase diagram of a model blend containing poly(methyl methacrylate) and poly(styrene-ran-acrylonitrile). Physical information is derived from a Flory–Huggins representation of the spinodal curve, which is integrated into the Bayesian optimization process as a structured prior mean that acts as a soft constraint. Implemented as a human-in-the-loop workflow, the approach leverages optical imaging of film cloudiness with iterative Gaussian process surrogate modeling and a parameter selection decision policy to identify the composition-temperature conditions for sequential iterations. Convergence of kernel and Flory–Huggins-based hyperparameters provided a stopping criterion, ensuring an objective and interpretable termination of the experimental campaign. The framework recovered the known lower critical solution temperature (∼160 °C), while increasing material efficiency through targeted sampling. This work establishes a proof-of-concept for the application of Bayesian optimization workflows to study polymer blend miscibility.

36 MATERIALS SCIENCE↗

Building a Real-Time Flood Prediction Model for Improving Early Warning Systems in Ellicott City, Maryland

As flood events in the United States grow in frequency and intensity, the use of applied remote sensing analyses is increasingly necessary for effective flood monitoring and warning systems. The NASA DEVELOP National Program partnered with the local government of Howard County, Maryland, to investigate the use of machine learning for advanced flood risk detection, and to test the feasibility of integrating this approach into the county’s flood early warning system. To strengthen the efforts of the Howard County Office of Emergency Management (OEM), the project developed a statistical model capable of hindcasting the two severe flash flood events that devastated Ellicott City and transitioned to a ‘Long Short-Term Memory’ based sequence-to-sequence deep learning model with 8-hour forecast capability. The team combined data inputs from public sources including river and precipitation gauges, NASA and NOAA Earth observations, and numerical weather model products using scripts written in the Google Colaboratory Python scripting environment. In addition to designing the deep learning architecture, the team trained and tested the model, and evaluated its performance using Nash-Sutcliffe Efficiency. The final product, the Sequentially Trained Real-time EstimAted Model (STREAM) predicts stage height for the Hudson Branch gauge in Ellicott City using data products available in near real-time, including the High-Resolution Rapid Refresh model’s accumulated precipitation forecasts supplemented by stream gauge data from the OEM and the U.S. Geological Survey. STREAM was incorporated into an online dashboard in a user-friendly interface capable of triggering the alarms that initiate emergency response protocols up to 8 hours in advance of a predicted severe flood event. The project demonstrated the potential for the integration of open data and Earth observations into a flood risk forecasting tool capable of informing near real-time decision making.

NASA DEVELOP↗