SEARCH · Search NASA

Results for “decision-making under uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Optimal economic dispatch policy for prosumer with energy storage considering self-consumption demand

This paper analyzed the effects of self-consumption demand on the joint economic dispatch of prosumers (energy consumers who are also producers), particularly for prosumers with both energy storage and distributed energy sources (DERs). Studies in the existing literature on the economic dispatch scheduling policy of energy storage, mostly from the perspective of electricity merchants, do not address the impacts of self-consumption demand. However, due to the intermittent and high levels of uncertainty regarding DERs generation and the dynamic demand of the prosumer, production and consumption are not always simultaneous; there are two possible scenarios in each period depending on whether DERs generation can meet prosumers' self-consumption or not. Incorporating the self-consumption demand will pose modeling challenges since these two scenarios cannot occur simultaneously in each period, and different scenarios require different decisions for prosumers. Further, this paper analyzed the two scenarios separately to find the optimal storage scheduling strategy, and the results were combined to get the optimal global solution. We focused on prosumers' economic decision-making while considering self-consumption demand and the physical constraints of a battery based on dynamic programming. Our study showed that the feasible state of charge (SOC) range of storage can be segmented into several sub-ranges by SOC reference points under the above two scenarios. As a result, a prosumer's optimal scheduling can be uniquely and conveniently selected based on the sub-ranges within which the current SOC falls. The results, therefore, provided multistage decision-making guidance for prosumers with energy storage.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2026

Integrating science for water security governance

Hydrological extremes are intensifying globally, increasing the complexity of decisions required to ensure water security. Advances in hydrological science, modeling, and data systems have expanded the technical frontier of water research, yet uptake of scientific insights in policy and management decisions remains limited. This persistent science–policy gap is not primarily a failure of knowledge generation or robustness, but an institutional challenge shaped by how scientific and governance systems are organized, coordinated, and connected to support the effective use of scientific knowledge. These challenges are particularly pronounced in multi-level and transboundary water governance, where decisions span jurisdictions and require coordination across institutional and political boundaries. We synthesize research at the science–policy interface and evidence from water security initiatives to show how institutional arrangements, scientific tool development, and research practices enable or constrain the sustained use of scientific knowledge in water-security governance processes. Building on these insights, we develop ‘shared decision infrastructure’ as a framing to describe how scientific knowledge is embedded within the institutional, relational, and procedural arrangements that connect science to decision-making processes over time. We translate this framing into a practical intervention roadmap centered on institutional design, tool translation, sustained co-production, and outcome-oriented evaluation to support the integration of science into ongoing governance processes. By positioning science as shared decision infrastructure, the roadmap clarifies how researchers can design scientific efforts that support more coordinated, accountable, and adaptive water security decisions amid deepening uncertainty.

M whitney, Kristen [NASA Goddard Space Flight Cent↗

DOE OSTI2026

American cities in a time of global environmental change: the case of the Baltimore Social-Environmental Collaborative

The Baltimore Social-Environmental Collaborative (BSEC) Urban Integrated Field Laboratory seeks a new paradigm for urban climate research. Motivated by deep uncertainties in urban climate and the future of urban systems, BSEC works collaboratively across institutions and stakeholder groups to co-generate the science needed to advance energy security and resilience to extreme events across the city of Baltimore, Maryland, USA, and to do so in a manner that can inform similar efforts in other cities. BSEC begins with stakeholder priorities (health, affordable energy, etc) and designs observation networks and models to deliver climate science to address them. This takes the form of an iterative collaborative cycle, in which an initial research strategy is repeatedly updated in conversation with community partners, and researchers and stakeholders learn from each other. To date, this cycle has included multiple rounds of collaborative deliberation on urban heat mitigation, in which a multicriteria decision tool has been updated with more community-relevant spatial structure and modified optimization metrics. The guiding objective of this cycle is to inform potential ‘secure and resilient pathways’ for energy and infrastructure. In doing so, BSEC addresses fundamental urban science questions in natural and social sciences. It also tests our ability to integrate this science in a manner that advances participatory decision-making for urban resilience.

climate↗

DOE OSTI2022

Decision Science for Machine Learning (DeSciML)

The increasing use of machine learning (ML) models to support high-consequence decision making drives a need to increase the rigor of ML-based decision making. Critical problems ranging from climate change to nonproliferation monitoring rely on machine learning for aspects of their analyses. Likewise, future technologies, such as incorporation of data-driven methods into the stockpile surveillance and predictive failure analysis for weapons components, will all rely on decision-making that incorporates the output of machine learning models. In this project, our main focus was the development of decision scientific methods that combine uncertainty estimates for machine learning predictions, with a domain-specific model of error costs. Other focus areas include uncertainty measurement in ML predictions, designing decision rules using multiobjecive optimization, the value of uncertainty reduction, and decision-tailored uncertainty quantification for probability estimates. By laying foundations for rigorous decision making based on the predictions of machine learning models, these approaches are directly relevant to every national security mission that applies, or will apply, machine learning to data, most of which entail some decision context.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Extreme-scale stochastic optimization and simulation via learning-enhanced decomposition and parallelization (Final Technical Report)

Stochastic optimization and simulation models ubiquitously arise in designing and operating complex service/engineering systems. They can be extreme in scale due to high-dimensional data and decisions, and can also involve decisions made sequentially in response to newly revealed data, both causing significant computational challenge. The objective of this research is to explore a unified framework that integrates machine learning with discrete optimization and risk-averse modeling, to improve the efficiency of decomposition paradigms for stochastic optimization and simulations at extreme scale. The models we consider represent a broad class of complex decision-making problems, where 0-1 or continuous decisions are made before and/or after knowing multiple sources of uncertainties that could be correlated. We will employ machine learning methods to dynamically decide and prioritize computational procedures, including cut generation, branching, and bounding of the optimal objective. Furthermore, the research will shed new lights on the traditional decomposition algorithms for extreme-scale computing. Deliverables of the research include new modeling and computational methods for advancing the state-of-the-art research in optimization and simulation, bringing many relevant risk-averse, data-driven optimization problems in practice within the range of tractability. Examples include distributed computing server scheduling and sensor deployment for monitoring critical infrastructures. Success in this effort will enable progress in solving multiple extreme-scale problems in the complex system design and operations arising from DoE missions in energy, environment, and national security.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Transmission Operator Workflows for Real-Time Reliability Studies: A Review of Control Room Practices and Naturalistic Decision Making

This report provides an overview of real-time reliability study tools and their use by power system operators in the control room environment. After introducing some of the nuances of the control room environment and the differences in perspectives between power system engineers and operators, the roles and responsibilities of key entities involved in RTCA workflows are introduced. These are specifically the transmission system operator (TOP) and reliability coordinator (RC), which are required to run tools such as real-time contingency analysis (RTCA) as part of a real-time reliability assessment every 30 minutes, as dictated by a series of standards issued by the North American Electric Reliability Corporation (NERC). The process by which power systems operators operate the grid is discussed in terms of naturalistic decision making (NDM) and the recognition-primed decision-making (RPD) model. This cognitive model describe how experts working in high-risk, high-stress environments make safety-critical decisions under uncertainty and time pressure. For power system operators, the mental simulations involved in the traditional RPD model are supplemented by physics-based simulations using numerical tools, such as RTCA, to improve situational awareness and effectiveness of control actions. Next, a generic workflow is introduced to describe operator decision making for running RTCA tools and responding to system violations on a pre-contingent basis. The types of analysis performed and control actions chosen by power system operators are described in detail. The overall high-level workflow is then expanded in subsequent sections, with special attention given to high-voltage violations, low-voltage violations, and thermal overloads. Each type of violation is described in detail, with explanations of common causes, impacts on equipment and customers, and mitigation strategies. An additional workflow diagram is provided for each type of violation.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Circularity Futures Workshop Series: Summary Report

The aim of this report is to synthesize key feedback received from the three-part Circularity Futures workshop series held in Spring 2024. The workshop series was conducted by the National Renewable Energy Laboratory (NREL) on behalf of U.S. Department of Energy, Office Energy Efficiency and Renewable Energy (EERE), and was broken into three workshops: Workshop 1 - Circularity Analysis Needs and Priorities; Workshop 2 - Circularity Metrics and Indicators; and Workshop 3 - Circularity Data. Together, the workshops focused on identifying the existing priorities and gaps in the circularity modeling space, understanding different stakeholders' use and interpretation of circularity metrics and indicators, identifying common data gaps and data quality challenges, and assessing the robustness of available solutions. The workshop series brought a diverse group of stakeholders - including representatives from U.S. government offices, national labs, nonprofit organizations, industry, and academia - to collect first-hand feedback on needs, priorities, challenges and opportunities in the circularity modeling and analysis space. The workshop discussions highlighted numerous common needs, priorities and challenges among the interviewed groups. Several topics were frequently discussed, including: 1) Circularity as a pathway for sustainable economic growth: While circularity is generally defined in terms of resource conservation and reducing wasteful disposal of materials, participants agreed that circular strategies should serve broader economic, environmental, and social goals. It is therefore crucial for circularity analysis to look beyond waste reduction and instead evaluate a variety of impact metrics such as cost savings, job creation, air quality, and pollutant emissions. Mutli-criteria decision-making frameworks may be useful for making sense of disparate metrics and evaluating tradeoffs between impact categories.; 2) Economic and social factors are not well understood: Underdevelopment of existing end-of-life (EOL) management infrastructure, inconsistent standardization codes and policy space in reusing recycled content, and suboptimal collection and sorting strategies collectively contribute to uncertainty about the economic potential of circular pathways. The latter observation is consistent among all technologies but more emphasized for renewable energy systems. Social impacts of circularity practices are less understood and less researched than other sustainability aspects.; 3) Inconsistent methods for assessing emerging technologies: LCA and TEA results vary widely depending on the assumptions made with regards to market adoption of new technologies. Emerging technologies suffer limited availability of data needed to conduct a robust circularity analysis. Yet, understanding projected impacts of proposed nascent technology is a key need for different stakeholder groups.; and 4) Lack of temporally and geospatially explicit data: There is a need for open data that represents variations in circularity technologies over time and location. The lack thereof leads to aggregated and potentially misrepresented results in circularity analysis. Sensitivity analyses should be included to verify whether options perceived as more sustainable align with real-world practices.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2026

Testing Recommendations for Thermal Failure Analysis

Minimizing ranges of uncertainty in failure QOIs is vital for making informed engineering decisions. To improve our decision-making ability, it is necessary to have a QOI representation that captures more complex failure modes. Capturing the physics of complex time-dependent temperature failure in explicit detail can be computationally and experimentally expensive. To avoid this blooming of complication and costs, an empirical QOI component-level failure/damage model may be sufficient to help identify component-level failure. In this context, failure and damage are abstractions of more complicated processes that lead to the cessation of normal operating behavior, not a representation of mechanical damage, such as void formation. Such a model requires minimal increases in computational cost and experimental burden. The proposed QOI serves as an indicator that something has likely failed, similar to the previous critical failure temperatures or pressures. This QOI will not replace any existing failure QOIs. It will be calculated alongside them, allowing for preservation of legacy style QOI measurements with new context from the new QOI.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Are long-term climate projections useful for on-farm adaptation decisions?

The current literature on climate services for farmers predominantly focuses on seasonal forecasts, with an assumption that longer-term climate projections may not be suitable for informing farming decisions. In this paper, we explore whether certain types of long-term climate projections may be useful for some specific types of farming decisions. Through interviews with almond tree crop farmers and farm advisors in California, we examine how farmers perceive the utility and accuracy levels of long-term climate projections and identify the types of projections that they may find useful. The interviews revealed that farmers often perceive long-term climate projections as an extension of weather forecasts, which can lead to their initial skepticism of the utility of such information. However, we also found that when farmers were presented with long-term trends or shifts in crop-specific agroclimatic metrics (such as chill hours or summer heat), they immediately perceived these as valuable for their decision-making. Hence, the manner in which long-term projections are framed, presented, and discussed with farmers can heavily influence their perception of the potential utility of such projections. The iterative conversations as part of the exploratory interview questions, served as a tool for “ joint construction of meaning” of complex and ambiguous terms such as “long-term climate projections,” “long-term decisions” and “uncertainty.” This in-turn supported a joint identification (and understanding) of the types of information that can potentially be useful for on-farm adaptive decisions, where the farmer and the interviewer both improvise and iterate to find the best types of projections that fit specific decision-contexts. Overall, this research identifies both the types of long-term climate information that farmers may consider useful, and the engagement processes that are able to effectively elicit farmers' long-term information needs.

Jagannathan, Kripa↗

NASA NTRS2016

Climate Science for Decision-Making in the New York Metropolitan Region

New York City is one of the world's most vulnerable cities to coastal flooding, due to a high concentration of population and assets near a coastline exposed to warm-season tropical storms and cold -season Nor'easter storms. Among U.S. cities, New York City is second only to New Orleans in population living less than 4 ft above the local high tide. By the 2050s, average annual losses due to coastal flooding alone could exceed $2 billion for the combined New York City-Newark region. Perhaps the most iconic example of a vulnerable New York City asset is the financial district located at the southern tip of Manhattan, however low-lying coastal assets include the full complement of major highways, subways and tunnels, hospitals, schools, wastewater treatment plants, food distribution centers, and people's homes. Given the magnitude of the assets at risk, a compelling case can be made that long-term adaptation makes economic sense for New York City. Given New York's access to economic, human, and technological resources for resilience measures, the City may be able to achieve this resilience. The city's political environment-New York City is a place where climate science is generally not a partisan issue-and the city's experience with uncertainty and overall risk framing (e.g., financing of bond issues for multi-billion dollar infrastructure with multidecade expected lifetimes), encourage climate risk framing.

Horton, Radley↗

NASA NTRS2016

A Satellite Data-Driven, Client-Server Decision Support Application for Agricultural Water Resources Management

Water cycle extremes such as droughts and floods present a challenge for water managers and for policy makers responsible for the administration of water supplies in agricultural regions. In addition to the inherent uncertainties associated with forecasting extreme weather events, water planners need to anticipate water demands and water user behavior in a typical circumstances. This requires the use decision support systems capable of simulating agricultural water demand with the latest available data. Unfortunately, managers from local and regional agencies often use different datasets of variable quality, which complicates coordinated action. In previous work we have demonstrated novel methodologies to use satellite-based observational technologies, in conjunction with hydro-economic models and state of the art data assimilation methods, to enable robust regional assessment and prediction of drought impacts on agricultural production, water resources, and land allocation. These methods create an opportunity for new, cost-effective analysis tools to support policy and decision-making over large spatial extents. The methods can be driven with information from existing satellite-derived operational products, such as the Satellite Irrigation Management Support system (SIMS) operational over California, the Cropland Data Layer (CDL), and using a modified light-use efficiency algorithm to retrieve crop yield from the synergistic use of MODIS and Landsat imagery. Here we present an integration of this modeling framework in a client-server architecture based on the Hydra platform. Assimilation and processing of resource intensive remote sensing data, as well as hydrologic and other ancillary information occur on the server side. This information is processed and summarized as attributes in water demand nodes that are part of a vector description of the water distribution network. With this architecture, our decision support system becomes a light weight 'app' that connects to the server to retrieve the latest information regarding water demands, land use, yields and hydrologic information required to run different management scenarios. Furthermore, this architecture ensures all agencies and teams involved in water management use the same, up-to-date information in their simulations.

Agricultural↗

NASA NTRS2016

Unlocking the Full Potential of Earth Observation During the 2015 Texas Flood Disaster

Intense rainfall during late April and early May 2015 in Texas and Oklahoma led to widespread and sustained flooding in several river basins. Texas state agencies relevant to emergency response were activated when severe weather then ensued for 6 weeks from 8 May until 19 June following Tropical Storm Bill. An international team of scientists and flood response experts assembled and collaborated with decision-making authorities for user-driven high-resolution satellite acquisitions over the most critical areas; while experimental automated flood mapping techniques provided daily ongoing monitoring. This allowed mapping of flood inundation from an unprecedented number of spaceborne and airborne images. In fact, a total of 27,174 images have been ingested to the USGS Hazards Data Distribution System (HDDS) Explorer, except for the SAR images used. Based on the Texas flood use case, we describe the success of this effort as well as the limitations in fulfilling the needs of the decision-makers, and reflect upon these. In order to unlock the full potential for Earth observation data in flood disaster response, we suggest in a call for action(i) stronger collaboration from the onset between agencies, product developers, and decision-makers;(ii) quantification of uncertainties when combining data from different sources in order to augment information content; (iii) include a default role for the end-user in satellite acquisition planning; and(iv) proactive assimilation of methodologies and tools into the mandated agencies.

Texas↗

NASA NTRS2020

Improving Trust in Deep Neural Networks with Nearest Neighbors

Deep neural networks are used increasingly for perception and decision-making in UAVs. For example, they can be used to recognize objects from images and decide what actions the vehicle should take. While deep neural networks can perform very well at complex tasks, their decisions may be unintuitive to a human operator. When a human disagrees with a neural network prediction, due to the black box nature of deep neural networks, it can be unclear whether the system knows something the human does not or whether the system is malfunctioning. This uncertainty is problematic when it comes to ensuring safety. As a result, it is important to develop technologies for explaining neural network decisions for trust and safety. This paper explores a modification to the deep neural network classification layer to produce both a predicted label and an explanation to support its prediction. Specifically, at test time, we replace the final output layer of the neural network classifier by a k-nearest neighbor classifier. The nearest neighbor classifier produces 1) a predicted label through voting and 2) the nearest neighbors involved in the prediction, which represent the most similar examples from the training dataset. Because prediction and explanation are derived from the same underlying process, this approach guarantees that the explanations are always relevant to the predictions. We demonstrate the approach on a convolutional neural network for a UAV image classification task. We perform experiments using a forest trail image dataset and show empirically that the hybrid classifier can produce intuitive explanations without loss of predictive performance compared to the original neural network. We also show how the approach can be used to help identify potential issues in the network and training process.

Lee, Ritchie↗

NASA NTRS2020

Intelligent Contingency Management for Urban Air Mobility

The third aviation revolution is seeking to enable transportation where users have access to immediate and flexible air travel; the users dictate trip origin, destination and timing. One of the major components of this vision is urban air mobility (UAM) for the masses. UAM means a safe and efficient system for vehicles to move passengers and cargo within a city. In order to reach UAM’s full market potential the vehicle will have to be autonomous. One of the primary challenges of autonomous flight is dealing with off-nominal events, both common and unforeseen; thus, intelligent contingency management (ICM) is one of the enabling technologies. In this context, the vehicle has to be aware of its internal state and external environment at all times, ascertain its capability and make decisions about mission completion or modification. All of these functions require data to model and assess the environment and then take actions based on these models. Necessarily, there is uncertainty associated with the data and the models generated from it. Since we are dealing with safety-critical systems, one of the main challenges of ICM is to generate sufficient data and to minimize its uncertainty to enable practical and safe decision making. We propose an overall architecture that incorporates deterministic and learning algorithms together to assess vehicle capabilities, project these into the future and make decisions on mission management level. A layered approach allows for mature parts and technologies to be integrated into early highly automated vehicles before the final state of autonomy is reached.

data-driven systems↗

NASA NTRSDate not supplied

Runway Configuration Management with Offline Reinforcement Learning

Runway configuration management (RCM) is a challenging task, and it affects the efficiency of the National Airspace System (NAS) and airport surface operations significantly. Each airport, depending on the geometry, capacity, local climate patterns, etc. has multiple configurations for the runway usage for arriving and departing flights. Many factors such as the incoming/outgoing traffic load, wind direction and speed, convective weather, cloud ceiling and other environmental factors might affect the choice of a runway configuration at any point in time. However, other factors such as safety measures and regulations, noise abatement, capacity of each configuration, and preference of the air traffic controllers (ATCs) can also play a significant role in selecting the configuration. A sub-optimal selection of the runway configuration, or delay in making configuration changes might result in significant increase in taxi times for aircraft on the surface of the airport, fuel and energy use of the aircraft, and maintenance costs. It can also lead to safety concerns, such as an aircraft performing one or more go-arounds before being able to land. All these factors make RCM an extremely important and challenging decision-making process for the ATCs. The current state of practice sets the runway configuration by the ATCs based on relevant information available at the time including weather, traffic, noise abatement, safety bounds, etc. This makes the decision-making process subjective based on the accuracy of the available information and the bias in human decision making. Unfortunately, this approach yields poor results (e.g., significant delays) if the predicted outcomes are uncertain and their relative impact is not well understood. This is especially evident when the uncertainty increases the size of possible predicted outcomes (combinatorial explosion in possible scenarios) that cannot be handled by human reasoning. On the other hand, an automated approach based on machine intelligence can make use of historical data and search through all (or significant amount of) possible scenarios under uncertainty and make well-informed decisions.

Milad Memarzadeh↗

NASA NTRSDate not supplied

A Modeling Approach to Support Changeability Analysis and Management of Earth Observation Portfolios

Earth observing (EO) mission portfolios provide data on many geophysical parameters that collectively inform our understanding of the Earth system. To observe large-scale climate trends, it is important to collect data on many parameters over time. However, gaps may occur between missions due to formulation complexity, development delays, and uncertain events in operations. These gaps may impact the ability to provide sustained measurements. Developing a balanced EO mission portfolio is key to supporting study of the Earth system. Yet, decision-making at the portfolio level is a complex task because missions often address multiple parameters. Currently, the likelihood of data gaps is often assessed for an individual parameter, but decisions and events can impact data collection for many parameters. This poster introduces a modeling approach that supports management of the many parameter impacts of decisions and uncertain events. The approach provides a foundation for analyzing changeability in EO mission portfolios. Changeability analysis provides insight into a portfolio’s sensitivity to changes caused by decisions or uncertain events. The model is developed to analyze two aspects of changeability, robustness and flexibility, in EO mission portfolios. In context of these portfolios, the authors define robustness as a portfolio’s ability to provide sustained measurements despite future uncertainties, and they define flexibility as the availability of decision alternatives that contribute to parameter coverage at little detriment to the rest of the portfolio. The proposed model characterizes a portfolio’s sensitivity to change by capturing mission-parameter interconnectivity and redundancy in parameter coverage. The model can be analyzed to identify events that can create substantial gaps, thereby revealing focus areas for mitigation efforts. The model can also be analyzed to identify gap mitigation decisions that may improve parameter coverage with little detriment to the rest of the portfolio, thereby revealing favorable pathways for portfolio improvement. Developing a process for identifying these threats and opportunities will be the focus of future work. A parallel poster proposes a standard for visualizing and communicating portfolio-level impacts of decisions and uncertain events.

Lindsey Jacobson↗

NASA NTRS2024

Climate Extremes and Risks: Links Between Climate Science and Decision-Making

The World Climate Research Programme (WCRP) envisions a future where actionable climate information is universally accessible, supporting decision makers in preparing for and responding to climate change. In this perspective, we advocate for enhancing links between climate science and decision-making through a better and more decision-relevant understanding of climate impacts. The proposed framework comprises three pillars: climate science, impact science, and decision-making, focusing on generating seamless climate information from sub-seasonal, seasonal, decadal to century timescales informed by observed climate events and their impacts. The link between climate science and decision-making has strengthened in recent years, partly owing to undeniable impacts arising from disastrous weather extremes. Enhancing decision-relevant understanding involves utilizing lessons from past extreme events and implementing impact-based early warning systems to improve resilience. Integrated risk assessment and management require a comprehensive approach that encompasses good knowledge about possible impacts, hazard identification, monitoring, and communication of risks while acknowledging uncertainties inherent in climate predictions and projections, but not letting the uncertainty lead to decision paralysis. The importance of data accessibility, especially in the Global South, underscores the need for better coordination and resource allocation. Strategic frameworks should aim to enhance impact-related and open-access climate services around the world. Continuous improvements in predictive modeling and observational data are critical, as is ensuring that climate science remains relevant to decision makers locally and globally. Ultimately, fostering stronger collaborations and dedicated investments to process and tailor climate data will enhance societal preparedness, enabling communities to navigate the complexities of a changing climate effectively.

climate extremes↗

DOE OSTI2023

Operation Optimization Using Reinforcement Learning with Integrated Artificial Reasoning Framework

In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.

Kim, Junyung↗