Defender Policy Evaluation and Resource Allocation With MITRE ATT&CK Data and Evaluations.
Abstract not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
In large and complex systems, operational decision-making requires a systematic analysis with a vast amount of data from both process parameters and component status monitoring. In this paper, we present an integrated artificial reasoning approach for system state transition models that can help operational decision-making with explainable and traceable reasoning. The integrated artificial reasoning framework is a physics-based approach of defining the system structure in a Bayesian network, so we leveraged it in a Markov decision process (MDP) for finding optimal operational solutions. In our proposed framework, the MDP is implemented on a dynamic Bayesian network (DBN), which represents causalities in a system. The multilevel flow modeling was utilized in order to extract these causalities in a more efficient and objective manner. Since multilevel flow modeling is based on the fundamental energy and mass conservation laws, the target system is decomposed into several mass, energy, and information structures, which serve as the basis for a DBN. The MDP consists of the processes of finding a solution for the Bellman equation, which can be derived from the conditional probability equations of the constructed DBN. System operators can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from the component degradation process or random failures. We analyzed a simplified example system to illustrate finding an optimal operational policy with this approach.
Learning-based car-following control (LCC) of connected and autonomous vehicles (CAVs) is gaining significant attention with the advancement of computing power and data accessibility. While the flexibility and large model capacity of model-free architecture enable LCC to potentially outperform the model-based car-following (CF) model in improving traffic efficiency and mitigating congestion, the generalizability of LCC for traffic conditions different from the training environment/dataset is not well-understood. Herein, this study seeks to explore the impact of stop-and-go traffic in the training dataset on the generalizability of LCC. It uses the characteristics of lead vehicle trajectories to describe stop-and-go traffic, and links the theory of identifiability (i.e., obtaining a unique parameter estimation result using sensor measurements) to the generalizability of behavior cloning (BC) and policy-based deep reinforcement learning (DRL). Correspondingly, the study shows theoretically that: (i) stop-and-go traffic can enable the property of identifiability and enhance the control performance of BC-based LCC in different traffic conditions; (ii) stop-and-go traffic is not necessary for DRL-based LCC to generalize to different traffic conditions; (iii) DRL-based LCC trained with only constant-speed lead vehicle trajectories (not sufficient to ensure identifiability) can be generalized to different traffic conditions; and (iv) stop-and-go traffic increases variance in the training dataset, which improves the convergence of parameter estimation while negatively impacting the convergence of DRL to the optimal control policy. Numerical experiments validate the above findings, illustrating that BC-based LCC entails comprehensive training datasets for generalizing to different traffic conditions, while DRL-based LCC can achieve generalization with simple free-flow traffic training environments. This further suggests DRL as a more promising and cost-effective LCC approach to reduce operational costs, mitigate traffic congestion, and enhance safety and mobility, which can accelerate the deployment and acceptance of CAVs.
Here, we present a novel stochastic approach to binary optimization suited for optimal experimental design (OED) for Bayesian inverse problems governed by mathematical models such as partial differential equations. The OED utility function, namely, the regularized optimality criterion, is cast into a stochastic objective function in the form of an expectation over a multivariate Bernoulli distribution. The probabilistic objective is then solved by using a stochastic optimization routine to find an optimal observational policy. This formulation (a) is generally applicable to binary optimization problems with soft constraints and is ideal for OED and sensor placement problems; (b) does not require differentiability of the original objective function (e.g., a utility function in OED applications) with respect to the design variable, and thus it enables direct employment of sparsity-enforcing penalty functions such as $\ell_0$, without needing to utilize a continuation procedure or apply a rounding technique; (c) exhibits much lower computational cost than traditional gradient-based relaxation approaches; and (d) can be applied to both linear and nonlinear OED problems with proper choice of the utility function. The proposed approach is analyzed from an optimization perspective with detailed convergence analysis of the optimization approach and is also analyzed from a machine learning perspective with correspondence to policy gradient reinforcement learning. The approach is demonstrated numerically by using an idealized two-dimensional Bayesian linear inverse problem and validated by extensive numerical experiments carried out for sensor placement in a parameter identification setup.
Strategic environmental assessment (SEA) focuses primarily on assessing how policies, plans, and programs (PPPs) influence the sustainability of the involved regions. However, the processes of assessing policies and developing management strategies for pollution load and resource use are usually separate in the current SEA system. This study developed a policy management methodology to overcome the defects generated during the above processes. This work first devised a dynamic management framework using the methods of systems thinking, system dynamics, and Managing for Results (MFRs). Furthermore, a driving force-pressure-state-impact-response (DPSIR) indicator system was developed. The golf course installation policy was applied as a case study. Taiwan, counties of Taiwan, and the golf courses within those individual counties were identified as a system, subsystems, and objects, respectively. This study identified an object-linked double-layer framework with multi-stage-option to simultaneously to quantify golf courses in each subsystem and determine ratios of abatement and allocation for pollution load and resource use of each golf course. The DPSIR indicator values for each item of each golf course in each subsystem are calculated based on the options taken in the two decision layers. The summation of indicator values for all items of all golf courses in all subsystems according to various options is defined as the sustainability value of the policy. An optimization model and a system (IDPMS) were developed to obtain the greatest sustainability value of the policy, while golf course quantity, human activity intensity, total quantities of pollution load and resource use are simultaneously obtained. The solution method based on enumeration of multiple bounds for objectives and constraints (EMBOC) was developed for the problem with 1.95 x 10{sup 128} combinations of possible options to solve the optimal solution in ten minutes using a personal computer with 3.0 GHz CPU. This study obtain the policy with the optimal environmental sustainability value in Taiwan is 102 golf courses. Human activity intensity and total quantities of pollution load and resource use which are concurrently obtained are less than those of the existing policy and the existing quantities in 2006. The optimal solution remains unchanged under most sensitivity analysis conditions, unless the weights and constraints are extremely changed. The analytical results indicate that the proposed methodology can be used to assist the authorities for simultaneously generating and assessing the policy during the SEA process.
Work on the NYSERDA-MDAO project, presented at the OpenMDAO workshop.
We consider a price-based network revenue management problem with multiple products and multiple reusable resources. Each randomly arriving customer requests a product (service) that needs to occupy a sequence of reusable resources (servers). We adopt an incomplete information setting where the firm does not know the price-demand function for each product and the goal is to dynamically set prices of all products to maximize the total expected revenue of serving customers. We propose novel batched bandit learning algorithms for finding near-optimal pricing policies, and show that they admit a near-optimal cumulative regret bound of $O(J\sqrt{XT})$, where J, X, and T, are the numbers of products, candidate prices, and service periods, respectively. As part of our regret analysis, we develop the first finite-time mixing time analysis of an open network queueing system (i.e., the celebrated Jackson Network), which could be of independent interest. Our numerical studies show that the proposed approaches perform consistently well.
Here this paper investigates how the market impact of electricity merchants and uncertainty of wind generation affect their co-optimized scheduling policy, specifically for merchants who have both energy storage and wind plants. In the existing literature, merchants' trading actions are usually assumed not to affect market prices; however, a large-scale energy storage merchant's actions can affect market prices. To this end, we approximate the electricity price by a linear function of the quantity of power traded by the merchant in the reward function to achieve decision-making incorporating the market impact. This paper utilizes the dynamic programming approach to analyze merchants' optimal multi-period decision-making incorporating market impact, uncertain wind generation, and energy storage constraints. First, our results demonstrate that for a merchant with co-located energy storage facilities and wind power plants, the energy storage's feasible state of charge (SOC) range can be segmented into four possible sub-ranges by three analytically developed SOC reference points. The unique optimal trading decision can be achieved by comparing the current energy inventory and the SOC references of the next period. Second, our results show that market impact and uncertainties substantially change the optimal storage scheduling policy by impacting the values of the reference points. To mitigate the negative effect of the merchant's market impact on buying and selling actions, the merchant may reduce the amount of generating or pumping electricity each period to maximize profit. Moreover, we identify and investigate the trade-off between market price and transaction quantity. Our findings provide co-optimized scheduling guidance for electricity merchants with co-located energy storage and renewable power plants systems.
Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.
During dry and windy seasons, environmental conditions significantly increase the risk of wildfires, exposing power grids to disruptions caused by transmission line failures. Wildfire propagation exacerbates grid vulnerability, potentially leading to prolonged power outages. To address this challenge, we propose a multistage optimization model that dynamically adjusts transmission grid topology in response to wildfire propagation, aiming to develop an optimal response policy. By accounting for decision-dependent uncertainty, where line survival probabilities depend on usage, we employ distributionally robust optimization to model uncertainty in line survival distributions. We adapt the stochastic nested decomposition algorithm and derive a deterministic upper bound for its finite convergence. To enhance computational efficiency, we exploit the Lagrangian dual problem structure for a faster generation of Lagrangian cuts. Using realistic data from the California transmission grid, we demonstrate the superior performance of dynamic response policies against two-stage alternatives through a comprehensive case study. In addition, after solving the multistage formulation, we construct easy-to-implement policies that significantly reduce computational burden while maintaining good performance in real-time deployment. History: Accepted by Russell Bent, Area Editor for Network Optimization: Algorithms and Applications. Funding: This work was supported by the U.S. Department of Energy, Office of Electricity [Grant DE-AC02-05CH11231]. The work of R. Jiang was supported in part by the U.S. National Science Foundation, Division of Electrical, Communications and Cyber Systems [Grant ECCS-1845980] and the U.S. Air Force Office of Scientific Research [Grant FA9550-23-1-0323]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2025.1210 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2025.1210 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .
We present an approach for maximizing a global utility function by learning how to allocate resources in an unsupervised way. We expect interactions between allocation targets to be important and therefore propose to learn the reward structure for near-optimal allocation policies with a GNN. By relaxing the resource constraint, we can employ gradient-based optimization in contrast to more standard evolutionary algorithms. Our algorithm is motivated by a problem in modern astronomy, where one needs to select-based on limited initial information-among $10^9$ galaxies those whose detailed measurement will lead to optimal inference of the composition of the universe. Our technique presents a way of flexibly learning an allocation strategy by only requiring forward simulators for the physics of interest and the measurement process. We anticipate that our technique will also find applications in a range of resource allocation problems.
Optimization and sampling algorithms play a central role in science and engineering as they enable finding optimal predictions, policies, and recommendations, as well as expected and equilibrium states of complex systems. The notion of “optimality” is formalized by the choice of an objective function, while the notion of an “expected” state is specified by a probabilistic model for the distribution of states. Optimizing rugged objective functions and sampling multimodal distributions is computationally challenging, especially in high-dimensional problems. Here, for this reason, many optimization and sampling methods have been developed by researchers working in disparate fields such as Bayesian statistics, molecular dynamics, genetics, quantum chemistry, machine learning, weather forecasting, econometrics, and medical imaging.
States and the federal government are increasingly engaged in the challenges around decarbonizing the electric grid. In particular, regulators, consumers, stakeholders, and utilities recognize the need to carefully consider the role natural gas will play in a decarbonized future. A variety of technology and policy options to reduce greenhouse gas emissions associated with natural gas use are available, including energy efficiency programs, demand reduction tools, strategic electrification, and strategies to reduce emissions from natural gas production, transportation, and consumption. Low-carbon fuels – mainly renewable natural gas (RNG) and clean hydrogen – are being considered an important component of decarbonization goals. RNG and hydrogen may be able to meaningfully reduce emissions from processes independent of geologic natural gas, displacing emissions of methane, a powerful greenhouse gas. Although RNG and hydrogen are not cost-competitive today with geologic natural gas and are smaller in scale and potential than other decarbonization options, they can be explored as potential critical tools to decarbonize sectors that are difficult to electrify or shift off of natural gas entirely, such as air travel, industrial processes, maritime transport, long-distance trucking, space heating on cold days, and railroads (Nadel, 2022). The role of this report is to provide informational context for state utility regulators to understand the impacts of and challenges associated with broader integration of low-carbon fuels, followed by examples of state regulatory actions taken to date to facilitate the development of low-carbon fuels. Setting clear guidance to calculate the environmental benefits of low-carbon fuels and continuing federal and state investments in research and development to reduce costs relative to fossil fuels will be important steps to take to signal the desire to grow the market for these fuels. State public utility commissions may play a key role in setting regulatory frameworks for low-carbon fuels and ensuring that ratepayer funds, if utilized, are done so to further the public interest. This report is intended to summarize decisions that states have made to date on low-carbon fuels. In the spirit of understanding the current market and sharing information, this report provides success stories, and lessons learned across states as regulators implement varying strategies to achieve decarbonization objectives while maintaining their focus on affordability, safety, and reliability of the energy system. The report begins with an introduction of the role of natural gas in the U.S. economy (Section I) and background information on natural gas use, decarbonization, and low-carbon fuels (Section II). Next, the report describes the current market by discussing the scale of current production, emissions intensity, resource potential, and costs of low-carbon fuels compared to geologic natural gas (Section III). Following these sections, the report describes four strategies states have employed to facilitate low-carbon fuels: opening exploratory dockets, approving voluntary tariffs for customers, approving interconnection tariffs for producers, and considering portfolio-wide procurement targets (Section IV). This section lists states that have taken actions in each category, citing utility filings, commission decisions, stakeholder comments, and other relevant sources. Finally, the report concludes with suggested questions regulators may wish to consider regarding low-carbon fuels, in the interest of preparing to make decisions in the future (Section V). These questions include: Are there existing regulatory or technical barriers to voluntary purchases of low-carbon fuels? Can customers work with utilities to procure low-carbon fuels; are producers able to interconnect projects without significant barriers to entry? Should the infrastructure and/or commodity costs of low-carbon fuels be socialized among all ratepayers, or borne solely by the large commercial and industrial (C&I) customers currently driving the market? Should regulated natural gas and/or electric utilities own and operate low-carbon fuel production? How should regulators consider the unique decarbonization potential of low-carbon fuels, particularly for hard-to-abate sectors, in decision-making? Is additional direction or clarity from state policymakers needed? What no-regrets approaches can help facilitate both near-term RNG development and long-term development of hydrogen and other zero-carbon fuels? We collectively wish to express our gratitude to the U.S. Department of Energy, Office of Fossil Energy and Carbon Management, for supporting this report and other technical assistance resources for state regulators on natural gas topics. State regulators operate under a variety of policy environments, and states have vastly different types of energy resources, infrastructure, and customers. While there is no optimal regulatory, policy, or technological solution that will be successful in every state, state regulators can benefit by exchanging lessons learned with their peers across the country. We look forward to continued engagement with our fellow commissioners, commission staff, NARUC, the U.S. Department of Energy, and other stakeholders to develop sound regulation in the public interest.
Many advanced technical tools are available to prevent attacks on national infrastructure. Nevertheless, while traditional analyses of security problems have succeeded in producing good technical solutions, they have often ignored the human factor integral to these problems. Human attackers (who may be individuals or state-level attackers) expend substantial effort to breach security because they have the incentive for doing so. People involved in implementing security follow individual incentives, which need not align with global security concerns; consequently, desired security solutions are often implemented poorly, or not at all. This complex interplay between individual incentives and global (organizational and/or national) goals can be modeled and analyzed using game theoretic techniques. By analyzing not only what is possible, but also what is motivated, a holistic approach to security problems can be developed, informing policy and providing tools to policy makers. We study game theoretic models that unify several current incentive-based approaches to security, and develop simulation-based and mathematical optimization methods for analyzing such models that exploit the high-performance computing capabilities at Sandia. Our first model studies security in interdependent settings, offering a scalable local search heuristic to approximate optimal security decisions in general, and a linear programming approach, coupled with simulations of consequences, to optimally compute security in an important special case. Our second class of models addresses security patrolling problems when an adversary gets to observe the patrol location. We present a general framework, based on stochastic games, for computing optimal security policies in such settings, and present more scalable tools that apply in the important special cases. Our third contribution is a model of security that involves many defenders, but only models non-adaptive attackers (or natural disasters, inadvertent errors, etc). In this model, we demonstrate that the security decisions of many players result in global security configuration that is not very far from optimal, and is much more resilient to environment changes that an optimal solution. This positive effect dissipates, however, when the number of decision makers becomes too large.
The economic assessment of hybrid energy systems (HES) pairing battery energy storage systems (BESSs) and photovoltaics (PV) is highly important for advancing their deployment in power systems. This paper presents an innovative assessment framework, including an optimal control policy for dispatch under uncertainty and procedures for exploring control parameters that maximize economic benefits. The proposed dispatch policy consists of two steps using system forecast information. The first step is to determine whether a BESS will be used within an operational scheduling time frame based on the probability of events and their thresholds. Once the dispatch of BESS is triggered, a model predictive control (MPC) is carried out in the second step for scheduling using the expected value of system information. By exercising this policy with different thresholds, one can explore the trade-offs between short-term benefits and battery lifetime, and identify an optimal threshold that maximizes the total economic benefits within the battery lifetime. An evaluation study in a real-world HES project is presented to illustrate the proposed framework. Compared with traditional optimal dispatch algorithms, the proposed method can significantly improve the economic benefits of an HES scheduled under forecast uncertainties.
Predicting the population-level effects of an infectious disease intervention that incorporate multiple modes of intervention is complicated by the joint non-linear dynamics of both infection transmission and the intervention itself. In this paper, we consider the sensitivity of Dynamic Optimal Control Profiles (DOCPs) for the optimal joint investment in both a contagiousness and susceptibility-based control of HIV to bio-behavioral, economic, and programmatic assumptions. The DOCP is calculated using recently developed numerical algorithms that allow controls to be represented by a set of piecewise constant functions that maintain a constant yearly budget. Our transmission model assumes multiple stages of HIV infection corresponding to acute and chronic infection and both within- and between-individual behavioral heterogeneity. We parameterize a baseline scenario from a longitudinal study of sexual behavior in MSM and consider sensitivity of the DOCPs to deviations from that baseline scenario. In the baseline scenario, the primary determinant of the dominant control were programmatic factors, regardless of budget. In sensitivity analyses, the qualitative aspects of the optimal control policy were often robust to significant deviation in assumptions regarding transmission dynamics. In addition, we found several conditions in which long-term joint investment in both interventions was optimal. Our results suggest that modeling in the service of decision support for intervention design can improve population-level effects of a limited set of economic resources. We found that economic and programmatic factors were as important as the inherent transmission dynamics in determining population-level intervention effects. Given our finding that the DOCPs were robust to alternative biological and behavioral assumptions it may be possible to identify DOCPs even when the data are not sufficient to identify a transmission model.
Although solar costs are dropping rapidly, solar power is still more expensive than conventional and other renewable energy options. The solar sector still needs continuing government policy support. These policies are driven by objectives that go beyond the goal of achieving grid parity. The need to achieve multiple objectives and ensure sufficient political support for solar power makes it diffi cult for policy makers to design the optimal solar power policy. The dynamic and uncertain nature of the solar industry, combined with the constraints offered by broader economic, political and social conditions further complicates the task of policy making. This report presents an analysis of solar promotion policies in seven countries - Germany, Spain, the United States, Japan, China, Taiwan, and India - in terms of their outlook, objectives, policy mechanisms and outcomes. The report presents key insights, primarily in qualitative terms, and recommendations for two distinct audiences. The first audience consists of global policy makers who are exploring various mechanisms to increase the penetration of solar power in markets to mitigate climate change. The second audience consists of key Indian policy makers who are developing a long-term implementation plan under the Jawaharlal Nehru National Solar Mission and various state initiatives.
We report a feedback control method to remove grain boundaries and produce circular shaped colloidal crystals using morphing energy landscapes and reinforcement learning–based policies. We demonstrate this approach in optical microscopy and computer simulation experiments for colloidal particles in ac electric fields. First, we discover how tunable energy landscape shapes and orientations enhance grain boundary motion and crystal morphology relaxation. Next, reinforcement learning is used to develop an optimized control policy to actuate morphing energy landscapes to produce defect-free crystals orders of magnitude faster than natural relaxation times. Morphing energy landscapes mechanistically enable rapid crystal repair via anisotropic stresses to control defect and shape relaxation without melting. This method is scalable for up to at least N = 10 3 particles with mean process times scaling as N 0.5 . Further scalability is possible by controlling parallel local energy landscapes (e.g., periodic landscapes) to generate large-scale global defect-free hierarchical structures.