Search NASA⌕ Search

SEARCH · Search NASA

Results for “POLICIES”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Hydropower representation in water and energy system models: a review of divergences and call for reconciliation

Abstract Reservoir-based hydropower systems represent key interactions between water and energy systems and are being transformed under policy initiatives driven by increasing water and energy demand, the desire to reduce environmental impacts, and interacting effects of climate change. Such policies are often guided by complex system models, whereby divergence in system representations can potentially translate to incompatible planning outcomes, thereby undermining any planning that may rely on them. We review different approaches and assumptions in hydropower representation in water and energy systems. While the models and issues are relevant globally, the review focuses on applications in California given its extensive development of energy and water models for policy planning, but discusses the extent to which these observations apply to other regions. Structurally, both water-driven and energy-driven management models are similar. However, in energy models, hydropower is often represented as a single-priority output. Water management models typically allocate water for competing priorities, which are generally uninformed by dynamic electricity load demand, and often result in a lower priority for hydropower. In water models, constraints are increasingly resolved for non-energy components (e.g. inflow hydrology and non-energy water demand); few analogues exist for energy models. These limitations may result in inadequate representations of each respective sector, and vastly different planning outcomes for the same facilities between the two different sectors. These divergent modeling approaches manifest themselves in California where poorly reconciled outcomes may affect decisions in hydropower licensing, electricity grid flexibility and decarbonization, and planning for environmental water. Fully integrated water-energy models are computationally intensive and specific to certain regions, but better representation of each domain in respective efforts would help reconcile divergences in planning and management efforts related to hydropower across energy and water systems.

Rheinheimer, David E. (ORCID:0000000315259069)↗
DOE OSTI2023

Incorporating Operational Uncertainties into the Dispatch of an Integrated Solar and Storage System

The economic assessment of hybrid energy systems (HES) pairing battery energy storage systems (BESSs) and photovoltaics (PV) is highly important for advancing their deployment in power systems. This paper presents an innovative assessment framework, including an optimal control policy for dispatch under uncertainty and procedures for exploring control parameters that maximize economic benefits. The proposed dispatch policy consists of two steps using system forecast information. The first step is to determine whether a BESS will be used within an operational scheduling time frame based on the probability of events and their thresholds. Once the dispatch of BESS is triggered, a model predictive control (MPC) is carried out in the second step for scheduling using the expected value of system information. By exercising this policy with different thresholds, one can explore the trade-offs between short-term benefits and battery lifetime, and identify an optimal threshold that maximizes the total economic benefits within the battery lifetime. An evaluation study in a real-world HES project is presented to illustrate the proposed framework. Compared with traditional optimal dispatch algorithms, the proposed method can significantly improve the economic benefits of an HES scheduled under forecast uncertainties.

Ma, Xu↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗
DOE OSTI2021

On Distributed Model-Free Reinforcement Learning Control with Stability Guarantee

Distributed learning can enable scalable and effective decision making in numerous complex cyber-physical systems such as smart transportation, robotics swarm, power systems, etc. However, the stability of the system is usually not guaranteed in most existing learning paradigms; and this limitation can hinder the wide deployment of machine learning in decision making of safety-critical systems. This paper presents a stability guaranteed distributed reinforcement learning (SGDRL) framework for interconnected linear subsystems, without knowing the subsystem models. While the learning process requires data from a peer-to-peer (p2p) communication architecture, the control implementation of each subsystem is only based on its local state. The stability of the interconnected subsystems will be ensured by a diagonally dominant eigenvalue condition, which will then be used in a model-free RL algorithm to learn the feedback control gains. The RL algorithm structure follows an off-policy iterative framework, with interleaved policy evaluation and policy update steps. We numerically validate our theoretical results by performing simulations on four interconnected sub-systems.

Mukherjee, Sayak↗
DOE OSTI2023

A Modified Maximum Entropy Inverse Reinforcement Learning Approach for Microgrid Energy Scheduling

Increasing popularity of integrating distributed energy resources (DERs) into the power system brings a challenge to optimize the microgrid dispatch policy. The reinforcement learning methods suffer from a long-time problem with the theoretical assumption of the objective/reward function for the microgrid system. Although the traditional inverse reinforcement learning (IRL) approaches can solve this problem to some extent, they encounter a limitation of complex computations for state visitation frequency in the large and continuous state space. To alleviate this limitation, we propose a modified maximum entropy IRL (MMIRL) method to extract the reward function from the expert demonstrations for solving the microgrid energy scheduling problem. The proposed MMIRL algorithm is promising in recovering the reward function and learning the dispatch policy compared to conventional approaches. Case studies are performed in an energy arbitrage problem and a microgrid system with DERs. Results substantiate that the proposed MMIRL approach can learn the dispatch policy with more than 99% efficiency and outperforms other comparative methods.

artificial intelligence, reinforcement learning, m↗
DOE OSTI2023

Reinforcement Learning for Intentional Islanding in Resilient Power Transmission Systems

Intentional islanding is the process of identifying and deliberately decomposing the transmission network to form self-sustained islands from an endangered network during disruptions to improve resilience and security. Most existing intentional islanding models are offline resilience decision tools and hence do not provide outage responses in a timely manner. In this paper, a reinforcement learning (RL) based model for intentional islanding is developed, which offers real-time switching control, online deployability, and adaptability to varying system conditions. The intentional islanding process is formulated as a Markov decision process, where the optimal transmission switching policy is learned using the RL approach. The control policy is learned over an environment that encompasses a Power System Simulator for Engineering (PSS/E) model of the transmission network, facilitated by an interface to the standard openAI Gym framework. The proposed RL-based methodology aims to form stable and self-sustainable islands by ensuring voltage stability while reducing the power mismatch in the formed islands. A proximal policy optimization algorithm is designed, which is suitable for controlling the on/off status of the switches with multi-layer perceptron as value and actor networks. The effectiveness of the proposed framework in the self-recovery of the grid by island formation is applied on the modified IEEE 39-bus test network and validated by dynamic simulations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

An Edge-Cloud Integrated Solution for Buildings Demand Response Using Reinforcement Learning

Buildings, as major energy consumers, can provide great untapped demand response (DR) resources for grid services. However, their participation remains low in real-life. One major impediment for popularizing DR in buildings is the lack of cost-effective automation systems that can be widely adopted. Existing optimization-based smart building control algorithms suffer from high costs on both building-specific modeling and on-demand computing resources. To tackle these issues, this paper proposes a cost-effective edge-cloud integrated solution using reinforcement learning (RL). Beside RL’s ability to solve sequential optimal decision-making problems, its adaptability to easy-to-obtain building models and the off-line learning feature are likely to reduce the controller’s implementation cost. Using a surrogate building model learned automatically from building operation data, an RL agent learns an optimal control policy on cloud infrastructure, and the policy is then distributed to edge devices for execution. Simulation results demonstrate the control efficacy and the learning efficiency in buildings of different sizes. A preliminary cost analysis on a 4-zone commercial building shows the annual cost for optimal policy training is only 2.25% of the DR incentive received. Results of this study show a possible approach with higher return on investment for buildings to participate in DR programs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Multi‐Model Ensembles in Ecosystem Modeling: Challenges and Best Practices for Decision‐Making

Ecosystem models are increasingly central to the decision-making for environmental policy, conservation planning, and climate-related investments. Yet, the growing reliance on Multi-Model Ensembles (MMEs) of ecosystem models by practitioners and policymakers, sometimes under tight timelines and imperfect information, has frequently outpaced the scientific rigor required to ensure ensemble reliability. Here, MMEs refer to approaches that combine targeted predictions from multiple models with the expectation of improving robustness and quantifying predictive uncertainty. Poorly designed MMEs may create a false sense of confidence and lead to suboptimal policy and market decisions. This perspective argues that robust decision-making-relevant MMEs must be grounded on two pillars: (1) rigorous Model Intercomparison Projects (MIPs), which identify inter-model agreement and disagreement, characterize model uncertainties, and evaluate robustness with observationally based benchmarks—MIPs' diagnostic evaluation is so critical that it must be needed to drive MME's decision in model selection and weighting, especially when only a limited number of models available; and (2) co-design by both stakeholders and scientists to ensure that scenarios, metrics and uncertainty requirements provide decision-relevant information. Building upon the past success and lessons from the existing MIPs-MMEs efforts (e.g., climate/Earth system/crop), we derived the theoretical basis for MMEs, addressed their specific challenges in ecosystem modeling, and highlighted proper consideration of model numbers and diversity, risk of model inter-dependence, effective calibration of model parameters, possible overdue of some ecosystem model development, critical roles of open benchmark data across a wide range of conditions, and suggested use of Artificial Intelligence to support MIPs-MMEs. We highlighted the under-recognized opportunity for MIPs and MMEs to drive scientific progress and innovation through identifying better performing models, systematic benchmarking, feedback loops, and targeted model improvement. By following actionable best practice guidelines, MMEs can evolve from ad hoc aggregation of models into a trusted backbone of environmental policy and decision-making.

ecosystem modeling↗
DOE OSTI2021

Applying a science‐based systems perspective to dispel misconceptions about climate effects of forest bioenergy

Abstract The scientific literature contains contrasting findings about the climate effects of forest bioenergy, partly due to the wide diversity of bioenergy systems and associated contexts, but also due to differences in assessment methods. The climate effects of bioenergy must be accurately assessed to inform policy‐making, but the complexity of bioenergy systems and associated land, industry and energy systems raises challenges for assessment. We examine misconceptions about climate effects of forest bioenergy and discuss important considerations in assessing these effects and devising measures to incentivize sustainable bioenergy as a component of climate policy. The temporal and spatial system boundary and the reference (counterfactual) scenarios are key methodology choices that strongly influence results. Focussing on carbon balances of individual forest stands and comparing emissions at the point of combustion neglect system‐level interactions that influence the climate effects of forest bioenergy. We highlight the need for a systems approach, in assessing options and developing policy for forest bioenergy that: (1) considers the whole life cycle of bioenergy systems, including effects of the associated forest management and harvesting on landscape carbon balances; (2) identifies how forest bioenergy can best be deployed to support energy system transformation required to achieve climate goals; and (3) incentivizes those forest bioenergy systems that augment the mitigation value of the forest sector as a whole. Emphasis on short‐term emissions reduction targets can lead to decisions that make medium‐ to long‐term climate goals more difficult to achieve. The most important climate change mitigation measure is the transformation of energy, industry and transport systems so that fossil carbon remains underground. Narrow perspectives obscure the significant role that bioenergy can play by displacing fossil fuels now, and supporting energy system transition. Greater transparency and consistency is needed in greenhouse gas reporting and accounting related to bioenergy.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Reliability-Informed Life Cycle Warranty Cost and Life Cycle Analysis of Newly Manufactured and Remanufactured Units

Standard life cycle techniques such as life cycle warranty cost (LCWC) analysis and life cycle analysis (LCA) are used to respectively quantify the relative economical and environmental advantages of remanufactured goods while simultaneously identifying avenues for improvement. In this paper, we contribute to the literature on life cycle studies by incorporating reliability into LCWC analysis and LCA with the goal of improving long-term/multiple life cycle decision making. We develop a branched power-law model to incorporate the physical degradation mechanisms leading to reduced reuse rates of system parts over multiple life cycles. We then follow a standard LCA protocol to quantify the difference between a new unit and its remanufactured version in terms of environmental impact items such as abiotic depletion potential, global warming potential, and energy consumption. We then devise four practical warranty policies that vary in the choice of replacement and/or provision for extended warranty. All possible replacement scenarios for multiple life cycles are explored for each policy and a mathematically rigorous framework is provided, where the reliability information is used to calculate probabilistic LCWC and life cycle impact items. This reliability-informed LCWC analysis and LCA framework enables design engineers to compare design options and warranty policies by quantifying both economical and environmental impacts to aid in decision making. Although the framework is presented in a general form applicable to any engineered system, we demonstrate the utility of this framework by using a case study of an infinitely variable transmission used in agricultural equipment.

42 ENGINEERING↗
DOE OSTI2020

The relatively young and rural population may limit the spread and severity of COVID-19 in Africa: a modelling study

A novel coronavirus disease 2019 (COVID-19) has spread to all regions of the world. There is great uncertainty regarding how countries’ characteristics will affect the spread of the epidemic; to date, there are few studies that attempt to predict the spread of the epidemic in African countries. In this paper, we investigate the role of demographic patterns, urbanisation and comorbidities on the possible trajectories of COVID-19 in Ghana, Kenya and Senegal. We use an augmented deterministic Susceptible-Infected-Recovered model to predict the true spread of the disease, under the containment measures taken so far. We disaggregate the infected compartment into asymptomatic, mildly symptomatic and severely symptomatic to match observed clinical development of COVID-19. We also account for age structures, urbanisation and comorbidities (HIV, tuberculosis, anaemia). In our baseline model, we project that the peak of active cases will occur in July, subject to the effectiveness of policy measures. When accounting for the urbanisation, and factoring in comorbidities, the peak may occur between 2 June and 17 June (Ghana), 22 July and 29 August (Kenya) and, finally, 28 May and 15 June (Senegal). Successful containment policies could lead to lower rates of severe infections. While most cases will be mild, we project in the absence of policies further containing the spread, that between 0.78% and 1.03%, 0.61% and 1.22%, and 0.60% and 0.84% of individuals in Ghana, Kenya and Senegal, respectively, may develop severe symptoms at the time of the peak of the epidemic. Compared with Europe, Africa’s younger and rural population may modify the severity of the epidemic. The large youth population may lead to more infections but most of these infections will be asymptomatic or mild, and will probably go undetected. The higher prevalence of underlying conditions must be considered.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2024

Drug providers’ perspectives on antibiotic misuse practices in eastern Ethiopia: a qualitative study

Objective Antibiotic misuse includes using them to treat colds and influenza, obtaining them without a prescription, not finishing the prescribed course and sharing them with others. Although drug providers are well positioned to advise clients on proper stewardship practices, antibiotic misuse continues to rise in Ethiopia. It necessitates an understanding of why drug providers failed to limit such risky behaviours. This study aimed to explore drug providers’ perspectives on antibiotic misuse practices in eastern Ethiopia. Setting The study was conducted in rural Haramaya district and Harar town, eastern Ethiopia. Design and participants An exploratory qualitative study was undertaken between March and June 2023, among the 15 drug providers. In-depth interviews were conducted using pilot-tested, semistructured questions. The interviews were transcribed verbatim, translated into English and analysed thematically. The analyses considered the entire dataset and field notes. Results The study identified self-medication pressures, non-prescribed dispensing motives, insufficient regulatory functions and a lack of specific antibiotic use policy as the key contributors to antibiotic misuse. We found previous usage experience, a desire to avoid extra costs and a lack of essential diagnostics and antibiotics in public institutions as the key drivers of non-prescribed antibiotic access from private drug suppliers. Non-prescribed antibiotic dispensing in pharmacies was driven by client satisfaction, financial gain, business survival and market competition from informal sellers. Antibiotic misuse in the setting has also been linked to traditional and ineffective dispensing audits, inadequate regulatory oversights and policy gaps. Conclusion This study highlights profits and oversimplified access to antibiotics as the main motivations for their misuse. It also identifies the traditional antibiotic dispensing audit as an inefficient regulatory operation. Hence, enforcing specific antibiotic usage policy guidance that entails an automated practice audit, a responsible office and insurance coverage for persons with financial limitations can help optimise antibiotic use while reducing resistance consequences.

General & Internal Medicine↗
DOE OSTI2021

Distributionally Robust Partially Observable Markov Decision Process with Moment-Based Ambiguity

In this paper, we consider a distributionally robust partially observable Markov decision process (DR-POMDP), where the distribution of the transition-observation probabilities is unknown at the beginning of each decision period, but their realizations can be inferred using side information at the end of each period after an action being taken. We build an ambiguity set of the joint distribution using bounded moments via conic constraints and seek an optimal policy to maximize the worst-case (minimum) reward for any distribution in the set. We show that the value function of DR-POMDP is piecewise linear convex with respect to the belief state and propose a heuristic search value iteration method for obtaining lower and upper bounds of the value function. We conduct numerical studies and demonstrate the computational performance of our approach via testing instances of a dynamic epidemic control problem. Our results show that DR-POMDP can produce more robust policies under misspecified distributions of transition-observation probabilities as compared to POMDP but has less costly solutions than robust POMDP. The DR-POMDP policies are also insensitive to varying parameter in the ambiguity set and to noise added to the true transition-observation probability values obtained at the end of each decision period.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

SchedInspector: A Batch Job Scheduling Inspector Using Reinforcement Learning

Improving the performance of job executions is an important goal of HPC batch job schedulers, such as minimizing job waiting time, slowdown, or completion time. Such a goal is often accomplished using carefully designed heuristics based on job features, such as job size and job duration. However, these heuristics overlook important runtime factors (e.g., cluster availability and waiting job patterns), which may vary across time and make a previously sound scheduling decision not hold any longer. In this study, we propose a new approach to incorporate runtime factors into batch job scheduling for better job execution performance. The key idea is to add a scheduling inspector on top of the base job scheduler to scrutinize its scheduling decisions. The inspector will take the runtime factors into consideration and accordingly determine the fitness of the scheduled job. It then either accepts the scheduled job or rejects it and asks the base schedulers to try again later. We realize such an inspector, namely SchedInspector, by leveraging the intelligence of reinforcement learning. Through extensive experiments, we show SchedInspector can intelligently integrate the runtime factors into various batch job scheduling policies, including the state-of-the-art one, to gain better job execution performance, such as smaller average bounded job slowdown (up to 69% better) or average job waiting time (up to 52% better), across various real-world workloads. We also show that although rejecting scheduling decisions may leave the resources idle hence affect the system utilization, SchedInspector is able to achieve the job execution performance improvement with marginal impact on the system utilization (typically less than 1%). We consider one key advantage of SchedInspector is it automatically learns to work with and improve existing job scheduling policies without changing them, which makes it promising to serve as a generic enhancer for various batch job scheduling policies.

Zhang, Di↗
DOE OSTI2024

Virtual Time III, Part 3: Throttling and Message Cancellation

This is Part 3 of a trio of papers that unify in a natural way the two historically distinct parallel discrete event synchronization paradigms, optimistic and conservative, combining the best properties of both into a single framework called Unified Virtual Time (UVT). In this part, we survey the synchronization effects that can be achieved by restricting to corner cases the relationships permitted among the control variables, GVT, CVT, TVT, and LVT, which were defined in Part 1. Here we also survey various throttling policies from the literature and describe how they can be implemented in UVT by controlling the value of TVT, including policies that can take advantage of rollback in addition to LP blocking. A significant result is a new category of efficient and higher precision throttling algorithms for optimistic execution that are based on optimistic lookahead, defined in a way that is symmetric to what we now call the conservative lookahead information that is traditionally used for conservative synchronization. Finally, we present a novel algorithm allowing the choice between lazy and aggressive cancellation to be made on a message-by-message basis using either external logic expressed in the model code, or policy code internal to the simulator, or a mixture of both.

throttling↗
DOE OSTI2022

A Right Transfer Access Control Model of Internet of Things Based on Smart Contract

Sensor nodes play a crucial role in the promotion of development of Internet of Things (IoT). Through this transaction, RO defines access control policies in script form based on ABAC's access control model to grant access right. The identity of all users in the model is identified by address. This paper builds a more flexible right transfer access control model by means of combining the Attribute-Based Access Control model (ABAC) and blockchain technology. Owing to the characteristics of ABAC’s attributes and right association, the massive problems of some sensor nodes can be solved. At the same time, for the sake of addressing the dynamic problems such as node access and right transfer, right transfer contract (TS) and access control contract (CS) are employed on the chain to ensure efficient and safe transmission of rights. To solve on-chain storage problems and ensure transparency of the operation, the idea of Rollup in Ethereum expansion is used to upload the final state of protocol policy and right exchange to the chain. Any user can know the policy and current right transfer status at any time. Finally, comparative and security analysis show that the model presented here can solve IoT devices’ massive and dynamic problems more effectively and it is more secure than the traditional models.

Wang, Jiuru↗
DOE OSTI2023

PoliMOR

PoliMOR is a scalable, automated, and customizable policy engine framework for multi-tiered parallel file systems. It is composed of single-purpose agents that handle tasks such as gathering file metadata, making policy decisions, and then executing actions based on those policies. These agents are designed to communicate using distributed message queues, allowing the number of individual agents to be scaled up as needed. PoliMOR automates the data management tasks by precluding the need for admin intervention. The agents in PoliMOR can be customized to integrate any utilities/tools that perform tasks like metadata scanning and data placement management.

Brumgard, Christopher↗
DOE OSTI2026

Dynamic Transmission Line Switching Amid Wildfire-Prone Weather Under Decision-Dependent Uncertainty

During dry and windy seasons, environmental conditions significantly increase the risk of wildfires, exposing power grids to disruptions caused by transmission line failures. Wildfire propagation exacerbates grid vulnerability, potentially leading to prolonged power outages. To address this challenge, we propose a multistage optimization model that dynamically adjusts transmission grid topology in response to wildfire propagation, aiming to develop an optimal response policy. By accounting for decision-dependent uncertainty, where line survival probabilities depend on usage, we employ distributionally robust optimization to model uncertainty in line survival distributions. We adapt the stochastic nested decomposition algorithm and derive a deterministic upper bound for its finite convergence. To enhance computational efficiency, we exploit the Lagrangian dual problem structure for a faster generation of Lagrangian cuts. Using realistic data from the California transmission grid, we demonstrate the superior performance of dynamic response policies against two-stage alternatives through a comprehensive case study. In addition, after solving the multistage formulation, we construct easy-to-implement policies that significantly reduce computational burden while maintaining good performance in real-time deployment. History: Accepted by Russell Bent, Area Editor for Network Optimization: Algorithms and Applications. Funding: This work was supported by the U.S. Department of Energy, Office of Electricity [Grant DE-AC02-05CH11231]. The work of R. Jiang was supported in part by the U.S. National Science Foundation, Division of Electrical, Communications and Cyber Systems [Grant ECCS-1845980] and the U.S. Air Force Office of Scientific Research [Grant FA9550-23-1-0323]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2025.1210 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2025.1210 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

Estrada-Garcia, Juan-Alberto↗