Search NASA⌕ Search

SEARCH · Search NASA

Results for “policy framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning

In multi-timescale multi-agent reinforcement learning (MARL), agents interact across different timescales. In general, policies for time-dependent behaviors, such as those induced by multiple timescales, are non-stationary. Learning non-stationary policies is challenging and typically requires sophisticated or inefficient algorithms. Motivated by the prevalence of this control problem in real-world complex systems, we introduce a simple framework for learning non-stationary policies for multi-timescale MARL. Our approach uses available information about agent timescales to define and learn periodic multi-agent policies. In detail, we theoretically demonstrate that the effects of non-stationarity introduced by multiple timescales can be learned by a periodic multi-agent policy. To learn such policies, we propose a policy gradient algorithm that parameterizes the actor and critic with phase-functioned neural networks, which provide an inductive bias for periodicity. The framework's ability to effectively learn multi-timescale policies is validated on a gridworld and building energy management environment.

control↗
DOE OSTI2026

A Physics-Informed Reinforcement Learning Framework for Economic-Thermal Co-Optimization of Crypto Mining Data Centers: Preprint

The rapid expansion of cryptocurrency mining has created a new class of high-density data centers characterized by extreme thermal flux and high sensitivity to volatile economic markets. Traditional thermal management strategies, typically reliant on rule-based control, maintain static setpoints that fail to account for fluctuating electricity prices and cryptocurrency values - factors critical to mining profitability. To address this, we present a physics-informed reinforcement learning (PIRL) framework for economic-thermal co-optimization in crypto mining data centers. This framework consists of a proximal policy optimization (PPO) agent, a virtual testbed powered by high-fidelity physics-based models, and an interactive frontend dashboard. The PPO agent is trained using the virtual testbed and strict hardware safety limits. This physics-informed approach allows the agent to learn a stochastic policy that dynamically balances mining revenue against operational costs by co-optimizing HVAC cooling setpoints and IT computational hashrate. The simulation results demonstrate that the integrated framework achieved an 8.62% increase in net operational profit compared to traditional baseline strategies while strictly adhering to safety-critical temperature constraints (coolant supply temperature < 32 degrees C). This work provides a scalable template for the deployment of reinforcement learning in mission critical facilities where economic volatility and physical safety must be managed simultaneously.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Incorporating Operational Uncertainties into the Dispatch of an Integrated Solar and Storage System

The economic assessment of hybrid energy systems (HES) pairing battery energy storage systems (BESSs) and photovoltaics (PV) is highly important for advancing their deployment in power systems. This paper presents an innovative assessment framework, including an optimal control policy for dispatch under uncertainty and procedures for exploring control parameters that maximize economic benefits. The proposed dispatch policy consists of two steps using system forecast information. The first step is to determine whether a BESS will be used within an operational scheduling time frame based on the probability of events and their thresholds. Once the dispatch of BESS is triggered, a model predictive control (MPC) is carried out in the second step for scheduling using the expected value of system information. By exercising this policy with different thresholds, one can explore the trade-offs between short-term benefits and battery lifetime, and identify an optimal threshold that maximizes the total economic benefits within the battery lifetime. An evaluation study in a real-world HES project is presented to illustrate the proposed framework. Compared with traditional optimal dispatch algorithms, the proposed method can significantly improve the economic benefits of an HES scheduled under forecast uncertainties.

Ma, Xu↗
DOE OSTI2023

Illuminating Common Ground: Success Factors for Contiguous US Tribal Solar Energy

Tribal energy development is a complex multi-faceted topic. The objective of this work is to identify common themes across tribal solar energy deployment projects, focusing specifically on lessons learned and recommendations. Identifying these commonalities and learning from the experiences of tribes that have embarked in energy development efforts can help to inform the development of future tribal solar energy projects. A thematic qualitative analysis approach was used to analyze project reports and presentations for 41 tribal solar deployment projects (only within the contiguous United States) funded by the Office of Indian Energy Policy and Programs, applying a framework of success factors developed from the literature. The results of the qualitative analysis are described in four discrete parts: comprehensive and inclusive planning, fostering partnerships and collaboration, building capacity, and exercising and advancing tribal sovereignty. Each of the overarching themes inform recommendations for tribes to promote the success of solar projects.

14 SOLAR ENERGY↗
DOE OSTI2026

SENTRA: A Modular Computational Graph Framework for Critical Mineral and Materials Supply Chains: Part I: Network Construction Latent-Quantity Estimation, and Temporal Graph Forecasting

Global supply chains for critical minerals and materials are complex, evolving networks of countries, products, production stages, and trade relationships. Existing analytical approaches are limited by fragmented data and static network representations that do not capture the dynamic production dependencies linking raw materials, intermediate products, and final goods across multiple countries. Trade and production statistics provide only a partial view of domestic production, inventories, and material flows, making it difficult to identify indirect sourcing pathways, hidden dependencies, and embedded foreign exposures. This paper introduces the Supply Chain Exposure Network Tracking and Risk Assessment (SENTRA) framework, a modular graph-based computational framework for constructing, analyzing, and forecasting dynamic supply chain networks. As the first paper in a three-part methodological series, it establishes the computational foundation of SENTRA by constructing a temporal attributed multi-relational graph whose nodes represent product–country pairs and whose edges encode observed trade and within-country value-chain relationships. Statistical estimation and constrained optimization recover latent production, final demand, and product input dependency coefficients while enforcing economic accounting constraints. Graph-derived exposure measures quantify direct, transshipment, value-chain, and multi-hop supply chain dependencies independently of the forecasting model. A temporal graph forecasting architecture based on a relational graph neural network then forecasts the evolution of the graph under mass-balance constraints with distribution-free conformal uncertainty quantification. Validation on the global aluminum supply chain shows that the learned graph representations recover economically meaningful supply chain structure, accurately forecast out-of-sample trade relationships, and produce well-calibrated prediction intervals. Subsequent papers apply this computational foundation to exposure assessment, disruption analysis, and scenario-based policy analysis, and extend the framework to multimaterial supply chain modeling and decision support.

36 MATERIALS SCIENCE↗
DOE OSTI2023

IRIS-DMEM: Efficient Memory Management for Heterogeneous Computing

This paper proposes an efficient data memory management approach for the Intelligent RuntIme System (IRIS) heterogeneous computing framework along with new data transfer policies. IRIS provides a task-based programming model for extreme heterogeneous computing (e.g., CPU, GPU, DSP, FPGA) with support for today's most important programming languages (e.g., OpenMP, OpenCL, CUDA, HIP, OpenACC). However, the IRIS framework either forces the programmer to introduce data transfer commands for each task or relies on suboptimal memory management for automatic and transparent data transfers. The work described here extends IRIS with novel heterogeneous memory handling and introduces novel data transfer policies by employing the Distributed data MEMory handler (DMEM) for efficient and optimal movement of data among the various computing resources. The proposed approach achieves performance gains of up to 7× for tiled LU factorization and tiled DGEMM (i.e., matrix multiplication) benchmarks. Moreover, this approach also reduces data transfers by up to 71% when compared to previous IRIS heterogeneous memory management handlers. This work compares the performance results of the IRIS framework's novel DMEM with the StarPU runtime and MAGMA math library for GPUs. Experiments show a performance gain of up to 1.95× over StarPU and 2.1× over MAGMA.

Miniskar, Narasinga Rao↗
DOE OSTI2023

Assessing Impacts of Environmental Perturbations on Urban Biogenic Carbon Exchange in the Chicago Region

Carbon dioxide (CO 2 ) quantification is critical for assessing city-level carbon emissions and sustainable urban development. While urban vegetation has the potential to provide environmental benefits, such as heat and carbon mitigation, the CO 2 exchange from biogenic sectors and its impact from the environmental perturbations are often overlooked. It is also challenging to simulate the plant functions in the complex urban terrain. This study presents a processed-based modeling approach to assess the biogenic carbon fluxes from the vegetated areas over the Chicago Metropolitan Area (CMA) using the Weather Research and Forecast—Urban Biogenic Carbon exchange model. We investigate the change of CO 2 sink power in CMA under heatwaves and irrigation. The results indicate that the vegetation plays a significant role in the city's carbon portfolio and the landscaping management has the potential to reduce carbon emissions significantly. Furthermore, based on the competing mechanisms in the biogenic carbon balance identified in this study, we develop a novel Environmental Benefit Score metrics framework to identify the vulnerability and mitigation measures associated with nature-based solutions (NbS) within CMA. By using the generalized portable framework and our science-policy confluence analysis presented in this study, global cities can maximize the effectiveness of NbS and accelerate carbon neutrality.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Optimal Coordination of Electric Vehicles for Grid Services using Deep Reinforcement Learning

Recent research has shown the effectiveness of reinforcement learning (RL) in coordinating electric vehicles (EVs) with vehicle-to-grid capabilities for grid services. However, many of these studies rely on lookup table and deep Q-network techniques, which can be impractical when dealing with continuous states and actions. In addition, existing RL designs inadequately account for battery aging effects, EV user satisfaction, uncertain departure and arrival time, and trip distance, which may compromise effective coordination. This paper aims to bridge these gaps by developing an innovative deep deterministic policy gradient-based RL framework for optimal coordination of EVs. Case studies were carried out using a test system with 100 EVs, and numerical analysis results showed that the proposed RL framework can effectively coordinate EVs to maximize economic benefits and user satisfaction while ensuring the expected battery lifespan.

Das, Avijit↗
DOE OSTI2020

A metric framework for evaluating the resilience contribution of hydropower to the grid

This paper will describe a proposed framework for expressing the resilience of hydropower generation and provide initial case studies for three classes of hydropower, run-of-river hydropower, hydropower with reservoirs, and pumped storage hydropower. Hydropower has great flexibility to provide support during and after natural and man-made events that can disrupt critical infrastructure functionality. The concept of the framework provides for consideration of policy and rules, constraints of the water shed and other allocations of water, storage and plant capabilities to produce real and reactive power, and the strength of the delivery network. The paper details a resilience response metric that has inputs of state of storage and plant level constraints on real and reactive power production. Using the definition of resilience, based on maintaining a minimally normal operations, we provide a qualitative assessment of hydropower’s ability to address the various time scales comprising the “R”s of resilience.

13 HYDRO ENERGY↗
DOE OSTI2025

Threat Hunt Guide for BESS Environments

The rapid digitalization of the electric grid - driven by the integration of inverter-based resources (IBRs), battery energy storage systems (BESS), and advanced grid control platforms - has significantly enhanced grid efficiency, visibility, and flexibility. However, this evolution also introduces new cybersecurity risks, particularly through supply chain dependencies and operational blind spots at the grid edge. To address these challenges, Idaho National Laboratory (INL), through the Department of Energy (DOE) Office of Cybersecurity, Energy Security, and Emergency Response (CESER) Rapid Risk initiative, conducted a series of rapid risk assessment engagements with energy organizations across the United States. Drawing on lessons learned from these engagements, INL developed the following threat hunting guide for asset owners and operators (AOOs) to enhance their cybersecurity visibility within BESS and IBR systems. The guide demonstrates how to use passive network monitoring to baseline device behavior, detect adversarial activity, and investigate anomalies without disrupting operations. By implementing these practices, energy sector stakeholders can improve coordination between cybersecurity and operations teams and strengthen the resilience of distributed energy resources (DERs) within the modern power grid. Prior to implementing any network monitoring, packet capture, or threat hunting activity described in this guide, AOOs are strongly advised to review applicable governance frameworks, legal requirements, and organizational policies. This guide is intended for informational and educational purposes only. It does not replace compliance with any federal, state, or local cybersecurity mandates or industry standards. Implementation of described configurations, technologies, or analytic workflows is performed at the discretion and responsibility of the asset owner and operator.

25 - ENERGY STORAGE↗
NASA NTRS1977

Assessing the future of air freight

The role of air cargo in the current transportation system in the United States is explored. Methods for assessing the future role of this mode of transportation include the use of continuous-time recursive systems modeling for the simulation of different components of the air freight system, as well as for the development of alternative future scenarios which may result from different policy actions. A basic conceptual framework for conducting such a dynamic simulation is presented within the context of the air freight industry. Some research needs are identified and recommended for further research. The benefits, limitations, pitfalls, and problems usually associated with large scale systems models are examined.

Dajani, J. S.↗
DOE OSTI2023

Efficient learning of power grid voltage control strategies via model-based deep reinforcement learning

Here this article proposes a model-based deep reinforcement learning (DRL) method to design emergency control strategies for short-term voltage stability problems in power systems. Recent advances show promising results for model-free DRL-based methods in power systems control problems. But in power systems applications, these model-free methods have certain issues related to training time (clock time) and sample efficiency; both are critical for making state-of-the-art DRL algorithms practically applicable. DRL-agent learns an optimal policy via a trial-and-error method while interacting with the real-world environment. It is also desirable to minimize the direct interaction of the DRL agent with the real-world power grid due to its safety-critical nature. Additionally, the state-of-the-art DRL-based policies are mostly trained using a physics-based grid simulator where dynamic simulation is computationally intensive, lowering the training efficiency. We propose a novel model-based DRL framework where a deep neural network (DNN)-based dynamic surrogate model (SM), instead of a real-world power grid or physics-based simulation, is utilized within the policy learning framework, making the process faster and more sample efficient. However, having stable training in model-based DRL is challenging because of the complex system dynamics of large-scale power systems. We addressed these issues by incorporating imitation learning to have a warm start in policy learning, reward-shaping, and multi-step loss in surrogate model training. Finally, we achieved 97.5% reduction in samples and 87.7% reduction in training time for an application to the IEEE 300-bus test system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

A Framework for Non-Pipeline Alternatives Analysis and Review of Existing Approaches

States are increasingly interested in gas distribution system planning and the role that demand-side resources can play in maintaining an affordable, reliable, equitable, and low-carbon energy system. In particular, public utility commissions and utilities are exploring the role of non-pipeline alternatives (NPA) – an investment or activity that defers, reduces, or avoids the need to construct or replace a pipeline – in gas distribution system planning. NPAs are an emerging cost and risk mitigation tool that can provide gas utilities with an opportunity to reduce emissions, gas system costs, and customer risk by avoiding unnecessary infrastructure spending. Rather than address system issues with more costly and long-lived traditional capital projects, utilities can leverage demand-side NPA resources such as energy efficiency and electrification as well as supply-side NPA resources to meet system needs. However, there is limited guidance on performing an NPA and even fewer lessons learned. To fill this gap, the U.S. Department of Energy funded two reports on NPAs. The first report is a literature review that focuses on four states' NPA policies. The second report offers a framework for evaluating NPAs that states can adapt to meet their policy needs.

03 NATURAL GAS↗
DOE OSTI2022

Community Solar Policy Screening Workbook for Multifamily Affordable Housing Building Portfolios

This workbook provides a framework to help Multifamily Affordable Housing (MFAH) housing providers evaluate state-level policy landscapes as they relate to MFAH community solar development. By exploring which programs and policies aid in the technical and economic feasibility of MFAH solar projects, building owners can gain an understanding of where in their portfolios to begin developing solar. This workbook also walks through the steps necessary for MFAH building owners to assess their portfolios using this framework. It was developed by the National Renewable Energy Laboratory for the National Community Solar Partnership (NCSP) Multifamily Affordable Housing Collaborative. Learn more about the NCSP and the Collaborative at https://www.energy.gov/communitysolar/community-solar.

14 SOLAR ENERGY↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗