Search NASA⌕ Search

SEARCH · Search NASA

Results for “policy driven computation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Integration of scanning probe microscope with high-performance computing: Fixed-policy and reward-driven workflows implementation

The rapid development of computation power and machine learning algorithms has paved the way for automating scientific discovery with a scanning probe microscope (SPM). The key elements toward operationalization of the automated SPM are the interface to enable SPM control from Python codes, availability of high computing power, and development of workflows for scientific discovery. Here, we build a Python interface library that enables controlling an SPM from either a local computer or a remote high-performance computer, which satisfies the high computation power need of machine learning algorithms in autonomous workflows. We further introduce a general platform to abstract the operations of SPM in scientific discovery into fixed-policy or reward-driven workflows. Furthermore, our work provides a full infrastructure to build automated SPM workflows for both routine operations and autonomous scientific discovery with machine learning.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

FIRE: A Failure-Adaptive RL Framework for Edge Computing Migrations

In edge computing, users' service profiles are migrated between edge servers due to user mobility. Reinforcement Learning (RL) frameworks have been proposed to do so, often trained on simulated data. However, existing RL frameworks overlook occasional server failures, which although rare, impact latency-sensitive applications like AR/VR and real- time obstacle detection. These rare failures, being not adequately represented in historical training data, pose a challenge for data-driven RL algorithms. We introduce FIRE, a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment. We propose FIRE-ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function. FIRE considers delay, migration, failure, and backup placement costs across individual and shared service profiles. We prove FIRE-ImRE's boundedness and convergence to optimality. Next, we introduce novel deep Q-learning (FIRE-ImDQL) and actor critic (FIRE-ImACRE) versions of our algorithm to enhance scalability. Here, we extend our framework to accommodate users with varying risk tolerances of rare failure events. Through trace-driven experiments, we show that FIRE reduces edge computing costs compared to vanilla RL and the greedy baseline in the event of failures.

Edge computing↗
DOE OSTI2025

Evaluating HPC Scheduling Strategies for Urgent Workloads

Scientific computing centers increasingly face workloads with diverse urgency requirements, driven by applications that demand rapid or even immediate execution. Appropriately configured scheduling policies can significantly improve both user satisfaction and overall cluster utilization. In this work, we present a systematic analysis of scheduler configurations under scenarios where a fraction of jobs have urgent computing needs. We evaluate multiple job scheduling simulators, develop a lightweight job-submission emulation framework, and create tools to analyze and visualize the resulting scheduling data. Our study identifies key trade-offs between responsiveness, fairness, and efficiency, and offers a set of practical scheduling configurations (particularly for Slurm) that can be tailored to HPC environments supporting mixed-urgency workloads.

Maheshwari, Ketan [ORNL] (ORCID:000000033800662X)↗
DOE OSTI2022

Koopman-based Differentiable Predictive Control for the Dynamics-Aware Economic Dispatch Problem

The dynamics-aware economic dispatch (DED) problem embeds low-level generator dynamics and operational constraints to enable near real-time scheduling of generation units in a power network. DED produces a more dynamic supervisory control policy than traditional economic dispatch (T-ED) that reduces overall generation costs. However, the incorporation of differential equations that govern the system dynamics makes DED an optimization problem that is computationally prohibitive to solve. In this work, we present a new data-driven approach based on differentiable programming to efficiently obtain offline parametric solutions to the underlying DED problem. In particular, we employ the recently proposed differentiable predictive control (DPC) for offline learning of explicit neural control policies based on identified Koopman operator (KO) model of the system dynamics. We demonstrate the high solution quality and five orders of magnitude computational-time savings of the DPC method over the original optimization-based DED approach on a 9-bus test power grid network.

King, Ethan↗
DOE OSTI2022

Sequential Decision Making (SDM) for Mesh Refinement and Model Selection in Multiscale, Multi-Physics Applications

Intelligent automation and decision support are needed to enhance computational efficiency and robustness in multiscale and multi-physics problems, including materials science, manufacturing, and climate and weather modeling. Current scientific computing approaches for enabling decisions by scientists fail to explore the role of learning, reasoning, and probabilistic planning. Often these decisions are not performed in real-time during the computation but are made prior to the start of the computation, which must be interrupted in order to make changes to the prior choices. Such interruptions at different stages of the computation increase the total computing time and the need for a human expert to frequently monitor the results. State of art scientific computing methods consist of rule-based algorithms that cannot automatically adapt to a dynamically changing computing environment. The development of a Sequential Decision Making (SDM) framework will automate scientific computing by optimizing the policies for mesh refinement, time-stepping, model and algorithm selection, resource allocation, and pre and post-processing. Our agent SDM framework for scientific computing will consist of data-driven learning (Classifier), automated reasoning (contextual knowledge), and probabilistic planning (Reinforcement Learning). In this project, we focused on three problems to demonstrate our SDM framework on a set of ordinary and partial differential equations. Classification of Lorenz system regions using Feed-Forward Neural Networks examined learning in the SDM framework. On the other hand, reasoning and planning in the SDM framework were used in two problems: adaptive time-stepping for nonlinear ODEs using on-policy RL algorithms, and adaptive mesh refinement for 2-D PDEs using off-policy RL algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Challenges and Opportunities in Magnetospheric Space Weather Prediction

Abstract Space Weather is the study of the dynamics of the coupled Solar‐Terrestrial environment, as these dynamics impact technological systems and human activity. This paper reviews a selection of the advances, challenges, and new opportunities for magnetospheric space weather, and while the focus is on specific phenomena, many other aspects of space weather will have similar challenges and needs. As a scientific field with direct applications, the field of space weather is partly driven by imperatives from both policy and operations. We provide an introduction to some of the context in which the field exists and discuss how this might shape future developments and norms within the space weather enterprise. We briefly examine benchmarking, as a policy and operationally driven activity, as it provides immediate societal relevance and an opportunity to stretch scientific understanding. As numerical space weather prediction now becomes routine, and exascale computing is in the near future, we identify challenges relating to computational expense and big data, capturing and accounting for uncertainties, and specification of boundary conditions. Here, as with the observations supporting numerical space weather prediction, the key challenge lies in extending the lead time of predictions. We also discuss the role of data, particularly in regard to model validation and empirical modeling. Due to the growing societal impact of space weather, we also examine the relationships between space weather and its terrestrial counterpart and look at the importance of continuous evaluation, monitoring progress in predictive capability, and communication with researchers, forecasters, and end users.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Towards Sustainable Post-Exascale Leadership Computing

As computing systems approach the limits of traditional silicon technology, the diminishing returns in performance per watt present a significant barrier to sustaining growth in HPC. From a large-scale scientific supercomputing facility point of view, we propose a multifaceted strategy toward specialized hardware and architectures that are optimized for energy efficiency in specific applications. We also emphasize the need for integrating energy-aware practices across all levels of HPC, from system design and software development to operational policies. We discuss strategic opportunities such as the adoption of application-specific accelerators, the development of energy-efficient algorithms, and the implementation of data-driven operational analytics. Our goal is to develop a comprehensive roadmap ensuring that future leadership systems at OLCF can meet scientific demands while operating within stringent energy budgets, thereby supporting sustainable computing growth.

Shin, Woong↗
DOE OSTI2018

Runtime Scheduling Policies for Distributed Graph Algorithms

In this paper we explore scheduling and runtime system support for unordered distributed graph computations that rely on optimistic (speculative) execution. Performance of such algorithms is impacted by two competing trends: the higher degree of parallelism enabled by optimistic execution in turn requires substantial runtime support. To address the potentially high overhead and scheduling complexity introduced by the runtime, we investigate customizable scheduling policies that augment the scheduler of the underlying runtime to adapt it to a specific graph application. We present several implementations of Distributed Control (DC), a data-driven unordered approach with work prioritization and demonstrate that customizable scheduling policies result in the most efficient implementation, outperforming the well-known -stepping Single-Source Shortest Paths (SSSP) and Jones-Plassmann vertex-coloring algorithms. We apply two scheduling techniques, flow control and adaptive frequency of network progress, which allow application-level control over the balance of domain work and the runtime work. Experimental results show the benefit of such application-aware scheduling for irregular distributed graph algorithms.

Firoz, Jesun S.↗
DOE OSTI2024

Toward Accelerating Discovery via Physics-Driven and Interactive Multifidelity Bayesian Optimization

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and often nondifferentiable parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, processing spaces, and molecular embedding spaces. Often these systems are expensive or time consuming to evaluate a single instance, and hence classical approaches based on exhaustive grid or random search are too data intensive. This resulted in strong interest toward active learning methods such as Bayesian optimization (BO) where the adaptive exploration occurs based on human learning (discovery) objective. However, classical BO is based on a predefined optimization target, and policies balancing exploration and exploitation are purely data driven. In practical settings, the domain expert can pose prior knowledge of the system in the form of partially known physics laws and exploration policies often vary during the experiment. Here, we propose an interactive workflow building on multifidelity BO (MFBO), starting with classical (data-driven) MFBO, then expand to a proposed structured (physics-driven) structured MFBO (sMFBO), and finally extend it to allow human-in-the-loop interactive interactive MFBO (iMFBO) workflows for adaptive and domain expert aligned exploration. These approaches are demonstrated over highly nonsmooth multifidelity simulation data generated from an Ising model, considering spin–spin interaction as parameter space, lattice sizes as fidelity spaces, and the objective as maximizing heat capacity. Detailed analysis and comparison show the impact of physics knowledge injection and real-time human decisions for improved exploration with increased alignment to ground truth. Here, the associated notebooks allow to reproduce the reported analyses and apply them to other systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bi-Level Adaptive Storage Expansion Strategy for Microgrids Using Deep Reinforcement Learning

Battery energy storage (BES) is a versatile resource for the secure and economic operation of microgrids (MGs). Prevailing stochastic optimization-based approaches for BES expansion planning for MGs are computationally complicated. This work proposes a data-driven bi-level multi-period BES expansion planning framework to determine the siting, sizing, and timing of BES installations. The proposed planning framework unifies deep reinforcement learning (DRL) and linear programming, thereby decoupling the determinations for the integer and continuous decision variables in two time scales, respectively. In the upper level, a rainbow DRL agent with quantile regression is trained to provide dynamic planning policies to accommodate stochastic renewable energy resources (RESs), load, and battery price changes efficiently. Further, the lower level computes the optimal operation of MGs with frequency constraints to hedge the islanding contingency. The two levels communicate with one another by exchanging storage configuration and operating expenses in order to accomplish the shared goal of minimizing investment and operation costs. Comparative case studies on an MG are carried out to demonstrate the superiority of the proposed DRL-based solution to the mixed-integer linear programming counterpart on efficiency, scalability, and adaptability.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Uncertainty Quantification for Capacity Expansion Planning

This report quantifies the uncertainty in output decisions from a Capacity Expansion Planning (CEP) model. The need to understand how uncertainties within CEP models and modeling assumptions affect Quantities of Interest (QoIs) such as expansion and operating costs, as well as expansion decisions remains an ongoing challenge in scientific research and industrial operations. This area of research is particularly important for models which seek to capture how large networks will evolve and operate under increased sources of variable generation, i.e., higher penetration of renewable technologies such as solar and wind generators. Uncertainty quantification (UQ) of CEP models which estimate expansion costs and decisions, and production cost models which estimate operating costs and dispatch decisions, is a key focus of research at NREL. The Regional Energy Deployment System (ReEDS) represents a state-of-the-art CEP model and considers a range of possible grid evolutions in an attempt to identify key drivers, ramifications, and decisions which contribute to better informed investment and policy decisions. However, research to quantify how uncertainties and model assumptions, such as unit commitment (UC), within ReEDS may be affecting its outputs remains challenging due to to size and complexity of the model

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Defense Technical Assistance - Infographic 4/24/25

Defense Technical Assistance (TA) Infographic designed to inform potential pilot partnerships with military installation about the TA opportunities available. - Secure and reliable energy is crucial for the operational readiness and resilience of military installations. The Department of Energy’s Grid Deployment Office has funded a new initiative at Idaho National Laboratory to conduct technical, on-site assessments that aim to secure the power infrastructure of military bases reliant on civilian utilities for their operations, both CONUS and OCONUS. Including how the assessments work and INL capabilities, partnership model as well as outcomes.

21 - SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLAN↗
DOE OSTI2026

Powered By SAM [Slides]

The System Advisor Model(TM) (SAM) is a free, open-source desktop application for techno-economic analysis of energy technologies. By combining detailed performance modeling with financial analysis, SAM allows users to assess technology trade-offs, explore future scenarios, and make informed decisions about energy investments. Users also have access to model details and the ability to embed SAM's core models in their own applications. This webinar, hosted by National Laboratory of the Rockies researchers Janine Keith and Matt Prilliman, highlights how this widely used modeling tool supports data-driven decision-making for energy systems.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Characterizing climate pathways using feature importance on echo state networks

The 2022 National Defense Strategy of the United States listed climate change as a serious threat to national security. Climate intervention methods, such as stratospheric aerosol injection, have been proposed as mitigation strategies, but the downstream effects of such actions on a complex climate system are not well understood. The development of algorithmic techniques for quantifying relationships between source and impact variables related to a climate event (i.e., a climate pathway) would help inform policy decisions. Data-driven deep learning models have become powerful tools for modeling highly nonlinear relationships and may provide a route to characterize climate variable relationships. In this paper, we explore the use of an echo state network (ESN) for characterizing climate pathways. ESNs are a computationally efficient neural network variation designed for temporal data, and recent work proposes ESNs as a useful tool for forecasting spatiotemporal climate data. However, ESNs are noninterpretable black-box models along with other neural networks. The lack of model transparency poses a hurdle for understanding variable relationships. We address this issue by developing feature importance methods for ESNs in the context of spatiotemporal data to quantify variable relationships captured by the model. We conduct a simulation study to assess and compare the feature importance techniques, and we demonstrate the approach on reanalysis climate data. In the climate application, we consider a time period that includes the 1991 volcanic eruption of Mount Pinatubo. This event was a significant stratospheric aerosol injection, which acts as a proxy for an anthropogenic stratospheric aerosol injection. Furthermore, we are able to use the proposed approach to characterize relationships between pathway variables associated with this event that agree with relationships previously identified by climate scientists.

black-box models↗
DOE OSTI2022

Multiscale Electricity Modeling for Evaluating Carbon Capture and Sequestration Technologies (MEME-CCS)

This effort employs and adapts a rigorous multiscale electricity modeling platform at the National Renewable Energy Laboratory (NREL) to evaluate carbon capture and sequestration (CCS) and negative emissions technologies (NET) from the ARPA-E FLECCS program. NREL's modeling platform includes the Regional Energy Deployment System (ReEDS) electric sector capacity expansion model, which projects future electricity generation mixes at sub-state-level resolution that are downscaled to the unit-level to enable hourly, zonal or nodal electricity production cost modeling in the PLEXOS model. The ReEDS-PLEXOS modeling suite is well-established for examining electric sector futures with high renewable energy penetrations, energy storage, electrification, and distributed generation. The key advancement proposed herein utilizes collaboration with CCS experts at the University of Wyoming (U.WY) and the FLECCS technology development teams to create innovative methods for representing CCS and NET in the ReEDS and PLEXOS models. Expanded technology options and new operational parameterizations are integrated into these models to allow an unprecedented combination of scope and resolution for exploring the future of CCS and NET. The resulting capabilities permit wide-ranging scenario analysis to assess CCS and NET deployment under alternative scenarios of CO2 prices and competitiveness of flexible CCS and NET technologies. We demonstrate sample deployment and operational outcomes to show how these models are being used to assess the future potential for FLECCS technologies and their impacts on the U.S. electricity system. These products will help an emerging CCS/NET industry in the United States by providing economics-driven guidance to technology developers while informing policy and investment decisions in the public and private sectors.

capacity expansion↗
DOE OSTI2018

High performance Data Driven Agent-based Modeling Framework for Simulation of Commute Mode Choices in Metropolitan Area

Active transportation, human-powered transportation modes such as walking and bicycling, not only reduces the carbon footprint from the transportation sector but also promotes healthy living by offering opportunities for people to build physical activity into their daily routine. To encourage active transportation through urban planning and public campaigns, it is of significant importance to infer factors that substantially influence commuters in their transportation mode choice process. This necessitates a flexible and repeatable tool that can evaluate how a policy is perceived by individual commuters and convert their decisions into macro level understanding. This paper introduces one such effort that is specifically designed for studies of transportation mode choices in metropolitan areas. It provides results from a high-resolution data driven simulation based on high performance computing implementation of the agent-based model framework for home-to-work commute trips. The framework uses a graph-partition based technique that can leverage the interaction structure of agents within a geographic proximity and can boost the simulation execution time. Further, based on a flexible design, it can run ABM with different levels of computing resources-from multi core workstations to an HPC grid. The framework has been tested on the Titan Cray XK7 supercomputer of the Oak Ridge Leadership Computing Facility.

Park, Byung↗
DOE OSTI2020

Enzyme activities predicted by metabolite concentrations and solvent capacity in the cell

Experimental measurements or computational model predictions of the post-translational regulation of enzymes needed in a metabolic pathway is a difficult problem. Consequently, regulation is mostly known only for well-studied reactions of central metabolism in various model organisms. In this study, we use two approaches to predict enzyme regulation policies and investigate the hypothesis that regulation is driven by the need to maintain the solvent capacity in the cell. The first predictive method uses a statistical thermodynamics and metabolic control theory framework while the second method is performed using a hybrid optimization–reinforcement learning approach. Efficient regulation schemes were learned from experimental data that either agree with theoretical calculations or result in a higher cell fitness using maximum useful work as a metric. As previously hypothesized, regulation is herein shown to control the concentrations of both immediate and downstream product concentrations at physiological levels. Model predictions provide the following two novel general principles: (1) the regulation itself causes the reactions to be much further from equilibrium instead of the common assumption that highly non-equilibrium reactions are the targets for regulation; and (2) the minimal regulation needed to maintain metabolite levels at physiological concentrations maximizes the free energy dissipation rate instead of preserving a specific energy charge. The resulting energy dissipation rate is an emergent property of regulation which may be represented by a high value of the adenylate energy charge. In addition, the predictions demonstrate that the amount of regulation needed can be minimized if it is applied at the beginning or branch point of a pathway, in agreement with common notions. The approach is demonstrated for three pathways in the central metabolism of E. coli (gluconeogenesis, glycolysis-tricarboxylic acid (TCA) and pentose phosphate-TCA) that each require different regulation schemes. It is shown quantitatively that hexokinase, glucose 6-phosphate dehydrogenase and glyceraldehyde phosphate dehydrogenase, all branch points of pathways, play the largest roles in regulating central metabolism.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A Novel Deep Reinforcement Learning Approach to Traffic Signal Control with Connected Vehicles

The advent of connected vehicle (CV) technology offers new possibilities for a revolution in future transportation systems. With the availability of real-time traffic data from CVs, it is possible to more effectively optimize traffic signals to reduce congestion, increase fuel efficiency, and enhance road safety. The success of CV-based signal control depends on an accurate and computationally efficient model that accounts for the stochastic and nonlinear nature of the traffic flow. Without the necessity of prior knowledge of the traffic system’s model architecture, reinforcement learning (RL) is a promising tool to acquire the control policy through observing the transition of the traffic states. In this paper, we propose a novel data-driven traffic signal control method that leverages the latest in deep learning and reinforcement learning techniques. By incorporating a compressed representation of the traffic states, the proposed method overcomes the limitations of the existing methods in defining the action space to include more practical and flexible signal phases. The simulation results demonstrate the convergence and robust performance of the proposed method against several existing benchmark methods in terms of average vehicle speeds, queue length, wait time, and traffic density.

42 ENGINEERING↗