Search NASA⌕ Search

SEARCH · Search NASA

Results for “policy evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Towards Off-policy Evaluation as a Prerequisite for Real-world Reinforcement Learning in Building Control

We present an initial study of off-policy evaluation (OPE), a problem prerequisite to real-world reinforcement learning (RL), in the context of building control. OPE is the problem of estimating a policy's performance without running it on the actual system, using historical data from the existing controller. It enables the control engineers to ensure a new, pretrained policy satisfies the performance requirements and safety constraints of a real-world system, prior to interacting with it. While many methods have been developed for OPE, no study has evaluated which ones are suitable for building operational data, which are generated by deterministic policies and have limited coverage of the state-action space. After reviewing existing works and their assumptions, we adopted the approximate model (AM) method. Furthermore, we used bootstrapping to quantify uncertainty and correct for bias. In a simulation study, we evaluated the proposed approach on 10 policies pretrained with imitation learning. On average, the AM method estimated the energy and comfort costs with 1.84% and 14.1% error, respectively.

Chen, B↗
DOE OSTI2022

Cordon screen: A cordon-based congestion pricing policy evaluation method for U.S. cities

Global trends toward urbanization will exacerbate traffic congestion, delays in economic productivity, and air pollution issues for growing cities. Traffic congestion pricing is one method available to help ameliorate these concerns. New York City is on the verge of implementing a cordon-based traffic congestion pricing policy around its central business district. For budget-constrained municipalities, evaluating implementation of such policy could be costly. This article proposes a sketch-planning methodology, called Cordon Screen, for major U.S. cities to evaluate the net income, traffic mitigation, and avoided pollution emissions from cordon-based traffic congestion pricing. This method relies on national datasets and limited user-specific data inputs, along with a range of user-selectable assumptions informed by academic literature to deliver order-of-magnitude results. The numerous limitations of this method are acceptable for preliminary policy evaluation to determine if greater financial investment to obtain more accurate results is justified. The Denver metropolitan area is used to demonstrate Cordon Screen capabilities, with mid-range assumption results suggesting the policy is most effective at generating net income and increasing vehicle speeds on major interstates. For Denver, the policy is comparably less effective at reducing air pollution and increasing speeds on minor roadways. Validation against early implementation results from the London cordon are acceptable. Still, users should discount revenue generation projections. Choice of cordon area may be the most difficult obstacle when using the Cordon Screen. With refinement, Cordon Screen could serve as a low-cost, open-source planning evaluation tool for growing and congested U.S. cities.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Defender Policy Evaluation and Resource Allocation against MITRE ATT&CK Data and Evaluations

Protecting against multi-step attacks of uncertain duration and timing forces defenders into an indefinite, always ongoing, resource-intensive response. To effectively allocate resources, a defender must be able to analyze multi-step attacks under assumption of constantly allocating resources against an uncertain stream of potentially undetected attacks. To achieve this goal, we present a novel methodology that applies a game-theoretic approach to the attack, attacker, and defender data derived from MITRE´s ATT&CK ® Framework. Time to complete attack steps is drawn from a probability distribution determined by attacker and defender strategies and capabilities. This constraints attack success parameters and enables comparing different defender resource allocation strategies. By approximating attacker-defender games as Markov processes, we represent the attacker-defender interaction, estimate the attack success parameters, determine the effects of attacker and defender strategies, and maximize opportunities for defender strategy improvements against an uncertain stream of attacks. This novel representation and analysis of multi-step attacks enables defender policy optimization and resource allocation, which we illustrate using the data from MITRE´ s APT3 ATT&CK ® Framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

On Distributed Model-Free Reinforcement Learning Control with Stability Guarantee

Distributed learning can enable scalable and effective decision making in numerous complex cyber-physical systems such as smart transportation, robotics swarm, power systems, etc. However, the stability of the system is usually not guaranteed in most existing learning paradigms; and this limitation can hinder the wide deployment of machine learning in decision making of safety-critical systems. This paper presents a stability guaranteed distributed reinforcement learning (SGDRL) framework for interconnected linear subsystems, without knowing the subsystem models. While the learning process requires data from a peer-to-peer (p2p) communication architecture, the control implementation of each subsystem is only based on its local state. The stability of the interconnected subsystems will be ensured by a diagonally dominant eigenvalue condition, which will then be used in a model-free RL algorithm to learn the feedback control gains. The RL algorithm structure follows an off-policy iterative framework, with interleaved policy evaluation and policy update steps. We numerically validate our theoretical results by performing simulations on four interconnected sub-systems.

Mukherjee, Sayak↗
DOE OSTI2020

Taking stock of national climate policies to evaluate implementation of the Paris Agreement

Many countries have implemented national climate policies to accomplish pledged Nationally Determined Contributions and to contribute to the temperature objectives of the Paris Agreement on climate change. In 2023, the global stocktake will assess the combined effort of countries. Here, based on a public policy database and a multi-model scenario analysis, we show that implementation of current policies leaves a median emission gap of 22.4 to 28.2 GtCO 2 eq by 2030 with the optimal pathways to implement the well below 2 °C and 1.5 °C Paris goals. If Nationally Determined Contributions would be fully implemented, this gap would be reduced by a third. Interestingly, the countries evaluated were found to not achieve their pledged contributions with implemented policies (implementation gap), or to have an ambition gap with optimal pathways towards well below 2 °C. This shows that all countries would need to accelerate the implementation of policies for renewable technologies, while efficiency improvements are especially important in emerging countries and fossilfuel- dependent countries.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

HPC Digital Twins for Evaluating Scheduling Policies, Incentive Structures and their Impact on Power and Cooling

Schedulers are critical for optimal resource utilization in high-performance computing. Traditional methods to evaluate sched- ulers are limited to post-deployment analysis, or simulators, which do not model associated infrastructure. In this work, we present the first-of-its-kind integration of scheduling and digital twins in HPC. This enables what-if studies to understand the impact of parameter configurations and scheduling decisions on the physical assets, even before deployment, or regarching changes not easily realizable in production. We (1) provide the first digital twin framework extended with scheduling capabilities, (2) integrate various top-tier HPC systems given their publicly available datasets, (3) implement extensions to integrate external scheduling simulators. Finally, we show how to (4) implement and evaluate incentive structures, as- well-as (5) evaluate machine learning based scheduling, in such novel digital-twin based meta-framework to prototype scheduling. Our work enables what-if scenarios of HPC systems to evaluate sustainability, and the impact on the simulated system.

Maiterth, Matthias [ORNL] (ORCID:000000018698460X)↗
DOE OSTI2026

A Real2Sim Digital Twin Pipeline for Photorealistic Robot Simulation: Evaluating VLA Policy Deployment on a Bimanual Mobile Robot

Digital twins that are automatically constructed from robot sensor data offer a promising pathway for scalable Real2Sim and Sim2Real transfer. However, it remains an open question whether photorealistic reconstruction alone is sufficient to support reliable deployment of vision-language-action (VLA) policies. We present a generative-AI-assisted Real2Sim pipeline that generates simulation-ready digital twins from real-world RGB observations with minimal manual intervention. The pipeline uses prompted segmentation to isolate scene components and a generative 3D model to directly produce simulation assets, eliminating the need for traditional multi-view reconstruction or manual 3D modeling.\r\nTo evaluate simulation fidelity, we deploy and compare policies from two VLA models in both the real robot and the reconstructed\r\nsimulation under identical tasks and initial conditions. We compare joint-level action trajectories and analyze how divergence evolves over time in closed-loop execution. Although the reconstructed environments are visually accurate, we observe increasing trajectory divergence during closedloop operation. These results indicate that photorealistic reconstruction alone is insufficient to preserve closed-loop control behavior\r\nin VLA policies, particularly in contact-rich manipulation settings where small perceptual errors compound over time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Methods and system for siting advanced nuclear reactors and evaluating energy policy concerns

There is a growing sociopolitical desire to develop cleaner energy sources in the United States and maintain energy security. Regardless of politics, many coal-fired electric plants have already been shut down and many utilities are vowing to retire their current coal-fired assets within the next two decades. Replacement power assets require consideration of appropriate siting. A geographic information system (GIS)-based multicriteria decision analysis approach is useful to assist utility and energy companies, as well as policymakers, to evaluate potential areas for siting new plants in the contiguous United States. A GIS-based framework is simply a database of location information that allows for mapping, querying, modeling, and analyzing data based on location. The spatial output can be structured to be visual, allowing for easier analysis of location data. The need to site additional power assets, including renewable resources and clean power sources, such as nuclear, led to the development of the Oak Ridge Siting Analysis for power Generation Expansion (OR-SAGE) tool discussed in this paper. The tool takes inputs such as population growth, water availability, environmental indicators, and tectonic and geological hazards to provide an in-depth visual analysis for siting options. Energy companies and other stakeholders can use OR-SAGE to procure feedback quickly and effectively on land suitability based on technology specific inputs. Policymakers can use OR-SAGE to analyze the impacts of future energy technology decisions, while balancing competing resource use. Overall, this paper discusses the recent use of OR-SAGE for these purposes and plans for future development.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Assessing the Level of Inter-Sectoral Policy Integration for Governance in the Water–Energy Nexus: A Comparative Study of Los Angeles and Beijing

As concerns around water scarcity and energy security increase, so too has interest in the connections between these resources, through a concept called the water–energy nexus. Efforts to improve the integration of water and energy management and to understand their cross-sector relevance are growing. In particular, this paper develops a better empirical understanding on the extent to which governance settings hinder and/or enable policy coherence between the water and energy sectors through a comparative analysis of two case studies, namely, Los Angeles County, California, the United States, and the city of Beijing, China. This paper examines the extent to which the institutional context enables policy coordination within (vertically) and between (horizontally) the water and energy sectors in Beijing and Los Angeles. To do so, we propose a framework for analyzing policy integration for the water energy nexus based on environmental policy integration (EPI). The results highlight the multiple and flexible approaches of EPI in nexus governance, not least with regards to horizontal and vertical policy integration, but also in terms of explicit (i.e., intended) and implicit (i.e., unintended) coordination. The level of nexus-focused policy integration is highly dependent on the motivation at the local context and the criteria to evaluate policy success in each sector.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Relative Cost-Effectiveness of Electricity and Transportation Policies as a Means to Reduce CO2 Emissions in the United States: A Multi-Model Assessment

Two common energy policy instruments in the United States are tax incentives and technology standards. Although these instruments have been shown to be less cost-effective as a means to reduce CO2 emissions than direct emissions pricing mechanisms, it can be challenging to compare the CO2 emissions reduction costs of such policies across sectors, given the wide range in estimates for any given policy and inconsistencies in how such estimates are constructed across studies. This study addresses this analytical gap by simultaneously comparing the cost-effectiveness of policies across the electricity and transportation sectors using three publicly available US energy system models (EM-NEMS, ReEDS, and GCAM-USA). Four policies are explicitly compared: wind and solar tax credits, a renewable portfolio standard (RPS), a renewable fuel standard (RFS), and an electric vehicle (EV) tax credit. An economy-wide carbon tax is used as a benchmark for cost-effectiveness. Results from this study confirm prior insights about the cost-effectiveness of economy-wide carbon pricing relative to sectoral instruments but also reveal several novel insights about particular sectoral policies. Specifically, this study finds that (1) current electricity tax incentives provide uneven support for wind and solar technologies, (2) despite known inefficiencies, renewable energy policies in the electricity sector are less expensive than earlier estimates due to technology advancement and changes in market conditions, (3) within transportation, an expanded RFS with increasing advanced biofuel targets is more cost-effective than an EV tax credit extension under plausible assumptions, (4) EV incentives lead to a rebound in conventional vehicle fuel economy that further erodes cost-effectiveness, and (5) the change in policy costs over time is not known a priori, but the relative cost ordering among these policies does not depend on the timeframe of analysis. These results are largely robust to the underlying modeling framework, increasing the confidence with which they can be applied to climate policy evaluation.

economics↗
DOE OSTI2022

Enhanced Oblique Decision Tree Enabled Policy Extraction for Deep Reinforcement Learning in Power System Emergency Control

Deep reinforcement learning (DRL) algorithms have successfully solved many challenging problems in various power system control scenarios. However, their decision-making process is usually regarded as black-boxes. Furthermore, how DRL models interact with human intelligence remains an open problem. Thus, this paper proposes a policy extraction framework to extract a complex DRL model into an explainable policy. This framework includes three parts: 1) DRL training and data generation. We train an agent for a specific control task and generate data, which contains the control policy of the agent. 2) Policy extraction. We propose an information gain rate based weighted oblique decision tree (IGR-WODT) for DRL policy extraction. 3) Policy evaluation. We define three metrics to evaluate the performance of the proposed approach. A case study for the under-voltage load shedding problem shows that the IGR-WODT presents a performance enhancement compared with DRL, weighted oblique decision tree, and univariate decision tree. The proposed policy extraction method could provide an intuitive explanation of the neural network decision-making process to the dispatchers when making final decisions on power grid operation. Also, the resulted rule-based controller could replace the deep neural network-based controller in many field edge devices with limited computing resources, providing comparable performance.

deep reinforcement learning↗
DOE OSTI2024

Leveraging NREL's ResStock & ComStock Dataset to Evaluate Building Stock Electrification: Preprint

Residential and commercial buildings accounted for 40% of U.S. energy consumption in 2022 and represent a significant opportunity for decarbonization through energy efficiency and electrification, and for grid planning. Building stock energy modeling is a powerful tool that can evaluate what-if scenarios as utilities, municipalities, policymakers, building owners and others work towards equitable building decarbonization and climate goals. This presentation will highlight several high-impact use cases of the National Renewable Energy Laboratory (NREL)'s highly granular, bottom-up building stock energy modeling tools, ResStock and ComStock. These use cases cover a wide range of project scale, from neighborhood electrification analysis and municipality long-term energy planning, to state energy code development and national policy evaluation. This presentation will showcase specific real-world applications for which ResStock and ComStock have been utilized across the country, including California codes and standards cost-effectiveness analysis, New York City affordable housing electrification cost gap analysis, and California targeted electrification and gas decommissioning analysis. For each use case, this presentation will illustrate how ResStock and ComStock played a crucial role in accurately characterizing regional building stocks, providing discrete and aggregated end-use load shapes, and calculating lifecycle consumption, emissions, and costs for a variety of building electrification strategies and scenarios. Finally, this presentation will demonstrate how the data provided by ResStock and ComStock can help unlock significant outcomes for these use cases, including but not limited to, customer bill impact, incentive and program design, and energy equity analyses.

building stock modeling↗
DOE OSTI2021

A modeling framework for designing and evaluating curbside traffic management policies at Dallas-Fort Worth International Airport

Emerging mobility technologies are changing the transportation system landscape. This is especially evident at airports, such as the Dallas-Fort Worth International Airport (DFW). Without careful analysis, these changes could lead to inefficient and costly airport operations. This paper presents a modeling framework that integrates travel mode encoding, demand projection, and microsimulation to enable airports to develop, simulate, and evaluate curbside traffic managements policies and measure their impact. Here, the framework is utilized to analyze several traffic scenarios and policies for DFW: a baseline scenario which represents DFW traffic pattern as observed in 2018 and projected to 2045, a transit network company (TNC) electrification policy, a TNC queuing policy, a policy that increased transit ridership, a bus-only policy which considers the use of only buses inside DFW, an autonomous vehicle (AV) policy which investigates the impact of autonomous vehicle (AV) adoption on airport operations, and an example COVID-19 scenario which models the impact of the COVID19 pandemic. The simulations’ results demonstrate that: increasing the DFW transit ridership postpones the need for airport curbside expansion the most; encouraging shared-mobility with the bus-only policy produces the most savings in curbside congestion delays; automation and electrification for all passenger vehicle trips to/from DFW generates the most saving in fuel consumption and emissions; and uncontrolled AV adoption incurs the highest increase in fuel consumption, delay, and emissions and could require immediate airport capacity extension. Without policy intervention or investment in additional infrastructure capacity, these results predict the current operations would face significant congestion on high demand days starting as early as 2028. While derived in close partnership with DFW, the methodology presented here can be generalized to any airport.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

From Bricks to Clicks: Mapping the White Space in Building Innovation

It is a critical national imperative to transform the buildings sector, yet innovation is impeded by deployment failures that leave promising technologies stranded. Conventional market reports and techno-economic analysis provide an insufficient understanding of markets and resource allocation for emerging building technologies. They omit crucial commercialization factors such as ecosystem maturity and adoption friction, where the coordinated participation of a network of suppliers, contractors, financiers, regulators, and integrators is required to scale solutions. This study addresses these gaps by introducing an evaluation framework grounded in front-line data from six years of the DOE's IMPEL incubator, comprising experience from 300 building-sector innovators and the adjacent, complex ecosystem. Our methodology synthesizes top-down market analysis with bottom-up, practitioner-level data across five megatrends: (M1) Affordable materials and industrialized construction; (M2) Healthy and efficient mechanical systems; (M3) Intelligent building operations; (M4) Buildings as grid assets; and (M5) High-density power and cooling for data centers and therein identify twelve "white space" technology opportunities. Next, we develop a multi-criteria scoring rubric to rank these opportunities based on parameters, i.e., Affordability, Quality of Life, Reliability, and Security, yielding composite ‘Demand’ and ‘Maturity’ indices. Our results indicate that the most significant white spaces may not be incremental products but a new class of ‘Ecosystem Enablers’, such as logistics platforms, orchestration layers, and automated compliance software that solve structural deployment gaps. This paper summarizes this transparent, evidence-based, practitioner-informed evaluation framework for policymakers and investors to re-evaluate policy and resource allocation and unlock scalable market transformation.

Singh, Reshma↗
NASA NTRS1977

Interactions of technology and society: Impacts of improved airtransport. A study of airports at the grass roots

The feasibility of applying a particular conception of technology and social change to specific examples of technological development was investigated. The social and economic effects of improved airport capabilities on rural communities were examined. Factors which led to the successful implementation of a plan to construct sixty small airports in Ohio are explored and implications derived for forming public policies, evaluating air transportation development, and assessing technology.

Laporte, T.↗
NASA NTRS2012

Mission Assurance Modeling and Simulation: A Cyber Security Roadmap

This paper proposes a cyber security modeling and simulation roadmap to enhance mission assurance governance and establish risk reduction processes within constrained budgets. The term mission assurance stems from risk management work by Carnegie Mellon's Software Engineering Institute in the late 19905. By 2010, the Defense Information Systems Agency revised its cyber strategy and established the Program Executive Officer-Mission Assurance. This highlights a shift from simply protecting data to balancing risk and begins a necessary dialogue to establish a cyber security roadmap. The Military Operations Research Society has recommended a cyber community of practice, recognizing there are too few professionals having both cyber and analytic experience. The authors characterize the limited body of knowledge in this symbiotic relationship. This paper identifies operational and research requirements for mission assurance M&S supporting defense and homeland security. M&S techniques are needed for enterprise oversight of cyber investments, test and evaluation, policy, training, and analysis.

Gendron, Gerald↗