SEARCH · Search NASA

Results for “counterfactuals”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

9 records

All sources NASA NTRS DOE OSTI

NASA NTRS2014

CLEANing the Reward: Counterfactual Actions to Remove Exploratory Action Noise in Multiagent Learning

Learning in multiagent systems can be slow because agents must learn both how to behave in a complex environment and how to account for the actions of other agents. The inability of an agent to distinguish between the true environmental dynamics and those caused by the stochastic exploratory actions of other agents creates noise in each agent's reward signal. This learning noise can have unforeseen and often undesirable effects on the resultant system performance. We define such noise as exploratory action noise, demonstrate the critical impact it can have on the learning process in multiagent settings, and introduce a reward structure to effectively remove such noise from each agent's reward signal. In particular, we introduce Coordinated Learning without Exploratory Action Noise (CLEAN) rewards and empirically demonstrate their benefits

Reinforcement Learning↗

NASA NTRS2006

QUICR-learning for Multi-Agent Coordination

Coordinating multiple agents that need to perform a sequence of actions to maximize a system level reward requires solving two distinct credit assignment problems. First, credit must be assigned for an action taken at time step t that results in a reward at time step t > t. Second, credit must be assigned for the contribution of agent i to the overall system performance. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning. The second credit assignment problem is typically addressed by creating custom reward functions. To address both credit assignment problems simultaneously, we propose the "Q Updates with Immediate Counterfactual Rewards-learning" (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. QUICR-learning is based on previous work on single-time-step counterfactual rewards described by the collectives framework. Results on a traffic congestion problem shows that QUICR-learning is significantly better than a Q-learner using collectives-based (single-time-step counterfactual) rewards. In addition QUICR-learning provides significant gains over conventional and local Q-learning. Additional results on a multi-agent grid-world problem show that the improvements due to QUICR-learning are not domain specific and can provide up to a ten fold increase in performance over existing methods.

Agogino, Adrian K.↗

NASA NTRS2005

Quicker Q-Learning in Multi-Agent Systems

Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.

Agogino, Adrian K.↗

NASA NTRS2017

Characterizing Agricultural Impacts of Recent Large-Scale US Droughts and Changing Technology and Management

Process-based agricultural models, applied in novel ways, can reproduce historical crop yield anomalies in the US, with median absolute deviation from observations of 6.7% at national-level and 11% at state-level. In seasons for which drought is the overriding factor, performance is further improved. Historical counterfactual scenarios for the 1988 and 2012 droughts show that changes in agricultural technologies and management have reduced system-level drought sensitivity in US maize production by about 25% in the intervening years. Finally, we estimate the economic costs of the two droughts in terms of insured and uninsured crop losses in each US county (for a total, adjusted for inflation, of $9 billion in 1988 and $21.6 billion in 2012). We compare these with cost estimates from the counterfactual scenarios and with crop indemnity data where available. Model based measures are capable of accurately reproducing the direct agro-economic losses associated with extreme drought and can be used to characterize and compare events that occurred under very different conditions. This work suggests new approaches to modeling, monitoring, forecasting, and evaluating drought impacts on agriculture, as well as evaluating technological changes to inform adaptation strategies for future climate change and extreme events.

Climate extremes; Drought impacts; Agriculture; Se↗

NASA NTRS1992

Minimax confidence intervals in geomagnetism

The present paper uses theory of Donoho (1989) to find lower bounds on the lengths of optimally short fixed-length confidence intervals (minimax confidence intervals) for Gauss coefficients of the field of degree 1-12 using the heat flow constraint. The bounds on optimal minimax intervals are about 40 percent shorter than Backus' intervals: no procedure for producing fixed-length confidence intervals, linear or nonlinear, can give intervals shorter than about 60 percent the length of Backus' in this problem. While both methods rigorously account for the fact that core field models are infinite-dimensional, the application of the techniques to the geomagnetic problem involves approximations and counterfactual assumptions about the data errors, and so these results are likely to be extremely optimistic estimates of the actual uncertainty in Gauss coefficients.

Stark, Philip B.↗

NASA NTRS2005

What If? Paths Not Taken

I think the point of this exercise in counterfactual thinking is two-fold first, to recognize that not only have choices been made in the past that defined the character of what has happened and that different choices were possible and would have led to different outcomes, and, second, that we are currently making similar choices for the future. Today s choices obviously will have significant long-term consequences for space development. Decision-makers have an image of a desirable future when they make choices, but they also realize that the link between current choice and desired result is always uncertain. As the philosopher Yogi Berra is often quoted as having said, "making predictions is hard, especially when they are about the future."

Logsdon, John M.↗

NASA NTRS2013

Global Premature Mortality Due to Anthropogenic Outdoor Air Pollution and the Contribution of Past Climate Change

Increased concentrations of ozone and fine particulate matter (PM2.5) since preindustrial times reflect increased emissions, but also contributions of past climate change. Here we use modeled concentrations from an ensemble of chemistry–climate models to estimate the global burden of anthropogenic outdoor air pollution on present-day premature human mortality, and the component of that burden attributable to past climate change. Using simulated concentrations for 2000 and 1850 and concentration–response functions (CRFs), we estimate that, at present, 470 000 (95% confidence interval, 140 000 to 900 000) premature respiratory deaths are associated globally and annually with anthropogenic ozone, and 2.1 (1.3 to 3.0) million deaths with anthropogenic PM2.5-related cardiopulmonary diseases (93%) and lung cancer (7%). These estimates are smaller than ones from previous studies because we use modeled 1850 air pollution rather than a counterfactual low concentration, and because of different emissions. Uncertainty in CRFs contributes more to overall uncertainty than the spread of model results. Mortality attributed to the effects of past climate change on air quality is considerably smaller than the global burden: 1500 (−20 000 to 27 000) deaths yr (exp -1) due to ozone and 2200 (−350 000 to 140 000) due to PM2.5. The small multi-model means are coincidental, as there are larger ranges of results for individual models, reflected in the large uncertainties, with some models suggesting that past climate change has reduced air pollution mortality.

human health↗

NASA NTRS2021

Global Impact of COVID-19 Restrictions on the Atmospheric Concentrations of Nitrogen Dioxide and Ozone

Social-distancing to combat the COVID-19 pandemic has led to widespread reductions in air pollutant emissions. Quantifying these changes requires a business-as-usual counterfactual that accounts for the synoptic and seasonal variability of air pollutants. We use a machine learning algorithm driven by information from the NASA GEOS-CF model to assess changes in nitrogen dioxide (NO2) and ozone (O3) at 5,756 observation sites in countries from January through June 2020. Reductions in NO2 coincide with timing and intensity of COVID-19 restrictions, ranging from 60% in severely affected cities (e.g., Wuhan, Milan) to little change (e.g., Rio de Janeiro, Taipei). On average, NO2 concentrations were (13-23) % lower than business as usual from February 2020 onward. China experienced the earliest and steepest decline, but concentrations since April have mostly recovered and remained within 5% to the business-as-usual estimate. NO2 reductions in Europe and the US have been more gradual with a halting recovery starting in late March. We estimate that the global NOx (NO+NO2) emission reduction during the first 6 months of 2020 amounted to 3.1 (2.6-3.6) TgN, equivalent to 5.5(4.7-6.4) % of the annual anthropogenic total. The response of surface O3 is complicated by competing influences of non-linear atmospheric chemistry. While surface O3 increased by up to 50% in some locations, we find the overall net impact on daily average O3 between February -June 2020 to be small. However, our analysis indicates a flattening of the O3 diurnal cycle with an increase in nighttime ozone due to reduced titration and a decrease in daytime ozone, reflecting a reduction in photochemical production.

Christoph A Keller↗

NASA NTRS2024

Scenario Setup and Forcing Data for Impact Model Evaluation and Impact Attribution Within the Third Round of the Inter-Sectoral Model Intercomparison Project (ISIMIP3a)

This paper describes the rationale and the protocol of the first component of the third simulation round of the Inter-Sectoral Impact Model Intercomparison Project (ISIMIP3a, http://www.isimip.org, last access: 2 November 2023) and the associated set of climate-related and direct human forcing data (CRF and DHF, respectively). The observation-based climate-related forcings for the first time include high-resolution observational climate forcings derived by orographic downscaling, monthly to hourly coastal water levels, and wind fields associated with historical tropical cyclones. The DHFs include land use patterns, population densities, information about water and agricultural management, and fishing intensities. The ISIMIP3a impact model simulations driven by these observation-based climate-related and direct human forcings are designed to test to what degree the impact models can explain observed changes in natural and human systems. In a second set of ISIMIP3a experiments the participating impact models are forced by the same DHFs but a counterfactual set of atmospheric forcings and coastal water levels where observed trends have been removed. These experiments are designed to allow for the attribution of observed changes in natural, human, and managed systems to climate change, rising CH 4 and CO 2 concentrations, and sea level rise according to the definition of the Working Group II contribution to the IPCC AR6.

Inter Sectoral Impact Model Intercomparison Projec↗