Eliciting Experts' Judgments about Uncertain Probabilities: A Brief Primer to Using Expert Elicitation Well.
Abstract not provided.
Butler, Karin↗
SEARCH · Search NASA
Results for “expert elicitation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Expert elicitation survey predicts 37% to 49% declines in wind energy costs by 2050
Wind energy has experienced accelerated cost reduction over the past five years—far greater than predicted in a 2015 expert elicitation. Here we report results from a new survey on wind costs, compare those with previous results and discuss the accuracy of the earlier predictions. In this work, we show that experts in 2020 expect future onshore and offshore wind costs to decline 37–49% by 2050, resulting in costs 50% lower than predicted in 2015. This is due to cost reductions witnessed over the past five years and expected continued advancements. If realized, these costs might allow wind to play a larger role in energy supply than previously anticipated. Considering both surveys, we also conclude that there is considerable uncertainty about future costs. Our results illustrate the importance of considering cost uncertainty, highlight the value and limits of using experts to reveal those uncertainties, and yield possible lessons for energy modellers and expert elicitation.
17 WIND ENERGY↗
The Utility in Conjoint Analysis as a Fast Expert Elicitation Technique
This paper presents an interface and analysis technique for quickly conducting expert elicitation with the goal of determining entity importance. Our interface deploys a two-alternative choice experiment that is capable of representing knowledge graphs in an easy to interpret fashion for users with limited experience with knowledge graphs. Our analysis methodology takes advantage of conjoint analysis techniques and provides entity weights for many SMEs simultaneously. The results largely align with individual participant fits.
Conjoint Analysis, interface, Expert Elicitation, ↗
Expert Elicitation for Tidal Energy Levelized Cost of Energy: Present and Future
In accordance with the Government Performance and Results Act (GPRA), H2O annually assesses marine energy technology development resulting from government-funded research and development programs and strategy. For GPRA reporting, H2O uses the levelized cost of energy (LCOE) - which represents the total system cost per unit of energy produced - to measure the progression of marine energy technology development, assess the impact of their research and development programs, and identify future research priorities. To support H2O's GPRA reporting requirements and inform future strategy, the National Laboratory of the Rockies conducted a tidal energy LCOE expert elicitation study to estimate present and future LCOE[AB2.1]. This report describes the motivation and background for the elicitation study, the methodology used to conduct the study, and the study results. It also provides future recommendations for accelerated tidal energy LCOE reduction based on feedback from study participants.
16 TIDAL AND WAVE POWER↗
Expert Elicitation on Wind Farm Control
Wind farm control is an active and growing field of research in which the control actions of individual turbines in a farm are coordinated, accounting for inter-turbine aerodynamic interaction, to improve the overall performance of the wind farm and to reduce costs. The primary objectives of wind farm control include increasing power production, reducing turbine loads, and providing electricity grid support services. Additional objectives include improving reliability or reducing external impacts to the environment and communities. In 2019, a European research project (FarmConners) was started with the main goal of providing an overview of the state-of-the-art in wind farm control, identifying consensus of research findings, data sets, and best practices, providing a summary of the main research challenges, and establishing a roadmap on how to address these challenges. Complementary to the FarmConners project, an IEA Wind Topical Expert Meeting (TEM) and two rounds of surveys among experts were performed. From these events we can clearly identify an interest in more public validation campaigns. Additionally, a deeper understanding of the mechanical loads and the uncertainties concerning the effectiveness of wind farm control are considered two major research gaps.
17 WIND ENERGY↗
Expert Elicitation for Wave Energy LCOE Futures
As the world faces increasing threats from climate change, the importance of developing renewable energy technologies and reducing their costs have similarly increased. Marine energy technologies (which include wave, tidal, ocean current, ocean thermal and salinity gradient resources) are often referred to as the most nascent and newest suite of renewable technologies under development. There are vast marine energy resources available around the world and within U.S. territorial waters, and as the technologies have continued to develop, the long-term trajectory of cost reductions and performance improvements is of particular interest. This study specifically investigates the long-term cost reduction potential for commercial wave energy technologies, as wave energy represents the largest marine energy resource available to the continental U.S. Similar studies focused on other resources and technologies may follow in the future.
16 TIDAL AND WAVE POWER↗
An Expert Elicitation Method for Estimating Probabilistic Schedule Risk During Conceptual System Design.
Abstract not provided.
Pedersen, Robert Douglas↗
Reliability Prediction using FMEA, FTA, and Related Techniques [Slides]
Summary: We presented a reliability analysis framework. We made point estimates of reliability using reliability block diagrams, fault trees, success trees and estimates with uncertainty using expert elicitation, Monte Carlo simulation, Bayesian analysis. Expert elicitation of failure modes and probabilities is labor-intensive, but critical. Bayesian analysis updates information from expert elicitation with data from reliability and aging tests (aging/compatibility data are needed to estimate lower-bound reliabilities at end of life). Estimation by more than one method helps insure consistency and accuracy.
97 MATHEMATICS AND COMPUTING↗
The Potential of Hydrogeodesy to Address Water-Related and Sustainability Challenges
Increasing climatic and human pressures are changing the world's water resources and hydrological processes at unprecedented rates. Understanding these changes requires comprehensive monitoring of water resources. Hydrogeodesy, the science that measures the Earth's solid and aquatic surfaces, gravity field, and their changes over time, delivers a range of novel monitoring tools that are complementary to traditional hydrological methods. It encompasses geodetic technologies such as Altimetry, Interferometric Synthetic Aperture Radar (InSAR), Gravimetry, and Global Navigation Satellite Systems (GNSS). Beyond quantifying these changes, there is a need to understand how hydrogeodesy can contribute to more ambitious goals dealing with water-related and sustainability sciences. Addressing this need, we combine a meta-analysis of over 3,000 articles to chart the range, trends, and applications of satellite-based hydrogeodesy with an expert elicitation that systematically assesses the potential of hydrogeodesy. We find a growing body of literature relating to the advancements in hydrogeodetic methods, their accuracy and precision, and their inclusion in hydrological modeling, with a considerably smaller portion related to understanding hydrological processes, water management, and sustainability sciences. The meta-analysis also shows that while lakes, groundwater and glaciers are commonly monitored by these technologies, wetlands or permafrost could benefit from a wider range of applications. In turn, the expert elicitation envisages the potential of hydrogeodesy to help solve the 23 Unsolved Questions of the International Association of Hydrological Sciences and advance knowledge as guidance toward a safe operating space for humanity. It also highlights how this potential can be maximized by combining hydrogeodetic technologies simultaneously, exploiting artificial intelligence, and accurately integrating other Earth science disciplines. Finally, we call for a coordinated way forward to include hydrogeodesy in tertiary education and broaden its application to water-related and sustainability sciences in order to exploit its full potential.
54 ENVIRONMENTAL SCIENCES↗
Methods for R&D Portfolio Analysis and Evaluation (Workshop Report)
The Workshop on Methods for R&D Portfolio Analysis and Evaluation convened on 17–18 July 2019 at the National Renewable Energy Laboratory in Golden, Colorado, and examined strengths and weaknesses of the various methodologies applicable to R&D portfolio modeling, analysis, and decision support, given pragmatic constraints such as data availability, uncertainties in estimating the impact of R&D spending, and practical operational overheads. Participants employed their deep expertise in approaches such as stochastic optimization, real options, Monte-Carlo analysis, Bayesian networks, decision theory, complex systems analysis, deep uncertainty, and technology-evolution modeling to critique the initial example models developed by the project’s core team and to conduct thought experiments grounded in real-life technology models, progress data, expert elicitation, and portfolio information. This engagement of participants’ methodological expertise with the practical requirements of real-life portfolio decision support yielded ideas for improved approaches, alternative methodological hypotheses, and hybridization of methodologies that are well-grounded theoretically, computationally sound, and realistically executable given data availability and other practical constraints.
32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
Methods for R&D Portfolio Analysis and Evaluation (Workshop Report)
The Workshop on Methods for R&D Portfolio Analysis and Evaluation convened on 17-18 July 2019 at the National Renewable Energy Laboratory in Golden, Colorado, and examined strengths and weaknesses of the various methodologies applicable to R&D portfolio modeling, analysis, and decision support, given pragmatic constraints such as data availability, uncertainties in estimating the impact of R&D spending, and practical operational overheads. Participants employed their deep expertise in approaches such as stochastic optimization, real options, Monte-Carlo analysis, Bayesian networks, decision theory, complex systems analysis, deep uncertainty, and technology-evolution modeling to critique the initial example models developed by the project’s core team and to conduct thought experiments grounded in real-life technology models, progress data, expert elicitation, and portfolio information. This engagement of participants’ methodological expertise with the practical requirements of real-life portfolio decision support yielded ideas for improved approaches, alternative methodological hypotheses, and hybridization of methodologies that are well-grounded theoretically, computationally sound, and realistically executable given data availability and other practical constraints.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Energy sector portfolio analysis with uncertainty
Governments are dealing with the challenge of how to efficiently invest in research and development portfolios related to energy technologies. Research and development investment decisions in the energy space are especially difficult due to numerous risks and uncertainties, and due to the complexity of energy's interactions with the broad economy. Historically, much of the U.S. Department of Energy's in-depth research and development analyses focused on assessing the impact of a research and development activity in isolation from other available opportunities and did not substantially consider risk and uncertainty. Endeavoring to combine integrated energy-economy modeling with uncertainty analysis and technology-specific research and development activities, the U.S. Department of Energy commissioned the development of the Stochastic Energy Deployment System to support and improve public energy research and development decision-making. The Stochastic Energy Deployment System draws from expert-elicited probability distributions for research and development-driven improvements in technology cost and performance, and it uses Monte Carlo simulations to evaluate the likelihood of outcomes within a system dynamics energy-economy model. The framework estimates the uncertain benefits and costs of various research and development portfolios and provides insight into the probability of meeting national technology goals, while accounting for interactions with the larger economy and for interactions among research and development investments spanning many energy sectors.
29 ENERGY PLANNING, POLICY, AND ECONOMY↗
Rapidly Changing Range Limits in a Warming World: Critical Data Limitations and Knowledge Gaps for Advancing Understanding of Mangrove Range Dynamics in the Southeastern USA
Climate change is altering species’ range limits and transforming ecosystems. For example, warming temperatures are leading to the range expansion of tropical, cold-sensitive species at the expense of their cold-tolerant counterparts. In some temperate and subtropical coastal wetlands, warming winters are enabling mangrove forest encroachment into salt marsh, which is a major regime shift that has significant ecological and societal ramifications. Here, we synthesized existing data and expert knowledge to assess the distribution of mangroves near rapidly changing range limits in the southeastern USA. We used expert elicitation to identify data limitations and highlight knowledge gaps for advancing understanding of past, current, and future range dynamics. Mangroves near poleward range limits are often shorter, wider, and more shrublike compared to their tropical counterparts that grow as tall forests in freeze-free, resource-rich environments. The northern range limits of mangroves in the southeastern USA are particularly dynamic and climate sensitive due to abundance of suitable coastal wetland habitat and the exposure of mangroves to winter temperature extremes that are much colder than comparable range limits on other continents. Thus, there is need for methodological refinements and improved spatiotemporal data regarding changes in mangrove structure and abundance near northern range limits in the southeastern USA. Advancing understanding of rapidly changing range limits is critical for foundation plant species such as mangroves, as it provides a basis for anticipating and preparing for the cascading effects of climate-induced species redistribution on ecosystems and the human communities that depend on their ecosystem services.
54 ENVIRONMENTAL SCIENCES↗
Modeling efficient and equitable distribution of COVID-19 vaccines
Producing and distributing COVID-19 vaccine during the pandemic is a major logistical challenge requiring careful planning and efficient execution. This report presents information on logistical, policy and technical issues relevant to rapidly fielding a COVID-19 vaccination program. For this study we (a) conducted literature review and subject matter expert elicitation to understand current vaccine manufacturing and distribution capabilities and vaccine allocation strategies, (b) designed a baseline vaccine distribution strategy and modeling strategy to provide insight into the potential for targeted distribution of limited initial vaccine supplies, and (c) developed parametric interfaces to enable vaccine distribution scenarios to be analyzed in depth with Sandias Adaptive Recovery Model that will allow us evaluate the additional sub- populations and alternative distribution scenarios from a public health benefit and associated economic disruption Principal issues, challenges, and complexities that complicate COVID-19 vaccine delivery identified in our literature and subject matter expert investigation include these items: The United States has not mounted an urgent nationwide vaccination campaign in recent history. The existing global manufacturing and distribution infrastructure are not able to produce enough vaccine for the population immediately. Vaccines, once available will be scarce resources. Prioritization for vaccine allocation will be built on existing distribution networks. Vaccine distribution may not have a universal impact on disease transmission and morbidity because of scarcity, priority population demographics, and underlying disease transmission rates. Considerations for designing a vaccine distribution strategy are discussed. A baseline distribution strategy is designed and tested using the Adaptive Recovery Model, which couples a deterministic compartmental epidemiological model and a stochastic network model. We show the impact of this vaccine distribution strategy on hospitalizations, mortality, and contact tracing requirements. This model can be used to quantitatively evaluate alternative distribution scenarios, guiding policy decisions as vaccine candidates are narrowed down.
59 BASIC BIOLOGICAL SCIENCES↗
A research agenda for the science of actionable knowledge: Drawing from a review of the most misguided to the most enlightened claims in the science-policy interface literature
Linking science with action affords a prime opportunity to leverage greater societal impact from research and increase the use of evidence in decision-making. Success in these areas depends critically upon processes of producing and mobilizing knowledge, as well as supporting and making decisions. For decades, scholars have idealized and described these social processes in different ways, resulting in numerous assumptions that now variously guide engagements at the interface of science and society. We systematically catalog these assumptions based on prior research on the science-policy interface, and further distill them into a set of 26 claims. We then elicit expert perspectives (n = 16) about these claims to assess the extent to which they are accurate or merit further examination. Out of this process, we construct a research agenda to motivate future scientific research on actionable knowledge, prioritizing areas that experts identified as critical gaps in understanding of the science-society interface. The resulting agenda focuses on how to define success, support intermediaries, build trust, and evaluate the importance of consensus and its alternatives – all in the diverse contexts of science-society-decision-making interactions. We further raise questions about the centrality of knowledge in these interactions, discussing how a governance lens might be generative of efforts to support more equitable processes and outcomes. We offer these suggestions with hopes of furthering the science of actionable knowledge as a transdisciplinary area of inquiry.
99 GENERAL AND MISCELLANEOUS↗
Governance and resilience as entry points for transforming food systems in the countdown to 2030
Due to complex interactions, changes in any one area of food systems are likely to impact—and possibly depend on—changes in other areas. Here we present the first annual monitoring update of the indicator framework proposed by the Food Systems Countdown Initiative, with new qualitative analysis elucidating interactions across indicators. Since 2000, we find that 20 of 42 indicators with time series have been trending in a desirable direction, indicating modest positive change. Qualitative expert elicitation assessed governance and resilience indicators to be most connected to other indicators across themes, highlighting entry points for action—particularly governance action. Literature review and country case studies add context to the assessed interactions across diets, environment, livelihoods, governance and resilience indicators, helping different actors understand and navigate food systems towards desirable change.
Schneider, Kate R. [Johns Hopkins Univ., Washingto↗
An open-access simulated earthquake ground-motion database for an M7 Hayward Fault earthquake in the San Francisco Bay Region
Comprehensive understanding of earthquake ground motions, particularly in the near-fault region of large-magnitude events, is limited by gaps in strong-motion data. This challenge is prominent in areas with high seismic hazard but infrequent large earthquakes where data is sparse and difficult to interpret. These data limitations lead to uncertainties in the development of site-specific ground motions, which are crucial for engineering risk assessments. To address these challenges, physics-based regional-scale ground-motion simulations have been developed. With the emergence of exaflop-scale computing ecosystems, it is now possible to simulate regional earthquake processes at unprecedented fidelity and generate the large number of fault rupture realizations necessary to characterize both intra- and inter-event ground-motion variability. This article introduces a new database of simulated earthquake ground motions, created for applications in earthquake engineering, earthquake planning, and emergency response. The inaugural version of the database features simulated ground motions for a magnitude 7 Hayward Fault earthquake in the San Francisco Bay Region (SFBR), using the EarthQuake SIMulation (EQSIM) simulation framework and the Graves–Pitarka kinematic rupture model. The aim is to provide high-fidelity, spatially dense, three-component motions generated on the Department of Energy’s (DOE) newest generation of graphics processing unit (GPU)-accelerated supercomputers. These motions are being made openly available to the engineering, scientific, and disaster planning communities. In addition, this work develops protocols for the efficient dissemination of these large data sets and emphasizes community engagement to build confidence in their application. This article discusses the methodology behind the data, underlying software verification and validation, scalable data management, and a user interface for data access. The goal is to facilitate widespread use and elicit expert feedback to maximize the utility and exploitation of simulated motions. While the initial focus is on the San Francisco Region, simulations for additional regions will be added as the DOE program progresses.
Simulated ground-motion database↗
The Technical and Economic Potential of the H2@Scale Hydrogen Concept within the United States
The U.S. energy system is evolving as society and technologies change. Renewable electricity generation - especially from wind and solar - is growing rapidly, and alternative energy sources are being developed and implemented across the residential, commercial, transportation, and industrial sectors to take advantage of their cost, security, and health benefits. Systemic changes present numerous challenges to grid resiliency and energy affordability, creating a need for synergistic solutions that satisfy multiple applications while yielding system-wide cost and emissions benefits. One such solution is an integrated hydrogen energy system. This is the focus of H2@Scale - a U.S. Department of Energy (DOE) initiative led by the Office of Energy Efficiency and Renewable Energy’s Hydrogen and Fuel Technologies Office. H2@Scale brings together stakeholders to advance affordable hydrogen production, transport, storage, and utilization in multiple energy sectors. The H2@Scale concept involves hydrogen as an energy intermediate. Hydrogen can be produced from various conventional and renewable energy sources including as a responsive load on the electric grid. Hydrogen has many current applications and many more potential applications, such as energy for transportation - used directly in fuel cell electric vehicles (FCEVs), as a feedstock for synthetic fuels, and to upgrade oil and biomass - feedstock for industry (e.g., for ammonia production, metals refining, and other end uses), heat for industry and buildings, and electricity storage. Owing to its flexibility and fungibility, a hydrogen intermediate could link energy sources that have surplus availability to markets that require energy or chemical feedstocks, benefiting both. This document builds upon a growing body of analyses of hydrogen as an energy intermediate by reporting the results from our initial analysis of the potential impacts of the H2@Scale vision by the mid-21 st century for the 48 contiguous U.S. states. Previous estimates have been based on expert elicitation and focused on hydrogen demands. We build upon them, first, by estimating hydrogen’s serviceable consumption potential for possible hydrogen applications and the technical potential for producing hydrogen from various resources. We define the serviceable consumption potential as the quantity of hydrogen that would be consumed to serve the portion of the market that could be captured without considering economics (i.e., if the price of hydrogen were $0/kg over an extended period); thus, it can be considered an upper bound for the size of the market. We define the technical potential as the resource potential constrained by real-world geography and system performance, but not by economics. We then compare the cumulative serviceable consumption potential with the technical potential of a number of possible sources. Second, we estimate economic potential: the quantity of hydrogen at an equilibrium price at which suppliers are willing to sell and consumers are willing to buy the same quantity of hydrogen. We believe this method provides a deeper understanding than was available in the previous analyses. We develop economic potentials for multiple scenarios across various market and technology-advancement assumptions.
08 HYDROGEN↗