Visualizing Uncertainty: Design Choices Impact Decision Making
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Design under uncertainty has significantly grown in research developments during the past decade. Additionally, machine learning (ML) and explainable ML (XML) have offered various opportunities to provide reliable predictable models. The current article investigates the use of finite element modeling (FEM), ML and XML predictions, and uncertain-based design of carbon-carbon (C-C) composites for use in ultra-high temperatures. A C-C composite concentrating solar power (CSP) as a microvascular receiver is considered as a case study. These C-C composites are fiber composites with directly integrated carbonized microchannels to form a lightweight, high-absorptivity material that includes an embedded microvascular network of channels. The topology of these microchannels is engineered to optimize heat transfer to a supercritical carbon dioxide (sCO2) heat transfer fluid. The mechanical characterization of C-C composites is highly challenging. Thus, designing every component made of C-C composites for ultra-high temperature applications needs an uncertainty-based analysis. As a part of a comprehensive project on the development of a novel carbonized microvascular C-C composite, this paper explores C-C composite sensitivity analysis, FEM, ML prediction, and XML analysis. The resulting composite can then be carbonized and coated with an oxidation-resistant coating to form a thermally efficient and mechanically robust C-C composite. An ANSYS 3-D-FE model was used to analyze the CSP’s stress/strain. To consider the variability in the mechanical and thermal properties of C-C composites, various mechanical properties are considered as the ANSYS FEM’s input. A synthetic dataset from 730 ANSYS runs was produced to feed into the ML and XML algorithms for uncertainty analysis and prediction. The ML and XML algorithms could accurately predict the CSP stresses/strains.
Remanufacturing is a well-established end-of-life (EOL) strategy that promises significant savings in energy and carbon emissions. However, the current design practices are not remanufacturing-inclusive, i.e., the majority of products are designed for a single life cycle. As a result, potential products that can sustain multiple life cycles are deprived of additional benefits of being designed for remanufacturing, such as reduced material usage, lower cost, and improved environmental impact. Moreover, the uncertainty in design, material selection, and economics are not considered to produce remanufacturable designs. Accordingly, this research proposes a design for remanufacturing (DfRem) framework that accounts for design uncertainty and material price volatility. The framework systematically explores the design space, performs design optimization under uncertainty, followed by topology optimization to provide additional mass savings, and finally, a price volatility analysis for plausible design material choices. The candidate designs are evaluated based on their design mass, material price volatility, failure mode characteristics, carbon footprint, and embodied energy impacts. The proposed framework's utility is demonstrated via the use of an engine cylinder head case study subjected to thermo-mechanical loads along with fatigue and wear failure. Considering grey cast iron and aluminum alloy as the design material choices, it was found that the cast iron design reduced the initial design mass by 6% as opposed to a 5% decrease for aluminum. On the other hand, about 8% area of the cast iron design failed due to fatigue, compared to 3% for aluminum. Here, we further observed that although the aluminum design provided better mechanical performance than the cast iron design, this material was more expensive and volatile in price.
This report documents the program and the outcomes of Dagstuhl Seminar 22331 "Visualization and Decision Making Design Under Uncertainty". The seminar brought together 33 researchers and practitioners from different domains concerned with visualization and decision making under uncertainty including visualization, visual analytics, human-computer interaction, artificial intelligence, climate research, geography and geology. The programme was organized in two parts: In the first part which lasted two days, participants gave short talks where they discussed current practices and the uncertainty visualization challenges they encountered in their own research. At the end of day two, participants brainstormed collectively around the main uncertainty visualization research challenges across domains and applications. In the second part, participants voted for the following three main challenges they wished to discuss for the remainder of the seminar (one and a half days): applications, human-centered uncertainty visualization, a design process for uncertainty visualization. Thus three break-out groups were formed to discuss these challenges. Abstracts for the individual talks and the break-out group activities are included in this report.
Throughout the diverse range of science and engineering applications, there is a growing desire to develop computational methods that can reliably predict the behavior of complex systems. Specifically, there is a strategic need for tools that can robustly forecast the behavior of complex physical systems, where data may be high-dimensional, noisy, or sparse, and models of the system may be time-dependent or include uncertainty. We use mystic to build tools that leverage statistical learning, physics-informed learning, and active learning in the efficient generation of reliably predictive surrogates for complex physical systems. mystic is a robust, proven, open-source optimization and uncertainty quantification toolkit with over a decade of use in the design and optimization of neutron instrumentation, solar-powered drones, and gasguns, and in iterative tuning of models for Raman spectroscopy and elastoplastic materials strength. Recent developments have focused on automated learning of statistically robust surrogates under uncertainty, with applications in materials in extreme environments, nanostructures, materials simulations and strength models, and the failure of shielding under particle radiation. In 2020, McKerns demonstrated active learning of optimally robust surrogates with respect to new simulated data for molecular dynamics simulations of materials mixing in warm dense matter, and is currently applying active learning to the automated steering of particle accelerator beams and the optimal design and control of quantum optical sensor instrumentation.
In this paper, the development of a fission-gas collecting and physical-analysis-enabling instrument was proposed for small-volume determination. Analysis specifications require a design capable of accurately and repeatably determining volumes in the range of 0.07–2.5 mL. This system relies on a series of gas expansions originating from a cylinder with known internal volume. The combined gas law is used to derive the unknown volumes from these expansions. Initial system designs included one of two known volumes, 11.85 ± 0.34 mL and 5.807 ± 0.078 mL, with a manifold volume of 32 mL. Results obtained from modeling this system’s operation showed that 0.07 mL can be determined with a relative expanded uncertainty greater than 300% (k = 2) for a single replicate, which was unacceptable for the proposed experimental design. Initial modeling showed that the volume connecting the known volume and rodlet, i.e., the manifold volume, and the sensitivity of the pressure sensor were key contributors to the expanded uncertainty of the measured rodlet volume. The system’s design limited the available options for pressure sensors, so emphasis was placed on the design of the manifold volume. The final system design reduced the manifold volume to 17 mL. These changes in design, combined with replicate analysis, were able to reduce the relative expanded uncertainty by ±12% (k = 2) for the 0.07 mL volume.
Offshore wind farms (OWFs) are set to significantly contribute to global decarbonization efforts. Developers often use a sequential approach to optimize design variables and market participation for grid-integrated offshore wind farms. However, this method can lead to sub-optimal system performance, and uncertainties associated with renewable resources are often overlooked in decision-making. Here, this paper proposes a control co-design approach, optimizing design and control decisions for integrating OWFs into the power grid while considering energy market and primary frequency market participation. Additionally, we introduce optimal sizing solutions for energy storage systems deployed onshore to enhance revenue for OWF developers over time. This framework addresses uncertainties related to wind resources and energy prices. We analyze five U.S. west-coast offshore wind farm locations and potential interconnection points, as identified by the Bureau of Ocean Energy Management (BOEM). Results show that optimized control co-design solutions can increase market revenue by 3.2% and provide flexibility in managing wind resource uncertainties.
Engineering a robust hydraulic connection between wells is one of the most difficult aspects of enhanced geothermal systems (EGS). Designing and constructing such hydraulic connections requires and understanding of the in situ state of stress and the heterogeneities and discontinuities that naturally exist and may control the stimulation. Even with comprehensive stress and formation characterization programs substantial uncertainty remains in these key parameters. This is especially the case in high-temperature EGS environments where drilling conditions are often difficult and a far fewer logging and testing options are available. This paper presents a new approach for explicitly quantifying the uncertainties in the state of stress using a Bayesian Markov Chain Monte Carlo method. This approach produces a probability distribution for the stress tensor, including a general 3D orientation, that reflects the uncertainties in all the observations or indicators used to constrain the stress state. This method is demonstrated on the characterization data for the EGS Collab Experiment 2 site. The output of the analysis is used to guide the design of the planned stimulations. In the case of research projects like EGS Collab, explicitly quantifying the uncertainties in the stress state allow for more rigorous hypothesis testing by allowing conclusions drawn from the experiments to be interpreted in the context of the uncertain knowledge about conditions in the test bed.
Accurate nuclear data are required for simulations of many applications including nuclear criticality safety. Actinide nuclear data at intermediate energies (from 1 to 100s of keV) are imprecise and inaccurate, because of scarce differential data, and an insufficient theory approach to capture the structures expected in the data to yield evaluated nuclear data, and lack of integral data for proper validation. This is a known deficiency but has proved challenging to address. More specifically, only 5% of integral experiments in the International Criticality Safety Benchmark Evaluation Project (ICSBEP) benchmark suite address intermediate energies (Fig. 1). Associated calculated effective multiplication factor, k eff , values for these experiments are far outside the experimental uncertainties and are 25× further from experiment than for fast energies. These differences could either stem from systematic biases in nuclear data, experiments or both. The goal of the PARADIGM (PARallel Approach of Differential and InteGral Measurements) project is to significantly reduce (by more than tens of percent) the uncertainties of intermediate energy actinide nuclear data. The PARADIGM project designed and intends to execute LANSCE (Los Alamos Neutron Science CEnter) and NCERC (National Criticality Experiments Research Center) intermediate experiments in parallel. They will specifically address a high priority nuclear data need—reducing bias and uncertainty in intermediate plutonium nuclear data. The two experiment will achieve that by informing each other and nuclear theory. By doing all these steps in parallel, the timeline to deliver improved nuclear data to users will significantly be reduced. This work will focus on the integral experiment final design and the balance of design and modeling simplicity while minimizing experiment uncertainty.
Error mitigation techniques are crucial to achieving near-term quantum advantage. Classical postprocessing of quantum computation outcomes is a popular approach for error mitigation, which includes methods, such as zero noise extrapolation, virtual distillation, and learning-based error mitigation. However, these techniques have limitations due to the propagation of uncertainty resulting from the finite shot number of a quantum measurement. In this work, we introduce general and unbiased methods for quantifying the uncertainty and error of error-mitigated observables based on the strategic sampling of error mitigation outcomes. We then extend our approach to demonstrate the optimization of performance and robustness of error mitigation under uncertainty. To illustrate our methods, we apply them to zero noise extrapolation and Clifford date regression in the ground state of the XY model simulated using depolarizing and International Business Machines Corporation (IBM) Toronto noise models, respectively. In particular, we optimize the choice of noise levels and the allocation of shots for zero noise extrapolation and the distribution of the training circuits for Clifford data regression. While our methods are readily applicable to any postprocessing-based error mitigation approach, in practice they must not be prohibitively expensive—even though they perform optimizations of the error mitigation hyperparameters requiring sampling of a statistical distribution of error mitigation outcomes. By leveraging surrogate-based optimization, we show that our methods can efficiently perform optimal design for a zero noise extrapolation implementation. We then further demonstrate the transferability of learned zero noise extrapolation hyperparameters to other similar circuits.
Not Available
Uncertainty is an important aspect to data understanding. Without awareness of the variability, error, or reliability of a dataset, the ability to make decisions on that data is limited. However, practices around uncertainty visualization remain domain-specific, rooted in convention, and in many instances, absent entirely. Part of the reason for this may be a lack of established guidelines for navigating difficult choices of when uncertainty should be added, how to visualize uncertainty, and how to evaluate its effectiveness. Unsurprisingly, the inclusion of uncertainty into visualizations is a major challenge to visualization. As work concerned with uncertainty visualization grows, it has become clear that simple visual additions of uncertainty information to traditional visualization methods do not appropriately convey the meaning of the uncertainty, pose many perceptual challenges, and, in the worst case, can lead a viewer to a completely wrong understanding of the data. These challenges are the driving motivator for this special issue.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
PSUADE (Problem Solving testbed for Uncertainty Analysis and Design Exploration) is a mathematical software useful for performing uncertainty quantification and sensitivity analysis.