SEARCH · Search NASA

Results for “uncertainty in complex systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

More Data Needed for Failure Rate Estimation, Validation, and Uncertainty Reduction

Current Environmental Control and Life Support System (ECLSS) development and test activities are not generating data fast enough to provide statistically-supportable precise Orbital Replacement Unit (ORU) failure rate estimates for future missions. Accurate and precise failure rate estimates are critical for missions beyond Low Earth Orbit (LEO) because current risk mitigation approaches – namely regular resupply and rapid abort capabilities – will not be available. Safe operations will depend on mission planners’ ability to accurately forecast spares demand and efficiently provide the necessary resources. However, even after more than a decade of operations on board the International Space Station (ISS), a significant amount of uncertainty remains in failure rate estimates. Uncertain or inaccurate failure rates result in increased risk and spares mass for future missions. A Bayesian failure rate estimation approach, such as the one currently implemented by the ISS Program, can help reduce uncertainty by incorporating engineering judgement into failure rate estimates. However, experience on the ISS and with other complex systems shows that these prior failure rate estimates are often inaccurate. In addition, prior failure rate estimates are typically point values; some level of uncertainty must be added to convert these into probability distributions for Bayesian updating, and there are several potential methods for doing so. Due to the low rate of data collection, these subjective (and often inaccurate) prior estimates currently have a strong influence on the end result. This paper examines the challenges associated with failure rate estimation, validation, and uncertainty reduction in the context of ECLSS development for beyond-LEO missions. A variety of techniques for generating and updating Bayesian priors are discussed and evaluated using both real-world and simulated data. Potential solutions for improving failure rate estimation, including testing additional units, are analyzed and discussed, and a set of recommendations are provided for next-generation system development activities.

Supportability↗

DOE OSTI2023

Increased exposure of coastal cities to sea-level rise due to internal climate variability

Adaptation to future sea-level rise is based on projections of continuously improving climate models. These projections are accompanied by inherent uncertainties, including those due to internal climate variability (ICV). The ICV arises from complex and unpredictable interactions within and between climate-system components, rendering its impact irreducible. Although neglecting this uncertainty can lead to an underestimation of future sea-level rise, its estimation and impacts have not been fully explored. Combining the Community Earth System Model version 1 Large Ensemble experiments with power-law statistics, we show that, by 2100, if the ICV uncertainty reaches its upper limit, new sea-level-rise hotspots would appear in Southeast Asian megacities (Chennai, Kolkata, Yangon, Bangkok, Ho Chi Minh City and Manila), in western tropical Pacific Islands and the Western Indian Ocean. Here, the better the ICV uncertainty is taken into account and correctly estimated, the more effective adaptation strategies can be elaborated with confidence and actions to follow.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Uncertainty-Aware, Structure-Preserving Machine Learning Approach for Domain Shift Detection From Nonlinear Dynamic Responses of Structural Systems

Complex structural systems deployed for aerospace, civil, or mechanical applications must operate reliably under varying operational conditions. Structural health monitoring (SHM) systems help ensure the reliability of these systems by providing continuous monitoring of the state of the structure. SHM relies on synthesizing measured data with a predictive model to make informed decisions about structural states. However, these models—which may be thought of as a form of a digital twin—need to be updated continuously as structural changes (e.g., due to damage) arise. We propose an uncertainty-aware machine learning model that enforces distance preservation of the original input state space and then encodes a distance-aware mechanism via a Gaussian process (GP) kernel. The proposed approach leverages the spectral-normalized neural GP algorithm to combine the flexibility of neural networks with the advantages of GP, subjected to structure-preserving constraints, to produce an uncertainty-aware model. This model is used to detect domain shift due to structural changes that cannot be observed directly because they may be spatially isolated (e.g., inside a joint or localized damage). This work leverages detection theory to detect domain shift systematically given statistical features of the prediction variance produced by the model. The proposed approach is demonstrated on a nonlinear structure being subjected to damage conditions. In conclusion, it is shown that the proposed approach is able to rely on distances of the transformed input state space to predict increased variance in shifted domains while being robust to normative changes.

Algorithms↗

DOE OSTI2024

Advances on CHP District Energy and Microgrids Deployment: Simplified Tool for Rapidly Deploying Feasibility Analytics for the Non-Technical User (Final Technical Report)

Community energy systems have proven to have the potential to improve cost efficiency, resilience, and decarbonize. However, investing in community energy systems such as community microgrids or district energy systems is a complex decision due to the high initial investment and the uncertainties associated with the long development time and lifecycle of the project. Tools that make feasibility assessments accessible to non-technical users like investors, policymakers, and other stakeholders will result in more feasibility analyses completed, more candidate projects identified, and more community energy systems deployed. The pilot tool developed under this award is named Energy Fellow. Energy Fellow allows technical and non-technical users to complete feasibility analyses for district energy systems and community microgrids. This is the first software tool of its kind designed for non-technical users and available at no cost. Its scope was adjusted to a 25x25-mile region within the Houston area in Texas to make its development compatible with the funding available. However, the findings and models developed make this pilot tool easily scalable to the US. The lessons learned during the design, implementation, and testing stages have helped find trade-off solutions to software and hardware challenges related to implementing 3D models in online tools. Green software strategies has been successfully applied to the design and operations of the tool, and the team has researched the aspects of the (non-technical) user experience that will make commercial developments of this tool even more impactful.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2024

A variance deconvolution estimator for efficient uncertainty quantification in Monte Carlo radiation transport applications

Monte Carlo simulations are at the heart of many high-fidelity simulations and analyses for radiation transport systems. As is the case with any complex computational model, it is important to propagate sources of input uncertainty and characterize how they affect model output. Unfortunately, uncertainty quantification (UQ) is made difficult by the stochastic variability that Monte Carlo transport solvers introduce. The standard method to avoid corrupting the UQ statistics with the transport solver noise is to increase the number of particle histories, resulting in very high computational costs. In this contribution, we propose and analyze a sampling estimator based on the law of total variance to compute UQ variance even in the presence of residual noise from Monte Carlo transport calculations. Here, we rigorously derive the statistical properties of the new variance estimator, compare its performance to that of the standard method, and demonstrate its use on neutral particle transport model problems involving both attenuation and scattering physics. We illustrate, both analytically and numerically, the estimator’s statistical performance as a function of available computational budget and the distribution of that budget between UQ samples and particle histories. We show analytically and corroborate numerically that the new estimator is unbiased, unlike the standard approach, and is more accurate and precise than the standard estimator for the same computational budget.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Advancing uncertainty characterization for understanding projected water scarcity in multi-sector, multi-actor river basins across scales.

Invited Talk at AGU Fall Meeting 2023 Modeling how human institutions and infrastructure interact with the water cycle is essential to better understand the vulnerability, and resilience of water resources systems from the local to the global scale. This is especially true when investigating multi-sector, multi-actor responses to the effects of long-term changes and short-term shocks. It is also well recognized that uncertainty present throughout the modeling cycle (e.g., in data, functional relations, and model coupling approaches) limits our ability to trace the interactive dynamics of human-water systems, as well as to quantify their implications for management and planning. Systems with large numbers of diverse stakeholders further compound this challenge, as uncertain drivers and complex dynamics might have very disparate effects on water users. The work presented is conducted through the Integrated Multisector Multiscale Modeling (IM3) Science Focus Area, which explores how human and natural systems co-evolve in response to change. Through this multi-year effort, we have developed exploratory modeling methods to better characterize how the uncertain human and natural drivers of water scarcity yield consequential vulnerabilities in institutionally complex, multi-sectoral systems. This talk will specifically present on a series of uncertainty characterization experiments performed in the Upper Colorado River Basin, a sub-basin of the Colorado with thousands of water users. These complementary and systematic experiments are aimed at understanding: How are various uncertain stressors (e.g., climate change, demand growth) affecting the diverse water users of this basin in terms of water shortage? What are the key drivers of this shortage for each user? Methods and results from this work are used to address additional questions on the ability of adaptation to modulate the effects of these uncertain drivers, and on their compounding effects across spatial scales and through sectoral interactions.

Hadjimichael, Antonia↗

DOE OSTI2025

Navigating Uncertainty: Challenges in Visualizing Ensemble Data and Surrogate Models for Decision Systems

Uncertainty visualization plays a critical role in transforming ensemble simulation data into actionable insights by effectively communicating various dimensions of uncertainty within a system. The emergence of artificial intelligence-driven surrogate models trained on multirun ensemble data offers a transformative opportunity to replace computationally intensive simulations with fast estimates, enabling users to explore data spaces with unprecedented depth and interactivity. However, integrating ensemble data and surrogate models into decision-making workflows and tools introduces novel challenges for uncertainty visualization. These include reconciling and clearly communicating the unique uncertainties associated with ensembles and their surrogate model estimates, and leveraging these approximations to inform actionable decisions. This work explores these challenges in the context of high-dimensional data visualization, bridging discrete datasets with their continuous representations and addressing the complexities of systems that support iterative navigation between input and output spaces. We evaluate the role of uncertainty visualization in fostering intuitive, actionable interactions and identify critical hurdles in advancing this frontier of computational simulation.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Infrastructure first principles for the Anthropocene

Abstract There appears to be a growing decoupling between the conditions that infrastructures were designed for and today’s rapidly changing environments. Infrastructures today are largely predicated on the technologies, goals, and governance structures from a century ago. While infrastructures continue to deliver untold value, there is growing evidence that these critical, basic, and lifeline systems appear ill-equipped to confront the volatility, uncertainty, accelerating conditions, and complexity that define them and their changing environments. Innovative and disruptive first principles are needed to guide infrastructures in the Anthropocene. Drawing from emerging infrastructure research and disciplines that appear better able to confront disruption and change, a novel set of first principles are identified: (1) Plan for complex conditions and surprise; (2) Recouple with agility and flexibility; (3) Govern for exploration and instability; (4) Build consensus as control decentralizes; (5) Restructure to engage with porous boundaries; and, (6) Cyberthreat planning is now mission critical. These principles should guide infrastructure planning recognizing the changing nature and increasingly obsolete boundaries that have defined engineered systems in the modern era.

Chester, Mikhail (ORCID:0000000293542102)↗

DOE OSTI2021

Analysis of secondary organic aerosol simulation bias in the Community Earth System Model (CESM2.1)

Organic aerosol (OA) has been considered as one of the most important uncertainties in climate modeling due to the complexity in presenting its chemical production and depletion mechanisms. To better understand the capability of climate models and probe into the associated uncertainties in simulating OA, we evaluate the Community Earth System Model version 2.1 (CESM2.1) configured with the Community Atmosphere Model version 6 (CAM6) with comprehensive tropospheric and stratospheric chemistry representation (CAM6-Chem) through a long-term simulation (1988–2019) with observations collected from multiple datasets in the United States. We find that CESM generally reproduces the interannual variation and seasonal cycle of OA mass concentration at surface layer with a correlation of 0.40 compared to ground observations and systematically overestimates (69%) in summer and underestimates (-19%) in winter. Through a series of sensitivity simulations, we reveal that modeling bias is primarily related to the dominant fraction of monoterpene-formed secondary organic aerosol (SOA), and a strong positive correlation of 0.67 is found between monoterpene emission and modeling bias in the eastern US during summer. In terms of vertical profile, the model prominently underestimates OA and monoterpene concentrations by 37%–99% and 82%–99%, respectively, in the upper air (>500 m) as validated against aircraft observations. Our study suggests that the current volatility basis set (VBS) scheme applied in CESM might be parameterized with monoterpene SOA yields that are too high, which subsequently results in strong SOA production near the emission source area. We also find that the model has difficulty in reproducing the decreasing trend of surface OA in the southeastern US probably because of employing pure gas VBS to represent isoprene SOA which is in reality mainly formed through multiphase chemistry; thus, the influence of aerosol acidity and sulfate particle change on isoprene SOA formation has not been fully considered in the model. This study reveals the urgent need to improve the SOA modeling in climate models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

NASA NTRS2007

Probabilistic Reasoning for Robustness in Automated Planning

A general-purpose computer program for planning the actions of a spacecraft or other complex system has been augmented by incorporating a subprogram that reasons about uncertainties in such continuous variables as times taken to perform tasks and amounts of resources to be consumed. This subprogram computes parametric probability distributions for time and resource variables on the basis of user-supplied models of actions and resources that they consume. The current system accepts bounded Gaussian distributions over action duration and resource use. The distributions are then combined during planning to determine the net probability distribution of each resource at any time point. In addition to a full combinatoric approach, several approximations for arriving at these combined distributions are available, including maximum-likelihood and pessimistic algorithms. Each such probability distribution can then be integrated to obtain a probability that execution of the plan under consideration would violate any constraints on the resource. The key idea is to use these probabilities of conflict to score potential plans and drive a search toward planning low-risk actions. An output plan provides a balance between the user s specified averseness to risk and other measures of optimality.

Schaffer, Steven↗

NASA NTRS2018

Evaluating the Assumptions in an Empirical Jet-Surface Interaction Noise Model

A set of empirical jet-surface interaction noise models, developed for single-stream round nozzles exhausting over a simple surface in a static ambient, are evaluated for use in more realistic applications that include multi-stream nozzle systems, multi-plane surface geometries, and a flight-stream. The simple-single-stream models have several advantages when used in system-level noise studies: they are robust, they are quickly computed, and they are generally applicable to a wide range of configurations. However, these models require simplifying assumptions when applied to more complex jet exhaust systems; for example, previous work on multi-stream jets used an empirical formula to compute a single-stream equivalent jet potential core length that could be used to predict the noise using simple-single-stream jet-surface interaction models. This paper considers the effect of flight and multi-plane surfaces using a similar approach: introducing assumptions to simplify the complex system, applying the simple-single-stream models, and evaluating the uncertainty.

Jet Noise↗

NASA NTRS2018

Evaluating the Assumptions in an Empirical Jet-Surface Interaction Noise Model

A set of empirical jet-surface interaction noise models, developed for single-stream round nozzles exhausting over a simple surface in a static ambient, are evaluated for use in more realistic applications that include multi-stream nozzle systems, multi-plane surface geometries, and a flight-steam. The simple-single-stream models have several advantages when used in system-level noise studies: they are robust, they are quickly computed, and they are generally applicable to a wide range of configurations. However, these models require simplifying assumptions when applied to more complex jet exhaust systems; for example, previous work on multi-stream jets used an empirical formula to compute a single-stream equivalent jet potential core length that could be used to predict the noise using simple-single-stream jet-surface interaction models. This paper considers the effect of flight and multi-plane surfaces using a similar approach: introducing assumptions to simplify the complex system, applying the simple-single-stream models, and evaluating the uncertainty.

Jet Noise↗

DOE OSTI2022

On the characterization of biases arising from methods and approximations used for sensitivity analyses

The preliminary assessment of the performance and safety of advanced reactors, as well as the identification of research and development needs, rely on computer simulations due to the lack of operational experience. Those simulations should be based on verified and validated computational tools, calculation schemes, and nuclear data libraries, and should be accompanied of a proper characterization of the involved uncertainties, providing confidence in the computational-based predictions. Thus, sensitivity and uncertainty studies, together with integral experiments, play an essential role in that process. Sensitivities are then a critical element and guidelines about how to produce accurate enough sensitivities depending on the subsequent analyses to be performed are of interest. The present study addresses the sensitivities biases arising from the use of different nuclear data libraries, computational methods and the assumption of modelling simplifications. Not only sensitivities for multiplication factor but also for safety-relevant reactivity responses have been analyzed. In the study, propagated uncertainties are also included for the considered parameters to illustrate the impact of the sensitivity profiles obtained for each case on the final uncertainty. Then, relevant recommendations are given for sensitivity analyses of highly complex systems. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

NASA NTRSDate not supplied

Reduced-order Modeling for Spacecraft Thermal-structural Applications

Leveraging the speed of reduced-order models (ROMs), thermal analysis teams have previously had access to rapid optimization, sensitivity studies, rapid model correlation, and uncertainty quantification, to name a few. An advantage of the developed approach is its relative robustness, which enabled expansion to include structural models in addition to thermal models leading to both rapid thermal-structural and rapid structural-only analyses. This method was tested for a sample thermal-structural application and showed good performance. It was then applied to a complex NASA mission and showed how ROMs could be used to quickly evaluate system uncertainties. Finally, these methods were applied to a structural-only application and used to better optimize and understand the structural design.

Derek Hengeveld↗

DOE OSTI2020

A Hybrid Stochastic-Interval Operation Strategy for Multi-Energy Microgrids

Increasing deployment of distributed energy resources intensifies interactions among electric, gas, heating, and cooling systems. Inherent uncertainties ranging from internal parameters to external inputs also impose significant challenges on system operation. A coordinated scheduling method is, therefore, desired for facilitating efficient operation of multi-energy microgrids under multiple uncertainties. In this paper, a deterministic coordinated scheduling model is developed first with detailed modeling of system dynamics and device parameters to fully characterize complex interactions among multi-energy carriers. Then, a hybrid stochastic-interval method is proposed to account for heterogeneous uncertainties in decisions of operational strategies. Specifically, performance and efficiency uncertainties of distributed energy resources, and injection uncertainty of renewable sources and multienergy demands are characterized by interval-based uncertain operation regions and probability distributions, respectively. A scenario-based two-stage algorithm is developed to solve the problem so that multiple uncertainties are preserved in the entire decision-making process. Moreover, adjustable operational strategies are determined to hedge uncertainty by considering operators’ risk preferences. The validity of the proposed method is verified by comprehensive case studies on a test system.

hybrid stochastic-interval approach↗

NASA NTRS2017

Coupled Data Assimilation for Integrated Earth System Analysis and Prediction: Goals, Challenges, and Recommendations

The purpose of this report is to identify fundamental issues for coupled data assimilation (CDA), such as gaps in science and limitations in forecasting systems, in order to provide guidance to the World Meteorological Organization (WMO) on how to facilitate more rapid progress internationally. Coupled Earth system modeling provides the opportunity to extend skillful atmospheric forecasts beyond the traditional two-week barrier by extracting skill from low-frequency state components such as the land, ocean, and sea ice. More generally, coupled models are needed to support seamless prediction systems that span timescales from weather, subseasonal to seasonal (S2S), multiyear, and decadal. Therefore, initialization methods are needed for coupled Earth system models, either applied to each individual component (called Weakly Coupled Data Assimilation - WCDA) or applied the coupled Earth system model as a whole (called Strongly Coupled Data Assimilation - SCDA). Using CDA, in which model forecasts and potentially the state estimation are performed jointly, each model domain benefits from observations in other domains either directly using error covariance information known at the time of the analysis (SCDA), or indirectly through flux interactions at the model boundaries (WCDA). Because the non-atmospheric domains are generally under-observed compared to the atmosphere, CDA provides a significant advantage over single-domain analyses. Next, we provide a synopsis of goals, challenges, and recommendations to advance CDA: Goals: (a) Extend predictive skill beyond the current capability of NWP (e.g. as demonstrated by improving forecast skill scores), (b) produce physically consistent initial conditions for coupled numerical prediction systems and reanalyses (including consistent fluxes at the domain interfaces), (c) make best use of existing observations by allowing observations from each domain to influence and improve the full earth system analysis, (d) develop a robust observation-based identification and understanding of mechanisms that determine the variability of weather and climate, (e) identify critical weaknesses in coupled models and the earth observing system, (f) generate full-field estimates of unobserved or sparsely observed variables, (g) improve the estimation of the external forcings causing changes to climate, (h) transition successes from idealized CDA experiments to real-world applications. Challenges: (a) Modeling at the interfaces between interacting components of coupled Earth system models may be inadequate for estimating uncertainty or error covariances between domains, (b) current data assimilation methods may be insufficient to simultaneously analyze domains containing multiple spatiotemporal scales of interest, (c) there is no standardization of observation data or their delivery systems across domains, (d) the size and complexity of many large-scale coupled Earth system models makes it is difficult to accurately represent uncertainty due to model parameters and coupling parameters, (e) model errors lead to local biases that can transfer between the different Earth system components and lead to coupled model biases and long-term model drift, (e) information propagation across model components with different spatiotemporal scales is extremely complicated, and must be improved in current coupled modeling frameworks, (h) there is insufficient knowledge on how to represent evolving errors in non-atmospheric model components (e.g. as sea ice, land and ocean) on the timescales of NWP.

CDA↗

DOE OSTI2021

Modular hybrid modeling to increase efficiency, explore structural uncertainty, and allow multidimensional complexity scaling in land surface models

Land surface models (LSMs) are indispensable tools for predicting hydrologic extremes, as well as a particularly uncertain component of Earth system models that has stubbornly resisted the convergence in projections over several successive generations of model intercomparisons. This uncertainty in LSMs is poorly quantified and poorly attributed to specific processes, which has hampered efforts to focus research in reducing uncertainty. This has resulted from sparse sampling of the possible uncertainty space—which is high-dimensional and has contributions from parametric, structural, initial, and boundary condition uncertainties—as an artifact of CMIP-type ensembles of opportunity and limitations inherent in observational benchmarks. A new approach is needed to understand and reduce this uncertainty, based around individual LSMs that can represent the breadth of assumptions represented in current CMIP-type efforts, while at the same time exploring that uncertainty in a systematic way, confronting multiple types of observations, and where justified, replacing process representations with ML-driven emulators. We propose an approach of modular hybrid modeling to address these challenges.

58 GEOSCIENCES↗

DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗