Results for “inference methods” — Page 27

DOE OSTI2017

UQTk Version 3.0.4 User Manual

The UQ Toolkit (UQTk) is a collection of libraries and tools for the quantification of uncertainty in numerical model predictions. Version 3.0.4 offers intrusive and non-intrusive methods for propagating input uncertainties through computational models, tools for sensitivity analysis, methods for sparse surrogate construction, and Bayesian inference tools for inferring parameters from experimental data. This manual discusses the download and installation process for UQTk, provides pointers to the UQ methods used in the toolkit, and describes some of the examples provided with the toolkit.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

UQTk Version 3.1.2 User Manual

The UQ Toolkit (UQTk) is a collection of libraries and tools for the quantification of uncertainty in numerical model predictions. Version 3.1.2 offers intrusive and non-intrusive methods for propagating input uncertainties through computational models, tools for sensitivity analysis, methods for sparse surrogate construction, and Bayesian inference tools for inferring parameters from experimental data. This manual discusses the download and installation process for UQTk, provides pointers to the UQ methods used in the toolkit, and describes some of the examples provided with the toolkit.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1993

Object/rule integration in CLIPS

This paper gives a brief overview of the C Language Integrated Production System (CLIPS) with a focus on the object-oriented features. The advantages of an object data representation over the traditional working memory element (WME), i.e., facts, are discussed, and the implementation of the Rete inference algorithm in CLIPS is presented in detail. A few methods for achieving pattern-matching on objects with the current inference engine are given, and finally, the paper examines the modifications necessary to the Rete algorithm to allow direct object pattern-matching.

Donnell, Brian L.↗

NASA NTRS2018

An Artificial Neural Network for Inferring Solar Wind Proxies at Mars

We present a novel method to determine solar wind proxies from sheath measurements at Mars. Specifically, we develop an artificial neural network (ANN) to simultaneously infer seven solar wind proxies: ion density, ion speed, ion temperature, and interplanetary magnetic field magnitude and its vector components, using spacecraft measurements of ion moments, magnetic field magnitude, magnetic field components in the sheath, and the solar extreme ultraviolet flux. The ANN was trained and tested using3 years of data from the Mars Atmosphere and Volatile EvolutioN (MAVEN) spacecraft. When compared with MAVEN spacecraft's in situ measured values of the solar wind parameters, we find that the ANN proxies for the solar wind ion density, ion speed, ion temperature, and interplanetary magnetic field magnitude havepercentage differences of 50% or less for 84.4%, 99.9%, 86.8%, and 79.8% of the instances, respectively. Forthe cone angle and clock angle proxies, 69.1% and 53.3% of instances, respectively, have angle differences of 30* or less.

Neural Network↗

DOE OSTI2023

Neural networks for parameter estimation in intractable models

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Advanced data analysis in inertial confinement fusion and high energy density physics

Bayesian analysis enables flexible and rigorous definition of statistical model assumptions with well-characterized propagation of uncertainties and resulting inferences for single-shot, repeated, or even cross-platform data. This approach has a strong history of application to a variety of problems in physical sciences ranging from inference of particle mass from multi-source high-energy particle data to analysis of black-hole characteristics from gravitational wave observations. The recent adoption of Bayesian statistics for analysis and design of high-energy density physics (HEDP) and inertial confinement fusion (ICF) experiments has provided invaluable gains in expert understanding and experiment performance. In this Review, we discuss the basic theory and practical application of the Bayesian statistics framework. We highlight a variety of studies from the HEDP and ICF literature, demonstrating the power of this technique. Due to the computational complexity of multi-physics models needed to analyze HEDP and ICF experiments, Bayesian inference is often not computationally tractable. Two sections are devoted to a review of statistical approximations, efficient inference algorithms, and data-driven methods, such as deep-learning and dimensionality reduction, which play a significant role in enabling use of the Bayesian framework. We provide additional discussion of various applications of Bayesian and machine learning methods that appear to be sparse in the HEDP and ICF literature constituting possible next steps for the community. We conclude by highlighting community needs, the resolution of which will improve trust in data-driven methods that have proven critical for accelerating the design and discovery cycle in many application areas.

47 OTHER INSTRUMENTATION↗

NASA NTRS1985

Estimating the R-curve from residual strength data

A method is presented for estimating the crack-extension resistance curve (R-curve) from residual-strength (maximum load against original crack length) data for precracked fracture specimens. The method allows additional information to be inferred from simple test results, and that information can be used to estimate the failure loads of more complicated structures of the same material and thickness. The fundamentals of the R-curve concept are reviewed first. Then the analytical basis for the estimation method is presented. The estimation method has been verified in two ways. Data from the literature (involving several materials and different types of specimens) are used to show that the estimated R-curve is in good agreement with the measured R-curve. A recent predictive blind round-robin program offers a more crucial test. When the actual failure loads are disclosed, the predictions are found to be in good agreement.

Orange, T. W.↗

DOE OSTI2019

Scalable and efficient learning from crowds with Gaussian processes

Over the last few years, multiply-annotated data has become a very popular source of information. Online platforms such as Amazon Mechanical Turk have revolutionized the labelling process needed for any classification task, sharing the effort between a number of annotators (instead of the classical single expert). This crowdsourcing approach has introduced new challenging problems, such as handling disagreements on the annotated samples or combining the unknown expertise of the annotators. Probabilistic methods, such as Gaussian Processes (GP), have proven successful to model this new crowdsourcing scenario. However, GPs do not scale up well with the training set size, which makes them prohibitive for medium-to-large datasets (beyond 10K training instances). This constitutes a serious limitation for current real-world applications. In this work, we introduce two scalable and efficient GP-based crowdsourcing methods that allow for processing previously-prohibitive datasets. The first one is an efficient and fast approximation to GP with squared exponential (SE) kernel. The second allows for learning a more flexible kernel at the expense of a heavier training (but still scalable to large datasets). Since the latter is not a GP-SE approximation, it can be also considered as a whole new scalable and efficient crowdsourcing method, useful for any dataset size. Both methods use Fourier features and variational inference, can predict the class of new samples, and estimate the expertise of the involved annotators. A complete experimentation compares them with state-of-the-art probabilistic approaches in synthetic and real crowdsourcing datasets of different sizes. Finally, they stand out as the best performing approach for large scale problems. Moreover, the second method is competitive with the current state-of-the-art for small datasets.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

NASA NTRS1976

Remote probing of atmospheric particulates from radiation extinction experiments: A review of methods

The existing methodology for reconstructing the particle size distribution and inferring the refractive index of absorbing and scattering atmospheric particulates is critically reviewed. Emphasis is placed on method capabilities and shortcomings and, wherever possible, on achievable accuracy. The nature of the associated remote probing problem is analyzed with regard to the effects of the particulates on EM wave propagation in the atmosphere. The parameterization of size distribution is studied within the unifying framework of Pearson's distribution curves. The inversions of extinction measurements and their ratios are considered separately, and the potentialities of each type of measurement are identified. Work lacking in each of the methods reviewed is indicated. A method of determining both the effective complex refractive index and size distribution model parameters from the same data is also presented. Lastly, determination from extinction ratio data of the complex refractive index independent of size distribution is discussed and error analyzed.

Fymat, A. L.↗

NASA NTRS1982

Time Domain Identification of an Optimal Control Pilot Model with Emphasis on the Objective Function

A method for the identification of the pilot's control compensation using time domain techniques is proposed. From this information we hope to infer a quadratic cost function, supported by the data, that represents a reasonable expression for the pilot's control objective in the task being performed, or an inferred piloting strategy. The objectives for this method are: (1) obtain a better understanding of the fundamental piloting techniques in complex tasks, such as landing approach; (2) the development of a metric measurable in simulations and flight test that correlate with subjective pilot opinion; and (3) to further validate pilot models and pilot vehicle analysis methods.

Schmidt, D. K.↗

DOE OSTI2019

Active Subspace Methods for Data-Intensive Inverse Problems (Final Report)

Fostered by advances in computing, simulation and data science have emerged as the third and fourth paradigms enabling scientific discovery. Many mathematical challenges arise when integrating these paradigms to produce new knowledge. One of the most important challenges is the inverse or calibration problem: find the inputs of a simulation such that its outputs agree with a given set of measurements. As both the cost of the complex simulations and the volume of measurement data sets increase, intuition-driven trial-and-error methods for calibration quickly become untenable; modern scientists need more computationally efficient and mathematically rigorous methods. When measurements and simulated predictions contain uncertainty, the Bayesian perspective on statistical inference provides a popular mathematical framework for developing computational methods to solve inverse problems. Markov chain Monte Carlo (MCMC) methods are a mainstay of this framework; they characterize a probability distribution on the space of model inputs that is conditioned on the given measurements. However, despite recent advances, MCMC methods remain impractical for expensive simulations with more than a handful of inputs. At its heart, MCMC searches the space of inputs for corresponding outputs that match measurements, and so it is afflicted with the curse of dimensionality. Additionally, MCMC methods become less efficient as the volume of the measurement data increases, particularly if the measurements are redundant. To make MCMC—and Bayesian methods, in general—amenable to today’s computer models with millions of inputs and terabytes of measurement data, one must devise methods to reduce the dimension of both the simulation inputs and the measurements. For many models used in practice, the outputs vary primarily on a low-dimensional manifold of the nominally high-dimensional input space. Recently developed active subspace methods discover this low-dimensional manifold and exploit it to enable analyses typically suited for low-dimensional settings (e.g., statistical inversion of complex systems). We hypothesize that active subspace methods can provide the necessary dimension reduction, enabling existing Bayesian methods to find solutions for a large class of otherwise intractable statistical inverse problems. We will test the hypothesis with a series of mathematical analyses and computational experiments that will proceed along three research veins: (i) integrating active subspace methods into methods for statistical inversion, (ii) addressing current challenges in active subspace methods, and (iii) incorporating methods for nonlinear dimensionality reduction into active subspace methods. We will apply the methods we develop to real inverse problems in chemical kinetics and turbulent flame modeling. Our team combines expertise in methods for inverse problems, dimension reduction, and reduced order modeling. With a history of active collaboration and thoughtful engagement, we bring enthusiasm and fresh perspectives to a well studied subject. We are excited by the potential opportunity to work together to extend and apply active subspace methods to data-intensive inverse problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Active Subspace Methods for Data-Intensive Inverse Problems

Fostered by advances in computing, simulation and data science have emerged as the third and fourth paradigms enabling scientific discovery. Many mathematical challenges arise when integrating these paradigms to produce new knowledge. One of the most important challenges is the inverse or calibration problem: find the inputs of a simulation such that its outputs agree with a given set of measurements. As both the cost of the complex simulations and the volume of measurement data sets increase, intuition-driven trial-and-error methods for calibration quickly become untenable; modern scientists need more computationally efficient and mathematically rigorous methods. When measurements and simulated predictions contain uncertainty, the Bayesian perspective on statistical inference provides a popular mathematical framework for developing computational methods to solve inverse problems. Markov chain Monte Carlo (MCMC) methods are a mainstay of this framework; they characterize a probability distribution on the space of model inputs that is conditioned on the given measurements. However, despite recent advances, MCMC methods remain impractical for expensive simulations with more than a handful of inputs. At its heart, MCMC searches the space of inputs for corresponding outputs that match measurements, and so it is afflicted with the curse of dimensionality. Additionally, MCMC methods become less efficient as the volume of the measurement data increases, particularly if the measurements are redundant. To make MCMC—and Bayesian methods, in general—amenable to today’s computer models with millions of inputs and terabytes of measurement data, one must devise methods to reduce the dimension of both the simulation inputs and the measurements. For many models used in practice, the outputs vary primarily on a low-dimensional manifold of the nominally high-dimensional input space. Recently developed active subspace methods discover this low-dimensional manifold and exploit it to enable analyses typically suited for low-dimensional settings (e.g., statistical inversion of complex systems). We hypothesize that active subspace methods can provide the necessary dimension reduction, enabling existing Bayesian methods to find solutions for a large class of otherwise intractable statistical inverse problems. We will test the hypothesis with a series of mathematical analyses and computational experiments that will proceed along three research veins: (i) integrating active subspace methods into methods for statistical inversion, (ii) addressing current challenges in active subspace methods, and (iii) incorporating methods for nonlinear dimensionality reduction into active subspace methods. We will apply the methods we develop to real inverse problems in chemical kinetics and turbulent flame modeling. Our team combines expertise in methods for inverse problems, dimension reduction, and reduced order modeling. With a history of active collaboration and thoughtful engagement, we bring enthusiasm and fresh perspectives to a well studied subject. We are excited by the potential opportunity to work together to extend and apply active subspace methods to data-intensive inverse problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

An expectation–maximization framework for comprehensive prediction of isoform-specific functions

Advances in RNA sequencing technologies have achieved an unprecedented accuracy in the quantification of mRNA isoforms, but our knowledge of isoform-specific functions has lagged behind. There is a need to understand the functional consequences of differential splicing, which could be supported by the generation of accurate and comprehensive isoform-specific gene ontology annotations. We present isoform interpretation, a method that uses expectation–maximization to infer isoform-specific functions based on the relationship between sequence and functional isoform similarity. We predicted isoform-specific functional annotations for 85 617 isoforms of 17 900 protein-coding human genes spanning a range of 17 430 distinct gene ontology terms. Comparison with a gold-standard corpus of manually annotated human isoform functions showed that isoform interpretation significantly outperforms state-of-the-art competing methods. We provide experimental evidence that functionally related isoforms predicted by isoform interpretation show a higher degree of domain sharing and expression correlation than functionally related genes. We also show that isoform sequence similarity correlates better with inferred isoform function than with gene-level function.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2026

Hierarchical Gaussian Random Field Sampling for Multilevel Markov Chain Monte Carlo: Coupling Stochastic Partial Differential Equation and the Karhunen–Loève Decomposition

This work introduces structure preserving hierarchical decompositions for sampling Gaussian random fields (GRFs) within the context of multilevel Bayesian inference in high-dimensional space. Existing scalable hierarchical sampling methods, such as those based on stochastic partial differential equations (SPDEs), often reduce the dimensionality of the sample space at the cost of accuracy of inference. Other approaches, such that those based on Karhunen-Loève (KL) expansions, offer sample space dimensionality reduction but sacrifice GRF representation accuracy and ergodicity of the Markov chain Monte Carlo (MCMC) sampler and are computationally expensive for high-dimensional problems. The proposed method integrates the dimensionality reduction capabilities of KL expansions with the scalability of SPDE-based sampling, thereby providing a robust, unified framework for high-dimensional uncertainty quantification (UQ) that is scalable and accurate, preserves ergodicity, and offers dimensionality reduction of the sample space. The hierarchy in our multilevel algorithm is derived from the geometric multigrid hierarchy. By constructing a hierarchical decomposition that maintains the covariance structure across the levels in the hierarchy, the approach enables efficient coarse-to-fine sampling while ensuring that all samples are drawn from the desired distribution. The effectiveness of the proposed method is demonstrated on a benchmark subsurface flow problem, demonstrating its effectiveness in improving computational efficiency and statistical accuracy. Furthermore, our proposed technique is more efficient and accurate and displays better convergence properties than existing methods for high-dimensional Bayesian inference problems.

Gaussian random fields↗

DOE OSTI2023

Population of Merging Compact Binaries Inferred Using Gravitational Waves through GWTC-3

We report on the population properties of compact binary mergers inferred from gravitational-wave observations of these systems during the first three LIGO-Virgo observing runs. The Gravitational-Wave Transient Catalog 3 (GWTC-3) contains signals consistent with three classes of binary mergers: binary black hole, binary neutron star, and neutron star–black hole mergers. We infer the binary neutron star merger rate to be between 10 and 1700 Gpc − 3 yr − 1 and the neutron star–black hole merger rate to be between 7.8 and 140 Gpc − 3 yr − 1 , assuming a constant rate density in the comoving frame and taking the union of 90% credible intervals for methods used in this work. We infer the binary black hole merger rate, allowing for evolution with redshift, to be between 17.9 and 44 Gpc − 3 yr − 1 at a fiducial redshift ( z = 0.2 ). The rate of binary black hole mergers is observed to increase with redshift at a rate proportional to ( 1 + z ) κ with κ = 2. 9 − 1.8 + 1.7 for z ≲ 1 . Using both binary neutron star and neutron star–black hole binaries, we obtain a broad, relatively flat neutron star mass distribution extending from 1.2 − 0.2 + 0.1 to 2.0 − 0.3 + 0.3 M ⊙ . We confidently determine that the merger rate as a function of mass sharply declines after the expected maximum neutron star mass, but cannot yet confirm or rule out the existence of a lower mass gap between neutron stars and black holes. We also find the binary black hole mass distribution has localized over- and underdensities relative to a power-law distribution, with peaks emerging at chirp masses of 8.3 − 0.5 + 0.3 and 27.9 − 1.8 + 1.9 M ⊙ . While we continue to find that the mass distribution of a binary’s more massive component strongly decreases as a function of primary mass, we observe no evidence of a strongly suppressed merger rate above approximately 60 M ⊙ , which would indicate the presence of a upper mass gap. Observed black hole spins are small, with half of spin magnitudes below χ i ≈ 0.25 . While the majority of spins are preferentially aligned with the orbital angular momentum, we infer evidence of antialigned spins among the binary population. We observe an increase in spin magnitude for systems with more unequal-mass ratio. We also observe evidence of misalignment of spins relative to the orbital angular momentum. Published by the American Physical Society 2023

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2018

An approach to quantum-computational hydrologic inverse analysis

Making predictions about flow and transport in an aquifer requires knowledge of the heterogeneous properties of the aquifer such as permeability. Computational methods for inverse analysis are commonly used to infer these properties from quantities that are more readily observable such as hydraulic head. We present a method for computational inverse analysis that utilizes a type of quantum computer called a quantum annealer. While quantum computing is in an early stage compared to classical computing, we demonstrate that it is sufficiently developed that it can be used to solve certain subsurface flow problems. We utilize a D-Wave 2X quantum annealer to solve 1D and 2D hydrologic inverse problems that, while small by modern standards, are similar in size and sometimes larger than hydrologic inverse problems that were solved with early classical computers. Our results and the rapid progress being made with quantum computing hardware indicate that the era of quantum-computational hydrology may not be too far in the future.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS1994

Redshift data and statistical inference

Frequency histograms and the 'power spectrum analysis' (PSA) method, the latter developed by Yu & Peebles (1969), have been widely employed as techniques for establishing the existence of periodicities. We provide a formal analysis of these two classes of methods, including controlled numerical experiments, to better understand their proper use and application. In particular, we note that typical published applications of frequency histograms commonly employ far greater numbers of class intervals or bins than is advisable by statistical theory sometimes giving rise to the appearance of spurious patterns. The PSA method generates a sequence of random numbers from observational data which, it is claimed, is exponentially distributed with unit mean and variance, essentially independent of the distribution of the original data. We show that the derived random processes is nonstationary and produces a small but systematic bias in the usual estimate of the mean and variance. Although the derived variable may be reasonably described by an exponential distribution, the tail of the distribution is far removed from that of an exponential, thereby rendering statistical inference and confidence testing based on the tail of the distribution completely unreliable. Finally, we examine a number of astronomical examples wherein these methods have been used giving rise to widespread acceptance of statistically unconfirmed conclusions.

Newman, William I.↗

NASA NTRSDate not supplied

Probabilistic Calibration of Expensive Models using Efficiently Trained Surrogates

Calibration of computational models in the presence of uncertainty is often cast as a Bayesian inference problem and solved via sampling methods, e.g., Markov chain Monte Carlo. When the computational model is expensive, this task becomes intractable due to the large number of samples required to accurately estimate the posterior distribution of the calibration parameters. A popular solution to this problem is to use machine learning to develop a faster-to-evaluate, lower-fidelity substitute for the original model to serve as a surrogate while solving the inference problem. Although considered an offline cost, generating training data to construct this surrogate model can still be an expensive task in practice. An active learning algorithm is presented that focuses training on improving surrogate accuracy specifically in and around the bulk of the posterior distribution, as this is where the model is exercised during calibration. Candidate samples are drawn from families of distributions related to an approximation of the posterior. The sample maximizing predictive variance is then selected for evaluation by the original computational model, yielding a label for the training point. Iterating this approach increases efficiency relative to space filling designs (e.g., Latin hypercube sampling) by avoiding low probability points. Practical considerations are discussed, including the benefits of using a sequential Monte Carlo sampling approach, convergence heuristics, and the importance of both exploration and exploitation given that the true posterior is unknown a priori.

uncertainty quantification↗