Search NASA⌕ Search

SEARCH · Search NASA

Results for “Gaussian Process Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21

Multi-fidelity modeling to predict the rheological properties of a suspension of fibers using neural networks and Gaussian processes

Unveiling the rheological properties of fiber suspensions is of paramount interest to many industrial applications. There are multiple factors, such as fiber aspect ratio and volume fraction, that play a significant role in altering the rheological behavior of suspensions. Three-dimensional (3D) numerical simulations of coupled differential equations of the suspension of fibers are computationally expensive and time-consuming. Machine learning algorithms can be trained on the available data and make predictions for the cases where no numerical data are available. However, some widely used machine learning surrogates, such as neural networks, require a relatively large training dataset to produce accurate predictions. Multi-fidelity models, which combine high-fidelity data from numerical simulations and less expensive lower fidelity data from resources such as simplified constitutive equations, can pave the way for more accurate predictions. Here, we focus on neural networks and the Gaussian processes with two levels of fidelity, i.e., high and low fidelity networks, to predict the steady-state rheological properties, and compare them to the single-fidelity network. High-fidelity data are obtained from direct numerical simulations based on an immersed boundary method to couple the fluid and solid motion. The low-fidelity data are produced by using constitutive equations. Multiple neural networks and the Gaussian process structures are used for the hyperparameter tuning purpose. Results indicate that with the best choice of hyperparameters, both the multi-fidelity Gaussian processes and neural networks are capable of making predictions with a high level of accuracy with neural networks demonstrating marginally better performance.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Non-Gaussianity and Large Scale Structure in a two-field Inflationary model

Single-field inflationary models predict nearly Gaussian initial conditions, and hence a detection of non-Gaussianity would be a signature of the more complex inflationary scenarios. In this paper we study the effect on the cosmic microwave background and on large-scale structure from primordial non-Gaussianity in a two-field inflationary model in which both the inflaton and curvaton contribute to the density perturbations. We show that in addition to the previously described enhancement of the galaxy bias on large scales, this setup results in large-scale stochasticity. We provide joint constraints on the local non-Gaussianity parameter f*{sub NL} and the ratio {zeta} of the amplitude of primordial perturbations due to the inflaton and curvaton using WMAP and Sloan Digital Sky Survey data.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

CLASSIFYING X-RAY BINARIES: A PROBABILISTIC APPROACH

In X-ray binary star systems consisting of a compact object that accretes material from an orbiting secondary star, there is no straightforward means to decide whether the compact object is a black hole or a neutron star. To assist in this process, we develop a Bayesian statistical model that makes use of the fact that X-ray binary systems appear to cluster based on their compact object type when viewed from a three-dimensional coordinate system derived from X-ray spectral data where the first coordinate is the ratio of counts in the mid- to low-energy band (color 1), the second coordinate is the ratio of counts in the high- to low-energy band (color 2), and the third coordinate is the sum of counts in all three bands. We use this model to estimate the probabilities of an X-ray binary system containing a black hole, non-pulsing neutron star, or pulsing neutron star. In particular, we utilize a latent variable model in which the latent variables follow a Gaussian process prior distribution, and hence we are able to induce the spatial correlation which we believe exists between systems of the same type. The utility of this approach is demonstrated by the accurate prediction of system types using Rossi X-ray Timing Explorer All Sky Monitor data, but it is not flawless. In particular, non-pulsing neutron systems containing “bursters” that are close to the boundary demarcating systems containing black holes tend to be classified as black hole systems. As a byproduct of our analyses, we provide the astronomer with the public R code which can be used to predict the compact object type of XRBs given training data.

79 ASTRONOMY AND ASTROPHYSICS↗

Comparison of Machine Learning-Based Predictive Models of the Nutrient Loads Delivered from the Mississippi/Atchafalaya River Basin to the Gulf of Mexico

Predicting nutrient loads is essential to understanding and managing one of the environmental issues faced by the northern Gulf of Mexico hypoxic zone, which poses a severe threat to the Gulf’s healthy ecosystem and economy. The development of hypoxia in the Gulf of Mexico is strongly associated with the eutrophication process initiated by excessive nutrient loads. Due to the complexities in the excessive nutrient loads to the Gulf of Mexico, it is challenging to understand and predict the underlying temporal variation of nutrient loads. The study was aimed at identifying an optimal predictive machine learning model to capture and predict nonlinear behavior of the nutrient loads delivered from the Mississippi/Atchafalaya River Basin (MARB) to the Gulf of Mexico. For this purpose, monthly nutrient loads (N and P) in tons were collected from US Geological Survey (USGS) monitoring station 07373420 from 1980 to 2020. Machine learning models—including autoregressive integrated moving average (ARIMA), gaussian process regression (GPR), single-layer multilayer perceptron (MLP), and a long short-term memory (LSTM) with the single hidden layer—were developed to predict the monthly nutrient loads, and model performances were evaluated by standard assessment metrics—Root Mean Square Error (RMSE) and Correlation Coefficient (R). The residuals of predictive models were examined by the Durbin–Watson statistic. The results showed that MLP and LSTM persistently achieved better accuracy in predicting monthly TN and TP loads compared to GPR and ARIMA. In addition, GPR models achieved slightly better test RMSE score than ARIMA models while their correlation coefficients are much lower than ARIMA models. Moreover, MLP performed slightly better than LSTM in predicting monthly TP loads while LSTM slightly outperformed for TN loads. Furthermore, it was found that the optimizer and number of inputs didn’t show effects on the LSTM performance while they exhibited impacts on MLP outcomes. This study explores the capability of machine learning models to accurately predict nonlinearly fluctuating nutrient loads delivered to the Gulf of Mexico. Further efforts focus on improving the accuracy of forecasting using hybrid models which combine several machine learning models with superior predictive performance for nutrient fluxes throughout the MARB.

54 ENVIRONMENTAL SCIENCES↗

A Gaussian Process Based Online Change Detection Algorithm for Monitoring Periodic Time Series

Online time series change detection is a critical component of many monitoring systems, such as space and air-borne remote sensing instruments, cardiac monitors, and network traffic profilers, which continuously analyze observations recorded by sensors. Data collected by such sensors typically has a periodic (seasonal) component. Most existing time series change detection methods are not directly applicable to handle such data, either because they are not designed to handle periodic time series or because they cannot operate in an online mode. We propose an online change detection algorithm which can handle periodic time series. The algorithm uses a Gaussian process based non-parametric time series prediction model and monitors the difference between the predictions and actual observations within a statistically principled control chart framework to identify changes. A key challenge in using Gaussian process in an online mode is the need to solve a large system of equations involving the associated covariance matrix which grows with every time step. The proposed algorithm exploits the special structure of the covariance matrix and can analyze a time series of length T in O(T^2) time while maintaining a O(T) memory footprint, compared to O(T^4) time and O(T^2) memory requirement of standard matrix manipulation methods. We experimentally demonstrate the superiority of the proposed algorithm over several existing time series change detection algorithms on a set of synthetic and real time series. Finally, we illustrate the effectiveness of the proposed algorithm for identifying land use land cover changes using Normalized Difference Vegetation Index (NDVI) data collected for an agricultural region in Iowa state, USA. Our algorithm is able to detect different types of changes in a NDVI validation data set (with ~80% accuracy) which occur due to crop type changes as well as disruptive changes (e.g., natural disasters).

54 ENVIRONMENTAL SCIENCES↗

Neutron Drip Line in the Ca Region from Bayesian Model Averaging

The region of heavy calcium isotopes forms the frontier of experimental and theoretical nuclear structure research where the basic concepts of nuclear physics are put to stringent test. The recent discovery of the extremely neutron-rich nuclei around 60 Ca and the experimental determination of masses for 55-57 Ca provide unique information about the binding energy surface in this region. To assess the impact of these experimental discoveries on the nuclear landscape’s extent, we use global mass models and statistical machine learning to make predictions, with quantified levels of certainty, for bound nuclides between Si and Ti. Using a Bayesian model averaging analysis based on Gaussian-process-based extrapolations we introduce the posterior probability for each nucleus to be bound to neutron emission. We discover that extrapolations for drip-line locations, at which the nuclear binding ends, are consistent across the global mass models used, in spite of significant variations between their raw predictions. In particular, considering the current experimental information and current global mass models, we suggest that 68 Ca has an average posterior probability p ex ≈ 76 % to be bound to two-neutron emission while the nucleus 61 Ca is likely to decay by emitting a neutron (p ex ≈ 46 % ).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Scaled Vecchia Approximation for Fast Computer-Model Emulation

Many scientific phenomena are studied using computer experiments consisting of multiple runs of a computer model while varying the input settings. Gaussian processes (GPs) are a popular tool for the analysis of computer experiments, enabling interpolation between input settings, but direct GP inference is computationally infeasible for large datasets. We adapt and extend a powerful class of GP methods from spatial statistics to enable the scalable analysis and emulation of large computer experiments. Specifically, we apply Vecchia’s ordered conditional approximation in a transformed input space, with each input scaled according to how strongly it relates to the computer-model response. The scaling is learned from the data by estimating parameters in the GP covariance function using Fisher scoring. Our methods are highly scalable, enabling estimation, joint prediction, and simulation in near-linear time in the number of model runs. In several numerical examples, our approach substantially outperformed existing methods.

97 MATHEMATICS AND COMPUTING↗

Transfer learning of memory kernels for transferable coarse-graining of polymer dynamics

The present work concerns the transferability of coarse-grained (CG) modeling in reproducing the dynamic properties of the reference atomistic systems across a range of parameters. In particular, we focus on implicit-solvent CG modeling of polymer solutions. The CG model is based on the generalized Langevin equation, where the memory kernel plays the critical role in determining the dynamics in all time scales. Thus, we propose methods for transfer learning of memory kernels. The key ingredient of our methods is Gaussian process regression. By integration with the model order reduction via proper orthogonal decomposition and the active learning technique, the transfer learning can be practically efficient and requires minimum training data. Through two example polymer solution systems, we demonstrate the accuracy and efficiency of the proposed transfer learning methods in the construction of transferable memory kernels. The transferability allows for out-of-sample predictions, even in the extrapolated domain of parameters. Built on the transferable memory kernels, the CG models can reproduce the dynamic properties of polymers in all time scales at different thermodynamic conditions (such as temperature and solvent viscosity) and for different systems with varying concentrations and lengths of polymers.

Ma, Zhan↗

A Machine Learning based Approach of Estimating Equivalent Circuit Model Parameters at Different SoCs of Li-ion Batteries from Voltage Relaxation

Abstract: In this study, an approach of estimating the equivalent circuit model (ECM) parameters for Li-ion batteries (LIBs) is proposed based on the voltage value at different intervals while relaxing the LIB after discharge. The typical approach for estimating ECM parameters of a LIB is to conduct electrochemical impedance spectroscopy (EIS) measurements at different frequencies and fit them to a predefined circuit model, which requires additional measuring arrangements and specialized devices. The proposed methodology utilizes four different voltages at 0s, 60s, 360s, and 1800s alongside the specific state of charge (SoC) value for a specific constant discharge current value of ~1C until the relaxation stage to train and evaluate three regression-based machine learning models— Support Vector Regression (SVR), Extreme Gradient Boosting (XGBoost), and Gaussian Process Regression (GPR)—for estimating the ECM parameters of the selected model. Bayesian optimization is employed for hyperparameter tuning to achieve optimal performance for all the regressor models, among which, the GPR provided the best performance with the root-mean-squared error (RMSE) of less than 4x10-4 on average for the resistive components and less than 0.27 for capacitive components with excellent R2 scores. The simplicity of the approach enables it to eliminate the need for sophisticated measuring equipment and computation power.

Sagar, Md. Samiul [The University of Alabama (UA)]↗

Computer-aided detection using non-convolutional neural network Gaussian processes

Deep convolutional neural networks (CNNs) have in recent years achieved record-breaking performance on many image classification tasks and are therefore well-suited for computer aided detection (CAD). The need for uncertainty quantification for CAD motivates the need for a probabilistic framework for deep learning. The most well-known probabilistic neural network model is the Bayesian neural network (BNN), but BNNs are notoriously difficult to sample for large complex network architectures, and as such their use is restricted to small problems. It is known that the limit of BNNs as their widths increase toward infinity is a Gaussian process (GP), and there has been considerable research interest in these infinitely wide BNNs. Recently, this classic result has been extended to deep architectures in what is termed the neural network Gaussian process (NNGP) model. In this work, we implement an NNGP model and apply it to the ChestXRay14 dataset at the full resolution of 1024x1024 pixels. Even without any convolutional aspects to the network architecture and without any data augmentation, our five layer deep NNGP model outperforms other non-convolutional models and therefore helps to narrow the performance gap between non-convolutional and convolutional models. Our NNGP model is fully Bayesian and therefore offers uncertainty information through its predictive variance that can be used to formulate a predictive confidence measure. We show that the performance of the NNGP model is significantly boosted after low-confidence predictions are rejected, suggesting that convolution is most beneficial only for these low-confidence examples. Finally, our results indicate that an extremely large fully-connected neural network with appropriate regularization could perform as well as the NNGP if not for the computational bottleneck resulting from the large number of model parameters.

Agrawal, Devanshu↗

A Data-Driven Nonparametric Approach for Probabilistic Load-Margin Assessment Considering Wind Power Penetration

A modern power system is characterized by an increasing penetration of wind power, which results in large uncertainties in its states. These uncertainties must be quantified properly; otherwise, the system security may be threatened. Facing this challenge, here we propose a cost-effective, data-driven approach to assessing a power system's load margin probabilistically. Using actual wind data, a kernel density estimator is applied to infer the nonparametric wind speed distributions, which are further merged into the framework of a vine copula. The latter enables us to simulate complex multivariate and highly dependent model inputs with a variety of bivariate copulae that precisely represent the tail dependence in the correlated samples. Furthermore, to reduce the prohibitive computational time of traditional Monte-Carlo simulations that process a large amount of samples, we propose to use a nonparametric, Gaussian-process-emulator-based reduced-order model to replace the original complicated continuation power-flow model through a Bayesian-learning framework. To accelerate the convergence rate of this Bayesian algorithm, a truncated polynomial chaos surrogate, which serves as a highly efficient, parametric Bayesian prior, is developed. This emulator allows us to execute the time-consuming continuation power-flow solver at the sampled values with a negligible computational cost. Results of simulations that are performed on several test systems reveal the impressive performance of the proposed method in the probabilistic load-margin assessment.

17 WIND ENERGY↗

Bayesian Treed Calibration: An Application to Carbon Capture With AX Sorbent

In cases where field or experimental measurements are not available, computer models can model real physical or engineering systems to reproduce their outcomes. They are usually calibrated in light of experimental data to create a better representation of the real system. Statistical methods, based on Gaussian processes, for calibration and prediction have been especially important when the computer models are expensive and experimental data limited. In this paper, we develop the Bayesian treed calibration (BTC) as an extension of standard Gaussian process calibration methods to deal with non-stationarity computer models and/or their discrepancy from the field (or experimental) data. Our proposed method partitions both the calibration and observable input space, based on a binary tree partitioning, into sub-regions where existing model calibration methods can be applied to connect a computer model with the real system. The estimation of the parameters in the proposed model is carried out using Markov chain Monte Carlo (MCMC) computational techniques. Different strategies have been applied to improve mixing. We illustrate our method in two artificial examples and a real application that concerns the capture of carbon dioxide with AX amine based sorbents. The source code and the examples analyzed in this paper are available as part of the supplementary materials.

Bayesian tree↗

Machine Learning Based Parallel I/O Predictive Modeling: A Case Study on Lustre File Systems

Parallel I/O hardware and software infrastructure is a key contributor to performance variability for applications running on large-scale HPC systems. This variability confounds efforts to predict application performance for characterization, modeling, optimization, and job scheduling. We propose a modeling approach that improves predictive ability by explicitly treating the variability and by leveraging the sensitivity of application parameters on performance to group applications with similar characteristics. We develop a Gaussian process-based machine learning algorithm to model I/O performance and its variability as a function of application and file system characteristics. We demonstrate the effectiveness of the proposed approach using data collected from the Edison system at the National Energy Research Scientific Computing Center. The results show that the proposed sensitivity-based models are better at prediction when compared with application-partitioned or unpartitioned models. We highlight modeling techniques that are robust to the outliers that can occur in production parallel file systems. Using the developed metrics and modeling approach, we provide insights into the file system metrics that have a significant impact on I/O performance.

I/O performance variability↗

Universality for 1d Random Band Matrices: Sigma-Model Approximation

The paper continues the development of the rigorous supersymmetric transfer matrix approach to the random band matrices started in (J Stat Phys 164:1233–1260, 2016; Commun Math Phys 351:1009–1044, 2017). We consider random Hermitian block band matrices consisting of W×W random Gaussian blocks (parametrized by j,k∈Λ=[1,n]{sup d}∩ℤ{sup d}) with a fixed entry’s variance J{sub jk}=δ{sub j,k}W{sup −1}+βΔ{sub j,k}W{sup −2}, β>0 in each block. Taking the limit W→∞ with fixed n and β, we derive the sigma-model approximation of the second correlation function similar to Efetov’s one. Then, considering the limit β,n→∞, we prove that in the dimension d=1 the behaviour of the sigma-model approximation in the bulk of the spectrum, as β≫n, is determined by the classical Wigner–Dyson statistics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Kinetics Modeling and Reactor Design Study of Glucose-to-Terpenes Cell-Free Conversion

Cell-free systems offer many advantages over traditional biological conversion by eliminating biological growth constraints. It also offers easy manipulation and finetuning of the reaction conditions for each individual enzyme. The conversion of cellulosic glucose to Limonene, a terpene, is a promising pathway for producing fuels and chemicals. Recent advances in developing cell-free systems focuses on bench scale optimization of terpene yield and to demonstrate its feasibility towards commercialization [1,2]. There is significant knowledge gap regarding reaction kinetics of these cell-free systems to further study how it will perform at larger scale. We present here, our studies on reaction kinetics and reactor design implications of cell-free glucose to Limonene conversion to facilitate the further development and commercialization of this process. We developed a novel kinetic model based on the metabolic-network structure of the cell-free system with multi-substrate reversible Michaelis-Menten rate law. To estimate kinetic parameters for this system of rate equations, we employed Bayesian optimization to perform global search with the assistance of gaussian processes to balance exploration and exploitation. The model parameters estimated showed good results compared with experimental data. The estimated parameters were used to perform sensitivity analysis. We found that Hexokinase is one of the most critical enzymes that affect the conversion of the glucose. We also observed that abundance of co-factors is also critical to the conversion of glucose to limonene. We investigated packed bed reactors with enzymes immobilized on the surface of particles to convert glucose stream into Limonene for larger scale production. The reactor design such as particle size, enzyme loading, and flow rate are found to be critical for improving yields. [1] Dudley, Q.M., Nash, C.J. and Jewett, M.C., 2019. Synthetic Biology, 4(1), p.ysz003. [2] Korman, T.P., Opgenorth, P.H. and Bowie, J.U., 2017. Nature communications, 8(1), p.15526.

09 BIOMASS FUELS↗

Benchmarking FFTF LOFWOS Test# 13 using SAM code: Baseline model development and uncertainty quantification

The development and deployment of advanced reactors, such as the sodium-cooled fast reactor (SFR), relies on sophisticated modeling tools to ensure the safety of the design under various transients. The predictive capability of these advanced modeling tools requires validation to garner trust in supporting the licensing of the advanced reactors. For this reason, the International Atomic Energy Agency (IAEA) initiated a coordinated research project (CRP) in 2018 for the analysis of the Fast Flux Test Facility (FFTF) Loss of Flow Without Scram (LOFWOS) Test #13.In this study, we present and discuss the benchmarking efforts of the modern system code SAM on the FFTF LOFWOS Test #13. Further, the SAM baseline model was developed according to the benchmark specification, which included a detailed core model with reactivity feedback. Generally, good agreement was observed between the baseline results and benchmark measurements; however, discrepancies persisted, particularly in predicted fuel assembly coolant outlet temperatures. Utilizing the baseline model, uncertainty quantification (UQ) and sensitivity analysis (SA) were conducted with the assistance of various statistical learning and machine learning methods, including kernel density estimation, Gaussian processes, and Sobol indices. Following the baseline model prediction and UQ and SA results, we discuss the reasons for the simulation discrepancies and propose further improvements to the model. This benchmarking effort adheres to the best-estimate plus uncertainty approach and can serve as a valuable example for supporting risk-informed licensing of advanced reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Physical Parameters of the Multiplanet Systems HD 106315 and GJ 9827

HD 106315 and GJ 9827 are two bright, nearby stars that host multiple super-Earths and sub-Neptunes discovered by K2 that are well suited for atmospheric characterization. We refined the planets’ ephemerides through Spitzer transits, enabling accurate transit prediction required for future atmospheric characterization through transmission spectroscopy. Through a multiyear high-cadence observing campaign with Keck/High Resolution Echelle Spectrometer and Magellan/Planet Finder Spectrograph, we improved the planets’ mass measurements in anticipation of Hubble Space Telescope transmission spectroscopy. For GJ 9827, we modeled activity-induced radial velocity signals with a Gaussian process informed by the Calcium II H&K lines in order to more accurately model the effect of stellar noise on our data. We measured planet masses of M {sub b} = 4.87 ± 0.37 M {sub ⊕}, M {sub c} = 1.92 ± 0.49 M {sub ⊕}, and M {sub d} = 3.42 ± 0.62 M {sub ⊕}. For HD 106315, we found that such activity radial velocity decorrelation was not effective due to the reduced presence of spots and speculate that this may extend to other hot stars as well (T {sub eff} > 6200 K). We measured planet masses of M {sub b} = 10.5 ± 3.1 M {sub ⊕} and M {sub c} = 12.0 ± 3.8 M {sub ⊕}. We investigated all of the planets’ compositions through comparison of their masses and radii to a range of interior models. GJ 9827 b and GJ 9827 c are both consistent with a 50/50 rock-iron composition, GJ 9827 d and HD 106315 b both require additional volatiles and are consistent with moderate amounts of water or hydrogen/helium, and HD 106315 c is consistent with a ∼10% hydrogen/helium envelope surrounding an Earth-like rock and iron core.

47 OTHER INSTRUMENTATION↗

Advances in Kriging-Based Autonomous X-Ray Scattering Experiments

Autonomous experimentation is an emerging paradigm for scientific discovery, wherein measurement instruments are augmented with decision-making algorithms, allowing them to autonomously explore parameter spaces of interest. We have recently demonstrated a generalized approach to autonomous experimental control, based on generating a surrogate model to interpolate experimental data, and a corresponding uncertainty model, which are computed using a Gaussian process regression known as ordinary Kriging (OK). We demonstrated the successful application of this method to exploring materials science problems using x-ray scattering measurements at a synchrotron beamline. Here, we report several improvements to this methodology that overcome limitations of traditional Kriging methods. The variogram underlying OK is global and thus insensitive to local data variation. We augment the Kriging variance with model-based measures, for instance providing local sensitivity by including the gradient of the surrogate model. As with most statistical regression methods, OK minimizes the number of measurements required to achieve a particular model quality. However, in practice this may not be the most stringent experimental constraint; e.g. the goal may instead be to minimize experiment duration or material usage. We define an adaptive cost function, allowing the autonomous method to balance information gain against measured experimental cost. We provide synthetic and experimental demonstrations, validating that this improved algorithm yields more efficient autonomous data collection.

36 MATERIALS SCIENCE↗