SEARCH · Search NASA

Results for “log-likelihood”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

17 records

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Inverse Design of Two-Dimensional Airfoils Using Conditional Generative Models and Surrogate Log-Likelihoods

Abstract This paper shows how to use conditional generative models in two-dimensional (2D) airfoil optimization to probabilistically predict good initialization points within the vicinity of the optima given the input boundary conditions, thus warm starting and accelerating further optimization. We accommodate the possibility of multiple optimal designs corresponding to the same input boundary condition and take this inversion ambiguity into account when designing our prediction framework. To this end, we first employ the conditional formulation of our previous work BézierGAN–Conditional BézierGAN (CBGAN)—as a baseline, then introduce its sibling conditional entropic BézierGAN (CEBGAN), which is based on optimal transport regularized with entropy. Compared with CBGAN, CEBGAN overcomes mode collapse plaguing conventional GANs, improves the average lift-drag (Cl/Cd) efficiency of airfoil predictions from 80.8% of the optimal value to 95.8%, and meanwhile accelerates the training process by 30.7%. Furthermore, we investigate the unique ability of CEBGAN to produce a log-likelihood lower bound that may help select generated samples of higher performance (e.g., aerodynamic performance). In addition, we provide insights into the performance differences between these two models with low-dimensional toy problems and visualizations. These results and the probabilistic formulation of this inverse problem justify the extension of our GAN-based inverse design paradigm to other inverse design problems or broader inverse problems.

Engineering↗

DOE OSTI2021

Training Restricted Boltzmann Machines With a D-Wave Quantum Annealer

Restricted Boltzmann Machine (RBM) is an energy-based, undirected graphical model. It is commonly used for unsupervised and supervised machine learning. Typically, RBM is trained using contrastive divergence (CD). However, training with CD is slow and does not estimate the exact gradient of the log-likelihood cost function. In this work, the model expectation of gradient learning for RBM has been calculated using a quantum annealer (D-Wave 2000Q), where obtaining samples is faster than Markov chain Monte Carlo (MCMC) used in CD. Training and classification results of RBM trained using quantum annealing are compared with the CD-based method. The performance of the two approaches is compared with respect to the classification accuracies, image reconstruction, and log-likelihood results. The classification accuracy results indicate comparable performances of the two methods. Image reconstruction and log-likelihood results show improved performance of the CD-based method. It is shown that the samples obtained from quantum annealer can be used to train an RBM on a 64-bit “bars and stripes” dataset with classification performance similar to an RBM trained with CD. Though training based on CD showed improved learning performance, training using a quantum annealer could be useful as it eliminates computationally expensive MCMC steps of CD.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2021

Statistical interpretation of sterile neutrino oscillation searches at reactors

A considerable experimental effort is currently under way to test the persistent hints for oscillations due to an eV-scale sterile neutrino in the data of various reactor neutrino experiments. The assessment of the statistical significance of these hints is usually based on Wilks’ theorem, whereby the assumption is made that the log-likelihood is χ 2 -distributed. However, it is well known that the preconditions for the validity of Wilks’ theorem are not fulfilled for neutrino oscillation experiments. In this work we derive a simple asymptotic form of the actual distribution of the log-likelihood based on reinterpreting the problem as fitting white Gaussian noise. From this formalism we show that, even in the absence of a sterile neutrino, the expectation value for the maximum likelihood estimate of the mixing angle remains non-zero with attendant large values of the log-likelihood. Our analytical results are then confirmed by numerical simulations of a toy reactor experiment. Finally, we apply this framework to the data of the Neutrino-4 experiment and show that the null hypothesis of no-oscillation is rejected at the 2.6 σ level, compared to 3.2 σ obtained under the assumption that Wilks’ theorem applies.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

We derive an asymptotic expansion for the log-likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest. © 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Reconstructing porous media using generative flow networks

One area of intense scientific interest for the study of sandstones, carbonates, and shale at the pore scale is the use of limited image and petrophysical data to generate multiple realizations of a rock’s pore structure. Such images aid efforts to quantify uncertainty in petrophysical properties, including porosity–permeability transforms. We develop and evaluate a deep learning-based method to synthesize porous media volumes using a so-called generative flow model trained on x-ray microscope images of rock texture and pore structure. These models are optimized on a log-likelihood objective and they synthesize large and realistic three-dimensional images. Further, we demonstrate the rapid generation of sandstone image volumes that display realism as gauged by quantitative comparison of topological features using Minkowski functionals of porosity, specific surface area, and the Euler–Poincaré characteristic (i.e., pore connectivity). We also evaluate the single-phase permeability using Navier–Stokes and lattice Boltzmann methods and show that transport properties of the generated samples match measured trends.

58 GEOSCIENCES↗

DOE OSTI2024

Bi-fidelity variational auto-encoder for uncertainty quantification

Quantifying the uncertainty of quantities of interest (QoIs) from physical systems is a primary objective in model validation. However, achieving this goal entails balancing the need for computational efficiency with the requirement for numerical accuracy. To address this trade-off, we propose a novel bi-fidelity formulation of variational auto-encoders (BF-VAE) designed to estimate the uncertainty associated with a QoI from low-fidelity (LF) and high-fidelity (HF) samples of the QoI. Here, this model allows for the approximation of the statistics of the HF QoI by leveraging information derived from its LF counterpart. Specifically, we design a bi-fidelity auto-regressive model in the latent space which is integrated within the VAE’s probabilistic encoder–decoder structure. An effective algorithm is proposed to maximize the variational lower bound of the HF log-likelihood in the presence of limited HF data, resulting in the synthesis of HF realizations with a reduced computational cost. Additionally, we introduce the concept of the bi-fidelity information bottleneck (BF-IB) to provide an information-theoretic interpretation of the proposed BF-VAE model. Our numerical results demonstrate that the BF-VAE leads to considerably improved accuracy, as compared to a VAE trained using only HF data, when limited HF data is available.

42 ENGINEERING↗

DOE OSTI2020

Extended experimental inferential structure determination method in determining the structural ensembles of disordered protein states

Proteins with intrinsic or unfolded state disorder comprise a new frontier in structural biology, requiring the characterization of diverse and dynamic structural ensembles. Here we introduce a comprehensive Bayesian framework, the Extended Experimental Inferential Structure Determination (X-EISD) method, which calculates the maximum log-likelihood of a disordered protein ensemble. X-EISD accounts for the uncertainties of a range of experimental data and back-calculation models from structures, including NMR chemical shifts, J-couplings, Nuclear Overhauser Effects (NOEs), paramagnetic relaxation enhancements (PREs), residual dipolar couplings (RDCs), hydrodynamic radii (R h ), single molecule fluorescence Förster resonance energy transfer (smFRET) and small angle X-ray scattering (SAXS). We apply X-EISD to the joint optimization against experimental data for the unfolded drkN SH3 domain and find that combining a local data type, such as chemical shifts or J-couplings, paired with long-ranged restraints such as NOEs, PREs or smFRET, yields structural ensembles in good agreement with all other data types if combined with representative IDP conformers.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Transfer Learning-Based Independent Component Analysis

Understanding the underlying component structure is crucial for multivariate signal analysis. Among all the techniques that try to learn the latent structure, independent component analysis (ICA) is one of the most important and popular methods, which aims to extract independent components from multivariate signals and enables further analysis. For example, in electroencephalogram (EEG) analysis, artifacts filtering and disease detection are conducted based on the independent components of the signals. One critical challenge in existing ICA approaches is that the component extraction accuracy may degrade when the available data of a unit are limited. To address this issue, this paper proposes a transfer learning-based ICA method by innovatively transferring component distribution from a source domain, so that accurate component extraction results can be achieved even when only limited data are available in the target domain. To the best of our knowledge, this is the first work that leverages transfer learning to improve ICA accuracy with limited available data. In particular, we first extract all the independent components from the source domain by maximizing the log-likelihood function with a Newton-like method on a smooth manifold. Then for the target domain, the component with the largest negentropy is extracted in each round. To effectively leverage the knowledge from the source domain and to prevent the negative transfer, we try to find a component in the source domain that matches the component we are extracting. The probability density function of the matched component will then be used to improve the component extraction accuracy if such matched component can be found; otherwise, no knowledge will be transferred. Finally, numerical simulations and a case study with electrocardiogram (ECG) data are conducted, showing the effectiveness of the proposed method in transferring knowledge and reducing negative transfer.

42 ENGINEERING↗

DOE OSTI2020

A Robust Segmented Mixed Effect Regression Model for Baseline Electricity Consumption Forecasting

Renewable energy production has been surging around the world in recent years. To mitigate the increasing uncertainty and intermittency of the renewable generation, proactive demand response algorithms and programs are proposed and developed to further improve the utilization of load flexibility and increase the efficiency of power system operation. One of the biggest challenges to efficient control and operation of demand response resources is how to forecast the baseline electricity consumption and estimate the load impact from demand response resources accurately. In this paper, we propose a mixed effect segmented regression model and a new robust estimate for forecasting the baseline electricity consumption in Southern California, USA, by combining the ideas of random effect regression model, segmented regression model, and the least trimmed squares estimate. Since the log-likelihood of the considered model is not differentiable at breakpoints, we propose a new backfitting algorithm to estimate the unknown parameters. The estimation performance of the new estimation procedure has been demonstrated with both simulation studies and the real data application for the electric load baseline forecasting in Southern California.

42 ENGINEERING↗

DOE OSTI2021

The Anatomy of an Unusual Edge-on Protoplanetary Disk. I. Dust Settling in a Cold Disk

As the earliest stage of planet formation, massive, optically thick, and gas-rich protoplanetary disks provide key insights into the physics of star and planet formation. When viewed edge-on, high-resolution images offer a unique opportunity to study both the radial and vertical structures of these disks and relate this to vertical settling, radial drift, grain growth, and changes in the midplane temperatures. In this work, we present multi-epoch Hubble Space Telescope and Keck scattered light images, and an Atacama Large Millimeter/submillimeter Array 1.3 mm continuum map for the remarkably flat edge-on protoplanetary disk SSTC2DJ163131.2–242627, a young solar-type star in ρ Ophiuchus. We model the 0.8 μm and 1.3 mm images in separate Markov Chain Monte Carlo (MCMC) runs to investigate the geometry and dust properties of the disk using the MCFOST radiative transfer code. In scattered light, we are sensitive to the smaller dust grains in the surface layers of the disk, while the submillimeter dust continuum observations probe larger grains closer to the disk midplane. An MCMC run combining both data sets using a covariance-based log-likelihood estimation was marginally successful, implying insufficient complexity in our disk model. The disk is well characterized by a flared disk model with an exponentially tapered outer edge viewed nearly edge-on, though some degree of dust settling is required to reproduce the vertically thin profile and lack of apparent flaring. A colder than expected disk midplane, evidence for dust settling, and residual radial substructures all point to a more complex radial density profile to be probed with future, higher-resolution observations.

47 OTHER INSTRUMENTATION↗

DOE OSTI2026

Red Noise–based False Alarm Thresholds for Astrophysical Periodograms via Whittle’s Approximation to the Likelihood

Astronomers who search for periodic signals using Lomb–Scargle periodograms rely on false alarm level (FAL) estimates to identify statistically significant peaks. Although FALs are often calculated from white noise models, many astronomical time series suffer from red noise. Prewhitening is a statistical technique in which a continuum model is subtracted from the log power spectrum estimate, after which the observer can proceed with a white-noise treatment. Here we present a prewhitening-based method of calculating frequency-dependent FALs. We fit power laws and autoregressive models of order 1 to each Lomb–Scargle periodogram by minimizing the Whittle approximation to the negative log-likelihood (NLL), then calculate FALs based on the best-fit model power spectrum. Our technique is a novel extension of the Whittle NLL to datasets with uneven time sampling. We demonstrate FAL calculations using observations of α Cen B, GJ 581, HD 192310, synthetic data from the radial velocity (RV) fitting challenge, and Kepler observations of a differential rotator. The Kepler data analysis shows that only true rotation signals are detected by red noise FALs, while white noise FALs suggest all spurious peaks in the low-frequency range are significant. A high-frequency sinusoid injected into α Cen B logR$'$ HK observations exceeds the 1% red noise FAL despite having only 8.9% of the power of the dominant rotation signal. In a periodogram of HD 192310 RVs, peaks associated with differential rotation and planets are detected against the 5% red noise FAL without iterative model fitting or subtraction. The software for calculating red noise–based FALs is available on GitHub.

Astrostatistics (1882)↗

DOE OSTI2023

Limits on Leptonic TeV Emission from the Cygnus Cocoon with Swift-XRT

$\gamma$-ray observations of the Cygnus Cocoon, an extended source surrounding the Cygnus X star-forming region, suggest the presence of a cosmic ray accelerator reaching energies up to a few PeV. The very-high-energy (VHE; 0.1-100~TeV) $\gamma$-ray emission may be explained by the interaction of cosmic-ray hadrons with matter inside the Cocoon, but an origin of inverse Compton radiation by relativistic electrons cannot be ruled out. Inverse Compton $\gamma$-rays at VHE are accompanied by synchrotron radiation peaked in X-rays. Hence, X-ray observations may probe the electron population and magnetic field of the source. We observed eleven fields in or near the Cygnus Cocoon with the Neil Gehrels Swift Observatory's X-Ray Telescope (Swift-XRT) totaling 110 ksec. We fit the fields to a Galactic and extra-galactic background model and performed a log-likelihood ratio test for an additional diffuse component. We found no significant additional emission and established upper limits in each field. By assuming that the X-ray intensity traces the TeV intensity and follows an dN/dE ∝ E -2.5 spectrum, we obtained a 90% upper limit of F X < 8.7 x 10 -11 erg cm -2 s -1 or 5.2 x10 -11 erg cm -2 s -1 on the X-ray flux of the entire Cygnus Cocoon between 2 and 10 keV depending on the choice of hydrogen column density model. This suggests that no more than one quarter of the $\gamma$-ray flux at 1 TeV is produced by inverse Compton scattering, when assuming an equipartition magnetic field of ~20 μG.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Search For Excited Cascade Hypersons (¿*-) Using the CLAS12 Spectrometer at Jefferson Laboratory

The number of experimentally observed doubly strange Cascade states to date is far fewer than has been predicted theoretically. The CLAS12 Very Strange physics program (E12- 11-005A) at the Thomas Jefferson National Accelerator Facility aims to study the electroproduction of these states using the newly upgraded CEBAF Large Acceptance Spectrometer for 12 GeV (CLAS12) in experimental Hall B. In this project, the reaction ep ! e0K+K+??? ! e0K+K+K?(?=?0) is studied using CLAS12 Run Group A (RG-A) data sets taken by impinging electron beams of 10:2 and 10:6 GeV energies on an LH2 target. Scattered electrons are detected with either the Forward Detector (FD), covering a polar angle range of 5? to 35? to study electroproduction, or with the Forward Tagger (FT), covering a polar angle range of 2:5? to 4:5?, to study quasi-real photoproduction. The CLAS12 detector with nearly a 4? solid angle coverage is used to detect scattered electrons and charged kaons in the ?final state. ?=?0 hyperons are reconstructed using the missing mass technique to explore intermediate doubly-strange hyperons (???) that decay to K? and ?=?0. No statistically significant ??? states other than the ???(1530) were found in the missing mass spectra based on the currently available statistics in the CLAS12-FD acceptance only. A maximum log-likelihood method is implemented to determine the statistical significance and the upper limits on the ???(1820) electroproduction cross section by constructing 95% confidence level boundaries on the ???(1820) yields. Additional research was conducted to investigate the upper limit electroproduction cross section of the reaction ep ! e0K+K+??? ! e0K+K+K?(?=?0) as a function of the electroproduced ??? mass and the differential cross section ( d?dMM(e0K+K+) ) as a function of missing mass MM(e0K+K+) for electroproduction and quasi-real photoproduction processes.

Khanal, Achyut↗

DOE OSTI2024

Maximum Likelihood Estimation: Some Basics

The maximum likelihood estimation is a general estimation procedure. It is often compared to estimation procedures like the ordinary least squares regression or generalized method of moments, to name a few. We discuss some basics about the maximum likelihood estimation, its advantages and disadvantages, and provide an example application to a gamma distribution function.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2025

Maximum Likelihood Estimation: Some Basics

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Estimating Large Global Significances with a New Monte Carlo Extrapolation Method

In particle physics, it is needed to evaluate the possibility that excesses of events in mass spectra are due to statistical fluctuations as quantified by the standards of local and global significances. Without prior knowledge of a particle’s mass, it is especially critical to estimate its global significance. The usual approach is to count the number of times a significance limit is exceeded in a collection of simulated Monte Carlo (MC) “toy experiments”. To demonstrate this conventional method for global significance, we performed simulation studies according to a recent Compact Muon Solenoid (CMS) result to show its effectiveness. However, this counting method is not practical for computing large global significances. To address this problem, we developed a new “extrapolation” method to evaluate the global significance. We compared the global significance estimated by our new method with that of the conventional approach, and verified its feasibility and effectiveness. This method is also applicable for cases where only small toy MC samples are available. In this approach, the significance is calculated based on p-values, assuming symmetrical Gaussian distributions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗