Search NASA⌕ Search

SEARCH · Search NASA

Results for “Machine Learning Cosmology”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

The Sensitivity of GPz Estimates of Photo-z Posterior PDFs to Realistically Complex Training Set Imperfections

The accurate estimation of photometric redshifts is crucial to many upcoming galaxy surveys, for example, the Vera C. Rubin Observatory Legacy Survey of Space and Time (LSST). Almost all Rubin extragalactic and cosmological science requires accurate and precise calculation of photometric redshifts; many diverse approaches to this problem are currently in the process of being developed, validated, and tested. In this work, we use the photometric redshift code GPz to examine two realistically complex training set imperfections scenarios for machine learning based photometric redshift calculation: (i) where the spectroscopic training set has a very different distribution in color–magnitude space to the test set, and (ii) where the effect of emission line confusion causes a fraction of the training spectroscopic sample to not have the true redshift. By evaluating the sensitivity of GPz to a range of increasingly severe imperfections, with a range of metrics (both of photo-z point estimates as well as posterior probability distribution functions, PDFs), we quantify the degree to which predictions get worse with higher degrees of degradation. In particular, we find that there is a substantial drop-off in photo-z quality when line-confusion goes above ~1%, and sample incompleteness below a redshift of 1.5, for an experimental setup using data from the Buzzard Flock synthetic sky catalogs.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2019

Hierarchical Modeling and Statistical Calibration for Photometric Redshifts

The cosmological exploitation of modern photometric galaxy surveys requires both accurate (unbiased) and precise (narrow) redshift probability distributions derived from broadband photometry. Existing methodologies do not meet these requirements. Standard template fitting delivers interpretable models and errors, but lacks the flexibility to learn inaccuracies in the observed photometry or spectral templates. Machine learning addresses those issues, but requires representative training data, and the resulting models and uncertainties cannot be interpreted in the context of a physical model or outside of the training data. We present a hierarchical modeling approach simultaneously addressing the issues of flexibility, interpretability, and generalization. It combines template fitting with flexible (machine-learning-like) models to correct the spectral templates, model their redshift distributions, and recalibrate the photometric observations. By optimizing the full posterior distribution of the model and solving for its (thousands of) parameters, one can perform a global statistical calibration of the data and the spectral energy distribution (SED) model. We apply this approach to the public Dark Energy Survey Science Verification data and show that it provides more accurate and compact redshift posterior distributions than existing methods, as well as insights into residual photometric and SED systematics. Here, the model is causal and makes predictions for future data (e.g., additional photometric bandpasses), and its internal parameters and components are interpretable. This approach does not formally require the training data to be complete or representative; in principle, it can even work in regimes in which few or no spectroscopic redshifts are available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Inferring dark matter substructure with astrometric lensing beyond the power spectrum

Abstract Astrometry—the precise measurement of positions and motions of celestial objects—has emerged as a promising avenue for characterizing the dark matter population in our Galaxy. By leveraging recent advances in simulation-based inference and neural network architectures, we introduce a novel method to search for global dark matter-induced gravitational lensing signatures in astrometric datasets. Our method based on neural likelihood-ratio estimation shows significantly enhanced sensitivity to a cold dark matter population and more favorable scaling with measurement noise compared to existing approaches based on two-point correlation statistics. We demonstrate the real-world viability of our method by showing it to be robust to non-trivial modeled as well as unmodeled noise features expected in astrometric measurements. This establishes machine learning as a powerful tool for characterizing dark matter using astrometric data.

convolutional neural networks (1938)↗
DOE OSTI2022

Identifying Transient Candidates in the Dark Energy Survey Using Convolutional Neural Networks

The ability to discover new transient candidates via image differencing without direct human intervention is an important task in observational astronomy. For these kind of image classification problems, machine learning techniques such as Convolutional Neural Networks (CNNs) have shown remarkable success. In this work, we present the results of an automated transient candidate identification on images with CNNs for an extant data set from the Dark Energy Survey Supernova program, whose main focus was on using Type Ia supernovae for cosmology. By performing an architecture search of CNNs, we identify networks that efficiently select non-artifacts (e.g., supernovae, variable stars, AGN, etc.) from artifacts (image defects, mis-subtractions, etc.), achieving the efficiency of previous work performed with random Forests, without the need to expend any effort in feature identification. The CNNs also help us identify a subset of mislabeled images. Performing a relabeling of the images in this subset, the resulting classification with CNNs is significantly better than previous results, lowering the false positive rate by 27% at a fixed missed detection rate of 0.05.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Snowmass2021 theory frontier white paper: Astrophysical and cosmological probes of dark matter

While astrophysical and cosmological probes provide a remarkably precise and consistent picture of the quantity and general properties of dark matter, its fundamental nature remains one of the most significant open questions in physics. Obtaining a more comprehensive understanding of dark matter within the next decade will require overcoming a number of theoretical challenges: the groundwork for these strides is being laid now, yet much remains to be done. Chief among the upcoming challenges is establishing the theoretical foundation needed to harness the full potential of new observables in the astrophysical and cosmological domains, spanning the early Universe to the inner portions of galaxies and the stars therein. Identifying the nature of dark matter will also entail repurposing and implementing a wide range of theoretical techniques from outside the typical toolkit of astrophysics, ranging from effective field theory to the dramatically evolving world of machine learning and artificial-intelligence-based statistical inference. Through this work, the theory frontier will be at the heart of dark matter discoveries in the upcoming decade.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2020

ECP Report: Update on Proxy Applications and Vendor Interactions

The ExaLearn miniGAN team (Ellis and Rajamanickam) have released miniGAN, a generative adversarial network(GAN) proxy application, through the ECP proxy application suite. miniGAN is the first machine learning proxy application in the suite (note: the ECP CANDLE project did previously release some benchmarks) and models the performance for training generator and discriminator networks. The GAN's generator and discriminator generate plausible 2D/3D maps and identify fake maps, respectively. miniGAN aims to be a proxy application for related applications in cosmology (CosmoFlow, ExaGAN) and wind energy (ExaWind). miniGAN has been developed so that optimized mathematical kernels (e.g., kernels provided by Kokkos Kernels) can be plugged into to the proxy application to explore potential performance improvements. miniGAN has been released as open source software and is available through the ECP proxy application website (https://proxyapps.exascaleproject.ordecp-proxy-appssuite/) and on GitHub (https://github.com/SandiaMLMiniApps/miniGAN). As part of this release, a generator is provided to generate a data set (series of images) that are inputs to the proxy application.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Deep inference of simulated strong lenses in ground-based surveys

The large number of strong lenses discoverable in future astronomical surveys will likely enhance the value of strong gravitational lensing as a cosmic probe of dark energy and dark matter. However, leveraging the increased statistical power of such large samples will require further development of automated lens modeling techniques. We show that deep learning and simulation-based inference (SBI) methods produce informative and reliable estimates of parameter posteriors for strong lensing systems in ground-based surveys. We present the examination and comparison of two approaches to lens parameter estimation for strong galaxy-galaxy lenses — Neural Posterior Estimation (NPE) and Bayesian Neural Networks (BNNs). We perform inference on 1-, 5-, and 12-parameter lens models for ground-based imaging data that mimics the Dark Energy Survey (DES). We find that NPE outperforms BNNs, producing posterior distributions that are more accurate, precise, and well-calibrated for most parameters. For the 12-parameter NPE model, the calibration is consistently within <10% of optimal calibration for all parameters, while the BNN is rarely within 20% of optimal calibration for any of the parameters. Similarly, residuals for most of the parameters are smaller (by up to an order of magnitude) with the NPE model than the BNN model. This work takes important steps in the systematic comparison of methods for different levels of model complexity.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2019

Toward Rate Estimation for Transient Surveys. I. Assessing Transient Detectability and Volume Sensitivity for iPTF

The past couple of decades have seen an emergence of transient detection facilities in various avenues of time-domain astronomy that have provided us with a rich data set of transients. The rates of these transients have implications in star formation, progenitor models, evolution channels, and cosmology measurements. The crucial component of any rate calculation is the detectability and spacetime volume sensitivity of a survey to a particular transient type as a function of many intrinsic and extrinsic parameters. Fully sampling that multidimensional parameter space is challenging. Instead, we present a scheme to assess the detectability of transients using supervised machine learning. The data product is a classifier that determines the detection likelihood of sources resulting from an image subtraction pipeline associated with time-domain survey telescopes, taking into consideration the intrinsic properties of the transients and the observing conditions. We apply our method to assess the spacetime volume sensitivity of type Ia supernovae (SNe Ia) in the intermediate Palomar Transient Factory (iPTF) and obtain the result, &lang; VT &rang; Ia = (2.93 ± 0.21) × 10 -2 Gpc 3 yr. With rate estimates in the literature, this volume sensitivity gives a count of 680-1160 SNe Ia detectable by iPTF, which is consistent with the archival data. With a view toward wider applicability of this technique we do a preliminary computation for long-duration type IIp supernovae (SNe IIp) and find &lang; VT &rang; IIa = (7.80 ± 0.76) × 10 -4 Gpc 3 yr. This classifier can be used for computationally fast spacetime volume sensitivity calculation of any generic transient type using their light-curve properties. Hence, it can be used as a tool to facilitate calculation of transient rates in a range of time-domain surveys, given suitable training sets.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Effective cosmic density field reconstruction with convolutional neural network

ABSTRACT We present a cosmic density field reconstruction method that augments the traditional reconstruction algorithms with a convolutional neural network (CNN). Following previous work, the key component of our method is to use the reconstructed density field as the input to the neural network. We extend this previous work by exploring how the performance of these reconstruction ideas depends on the input reconstruction algorithm, the reconstruction parameters, and the shot noise of the density field, as well as the robustness of the method. We build an eight-layer CNN and train the network with reconstructed density fields computed from the Quijote suite of simulations. The reconstructed density fields are generated by both the standard algorithm and a new iterative algorithm. In real space at z = 0, we find that the reconstructed field is 90 per cent correlated with the true initial density out to $k\sim 0.5 \, \mathrm{ h}\, \rm {Mpc}^{-1}$, a significant improvement over $k\sim 0.2 \, \mathrm{ h}\, \rm {Mpc}^{-1}$ achieved by the input reconstruction algorithms. We find similar improvements in redshift space, including an improved removal of redshift space distortions at small scales. We also find that the method is robust across changes in cosmology. Additionally, the CNN removes much of the variance from the choice of different reconstruction algorithms and reconstruction parameters. However, the effectiveness decreases with increasing shot noise, suggesting that such an approach is best suited to high density samples. This work highlights the additional information in the density field beyond linear scales as well as the power of complementing traditional analysis approaches with machine learning techniques.

Astronomy & Astrophysics↗
DOE OSTI2021

Machine learning of high dimensional data on a noisy quantum processor

Abstract Quantum kernel methods show promise for accelerating data analysis by efficiently learning relationships between input data points that have been encoded into an exponentially large Hilbert space. While this technique has been used successfully in small-scale experiments on synthetic datasets, the practical challenges of scaling to large circuits on noisy hardware have not been thoroughly addressed. Here, we present our findings from experimentally implementing a quantum kernel classifier on real high-dimensional data taken from the domain of cosmology using Google’s universal quantum processor, Sycamore. We construct a circuit ansatz that preserves kernel magnitudes that typically otherwise vanish due to an exponentially growing Hilbert space, and implement error mitigation specific to the task of computing quantum kernels on near-term hardware. Our experiment utilizes 17 qubits to classify uncompressed 67 dimensional data resulting in classification accuracy on a test set that is comparable to noiseless simulation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Milky Way Satellite Census. I. The Observational Selection Function for Milky Way Satellites in DES Y3 and Pan-STARRS DR1

We report the results of a systematic search for ultra-faint Milky Way satellite galaxies using data from the Dark Energy Survey (DES) and Pan-STARRS1 (PS1). Together, DES and PS1 provide multi-band photometry in optical/near-infrared wavelengths over ~80% of the sky. Our search for satellite galaxies targets ~25,000 deg$^2$ of the high-Galactic-latitude sky reaching a 10$\sigma$ point-source depth of $\gtrsim$ 22.5 mag in the $g$ and $r$ bands. While satellite galaxy searches have been performed independently on DES and PS1 before, this is the first time that a self-consistent search is performed across both data sets. We do not detect any new high-significance satellite galaxy candidates, while recovering the majority of satellites previously detected in surveys of comparable depth. We characterize the sensitivity of our search using a large set of simulated satellites injected into the survey data. We use these simulations to derive both analytic and machine-learning models that accurately predict the detectability of Milky Way satellites as a function of their distance, size, luminosity, and location on the sky. To demonstrate the utility of this observational selection function, we calculate the luminosity function of Milky Way satellite galaxies, assuming that the known population of satellite galaxies is representative of the underlying distribution. We provide access to our observational selection function to facilitate comparisons with cosmological models of galaxy formation and evolution.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗
DOE OSTI2017

CMU DeepLens: deep learning for automatic image-based galaxy–galaxy strong lens finding

Galaxy-scale strong gravitational lensing can not only provide a valuable probe of the dark matter distribution of massive galaxies, but also provide valuable cosmological constraints, either by studying the population of strong lenses or by measuring time delays in lensed quasars. Due to the rarity of galaxy-scale strongly lensed systems, fast and reliable automated lens finding methods will be essential in the era of large surveys such as Large Synoptic Survey Telescope, Euclid and Wide-Field Infrared Survey Telescope. To tackle this challenge, we introduce CMU DeepLens, a new fully automated galaxy-galaxy lens finding method based on deep learning. This supervised machine learning approach does not require any tuning after the training step which only requires realistic image simulations of strongly lensed systems. We train and validate our model on a set of 20 000 LSST-like mock observations including a range of lensed systems of various sizes and signal-to-noise ratios (S/N). We find on our simulated data set that for a rejection rate of non-lenses of 99 per cent, a completeness of 90 per cent can be achieved for lenses with Einstein radii larger than 1.4 arcsec and S/N larger than 20 on individual g-band LSST exposures. Finally, we emphasize the importance of realistically complex simulations for training such machine learning methods by demonstrating that the performance of models of significantly different complexities cannot be distinguished on simpler simulations. We make our code publicly available at https://github.com/McWilliamsCenter/CMUDeepLens.

gravitational lensing: strong↗
DOE OSTI2018

Dark Energy Survey Year 1 results: measurement of the baryon acoustic oscillation scale in the distribution of galaxies to redshift 1

ABSTRACT We present angular diameter distance measurements obtained by locating the baryon acoustic oscillations (BAO) scale in the distribution of galaxies selected from the first year of Dark Energy Survey data. We consider a sample of over 1.3 million galaxies distributed over a footprint of 1336 deg2 with 0.6 < $z$photo < 1 and a typical redshift uncertainty of 0.03(1 + $z$). This sample was selected, as fully described in a companion paper, using a colour/magnitude selection that optimizes trade-offs between number density and redshift uncertainty. We investigate the BAO signal in the projected clustering using three conventions, the angular separation, the comoving transverse separation, and spherical harmonics. Further, we compare results obtained from template-based and machine-learning photometric redshift determinations. We use 1800 simulations that approximate our sample in order to produce covariance matrices and allow us to validate our distance scale measurement methodology. We measure the angular diameter distance, DA, at the effective redshift of our sample divided by the true physical scale of the BAO feature, rd. We obtain close to a 4 per cent distance measurement of DA($z$eff = 0.81)/rd = 10.75 ± 0.43. These results are consistent with the flat Λ cold dark matter concordance cosmological model supported by numerous other recent experimental results.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Amortized simulation-based frequentist inference for tractable and intractable likelihoods

Abstract High-fidelity simulators that connect theoretical models with observations are indispensable tools in many sciences. If the likelihood is known, inference can proceed using standard techniques. However, when the likelihood is intractable or unknown, a simulator makes it possible to infer the parameters of a theoretical model directly from real and simulated observations when coupled with machine learning. We introduce an extension of the recently proposed likelihood-free frequentist inference ( LF2I ) approach that makes it possible to construct confidence sets with the p -value function and to use the same function to check the coverage explicitly at any given parameter point. Like LF2I , this extension yields provably valid confidence sets in parameter inference problems for which a high-fidelity simulator is available. The utility of our algorithm is illustrated by applying it to three pedagogically interesting examples: the first is from cosmology, the second from high-energy physics and astronomy, both with tractable likelihoods, while the third, with an intractable likelihood, is from epidemiology 3 3 Code to reproduce all of our results is available on https://github.com/AliAlkadhim/ALFFI . .

Computer Science↗
DOE OSTI2018

Star-galaxy classification in the Dark Energy Survey Y1 dataset

Here, we perform a comparison of different approaches to star-galaxy classification using the broad-band photometric data from Year 1 of the Dark Energy Survey. We explore and compare different techniques to address this problem: morphological versus using additional flux information, differences between machine-learning, cut based and template fitting including Hierarchical Bayesian methods and catalogue-based versus image-based analyses. This is done by performing a wide range of tests with and without external ‘truth’ information, which can be ported to other similar datasets. We make a broad evaluation of the performance of the classifiers in two science cases with DES data that are most affected by this systematic effect: large-scale structure and Milky Way studies. In general, even though the default morphological classifiers used for DES Y1 cosmology studies are sufficient to maintain a low level of systematic contamination from stellar mis-classification, contamination can be reduced to the O (1%) level by using multi-epoch and infrared information from external datasets. For Milky Way studies the stellar sample can be augmented by ~20% for a given flux limit. Reference catalogues used in this work are available at http://des.ncsa.illinois.edu/releases/y1a1.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Comparing Automated Posterior Estimation Techniques for Modeling Strong Lenses In Ground-based Survey Data

Current and future ground-based cosmological surveys, such as the Dark Energy Survey (DES), and the Vera Rubin Observatory Legacy Survey of Space and Time (LSST), are predicted to discover thousands to tens of thousands of strong gravitational lenses. The large number of strong lenses discoverable in future surveys will make strong lensing a highly competitive and complementary cosmic probe. However, conventional lens modeling techniques are unable to scale up to the sheer number of lenses that will be discovered through upcoming surveys. Therefore, the use of automated lens analysis techniques is necessary. We demonstrate that machine learning methods can be used to automate the inference of informative model posteriors of strong lensing systems in ground-based surveys with credible uncertainty estimation. We present two Simulation-Based Inference (SBI) approaches for lens parameter estimation of galaxy-galaxy lenses. We demonstrate applications of Neural Posteriors Estima tors (NPEs) and Bayesian Neural Network (BNNs) to automate the inference of a 12-parameter lensing system for DES-like ground-based imaging data. We apply a suite of diagnostics (e.g., posterior coverage and SBC) to validate the performance of our methods. We find that NPEs outperform the BNN, producing posterior distributions that are for the most part both more accurate and more precise; in particular, several source-light model parameters are systematically biased in the BNN implementation.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Self-consistent redshift estimation using correlation functions without a spectroscopic reference sample

ABSTRACT We present a new method to estimate redshift distributions and galaxy-dark matter bias parameters using correlation functions in a fully data driven and self-consistent manner. Unlike other machine learning, template, or correlation redshift methods, this approach does not require a reference sample with known redshifts. By measuring the projected cross- and auto-correlations of different galaxy sub-samples, e.g. as chosen by simple cells in colour–magnitude space, we are able to estimate the galaxy-dark matter bias model parameters, and the shape of the redshift distributions of each sub-sample. This method fully marginalizes over a flexible parametrization of the redshift distribution and galaxy-dark matter bias parameters of sub-samples of galaxies, and thus provides a general Bayesian framework to incorporate redshift uncertainty into the cosmological analysis in a data-driven, consistent, and reproducible manner. This result is improved by an order of magnitude by including cross-correlations with the cosmic microwave background and with galaxy–galaxy lensing. We showcase how this method could be applied to real galaxies. By using idealized data vectors, in which all galaxy-dark matter model parameters and redshift distributions are known, this method is demonstrated to recover unbiased estimates on important quantities, such as the offset Δz between the mean of the true and estimated redshift distribution and the 68 per cent, 95 per cent, and 99.5 per cent widths of the redshift distribution to an accuracy required by current and future surveys.

79 ASTRONOMY AND ASTROPHYSICS↗