Search NASA⌕ Search

SEARCH · Search NASA

Results for “data set”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Two Weeks In The Life of Skybridge

This report documents a new large public data set for researchers studying the behavior of large commodity high-performance computing systems. Such large data sets are typically confined within institutions and access to them is limited to institutional partners. We provide it to promote HPC research more widely. The data set provides a two week time series of performance data collected once per minute using the Lightweight Distributed Metric Service from the system Skybridge at Sandia National Laboratories and the corresponding job-level accounting information. General system log information is not provided.

97 MATHEMATICS AND COMPUTING↗

Efficient First-Order Algorithms for Large-Scale, Non-Smooth Maximum Entropy Models with Application to Wildfire Science

Maximum entropy (MaxEnt) models are a class of statistical models that use the maximum entropy principle to estimate probability distributions from data. Due to the size of modern data sets, MaxEnt models need efficient optimization algorithms to scale well for big data applications. State-of-the-art algorithms for MaxEnt models, however, were not originally designed to handle big data sets; these algorithms either rely on technical devices that may yield unreliable numerical results, scale poorly, or require smoothness assumptions that many practical MaxEnt models lack. In this paper, we present novel optimization algorithms that overcome the shortcomings of state-of-the-art algorithms for training large-scale, non-smooth MaxEnt models. Our proposed first-order algorithms leverage the Kullback–Leibler divergence to train large-scale and non-smooth MaxEnt models efficiently. For MaxEnt models with discrete probability distribution of n elements built from samples, each containing m features, the stepsize parameter estimation and iterations in our algorithms scale on the order of O(mn) operations and can be trivially parallelized. Moreover, the strong ℓ1 convexity of the Kullback–Leibler divergence allows for larger stepsize parameters, thereby speeding up the convergence rate of our algorithms. To illustrate the efficiency of our novel algorithms, we consider the problem of estimating probabilities of fire occurrences as a function of ecological features in the Western US MTBS-Interagency wildfire data set. Our numerical results show that our algorithms outperform the state of the art by one order of magnitude and yield results that agree with physical models of wildfire occurrence and previous statistical analyses of wildfire drivers.

Physics↗

Migrating data from TcSE to DOORS : an evaluation of the T-Plan Integrator software application.

This report describes our evaluation of the T-Plan Integrator software application as it was used to transfer a real data set from the Teamcenter for Systems Engineering (TcSE) software application to the DOORS software application. The T-Plan Integrator was evaluated to determine if it would meet the needs of Sandia National Laboratories to migrate our existing data sets from TcSE to DOORS. This report presents the struggles of migrating data and focuses on how the Integrator can be used to map a data set and its data architecture from TcSE to DOORS. Finally, this report describes how the bulk of the migration can take place using the Integrator; however, about 20-30% of the data would need to be transferred from TcSE to DOORS manually. This report does not evaluate the transfer of data from DOORS to TcSE.

99 GENERAL AND MISCELLANEOUS↗

The International Soil Moisture Network: serving Earth system science for over a decade

In 2009, the International Soil Moisture Network (ISMN) was initiated as a community effort, funded by the European Space Agency, to serve as a centralised data hosting facility for globally available in situ soil moisture measurements (Dorigo et al., 2011b, a). The ISMN brings together in situ soil moisture measurements collected and freely shared by a multitude of organisations, harmonises them in terms of units and sampling rates, applies advanced quality control, and stores them in a database. Users can freely retrieve the data from this database through an online web portal (https://ismn.earth/en/, last access: 28 October 2021). Meanwhile, the ISMN has evolved into the primary in situ soil moisture reference database worldwide, as evidenced by more than 3000 active users and over 1000 scientific publications referencing the data sets provided by the network. As of July 2021, the ISMN now contains the data of 71 networks and 2842 stations located all over the globe, with a time period spanning from 1952 to the present. The number of networks and stations covered by the ISMN is still growing, and approximately 70% of the data sets contained in the database continue to be updated on a regular or irregular basis. The main scope of this paper is to inform readers about the evolution of the ISMN over the past decade, including a description of network and data set updates and quality control procedures. A comprehensive review of the existing literature making use of ISMN data is also provided in order to identify current limitations in functionality and data usage and to shape priorities for the next decade of operations of this unique community-based data repository.

54 ENVIRONMENTAL SCIENCES↗

Exploring the Effects of Population and Employment Characteristics on Truck Flows: An Analysis of NextGen NHTS Origin-Destination Data

Truck transportation remains the dominant mode of US freight transportation because of its advantages, such as the flexibility of accessing pickup and drop-off points and faster delivery. Because of the massive freight volume transported by trucks, understanding the effects of population and employment characteristics on truck flows is critical for better transportation planning and investment decisions. The US Federal Highway Administration published a truck travel origin-destination data set as part of the Next Generation National Household Travel Survey program. This data set contains the total number of truck trips in 2020 within and between 583 predefined zones encompassing metropolitan and nonmetropolitan statistical areas within each state and Washington, DC. In this study, origin-destination-level truck trip flow data was augmented to include zone-level population and employment characteristics from the US Census Bureau. Census population and County Business Patterns data were included. The final data set was used to train a machine learning algorithm-based model, Extreme Gradient Boosting (XGBoost), where the target variable is the number of total truck trips. Shapley Additive ExPlanation (SHAP) was adopted to explain the model results. Results showed that the distance between the zones was the most important variable and had a nonlinear relationship with truck flows.

Uddin, Majbah↗

Novel Histogram Based Unsupervised Classification Technique to Determine Natural Classes From Biophysically Relevant Fit Parameters to Hyperspectral Data

Hyperspectral image analysis has benefited from an array of methods that take advantage of the increased spectral depth compared to multispectral sensors; however, the focus of these developments has been on supervised classification methods. Lack of a priori knowledge regarding land cover characteristics can make unsupervised classification methods preferable under certain circumstances. An unsupervised classification technique is presented in this paper that utilizes physically relevant basis functions to model the reflectance spectra. These fit parameters used to generate the basis functions allow clustering based on spectral characteristics rather than spectral channels and provide both noise and data reduction. Histogram splitting of the fit parameters is then used as a means of producing an unsupervised classification. Unlike current unsupervised classification techniques that rely primarily on Euclidian distance measures to determine similarity, the unsupervised classification technique uses the natural splitting of the fit parameters associated with the basis functions creating clusters that are similar in terms of physical parameters. The data set used in this work utilizes the publicly available data collected at Indian Pines, Indiana. This data set provides reference data allowing for comparisons of the efficacy of different unsupervised data analysis. The unsupervised histogram splitting technique presented in this paper is shown to be better than the standard unsupervised ISODATA clustering technique with an overall accuracy of 34.3/19.0% before merging and 40.9/39.2% after merging. Finally, this improvement is also seen as an improvement of kappa before/after merging of 24.8/30.5 for the histogram splitting technique compared to 15.8/28.5 for ISODATA.

58 GEOSCIENCES↗

The Dark Energy Survey Data Release I

We describe the first public data release of the Dark Energy Survey, DES DR1, consisting of reduced single-epoch images, co-added images, co-added source catalogs, and associated products and services assembled over the first 3 yr of DES science operations. DES DR1 is based on optical/near-infrared imaging from 345 distinct nights (2013 August to 2016 February) by the Dark Energy Camera mounted on the 4 m Blanco telescope at the Cerro Tololo InterAmerican Observatory in Chile. We release data from the DES wide-area survey covering similar to 5000 deg(2) of the southern Galactic cap in five broad photometric bands, grizY. DES DR1 has a median delivered point-spread function of g = 1.12, r = 0.96, i = 0.88, z = 0.84, and Y = 0.'' 90 FWHM, a photometric precision of <1% in all bands, and an astrometric precision of 151 mas. The median co-added catalog depth for a 1.'' 95 diameter aperture at signal-to-noise ratio (S/N) = 10 is g = 24.33, r = 24.08, i = 23.44, z = 22.69, and Y = 21.44 mag. DES DR1 includes nearly 400 million distinct astronomical objects detected in similar to 10,000 co-add tiles of size 0.534 deg(2) produced from similar to 39,000 individual exposures. Benchmark galaxy and stellar samples contain similar to 310 million and similar to 80 million objects, respectively, following a basic object quality selection. These data are accessible through a range of interfaces, including query web clients, image cutout servers, jupyter notebooks, and an interactive co-add image visualization tool. DES DR1 constitutes the largest photometric data set to date at the achieved depth and photometric precision.

Abbott, T. M. C.↗

Assimilating compact phase space retrievals of atmospheric composition with WRF-Chem/DART: a regional chemical transport/ensemble Kalman filter data assimilation system

This study introduces the Weather Research and Forecasting Model with chemistry/Data Assimilation Research Testbed (WRF-Chem/DART) chemical transport forecasting/data assimilation system together with the assimilation of ''compact phase space retrievals'' of satellite-derived atmospheric composition products. WRF-Chem is a state-of-the-art chemical transport model. DART is a flexible software environment for researching ensemble data assimilation with different assimilation and forecast model options. DART's primary assimilation tool is the ensemble adjustment Kalman filter. WRF-Chem/DART is applied to the assimilation of Terra/Measurement of Pollution in the Troposphere (MOPITT) carbon monoxide (CO) trace gas retrieval profiles. Those CO observations are first assimilated as quasi-optimal retrievals (QORs). Our results show that assimilation of the CO retrievals (i) reduced WRF-Chem's CO bias in retrieval and state space, and (ii) improved the CO forecast skill by reducing the Root Mean Square Error (RMSE) and increasing the Coefficient of Determination ($R^2$). Those CO forecast improvements were significant at the 95 % level. Trace gas retrieval data sets contain (i) large amounts of data with limited information content per observation, (ii) error covariance cross-correlations, and (iii) contributions from the retrieval prior profile that should be removed before assimilation. Those characteristics present challenges to the assimilation of retrievals. This paper addresses those challenges by introducing the assimilation of compact phase space retrievals (CPSRs). CPSRs are obtained by preprocessing retrieval data sets with an algorithm that (i) compresses the retrieval data, (ii) diagonalizes the error covariance, and (iii) removes the retrieval prior profile contribution. Most modern ensemble assimilation algorithms can efficiently assimilate CPSRs. Our results show that assimilation of MOPITT CO CPSRs reduced the number of observations (and assimilation computation costs) by ~ 35 %, while providing CO forecast improvements comparable to or better than with the assimilation of MOPITT CO QORs.

58 GEOSCIENCES↗

Consequences of neglecting the interannual variability of the solar resource: A case study of photovoltaic power among the Hawaiian Islands

The interannual variability of the solar irradiance and meteorological conditions are often ignored in favor of single-year data sets for modeling power generation and evaluating the economic value of photovoltaic (PV) power systems. Yet interannual variability significantly impacts the generation from one year to another of renewable power systems such as wind and PV. Consequently, the interannual variability of power generation corresponds to the interannual variability of capital returns on investment. The penetration of PV systems within the Hawaiian Electric Companies' portfolio has rapidly accelerated in recent years and is expected to continue to increase given the state's energy objectives laid out by the Hawaii Clean Energy Initiative. We use the National Solar Radiation Database (1998-2015) to characterize the interannual variability of the solar irradiance and meteorological conditions across the State of Hawaii. These data sets are passed to the National Renewable Energy Laboratory's System Advisory Model (SAM) to calculate an 18-year PV power generation data set to characterize the variability of PV power generation. We calculate the interannual coefficient of variability (COV) for annual average global horizontal irradiance (GHI) on the order of 2% and COV for annual capacity factor on the order of 3% across the Hawaiian archipelago. Regarding the interannual variability of seasonal trends, we calculate the COV for monthly average GHI values on the order of 5% and COV for monthly capacity factor on the order of 10%. We model residential-scale and utility-scale PV systems and calculate the economic returns of each system via the payback period and the net present value. We demonstrate that studies based on single-year data sets for economic evaluations reach conclusions that deviate from the true values realized by accounting for interannual variability.

14 SOLAR ENERGY↗

Uncertainty in gridded CO 2 emissions estimates

We are interested in the spatial distribution of fossil-fuel-related emissions of CO 2 for both geochemical and geopolitical reasons, but it is important to understand the uncertainty that exists in spatially explicit emissions estimates. Working from one of the widely used gridded data sets of CO 2 emissions, we examine the elements of uncertainty, focusing on gridded data for the United States at the scale of 1° latitude by 1° longitude. Uncertainty is introduced in the magnitude of total United States emissions, the magnitude and location of large point sources, the magnitude and distribution of non-point sources, and from the use of proxy data to characterize emissions. For the United States, we develop estimates of the contribution of each component of uncertainty. At 1° resolution, in most grid cells, the largest contribution to uncertainty comes from how well the distribution of the proxy (in this case population density) represents the distribution of emissions. In other grid cells, the magnitude and location of large point sources make the major contribution to uncertainty. Uncertainty in population density can be important where a large gradient in population density occurs near a grid cell boundary. Uncertainty is strongly scale-dependent with uncertainty increasing as grid size decreases. In conclusion, uncertainty for our data set with 1° grid cells for the United States is typically on the order of ±150%, but this is perhaps not excessive in a data set where emissions per grid cell vary over 8 orders of magnitude.

54 ENVIRONMENTAL SCIENCES↗

Albedo Data for Bifacial PV Systems Update

For use by the PV and financial communities to better estimate the performance and to reduce the risk of bifacial PV systems, data sets of ground albedo and associated meteorological data were developed by using existing measurement network data and data contributed by the PV industry. The data sets include time-series data as well as summary information of tabular monthly and yearly data and plots of monthly and hourly albedo values. Complete information is presented in a user’s guide and data are available for download from NREL’s DuraMAT website.

albedo↗

Impact of heavy quark and quarkonium data on nuclear gluon PDFs

A clear understanding of nuclear parton distribution functions (nPDFs) plays a crucial role in the interpretation of collider data taken at the Relativistic Heavy Ion Collider, the Large Hadron Collider (LHC), and in the near future at the Electron-Ion Collider. Even with the recent inclusions of vector boson and light meson production data, the uncertainty of the gluon PDF remains substantial and limits the interpretation of heavy ion collision data. To obtain new constraints on the nuclear gluon PDF, we extend our recent nCTEQ15WZ+SIH analysis to inclusive quarkonium and open heavy-flavor meson production data from the LHC. This vast new data set covers a wide kinematic range and puts strong constraints on the nuclear gluon PDF down to x ≲ 10 -5 . The theoretical predictions for these data sets are obtained from a data-driven approach, where proton-proton data are used to determine effective scattering matrix elements. This approach is validated with detailed comparisons to existing next-to-leading order calculations in nonrelativistic QCD for quarkonia and in the general-mass variable-flavor-number scheme for the open heavy-flavored mesons. In addition, the uncertainties from the data-driven approach are determined using the Hessian method and accounted for in the PDF fits. This extension of our previous analyses represents an important step toward the next generation of PDFs not only by including new data sets, but also by exploring new methods for future analyses.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Solar Resource Calibration, Measurement, and Dissemination: Final Report FY 2016-FY 2018

The Solar Resource Calibration, Measurement, and Dissemination project supports the U.S. Department of Energy's Solar Energy Technologies Office (SETO) initiative to make solar energy cost-competitive with other forms of electricity by improving the tools and methods to measure and model solar radiation and therefore reduce uncertainty in predicting solar output and improve the bankability of solar projects. This project has three tasks that conduct research on advancing solar resource measurements and modeling: Task 1: Applied solar Radiation Measurements. Task 2: National Solar Radiation Data Base (NSRDB). Task 3: Knowledge Sharing. The National Renewable Energy Laboratory's (NREL's) Solar Radiation Research Laboratory (SRRL) activities ensure traceable solar measurements throughout the United States that are essential for project feasibility, due diligence, financing, and plant operations. The SRRL houses NREL's Baseline Measurement System (BMS), which provides a platform for improving the accuracy of solar measurements through research to improve the calibration and characterization of instruments and enable the development of new measurement technology and standards. The platform contains a variety of instruments measuring solar radiation and other meteorological parameters. Following best practices, BMS instruments are calibrated to ensure traceability to the World Radiometric Reference in compliance with ISO/IEC 17025 accreditation requirements. The BMS data sets are quality-controlled and made publicly available through the Measurement and Instrumentation Data Center website. The NSRDB is the most accessed public database of solar irradiance and associated weather parameters for use in energy modeling. It supports SETO's goals of reducing barriers to high penetrations of solar technologies by providing easy access to high-quality, foundational data essential for innovative product development and downstream modeling through the NSRDB website. Data are currently available for the contiguous United States and beyond for 20 years covering 1998-2017. The Physical Solar Model (PSM) that underpins the current NSRDB data sets is a unique physics-based model that has been developed at NREL and has opened the door to the use of next-generation satellite data sets for solar resource assessment and forecasting. Through the American Society for Testing Materials (ASTM) International and other international standards organizations, NREL continues to provide leadership in the development of standards that are relevant to the measurement and modeling of solar radiation for solar energy applications. Standards developed cover the calibration method of pyrheliometers (direct normal irradiance measurements), pyranometers (global horizontal irradiance measurements), spectroradiometers, ultraviolet radiometers, correct field deployment of these instruments, spectral distribution reference tables for photovoltaic modeling, and transfer of indoor calibrations. Further, NREL leads the International Energy Agency (IEA) Photovoltaic Power Systems Programme Task 16: Subtask 1, which focuses on the evaluation of current and emerging resource assessment methodologies. NREL and the IEA updated the Best Practices Handbook for the Collection and Use of Solar Resource Data for Solar Energy Applications, which is widely used by the solar industry.

14 SOLAR ENERGY↗

Heavy-Duty Vehicle Activity Updates for MOVES Using NREL Fleet DNA and CE-CERT Data

The U.S. Environmental Protection Agency's (EPA's) Motor Vehicle Emission Simulator (MOVES) is a publicly available tool used by researchers and policymakers to help understand motor vehicle emission sources at a national, county, and project level. Estimates of heavy-duty activity in the most recent version of the model at the time this work was conducted, MOVES2014, was identified as an area in need of improvement. The start activity in MOVES2014 is based on a limited and dated data set. In addition, MOVES2014 relies on drive cycles that represent on-network activity but do not account for idling activity that occurs on off-network roads, such as at a distribution center, while the truck is queuing or during loading and unloading. As a result, MOVES2014 may currently underestimate the number of starts and idle and soak time for heavy-duty trucks in real-world operation. The National Renewable Energy Laboratory (NREL) has previously leveraged its expansive Fleet DNA database of heavy-duty vehicles to idle and start activity for six of the nine heavy-duty vehicle source types of classes in the MOVES model. The data available in Fleet DNA from 416 conventional, diesel-powered vehicles provided activity estimates from more than 120,000 hours of operation throughout 14,682 vehicle days between October 2006 and January 2016. NREL calculated start fraction, starts per day, soak fraction, and idle fraction by hour of the day for each vehicle type, state, and vocation, and provided results in .CSV files that can be translated to MOVES table inputs. The idle and start activity from this initial analysis of Fleet DNA data was used to develop default idle and start data for heavy-duty vehicles in MOVES3. Satisfied with the results from the Fleet DNA data used for MOVES3, the EPA asked NREL to extend this start/soak/idle analysis using additional data from a larger number of vehicles for a potential future update to the MOVES model. Such a data set was achieved from a project led by the University of California at Riverside, College of Engineering, Center for Environmental Research & Technology (CE-CERT) and funded by California Air Resources Board. Specifically, this data set consists of 90 heavy-duty vehicles operated mainly in California, which can be separated into five of the nine heavy-duty vehicle classes in the MOVES model. In addition, the heavy-duty activity database collected by CE-CERT provided activity estimates from more than 44,000 hours of operation throughout 4,724 vehicle days between November 2014 and September 2016. This report details the analysis of the heavy-duty activity database collected from the University of California at Riverside by providing graphical analysis and context for the start, soak, and idle distributions. The comparison of the related results from both the Fleet DNA and CE-CERT data sets are documented as well.

33 ADVANCED PROPULSION SYSTEMS↗

An improved method to increase the predictive accuracy of the ECR technique

The technique of electrical conductivity relaxation (ECR) is a widely utilized method for obtaining oxygen chemical surface exchange (k) and bulk diffusion (D) coefficients for mixed ionic electronic conductors. Although the technique is conceptually simple, its practical application does not frequently produce reliably accurate values of k and D, and many examples of poor agreement can be found in the published literature. In this present paper, methods for enhancing the accuracy of the ECR technique relative to data processing are described in detail. Physically accurate determination of the actual starting time (t{sub 0}) and improved methods for data fitting of ECR measurements are discussed. To test the methods described, generated relaxation data sets are analyzed by using non-linear-least-square (NLLS) methods. The results showed that fixing t{sub 0} based on an ideally stirred reactor model improves the fitted results compared to treating t{sub 0} as a fitted parameter. Additionally, evaluation of the global minimum in the fitted error is made by analyzing multiple samples of varying thickness, which improves parameter evaluation compared to analysis of a single relaxation data set. The improved accuracy of the computed values of k and D is described in comparison to conventional methods.

Bulk diffusion coefficient↗

Identification of functional elements and regulatory circuits by Drosophila modENCODE

To gain insight into how genomic information is translated into cellular and developmental programs, the Drosophila model organism Encyclopedia of DNA Elements (modENCODE) project is comprehensively mapping transcripts, histone modifications, chromosomal proteins, transcription factors, replication proteins and intermediates, and nucleosome properties across a developmental time course and in multiple cell lines. We have generated more than 700 data sets and discovered protein-coding, noncoding, RNA regulatory, replication, and chromatin elements, more than tripling the annotated portion of the Drosophila genome. Correlated activity patterns of these elements reveal a functional regulatory network, which predicts putative new functions for genes, reveals stage- and tissue-specific regulators, and enables gene-expression prediction. Our results provide a foundation for directed experimental and computational studies in Drosophila and related species and also a model for systematic data integration toward comprehensive genomic and functional annotation. Several years after the complete genetic sequencing of many species, it is still unclear how to translate genomic information into a functional map of cellular and developmental programs. The Encyclopedia of DNA Elements (ENCODE) (1) and model organism ENCODE (modENCODE) (2) projects use diverse genomic assays to comprehensively annotate the Homo sapiens (human), Drosophila melanogaster (fruit fly), and Caenorhabditis elegans (worm) genomes, through systematic generation and computational integration of functional genomic data sets. Previous genomic studies in flies have made seminal contributions to our understanding of basic biological mechanisms and genome functions, facilitated by genetic, experimental, computational, and manual annotation of the euchromatic and heterochromatic genome (3), small genome size, short life cycle, and a deep knowledge of development, gene function, and chromosome biology. The functions of {approx}40% of the protein and nonprotein-coding genes [FlyBase 5.12 (4)] have been determined from cDNA collections (5, 6), manual curation of gene models (7), gene mutations and comprehensive genome-wide RNA interference screens (8-10), and comparative genomic analyses (11, 12). The Drosophila modENCODE project has generated more than 700 data sets that profile transcripts, histone modifications and physical nucleosome properties, general and specific transcription factors (TFs), and replication programs in cell lines, isolated tissues, and whole organisms across several developmental stages (Fig. 1). Here, we computationally integrate these data sets and report (i) improved and additional genome annotations, including full-length proteincoding genes and peptides as short as 21 amino acids; (ii) noncoding transcripts, including 132 candidate structural RNAs and 1608 nonstructural transcripts; (iii) additional Argonaute (Ago)-associated small RNA genes and pathways, including new microRNAs (miRNAs) encoded within protein-coding exons and endogenous small interfering RNAs (siRNAs) from 3-inch untranslated regions; (iv) chromatin 'states' defined by combinatorial patterns of 18 chromatin marks that are associated with distinct functions and properties; (v) regions of high TF occupancy and replication activity with likely epigenetic regulation; (vi)mixed TF and miRNA regulatory networks with hierarchical structure and enriched feed-forward loops; (vii) coexpression- and co-regulation-based functional annotations for nearly 3000 genes; (viii) stage- and tissue-specific regulators; and (ix) predictive models of gene expression levels and regulator function.

60 APPLIED LIFE SCIENCES↗

Astronomaly at scale: searching for anomalies amongst 4 million galaxies

ABSTRACT Modern astronomical surveys are producing data sets of unprecedented size and richness, increasing the potential for high-impact scientific discovery. This possibility, coupled with the challenge of exploring a large number of sources, has led to the development of novel machine-learning-based anomaly detection approaches, such as astronomaly. For the first time, we test the scalability of astronomaly by applying it to almost 4 million images of galaxies from the Dark Energy Camera Legacy Survey. We use a trained deep learning algorithm to learn useful representations of the images and pass these to the anomaly detection algorithm isolation forest, coupled with astronomaly’s active learning method, to discover interesting sources. We find that data selection criteria have a significant impact on the trade-off between finding rare sources such as strong lenses and introducing artefacts into the data set. We demonstrate that active learning is required to identify the most interesting sources and reduce artefacts, while anomaly detection methods alone are insufficient. Using astronomaly, we find 1635 anomalies among the top 2000 sources in the data set after applying active learning, including eight strong gravitational lens candidates, 1609 galaxy merger candidates, and 18 previously unidentified sources exhibiting highly unusual morphology. Our results show that by leveraging the human–machine interface, astronomaly is able to rapidly identify sources of scientific interest even in large data sets.

Astronomy & Astrophysics↗

PARAFAC_T1

This is a method of performing trilinear analysis on large data sets using a modification of the PARAFAC-ALS algorithm. It iteratively decomposes the data matrix into a core matrix and three loading matrices based on the Tucker1 model. The algorithm is particularly useful for data sets that are too large to upload into a computer?s main memory. While the performance advantage in utilizing our algorithm is dependent on the number of data elements and dimensions of the data array, we have seen a significant performance improvement over operating PARAFAC-ALS on the full data set. SAND2020-12649 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Van Benthem, Mark↗