Search NASA⌕ Search

SEARCH · Search NASA

Results for “randomized sampling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Basic properties of eutectic chloride salts NaCl-KCl-ZnCl 2 and NaCl-KCl-MgCl 2 as HTFs and thermal storage media measured using simultaneous DSC-TGA

Concentrated solar power (CSP) technologies tend to work at high temperatures for better energy efficiency, which need high temperature heat transfer fluids (HTF) and thermal storage. A good HTF should have low melting point and be thermally stable at high temperatures. A thermal storage medium needs a high heat capacity in order to store more energy in per unit of mass or volume. The objective of this work is to study the basic thermophysical properties of several eutectic molten salts for HTF and thermal storage media, which can work at upper limit temperatures of 850 °C. The eutectic salts are mixtures by NaCl-KCl-ZnCl 2 , and by NaCl-KCl-MgCl 2 , respectively. The heat capacity, melting point, and heat of fusion of the eutectic salts at different temperatures were measured using Differential Scanning Calorimetry (DSC) and Thermal Gravimetric Analysis (TGA) simultaneously. Details of the measurements and obtained experience of operating the equipment are provided in the work. For each of the eutectic salts multiple samples were tested to check repeatability and to minimize random errors. Finally, in theoretical analysis, mixing rules from literature was used to estimate the heat capacity of the new HTFs, which showed good agreement with experimental data.

14 SOLAR ENERGY↗
DOE OSTI2019

Challenges and opportunities in attosecond and XFEL science

In 2009, the world’s first hard X-FEL brought a ten-billion-fold gain in X-ray peak brightness. This fuelled the avidly pursued, but still elusive, dream of single-shot imaging of non-crystalline samples with near-atomic resolution. The overarching concept of ‘diffract-before-destroy’ imaging is that a powerful burst of ultrafast X-rays can produce a coherent diffraction pattern of a nanoscale object — before the object explodes due to Coulomb repulsion caused by a massively parallel ionization of the atoms in the sample. A collection of snapshots of identical objects in random orientations then provide information from which a 3D structure can be retrieved.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Hot Stars with Kepler Planets Have High Obliquities

It has been known for a decade that hot stars with hot Jupiters tend to have high obliquities. Less is known about the degree of spin–orbit alignment for hot stars with other kinds of planets. Here, we reassess the obliquities of hot Kepler stars with transiting planets smaller than Neptune, based on spectroscopic measurements of their projected rotation velocities (vsin i). The basis of the method is that a lower obliquity—all other things being equal—causes sin i to be closer to unity and increases the value of vsin i. We sought evidence for this effect using a sample of 150 Kepler stars with effective temperatures between 5950 and 6550 K and a control sample of 101 stars with matching spectroscopic properties and random orientations. The planet hosts have systematically higher values of vsin i than the control stars, but not by enough to be compatible with perfect spin–orbit alignment. The mean value of sin i is 0.856 ± 0.036, which is 4σ away from unity (perfect alignment), and 2σ away from π/4 (random orientations). There is also evidence that the hottest stars have a broader obliquity distribution: when modeled separately, the stars cooler than 6250 K have 〈sin i〉=0.928±0.042 while the hotter stars are consistent with random orientations. This is similar to the pattern previously noted for stars with hot Jupiters. Based on these results, obliquity excitation for early-G and late-F stars appears to be a general outcome of star and planet formation, rather than being exclusively linked to hot Jupiter formation.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Effect of Heat Treatment on Microstructure and Mechanical Property of 316L Stainless Steel Produced by Laser Powder Bed Fusion

The advanced non-light water reactor designs (Gen IV reactors), including molten salt/ very high temperature/ sodium-cooled and lead-cooled fast reactors, typically operate at higher temperatures and more extreme radiation conditions than light water reactors. An intrinsic part of the deployment and progress of Gen IV reactor designs is selecting the most suitable structural material for a specific application. Additive manufacturing (AM), a fairly new process of making physical, three-dimensional objects from a computer design file, is going to completely change the way of design, build and certify nuclear systems. It offers a range of opportunities to produce complex geometries from existing materials, offers new routes for processing of previously difficult to process materials, allows for design of new high-performance materials, and finally facilitates hybridization of dissimilar materials. This emerging technology has successfully produced cars, wind turbine blade molds and even live cells. It could also open up big opportunities for the nuclear industry to quickly deploy technologies at a fraction of the cost. So far, AM techniques have been preliminarily applied in the field of nuclear reactors, including the classical parts such as the pressure vessel of a small reactor with 508-III steel, the bottom nozzle of a fuel assembly with 304L steel, the fuel cladding with zirconium alloy and the integrated impeller of a pump and the multi-channel valve body with 316L steel [6,7]. The AM applications for operating nuclear reactors started in auxiliary plant components and have slowly migrated to metallic reactors and core components, but many of these are not safety critical components. Although many parts used for nuclear reactors have been fabricated by AM techniques, practical applications in engineering are still a long way off due to the uncertainty factors focused on the processing, material properties, analysis methods and application standards, which feeds the safety and life-cycle of the nuclear reactor. Due to rapid, repeated heating and cooling during production, a high dislocation density was present in the AM material. This microstructure feature is unstable at elevated temperature while high temperature is one of the typical operation environments for nuclear reactors. Thus, it is important to understand the thermal effect on the microstructure of AM material. The objectives of this study are to investigate the effect of heat treatment on the microstructure and mechanical properties of 316L stainless steel produced by laser powder bed fusion additive manufacturing, and to determine an appropriate heat treatment practice that will be applied to the lightweight AM lattice-structured material with the same chemistry. The heat treatment study consisted of annealing the samples at a temperature range of 800 to 1200 oC with a 50 oC increment for different times (1-24 hours), followed by vacuum or air cooling. Microstructural characterization was carried out by Scanning Electron Microscope (SEM). Grain size and crystallographic orientation were investigated by Electron Backscatter Diffraction (EBSD). Vickers hardness tests with a 0.5 kg load were employed to determine the hardness of samples after different heat treatments. After heat treatment, the random crystallographic orientation was preserved, and the volume fraction of high-angle grain boundaries (grain boundary misorientation =15 oC) remained the same. The dislocation density decreased with annealing temperature due to recovery. The fine subgrain structures in the as-printed specimen were quite stable up to 1200 oC. Minimal recrystallization was observed up to 1200 oC. Recrystallization initiated only after 8.5 hours at 1200 oC. The SEM images did not show obvious dependence of microstructure on cooling rate. The hardness of the specimens decreased with increasing annealing temperature as a result of the decrease in dislocation density. It is interesting to note that the AM material showed very similar hardness to the wrought material when annealing at similar temperature, although the microstructures are very different. Annealing at 1050 oC for 1 hour followed by air cooling was selected as the heat treatment procedure for the lattice designed lightweight AM 316L material.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Photometric redshift estimation of BASS DR3 quasars by machine learning

ABSTRACT Correlating Beijing–Arizona Sky Survey (BASS) data release 3 (DR3) catalogue with the ALLWISE data base, the data from optical and infrared information are obtained. The quasars from Sloan Digital Sky Survey are taken as training and test samples while those from LAMOST are considered as external test sample. We propose two schemes to construct the redshift estimation models with XGBoost, CatBoost, and Random Forest. One scheme (namely one-step model) is to predict photometric redshifts directly based on the optimal models created by these three algorithms; the other scheme (namely two-step model) is to first classify the data into low- and high-redshift data sets, and then predict photometric redshifts of these two data sets separately. For one-step model, the performance of these three algorithms on photometric redshift estimation is compared with different training samples, and CatBoost is superior to XGBoost and Random Forest. For two-step model, the performances of these three algorithms on the classification of low and high redshift subsamples are compared, and CatBoost still shows the best performance. Therefore, CatBoost is regarded as the core algorithm of classification and regression in two-step model. In contrast to one-step model, two-step model is optimal when predicting photometric redshift of quasars, especially for high-redshift quasars. Finally, the two models are applied to predict photometric redshifts of all quasar candidates of BASS DR3. The number of high-redshift quasar candidates is 3938 (redshift ≥3.5) and 121 (redshift ≥4.5) by two-step model. The predicted result will be helpful for quasar research and follow-up observation of high-redshift quasars.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Sampling-Based Motion Planning Algorithms for Replanning and Spatial Load Balancing

The common theme of this dissertation is sampling-based motion planning with the two key contributions being in the area of replanning and spatial load balancing for robotic systems. Here, we begin by recalling two sampling-based motion planners: the asymptotically optimal rapidly-exploring random tree (RRT*), and the asymptotically optimal probabilistic roadmap (PRM*). We also provide a brief background on collision cones and the Distributed Reactive Collision Avoidance (DRCA) algorithm. The next four chapters detail novel contributions for motion replanning in environments with unexpected static obstacles, for multi-agent collision avoidance, and spatial load balancing. First, we show improved performance of the RRT* when using the proposed Grandparent-Connection (GP) or Focused-Refinement (FR) algorithms. Next, the Goal Tree algorithm for replanning with unexpected static obstacles is detailed and proven to be asymptotically optimal. A multi-agent collision avoidance problem in obstacle environments is approached via the RRT*, leading to the novel Sampling-Based Collision Avoidance (SBCA) algorithm. The SBCA algorithm is proven to guarantee collision free trajectories for all of the agents, even when subject to uncertainties in the knowledge of the other agents’ positions and velocities. Given that a solution exists, we prove that livelocks and deadlock will lead to the cost to the goal being decreased. We introduce a new deconfliction maneuver that decreases the cost-to-come at each step. This new maneuver removes the possibility of livelocks and allows a result to be formed that proves convergence to the goal configurations. Finally, we present a limited range Graph-based Spatial Load Balancing (GSLB) algorithm which fairly divides a non-convex space among multiple agents that are subject to differential constraints and have a limited travel distance. The GSLB is proven to converge to a solution when maximizing the area covered by the agents. The analysis for each of the above mentioned algorithms is confirmed in simulations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

THE ALFALFA H α SURVEY. I. PROJECT DESCRIPTION AND THE LOCAL STAR FORMATION RATE DENSITY FROM THE FALL SAMPLE

The ALFALFA H α survey utilizes a large sample of H i-selected galaxies from the ALFALFA survey to study star formation (SF) in the local universe. ALFALFA H α contains 1555 galaxies with distances between ∼20 and ∼100 Mpc. We have obtained continuum-subtracted narrowband H α images and broadband R images for each galaxy, creating one of the largest homogeneous sets of H α images ever assembled. Our procedures were designed to minimize the uncertainties related to the calculation of the local SF rate density (SFRD). The galaxy sample we constructed is as close to volume-limited as possible, is a robust statistical sample, and spans a wide range of galaxy environments. In this paper, we discuss the properties of our Fall sample of 565 galaxies, our procedure for deriving individual galaxy SF rates, and our method for calculating the local SFRD. We present a preliminary value of log(SFRD[ M {sub ⊙} yr{sup −1} Mpc{sup −3}]) = −1.747 ± 0.018 (random) ±0.05 (systematic) based on the 565 galaxies in our Fall sub-sample. Compared to the weighted average of SFRD values around z ≈ 2, our local value indicates a drop in the global SFRD of a factor of 10.2 over that lookback time.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

Maternal body burden of cadmium and offspring size at birth

Increasing evidence suggests an inverse association between cadmium (Cd) and size at birth, potentially greatest among female neonates. We evaluated whether greater maternal body burden of Cd is associated with reduced neonatal anthropometry (birthweight, birth length, head circumference, and ponderal index) and assessed whether these associations differ by infant sex. The analytic sample for the present study (n=396) was derived from a subcohort of 750 women randomly drawn from among all participants (N=4344) in the Omega Study, a prospective pregnancy cohort. Creatinine-corrected Cd in maternal clean-catch spot urine samples (U-Cd) was quantified by inductively coupled plasma mass spectrometry. Continuous log{sub 2}-transformed Cd (log{sub 2}-Cd) and U-Cd tertiles (low<0.29 μg/g creatinine, middle 0.29–0.42 μg/g creatinine, high≥0.43 μg/g creatinine) were used in multivariable linear regression models. Females had reduced birth length with greater U-Cd tertile, whereas males birth length marginally increased [β(95% CI) females: low=reference, middle=−0.59 cm (−1.37, 0.19), high=−0.83 cm (−1.69, 0.02), p-trend=0.08; males: low=reference, middle=0.18 cm (−0.59, 0.95), high=0.78 cm (–0.04, 1.60), p-trend=0.07; p for interaction=0.03]. The log{sub 2}-Cd by infant sex interaction was statistically significant for ponderal index [p=0.003; β(95% CI): female=0.25 kg/m{sup 3} (−0.20, 0.70); male=−0.63 kg/m{sup 3} (−1.01, −0.24)] and birth length [p<0.001; β(95% CI): female=−0.47 cm (−0.74, −0.20), male=0.32 cm (0.00, 0.65)]. Our findings suggest potential sex-specific reversal of Cd’s associations on birth length and contribute to the evidence suggesting Cd impairs fetal growth. - Highlights: • Cadmium levels in the general population potentially adversely affect size at birth. • Maternal urinary cadmium was inversely related to birth length among female infants. • For male infants, maternal cadmium was positively associated with birth length. • Maternal cadmium was also inversely related to ponderal index among male infants.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Leverage Score Sampling for Parametric PDEs (Final Technical Report)

This final technical report summarizes the accomplishments of work performed under DOE Office of Science Award DE-SC0022266, which is titled “Leverage Score Sampling for Parametric PDEs”. The goal of the project was to extend methods from Randomized Numerical Linear Algebra (RandNLA) to tackle central computational challenges in model order reduction and uncertainty quantification (UQ) for parametric partial differential equations (PDEs). In particular, we sought to use importance sampling methods originally developed for RandNLA to develop sample efficient active learning algorithms for approximating high-dimensional scalar functions, e.g. by polynomials, Gaussian process models, and simple neural networks. Such methods can be immediately applied to developing surrogate models or to approximating quantity of interest (QoI) surfaces. In the context of PDEs, each sample used for learning equates to the solution of the differential equation for a particular set of parameters, so sample efficiency translates to improved computational efficiency for a variety of downstream tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Examining AGN UV/Optical Variability beyond the Simple Damped Random Walk

We present damped harmonic oscillator (DHO) light-curve modeling for a sample of 12,714 spectroscopically confirmed quasars in the Sloan Digital Sky Survey Stripe 82 region. DHO is a second-order continuous-time autoregressive moving-average process, which can be fully described using four independent parameters: a natural oscillation frequency (ω 0 ), a damping ratio (ξ), a characteristic perturbation timescale (τ perturb ), and an amplitude for the perturbing white noise (σ ϵ ). The asymptotic variability amplitude of a DHO process is quantified by σ DHO —a function of ω 0 , ξ, τ perturb , and σ ϵ . We find that both τ perturb and σ ϵ follow different dependencies with rest-frame wavelength (λ RF ) on either side of 2500 Å, whereas σ DHO follows a single power-law relation with λ RF . After correcting for wavelength dependence, σ DHO exhibits anticorrelations with both the Eddington ratio and the black hole mass, while τ perturb —with a typical value of days in the rest frame—shows an anticorrelation with the bolometric luminosity. Modeling active galactic nuclei (AGN) variability as a DHO offers more insight into the workings of accretion disks close to the supermassive black holes at the center of AGN. The newly discovered short-term variability (characterized by τ perturb and σ ϵ ) and its correlation with bolometric luminosity pave the way for new algorithms that will derive fundamental properties (e.g., Eddington ratio) of AGN using photometric data alone.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

A Metabolomics Assay to Diagnose Citrus Huanglongbing Disease and to Aid in Assessment of Treatments to Prevent or Cure Infection

Citrus greening disease, or Huanglongbing (HLB), has devastated citrus crops globally in recent years. The causal bacterium, ‘ Candidatus Liberibacter asiaticus’, presents a sampling issue for qPCR diagnostics and results in a high false negative rate. In this work, we compared six metabolomics assays to identify HLB-infected citrus trees from leaf tissue extracted from 30 control and 30 HLB-infected trees. A liquid chromatography-mass spectrometry-based assay was most accurate. A final partial least squares-discriminant analysis (PLS-DA) model was trained and validated on 690 leaf samples with corresponding qPCR measures from three citrus varieties (Rio Red grapefruit, Hamlin sweet orange, and Valencia sweet orange) from orchards in Florida and Texas. Trees were naturally infected with HLB transmitted by the insect vector Diaphorina citri. In a randomized validation set, the assay was 99.9% accurate to classify diseased from nondiseased samples. This model was applied to samples from trees receiving plant defense-inducer compounds or biological treatments to prevent or cure HLB infection. From two trials, HLB-related metabolite abundances and PLS-DA scores were tracked longitudinally and compared with those of control trees. We demonstrate how our assay can assess tree health and the efficacy of HLB treatments and conclude that no trialed treatment was efficacious.

Plant Sciences↗
DOE OSTI2022

Physics Informed Reinforcement Learning for Power Grid Control using Augmented Random Search

Wide adoption of deep reinforcement learning need to overcome several challenges in energy system domain, including scalability, learning from limited samples, and high-dimensional continuous state and action spaces. In this paper, we integrated physics-based information from the normal generator operation state formula in the reinforcement learning agent's neural network loss function, and applied an augmented random search agent to optimize the generator control under dynamic contingency. Simulation results demonstrated the reliability performance improvements in training speed, reward convergence, sampling efficiency, scalability, and transferability.

physics informed ML, Physics Informed Neural Netwo↗
DOE OSTI2026

Quantum-Inspired Bayesian Sampling for Uncertainty Quantification and Machine Learning (Final Technical Report)

With increasing simulation and measurement data, machine learning and artificial intelligence have been widely used in computational decision-making of complex engineering systems. The resulting tools, such as uncertainty quantification solvers, reinforcement learning, and physics-informed machine learning, have achieved great success in critical DOE tasks such as material discovery and design, energy system modeling and control, and numerical weather and climate prediction. A core topic in scientific machine learning and artificial intelligence is Bayesian inference: given an observed data set, people want to estimate the posterior distribution of a (possibly large) number of hidden parameters. Due to the flexibility and weak assumptions, Bayesian sampling has been the mainstream Bayesian inference solvers despite the rapid progress of approximate Bayesian inference. Classical Bayesian sampling methods such as Markov-chain Monte Carlo suffer from a low-acceptance rate due to the random walk nature, therefore state-of-the-art techniques use Hamiltonian Monte Carlo and its variants to efficiently draw posterior samples in a high dimension. The key idea of Hamiltonian Monte Carlo and its variants is to simulate the Hamiltonian dynamics of a classical particle with a fixed mass, and their performance significantly degrades when the posterior distribution is highly spiky or has multiple modes. Leveraging the idea of quantum physics, this project has investigated new theory, algorithms and applications of Bayesian inference (especially Bayesian sampling). The main results include: (1) novel quantum-inspired Bayesian sampling methods that can lead to better accuracy for challenging multi-modal or spiky distributions, (2) more scalable machine learning framework leveraging tensor-compressed Bayesian inference, and (3) Bayesian and sampling approaches for verifying the robustness of continuous and binary neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Convergent behavior of extended stalk regions from staphylococcal surface proteins with widely divergent sequence patterns

Staphylococcus epidermidis and Staphylococcus aureus are highly problematic bacteria in hospital settings. A major challenge is their ability to form biofilms on abiotic or biotic surfaces. Biofilms are well-organized, multicellular bacterial aggregates that resist antibiotic treatment and often lead to recurrent infections. Bacterial cell wall-anchored (CWA) proteins are important players in biofilm formation and infection. Many have putative stalk-like regions or regions of low complexity near the cell wall-anchoring motif. Recent work demonstrated the strong propensity of the stalk region of S. epidermidis accumulation-associated protein (Aap) to remain highly extended under solution conditions that typically induce compaction. This behavior is consistent with the expected function of a stalk-like region that is covalently attached to the cell wall peptidoglycan and projects the adhesive domains of Aap away from the cell surface. In this study, we evaluate whether the ability to resist compaction is a common theme among stalk regions from various staphylococcal CWA proteins. Circular dichroism spectroscopy was used to examine secondary structure changes as a function of temperature and cosolvents along with sedimentation velocity analytical ultracentrifugation, size-exclusion chromatography, and SAXS to characterize structural characteristics in solution. All stalk regions tested are intrinsically disordered, lacking secondary structure beyond random coil and polyproline type II helix, and they all sample highly extended conformations. Remarkably, the Ser-Asp dipeptide repeat region of SdrC exhibited nearly identical behavior in solution when compared to the Aap Pro/Gly-rich region, despite highly divergent sequence patterns, indicating conservation of function by various distinct staphylococcal CWA protein stalk regions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Codiscovering graphical structure and functional relationships within data: A Gaussian Process framework for connecting the dots

Most problems within and beyond the scientific domain can be framed into one of the following three levels of complexity of function approximation. Type 1: Approximate an unknown function given input/output data. Type 2: Consider a collection of variables and functions, some of which are unknown, indexed by the nodes and hyperedges of a hypergraph (a generalized graph where edges can connect more than two vertices). Given partial observations of the variables of the hypergraph (satisfying the functional dependencies imposed by its structure), approximate all the unobserved variables and unknown functions. Type 3: Expanding on Type 2, if the hypergraph structure itself is unknown, use partial observations of the variables of the hypergraph to discover its structure and approximate its unknown functions. These hypergraphs offer a natural platform for organizing, communicating, and processing computational knowledge. While most scientific problems can be framed as the data-driven discovery of unknown functions in a computational hypergraph whose structure is known (Type 2), many require the data-driven discovery of the structure (connectivity) of the hypergraph itself (Type 3). We introduce an interpretable Gaussian Process (GP) framework for such (Type 3) problems that does not require randomization of the data, access to or control over its sampling, or sparsity of the unknown functions in a known or learned basis. Its polynomial complexity, which contrasts sharply with the super-exponential complexity of causal inference methods, is enabled by the nonlinear ANOVA capabilities of GPs used as a sensing mechanism.

Science & Technology - Other Topics↗
DOE OSTI2022

SDSS-IV MaNGA: spatial resolved properties of kinematically misaligned galaxies

We select 456 galaxies with kinematically misaligned gas and stellar components from 9456 parent galaxies in MaNGA, and classify them into 72 star-forming galaxies, 142 green-valley galaxies, and 242 quiescent galaxies. Comparing the spatial resolved properties of the misaligned galaxies with control samples closely match in the Dn4000 and stellar velocity dispersion, we find that: (1) the misaligned galaxies have lower values in V gas /σ gas and V star /σ star (the ratio between ordered to random motion of gas and stellar components) across the entire galaxies than their control samples; (2) the star-forming and green-valley misaligned galaxies have enhanced central concentrated star formation than their control galaxies. The difference in stellar population between quiescent misaligned galaxies and control samples is small; (3) gas-phase metallicity of the green valley and quiescent misaligned galaxies are lower than the control samples. For the star-forming misaligned galaxies, the difference in metallicity between the misaligned galaxies and their control samples strongly depends on how we select the control samples. All these observational results suggest that external gas accretion influences the evolution of star forming and green-valley galaxies, not only in kinematics/morphologies, but also in stellar populations. Furthermore, the quiescent misaligned galaxies have survived from different formation mechanisms.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2016

Machine- z : Rapid machine-learned redshift indicator for Swift gamma-ray bursts

Studies of high-redshift gamma-ray bursts (GRBs) provide important information about the early Universe such as the rates of stellar collapsars and mergers, the metallicity content, constraints on the re-ionization period, and probes of the Hubble expansion. Rapid selection of high-z candidates from GRB samples reported in real time by dedicated space missions such as Swift is the key to identifying the most distant bursts before the optical afterglow becomes too dim to warrant a good spectrum. Here, we introduce ‘machine-z’, a redshift prediction algorithm and a ‘high-z’ classifier for Swift GRBs based on machine learning. Our method relies exclusively on canonical data commonly available within the first few hours after the GRB trigger. Using a sample of 284 bursts with measured redshifts, we trained a randomized ensemble of decision trees (random forest) to perform both regression and classification. Cross-validated performance studies show that the correlation coefficient between machine-z predictions and the true redshift is nearly 0.6. At the same time, our high-z classifier can achieve 80 per cent recall of true high-redshift bursts, while incurring a false positive rate of 20 per cent. With 40 per cent false positive rate the classifier can achieve ~100 per cent recall. As a result, the most reliable selection of high-redshift GRBs is obtained by combining predictions from both the high-z classifier and the machine-z regressor.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Large Scale Caching and Streaming of Training Data for Online Deep Learning

The training of deep neural network models on large data remains a difficult problem, despite progress towards scalable techniques. In particular, there is a mismatch between the random but predetermined order in which AI flows select training samples and the streaming I/O patterns for which traditional HPC data storage (e.g., parallel file systems) are designed. In addition, as more data are obtained, it is feasible neither simply to train learning models incrementally, due to catastrophic forgetting (i.e., bias towards new samples), nor to train frequently from scratch, due to prohibitive time and/or resource constraints. In this paper, we study data management techniques that combine caching and streaming with rehearsal support in order to enable efficient access to training samples in both offline training and continual learning. We revisit state-of-art streaming approaches based on data pipelines that transparently handle prefetching, caching, shuffling, and data augmentation, and discuss the challenges and opportunities that arise when combining these methods with data-parallel training techniques. We also report on preliminary experiments that evaluate the I/O overheads involved in accessing the training samples from a parallel file system (PFS) under several concurrency scenarios, highlighting the impact of the PFS on the design of the data pipelines.

data pipelines↗