Search NASA⌕ Search

SEARCH · Search NASA

Results for “Distributed training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,279 records · Page 71

A Quantitative Study of Deep Learning Training on Heterogeneous Supercomputers

Deep learning (DL) has become a key technique for solving complex problems in scientific research and discovery. DL training for science is substantially challenging because it has to deal with massive quantities of multi-dimensional data. High- performance computing (HPC) supercomputers are increasingly being employed for meeting the exponentially growing demand for DL. Multiple GPUs and high-speed interconnect network are needed for supporting DL on HPC systems. However, the excessive use of GPUs without considering effective benefits leads to inefficient resource utilization of these expensive setups. In this paper, we conduct a quantitative analysis to gauge the efficacy of DL workloads on the latest HPC system and identify viability of next-generation DL-optimized heterogeneous supercomputers for enabling researchers to develop more efficient resource management and distributed DL middleware. We evaluate well- known DL models with large-scale datasets using the popular TensorFlow framework, and provide a thorough evaluation in- cluding scalability, accuracy, variability, storage resource, GPU- GPU/GPU-CPU data transfer, and GPU utilization. Our analysis reveals that the latest heterogeneous supercomputing cluster shows varying performance trend as compared to the existing literature for single- and multi-node training. To the best of our knowledge, this is the first work to conduct such a quantitative and comprehensive study of DL training on a supercomputing system with multiple GPUs.

Han, Jingoo↗

Accelerating Hamiltonian Monte Carlo for Bayesian inference in neural networks and neural operators

Hamiltonian Monte Carlo (HMC) is a powerful and accurate method to sample from the posterior distribution in Bayesian inference. However, HMC techniques are computationally demanding for Bayesian neural networks due to the high dimensionality of the network’s parameter space and the non-convexity of their posterior distributions. Therefore, various approximation techniques, such as variational inference (VI) or stochastic gradient MCMC, are often employed to infer the posterior distribution of the network parameters. Such approximations introduce inaccuracies in the inferred distributions, resulting in unreliable uncertainty estimates. In this work, we propose a hybrid approach that combines inexpensive VI and accurate HMC methods to efficiently and accurately quantify uncertainties in neural networks and neural operators. The proposed approach leverages an initial VI training on the full network. We examine the influence of individual parameters on the prediction uncertainty, which shows that a large proportion of the parameters do not contribute substantially to uncertainty in the network predictions. This information is then used to significantly reduce the dimension of the parameter space, and HMC is performed only for the subset of network parameters that strongly influence prediction uncertainties. This yields a framework for accelerating the full batch HMC for posterior inference in neural networks. We demonstrate the efficiency and accuracy of the proposed framework on deep neural networks and operator networks, showing that inference can be performed for large networks with tens to hundreds of thousands of parameters. Finally, we show that this method can effectively learn surrogates for complex physical systems by modeling the operator that maps from upstream conditions to wall-pressure data on a cone in hypersonic flow.

Bayesian inference↗

Machine learning for interpreting coherent X-ray speckle patterns

Speckle patterns produced by coherent X-ray have a close relationship with the internal structure of materials but quantitative inversion of the relationship to determine structure from speckle patterns is challenging. Here, in this work, we investigate the link between coherent X-ray speckle patterns and sample structures using a model 2D disk system and explore the ability of machine learning to learn aspects of the relationship. Specifically, we train a deep neural network to classify the coherent X-ray speckle patterns according to the disk number density in the corresponding structure. It is demonstrated that the classification system is accurate for both non-disperse and disperse size distributions.

36 MATERIALS SCIENCE↗

Z-Contrast Enhancement in Au–Pt Nanocatalysts by Correlative X-ray Absorption Spectroscopy and Electron Microscopy: Implications for Composition Determination

The properties of bimetallic nanoparticles (BNPs) vary widely as a function of their composition and size distributions. X-ray absorption fine structure analysis is commonly used to characterize their structure, but its application to elements that are close to each other in the periodic table is hampered by poor Z-contrast. For this study, we trained an artificial neural network to recognize the partial coordination numbers in AuPt NPs synthesized via peptide templating using their X-ray absorption near-edge structure spectra. This approach, combined with scanning transmission electron microscopy analysis, revealed unique details of this prototype catalytic system that has different forms of heterogeneities at both the intra- and inter-particle levels. Our method based on the enhancement of Z-contrast of metal species will have implications for compositional studies of BNPs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Acoustic Emission Source Location in Unidirectional Carbon-Fibre-Reinforced Plastic Plates Using Virtually Trained Artificial Neural Networks

Acoustic emission source location in a unidirectional carbon-fibre-reinforced plastic plate was attempted employing Artificial Neural Network (ANN) technology. The acoustic emission events were produced by a lead break, and the response wave received by piezoelectric sensors, type VS150-M resonant at 150 kHz. The waves were detected by a Vallen AMSY4 eight-channel instrumentation. The time of arrival, determined through the conventional threshold crossing technique, was used to measure the dependence of wave velocity on fibre orientation. A simple empirical formula, relying on classical lamination and suggested by wave propagation theory, was able to accurately model the experimental trend. Based on the formula, virtual training and testing data sets were generated for the case of a plate monitored by three transducers, and adopted to select two potentially effective ANN architectures. For final validation, experimental tests were carried out, positioning the source at predetermined points evenly distributed within the plate area. A very satisfactory correlation was found between the actual source locations and the ANN predictions.

36 MATERIALS SCIENCE↗

Resonant two- or three-photon ionization of noble-gas atoms captured by time-resolved photoelectron momentum spectroscopy

In this work, we present an all-electron, ab initio calculation of time-resolved photoelectron momentum distributions (PMDs) induced by resonant two-color, two- or three-photon ionization of helium and neon atoms, as recently measured by Villeneuve et al. [Science 356, 1150 (2017)] using a femtosecond infrared (IR) laser pulse and an attosecond pulse train (APT) produced with high harmonic generation as an extreme ultraviolet (XUV) photon source. In contrast to a monochromatic XUV pulse commonly used at the free-electron laser facilities, an APT is broadband and ultrashort (<1 fs), so that it could induce nonresonant one-photon transitions above ionization threshold as well as a resonantly excited state. They may interfere with each other upon subsequent absorption or emission of an IR photon, so that the resultant PMD loses clear orbital symmetry. Our time-dependent density functional calculation demonstrates that the selection of a particular excited state in the time-resolved PMD is nevertheless possible if energy and timing of an APT are adjusted properly.

74 ATOMIC AND MOLECULAR PHYSICS↗

Hybrid Entanglement Distribution between Remote Microwave Quantum Computers Empowered by Machine Learning

Superconducting microwave circuits with Josephson junctions, the major platform for quantum computing, can only reach the full capability when connected. This requires an efficient protocol to distribute microwave entanglement. While quantum computers typically use discrete-variable (DV) methods for information encoding, the entire continuous-variable (CV) degree of freedom in electromagnetic fields must be utilized to achieve the highest entanglement distribution rate. Here, we propose a hybrid protocol to resolve the incompatibility between DV microwave quantum computers and CV quantum communications. CV microwave entanglement is distributed using optical swapping of optical-microwave entanglement pairs. To interface with DV microwave quantum computers, we further design a hybrid circuit to simultaneously convert and distill high-quality DV entanglement from noisy CV entanglement. The hybrid circuit is trained with machine-learning algorithms, ensuring high entanglement fidelity and generation rate. In conclusion, our work not only provides a practical method to realize efficient quantum links for superconducting microwave quantum computers, but also opens avenues to bridge the gap between DV and CV quantum systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Electrical Load Forecasting Over Multihop Smart Metering Networks With Federated Learning

Electric load forecasting is essential for power management and stability in smart grids. This is mainly achieved via advanced metering infrastructure, where smart meters (SMs) record household energy data. Traditional machine learning (ML) methods are often employed for load forecasting, but require data sharing, which raises data privacy concerns. Federated learning (FL) can address this issue by running distributed ML models at local SMs without data exchange. However, current FL-based approaches struggle to achieve efficient load forecasting due to imbalanced data distribution across heterogeneous SMs. Here, this article presents a novel personalized FL (PFL) method for high-quality load forecasting in metering networks. A meta-learning-based strategy is developed to address data heterogeneity at local SMs in the collaborative training of local load forecasting models. Moreover, to minimize the load forecasting delays in our PFL model, we study a new latency optimization problem based on optimal resource allocation at SMs. A theoretical convergence analysis is also conducted to provide insights into FL design for federated load forecasting. Extensive simulations from real-world datasets show that our method outperforms existing approaches regarding better load forecasting and reduced operational latency costs.

Rahman, Ratun [Univ. of Alabama, Huntsville, AL (U↗

Diverse Super-Resolution (diversity_SR) [SWR-21-60]

Deep learning tools for enhancing the spatial resolution of wind data. The software is developed in Python using the TensorFlow deep learning package. Models for diversity super-resolution is provided. Included in the package are pretrained models with example code/data to perform the super-resolution as well as tools of training models for different enhancement- or data-types. The super-resolution is an inherently ill-conditioned problem, with multiple high-resolution fields plausibly mapping to the same coarse field. Considitional GANs provide a framework for generating a distribution of high-resolution realizations from a given low-resolution input. Stochastic estimation is used to inform the network of the expected degree and location of sub-grid diversity. The package includes a pretrained network to generate distributions of 10x-enhanced fields of wind data.

Glaws, Andrew↗

A Roadmap for NEAMS Capability Transfer

The vision of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program is to bring truly predictive modeling and simulation (M&S) capabilities to the nuclear engineering community in order to enable a new approach to the design and analysis of nuclear energy systems. From its inception, the NEAMS program has always envisioned a broad user base for its software and scientific products, including researchers within the DOE complex, nuclear industry technology developers and vendors, and operators. However activities to date have focused almost exclusively on interactions with NEAMS sponsors, who are also near-term users of NEAMS technologies. The task of the NEAMS Capability Transfer (CT) program element for FY2011 is to develop a comprehensive plan to support the program's needs for user outreach and technology transfer. In order to obtain community input to this plan, a 'NEAMS Capability Transfer Roadmapping Workshop' was held 4-5 April 2011 in Chattanooga, TN, and is summarized in this report. The 30 workshop participants represented the NEAMS program, the DOE and industrial user communities, and several outside programs. The workshop included a series of presentations providing an overview of the NEAMS program and presentations on the user outreach and technology transfer experiences of (1) The Advanced Simulation and Computing (ASC) program, (2) The Standardized Computer Analysis for Licensing Evaluation (SCALE) project, and (3) The Consortium for Advanced Simulation of Light Water Reactors (CASL), followed by discussion sessions. Based on the workshop and other discussions throughout the year, we make a number of recommendations of key areas for the NEAMS program to develop the user outreach and technology transfer activities: (1) Engage not only DOE, but also industrial users sooner and more often; (2) Engage with the Nuclear Regulatory Commission to facilitate their understanding and acceptance of NEAMS approach to predictive M&S; (3) Place requirements gathering from prospective users on a more formal footing, updating requirements on a regular basis and incorporate them into planning and execution of the project in a traceable fashion; (4) Seek out the best available data for validation purposes, and work with experimental programs to design and carry out new experiments that satisfy the need for data suitable for validation of high-fidelity M&S codes; (5) Develop and implement program-wide plans and policies for export control, licensing, and distribution of NEAMS software products; (6) Establish a program of sponsored alpha testing by experienced users in order to obtain feedback on NEAMS codes; (7) Provide technical support for NEAMS software products; (8) Develop and deliver documentation, tutorial materials, and live training classes; and (9) Be prepared to support outside users who wish to contribute to the codes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Hybrid Storage Solution

With the rise of artificial intelligence and machine learning, data sets used to train models have become increasingly large. The availability, accessibility and integrity of large data sets has become important to the research conducted at Los Alamos National Laboratory. Ceph is a storage solution suitable for use with critical data because of its distributed nature and ability to keep multiple copies of a file in different locations. The amount of data means that bandwidth, latency, and cost are important factors and the reason most storage solutions are on-premises. However, there are distinct advantages to hosting services in the cloud, namely scalability and ease-of-use. In this paper, we explore the possibility of provisioning a hybrid Ceph cluster that leverages the benefits of both cloud architectures and on-premise performance.

97 MATHEMATICS AND COMPUTING↗

Denver TMA assessment

This report describes the assessment of the Traffic Management Advisor (TMA) conducted at the Denver Center and TRACON, January 11-February 5, 1993. The assessment addressed the effectiveness of TMA for supporting various traffic management activities. At the Center, traffic management coordinators (TMC's) shadowed traffic operations, using TMA to make metering time and internal release decisions. At the TRACON, TMC's accessed TMA in an advisory mode for different planning activities such as staffing, distributing the traffic load, and changing the airport acceptance rate. These different opportunities for assessing TMA provide insight into TMA as a potential communication aid and planning tool. Findings from this assessment indicate that TMA can be used to support traffic management planning and decisions. TMC's at the center were able to use information provided by TMA to determine metering times as well as internal release times. At the TRACON, TMA supported decisions on airport configuration, airport acceptance rate, load distribution, proactive coordination with the center, and staffing. While findings of TMA use are generally positive, it must be kept in mind that this assessment is a snapshot in time. Not all TMA capabilities were assessed, nor were TMC's fully trained and knowledgeable on all features. Feature use will continue to evolve and strategies emerge as the TMC's gain experience with TMA over a variety of traffic situations and their understanding of TMA capabilities broadens with continued training.

Harwood, Kelly↗

Experimental and computational investigation of lift-enhancing tabs on a multi-element airfoil

An experimental and computational investigation of the effect of lift enhancing tabs on a two-element airfoil was conducted. The objective of the study was to develop an understanding of the flow physics associated with lift enhancing tabs on a multi-element airfoil. A NACA 63(sub 2)-215 ModB airfoil with a 30 percent chord Fowler flap was tested in the NASA Ames 7 by 10 foot wind tunnel. Lift enhancing tabs of various heights were tested on both the main element and the flap for a variety of flap riggings. Computations of the flow over the two-element airfoil were performed using the two-dimensional incompressible Navier-Stokes code INS2D-UP. The computer results predict all of the trends in the experimental data quite well. When the flow over the flap upper surface is attached, tabs mounted at the main element trailing edge (cove tabs) produce very little change in lift. At high flap deflections. however, the flow over the flap is separated and cove tabs produce large increases in lift and corresponding reductions in drag by eliminating the separated flow. Cove tabs permit high flap deflection angles to be achieved and reduce the sensitivity of the airfoil lift to the size of the flap gap. Tabs attached to the flap training edge (flap tabs) are effective at increasing lift without significantly increasing drag. A combination of a cove tab and a flap tab increased the airfoil lift coefficient by 11 percent relative to the highest lift tab coefficient achieved by any baseline configuration at an angle of attack of zero percent and the maximum lift coefficient was increased by more than 3 percent. A simple analytic model based on potential flow was developed to provide a more detailed understanding of how lift enhancing tabs work. The tabs were modeled by a point vortex at the training edge. Sensitivity relationships were derived which provide a mathematical basis for explaining the effects of lift enhancing tabs on a multi-element airfoil. Results of the modeling effort indicate that the dominant effects of the tabs on the pressure distribution of each element of the airfoil can be captured with a potential flow model for cases with no flow separation.

Ashby, Dale↗

Autonomous Mission Operations Roadmap

As light time delays increase, the number of such situations in which crew autonomy is the best way to conduct the mission is expected to increase. However, there are significant open questions regarding which functions to allocate to ground and crew as the time delays increase. In situations where the ideal solution is to allocate responsibility to the crew and the vehicle, a second question arises: should the activity be the responsibility of the crew or an automated vehicle function? More specifically, we must answer the following questions: What aspects of mission operation responsibilities (Plan, Train, Fly) should be allocated to ground based or vehicle based planning, monitoring, and control in the presence of significant light-time delay between the vehicle and the Earth?How should the allocated ground based planning, monitoring, and control be distributed across the flight control team and ground system automation? How should the allocated vehicle based planning, monitoring, and control be distributed between the flight crew and onboard system automation?When during the mission should responsibility shift from flight control team to crew or from crew to vehicle, and what should the process of shifting responsibility be as the mission progresses? NASA is developing a roadmap of capabilities for Autonomous Mission Operations for human spaceflight. This presentation will describe the current state of development of this roadmap, with specific attention to in-space inspection tasks that crews might perform with minimum assistance from the ground.

Mission Operations↗

A Decade of NASA Strategic Astrophysics Technology Investments: Technology Maturation, Infusion, and Other Benefits

NASA’s Astrophysics Division (APD) funds development of cutting-edge technology to enable its missions to achieve ambitious and groundbreaking science goals. These technology development efforts are managed by the Physics of the Cosmos (PCOS), Cosmic Origins (COR), and Exoplanet Exploration (ExE) Programs. The NASA Strategic Astrophysics Technology (SAT) Program was established in 2009 as a new technology maturation program to fill the gap in the Technology Readiness Level (TRL) range from 3 to 6. Since program inception, 100 SAT grants have been openly competed and awarded, along with dozens of direct-funded projects, leading to a host of technologies advancing their TRLs and/or being infused into space and suborbital missions and ground-based projects. We present the portfolio distribution in terms of specific technology areas addressed, including optics, detectors, coatings, coronagraphs, starshades, lasers, electronics, cooling systems, and micro-thruster subsystems. We show an analysis of the rate of TRL advances, infusion success stories, and other benefits such as training the future astrophysics workforce, including students and postdoctoral fellows hired by projects. Finally, we present APD’s current strategic technology maturation priorities for investment, enabling a range of future strategic astrophysics missions

NASA, astrophysics, technology development, optics↗

PPDIST, global 0.1° daily and 3-hourly precipitation probability distribution climatologies for 1979–2018

We introduce the Precipitation Probability DISTribution (PPDIST) dataset, a collection of global high-resolution (0.1°) observation-based climatologies (1979–2018) of the occurrence and peak intensity of precipitation (P) at daily and 3-hourly time-scales. The climatologies were produced using neural networks trained with daily P observations from 93,138 gauges and hourly P observations (resampled to 3-hourly) from 11,881 gauges worldwide. Mean validation coefficient of determination (R^(2)) values ranged from 0.76 to 0.80 for the daily P occurrence indices, and from 0.44 to 0.84 for the daily peak P intensity indices. The neural networks performed significantly better than current state-of-the-art reanalysis (ERA5) and satellite (IMERG) products for all P indices. Using a 0.1 mm 3 per h threshold, P was estimated to occur 12.2%, 7.4%, and 14.3% of the time, on average, over the global, land, and ocean domains, respectively. The highest P intensities were found over parts of Central America, India, and Southeast Asia, along the western equatorial coast of Africa, and in the intertropical convergence zone.

Hylke E. Beck↗

Fine particulate concentrations over East Asia derived from aerosols measured by the Advanced Himawari Imager using machine learning

Fine particulate matter with a diameter below 2.5 μm (PM 2.5 ) is deleterious to the cardiovascular and respiratory systems. It is often difficult to assess the effects of PM 2.5 on human health over regions with limited ground monitoring sites, especially in East Asia. As an alternative, we estimated near-surface PM 2.5 concentrations by analyzing Advanced Himawari Imager (AHI) Yonsei Aerosol Retrieval (YAER) products. This study incorporates daytime data for East Asia covering the Korean Peninsula, China, Japan, Southeast Asia, and southern Mongolia. We collocated AHI YAER product pixels with meteorological, land-cover, and other ancillary data for the period from March 2018 to February 2019. To estimate PM 2.5 concentrations over wide areas spanning many countries displaying various relationships between aerosol optical depth and PM 2.5 , monthly models were developed by considering both the spatial and temporal characteristics of ground-based PM 2.5 measurements. Random forest machine learning model estimated ground-level mass concentrations of PM 2.5 ; subsequent 10-fold cross validation (CV) yielded a CV R 2 value of 0.81 and a CV root mean squared error (RMSE) of 12.3 μg m -3 . We investigated the spatial pattern of PM 2.5 concentrations over multiple countries and seasonal variation in PM 2.5 concentrations. Diurnal variation of a severe PM 2.5 event in the Korean Peninsula was investigated as a case study. The model captured the extremely heterogeneous spatial distribution of PM 2.5 concentrations peaked around local noon. To measure the capability of the developed model to estimate PM 2.5 concentrations in areas with few in-situ data, its predictive performance was evaluated using a dataset independent of the training process with an R 2 of 0.60 and RMSE of 8.18 μg m −3 . This study demonstrates the potential for satellite-based PM 2.5 estimation for areas with insufficient measuring stations.

Pm2.5↗

From Simulation to Reality With Random Noise

The challenging environment of autonomous vehicle (AV) navigation necessitates certain functions be performed by deep neural networks. Optimizing these models involves collecting vast quantities of domain-specific training data and ensuring that the dataset is representative of expected conditions. High-fidelity simulation plays a vital role in making this process feasible, allowing a wide range of scenarios to be explored at low cost. However, learning from simulation introduces subtle biases into models, which can degrade real-world performance in unpredictable ways. This effect can be mitigated with learning schemes specialized to bridge distributional shifts (transfer learning). Given the complex nature of these methods, the underlying models, and their environments, meaningfully evaluating performance is notstraight forward. Many unrelated factors can effect an improvement in generalization accuracy, but a full ablation analysis is often difficult. To tease out signal from noise, it is necessary to understand how transfer learning performance is affected by noise itself. The goals of this paper are (i) to establish a domain randomization baseline for a simple classification transfer learning task and (ii) to validate the RRAV testbed as a platform for further research in sim-to-real learning. We generate imagery from a simulation of NASA Ames Research Center and train a small convolutional neural network (ConvNet) to classify position relative to a centerline. Further models are trained with different types of noise progressively added to the data. The models are deployed aboard the on-site test vehicle to test real-world performance. In our experiments, we find that such naive domain randomization raises sim-to-real accuracy from 64% to 79%, while training directly on real data yields an 89% accuracy ceiling. These results suggest that the isolated mechanism of domain randomization can significantly improve generalization.

simulation↗