Scalable uncertainty quantification for deep operator networks using randomized priors
Not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not provided.
FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.
An analytical model is described using linear programming for the optimum generation and distribution of energy demands among competing energy resources and different economic criteria. The model, which will be used as a general engineering tool in the analysis of the Deep Space Network ground facility, considers several essential decisions for better design and operation. The decisions sought for the particular energy application include: the optimum time to build an assembly of elements, inclusion of a storage medium of some type, and the size or capacity of the elements that will minimize the total life-cycle cost over a given number of years. The model, which is structured in multiple time divisions, employ the decomposition principle for large-size matrices, the branch-and-bound method in mixed-integer programming, and the revised simplex technique for efficient and economic computer use.
While machine learning has been making enormous strides in many technical areas, it is still massively underused in transmission electron microscopy. To address this, a convolutional neural network model was developed for reliable classification of crystal structures from small numbers of electron images and diffraction patterns with no preferred orientation. Diffraction data containing 571,340 individual crystals divided among seven families, 32 genera, and 230 space groups were used to train the network. Despite the highly imbalanced dataset, the network narrows down the space groups to the top two with over 70% confidence in the worst case and up to 95% in the common cases. As examples, we benchmarked against alloys to two-dimensional materials to cross-validate our deep-learning model against high-resolution transmission electron images and diffraction patterns. We present this result both as a research tool and deep-learning application for diffraction analysis.
Abstract The controlled introduction of elastic strains is an appealing strategy for modulating the physical properties of semiconductor materials. With the recent discovery of large elastic deformation in nanoscale specimens as diverse as silicon and diamond, employing this strategy to improve device performance necessitates first-principles computations of the fundamental electronic band structure and target figures-of-merit, through the design of an optimal straining pathway. Such simulations, however, call for approaches that combine deep learning algorithms and physics of deformation with band structure calculations to custom-design electronic and optical properties. Motivated by this challenge, we present here details of a machine learning framework involving convolutional neural networks to represent the topology and curvature of band structures in k -space. These calculations enable us to identify ways in which the physical properties can be altered through “deep” elastic strain engineering up to a large fraction of the ideal strain. Algorithms capable of active learning and informed by the underlying physics were presented here for predicting the bandgap and the band structure. By training a surrogate model with ab initio computational data, our method can identify the most efficient strain energy pathway to realize physical property changes. The power of this method is further demonstrated with results from the prediction of strain states that influence the effective electron mass. We illustrate the applications of the method with specific results for diamonds, although the general deep learning technique presented here is potentially useful for optimizing the physical properties of a wide variety of semiconductor materials.
The NASA Radiation Health Program has supported basic research over the last decade in radiation physics to develop ionizing radiation transport codes and corresponding data bases for the protection of astronauts from galactic and solar cosmic rays on future deep space missions. The codes describe the interactions of the incident radiations with shield materials where their content is modified by the atomic and nuclear reactions through which high energy heavy ions are fragmented into less massive reaction products and reaction products are produced as radiations as direct knockout of shield constituents or produced as de-excitation products in the reactions. This defines the radiation fields to which specific devices are subjected onboard a spacecraft. Similar reactions occur in the device itself which is the initiating event for the device response. An overview of the computational procedures and data base with some applications to photonic and data processing devices will be given.
Modern tokamaks have achieved significant fusion production, but further progress towards steady-state operation has been stymied by a host of kinetic and MHD instabilities. Control and identification of these instabilities is often complicated, warranting the application of data-driven methods to complement and improve physical understanding. In particular, Alfvén eigenmodes are a class of ubiquitous mixed kinetic and MHD instabilities that are important to identify and control because they can lead to loss of confinement and potential damage to the walls of a plasma device. In the present work, we use reservoir computing networks to classify Alfvén eigenmodes in a large labeled database of DIII-D discharges, covering a broad range of operational parameter space. However, despite the large parameter space, we show excellent classification and prediction performance, with an average hit rate of 91% and false alarm ratio of 7%, indicating promise for future implementation with additional diagnostic data and consolidation into a real-time control strategy.
Contaminated vadose zone materials are a potential source of long-term groundwater contamination at many sites across the Department of Energy (DOE) complex. Deep vadose zone contamination presents a particularly challenging remedial problem due to the difficultly of locating contaminants and the expense of access and ex situ treatment. In situ remediation techniques, whereby remedial amendments must be delivered to contaminated soils, have been identified as a potential alternative. However, amendment delivery is typically uncertain and post delivery remedial performance is often not well understood due to limited information available from sparsely spaced boreholes. Recent advancements in electrical resistivity tomography (ERT) are being used to address these challenges at the Hanford site by providing remote, three-dimensional images of contaminant distribution and four-dimensional (three spatial dimensions plus the time dimension) images of in-situ vadose zone remediation processes. These capabilities were recently demonstrated with a large scale surface characterization effort and during a smaller scale desiccation treatability test at the Hanford BC-Cribs area. The results of these efforts demonstrate the utility of leveraging high-performance computing resources to process ERT data for 3D reconnaissance mapping of subsurface contaminants and for detailed 3D monitoring of vadose zone remediation efforts. Two examples of ERT imaging at a former waste disposal facility have been demonstrated at different scales; a large scale reconnaissance survey spanning many hundreds of meters and a detailed monitoring application spanning a few tens of meters. Although ERT is applicable at many scales, resolution is generally governed by proximity to electrodes. For instance, the BCCT reconnaissance survey demonstrated herein lost resolution with depth (i.e. away from surface electrodes), and was unable to detect a high conductivity anomaly at approximately 75 m below ground surface in the cribs area; the deep anomalies were detected during highly focused borehole sampling. Conversely, the desiccation site characterization and monitoring demonstrated high resolution at a small scale, but only because of close proximity to survey electrodes. Although proximity to electrodes is not the only factor governing ERT resolution, it is often the primary factor for determining how an ERT system should be effectively deployed for a given application. Regardless of resolution limitations, the utility of ERT for remotely detecting wastes and monitoring subsurface processes is evident. ERT imaging is not a final detection or monitoring solution alone, but provides a necessary capability to remotely detect subsurface conditions and understand subsurface processes. Such a capability can significantly increase the efficacy and reduce the cost of remediation by guiding sampling efforts and informing remedial system design and operation. However, given the computational demands of 3D ERT characterization and monitoring inversions, the full imaging capability enabled by contemporary ERT survey systems can generally only be realized with high performance computing resources, both hardware and software. (authors)
This poster explores native parallel features in Fortran 2023 through the lens of supporting applications with libraries, compilers, and parallel runtimes. The language revision informally named Fortran 2008 introduced parallelism in the form of Single Program Multiple Data (SPMD) execution with two broad feature sets: (1) loop-level parallelism via do concurrent and (2) a Partitioned Global Address Space (PGAS) comprised of distributed “coarray” data structures. Fortran’s native parallelism has demonstrated high performance [1] and reduced the burden of inserting what sometimes amounts to more directives than code. Several compilers support both feature sets, typically by translating do concurrent into serial do loops annotated by parallel directives and by translating SPMD/PGAS features into direct calls to a communication library. Our research focuses primarily on two questions: (1) can the compiler’s parallel runtime library be developed in the language being compiled (Fortran) and (2) can we define an interface to the runtime that liberates compilers from being hardwired to one runtime and vice versa. We are answering these questions by developing the Parallel Runtime Interface for Fortran (PRIF) [2] and the Co-Array Fortran Framework of Efficient Interfaces to Network Environments (Caffeine) [3]. Caffeine is initially targeting adoption by LLVM Flang, a new open-source Fortran compiler developed by a broad community in industry, academia, and government labs. We are also exploring the use of these features in Inference-Engine, a deep learning library designed to facilitate neural network training and inference for high-performance computing applications written in modern Fortran.
Degradation mechanisms and sensor identification/selection resulted in a list of degradation modes and a list of sensors that are utilized in the diagnosis of these degradation modes. The sensor list is divided into primary and secondary indicators of the corresponding degradation modes. The signal conditioning requirements are discussed, describing the methods of producing the Space Shuttle Main Engine (SSME) post-hot-fire test data to be utilized by the Health Monitoring System. Development of the diagnostic logic and algorithms is also presented. The knowledge engineering approach, as utilized, includes the knowledge acquisition effort, characterization of the expert's problem solving strategy, conceptually defining the form of the applicable knowledge base, and rule base, and identifying an appropriate inferencing mechanism for the problem domain. The resulting logic flow graphs detail the diagnosis/prognosis procedure as followed by the experts. The nature and content of required support data and databases is also presented. The distinction between deep and shallow types of knowledge is identified. Computer coding of the Health Monitoring System is shown to follow the logical inferencing of the logic flow graphs/algorithms.
Estimation of riverbed profiles, also known as bathymetry, plays a vital role in many applications, such as safe and efficient inland navigation, prediction of bank erosion, land subsidence, and flood risk management. The high cost and complex logistics of direct bathymetry surveys, i.e, depth imaging, have encouraged the use of indirect measurements such as surface flow velocities. However, estimating high-resolution bathymetry from indirect measurements is an inverse problem that can be computationally challenging. Here, we propose a reduced-order model (ROM) based approach that utilizes a variational autoencoder (VAE), a type of deep neural network with a narrow layer in the middle, to compress bathymetry and flow velocity information and accelerate bathymetry inverse problems from flow velocity measurements. In our application, the shallow-water equations (SWE) with appropriate boundary conditions (BCs), e.g., the discharge and/or the free surface elevation, constitute the forward problem, to predict flow velocity. Then, ROMs of the SWEs are constructed on a nonlinear manifold of low dimensionality through a variational encoder and the bathymetry inversion problem is derived on the low-dimensional latent space in a Hierarchical Bayesian setting. Further, the reformulation allows variational inference with a small number (e.g., $\mathscr{O}$ (100) of ROM runs and efficient uncertainty quantification. We have tested our inversion approach on a one-mile reach of the Savannah River, GA, USA. Once the neural network is trained (offline stage), the proposed technique can perform the inversion operation orders of magnitude faster than traditional inversion methods that are commonly based on linear projections, such as principal component analysis (PCA), or the principal component geostatistical approach (PCGA). Furthermore, tests show that the algorithm can estimate the bathymetry with good accuracy even with sparse flow velocity measurements.
Fluid flow in the transonic regime finds relevance in aerospace engineering, particularly in the design of commercial air transportation vehicles. Computational fluid dynamics models of transonic flow for aerospace applications are computationally expensive to solve because of the high degrees of freedom as well as the coupled nature of the conservation laws. While these issues pose a bottleneck for the use of such models in aerospace design, computational costs can be significantly minimized by constructing special, structure-preserving surrogate models called reduced-order models. In this work, we propose a machine learning method to construct reduced-order models via deep neural networks and we demonstrate its ability to preserve accuracy with a significantly lower computational cost. In addition, our machine learning methodology is physics-informed and constrained through the utilization of an interpretable encoding by way of proper orthogonal decomposition. Application to the inviscid transonic flow past the RAE2822 airfoil under varying freestream Mach numbers and angles of attack, as well as airfoil shape parameters with a deforming mesh, shows that the proposed approach adapts to high-dimensional parameter variation well. Notably, the proposed framework precludes the knowledge of numerical operators utilized in the data generation phase, thereby demonstrating its potential utility in the fast exploration of design space for diverse engineering applications. Comparison against a projection-based nonintrusive model order reduction method demonstrates that the proposed approach produces comparable accuracy and yet is orders of magnitude computationally cheap to evaluate, despite being agnostic to the physics of the problem.
The evolving energy landscape requires novel tools to efficiently perform contingency analysis and reliability assessment of power grids, potentially in real-time. The high computational cost of traditional power flow solvers limits their applicability in practice. Machine learning (ML) surrogates such as deep neural networks (NNs) accelerate power flow solvers computations, enabling high-order contingency analysis and real-time decision-making by learning highly nonlinear functions and integrating grid topology via graph architectures. However, (graph) NNs lack predictive power away from training data and do not provide predictive confidence estimates. Here, we present a Bayesian residual graph NN that integrates knowledge from low-fidelity data via residual training and embeds granular quantification of uncertainties, improving trustworthiness critical for high-consequence decision-making. Applying Bayesian concepts to NNs is challenging due to the high-dimensionality of both the parameter space, complicating derivation of a meaningful prior, and the output space in large grid systems, requiring enhanced techniques to assess the predicted high-dimensional uncertainties. Our contributions include: (1) Deriving a prior for fully connected and graph NNs that leverages low-fidelity data to guide mean predictions and appropriately control prior predictive uncertainty. (2) Integrating this prior within an ensembling with anchoring scheme for efficient approximate posterior inference. (3) Deriving enhanced metrics to assess accuracy of both the mean and uncertainty predictions in high dimensions, appropriately accounting for correlations propagated through graph layers. The resulting Bayesian residual graph NN is tested on a contingency analysis task for 14-bus and 118-bus grids.
Interest in deep probabilistic graphical models has in-creased in recent years, due to their state-of-the-art performance on many machine learning applications. Such models are typically trained with the stochastic gradient method, which can take a significant number of iterations to converge. Since the computational cost of gradient estimation is prohibitive even for modestly sized models, training becomes slow and practically usable models are kept small. In this paper we propose a new, largely tuning-free algorithm to address this problem. Our approach derives novel majorization bounds based on the Schatten- norm. Intriguingly, the minimizers of these bounds can be interpreted as gradient methods in a non-Euclidean space. We thus propose using a stochastic gradient method in non-Euclidean space. We both provide simple conditions under which our algorithm is guaranteed to converge, and demonstrate empirically that our algorithm leads to dramatically faster training and improved predictive ability compared to stochastic gradient descent for both directed and undirected graphical models.
While deep learning models have seen increasing applications in protein science, few have been implemented for protein backbone generation—an important task in structure-based problems such as active site and interface design. We present a new approach to building class-specific backbones, using a variational auto-encoder to directly generate the 3D coordinates of immunoglobulins. Our model is torsion- and distance-aware, learns a high-resolution embedding of the dataset, and generates novel, high-quality structures compatible with existing design tools. We show that the Ig-VAE can be used with Rosetta to create a computational model of a SARS-CoV2-RBD binder via latent space sampling. We further demonstrate that the model’s generative prior is a powerful tool for guiding computational protein design, motivating a new paradigm under which backbone design is solved as constrained optimization problem in the latent space of a generative model.
It is widely known that neural networks (NNs) are universal approximators of continuous functions. However, a less known but powerful result is that a NN with a single hidden layer can accurately approximate any nonlinear continuous operator. This universal approximation theorem of operators is suggestive of the structure and potential of deep neural networks (DNNs) in learning continuous operators or complex systems from streams of scattered data. Here, in this work, we thus extend this theorem to DNNs. We design a new network with small generalization error, the deep operator network (DeepONet), which consists of a DNN for encoding the discrete input function space (branch net) and another DNN for encoding the domain of the output functions (trunk net). We demonstrate that DeepONet can learn various explicit operators, such as integrals and fractional Laplacians, as well as implicit operators that represent deterministic and stochastic differential equations. We study different formulations of the input function space and its effect on the generalization error for 16 different diverse applications.
The manipulation and control of nanoscale magnetic spin textures are of rising interest as they are potential foundational units in next-generation computing paradigms. Achieving this requires a quantitative understanding of the spin texture behavior under external stimuli using in situ experiments. Lorentz transmission electron microscopy (LTEM) enables real-space imaging of spin textures at the nanoscale, but quantitative characterization of in situ data is extremely challenging. Here, we present an AI-enabled phase-retrieval method based on integrating a generative deep image prior with an image formation forward model for LTEM. Our approach uses a single out-of-focus image for phase retrieval and achieves significantly higher accuracy and robustness to noise compared to existing methods. Furthermore, our method is capable of isolating sample heterogeneities from magnetic contrast, as shown by application to simulated and experimental data. This approach allows quantitative phase reconstruction of in situ data and can also enable near real-time quantitative magnetic imaging.
This paper will describe the use of digital Field Programmable Gate Arrays (FPGA) to contribute to advancing the state-of-the-art in software defined radio (SDR) transponder design for the emerging SmallSat and CubeSat industry and to provide advances for NASA as described in the TAO5 Communication and Navigation Roadmap (Ref 4). The use of software defined radios (SDR) has been around for a long time. A typical implementation of the SDR is to use a processor and write software to implement all the functions of filtering, carrier recovery, error correction, framing etc. Even with modern high speed and low power digital signal processors, high speed memories, and efficient coding, the compute intensive nature of digital filters, error correcting and other algorithms is too much for modern processors to get efficient use of the available bandwidth to the ground. By using FPGAs, these compute intensive tasks can be done in parallel, pipelined fashion and more efficiently use every clock cycle to significantly increase throughput while maintaining low power. These methods will implement digital radios with significant data rates in the X and Ka bands. Using these state-of-the-art technologies, unprecedented uplink and downlink capabilities can be achieved in a 1/2 U sized telemetry system. Additionally, modern FPGAs have embedded processing systems, such as ARM cores, integrated inside the FPGA allowing mundane tasks such as parameter commanding to occur easily and flexibly. Potential partners include other NASA centers, industry and the DOD. These assets are associated with small satellite demonstration flights, LEO and deep space applications. MSFC currently has an SDR transponder test-bed using Hardware-in-the-Loop techniques to evaluate and improve SDR technologies.