Search NASA⌕ Search

SEARCH · Search NASA

Results for “successive training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Building Neutron Scattering Infrastructure in Louisiana for Advanced Materials (Final Report)

The main objective of the Louisiana Consortium for Neutron Scattering (LaCNS) is to build a major neutron scattering infrastructure capable of treating both soft and hard materials. The goal is to create a sustainable effort having the quality, breadth, and depth necessary to produce competitive proposals including collaborative and center type proposals. Our scientific aim is to understand the role of coupling in emergent complex materials and its impact on the structure/property relationship and to explore how to tune the key couplings to guide the design of materials with the desired properties. This naturally includes building a base of users of the Spallation Neutron Source (SNS) and the High Flux Isotope Reactor (HFIR) in Louisiana; to train highly talented graduate and post-doctoral students in synthesis, neutron scattering, and simulation and modeling techniques, thereby helping to produce the next generation of scientist who use neutron scattering techniques as a crucial part of their research. The goal of our hard matter program is to understand the interplay between spin, charge, orbital, and lattice degrees of freedom in carefully selected complex materials. We have made considerable progress on number of complex systems including the oxide Sr3(Ru1-xMnx)2O7 where short-range magnetic ordering with anisotropic spin texture is initiated at the metal-insulator transition that clearly indicates a strong spin-charge coupling. Another critical area is in quantum materials. For example, in the topological semimetal BaMnSb2, we found a 3D canted antiferromagnetic Weyl semimetal with a 2D electronic structure and a nontrivial Berry phase. In addition, in the hexagonal chiral Mn1/3NbS2 system, we found soliton/soliton and soliton/antisoliton domain walls where the application of small fields or small currents can be used to control nanoscopic magnetic domains where the control of domain walls is crucially important for information storage. In addition, chemical transformation investigations were performed on the VISION instrument at Spallation Neutron Source (SNS). The soft matter effort was focused on understanding the role of non-covalent interactions on the structure and dynamics of fluid-based soft matter. One key focus was on sequence-defined (SD) amphiphilic peptoid polymers that allow encoding of molecular interactions and thereby systematic investigations of how charge directs the solution self-assembly of amphiphilic polymers in water. Another important area was on the dynamics of lipids self-assembled into membranes for exploring the permeability and mechanical using both neutron spin echo (NSE) spectroscopy and quasielastic neutron scattering (QENS) to distinguished between viscoelasticity and permeability at the molecular scale. Overall, the LaCNS project was quite successful, generating 145 publications and 245 presentations. Our graduate and post-doctoral students were well trained resulting in positions in national laboratories (Oak Ridge National Laboratory (ORNL), Argonne National Laboratory and Los Alamos National Laboratory), major research universities, and industry. We also developed a uniaxial pressure cell along with ORNL for SNS. A critical goal of this project was to establish a foundation for competing nationally in federally funded research programs. To this end, we were quite successful in generating over twenty-two federally and non-federally funded grants including awards from NSF, DOE, and DOD, and two early career awards. Equally important, we were able to secure a key major piece of instrumentation via a large ARO grant for a state-of-the-art electron microscope.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Respecting causality for training physics-informed neural networks

While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date PINNs have not been successful in simulating dynamical systems whose solution exhibits multi-scale, chaotic or turbulent behavior. In this work we attribute this shortcoming to the inability of existing PINNs formulations to respect the spatio-temporal causal structure that is inherent to the evolution of physical systems. We argue that this is a fundamental limitation and a key source of error that can ultimately steer PINN models to converge towards erroneous solutions. We address this pathology by proposing a simple re-formulation of PINNs loss functions that can explicitly account for physical causality during model training. We demonstrate that this simple modification alone is enough to introduce significant accuracy improvements, as well as a practical quantitative mechanism for assessing the convergence of a PINNs model. Here we provide state-of-the-art numerical results across a series of benchmarks for which existing PINNs formulations fail, including the chaotic Lorenz system, the Kuramoto–Sivashinsky equation in the chaotic regime, and the Navier–Stokes equations. To the best of our knowledge, this is the first time that PINNs have been successful in simulating such systems, introducing new opportunities for their applicability to problems of industrial complexity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Realizing the scientific program with polarized ion beams at the future BNL Electron Ion Collider

Polarized ion beams at the Electron Ion Collider (EIC) are essential to address some of the most important open questions at the twenty-first century frontiers of understanding of the fundamental structure of matter. Here, in this work, we summarize the science case and identify polarized 2 H, 3 He, 6 Li, and 7 Li ion beams as critical technology that will enable experiments which address the most important science. Furthermore, we discuss the required ion polarimetry and spin manipulation at the EIC. The current EIC accelerator design is presented. We identify a significant research and development effort across national and international laboratories and universities that is required over about a decade to realize the polarized ion beams and estimate (based on previous experience) that it will require about 20 full-time equivalent (FTE) over 10 years (or a total of about 200 FTE-years) of personnel, including graduate students, postdoctoral researchers, technicians, and engineers. Attracting, educating, and training a new generation of physicists in experimental spin techniques will be essential for the successful realization. Artificial intelligence and machine learning are seen as having significant potential for both acceleration of research and development and amplification of discovery in the optimal realization of this unique quantum technology on a cutting-edge collider. The research and development effort is synergistic with research in atomic physics and fusion energy science.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Adaptive Computing for Scale-Up Problems

Adaptive Computing is an application-agnostic outer loop framework to strategically deploy simulations and experiments to guide decision making for scale-up analysis. Resources are allocated over successive batches, which makes the allocation adaptive to some objective such as optimization or model training. The framework enables the characterization and management of uncertainties associated with predictive models of complex systems when scale-up questions lead to significant model extrapolation. A key advancement of this framework is its integration of multi-fidelity surrogate modeling, uncertainty management, and automated orchestration of various computing and experimentation resources into a single integrated software package. This enables efficient multi-fidelity modeling across multiple computing resources by incorporating real-world constraints such as relative queue times and throughput on individual machines into the multi-fidelity sampling decision. We discuss applications of this framework to problems in the renewable energy space, including biofuels production, material synthesis, perovskite crystal growth, and building electrical loads.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

EMPOWERED Distributed Energy Resources Permit Accelerator Pilot

This report summarizes the EMPOWERED Distributed Energy Resources (DER) Permit Accelerator Pilot project. Its primary objective was to streamline the design, installation, permitting, and inspections of targeted Distributed Energy Resource (DER) solutions. Streamlining these processes is critical to strengthening and accelerating the adoption of DER solutions nationwide. Moreover, streamlined processes help expedite the clean energy transition and expand energy resilience by improving understanding and capability among key stakeholders like property owners, code officials, and the general workforce. The project team accomplished this by developing and deploying a set of design and permitting guides for simple DER solutions, which they rolled out via a permitting pilot program. The project team developed nine permitting and inspection guides for simple DER solutions, as follows: 1. Single-Family and Duplex: Electric Vehicle Service Equipment (EVSE), and Storage, Solar + Storage (each in two code cycles) 2. Multi-family and Office: EVSE, Storage, Solar + Storage (each in two code cycles) 3. A permitting process guide to support the implementation of the permitting guides. The guides cover key code requirements via plan review and field inspection checklists for code officials and building owners to follow to streamline the permitting and inspections process for DER solutions. Following the development of the initial set of guides, the project team formed two cohorts, each made up of four jurisdictions, to participate in the Permit Accelerator Pilot. Each cohort received training and technical assistance to support them in incorporating the guides into use. The project successfully engaged jurisdictions in two disparate US regions: Chelsea, Somerville, Natick, and Norwood in metro Boston; and Pima County, Town of Gilbert, Flagstaff, and Sedona in Arizona. Feedback gathered from the project was valuable in refining the guides to better address jurisdictional needs. After the pilot period, the project team developed final versions of the guides based on extensive feedback from the cohorts and from external technical peer review. Feedback included needs and barriers related to implementation, as well as detailed technical improvements from external peer review. The final guides were presented in a series of webinars with a national reach, with 192 attendees from 35 states in the live session of the final webinar, and the final guides are posted online and available for use nationwide. Key findings: • The most receptive jurisdictions were those adjacent to other jurisdictions that had already adopted similar initiatives. The least receptive jurisdictions were those in the midst of adopting new code cycles. • The most common barrier to jurisdictional adoption of new guidelines were internal administrative and process delays, more so than technical barriers. • The main technical barrier to adoption proved to be the concern of fire hazard risks posed by energy storage systems. There is some apprehension in the code enforcement community about the safety of the batteries. Education coordinated with fire and safety services will help alleviate concerns and increase confidence in the acceptance of innovative technologies. • Opportunities to further leverage and advance the guides and related resources include outreach, education, and jurisdictional support; updates to the resources to align with code cycle changes and local requirements; and advancements supporting emerging technologies.

14 SOLAR ENERGY↗
DOE OSTI2024

Assessing Membership Inference Attacks under Distribution Shifts

Membership inference attacks (MIAs) exploit machine learning models to infer whether a data point was in the training set, posing significant privacy risks even with limited black-box access. These attacks rely on the attacker approximating the target model’s training distribution, yet the impact of distribution shifts between target and shadow models on MIA success remains underexplored. We systematically evaluate five types of distribution shifts —-cutout, jitter, Gaussian noise, label shift, and attribute shift —- at varying intensities. Our results reveal that these shifts affect MIA effectiveness in nuanced ways, with some reducing attack success while others exacerbate vulnerabilities, and the same shift can have opposite effects depending on the type of MIA. This highlights the complex interplay between distributional differences and attack performance, offering critical insights for improving model defenses against MIAs.

Shi, Yichuan [Massachusetts Institute of Technolog↗
DOE OSTI2022

Neural operator prediction of linear instability waves in high-speed boundary layers

Here, we investigate if neural operators can predict the linear evolution of instability waves in high-speed boundary layers. To this end, we extend the design of the DeepOnet to ensure accurate and robust predictions, and also to perform data assimilation. In particular, we train DeepONet to take as inputs an upstream disturbance and a downstream location of interest, and to provide as output the perturbation field downstream in the boundary layer. DeepONet thus approximates the linearized and parabolized Navier-Stokes operator for this flow. For successful application to the high-speed boundary layer problem, we add sample weighting and Fourier input features to the regular DeepONet formulation. Once trained, the DeepOnet can perform fast and accurate predictions of the downstream disturbances within the range of training frequencies (inside the distribution). In addition, we show that DeepONet can solve the inverse problem, where downstream wall measurements are adopted as input, and a trained network can predict the upstream disturbances that led to these observations. This capability, along with the forward predictions, allows us to perform a full data assimilation cycle efficiently: starting from wall-pressure data, we predict the upstream disturbance using the inverse DeepONet and its evolution using the forward DeepONet. Finally, we introduce three new metrics to benchmark the training, evaluation and break-even cost of neural operators.

42 ENGINEERING↗
DOE OSTI2025

Floating Offshore Wind US Manufacturing and Commercialization: Cooperative Research and Development (Final Report)

NREL assessed the supply chain and workforce considerations for the OCG-Wind floater technology, a floating semi-submersible offshore wind substructure, as well sharing vessel needs to inform their installation strategy. This technical assistance was in support of the FLoating Offshore Wind ReadINess (FLOWIN) Prize Phase 2 submission. NREL provided an assessment of domestic supplier capabilities for the main components of their floating offshore wind platform design and analyzed US regional and national supply chain constraints and gaps. Thirteen interviews with companies including steel distributors, forges, foundries, ports, large component fabricators, subcomponent fabricators, and secondary suppliers provided key insights such as 1) assembly ports are the key infrastructure barrier standing in the way of unlocking the domestic assembly and component fabrication for steel-based FOW platforms, 2) domestic steel producers can supply the types and quantities of steel necessary for FOW platforms, and 3) coordination between stakeholders will be a vital part of successfully developing the supply chain and infrastructure needed to domestically produce FOW platforms. In the workforce assessment, NREL documented a step-by-step approach to conduct a place-based assessment of the foundational workforce consideration for recruiting, upskilling, and retaining a workforce, such as supportive local and state policy, nearby education and training programs, and existing relevant industry. This approach was applied to Tacoma, Washington. Tacoma was indicated to have the potential be a successful location for fabrication and assembly of floating offshore wind energy in terms of workforce development. To share data on vessel requirements to install the OCG-Wind floater, NREL compiled resources that help answer the questions related to anchor handling tug vessels, shared a database of cable laying vessels, and answered questions on complying with the Jones Act.

17 WIND ENERGY↗
DOE OSTI2026

Augmented Reality Technologies for Radiation Safety Training: A Systematic Review of Sensor Integration and Visualization Approaches

This paper presents a comprehensive systematic review examining the application of augmented reality (AR) and sensor technologies for visualizing ionizing radiation in virtual training environments. The review methodology involved systematic identification and analysis of the relevant literature based on predetermined criteria including publication type, year of publication, application domain, and technological approach. The literature search encompassed publications from 2011 to 2021 across four major academic databases: Web of Science, Google Scholar, IEEE Xplore, and Scopus. Through rigorous screening following PRISMA 2020 guidelines, 23 research articles met the inclusion criteria for detailed analysis. From 404 initial database records, 360 were excluded during title/abstract screening (primarily for lacking AR components, radiation focus, or training applications) and 4 during full-text assessment (all for lacking sensor integration). The findings reveal that AR-based ionizing radiation visualization has been successfully implemented across diverse domains, including nuclear facility operations, medical procedures, CERN research activities, and educational and monitoring applications. The analysis identified multiple dimensions of impact, encompassing distinct benefits, emerging opportunities, and implementation challenges associated with AR deployment for ionizing radiation training. Each of these dimensions is comprehensively examined and documented within this review. Additionally, this study identifies critical research gaps that currently limit the full potential of AR technology in supporting ionizing radiation training programs. These gaps are systematically analyzed and discussed to establish clear directions for future research endeavors in this emerging field.

61 - RADIATION PROTECTION AND DOSIMETRY↗
DOE OSTI2024

Constraining Galaxy-Halo connection using machine learning

We investigate the potential of machine learning (ML) methods to model small-scale galaxy clustering for constraining Halo Occupation Distribution (HOD) parameters. Our analysis reveals that while many ML algorithms report good statistical fits, they often yield likelihood contours that are significantly biased in both mean values and variances relative to the true model parameters. This highlights the importance of careful data processing and algorithm selection in ML applications for galaxy clustering, as even seemingly robust methods can lead to biased results if not applied correctly. ML tools offer a promising approach to exploring the HOD parameter space with significantly reduced computational costs compared to traditional brute-force methods if their robustness is established. Using our ANN-based pipeline, we successfully recreate some standard results from recent literature. Properly restricting the HOD parameter space, transforming the training data, and carefully selecting ML algorithms are essential for achieving unbiased and robust predictions. Among the methods tested, artificial neural networks (ANNs) outperform random forests (RF) and ridge regression in predicting clustering statistics, when the HOD prior space is appropriately restricted. We demonstrate these findings using the projected two-point correlation function (w p (r p )), angular multipoles of the correlation function (ξ ℓ (r)), and the void probability function (VPF) of Luminous Red Galaxies from Dark Energy Spectroscopic Instrument mocks. Our results show that while combining w p (r p ) and VPF improves parameter constraints, adding the multipoles ξ 0 , ξ 2 , and ξ 4 to w p (r p ) does not significantly improve the constraints.

cosmology↗
DOE OSTI2021

PreMevE Update: Forecasting Ultra-Relativistic Electrons Inside Earth's Outer Radiation Belt

Energetic electrons inside Earth's Van Allen belts pose a major radiation threat to spaceborne electronics that often play vital roles in modern society. Ultra-relativistic electrons with energies greater than or equal to two megaelectron-volt (MeV) are of particular interest, and thus forecasting these ≥2 MeV electrons has a significant meaning to all space sectors. Here, we update the latest development of the predictive model for MeV electrons in the outer radiation belt. The new version, called PREdictive MEV Electron (PreMevE)-2E, forecasts ultra-relativistic electron flux distributions across the outer belt, with no need for in situ measurements of the trapped MeV electron population except at the geosynchronous orbit (GEO). Model inputs include precipitating electrons observed in low-Earthorbits by NOAA satellites, upstream solar wind speeds and densities from solar wind monitors, as well as ultra-relativistic electrons measured by one Los Alamos GEO satellite. We evaluated 32 supervised machine learning models that fall into four different classes of linear and neural network architectures, and successfully tested ensemble forecasting by using groups of top-performing models. All models are individually trained, validated, and tested by in situ electron data from NASA's Van Allen Probes mission. It is shown that the final ensemble model outperforms individual models at most L-shells, and this PreMevE-2E model can provide 25-h (~1-day) and 50-h (~2-day) forecasts with high mean performance efficiency and correlation values. Our results also suggest that this new model is dominated by nonlinear components at L-shells <~4 for ultra-relativistic electrons, different from the dominance of linear components for 1 MeV electrons as previously discovered.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Towards maintaining canine training aid integrity: Effects of environmental factors and operational use on the triacetone triperoxide polymer odor capture‐and‐release system

Abstract There are many factors that may affect the longevity of or guide the use of canine training aids. Literature to date has mainly focused on identifying the headspace volatiles associated with training aids or odors and only minimal research exists into how different variables may alter those volatiles. The current study examines several factors affecting canine training aids: humidity, air flow, transportation, and operational deployment, using the triacetone triperoxide polymer odor capture‐and‐release canine training aid (TATP POCR) as the target. The TATP POCR is an absorption‐based canine training aid developed to be used to safely train canines to detect the odor of the explosive TATP in operational settings. Comparisons of the TATP POCR to neat TATP are made throughout the manuscript. First, humidity increased the background components of the POCR matrix, as well as the amount of TATP recovered was above the POCR. Humidity thus affected the amount of TATP detected but did not prevent detection. Second, air flow lessened the lifetime of the TATP POCR. Third, the practice of using primary and secondary containment successfully prevented contamination, cross‐contamination, and significant target loss, thereby maintaining kit integrity. Finally, the absorption of background odors from training environments was not observed. TATP headspace concentrations between a Deployed and Control POCR kit were not significantly different at time 0 (i.e., upon opening), which suggests that the operational use does not affect the function of the TATP POCR system. This information provides pivotal evidence for explosives detection canine handlers or trainers who utilize the TATP POCR.

Legal Medicine↗
DOE OSTI2021

Chemometrics and Experimental Design for the Quantification of Nitrate Salts in Nitric Acid: Near-Infrared Spectroscopy Absorption Analysis

Implementing remote, real-time spectroscopic monitoring of radiochemical processing streams in hot cell environments requires efficiency and simplicity. The success of optical spectroscopy for the quantification of species in chemical systems highly depends on representative training sets and suitable validation sets. Selecting a training set (i.e., calibration standards) to build multivariate regression models is both time- and resource-consuming using standard one-factor-at-a-time approaches. This study describes the use of experimental design to generate spectral training sets and a validation set for the quantification of sodium nitrate (0–1 M) and nitric acid (0.1–10 M) using the near-infrared water band centered at 1440 nm. Partial least squares regression models were built from training sets generated by both D- and I-optimal experimental designs and a one-factor-at-a-time approach. The prediction performance of each model was evaluated by comparing the bias and standard error of prediction for statistical significance. D- and I-optimal designs reduced the number of samples required to build regression models compared with one-factor-at-a-time while also improving performance. Models must be confirmed against a validation sample set when minimizing the number of samples in the training set. The D-optimal design performed the best when considering both performance and efficiency by improving predictive capability and reducing number of samples in the training set by 64% compared with the one-factor-at-a-time approach. The experimental design approach objectively selects calibration and validation spectral data sets based on statistical criterion to optimize performance and minimize resources.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2019

Environmental Awareness Course 32461

Los Alamos National Laboratory (LANL or the Laboratory) is committed to an environmental management strategy aimed at reducing the Laboratory's environmental impacts, while still maintaining or increasing operating efficiencies. This training is designed to help you understand the critical importance of environmental management to the continuing success of the Laboratory and your personal role in this essential endeavor.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Brilliant Execution of Smart Labs: Employing Best Practices to Improve Safety and Reduce Energy for Sustainable Labs: Preprint

Laboratory buildings often dominate the energy use on research campuses, consuming three to four times the energy of a typical office building. In most cases, these facilities are defined as “mission critical,” with excessive energy use as simply an unfortunate truth that cannot be avoided without compromising safety. However, case studies indicate cost savings opportunities of up to 40% in laboratories (compared to standard practices) by employing energy efficient design strategies that also improve the safety of lab spaces (Better Buildings 2018). To help stakeholders implement improvement measures, the Smart Labs Toolkit describes a proven approach to optimizing performance of laboratories and critical control environments in new or existing facilities. The Toolkit outlines distinct phases for building a successful Smart Labs program that include: Plan, Assess, Optimize, and Manage. Each phase has specific tasks, training videos, and resources to educate users, support the efforts of key stakeholders, and maximize benefits for the organization in implementing high-performance labs. The Toolkit also includes case studies from the 17 Better Buildings Smart Labs Accelerator partners, sharing lessons learned and supporting the Smart Labs approach to safe, efficient world class science. This paper reviews the technology choices for implementing Smart Labs ventilation strategies, controls, and other efficiency measures. Better Building Smart Lab Accelerator partners provide case study examples of safety and energy-savings results of using Smart Labs approaches.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Uncertainty quantification of a physics-informed model based on sparse identification of a Thermal Energy Distribution System

Integrated energy systems (IES)s are crucial for enhancing the economy and efficiency of power generation sources (e.g., nuclear energy) necessary to unleash American energy dominance. These systems can be integrated with thermal energy storage (TES) and intermittent renewable energies to optimize overall energy use, peak-load regulation, and demand-side responses. However, the stabilization of energy generation, transport, and utilization introduces operational complexities that exceed the challenges of managing each sub-component individually. Currently, though IESs rely on human operators for efficiency and stability, reducing human error risk and enhancing performance through automation is highly desirable. Recent advances at Idaho National Laboratory have demonstrated successful control of the Thermal Energy Distributed System (TEDS). However, the automatic control system depends on a deterministic Sparse Identification of Nonlinear Dynamics with Control (SINDyC) model, which are trained based on simulation data from physics-based simulations. Because of uncertainties in physics-based simulation, SINDyC model results in large discrepancies against experimental data and cannot be reliably used in automatic control. In this paper, we present an innovative approach to address these discrepancies by quantifying uncertainties and developing a more robust model. We first generated trajectories by using first-principles physics codes to encapsulate the experiment. Next, we trained thousands of models by randomly sampling these trajectories. We then collapsed all those models into one probabilistic SINDyC by fitting a multivariate Gaussian distribution onto the resulting coefficient’s distribution. Despite its simplicity, our approach successfully produced 95% confidence intervals that captured the experimental trajectories. It even did so with a higher probability and better U-pooling score across six of the seven relevant quantities of interest (QoIs), as compared to other classical approaches. In conclusion, ongoing research is focusing on generating new experimental trajectories to validate this approach, and on employing Bayesian calibration to refine parametric uncertainties and guide future model development efforts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

A library of AI-assisted FAIR water cycle and related disturbance datasets to enable model training, parameterization and validation

This whitepaper is responsive to focal area Data acquisition and assimilation enabled by machine learning, AI, and advanced methods. Here we describe how FAIR (Findable, Accessible, Reusable, Interoperable) datasets related to water cycle extremes are essential for successful implementation of ML in Earth System and other models. We also describe how AI can be used to acquire and integrate water cycle data related to extreme events to create a library of FAIR datasets for training and evaluating algorithms.

58 GEOSCIENCES↗
DOE OSTI2022

Multi-variance replica exchange SGMCMC for inverse and forward problems via Bayesian PINN

Physics-informed neural network (PINN) has been successfully applied in solving a variety of nonlinear non-convex forward and inverse problems. However, the training is challenging because of the non-convex loss functions and the multiple optima in the Bayesian inverse problem. In this work, we propose a multi-variance replica exchange stochastic gradient Langevin dynamics method to tackle the challenge of the multiple local optima in the optimization and the challenge of the multiple modal posterior distribution in the inverse problem. Replica exchange methods are capable of escaping from the local traps and accelerating the convergence; two chains with different temperatures are designed where the low temperature chain aims for the local convergence, and the target of the high temperature chain is to travel globally and explore the whole loss function entropy landscape. However, it may not be efficient to solve mathematical inversion problems by using the vanilla replica method directly since the method doubles the computational cost in evaluating the forward solvers (likelihood functions) in the two chains. To address this issue, we propose to make different assumptions on the energy function estimation and this facilities one to use solvers of different fidelities in the likelihood function evaluation. More precisely, one can use a solver with low fidelity in the high temperature chain while using a solver with high fidelity in the low temperature chain. Our proposed method significantly lowers the computational cost in the high temperature chain, meanwhile preserving the accuracy and converging very fast. Here we give an unbiased estimate of the swapping rate and give an estimation of the discretization error of the scheme. To verify our idea, we design and solve four inverse problems which have multiple modes. The proposed method is also employed to train the Bayesian PINN to solve the forward and inverse problems; faster and more accurate convergence has been observed when compared to the stochastic gradient Langevin dynamics (SGLD) method and vanilla replica exchange methods.

97 MATHEMATICS AND COMPUTING↗