SEARCH · Search NASA

Results for “multi-modal networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Ultrafast CMOS image sensors and data-enabled super-resolution for multimodal radiographic imaging and tomography

We summarize recent progress in ultrafast Complementary Metal Oxide Semiconductor (CMOS) image sensor development and the application of neural networks for post-processing of CMOS and charge-coupled device (CCD) image data to achieve sub-pixel resolution (thus $super$-$resolution$). The combination of novel CMOS pixel designs and data-enabled image post-processing provides a promising path towards ultrafast high-resolution multi-modal radiographic imaging and tomography applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Multimodal representation learning for predicting molecule–disease relations

Motivation: Predicting molecule–disease indications and side effects is important for drug development and pharmacovigilance. Comprehensively mining molecule–molecule, molecule–disease and disease–disease semantic dependencies can potentially improve prediction performance. Methods: We introduce a Multi-Modal REpresentation Mapping Approach to Predicting molecular-disease relations (M2REMAP) by incorporating clinical semantics learned from electronic health records (EHR) of 12.6 million patients. Specifically, M2REMAP first learns a multimodal molecule representation that synthesizes chemical property and clinical semantic information by mapping molecule chemicals via a deep neural network onto the clinical semantic embedding space shared by drugs, diseases and other common clinical concepts. To infer molecule–disease relations, M2REMAP combines multimodal molecule representation and disease semantic embedding to jointly infer indications and side effects. Results: We extensively evaluate M2REMAP on molecule indications, side effects and interactions. Results show that incorporating EHR embeddings improves performance significantly, for example, attaining an improvement over the baseline models by 23.6% in PRC-AUC on indications and 23.9% on side effects. Further, M2REMAP overcomes the limitation of existing methods and effectively predicts drugs for novel diseases and emerging pathogens. Availability and implementation: The code is available at https://github.com/celehs/M2REMAP, and prediction results are provided at https://shiny.parse-health.org/drugs-diseases-dev/.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

Ultrafast CMOS image sensors and data-enabled super-resolution for multimodal radiographic imaging and tomography

We summarize recent progress in ultrafast Complementary Metal Oxide Semiconductor (CMOS) image sensor development and the application of neural networks for post-processing of CMOS and charge-coupled device (CCD) image data to achieve sub-pixel resolution (thus ‘super-resolution’). The combination of novel CMOS pixel designs and data-enabled image post-processing provides a promising path towards ultrafast high-resolution multi-modal radiographic imaging and tomography applications.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

CO 2 storage site characterization using ensemble-based approaches with deep generative models

Estimating spatially distributed properties such as permeability from available sparse measurements is a great challenge in efficient subsurface CO 2 storage operations. In this paper, a deep generative model that can accurately capture complex subsurface structure is tested with an ensemble-based inversion method for accurate and accelerated characterization of CO 2 storage sites. We chose Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) for its realistic reservoir property representation and Ensemble Smoother with Multiple Data Assimilation (ES-MDA) for its robust data fitting and uncertainty quantification capability. WGAN-GP are trained to generate high-dimensional permeability fields from a low-dimensional latent space and ES-MDA then updates the latent variables by assimilating available measurements. Several subsurface site characterization examples including Gaussian, channelized, and fractured reservoirs are used to evaluate the accuracy and computational efficiency of the proposed method and the main features of the unknown permeability fields are characterized accurately with reliable uncertainty quantification. Furthermore, the estimation performance is compared with a widely-used variational, i.e., optimization-based, inversion approach, and the proposed approach outperforms the variational inversion method in several benchmark cases. We explain such superior performance by visualizing the objective function in the latent space: because of nonlinear and aggressive dimension reduction via generative modeling, the objective function surface becomes extremely complex while the ensemble approximation can smooth out the multi-modal surface during the minimization. This suggests that the ensemble-based approach works well over the variational approach when combined with deep generative models at the cost of forward model runs unless convergence-ensuring modifications are implemented in the variational inversion.

42 ENGINEERING↗

DOE OSTI2021

Iterative self-organizing SCEne-LEvel sampling (ISOSCELES) for large-scale building extraction

Convolutional neural networks (CNN) provide state-of-the-art performance in many computer vision tasks, including those related to remote-sensing image analysis. Successfully training a CNN to generalize well to unseen data, however, requires training on samples that represent the full distribution of variation of both the target classes and their surrounding contexts. With remote sensing data, acquiring a sufficiently representative training set is a challenge due to both the inherent multi-modal variability of satellite or aerial imagery and the general high cost of labeling data. To address this challenge, we have developed ISOSCELES, an Iterative Self-Organizing SCEne LEvel Sampling method for hierarchical sampling of large image sets. Using affinity propagation, ISOSCELES automates the selection of highly representative training images. Compared to random sampling or using available reference data, the distribution of the training is principally data driven, reducing the chance of oversampling uninformative areas or undersampling informative ones. In comparison to manual sample selection by an analyst, ISOSCELES exploits descriptive features, spectral and/or textural, and eliminates human bias in sample selection. Using a hierarchical sampling approach, ISOSCELES can obtain a training set that reflects both between-scene variability, such as in viewing angle and time of day, and within-scene variability at the level of individual training samples. We verify the method by demonstrating its superiority to stratified random sampling in the challenging task of adapting a pre-trained model to a new image and spatial domain for country-scale building extraction. Using a pair of hand-labeled training sets comprising 1,987 sample image chips, a total of 496,000,000 individually labeled pixels, we show, across three distinct model architectures, an increase in accuracy, as measured by F1-score, of 2.2–4.2%.

42 ENGINEERING↗

DOE OSTI2023

Understanding and Improving Energy Efficiency of Regional Mobility Systems Leveraging System-Level Data

Increased congestion required urban Americans to travel 6.8 billion hours more and purchase 3.1 billion gallons of fuel for a congestion cost of $\$$153 billion, according to the 2019 Urban Mobility Report. How to effectively manage the regional mobility system and improve the energy efficiency presents a big challenge to public agencies. Recent years have witnessed massive multi-jurisdictional multi-modal system-level data from various sources, which provides an unprecedented opportunity to improve the mobility system and its energy efficiency. However, implications of system-level data for mobility and energy efficiency are unclear. Those system-level data sets are siloed, spatially and temporally sparse, biased, not unified, and lacking of insights for system management. Consequently, there is a real need to acquire, fuse, mine and learn from multi-source system-level data to prepare public agencies to deal more effectively with large-scale energy efficiency modeling, management and planning. This project proposes to intensively review inexpensive, replicable and openly-accessible data from multi-modal systems, develop a data-driven system-level modeling framework enabled and validated by data, identify the energy inefficiencies of mobility systems from infrastructure, vehicles, passenger systems, and quantify the benefits of system-level strategies to improve mobility/energy efficiency. In addition, this research develops models to effectively estimate energy consumption and emissions from various types of vehicles on the roadway networks, with high granularity and high fidelity. Traditional models often heavily rely on aggregated infrastructure or vehicle/passenger data, for example, the census survey, land-use, and traffic counts of one or several classes, which may lead to research gaps considering the emerging vehicle technologies. Those models do not contain individual vehicular information. We propose an integrated data-driven method that combines multiple network modeling components, featuring the utilization of state-wide vehicle registration data. The additional vehicle registration data improve the model performance, and produce high-resolution vehicle-specific estimates of emissions and network performance metrics. Two case studies on the Pittsburgh and Philadelphia regional network show that the proposed method can efficiently and effectively estimate the emissions of a large-scale network, and provide valuable information for evaluating common management strategies.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2022

Layer-wise Imaging Dataset from Powder Bed Additive Manufacturing Processes for Machine Learning Applications (Peregrine v2022-10)

36 MATERIALS SCIENCE↗

DOE OSTI2023

Layer-wise Imaging Dataset from Powder Bed Additive Manufacturing Processes for Machine Learning Applications (Peregrine v2022-10.1)

This release consists of six datasets which together include multi-modal layer-wise powder bed images from two different powder bed printing technologies. These datasets are designed primarily to facilitate the development and testing of new computer vision and machine learning based anomaly and defect detection algorithms. The authors provide both training data with corresponding ground truth pixel masks and evaluation data with corresponding baseline prediction pixel masks made by a trained neural network. The laser powder bed fusion (L-PBF) datasets are sourced from EOS M290 and AddUp FormUp 350 printers and the binder jet (BJ) dataset is sourced from an ExOne M-Flex printer. The materials represented in these datasets include 17-4 PH Stainless Steel, GammaPrint-700, Inconel 718, Maraging Steel, and H13 Steel. The sensor imaging modalities represented include visible-light (VL), temporally-integrated (i.e., long duration exposure) near-infrared (TI-NIR), and wide-band infrared (IR). To download the dataset: (1) Create a Globus account. (2) Create a Globus Endpoint on your computer. (3) Transfer the dataset from the OLCF DOI-DOWNLOADS Collection to your Collection. Common troubleshooting steps: (a) Confirm that the transfer is going from OLCF DOI-DOWNLOADS to your Collection. (b) Create an exception for Globus in your antivirus software so that it can create an Endpoint. (c) Manually create a Globus access directory (where the data will be downloaded) by going to the Preferences > Access tab.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Characterization of the degradation of gamma-irradiated elastomers using Raman spectroscopy

This report presents key findings from Raman spectroscopic analysis of gamma-irradiated rubber samples extracted from a laminated lead-damped rubber (LDR) seismic isolation device. The samples were exposed to gamma radiation from a 60Co source in a Foss Therapy Services gamma irradiator, reaching absorbed doses up to 1600 kGy. A distinct threshold near 400 kGy was identified, beyond which significant spectral changes were observed. Two Raman peaks - at approximately 425 cm-1 and 2440 cm-1 - were tracked as a function of dose using Gaussian fitting. The 425 cm-1 peak, attributed to sulfur–sulfur (S–S) bond stretching (resulting from vulcanization of the rubber), exhibited a dose-dependent upshift, indicating radiation-induced crosslinking within the sulfur-based polymer network. Conversely, the 2440 cm-1 peak, likely associated with vibrational modes of additives or impurities, showed a downward shift with increasing dose, suggesting chain scission and degradation of non-rubber constituents. These results provide first-of-a-kind insights into the microstructural evolution of elastomers under high-dose gamma irradiation and establish a preliminary dose threshold for significant degradation. Future work will incorporate multi-modal characterization—including Fourier Transform Infrared (FTIR) spectroscopy, scanning electrom microscopy (SEM) of the rubber surface morphology, thermogravimetric analysis (TGA) to determine changes in thermal stability, and mechanical testing—to correlate molecular-level changes with macroscopic performance of these elastomers as damping media in seismic isolation devices. These findings are expected to provide regulatory guidance and design criteria for qualifying low-damping rubber seismic isolators in advanced nuclear reactor applications.

36 - MATERIALS SCIENCE↗

DOE OSTI2022

Multi-modal characterization and simulation of human epileptic circuitry

Temporal lobe epilepsy is the fourth most common neurological disorder, with about 40% of patients not responding to pharmacological treatment. Increased cellular loss is linked to disease severity and pathological phenotypes such as heightened seizure propensity. While the hippocampus is the target of therapeutic interventions, the impact of the disease at the cellular level remains unclear. Here, we show that hippocampal granule cells change with disease progression as measured in living, resected hippocampal tissue excised from patients with epilepsy. We show that granule cells increase excitability and shorten response latency while also enlarging in cellular volume and spine density. Single-nucleus RNA sequencing combined with simulations ascribes the changes to three conductances: BK, Cav2.2, and Kir2.1. In a network model, we show that these changes related to disease progression bring the circuit into a more excitable state, while reversing them produces a less excitable, “early-disease-like” state.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Assessing the interfacial corrosion mechanism of Inconel 617 in chloride molten salt corrosion using multi-modal advanced characterization techniques

The United States Department of Energy (DOE) has committed to expanding the domestic clean energy portfolio in response to the rising challenges of energy security in the wake of climate change. Accordingly, the construction of a series of Generation IV reactor technologies are being demonstrated, including sodium-cooled, small modular, and molten chloride fast reactors (MCFRs). To date, there are no fully qualified structural materials for constructing MCFRs. A number of commercial structural alloys have been considered for the construction of MCFRs, including alloys from the Inconel and Hastelloy series. Informed qualification of structural materials for the construction of MCFRs in the future can only be ensured by expanding the current fundamental knowledgebase of information pertaining to material performance under environmental stressors relevant to operation of the reactor, including corrosion susceptibility. The purpose of this investigation is to illustrate how a correlative multi-modal electron microscopy characterization approach, including the novel application of focused-ion beam 3D reconstruction capabilities, can elucidate the corrosion mechanism of a candidate structural material Inconel 617 for MCFR in NaCl-MgCl 2 eutectic salt at 700°C for 1,000 h. Evidence of intergranular corrosion, Ni and Fe dealloying, and Cr-O enrichment along the grain boundary, which most likely corresponds to Cr 2 O 3 , is a phenomenon that has been documented in other Ni-based superalloys exposed to chloride molten salt systems. Additional corrosion products, including the formation of insoluble MgAl 2 O 4 , within the porous network produced by the salt attack is a novel observation. In addition, Mo 3 Si 5 and τ 2 precipitates are detected in the alloy bulk and are dissolved by the salt. Furthermore, the lack of detection of design γ' precipitates in Inconel 617 after 1,000 h could indicate that the molten salt corrosion mechanism has indirectly induced a phase transformation of Al 2 TiNi (τ 2 ) and Ni 3 (Al,Ti) (γ’) phase. This investigation provides a comprehensive understanding of molten salt corrosion mechanisms in a complex material system such as a commercial structural alloy for applications in MCFRs.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Multi-modality deep learning for pulse prediction in homogeneous nonlinear systems via parametric conversion

In this Letter, we introduce FusionNet, a multi-modality deep learning framework designed to predict and analyze output pulses in high-power rare-earth-doped laser systems driving parametric conversion in homogeneous guided nonlinear media. FusionNet integrates temporal, spectral, and physical experimental conditions to model ultrafast nonlinear phenomena, including parametric nonlinear frequency conversion, self-phase modulation, and cross-phase modulation in homogeneous guided systems such as gas-filled hollow-core fibers. These systems bridge physical models with experimental data, advancing our understanding of light-guiding principles and nonlinear interactions while expediting the design and optimization of on-demand high-power, high-brightness systems. Our results demonstrate a 73% reduction in prediction error and an 83% improvement in computational efficiency compared to conventional neural networks. This work establishes a new paradigm for accelerating parametric simulations and optimizing experimental designs in high-power laser systems, with further implications for high-precision spectroscopy, quantum information science, and distributed entangled interconnects.

47 OTHER INSTRUMENTATION↗

DOE OSTI2024

Synchrotron micro-computed tomography analysis of neutron-irradiated U-Mo fuel

The three-dimensional (3D) microstructure of neutron-irradiated uranium-10 wt.% molybdenum (U-10Mo) fuel with a burn-up of 9.8 × 10 21 fissions/cm 3 was characterized using a novel, multi-modal synchrotron micro-computed tomography approach combining propagation-based phase-contrast enhanced and absorption contrast techniques. The porosity development, porosity interconnectedness, swelling, composition, local thickness of the zirconium (Zr) diffusion barrier, and the influence of the fuel–cladding interaction on the local composition and pore morphology, were uniquely determined in 3D. Two cuboids were produced using a focused ion beam-scanning electron microscope at the Zr diffusion barrier–fuel interface and in the bulk fuel. The bulk fuel sample swelled by 53.3 [+9.7/−3.1]%, while the fuel near the Zr–fuel interface swelled by 63.3 [+14.7/−7.2]%. The average local thickness of the Zr diffusion barrier decreased by 53 %, compared to the expected pre-irradiated thickness. Four pore morphology regions were identified initiating parallel to the fuel–Zr interaction region: (1) an interaction layer of suppressed porosity, (2) a layer of elongated and interconnected porosity, (3) a transition zone of low porosity, and (4) a layer of unoriented porosity representative of the bulk fuel behavior. The increase in porosity near the diffusion barrier corresponded to a higher U concentration compared to that in the bulk fuel. The interconnected porosity in the fuel near the diffusion barrier was extensive and oriented parallel to the diffusion barrier, while the bulk fuel had more compact and isolated pore networks. The interaction layer, despite having suppressed porosity, was nearly 100 wt.% U. Porosity suppression at the diffusion barrier corresponds to the expected reduction in radiation-driven diffusion of Xe at the interface despite the anticipated increase in fission product nucleation originating from a higher U concentration. In conclusion, the novel 3D insights of the porosity, swelling, and compositional variations characterized herein can improve the fidelity of fuel performance codes for proliferation-resistant fuels for research and test reactors.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2021

MULTI-MODAL global surveillance methodology for predictive and on-demand characterization of localized processes using cube satellite platforms and deep learning techniques

This paper presents the work completed towards the development of a multi-modal global surveillance methodology using cube satellite (CubeSat) platforms and novel data analysis techniques. A CubeSat system equipped with adequate sensors and data analytics capabilities can autonomously characterize various phenomena of interest on the Earth’s surface. CubeSats are advantageous over conventional satellites in certain remote monitoring applications because of their reduced construction costs (due to the availability of commercially-off-the-shelf components) and are easier to launch. The CubeSat surveillance system developed in this paper focused on phenomena of interest surrounding the nuclear fuel cycle in support of nuclear non-proliferation and emergency response. To observe the phenomena, a constellation of 3U and 6U CubeSats deployed from the ISS with adequate components was chosen. Four different sensor configurations were identified for remote sensing: panchromatic/multispectral in the visible and near-infrared spectrum, multispectral in infrared spectrum, hyperspectral in infrared spectrum, and multispectral in ultraviolet spectrum. While a panchromatic/multispectral sensor configuration has CubeSat flight heritage at the required spatial resolutions, the other three sensor types need future 3 development to meet signature and system requirements. Once each sensor onboard the CubeSat system collects data on a target of interest, the onboard computers would then apply the deep learning-based characterization methodology developed in this paper to identify phenomena. Four surrogate datasets containing representative simplified “images” were created for each sensor type to train the characterization methodology. A convolutional neural network was applied to each dataset and produced recall rates for the phenomena between 89.7% - 99.3% and precision rates between 92.3% - 99.9%. Each phenomenon’s presence probability from each network is then combined into a final characterization solution for a target area. This paper covers multiple interdisciplinary areas to develop the foundation for a CubeSat surveillance system focused on phenomena surrounding the nuclear fuel cycle.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2025

Object detection with deep learning for rare event search in the GADGET II TPC

In the pursuit of identifying rare two-particle events within the GADGET II Time Projection Chamber (TPC), this paper presents a comprehensive approach for leveraging Convolutional Neural Networks (CNNs) and various data processing methods. To address the inherent complexities of 3D TPC track reconstructions, the data is expressed in 2D projections and 1D quantities. This approach capitalizes on the diverse data modalities of the TPC, allowing for the efficient representation of the distinct features of the 3D events, with no loss in topology uniqueness. Additionally, it leverages the computational efficiency of 2D CNNs and benefits from the extensive availability of pre-trained models. Given the scarcity of real training data for the rare events of interest, simulated events are used to train the models to detect real events. To account for potential distribution shifts when predominantly depending on simulations, significant perturbations are embedded within the simulations. This produces a broad parameter space that works to account for potential physics parameter and detector response variations and uncertainties. These parameter-varied simulations are used to train sensitive 2D CNN object detectors. When combined with 1D histogram peak detection algorithms, this multi-modal detection framework is highly adept at identifying rare, two-particle events in data taken during experiment 21072 at the Facility for Rare Isotope Beams (FRIB), demonstrating a 100% recall for events of interest. Here, we present the methods and outcomes of our investigation and discuss the potential future applications of these techniques.

Convolutional neural network↗

DOE OSTI2021

Striving to translate shale physics across ten orders of magnitude: What have we learned?

Shales will play an important role in the successful transition of energy from fossil-based resources to renewables in the coming decades. Aside from being a significant source of low-carbon intensity fuels, like natural gas, they also serve as geologic seals of subsurface formations that may be used to isolate nuclear waste, sequester CO 2 , or store intermittent energy (e.g., solar hydrogen). Despite their importance, shales pose significant engineering and environmental challenges due to their nanoporous structure and extreme heterogeneity that spans at least ~10 orders of magnitude in spatial scale. Two challenges inhibit a system-level understanding: (1) the physics of fluid flow and phase behavior in shales are poorly understood due to the dominant molecular interactions between minerals and fluids under confinement, and (2) the apparent lack of scale separation that prevents a reliable (closed) description of the physics at any single scale of observation. In this review, we focus on the latter issue and discuss scale translation, which in its broadest sense is transforming data or simulations from one spatiotemporal scale to another. While effective scale translation is not exclusive to shales, but all geologic porous media, the need for it is especially acute in shales given their high degree of heterogeneity. Classical theories like homogenization, while indispensable, fail when scales are not separated. Other methods, like numerical upscaling, scale-translate in only one direction: small to large, but not the reverse, called downscaling. However, the confluence of advances in three areas are bringing challenging problems such as shales within reach: increased computational power and scalable algorithms; high-resolution imaging and multi-modal data acquisition; and machine learning to process massive amounts of data. While these advances equip geoscientists with a wide array of experimental and computational tools, no individual tool can probe the entire gamut of heterogeneity in shales. Their effective use, therefore, requires an ability to bridge between various data types obtained at different scales. The aim of this review is to present a coherent account of computational and experimental methods that may be used to achieve just that, i.e., to perform scale translation. We provide a broader definition of scale translation, one that transcends classical homogenization and upscaling methods, but is consistent with them and accommodates notions like downscaling and data translation. After a brief introduction to homogenization, we review hybrid methods, numerical upscaling and its recent extensions, multiscale computing, high-resolution imaging, and machine learning. We place particular emphasis on multiscale computing and propose an algorithmic framework to bridge between the pore (micro) and Darcy (macro) scales. Throughout the paper, we draw comparisons between the various methods and highlight their (often hidden) similarities, differences, benefits, and pitfalls. We finally conclude with two case studies on shales that exemplify some of the methods presented.

58 GEOSCIENCES↗

DOE OSTI2026

Entropy-defect synergy for dual luminescence mechanism in spinel: Time-resolved anti-counterfeiting and fingerprint visualization

Multimodal luminescent materials, while promising for anti-counterfeiting, often lack dynamic time-dependent responses and controllable spatial distribution, limiting their encryption capabilities in the spatiotemporal dimension. Here, this work presents a coordinated control strategy based on entropy and defect engineering, and uses a backpropagation (BP) neural network for material screening to successfully prepare spinel Mg 0.8 (Fe 0.04 Co 0.04 Ni 0.04 Cu 0.04 Zn 0.04 )Cr 2 O 4 (MgA 5 CO) phosphors with time-dependent dynamic luminescence behavior. This phosphor simultaneously activated the d-d transition luminescence (∼618 nm) derived from Co 2+ /Cr 3+ and the defect luminescence (∼398 nm) related to zinc vacancies (V Zn ) in a single-phase solid solution. The phosphor exhibits a time-dependent color evolution from pink to purple under fixed-wavelength excitation, due to the different excited-state dynamics and decay lifetimes associated with the d-d transition and defect luminescence. Structural characterization and spectral analysis confirmed the existence of V Zn and its significant role in defect luminescence process. The fluorescent and dynamic luminescent properties of entropy-based spinel oxide enable its use in advanced anti-counterfeiting applications like fingerprint recognition and color-changing dedicated anti-counterfeiting mark, showing promise in high-end and time-dynamic anti-counterfeiting fields. This research not only developed a new type of fluorescent dynamic anti-counterfeiting material, but also provided a new idea for constructing advanced optical functional materials with multiple luminescence mechanisms.

Defect project↗

DOE OSTI2023

Iterative multi-task learning and inference from seismic images

Seismic interpretation aims to extract quantitative and interpretable attributes from a seismic image produced using some migration method to inform characteristics of a subsurface reservoir or target of interest. Current paradigms for computing seismic attributes mostly rely on single-task algorithms. We develop an iterative, multi-task machine learning method to learn and infer multiple attributes from a seismic image. This method is composed of two stages: a multi-task inference stage and a multi-modal, multi-task refinement stage. The basic mechanism of this method is that we train a multi-task inference neural network (NN) to estimate a set of attributes, including a relative geological time (RGT), a denoised higher-resolution (DHR) seismic image, and multiple fault attributes (including probability, dip, and strike), from a low-resolution, noisy seismic image; then we input the inferred attributes to a multi-task refinement NN to enhance the raw inference results iteratively. The two multi-task NNs are trained separately based on synthetic seismic images and associated attributes generated by a geological modeling algorithm. The software we intend to release is a PyTorch implementation of this multi-task learning method for both 2D and 3D cases along with scripts to run the training/validation. The algorithm and software can be a useful tool for automatic seismic interpretation.

Gao, Kai↗