Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep regression”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

On the use of neural networks for energy reconstruction in high-granularity calorimeters

We contrasted the performance of deep neural networks — Convolutional Neural Network (CNN) and Graph Neural Network (GNN) — to current state of the art energy regression methods in a finely 3D-segmented calorimeter simulated by GEANT4. This comparative benchmark gives us some insight to assess the particular latent signals neural network methods exploit to achieve superior resolution. A CNN trained solely on a pure sample of pions achieved substantial improvement in the energy resolution for both single pions and jets over the conventional approaches. It maintained good performance for electron and photon reconstruction. We also used the Graph Neural Network (GNN) with edge convolution to assess the importance of timing information in the shower development for improved energy reconstruction. We implement a simple simulation based correction to the energy sum derived from the fraction of energy deposited in the electromagnetic shower component. This serves as an approximate dual-readout analogue for our benchmark comparison. Although this study does not include the simulation of detector effects, such as electronic noise, the margin of improvement seems robust enough to suggest these benefits will endure in real-world application. We also find reason to infer that the CNN/GNN methods leverage latent features that concur with our current understanding of the physics of calorimeter measurement.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Understanding and Verifying Neural Networks

Deep Neural Networks (DNNs) have gained immense popularity in recent times and have widespread use in applications such as image classification, sentiment analysis, speech recognition and also in safety-critical applications such as autonomous driving. However, they suffer limitations such as lack of explainability and robustness which raise safety and security concerns in their usage. Further, the complex structure and large input spaces of DNNs act as an impediment to thorough verification and testing. The SafeDNN project at the Robust Software Engineering (RSE) group at NASA aims at exploring techniques to ensure that systems that use deep neural networks are safe, robust and interpretable. In this talk, I will be presenting our technique Prophecy that automatically infers formal properties of deep neural network models. The tool extracts patterns based on neuron activations as preconditions that imply certain desirable output properties of the model. I would be highlighting case studies that use Prophecy in obtaining explanations for network decisions, understanding correct and incorrect behavior, providing formal guarantees wrt safety and robustness, and debugging neural network models. We have applied the tool on image classification networks, neural network controllers providing turn advisories in unmanned aircrafts, regression models used for autonomous center-line tracking in aircrafts and neural network object detectors

Deep Neural Networks↗

Discovering a reaction–diffusion model for Alzheimer’s disease by combining PINNs with symbolic regression

Misfolded tau proteins play a critical role in the progression and pathology of Alzheimer's disease. Recent studies suggest that the spatio-temporal pattern of misfolded tau follows a reaction-diffusion type equation. However, the precise mathematical model and parameters that characterize the progression of misfolded protein across the brain remain incompletely understood. Here, we use deep learning and artificial intelligence to discover a mathematical model for the progression of Alzheimer's disease using longitudinal tau positron emission tomography from the Alzheimer's Disease Neuroimaging Initiative database. Specifically, we integrate physics informed neural networks (PINNs) and symbolic regression to discover a reaction-diffusion type partial differential equation for tau protein misfolding and spreading. First, we demonstrate the potential of our model and parameter discovery on synthetic data. Then, we apply our method to discover the best model and parameters to explain tau imaging data from 46 individuals who are likely to develop Alzheimer's disease and 30 healthy controls. Our symbolic regression discovers different misfolding models f(c) for two groups, with a faster misfolding for the Alzheimer's group, f(c) = 0.23c 3 – 1.34c 2 + 1.11c, than for the healthy control group, f(c) = –c 3 + 0.62c 2 + 0.39c. Our results suggest that PINNs, supplemented by symbolic regression, can discover a reaction-diffusion type model to explain misfolded tau protein concentrations in Alzheimer's disease. Furthermore, we expect our study to be the starting point for a more holistic analysis to provide image-based technologies for early diagnosis, and ideally early treatment of neurodegeneration in Alzheimer's disease and possibly other misfolding-protein based neurodegenerative disorders.

60 APPLIED LIFE SCIENCES↗

Machine learning Frenkel Hamiltonian parameters to accelerate simulations of exciton dynamics

In this manuscript, we develop multiple machine learning (ML) models to accelerate a scheme for parameterizing site-based models of exciton dynamics from all-atom configurations of condensed phase sexithiophene systems. This scheme encodes the details of a system’s specific molecular morphology in the correlated distributions of model parameters through the analysis of many single-molecule excited-state electronic-structure calculations. These calculations yield excitation energies for each molecule in the system and the network of pair-wise intermolecular electronic couplings. In this work, we demonstrate that the excitation energies can be accurately predicted using a kernel ridge regression (KRR) model with Coulomb matrix featurization. We present two ML models for predicting intermolecular couplings. The first one utilizes a deep neural network and bi-molecular featurization to predict the coupling directly, which we find to perform poorly. The second one utilizes a KRR model to predict unimolecular transition densities, which can subsequently be analyzed to compute the coupling. We find that the latter approach performs excellently, indicating that an effective, generalizable strategy for predicting simple bimolecular properties is through the indirect application of ML to predict higher-order unimolecular properties. Such an approach necessitates a much smaller feature space and can incorporate the insight of well-established molecular physics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Information Retrieval from SAGE II and MFRSR Multi-Spectral Extinction Measurements

Direct beam spectral extinction measurements of solar radiation contain important information on atmospheric composition in a form that is essentially free from multiple scattering contributions that otherwise tend to complicate the data analysis and information retrieval. Such direct beam extinction measurements are available from the solar occultation satellite-based measurements made by the Stratospheric and Aerosol Gas Experiment (SAGE II) instrument and by ground-based Multi-Filter Shadowband Radiometers (MFRSRs). The SAGE II data provide cross-sectional slices of the atmosphere twice per orbit at seven wavelengths between 385 and 1020 nm with approximately 1 km vertical resolution, while the MFRSR data provide atmospheric column measurements at six wavelengths between 415 and 940 nm but at one minute time intervals. We apply the same retrieval technique of simultaneous least-squares fit to the observed spectral extinctions to retrieve aerosol optical depth, effective radius and variance, and ozone, nitrogen dioxide, and water vapor amounts from the SAGE II and MFRSR measurements. The retrieval technique utilizes a physical model approach based on laboratory measurements of ozone and nitrogen dioxide extinction, line-by-line and numerical k-distribution calculations for water vapor absorption, and Mie scattering constraints on aerosol spectral extinction properties. The SAGE II measurements have the advantage of being self-calibrating in that deep space provides an effective zero point for the relative spectral extinctions. The MFRSR measurements require periodic clear-day Langley regression calibration events to maintain accurate knowledge of instrument calibration.

Lacis, Andrew A.↗

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of 10\textsuperscript{5}, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Domain Adaptation for Measurements of Strong Gravitational Lenses

Upcoming surveys are predicted to discover galaxy-scale strong lenses on the order of $10^5$, making deep learning methods necessary in lensing data analysis. Currently, there is insufficient real lensing data to train deep learning algorithms, but the alternative of training only on simulated data results in poor performance on real data. Domain Adaptation may be able to bridge the gap between simulated and real datasets. We utilize domain adaptation for the estimation of Einstein radius ($\Theta_E$) in simulated galaxy-scale gravitational lensing images with different levels of observational realism. We evaluate two domain adaptation techniques - Domain Adversarial Neural Networks (DANN) and Maximum Mean Discrepancy (MMD). We train on a source domain of simulated lenses and apply it to a target domain of lenses simulated to emulate noise conditions in the Dark Energy Survey (DES). We show that both domain adaptation techniques can significantly improve the model performance on the more complex target domain dataset. This work is the first application of domain adaptation for a regression task in strong lensing imaging analysis. Our results show the potential of using domain adaptation to perform analysis of future survey data with a deep neural network trained on simulated data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Three-dimensional flow contrast imaging of deep tissue using noncontact diffuse correlation tomography

This study extended our recently developed noncontact diffuse correlation spectroscopy flowmetry system into noncontact diffuse correlation tomography (ncDCT) for three-dimensional (3-D) flow imaging of deep tissue. A linear array of 15 photodetectors and two laser sources connected to a mobile lens-focusing system enabled automatic and noncontact scanning of flow in a region of interest. These boundary measurements were combined with a finite element framework for DCT image reconstruction implemented into an existing software package. This technique was tested in computer simulations and using a tissue-like phantom with anomaly flow contrast design. The cylindrical tube-shaped anomaly was clearly reconstructed in both simulation and phantom. Recovered and assigned flow contrast changes in anomaly were found to be highly correlated: regression slope = 1.00, R{sup 2} = 1.00, and p < 10{sup −5} in simulation and regression slope ≥ 0.97, R{sup 2} ≥ 0.96, and p < 10{sup −3} in phantom. These results exhibit promise of our ncDCT technique for 3-D imaging of deep tissue blood flow heterogeneities.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Deep ugrizY imaging and DEEP2/3 spectroscopy: a photometric redshift testbed for LSST and public release of data from the DEEP3 Galaxy Redshift Survey

ABSTRACT We present catalogues of calibrated photometry and spectroscopic redshifts in the Extended Groth Strip, intended for studies of photometric redshifts (photo-z’s). The data includes ugriz photometry from Canada–France–Hawaii Telescope Legacy Survey (CFHTLS) and Y-band photometry from the Subaru Suprime camera, as well as spectroscopic redshifts from the DEEP2, DEEP3, and 3D-HST surveys. These catalogues incorporate corrections to produce effectively matched-aperture photometry across all bands, based upon object size information available in the catalogue and Moffat profile point spread function fits. We test this catalogue with a simple machine learning-based photometric redshift algorithm based upon Random Forest regression, and find that the corrected aperture photometry leads to significant improvement in photo-z accuracy compared to the original SExtractor catalogues from CFHTLS and Subaru. The deep ugrizY photometry and spectroscopic redshifts are well suited for empirical tests of photometric redshift algorithms for LSST. The resulting catalogues are publicly available at http://d-scholarship.pitt.edu/36064/. We include a basic summary of the strategy of the DEEP3 Galaxy Redshift Survey to accompany the recent public release of DEEP3 data.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep Learning for Multigroup Cross-Section Representation in Two-Step Core Calculations

Here we investigate using deep learning, a type of machine-learning algorithm employing multiple layers of artificial neurons, for the mathematical representation of multigroup cross sections for use in the Griffin reactor multiphysics code for two-step deterministic neutronics calculations. A three-dimensional fuel element typical of a high-temperature gas reactor as well as a two-dimensional sodium-cooled fast reactor lattice are modeled using the Serpent Monte Carlo code, and multigroup macroscopic cross sections are generated for various state parameters to produce a training data set and a separate validation data set. A fully connected, feedforward neural network is trained using the open-source PyTorch machine-learning framework, and its accuracy is compared against the standard piecewise linear interpolation model. Additionally, we provide in this work a generic technique for propagating the cross-section model errors up to the k eff using sensitivity coefficients with the first-order uncertainty propagation rule. Quantifying the eigenvalue error due to the cross-section regression errors is especially practical for appropriately selecting the mathematical representation of the cross sections. We demonstrate that the artificial neural network model produces lower errors and therefore enables better accuracy relative to the piecewise linear model when the cross sections exhibit nonlinear dependencies; especially when a coarse grid is employed, where the errors can be halved by the artificial neural network. However, for linearly dependent multigroup cross sections as found for the sodium-cooled fast reactor case, a simpler linear regression outperforms deeper networks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Smart Methane Emission Detection System Development (Final Report)

Working with the Department of Energy's National Energy Technology Laboratory, Southwest Research Institute® (SwRI®) developed a system to identify methane leaks reliably, accurately, and autonomously at critical midstream sections of the natural gas distribution network in real-time for the purpose of mitigating methane emissions using Optical Gas Imaging (OGI) cameras. SwRI's Smart Leak Detection – Methane (SLED/M) adds a high degree of automation to the process of methane leak detection to minimize sources of human error, minimize response time to a leak event, and maximize midstream visibility. Furthermore, SwRI has been working towards integrating Quantitative OGI (QOGI) capabilities into this existing technology. By leveraging Deep Learning, SwRI now has the capability to estimate fugitive emission leak rates quickly and reliably, which allows operators to detect emissions, quantify leak rate, prioritize repairs, and validate the repairs in a single instrument. The next generation QOGI technology leverages the same cameras used in Leak Detection and Repair (LDAR) programs, with improvements in safety and speed for traditional quantification-based repairs, ultimately leading to less overhead cost for the operators. The goals for this research were to develop two types of models with the following goals: Run in real-time on the edge (≥ 12 Hz), Classification: Achieve less than 5% false positive detection, Classification: Achieve ≥ 95% methane plume detection rate, Regression: achieve ≤ 10 standard cubic feet per hour (scfh) prediction > 70% of the time. In order to achieve these results, multiple infrared (IR) and other sensors were investigated in tandem with the midwave IR (MWIR) OGI to provide additional information to train the underlying models. Information on atmospheric conditions including humidity, temperature, pressure, and solar radiation was provided by a weather station. Several machine learning and deep learning architectures and methods, including looking at quantized classification networks and regressions networks, were explored. As further data was collected, curated, and labeled, it allowed for more refined regressive networks to be adequately trained, leading to better insight into the true flow rates being observed. An important valuable deliverable of this research effort was the development of an advanced network which underwent multiple iterations capable of giving a continuous output. The current network has a predicted mean average percentage error (MAPE) of 12.3% just outside our target goal of 10.00%, but an accuracy of 97.78% at ±50 scfh, well within the overall goal for the Department of Energy (DOE) program. Upon closer inspection, it was observed that more than 10% of datapoints contributing to the MAPE predictions were the result of low flow rate predictions and are beyond the sensitivity of instrument measurement as a result of normal operational variation and noise.

03 NATURAL GAS↗

Smart Methane Emission Detection System Development (Final Report)

Working with the Department of Energy’s National Energy Technology Laboratory, Southwest Research Institute® (SwRI®) developed a system to identify methane leaks reliably, accurately, and autonomously at critical midstream sections of the natural gas distribution network in real- time for the purpose of mitigating methane emissions using Optical Gas Imaging (OGI) cameras. SwRI’s Smart Leak Detection – Methane (SLED/M) adds a high degree of automation to the process of methane leak detection to minimize sources of human error, minimize response time to a leak event, and maximize midstream visibility. Furthermore, SwRI has been working towards integrating Quantitative OGI (QOGI) capabilities into this existing technology. By leveraging Deep Learning, SwRI now has the capability to estimate fugitive emission leak rates quickly and reliably, which allows operators to detect emissions, quantify leak rate, prioritize repairs, and validate the repairs in a single instrument. The next generation QOGI technology leverages the same cameras used in Leak Detection and Repair (LDAR) programs, with improvements in safety and speed for traditional quantification-based repairs, ultimately leading to less overhead cost for the operators. The goals for this research were to develop two types of models with the following goals: 1. Run in real-time on the edge (≥ 12 Hz) 2. Classification: Achieve less than 5% false positive detection 3. Classification: Achieve ≥ 95% methane plume detection rate 4. Regression: achieve ≤ 10 standard cubic feet per hour (scfh) prediction > 70% of the time In order to achieve these results, multiple infrared (IR) and other sensors were investigated in tandem with the midwave IR (MWIR) OGI to provide additional information to train the underlying models. Information on atmospheric conditions including humidity, temperature, pressure, and solar radiation was provided by a weather station. Several machine learning and deep learning architectures and methods, including looking at quantized classification networks and regressions networks, were explored. As further data was collected, curated, and labeled, it allowed for more refined regressive networks to be adequately trained, leading to better insight into the true flow rates being observed. An important valuable deliverable of this research effort was the development of an advanced network which underwent multiple iterations capable of giving a continuous output. The current network has a predicted mean average percentage error (MAPE) of 12.3% just outside our target goal of 10.00%, but an accuracy of 97.78% at ±50 scfh, well within the overall goal for the Department of Energy (DOE) program. Upon closer inspection, it was observed that more than 10% of datapoints contributing to the MAPE predictions were the result of low flow rate predictions and are beyond the sensitivity of instrument measurement as a result of normal operational variation and noise.

03 NATURAL GAS↗

Physics-Informed Machine Learning with Application to Solid Oxide Fuel Cell System Modeling and Optimization

Traditional data-driven machine learning (ML) techniques were combined with the physics-based SOFC-MP model toward improving SOFC system-level performance prediction. Four different physics-informed ML methods were proposed to demonstrate better prediction accuracy compared with traditional ML methods. First, it was preferred that the existing Kriging-based surrogate model not be entirely abandoned, so errors between Kriging regression prediction and true solution are learned using ML. The goal was to find the pattern of the error distribution as a function of model input so discrepancies between the Kriging prediction and true solution could be reduced by adding the additional ML-trained error term to the existing prediction. Second, deep neural networks (DNN) coupled with the mass balance model (MBM) have been developed to significantly decrease the reduced order model (ROM) prediction error with fewer training data compared to the traditional Kriging-based ROM and traditional DNN regression approaches. Third, DNN regression coupled with a neural networks (NN) classifier and MBM was developed. It provides superior performance on classifying the physically operational conditions for natural gas fuel cell (NGFC) than traditional classification approaches. Finally, two kinds of DNN transferring were studied and tested. One was transferring DNN for low-fidelity model data to a high-fidelity model. The second one was transferring DNN from state-of-the-art NGFC to advanced NGFC. These physic-informed ML frameworks can serve as a solid foundation for estimating U.S. Department of Energy-targeted fuel cell performance and exhibit great potential to be employed to other engineering applications.

30 DIRECT ENERGY CONVERSION↗

Modeling the Echelle Spectra Continuum with Alpha Shapes and Local Regression Fitting

Continuum normalization of echelle spectra is an important data analysis step that is difficult to automate. Polynomial fitting requires a reasonably high-order model to follow the steep slope of the blaze function. However, in the presence of deep spectral lines, a high-order polynomial fit can result in ripples in the normalized continuum that increase errors in spectral analysis. Here, we present two algorithms for flattening the spectrum continuum. The Alpha-shape Fitting to Spectrum algorithm is completely data driven, using an alpha shape to obtain an initial estimate of the blaze function. The Alpha-shape and Lab Source Fitting to Spectrum algorithm incorporates a continuum constraint from a laboratory source reference spectrum for the blaze function estimation. These algorithms are tested on a simulated spectrum, where we demonstrate improved normalization compared to polynomial regression for continuum fitting. We show an additional application, using the algorithms for mitigation of spatially correlated quantum efficiency variations and fringing in the charge-coupled device detector of the EXtreme PREcision Spectrometer.

79 ASTRONOMY AND ASTROPHYSICS↗

Deep Learning Classification of Cheatgrass Invasion in the Western United States Using Biophysical and Remote Sensing Data

Cheatgrass (Bromus tectorum) invasion is driving an emerging cycle of increased fire frequency and irreversible loss of wildlife habitat in the western US. Yet, detailed spatial information about its occurrence is still lacking for much of its presumably invaded range. Deep learning (DL) has demonstrated success for remote sensing applications but is less tested on more challenging tasks like identifying biological invasions using sub-pixel phenomena. We compare two DL architectures and the more conventional Random Forest and Logistic Regression methods to improve upon a previous effort to map cheatgrass occurrence at >2% canopy cover. High-dimensional sets of biophysical, MODIS, and Landsat-7 ETM+ predictor variables are also compared to evaluate different multi-modal data strategies. All model configurations improved results relative to the case study and accuracy generally improved by combining data from both sensors with biophysical data. Cheatgrass occurrence is mapped at 30 m ground sample distance (GSD) with an estimated 78.1% accuracy, compared to 250-m GSD and 71% map accuracy in the case study. Furthermore, DL is shown to be competitive with well-established machine learning methods in a limited data regime, suggesting it can be an effective tool for mapping biological invasions and more broadly for multi-modal remote sensing applications.

54 ENVIRONMENTAL SCIENCES↗

Aerosol and cloud microphysics covariability in the northeast Pacific boundary layer estimated with ship-based and satellite remote sensing observations: NE Pacific Aerosol-Cloud Interactions

Ship measurements collected over the northeast Pacific along transects between the port of Los Angeles (33.7°N, 118.2°W) and Honolulu (21.3°N, 157.8°W) during May to August 2013 were utilized to investigate the covariability between marine low cloud microphysical and aerosol properties. Ship-based retrievals of cloud optical depth (τ) from a Sun photometer and liquid water path (LWP) from a microwave radiometer were combined to derive cloud droplet number concentration Nd and compute a cloud-aerosol interaction (ACI) metric defined as ACICCN = ∂ ln(Nd)/∂ ln(CCN), with CCN denoting the cloud condensation nuclei concentration measured at 0.4% (CCN0.4) and 0.3% (CCN0.3) supersaturation. Analysis of CCN0.4, accumulation mode aerosol concentration (Na), and extinction coefficient (σext) indicates that Na and σext can be used as CCN0.4 proxies for estimating ACI. ACICCN derived from 10 min averaged Nd and CCN0.4 and CCN0.3, and CCN0.4 regressions using Na and σext, produce high ACICCN: near 1.0, that is, a fractional change in aerosols is associated with an equivalent fractional change in Nd. ACICCN computed in deep boundary layers was small (ACICCN = 0.60), indicating that surface aerosol measurements inadequately represent the aerosol variability below clouds. Satellite cloud retrievals from MODerate-resolution Imaging Spectroradiometer and GOES-15 data were compared against ship-based retrievals and further analyzed to compute a satellite-based ACICCN. Satellite data correlated well with their ship-based counterparts with linear correlation coefficients equal to or greater than 0.78. Combined satellite Nd and ship-based CCN0.4 and Na yielded a maximum ACICCN = 0.88–0.92, a value slightly less than the ship-based ACICCN, but still consistent with aircraft-based studies in the eastern Pacific.

Painemal, David↗

Deep machine learning-assisted multiphoton microscopy to reduce light exposure and expedite imaging

Two-photon excitation fluorescence (2PEF) allows imaging of tissue up to about one millimeter in thickness. Typically, reducing fluorescence excitation exposure reduces the quality of the image. However, using deep learning super resolution techniques, these low-resolution images can be converted to high-resolution images. This work explores improving human tissue imaging by applying deep learning to maximize image quality while reducing fluorescence excitation exposure. We analyze two methods: a method based on U-Net, and a patch-based regression method. Both methods are evaluated on a skin dataset and an eye dataset. The eye dataset includes 1200 paired high power and low power images of retinal organoids. The skin dataset contains multiple frames of each sample of human skin. High-resolution images were formed by averaging 70 frames for each sample and low-resolution images were formed by averaging the first 7 and 15 frames for each sample. The skin dataset includes 550 images for each of the resolution levels. We track two measures of performance for the two methods: mean squared error (MSE) and structural similarity index measure (SSIM). For the eye dataset, the patches method achieves an average MSE of 27,611 compared to 146,855 for the U-Net method, and an average SSIM of 0.636 compared to 0.607 for the U-Net method. For the skin dataset, the patches method achieves an average MSE of 3.768 compared to 4.032 for the U-Net method, and an average SSIM of 0.824 compared to 0.783 for the U-Net method. Despite better performance on image quality, the patches method is worse than the U-Net method when comparing the speed of prediction, taking 303 seconds to predict one image compared to less than one second for the U-Net method.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Deep Multimodal Representation Learning Framework for Accurate Molecular Properties Prediction

Drug discovery is a complex and challenging process, requiring the optimization of candidate compounds to identify those with the potential to become safe and effective drugs. Predicting molecular properties is an indispensable step in the drug discovery pipeline. Traditionally, this process is costly and time-intensive, involving multiple rounds of experiments and clinical trials, rendering it impractical for every candidate compound. Deep learning techniques have emerged as a promising approach to drug discovery to reduce the cost and time required to identify novel drugs. However, prevalent research in deep learning models focused on predicting molecular properties has primarily fixated on single-modal models, which utilize a single modality of data, neglecting the potential benefits of combining different data modalities. To overcome this limitation, we introduce MRL-Mol: a deep \textbf{M}ultimodal \textbf{R}epresentation \textbf{L}earning framework for accurate \textbf{Mol}ecular properties prediction. MRL-Mol harnesses three data modalities: sequence, graph, and image, augmenting the depth of comprehension. Leveraging a large-scale unlabeled dataset~($\sim$1M unique molecules), we pretrain MRL-Mol to extract inter- and intra-modal information. Our study demonstrates the superior performance of MRL-Mol in predicting molecular properties across six benchmark datasets, including both classification and regression tasks. Notably, MRL-Mol outperforms other state-of-the-art molecular properties prediction models. These findings suggest that by combining information from multiple data modalities, MRL-Mol can comprehend molecules better than single-modal deep learning models and identify molecular properties with better accuracy.

Yang, Yuxin↗