SEARCH · Search NASA

Results for “deep metric learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

192 records · Page 11

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Reinforcement Learning as a Parsimonious Alternative to Prediction Cascades: A Case Study on Image Segmentation

Deep learning architectures have achieved state-of-the-art (SOTA) performance on computer vision tasks such as object detection and image segmentation. This may be attributed to the use of over-parameterized, monolithic deep learning architectures executed on large datasets. Although such large architectures lead to increased accuracy, this is usually accompanied by a larger increase in computation and memory requirements during inference. While this is a non-issue in traditional machine learning (ML) pipelines, the recent confluence of machine learning and fields like the Internet of Things (IoT) has rendered such large architectures infeasible for execution in low-resource settings. For some datasets, large monolithic pipelines may be overkill for simpler inputs. To address this problem, previous efforts have proposed decision cascades where inputs are passed through models of increasing complexity until the desired performance is achieved. However, we argue that cascaded prediction leads to sub-optimal throughput and increased computational cost due to wasteful intermediate computations. To address this, we propose PaSeR (Parsimonious Segmentation with Reinforcement Learning) a non-cascading, cost-aware learning pipeline as an efficient alternative to cascaded decision architectures. Through experimental evaluation on both real-world and standard datasets, we demonstrate that PaSeR achieves better accuracy while minimizing computational cost relative to cascaded models. Further, we introduce a new metric IoU/GigaFlop to evaluate the balance between cost and performance. On the real-world task of battery material phase segmentation, PaSeR yields 179% improvement over SOTA MatPhase model and a 196% improvement over IDK Cascades under the IoU/GigaFlop metric. We also demonstrate PaSeR’s adaptability to complementary models trained on a noisy MNIST dataset, where it outperforms all baselines on IoU/GigaFlop by an average of 44%.

Srikshan, Bharat↗

DOE OSTI2020

Neural networks for modeling and control of particle accelerators

Charged particle accelerators support a wide variety of scientific, industrial, and medical applications. They range in scale and complexity from systems with just a few components for beam acceleration and manipulation, to large scientific user facilities that span many kilometers and have hundreds-to-thousands of individually-controllable components. Specific operational requirements must be met by adjusting the many controllable variables of the accelerator. Meeting these requirements can be challenging, both in terms of the ability to achieve specific beam quality metrics in a reliable fashion and in terms of the time needed to set up and maintain the optimal operating conditions. One avenue toward addressing this challenge is to incorporate techniques from the fields of machine learning (ML) and artificial intelligence (AI) into the way particle accelerators are modeled and controlled. While many promising approaches within AI/ML could be used for particle accelerators, this dissertation focuses on approaches based on neural networks. Neural networks are particularly well-suited to modeling, control, and diagnostic analysis of nonlinear systems, as well as systems with large parameter spaces. They are also very appealing for their ability to process high-dimensional data types, such as images and time series (both of which are ubiquitous in particle accelerators). In this work, key studies that demonstrated the potential utility of modern neural network-based approaches to modeling and control of particle accelerators are presented. The context for this work is important: at the start of this work in 2012, there was little interest in AI/ML in the particle accelerator community, and many of the advances in neural networks and deep learning that enabled its present success had not yet been made at that time. As such, this work was both an exploration of possible application areas and a generator of proof-of-concept demonstrations in these areas.

43 PARTICLE ACCELERATORS↗

DOE OSTI2021

Development of a Deep Learning Model for Predicting the Drag Coefficients of Spherical and Non-Spherical Particles,

There is yet to be a well-established drag model for non-spherical particles required in a particle-laden flow that could cover a wide range of sphericities. This talk will explore the development of a general drag model for non-spherical particles by applying deep learning using available experimental data available in the literature. The integration of several raw experimental measurements from different sources and research directions allows the training of robust Artificial Intelligence and Machine Learning (ML) models. Neural networks are an ML approach inspired by the inner biological workings of the brain. This work aims to develop a Deep Neural Network (DNN) that predicts drag coefficient values with the ability to adapt appropriately to unseen data. Given the limited number of data points available and the variance found within the data collected from various sources, challenges may arise when looking to train the model. Our study tests and implements various model regularization techniques and assesses different loss and activation functions for the proposed DNN. The proposed model considers a broader range of features other than sphericity and Reynold number. These features include density ratio, solid volume fraction, lengthwise and crosswise sphericity, and more. Furthermore, we present the features that play a significant role in predicting different drag coefficients through feature importance. Within the investigated parameter ranges in this study, the following conclusions can be achieved and summarized below: • An improved drag coefficient model can be developed by considering more features such as, aspect ratio, lengthwise sphericity, crosswise sphericity, and density ratio. • DNN model can predict better results compared to traditional methods using MAE metric. • The proposed model addresses data challenges such as limited data and extreme data points through expanded feature-set and regularization. • Three major features that mostly affect the drag coefficient were identified from a feature importance analysis.

Presa-Reyes, Maria↗

DOE OSTI2024

Risk-Aware Framework Development for Disruption Prediction: Alcator C-Mod and DIII-D Survival Analysis

Abstract Survival regression models can achieve longer warning times at similar receiver operating characteristic performance than previously investigated models. Survival regression models are also shown to predict the time until a disruption will occur with lower error than other predictors. Time-to-event predictions from time-series data can be obtained with a survival analysis statistical framework, and there have been many tools developed for this task which we aim to apply to disruption prediction. Using the open-source Auton-Survival package we have implemented disruption predictors with the survival regression models Cox Proportional Hazards, Deep Cox Proportional Hazards, and Deep Survival Machines. To compare with previous work, we also include predictors using a Random Forest binary classifier, and a conditional Kaplan-Meier formalism. We benchmarked the performance of these five predictors using experimental data from the Alcator C-Mod and DIII-D tokamaks by simulating alarms on each individual shot. We find that developing machine-relevant metrics to evaluate models is an important area for future work. While this study finds cases where disruptive conditions are not predicted, there are instances where the desired outcome is produced. Giving the plasma control system the expected time-to-disruption will allow it to determine the optimal actuator response in real time to minimize risk of damage to the device.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2020

Integrated Mid-Continent Stacked Carbon Storage Hub Project Phase II (Final Summary Report)

The Phase II Integrated Midcontinent Stacked Carbon Storage Hub (IMSCS-HUB) is part of the Carbon Storage Assurance Facility Enterprise (CarbonSAFE) established by the United States Department of Energy (DOE) National Energy Technology Laboratory (NETL). CarbonSAFE is phased to support the development of commercial-scale (50 million metric tonnes [Mt] over a 30-year period) carbon capture, utilization, and storage (CCUS) in the United States. The IMSCS-HUB study area comprises carbon dioxide (CO 2 ) sources in Iowa, Kansas, and Nebraska (the source corridor), and CO 2 sinks in Kansas and Nebraska (the storage corridor), representing the first large-scale project for the Midcontinent region. The stacked storage corridor is characterized by alternating sequences of deep saline formations, oil-bearing reservoirs, shale, and evaporite units that are conducive to vertically stacked CO 2 injection for geologic storage and enhanced oil recovery (EOR). Three sites within the IMSCS-HUB stacked storage corridor were evaluated in Phase II for commercial CCUS feasibility: one in southwest-central Nebraska, Sleepy Hollow Field (SHF), a second in southwestern Nebraska near Madrid (Madrid), and a third in southwestern Kansas, the Patterson Site (composed of the Patterson, Heinitz, Hartland, and Oslo fields). In Phase II, the team assessed the feasibility of storage complexes at the potential storage sites in Nebraska and Kansas to support a commercial-scale storage hub that integrates proven CO 2 capture technology and transport from nearby ethanol sources. Building on lessons learned from the DOE-NETL Regional Carbon Sequestration Partnerships (RCSPs), the Project Team has identified a clear strategy to meet DOE’s 2025 objective of commercial carbon capture and storage (CCS) implementation by developing a CO 2 market and infrastructure that relies on multiple ethanol-based CO 2 sources in the short term and the incorporation of multiple coal-fired power plant CO 2 sources when commercial capture is economically viable. The team also leveraged the updated 45Q tax credit to develop capture and transport infrastructure. Commercial-scale CCUS is feasible at two candidate storage sites studied, the Madrid, Nebraska Site and the Patterson Site in Kearny County, Kansas. The Sleepy Hollow Field in Nebraska was found to be an attractive candidate for stacked storage with CO 2 -EOR (Battelle 2020e). Outreach efforts facilitated engagement from industry, government, and research sectors (Battelle and GPI, 2020) and an outreach plan for future phases of the project was developed to address issues that are of concern in the IMSCS-HUB project area (Battelle, 2020f). All components of a CCUS project were determined to be feasible in the IMSCS-HUB region and Risk Mitigation Plan was developed and includes strategies to mitigate risks associated with each project component (Battelle, 2020j). A roadmap was developed to obtain the required UIC permits for an integrated CCUS project (Battelle, 2020k). The regional storage resource characterization demonstrated significant opportunity for commercial-scale projects in the IMSCS-HUB storage corridor with 577.4 Mt of stacked CO 2 storage capacity and the potential to produce 181.9 MMbbls of oil via EOR across 17 individual storage areas (Battelle and ARI, 2020). The pipeline assessment study found viable pipeline routes that connected 45Q-eligible ethanol plants, coal fired power plants, and other sources in the IMSCS-HUB corridor. The comprehensive results of subsurface characterization, modeling efforts, outreach assessment, and regulatory analysis from were integrated to develop a Detailed Commercial Development Plan for the IMSCS-HUB (Battelle, 2020n). Commercialization efforts will involve obtaining Class VI UIC permits, establishing and finalizing the pipeline route, and evaluating capture projects at participating CO 2 sources. Phases I and II of the IMSCS-HUB CarbonSAFE provide a strong foundation for safely, efficiently, and cost-effectively characterizing and permitting commercial-scale project sites in the region. The plan for implementation of commercial-scale CCUS projects in the IMSCS-HUB is aligned with the objectives of CarbonSAFE Phase III: Site Characterization and CO 2 Capture Assessment.

20 FOSSIL-FUELED POWER PLANTS↗

DOE OSTI2022

Inter-Domain Fusion for Enhanced Intrusion Detection in Power Systems: An Evidence Theoretic and Meta-Heuristic Approach

False alerts due to misconfigured or compromised intrusion detection systems (IDS) in industrial control system (ICS) networks can lead to severe economic and operational damage. However, research using deep learning to reduce false alerts often requires the physical and cyber sensor data to be trustworthy. Implicit trust is a major problem for artificial intelligence or machine learning (AI/ML) in cyber-physical system (CPS) security, because when these solutions are most urgently needed is also when they are most at risk (e.g., during an attack). To address this, the Inter-Domain Evidence theoretic Approach for Inference (IDEA-I) is proposed that reframes the detection problem as how to make good decisions given uncertainty. Specifically, an evidence theoretic approach leveraging Dempster–Shafer (DS) combination rules and their variants is proposed for reducing false alerts. A multi-hypothesis mass function model is designed that leverages probability scores obtained from supervised-learning classifiers. Using this model, a location-cum-domain-based fusion framework is proposed to evaluate the detector’s performance using disjunctive, conjunctive, and cautious conjunctive rules. The approach is demonstrated in a cyber-physical power system testbed, and the classifiers are trained with datasets from Man-In-The-Middle attack emulation in a large-scale synthetic electric grid. For evaluating the performance, we consider plausibility, belief, pignistic, and general Bayesian theorem-based metrics as decision functions. To improve the performance, a multi-objective-based genetic algorithm is proposed for feature selection considering the decision metrics as the fitness function. Finally, we present a software application to evaluate the DS fusion approaches with different parameters and architectures.

42 ENGINEERING↗

DOE OSTI2022

Stable parallel training of Wasserstein conditional generative adversarial neural networks

In this work, we propose a stable, parallel approach to train Wasserstein conditional generative adversarial neural networks (W-CGANs) under the constraint of a fixed computational budget. Differently from previous distributed GANs training techniques, our approach avoids inter-process communications, reduces the risk of mode collapse and enhances scalability by using multiple generators, each one of them concurrently trained on a single data label. The use of the Wasserstein metric also reduces the risk of cycling by stabilizing the training of each generator. We illustrate the approach on the CIFAR10, CIFAR100, and ImageNet1k datasets, three standard benchmark image datasets, maintaining the original resolution of the images for each dataset. Performance is assessed in terms of scalability and final accuracy within a limited fixed computational time and computational resources. To measure accuracy, we use the inception score, the Fréchet inception distance, and image quality. An improvement in inception score and Fréchet inception distance is shown in comparison to previous results obtained by performing the parallel approach on deep convolutional conditional generative adversarial neural networks as well as an improvement of image quality of the new images created by the GANs approach. Weak scaling is attained on both datasets using up to 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Comparative Assessment of U-Net-Based Deep Learning Models for Segmenting Microfractures and Pore Spaces in Digital Rocks

Segmentation of high-resolution X-ray microcomputed tomography (µCT) images is crucial in digital rock physics (DRP), affecting the characterization and analysis of microscale phenomena in the porous media. The complexity of geological structures and nonideal scanning conditions pose significant challenges to conventional image segmentation approaches. Motivated by the recent increasing popularity of deep learning (DL) techniques in image processing, this work undertakes a comparative study of DL models, specifically U-Net and its variants, for segmenting multiple targets with distinguished features in digital rocks, including discrete fracture networks (DFNs), pore spaces, and solid rock. Particularly, DFNs have a smaller volumetric fraction over others, bringing in a substantial challenge of imbalanced segmentation. The primary focus is to evaluate the architecture and feature enhancement strategies of various DL models, including U-Net, attention U-Net, residual U-Net, U-Net++, and residual U-Net++. The models were designed as 2.5D, utilizing a central 2D image and its two adjacent upper and lower 2D images as input to provide a pseudo-3D context. In addition, because the ground truth of segmentation was unknown for real-world digital rocks, we created a benchmark data set following the inverse operations of segmentation. The data synthesis started from the label images (i.e., solid rock, pore spaces, and DFNs), followed by simulating partial volume blurring, adding random background noise, and introducing ring artifacts to mimic real raw X-ray µCT images. The data set, which included various rock types (i.e., sandstone and artificial data), scanning resolution, and magnitudes of noise and artifacts, was divided into training and testing data sets with a 90% and 10% ratio, respectively. Moreover, in addition to the conventional pixel-wise evaluation metrics, the physics-based metric of the lattice-Boltzmann method (LBM) simulated permeability provided more comprehensive assessments. The results demonstrated that the residual connections, nested architectures, and redesigned skip connections contribute to the model performance and give the residual U-Net++ the highest accuracy. The improvements were mainly on the boundaries and small targets, especially the DFNs, which dominate the interconnectivity and therefore affect the permeability greatly. This study also rigorously evaluated the efficiency and generalization of each model, demonstrating that the sophisticated architectures achieved excellent practicability and maintained robust performance on completely unseen data, ensuring their suitability for diverse and challenging DRP applications.

58 GEOSCIENCES↗

DOE OSTI2022

Understanding the Physics Representation of Deep Learning Models in Environmental Applications

Deep learning (DL) models have been popular in earth and environmental modeling and analysis, which exhibit huge potential in capturing and reconstructing the non-linearity of relevant environmental processes. They are extensively used as analytical tools or emulators for multiple domains (atmosphere, land surface, ocean, and biogeochemistry). Despite their success, their internal working mechanism remains largely unknown. Such a lack of knowledge hinders the identification of physically consistent models that are fully adaptive to non-stationary climate, as well as the development of physics-informed machine learning such as physics-informed neural network (PINN). To establish preliminary knowledge and framework of such physics representation evaluation, this project focuses on an improved understanding of DL models in the environmental applications. DL models are increasingly applied to environmental modeling and prediction. However, they have been evaluated mostly from a performance perspective, and there is a gap in understanding how they represent the known physics internally. Such knowledge is especially critical when applying DL models under climate change conditions, where new inputs are likely outside the ranges of the training datasets. In this project, we reveal how the known physical processes are represented within DL models from both statistical and mechanistic perspectives. Leveraging the traditional model evaluations that focus more on the accuracies of predictions, we establish a framework that examines both the accuracy and physics representation of DL models. This analysis framework can identify DL models that make the correct predictions based on correct physics, thus enhancing the existing explainable artificial intelligence (explainable-AI) portfolio. It lays a foundation for developing novel metrics to evaluate the emerging DL models in environmental applications. This knowledge also informs the development of physics-informed DL models by revealing the direct connections between the known physical processes and specific model components or structures.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗

DOE OSTI2023

Reservoir Thermal Energy Storage Benchmarking (Rev. 3)

A benchmarking analysis of RTES research funded by GTO through the Beyond Batteries projects was conducted against the ESGC to see where they fit within the identified ESGC Use Cases. The projects were found to advance knowledge in multiple ESGC use cases, either directly or in some cases, indirectly as enabling technologies. This analysis is helpful to understand where RTES and associated research fits into the larger discussion around energy storage technologies. Also, a retrospective analysis of the Beyond Batteries projects was conducted to evaluate what the projects learned and how the results can be applied to advance the value of RTES. Major results of each of the studies are summarized in Table 2. Additionally, a comparative metrics analysis for RTES was completed to understand where RTES lies within the energy storage industry. Metrics for evaluation of RTES and its comparison to other storage technologies were selected and ranges of their values compiled. The selected metrics – LCOE (levelized cost of energy), capital costs, roundtrip efficiency, energy storage capacity, and storage time – were chosen based on data availability and have a particularly strong influence on the potential deployment of a storage technology. Charts which compare the metrics are presented in section 4.3 and show ranges for each of the 10 selected technologies. However, due to a lack of domestic operational facilities, values for RTES and for portions of the remaining technologies are based on theoretical modeling and studies of best-case scenarios. LCOE estimates for RTES fall within the lower reaches of Figure 15, but nevertheless amount to 2 – 5 times the ESGC Roadmap goal for LCOE, for example in the Facilitating and Evolving Grid Use Case. Capital costs for RTES sit on the higher end (Figure 16) but are expected to decrease as new projects are developed and the technology is refined. The theoretical roundtrip efficiency reported for RTES varies from mid to high percentages (Figure 17) with efficiencies upwards of 93% in modeled scenarios in the Portland Basin (Bershaw et al.,2020). RTES is also expected to have the largest energy storage capacities and longest storage times, likely matched only by lower efficiency hydrogen storage. To better assess the role that RTES could play in energy storage we examined it’s potential in the U.S. The potential depends on many factors. Recently, many researchers have started looking at deep sedimentary basins, depleted oil and gas fields, and basalt formations as potential targets for RTES development. The United States Geological Survey (USGS) has analyzed various cities and shown substantial RTES potential in the cooling sector (Pepin et al., 2021). By modeling RTES in low-quality groundwater (e.g., brackish), it is shown to be favorable across the U.S. with particular suitability in the Illinois Basin, Coastal Plains, and Basin and Range regions. Seasonal RTES operations have also been modeled in the Portland Basin by those at the USGS and Portland State University to simulate an RTES system supplying heating loads needed for the Oregon Health and Science University. Simulations suggest that high conductive heat loss in the initial years exists but tends to decrease with increasing time and development of the resource due to self-insulating nature of the basalts (Burns et al., 2020). Other national laboratory efforts are taking a close look at many of the technical issues involved with RTES (McLing et al., 2019, McLing et al., 2022). These include difficulties in understanding geochemical, hydrogeological, mechanical, and microbiological changes at such elevated temperatures and operational scenarios. Major gaps in research are identified and suggested for future work. With this increased focus to understand how to make RTES successful in the U.S., this technology could be a potential solution to many of the nation’s energy storage problems. For the energy independence of this country, the DOE should prioritize de-risking this technology by making future investments in pilot-scale demonstrations to attract potential investors.

15 GEOTHERMAL ENERGY↗

DOE OSTI2025

Failure Analysis–Informed Risk Assessment Framework for Geological Carbon Storage Using Numerical Simulation and Machine Learning

Geological carbon storage (GCS) is recognized as a critical technology for achieving large-scale reductions in anthropogenic carbon dioxide (CO 2 ) emissions. Ensuring long-term containment and safety requires robust risk assessment frameworks that account for geological uncertainty and identify potential failure scenarios. Among various indicators, the area of review (AoR) serves as a key metric for evaluating storage performance, regulatory compliance, and monitoring design, as it delineates the spatial extent impacted by pressure buildup and plume migration. However, conventional AoR-based risk assessments typically perturb parameters within narrow uncertainty bounds, potentially overlooking rare but high-impact events arising from extreme geological conditions. In this study, we present a failure analysis–informed risk assessment framework for large-scale GCS projects to improve site prescreening and monitoring design. A suite of 300 numerical simulations was generated using stochastic geological models that vary five key parameters: net-to-gross ratio, anisotropy azimuth, porosity multiplier, permeability multiplier, and vertical-to-horizontal permeability ratio. Among these, 200 realizations represent normal geological uncertainty, while 100 additional cases explore extreme yet plausible conditions for failure-case analysis. The AoR was simulated and computed from pressure and CO 2 saturation fields, where the baseline AoR boundary, representing the extent predicted under typical geological uncertainty, was defined as the union of 200 normal-range simulations, and failure was identified when extreme-range cases exceeded this baseline. Results show that incorporating broader parameter uncertainty produces significantly larger AoR extents, underscoring the potential underestimation of risk under conventional uncertainty ranges. Furthermore, spatial probability maps derived from failure-induced AoR exceedance identify regions requiring enhanced monitoring attention. Various machine learning (ML)–based classifiers were developed to predict failure occurrence from geological parameters, with the random forest model achieving the highest performance (F1-score of 0.986). Consistent findings from correlation coefficient, feature importance, and Sobol sensitivity analyses reveal that low net-to-gross ratios and permeability multipliers are the dominant risk drivers, reflecting reduced reservoir connectivity and limited pressure dissipation. Altogether, these results provide a novel framework for risk-informed site prescreening and monitoring design that explicitly considers rare but high-impact geological scenarios in GCS projects.

25 ENERGY STORAGE↗