Results for “Contrastive learning” — Page 13

DOE OSTI2021

Adaptive Machine Learning for Robust Diagnostics and Control of Time-Varying Particle Accelerator Components and Beams

Machine learning (ML) is growing in popularity for various particle accelerator applications including anomaly detection such as faulty beam position monitor or RF fault identification, for non-invasive diagnostics, and for creating surrogate models. ML methods such as neural networks (NN) are useful because they can learn input-output relationships in large complex systems based on large data sets. Once they are trained, methods such as NNs give instant predictions of complex phenomenon, which makes their use as surrogate models especially appealing for speeding up large parameter space searches which otherwise require computationally expensive simulations. However, quickly time varying systems are challenging for ML-based approaches because the actual system dynamics quickly drifts away from the description provided by any fixed data set, degrading the predictive power of any ML method, and limits their applicability for real time feedback control of quickly time-varying accelerator components and beams. In contrast to ML methods, adaptive model-independent feedback algorithms are by design robust to un-modeled changes and disturbances in dynamic systems, but are usually local in nature and susceptible to local extrema. In this work, we propose that the combination of adaptive feedback and machine learning, adaptive machine learning (AML), is a way to combine the global feature learning power of ML methods such as deep neural networks with the robustness of model-independent control. We present an overview of several ML and adaptive control methods, their strengths and limitations, and an overview of AML approaches.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Super-Resolution for Renewable Energy Resource Data with Wind from Reanalysis Data and Application to Ukraine

With a potentially increasing share of the electricity grid relying on wind to provide generating capacity and energy, there is an expanding global need for historically accurate, spatiotemporally continuous, high-resolution wind data. Conventional downscaling methods for generating these data based on numerical weather prediction have a high computational burden and require extensive tuning for historical accuracy. In this work, we present a novel deep learning-based spatiotemporal downscaling method using generative adversarial networks (GANs) for generating historically accurate high-resolution wind resource data from the European Centre for Medium-Range Weather Forecasting Reanalysis version 5 data (ERA5). In contrast to previous approaches, which used coarsened high-resolution data as low-resolution training data, we use true low-resolution simulation outputs. We show that by training a GAN model with ERA5 as the low-resolution input and Wind Integration National Dataset Toolkit (WTK) data as the high-resolution target, we achieved results comparable in historical accuracy and spatiotemporal variability to conventional dynamical downscaling. This GAN-based downscaling method additionally reduces computational costs over dynamical downscaling by two orders of magnitude. We applied this approach to downscale 30 km, hourly ERA5 data to 2 km, 5 min wind data for January 2000 through December 2023 at multiple hub heights over Ukraine, Moldova, and part of Romania. With WTK coverage limited to North America from 2007–2013, this is a significant spatiotemporal generalization. The geographic extent centered on Ukraine was motivated by stakeholders and energy-planning needs to rebuild the Ukrainian power grid in a decentralized manner. This 24-year data record is the first member of the super-resolution for renewable energy resource data with wind from the reanalysis data dataset (Sup3rWind).

17 WIND ENERGY↗

DOE OSTI2023

Quantifying uncertainty for deep learning based forecasting and flow-reconstruction using neural architecture search ensembles

Classical problems in computational physics such as data-driven forecasting and signal reconstruction from sparse sensors have recently seen an explosion in deep neural network (DNN) based algorithmic approaches. However, most DNN models do not provide uncertainty estimates, which are crucial for establishing the trustworthiness of these techniques in downstream decision making tasks and scenarios. In recent years, ensemble-based methods have achieved significant success for the uncertainty quantification in DNNs on a number of benchmark problems. However, their performance on real-world applications remains under-explored. In this work, we present an automated approach to DNN discovery and demonstrate how this may also be utilized for ensemble-based uncertainty quantification. Specifically, we propose the use of a scalable neural and hyperparameter architecture search for discovering an ensemble of DNN models for complex dynamical systems. We highlight how the proposed method not only discovers high-performing neural network ensembles for our tasks, but also quantifies uncertainty seamlessly. This is achieved by using genetic algorithms and Bayesian optimization for sampling the search space of neural network architectures and hyperparameters. Subsequently, a model selection approach is used to identify candidate models for an ensemble set construction. Afterwards, a variance decomposition approach is used to estimate the uncertainty of the predictions from the ensemble. We demonstrate the feasibility of this framework for two tasks — forecasting from historical data and flow reconstruction from sparse sensors for the sea-surface temperature. In conclusion, we demonstrate superior performance from the ensemble in contrast with individual high-performing models and other benchmarks.

Deep ensembles↗

DOE OSTI2025

Lessons Learned from Three Agrivoltaic Installations in New Jersey

Agrivoltaics is a new technology that has the potential to positively impact commercial farming by combining agricultural practices with the generation of solar energy. While some yield reduction is to be expected, resulting from less sunlight reaching the plant canopy and ground occupied by support structures, the generated electricity provides a low-risk supplemental income to farmers. In order to combine farming with electricity generation, agrivoltaic systems use a lower ground coverage ratio compared to normal solar farms and the PV panels are often mounted higher above the ground in order to facilitate the movement of agricultural equipment and to reduce the contrast between shaded and non-shaded areas. With funding provided from the state of New Jersey and the New Jersey Agricultural Experiment Station (NJAES), we designed and installed three unique agrivoltaic research systems at Rutgers/NJAES farms. These projects were recently completed and are generating electricity that is exported to the grid. This paper discusses the lessons we have learned along the way, including all the steps necessary to see an agrivoltaic project through to completion.

Both, A. J. (ORCID:0000000150845296)↗

DOE OSTI2024

Lessons Learned from Three Agrivoltaic Installations in New Jersey

Agrivoltaics is a new technology that has the potential to positively impact commercial farming by combining agricultural practices with the generation of solar energy. While some yield reduction is to be expected, resulting from less sunlight reaching the plant canopy and ground occupied by support structures, the generated electricity provides a low-risk supplemental income to farmers. In order to combine farming with electricity generation, agrivoltaic systems use a lower ground coverage ratio compared to normal solar farms and the PV panels are often mounted higher above the ground in order to facilitate the movement of agricultural equipment and to reduce the contrast between shaded and non-shaded areas. With funding provided from the state of New Jersey and the New Jersey Agricultural Experiment Station (NJAES), we designed and installed three unique agrivoltaic research systems at Rutgers/NJAES farms. These projects were recently completed and are generating electricity that is exported to the grid. This paper discusses the lessons we have learned along the way, including all the steps necessary to see an agrivoltaic project through to completion.

14 SOLAR ENERGY↗

DOE OSTI2024

Lessons Learned from Three Agrivoltaic Installations in New Jersey

Agrivoltaics is a new technology that has the potential to positively impact commercial farming by combining agricultural practices with the generation of solar energy. While some yield reduction is to be expected, resulting from less sunlight reaching the plant canopy and ground occupied by support structures, the generated electricity provides a low-risk supplemental income to farmers. In order to combine farming with electricity generation, agrivoltaic systems use a lower ground coverage ratio compared to normal solar farms and the PV panels are often mounted higher above the ground in order to facilitate the movement of agricultural equipment and to reduce the contrast between shaded and non-shaded areas. With funding provided from the state of New Jersey and the New Jersey Agricultural Experiment Station (NJAES), we designed and installed three unique agrivoltaic research systems at Rutgers/NJAES farms. These projects were recently completed and are generating electricity that is exported to the grid. This paper discusses the lessons we have learned along the way, including all the steps necessary to see an agrivoltaic project through to completion.

14 SOLAR ENERGY↗

DOE OSTI2021

Enlightened Education: Solar Engineering Design to Energize School Facilities

Here, this paper explores the potential for universities, colleges, and K-12 schools to implement solar electric infrastructure projects on their campuses that not only provide financial savings but also provide learning environments and instructional opportunities for students. A recent case study at Madison College is presented for a 1.85 MW photovoltaic system that is the largest solar rooftop installation in the State of Wisconsin. The system was designed with several unique features to facilitate public access, provide students with hands-on interaction, and compare and contrast several different types of solar equipment. Special engineering design considerations should be made when installing solar on schools, and recommended practices from the Madison College experience are detailed. Madison College completed a Solar Roadmap in order to prioritize and sequence investment in solar across the multiple buildings and campus locations operated by the college. The featured installation was the first project within that plan. A ten-step guide on how to create a solar roadmap is shared, so that other schools can learn from Madison College’s experience and replicate the process for their own institutions.

14 SOLAR ENERGY↗

DOE OSTI2020

Reconstruction and uncertainty quantification of lattice Hamiltonian model parameters from observations of microscopic degrees of freedom

The emergence of scanning probe and electron beam imaging techniques has allowed quantitative studies of atomic structure and minute details of electronic and vibrational structure on the level of individual atomic units. These microscopic descriptors, in turn, can be associated with local symmetry breaking phenomena, representing the stochastic manifestation of the underpinning generative physical model. In this work, we explore the reconstruction of exchange integrals in the Hamiltonian for a lattice model with two competing interactions from observations of microscopic degrees of freedom and establish the uncertainties and reliability of such analysis in a broad parameter-temperature space. In contrast to other approaches, we specifically specify a loss function inherent to thermodynamic systems and utilize it to estimate uncertainty in simulated realizations of different models. As an ancillary task, we develop a machine learning approach based on histogram clustering to predict phase diagrams efficiently using a reduced descriptor space. We further demonstrate that reconstruction is possible well above the phase transition and in the regions of parameter space when the macroscopic ground state of the system is poorly defined due to frustrated interactions. This suggests that this approach can be applied to the traditionally complex problems of condensed matter physics such as ferroelectric relaxors and morphotropic phase boundary systems, spin and cluster glasses, and quantum systems once the local descriptors linked to the relevant physical behaviors are known.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2020

Accelerating Ab Initio Simulation via Nested Monte Carlo and Machine Learned Reference Potentials

As a corollary of the rapid advances in computing, ab initio simulation is playing an increasingly important role in modeling materials at the atomic scale. Two strategies are possible, ab initio Monte Carlo (AIMC) and molecular dynamics (AIMD) simulation. The former benefits from exact sampling from the correct thermodynamic distribution, while the latter is typically more efficient with its collective all-atom coordinate updates. In this study, using a relatively simple test model comprised of helium and argon, we show that AIMC can be brought up to, and even above, the performance levels of AIMD via a hybrid nested sampling/machine learning (ML) strategy. Here, ML provides an accurate classical reference potential (up to three-body explicit interactions) that can pilot long collective Monte Carlo moves that are accepted or rejected in toto à la nested Monte Carlo (NMC); this is in contrast to the single move nature of a naive implementation. Our proposed method only requires a small up front expense from evaluating the ab initio energies and forces of (100) random configurations for training. Importantly, our method does not totally rely on the trained, assuredly imperfect, interaction. We show that high performance and exact sampling at the desired level of theory can be realized even when the trained interaction has appreciable differences from the ab initio potential. Remarkably, at the highest levels of performance realized via our approach, a pair of statistically uncorrelated atomic configurations can be generated with (1) ab initio calculations.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2025

Accurate segmentation of localized corrosion in structural alloys via deep learning

This study presents a deep learning-based approach for the automated segmentation of corrosion damage in scanning electron microscopy (SEM) images. The proposed method enables rapid and accurate segmentation of corrosion features in these SEM images, making it highly suitable for real-time applications such as automated microscopy. Specifically, a dedicated corrosion segmentation database tailored for this task is constructed. The newly constructed dataset, alongside data from two public databases, are employed to jointly train a deep learning-based model modified with a texture refinement module. Compared to the same model without the texture refinement module, the refined model substantially enhances the efficacy and efficiency of corrosion segmentation. Furthermore, the methodology developed here is extendable to segmentation tasks for other materials with similar resolution, texture, and contrast characteristics, thereby paving the way for accelerated and automated analysis in corrosion science and beyond.

Artificial Intelligence↗

DOE OSTI2026

Effect of Solvent on the Local Structure, Dynamics, and Vibrational Density of States in Sn-BEA Zeolite

Lewis acid zeolites are attractive catalysts for epoxidation and biomass valorization, as they are highly active and selective in the liquid phase and can operate at or near ambient conditions. While a rich experimental literature exists on liquid-phase Lewis acid zeolite catalysis, our understanding of the molecular organization and solvent dynamics in the vicinity of Lewis acid sites with differing metal site speciation remains limited. In this work, we investigate the molecular coordination and diffusion of two common solvents (methanol and water) around the closed and open Sn-BEA zeolite active sites using molecular dynamics simulations with a machine-learned interatomic potential trained on ab initio molecular dynamics trajectories. Molecular dynamics simulations reveal that introducing active sites significantly enhances local order in the first and second solvation shells compared to the pure silica case. For methanol, both closed and open active sites are singly coordinated, while more than two water molecules coordinate the open site. In contrast to methanol, we observed that water molecules dissociate, leading to the formation of additional Sn-OH and silanol groups away from the active site. The diffusion coefficients of water and methanol are functions of the solvent population in the pore. Here, our work provides insights into how active site speciation in Lewis acid zeolites affects solvent coordination, diffusion, and vibrational signature. This information is foundational for catalyst design and optimization of liquid-phase catalytic processes in zeolites. It also demonstrates the suitability of machine-learned interatomic potentials for modeling reactive systems, enabling sufficiently long trajectories for appropriate statistical averaging.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Postmortem and Antemortem Forensic Assessment of Pediatric Fracture Healing from Radiographs and Machine Learning Classification

A timeline of pediatric bone healing using fracture healing characteristics that can be assessed solely using radiographs would be practical for forensic casework, where the fracture event may precede death by days, months, or years. However, the dating of fractures from radiographs is difficult, imprecise, and lacks consensus, as only a few aspects of the healing process are visible on radiographs. Multiple studies in both the clinical and forensic literature have attempted to develop a usable scale to assess pediatric bone healing on radiographs using various healing characteristics. In contrast to the orthopedic definition, a fracture in forensic casework is only considered to be healed when the area around the fracture has been remodeled to the point that the fracture is difficult to detect on a radiograph or on the surface of the bone itself, a process that can take several years. We subjectively assessed visible characteristics of healing in radiograms of fractures occurring in 942 living children and adolescents. By dividing these assessments into learning and test (validation) sets, the accuracy of a newly proposed fracture healing scale was compared to a previous study. Two machine learning models were used to test predictions of the new scale. All three models produced similar estimates with substantial imprecision. Results corroborate the Malone model with an independent dataset and support the efficacy of using less complex models to estimate fracture age in children.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Gaussian process for calibration and control of GlueX Central Drift Chamber

We have developed and implemented a machine learning based system to calibrate and control the GlueX Central Drift Chamber at Jefferson Lab, VA, in near real-time. The system monitors environmental and experimental conditions during data taking and uses those as inputs to a Gaussian process (GP) with learned prior. The GP predicts calibration constants in order to recommend a high voltage (HV) setting for the detector that maintains consistent detector performance (gain and resolution) throughout data taking. This approach is in stark contrast to traditional detector operations in which the detector operates at fixed HV and its calibration parameters vary quite considerably with time. Additionally, the ML based system utilizes uncertainty quantification to correct the recommended control parameters when appropriate. We will present results from the ML system autonomously during the Charged Pion Polarizability (CPP) experiment conducted in Hall D at Jefferson Lab.

McSpadden, Helen↗

DOE OSTI2022

Gaussian process for calibration and control of GlueX Central Drift Chamber

We have developed and implemented a machine learning based system to calibrate and control the GlueX Central Drift Chamber at Jefferson Lab, VA, in near real-time. The system monitors environmental and experimental conditions during data taking and uses those as inputs to a Gaussian process (GP) with learned prior. The GP predicts calibration constants in order to recommend a high voltage (HV) setting for the detector that maintains consistent detector performance (gain and resolution) throughout data taking. This approach is in stark contrast to traditional detector operations in which the detector operates at fixed HV and its calibration parameters vary quite considerably with time. Additionally, the ML based system utilizes uncertainty quantification to correct the recommended control parameters when appropriate. We will present results from the ML system autonomously during the Charged Pion Polarizability (CPP) experiment conducted in Hall D at Jefferson Lab.

McSpadden, Helen↗

DOE OSTI2026

Integrating Maximum Entropy Production Theory and Machine Learning to Improve Global Evapotranspiration Modeling

Accurate estimation of terrestrial evapotranspiration (ET) is vital for understanding global water and energy cycles. However, current global ET estimations are not well constrained. This study introduces an integrated framework combining the Maximum Entropy Production (MEP) theory with Random Forest (RF) model to improve global ET estimation. Specifically, in contrast to direct ET estimation by the RF model, the integrated framework (MEP‐RF) trains to predict error of MEP‐simulated ET. MEP‐RF outperforms RF in spatiotemporal extrapolation. Attribution analysis with in situ observations reveals that the inputs of MEP are the most critical variables for the ET process, including net radiation, vegetated area, soil moisture, and surface temperature. We further drive MEP‐RF with global reanalysis and satellite data sets of these four inputs, yielding a global mean terrestrial ET of 548 mm/year, with 77% attributed to transpiration. The global ET increased at a rate of 0.85 mm/year per year during 2003–2021, primarily due to vegetation greening rather than rising temperature, while decreasing soil moisture led to decreasing regional ET. The integrated framework provides a novel approach for the estimation of global ET without the need for hard‐to‐obtain and thus uncertain inputs, such as wind speed, surface roughness, aerodynamic and canopy stomatal resistance. Therefore, MEP‐RF offers an independent method on existing global ET products. It represents a promising physically based approach that can be incorporated into Earth System Models to enhance water and energy cycle simulations.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Predicting solid state material platforms for quantum technologies

Semiconductor materials provide a compelling platform for quantum technologies (QT). However, identifying promising material hosts among the plethora of candidates is a major challenge. Therefore, we have developed a framework for the automated discovery of semiconductor platforms for QT using material informatics and machine learning methods. Different approaches were implemented to label data for training the supervised machine learning (ML) algorithms logistic regression, decision trees, random forests and gradient boosting. We find that an empirical approach relying exclusively on findings from the literature yields a clear separation between predicted suitable and unsuitable candidates. In contrast to expectations from the literature focusing on band gap and ionic character as important properties for QT compatibility, the ML methods highlight features related to symmetry and crystal structure, including bond length, orientation and radial distribution, as influential when predicting a material as suitable for QT.

36 MATERIALS SCIENCE↗

DOE OSTI2026

REV-INR: Regularized Evidential Implicit Neural Representation for Uncertainty-Aware Volume Visualization

Applications of Implicit Neural Representations (INRs) have emerged as a promising deep learning approach for compactly representing large volumetric datasets. These models can act as surrogates for volume data, enabling efficient storage and on-demand reconstruction via model predictions. However, conventional deterministic INRs only provide value predictions without insights into the model’s prediction uncertainty or the impact of inherent noisiness in the data. This limitation can lead to unreliable data interpretation and visualization due to prediction inaccuracies in the reconstructed volume. Identifying erroneous results extracted from model-predicted data may be infeasible, as raw data may be unavailable due to its large size. To address this challenge, we introduce REV-INR, Regularized Evidential Implicit Neural Representation, which learns to predict data values accurately along with the associated coordinate-level data uncertainty and model uncertainty using only a single forward pass of the trained REV-INR during inference. By comprehensively comparing and contrasting REV-INR with existing well-established deep uncertainty estimation methods, we show that REV-INR achieves the best volume reconstruction quality with robust data (aleatoric) and model (epistemic) uncertainty estimates using the fastest inference time. Consequently, we demonstrate that REV-INR facilitates assessment of the reliability and trustworthiness of the extracted isosurfaces and volume visualization results, enabling analyses to be solely driven by model-predicted data.

Saklani, Shanu [Indian Institute of Technology, Ka↗

DOE OSTI2021

An introduction to neuromorphic computing and its potential impact for unattended ground sensors

Neuromorphic computers are hardware systems that mimic the brain’s computational process phenomenology. This is in contrast to neural network accelerators, such as the Google TPU or the Intel Neural Compute Stick, which seek to accelerate the fundamental computation and data flows of neural network models used in the field of machine learning. Neuromorphic computers emulate the integrate and fire neuron dynamics of the brain to achieve a spiking communication architecture for computation. While neural networks are brain-inspired, they drastically oversimplify the brain’s computation model. Neuromorphic architectures are closer to the true computation model of the brain (albeit, still simplified). Neuromorphic computing models herald a 1000x power improvement over conventional CPU architectures. Sandia National Labs is a major contributor to the research community on neuromorphic systems by performing design analysis, evaluation, and algorithm development for neuromorphic computers. Space-based remote sensing development has been a focused target of funding for exploratory research into neuromorphic systems for their potential advantage in that program area; SNL has led some of these efforts. Recently, neuromorphic application evaluation has reached the NA-22 program area. This same exploratory research and algorithm development should penetrate the unattended ground sensor space for SNL’s mission partners and program areas. Neuromorphic computing paradigms offer a distinct advantage for the SWaP-constrained embedded systems of our diverse sponsor-driven program areas.

97 MATHEMATICS AND COMPUTING↗