Density regulation with disruption avoidance in next-generation tokamaks using a safe reinforcement learning-based controller
Not Available
Paruchuri, Sai Tej (ORCID:0000000323723888)↗
SEARCH · Search NASA
Results for “learning-based control”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
A Sequential Model Predictive and Deep Reinforcement Learning-Based Controller for Distribution System Outage Mitigation under Hurricane Events
This paper proposes a proactive outage mitigation framework for power distribution networks to withstand hurricane-induced disruptions. It leverages Model Predictive Control (MPC) to identify safe lines for proactive switching during hurricanes, minimizing the risk of cascading failures and voltage violations. The switching strategies optimized by MPC are sequentially integrated with a Deep Reinforcement Learning agent using the Advantage Actor-Critic algorithm, enabling dynamic line switching to maximize connected buses and minimize voltage violations in real time. Using a probabilistic hurricane model, the framework predicts line failures and adapts to varying conditions to enhance grid resilience. Simulations on the IEEE 123-bus system demonstrate its effectiveness in maintaining high connectivity and minimizing disruptions. Real-time testing with an RTDS confirms the practicality and reliability of the proposed approach.
Selim, Alaa [University of Connecticut]↗
Reinforcement Learning-Based Secondary Control Strategy for Voltage and Frequency Regulation in Islanded Inverter-Based Microgrids
This paper presents a reinforcement learning (RL) approach for secondary voltage and frequency control in islanded inverter-based microgrids. The proposed control strategy aims to restore voltage and frequency deviations caused by the primary droop control while ensuring proper power sharing between distributed generators. The RL agent is designed to provide correction signals to the primary control, considering communication delays and system constraints. The effectiveness of the proposed control strategy is validated through simulation results in MATLAB/Simulink environment, demonstrating superior performance in maintaining voltage and frequency within the nominal values.
Rodriguez Martinez, Omar Felipe [University of Pue↗
Adaptive Reinforcement Learning Control for Power Distribution in Multi-Output Resonant Converters
This paper presents an adaptive reinforcement learning (ARL)-based control framework for efficient power distribution in a multi-output resonant converter for UAV applications. The proposed system is based on a high-frequency isolated resonant architecture, where a single energy source supplies multiple propulsion loads through independently controlled output rectifiers, addressing the need for coordinated multi-motor power management. The ARL framework dynamically allocates output power by learning optimal phase-shift control actions under varying load demands and operating conditions. The agent autonomously determines control parameters that maximize conversion efficiency while ensuring accurate power sharing among multiple outputs. In addition, the proposed approach enables adaptive operation without requiring detailed system modeling or manual tuning. Experimental results demonstrate stable and efficient performance over a wide range of operating conditions, confirming the effectiveness and robustness of the learning-based control strategy for multi-output resonant converter system.
Asa, Erdem [ORNL] (ORCID:0000000190884812)↗
Machine Learning for Optimized Polarization at Jefferson Lab
Polarized cryo-targets and polarized photon beams are widely used in experiments at Jefferson Lab. Traditional methods for maintaining the optimal polarization involve manual adjustments throughout data taking by human shift takers. This may introduce some level of inconsistency simply due to the wide variety of experience and expertise of the shift takers themselves. Implementing machine learning-based control systems can improve the stability of the polarization without relying on human intervention. The cryo-target polarization is influenced by temperature, microwave energy, the distribution of paramagnetic radicals, as well as operational conditions including the radiation dose. Diamond radiators are used to generate linearly polarized photons from a primary electron beam. The energy spectrum of these photons can drift over time due to changes in the primary electron beam conditions and diamond degradation. As a first step towards automating the continuous optimization and control processes, uncertainty aware surrogate models have been developed to predict the polarization based on historical data. This talk will provide an overview of the use cases and models developed, highlighting the collaboration between data scientists and physicists at Jefferson Lab.
Jeske, Torri [Thomas Jefferson National Accelerato↗
LC-Opt: Benchmarking Reinforcement Learning and Agentic AI for End-to-End Liquid Cooling Optimization in Data Centers
Liquid cooling is critical for thermal management in high-density data centers with the rising AI workloads. However, machine learning-based controllers are essential to unlock greater energy efficiency and reliability, promoting sustainability. We present LC-Opt, a Sustainable Liquid Cooling (LC) benchmark environment, for reinforcement learning (RL) control strategies in energy-efficient liquid cooling of high-performance computing (HPC) systems. Built on the baseline of a high-fidelity digital twin of Oak Ridge National Lab's Frontier Supercomputer cooling system, LC-Opt provides detailed Modelica-based end-to-end models spanning site-level cooling towers to data center cabinets and server blade groups. RL agents optimize critical thermal controls like liquid supply temperature, flow rate, and granular valve actuation at the IT cabinet level, as well as cooling tower (CT) setpoints through a Gymnasium interface, with dynamic changes in workloads. This environment creates a multi-objective real-time optimization challenge balancing local thermal regulation and global energy efficiency, and also supports additional components like a heat recovery unit (HRU). We benchmark centralized and decentralized multi-agent RL approaches, demonstrate policy distillation into decision and regression trees for interpretable control, and explore LLM-based methods that explain control actions in natural language through an agentic mesh architecture designed to foster user trust and simplify system management. LC-Opt democratizes access to detailed, customizable liquid cooling models, enabling the ML community, operators, and vendors to develop sustainable data center liquid cooling control solutions.
Naug, Avisek [Hewlett Packard Enterprise]↗
Artificial Intelligence-Driven Management of Sustainable Energy Resources: Visibility, Operation, and Control
The rapid global transition toward sustainable energy resources (SERs) is reshaping how modern power systems are observed, optimized, and controlled. While SERs have significantly advanced decarbonization, their weather dependence, variability, and inverter-dominated characteristics challenge traditional, centralized, and deterministic grid operation. At the same time, the proliferation of high-resolution data from inverters, smart meters, and sensors offers unprecedented visibility into system dynamics. Yet, it also exceeds the analytical capability of conventional model-based approaches. Artificial intelligence (AI) provides a new foundation for addressing these challenges by bridging physical laws with data-driven learning, enabling accurate state awareness, adaptive operation, and coordinated control across distributed assets. This article examines how AI transforms the management of SER-rich power systems along three critical dimensions: 1) enhancing visibility by inferring behind-the-meter (BTM) activities, assessing SER flexibility, and reconstructing system states from sparse or noisy measurements; 2) improving operation through AI-enhanced SER service provision, volt/var control (VVC), and dynamic operating envelopes (DOE) for efficiency and security; and 3) advancing control by embedding learning-based intelligence into inverter coordination, voltage and frequency regulation, and long-term dispatch. Together, these developments reveal how AI can convert the variability of SERs from an operational challenge into a source of flexibility, resilience, and intelligence, paving the way toward sustainable, adaptive, and self-optimizing power systems.
24 POWER TRANSMISSION AND DISTRIBUTION↗
A User-Friendly GUI Tool for Automated Microstructural Analysis of Fiber-Reinforced Composites and Porous Structures
Understanding and quantifying microstructural features such as fiber orientation and porosity is critical for predicting the mechanical behavior and performance of fiber-reinforced polymer composites. Traditional manual analysis is time-consuming, subjective, and unsuitable for high-throughput datasets. We present a graphical user interface (GUI) application that automates the analysis of microscopy images to extract key microstructural metrics, including fiber orientation tensors, fiber orientation distribution, porosity and pore size distribution. The app integrates multiple image segmentation techniques including global and local thresholding, clustering, and region-based approaches, offering flexibility for different types of image qualities and features. Users can load microstructural images, select regions of interest and segmentation techniques tailored to their image dataset. It also addresses a critical challenge in fiber orientation analysis: the ambiguities caused by touching, overlapping, or partially cut fibers. It supports autorun examples for standardized workflows, enabling reproducible analysis and facilitating training and benchmarking. This tool significantly reduces manual intervention, enhances consistency, and accelerates data generation for structure–property modeling, process optimization, and digital materials research. The tool is intended for use by materials scientists, engineers, and researchers engaged in composite characterization, quality control, and machine learning-based microstructural studies.
Chawla, Komal [ORNL] (ORCID:0000000190327565)↗
Signal Whisperers: Enhancing Wireless Reception Using DRL-Guided Reflector Arrays
This paper presents a multi-agent reinforcement learning (MARL) approach for controlling adjustable metallic reflector arrays to enhance wireless signal reception in non-line-of-sight (NLOS) scenarios. Unlike conventional reconfigurable intelligent surfaces (RIS) that require complex channel estimation, our system employs a centralized training with decentralized execution (CTDE) paradigm where individual agents corresponding to reflector segments autonomously optimize reflector element orientation in three-dimensional space using spatial intelligence based on user location information. Through extensive ray-tracing simulations with dynamic user mobility, the proposed multi-agent beam-focusing framework demonstrates substantial performance improvements over single-agent reinforcement learning baselines, while maintaining rapid adaptation to user movement within one simulation step. Comprehensive evaluation across varying user densities and reflector configurations validates system scalability and robustness. The results demonstrate the potential of learning-based approaches for adaptive wireless propagation control.
deep reinforcement learning↗
Comprehensive assessment of deep reinforcement learning approaches for economic dispatch in nuclear-driven microgrids
As the electrical grid integrates more variable renewable energy sources such as wind and solar, the demand for distributed and flexible systems to address this increased variability becomes critical. Nuclear-driven microgrids provide a promising solution by offering stable generation to complement intermittent renewables, ensuring grid reliability and operating efficiency. This paper proposes a recurrent deep reinforcement learning framework for optimal economic dispatch in a nuclear-powered microgrid integrating renewable energy sources, small modular reactors, battery storage systems, and balance-of-plant dynamics. A three-agent control architecture is developed, where demand and renewable energy agents act as forecasters, and a reinforcement learning-based dispatch agent performs real-time energy allocation. A nonlinear programming formulation is first used to generate an optimal baseline for benchmarking. The proposed dispatch controller, based on Proximal Policy Optimization enhanced with Long Short-Term Memory networks, exploits temporal correlations in system dynamics by taking advantage of the time series used as inputs to improve policy robustness under uncertainty. Comparative analysis against established deep reinforcement learning methods, including Proximal Policy Optimization with a feedforward architecture, Soft Actor-Critic, and Twin Delayed Deep Deterministic Policy Gradient, demonstrates superior performance. Numerical results indicate that the proposed controller achieves a 0.39% cost reduction relative to the nonlinear programming benchmark and outperforms other learning-based methods by generating additional revenue of up to 0.35%. All reinforcement learning controllers compute dispatch actions in less than 0.3 s, resulting in a computational speedup of more than three orders of magnitude over the nonlinear programming baseline. The findings of this paper highlight their applicability for real-time operation and control in nuclear-integrated microgrids under volatile operating conditions.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Divide and conquer: separating the two probabilities in seismic phase picking
There are two fundamental probabilities in the seismic phase picking process—the probability of the existence of a seismic phase (detection probability) and the probability associated with the phase arrival time estimation (timing probability). The nearly ubiquitous approach in developing deep learning phase picking models is to use a kernel, such as a truncated Gaussian, to mask the labelled phase arrival time and train a segmentation model. Once a model is trained, the times of the peaks in the output are taken as phase arrival times (picks), and the height of the peaks are taken as ‘probability’ of the picks. Here, we show that this ‘probability’ represents neither the detection nor the timing probability because this approach forces the output to follow the shape of the kernel. We introduce an approach using two models to estimate these two distinct probabilities. We use a binary classifier with a calibrated confidence to address the detection probability and a multiclass classifier to obtain a probability mass function to address the timing probability. This new approach can make the deep learning-based phase picking process more interpretable and provide options to logically control seismic monitoring workflows.
58 GEOSCIENCES↗
Optimizing on-ramp merging for connected and automated vehicles: A hierarchical approach using deep reinforcement learning and optimal control
On-ramp merging for Connected and Automated Vehicles (CAVs) presents significant challenges in dynamic traffic environments. Traditional methods and recent learning-based approaches often fail to simultaneously address decision-making complexity and execution precision under fluctuating conditions. This study introduces a novel hierarchical framework that combines: (1) a high-level Deep Reinforcement Learning (DRL) module that coordinates merging sequences through Virtual Traffic Signals (VTS) with Yield/Green phases and (2) a low-level optimal controller generating collision-free speed trajectories via pseudospectral convex optimization. A convolutional autoencoder compresses high-dimensional traffic states to enhance responsiveness. Extensive simulations demonstrate a 12.5% improvement in mainline throughput a 28% reduction in emergency braking events, and 31.66% lower fuel consumption compared to baseline methods. Furthermore, the framework’s effectiveness in coordinating CAV merges highlights its potential for real-world deployment. Future work will extend validation to multi-lane scenarios with mixed traffic and large-scale multiple merging points.
Connected and automated vehicles↗
Machine Learning-Enabled Wearable Piezoelectric Acoustic Sensor for Real-Time Breast Abnormality Detection
In contemporary society, breast health has become a significant public health concern, particularly among women. According to statistics from the World Health Organization, both the incidence and mortality rates of breast tumors have steadily increased in recent years. Therefore, effective early-stage screening and postoperative monitoring are essential for maintaining breast health. However, conventional clinical diagnostic modalities are typically bulky, operationally complex, and unsuitable for continuous real-time monitoring, which limits their use in portable and everyday health management applications. To address these limitations, this study proposes a machine learning-integrated wearable piezoelectric sensing platform as an auxiliary tool for breast health assessment. The device consists of a PDMS matching layer embedded with flexible silver nanowires, a P(VDF-TrFE) piezoelectric layer, and a multi-channel low-noise signal acquisition circuit. It is capable of acquiring acoustic echo signals from tissue-mimicking environments and automatically evaluating signal validity using a convolutional neural network (CNN). By integrating piezoelectric sensing with deep learning-based signal analysis, the proposed system achieves a signal-to-noise ratio exceeding 70 dB and a real-time classification accuracy above 96% under controlled conditions. These results demonstrate that the platform provides a compact, portable, and intelligent approach for wearable sensing of mechanical heterogeneity and highlight its potential for future development in continuous biomedical monitoring technologies.
He, Shuaitong↗
Learning-Based Quantum Compilation: Translating QASM to QIR with CodeBERT
We propose a learning-based approach to quantum compilation by translating OpenQASM to Quantum Intermediate Representation (QIR) using a fine-tuned CodeBERT model. Trained on 10,000 synthetic QASM-QIR pairs, the model captures code semantics while addressing QIR verbosity and the 512-token limit via a custom token compression scheme. Finetuning was performed on the Frontier supercomputer, with results showing syntactic correctness and stable validation loss reduction. Our method moves toward enabling flexible, language-modeldriven quantum software tools. It also introduces syntax error handling and the possibility of incorporating classical control constructs, addressing limitations in existing rule-based compilers like qBraid-QIR. While the current model has been validated on quantum-only circuits, we propose future evaluations on hybrid quantum-classical examples. This poster will provide architecture insights, compression examples, training loss plots, and QIR outputs. Our work highlights the potential for scalable, adaptable compilation in future quantum toolchains.
Afrose, Sharmin [ORNL]↗
Deep Learning and Photogrammetric Reconstruction for Automated Crack Detection and Dimensional Measurement in Mining Operations
Surface crack detection and dimensional measurement at active mining sites present significant safety and operational challenges. Manual inspection methods are labor-intensive, spatially incomplete, and expose personnel to hazardous environments, while existing automated approaches have been developed primarily for concrete civil infrastructure and have not been validated on the complex, variable surfaces characteristic of mining environments. This dissertation presents an automated pipeline that integrates deep learning semantic segmentation with Structure-from-Motion photogrammetry to detect surface cracks and measure their aperture, length, and vertical displacement from standard RGB imagery acquired during routine Uncrewed Aerial Vehicle (UAV) survey operations, without requiring additional sensor hardware or manual measurement. The pipeline combines a U-Net architecture with an EfficientNet-B0 encoder, pretrained on the SDNET2018 concrete crack dataset and fine-tuned on a mining-specific dataset spanning laboratory concrete specimens, coal refuse impoundment embankments, and post-blast limestone quarry benches. Photogrammetric reconstruction is performed using COLMAP Structure-from-Motion and Multi-View Stereo, with crack segmentation masks projected into the reconstructed point cloud to enable three-dimensional vertical displacement measurement through local plane fitting and bimodal surface detection. The pipeline was validated across 36 controlled laboratory specimens at three imaging distances and four vertical displacement levels, achieving aperture measurement RMSE of 0.047 cm and R² of 0.954, and vertical displacement RMSE of 0.140 cm and R² of 0.966, against independent caliper measurements. Field application at a coal refuse impoundment in southwestern Pennsylvania detected 71 crack components across the embankment crest, with a dominant longitudinal crack exhibiting aperture values reaching 28 cm and a 95th percentile vertical displacement of 35.53 cm, consistent in magnitude and spatial distribution with simultaneously acquired LiDAR-derived estimates. Application across four post-blast limestone quarry bench datasets in California successfully characterized blast-induced fracture networks at ground sampling distances ranging from 0.59 to 1.23 cm/pixel, with detected crack geometries physically consistent with observable surface conditions at each site. The results demonstrate that deep learning-based crack detection and photogrammetric measurement can be integrated into routine UAV inspection workflows at mining sites, providing repeatable, scalable, and quantitative crack characterization across surface types, crack scales, and displacement magnitudes not previously addressed in the literature. The pipeline requires no dedicated surveying equipment beyond the UAV platforms already deployed at mine sites for survey and monitoring purposes, supporting practical adoption within existing operational workflows.
Crack detection, Dimensional Measurement↗
Unsupervised anomaly clustering via offset alignment in multivariate grid sensing data
Modern industries increasingly rely on multi-sensor technologies to acquire complex, high-dimensional data streams, enabling advanced monitoring and control systems. One critical application is online anomaly detection in electrical smart grids, where multivariate and multimodal sensing technologies play a vital role. However, detecting anomalies in such time-series data is challenging due to their inherent temporal dependencies and stochastic behavior. Traditional approaches based on supervised and semi-supervised learning methods depend on labeled datasets, which are often unavailable in real-world scenarios. While unsupervised methods have emerged as promising alternatives, these methods are highly susceptible to noise and outliers commonly present in sensing applications. Furthermore, deep learning-based anomaly detection methods, despite their performance, are often criticized for their black-box nature, limiting their applicability in safety-critical and online environments where interpretability and explainability are paramount. In this work, we propose an unsupervised anomaly clustering method leveraging a cyclic alignment-based offset detection algorithm for multivariate time-series signals. The proposed method is applied to multivariate data collected from vibrational, voltage, and magnetic field sensors deployed in a local grid substation. Our results demonstrate the robustness of the algorithm in accurately clustering various anomalies/events across different sensing modalities. Additionally, we compare the effectiveness of the proposed approach against a simple pattern-based anomaly detection method, which performs well for univariate data but fails to generalize to multivariate and multimodal time-series data.
Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗
Deep Reinforcement Learning for Distribution System Operations: A Tutorial and Survey
Here, the rapid evolution of modern electric power distribution systems into complex networks of interconnected active devices, distributed generation (DG), and storage poses increasing difficulties for system operators. The large-scale integration of distributed energy resources (DERs) and the rapid exchange of measurement data via communication networks present major opportunities for advancing grid operations but also introduce greater uncertainty, higher data dimensionality, more complex network and device models, and challenging control and optimization problems. Deep reinforcement learning (DRL) algorithms are promising in addressing these challenges. However, they have not been effectively adapted for power systems applications, requiring extensive customization for implementation and evaluation. This has resulted in reproducibility challenges and a steep learning curve for researchers new to applying DRL algorithms to the power systems domain. To bridge these gaps, this tutorial aims to serve as a valuable resource for researchers interested in exploring learning-based algorithms to operate active power distribution networks. Specifically, this work presents a generalized process for translating sequential decision-making problems in power distribution systems into Markov decision process (MDP) formulations, illustrated through concrete grid service examples. Additionally, we introduce a simple environment design strategy to develop and evaluate example DRL algorithms for distribution system applications, complete with an included code repository to guide users through environment construction.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Deep-learning-based domain adaptation for cavity fault prediction at Jefferson Laboratory
Superconducting radio-frequency (SRF) cavities are the core components of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab, providing high-power electron beams for nuclear physics experiments. The facility comprises 418 SRF cavities, and any fault in these cavities can lead to interruptions in the electron beam supply. Cavity faults are the leading cause of beam trips in CEBAF. Predicting and mitigating those faults before onset can help maintain normal operation. Existing models face challenges in distinguishing between normal and fault signals when changes occur in the underlying time-series data, from changes in control software, operational parameters, or the environment. This work proposes a deep learning domain adaptation model that leverages transfer learning to address fault prediction challenges by improving accuracy. The model is trained and fine-tuned using a dataset collected for faulty and normal operation using a data acquisition system in CEBAF. Our deep learning-based domain adaptation model achieves a prediction accuracy of 89.61% of the fault and normal signals. The developed model effectively predicts normal running signals compared to the baseline approach without domain adaptation. This capacity is essential for the fault prediction task in the CEBAF because of heavily imbalanced data containing vast amounts of normal signals. The model performs well for predicting faults several hundred milliseconds before the fault onset compared to other models where no adaptation is applied. Incorporating deep learning-based domain adaptation techniques will significantly improve the fault prediction performance.
Rahman, Md Monibor [Old Dominion Univ., Norfolk, V↗