Results for “Learning control” — Page 49

DOE OSTI2023

Creating a Simulation Platform for Research and Development of Advanced Control Methods

Advanced nuclear reactors are essential to meet the changing energy requirements throughout both the United States and the rest of world. In addition to other features, they are designed to enable deployment in remote locations and operate in a fully (or near-fully) autonomous manner, which will require a new control paradigm. To realize autonomously operating reactors, the U.S. Department of Energy’s Nuclear Energy Enabling Technologies Advanced Sensors and Instrumentation (NEET ASI) program conducts research and development into the enabling technologies and methods needed, including digital twins, machine learning, and risk modeling, in addition to various types of control methods. These technologies and methods are the key foundations needed to achieve fully autonomous systems. To develop and evaluate the technologies and methods necessary for achieving autonomous operations, it is critical to identify a software tool capable of integrating all the required elements. In surveying the available solutions, no software platforms were identified that could accomplish what was needed without introducing drawbacks. This challenge was the motivation for the current effort: to develop a software platform that can seamlessly integrate autonomouscontrol-enabling technologies and methods, allowing for accelerated research and development and transfer of ideas. The resulting platform, known as the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND), is Python-based, and leverages open-source tools to provide flexibility and facilitate building upon prior research. It is designed to enable advanced reactor developers to deploy and test advanced control technologies and methods coupled with their own models, solutions, and hardware. Given the substantial undertaking of developing such a platform, the current effort focused on laying down scalable, flexible software foundations and infrastructure, then demonstrating the platform via a use case. These foundations included developing generic modules, which contain the base variable and system blocks (the information and functional building blocks, respectively, that can be used to design a simulation) and the data handling and storage blocks needed to exchange information between the various blocks; as well as enablingtechnology-specific modules. This platform was evaluated via a use case, which was to simulate and control a process for the Microreactor Automated Control System (MACS) test bed. While MACS is not currently directly coupled to any specific microreactor physics, it was initially developed in concert with the Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor, and so the MARVEL physics are used here. As part of this use case, several enabling-technology-specific blocks within COMMAND were integrated, including a proportional integral derivative (PID) control block, a Reactor Excursion and Leak Analysis Program (RELAP5-3D) block, and an anomaly detection block. The COMMAND software platform was successfully demonstrated to achieve the scalability and flexibility objectives of this effort and will be leveraged by the program’s research efforts to advance state of the art control methodologies towards autonomous operations of advanced reactors. As new use cases are created and implemented, it is anticipated that COMMAND will continue to grow and evolve to meet new requirements.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Domain-aware Control-oriented Neural Models for Autonomous Underwater Vehicles

Conventional physics-based modeling is a time-consuming bottleneck in control design for complex nonlinear systems like autonomous underwater vehicles (AUVs). In contrast, purely data-driven models, require a large number of observations and lack operational guarantees for safety-critical systems. Data-driven models leveraging available partially characterized dynamics have potential to provide reliable systems models in a typical data-limited scenario for high value complex systems, thereby avoiding months of expensive expert modeling time. In this work we explore this middle-ground between expert-modeled and pure data-driven modeling. We present control-oriented parametric models with varying levels of domain-awareness that exploit known system structure and prior physics knowledge to create constrained deep neural dynamical system models. We employ universal differential equations to construct data-driven blackbox and graybox representations of the AUV dynamics. In addition, we explore a hybrid formulation that explicitly models the residual error related to imperfect graybox models. We compare the prediction performance of the learned models for different distributions of initial conditions and control inputs to assess their suitability for control.

Shaw Cortez, Wenceslao E.↗

DOE OSTI2025

Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)

This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2019

In Situ Quality Monitoring in AM Using Acoustic Emission: A Reinforcement Learning Approach

Additive manufacturing (AM) has attracted considerable attention in recent years. This technology overcomes the geometrical limits of workpieces produced with the traditional subtractive methods and so gives the opportunity to manufacture highly complex shapes. Unfortunately, the repeatability of the manufacturing process and the monitoring of quality are not reliable enough to be utilized in mass production. The quality monitoring of AM processes in commercial equipment has been largely based on temperature measurements of the process zone or high-resolution imaging of the layers. However, both techniques lack information about the physical phenomena taking place in the depth of the materials medium and this limits their reliability in real-life applications. To overcome those restrictions, we propose to combine acoustic emission and reinforcement learning. The former captures the information about the subsurface dynamics of the process. The latter is a branch of machine learning that allows interpreting the received data in terms of quality. The combination of both is an original method for in situ and real-time quality monitoring. Acoustic data were collected during a real process using a commercial AM machine. The process parameters were selected to achieve three levels of quality in terms of porosity concentration while manufacturing a stainless steel 316L cuboid shape. Using our method, we demonstrated that each level of quality produced unique acoustic signatures during the build that were recognized by the classifier. The classification accuracy reached in this work proves that the proposed method has high potential to be used as in situ and real-time monitoring of AM quality.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2018

A digital twin for rapid qualification of 3D printed metallic components

The customized production of complex components by 3D printing has been hailed as a potentially transformative tool in manufacturing with important applications in health care, automotive and aerospace industries. However, after about a quarter of a century of research and development, only a handful of commercial alloys can be printed and the market value of all 3D printed products now amounts to a negligible portion of the manufacturing economy. This difficulty is attributable to a remarkable diversity in structure and properties of the printed components and susceptibility to defects. In addition, the current practice of qualifying components by prolonged trial and error with expensive printing equipment and feed stock material confine the printed products to a niche market where the high product cost and the delay in the qualification are not critical factors. In this report we explain how a digital twin or a digital replica of the printing machine will reduce the number of trial and error tests to obtain desired product attributes and reduce the time required for part qualification to make the printed components cost effective. It is shown that a comprehensive digital twin of 3D printing machine consisting of mechanistic, control and statistical models of 3D printing, machine learning and big data can reduce the volume of trial and error testing, reduce defects and shorten time between the design and production.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Keras2c: A library for converting Keras neural networks to real-time compatible C

With the growth of machine learning models and neural networks in measurement and control systems comes the need to deploy these models in a way that is compatible with existing systems. Existing options for deploying neural networks either introduce very high latency, require expensive and time consuming work to integrate into existing code bases, or only support a very limited subset of model types. We have therefore developed a new method called Keras2c, which is a simple library for converting Keras/TensorFlow neural network models into real-time compatible C code. It supports a wide range of Keras layers and model types including multidimensional convolutions, recurrent layers, multi-input/output models, and shared layers. Keras2c re-implements the core components of Keras/TensorFlow required for predictive forward passes through neural networks in pure C, relying only on standard library functions considered safe for real-time use. The core functionality consists of ~ 1500 lines of code, making it lightweight and easy to integrate into existing codebases. Keras2c has been successfully tested in experiments and is currently in use on the plasma control system at the DIII-D National Fusion Facility at General Atomics in San Diego.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Kelvin Probe Force Microscopy Imaging of Plasticity in Hydrogenated Perovskite Nickelate Multilevel Neuromorphic Devices

Ion drift in nanoscale electronically inhomogeneous semiconductors is among the most important mechanisms being studied for designing neuromorphic computing hardware. However, nondestructive imaging of the ion drift in operando devices directly responsible for multiresistance states and synaptic memory represents a formidable challenge. Here, we present Kelvin probe force microscopy imaging of hydrogen-doped perovskite nickelate device channels subject to high-speed electric field pulses to directly visualize proton distribution by monitoring surface potential changes spatially, which is also supported with finite element-based electric field distribution studies. First-principles calculations provide mechanistic insights into the origin of surface potential changes as a function of hydrogen donor doping that serves as the contrast mechanism. We demonstrate 128 (7-bit) nonvolatile conductance levels in such devices relevant to in-memory computing applications. The synaptic plasticity measurements are implemented in spiking neural networks and show promising results for classification (SciKit Learn’s Iris and Wine data sets) and control (OpenAI’s CartPole-v1 and BipedalWalker-v3) simulation tasks.

Kelvin probe force microscopy↗

DOE OSTI2023

Variational actor-critic algorithms,

We introduce a class of variational actor-critic algorithms based on a variational formulation over both the value function and the policy. The objective function of the variational formulation consists of two parts: one for maximizing the value function and the other for minimizing the Bellman residual. Besides the vanilla gradient descent with both the value function and the policy updates, we propose two variants, the clipping method and the flipping method, in order to speed up the convergence. We also prove that, when the prefactor of the Bellman residual is sufficiently large, the fixed point of the algorithm is close to the optimal policy.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Enhancing SRF cavity stability and minimizing detuning with data-driven resonance control based on dynamic mode decomposition

Effective resonance control of superconducting radio frequency (SRF) cavities is critical for large machines like LCLS-II, as failure to achieve proper control can result in increased RF power consumption, higher cryogenic heat loads, and increased costs. To address this challenge, we have developed a machine learning (ML) model based on the dynamic mode decomposition method to represent the forced cavity dynamics. Using this model, we designed a model predictive controller (MPC) and demonstrated through simulation that the MPC can effectively stabilize the amplitude and phase of SRF cavities using only a frequency actuator, even in the presence of multiple mechanical modes. The lightweight and explicit ML model makes the controller suitable for direct implementation on field-programmable gate arrays, unlocking the full potential of SRF linacs like LCLS-II, enabling higher beam power and energy, and also serving as an advanced motion controller for various applications, such as photon beamlines and storage rings.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗

DOE OSTI2026

EPICS for small-scale laboratories with Python soft IOCs

While the Experimental Physics and Industrial Control System (EPICS) is widely used at large laboratories for slow controls and instrumentation, the deployment of a full EPICS installation can be difficult, with a steep learning curve to new users. Taking advantage of the pythonSoftIOC module, we developed an EPICS slow controls implementation for Jefferson Lab's Hall B cryotarget written entirely in Python and based on software IOCs that communicate with instruments over Ethernet. Here, this system ran successfully, interfacing with Jefferson Lab's full EPICS network, and we offer it as an example of the capabilities of pythonSoftIOC to build lightweight, yet robust and flexible instrumentation platforms that would be easily adapted for use at a small-scale laboratory. University groups can use these examples to build complete slow controls systems, from device communication to data archiving and display, using open-source, mature EPICS tools and student-friendly Python as an alternative to expensive and proprietary systems such as LabVIEW.

Computing↗

DOE OSTI2024

Four-dimensional phase-space reconstruction of flat and magnetized beams using neural networks and differentiable simulations

Beams with cross-plane coupling or extreme asymmetries between the two transverse phase spaces are often encountered in particle accelerators. Flat beams with large transverse-emittance ratios are critical for future linear colliders. Similarly, magnetized beams with significant cross-plane coupling are expected to enhance the performance of electron cooling in hadron beams. Preparing these beams requires precise control and characterization of the four-dimensional transverse phase space. In this study, we employ generative phase-space reconstruction techniques to rapidly characterize magnetized and flat-beam phase-space distributions using a conventional quadrupole-scan method. The reconstruction technique is experimentally demonstrated on an electron beam produced at the Argonne Wakefield Accelerator and successfully benchmarked against conventional diagnostics techniques. Specifically, we show that predicted beam parameters from the reconstructed phase-space distributions (e.g., as magnetization and flat-beam emittances) are in excellent agreement with those measured from the conventional diagnostic methods. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗

DOE OSTI2024

Enhancing Distribution System Resilience: A First-Order Meta-RL Algorithm for Critical Load Restoration

The increasing frequency of extreme events and the integration of distributed energy resources (DERs) into modern grids have elevated the need for resilient and efficient critical load restoration strategies in distribution systems. However, the stochastic nature of renewable DERs, limited energy resource availability and the intricate nonlinearities inherent in complex grid control problem make the problem challenging. Although reinforcement learning (RL) and warm-start RL methods have shown promising results, their performance often falls short in rapidly adapting to new, unseen situations and typically requires exhaustive problem-specific tuning. To address these gaps, we propose a First-Order Meta-based RL (FOM-RL) algorithm within an online framework for adaptive and robust critical load restoration. By harnessing local DERs as the enabling technology, FOM-RL allows the RL agent to swiftly adapt to new unseen scenarios by leveraging previously acquired knowledge of different tasks. Experimental results provide evidence that proposed algorithm learns more efficiently and showcases generalization capabilities across diverse set of operational scenarios. Moreover, a rigorous theoretical analysis yields a tight sublinear regret bound, sensitive to temporal variability, with a task-averaged optimality gap bounded by O(VM+D*/(Tsquare root(M))). These results suggest that optimality improves with task similarity and an increased number of tasks M, reaffirming the efficacy and scalability of the proposed approach in addressing the complexities of critical load restoration in distribution systems.

complexity theory↗

DOE OSTI2023

2022 Review of Data-Driven Plasma Science

Not Available

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2016

TH-AB-BRB-00: Research Opportunities with Digital Linear Accelerators

Current state-of-the art digital C-arm medical linear accelerators are capable of delivering radiation treatments with high level of automation, which affords coordinated motions of gantry, couch, and multileaf collimator (MLC) with dose rate modulations. The new machine capacity has shown the potential to bring substantially improved radiation dosimetry and/or delivery efficiency to many challenging diseases. Combining an integrated beam orientation optimization algorithm with automated machine navigation, markedly improved dose conformity has been achieved using 4ρ therapy. Trajectory modulated radiation therapy (TMAT) can be used to deliver highly conformal dose to partial breast or to carve complex dose distribution for therapy involving extended volumes such as total marrow and total lymph node treatment. Dynamic electron arc radiotherapy (DEAR) not only overcomes the deficiencies of conventional electron therapy in dose conformity and homogeneity but also achieves so without patient-specific shields. The combination of MLC and couch tracking provides improved motion management of thoracic and abdominal tumors. A substantial body of work has been done in these technological advances for clinical translation. The proposed symposium will provide a timely review of these exciting opportunities. Learning Objectives: Recognize the potential of using digitally controlled linacs for clinically significant improvements in delivered dose distributions for various treatment sites. Identify existing approaches to treatment planning, optimization and delivery for treatment techniques utilizing the advanced functions of digital linacs and venues for further development and improvement. Understand methods for testing and validating delivery system performance. Identify tools available on current delivery systems for implementation and control for such treatments. Obtain the update in clinical applications, trials and regulatory approval. K. Sheng, NIH U19AI067769, NIH R43CA183390, NIH R01CA188300, Varian Medical Systems V. Yu, Varian Medical Systems, AAPM Summer Undergraduate Fellowship, NSF graduate fellowship S. Nill, Elekta AB. Cancer Research UK under Programme C33589/A19727, NIHR Biomedical Research Centre at The Royal Marsden and The Institute of Cancer Research.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

TH-AB-BRB-01: Trajectory Modulated Arc Therapy: Application to Partial Breast Irradiation

Current state-of-the art digital C-arm medical linear accelerators are capable of delivering radiation treatments with high level of automation, which affords coordinated motions of gantry, couch, and multileaf collimator (MLC) with dose rate modulations. The new machine capacity has shown the potential to bring substantially improved radiation dosimetry and/or delivery efficiency to many challenging diseases. Combining an integrated beam orientation optimization algorithm with automated machine navigation, markedly improved dose conformity has been achieved using 4ρ therapy. Trajectory modulated radiation therapy (TMAT) can be used to deliver highly conformal dose to partial breast or to carve complex dose distribution for therapy involving extended volumes such as total marrow and total lymph node treatment. Dynamic electron arc radiotherapy (DEAR) not only overcomes the deficiencies of conventional electron therapy in dose conformity and homogeneity but also achieves so without patient-specific shields. The combination of MLC and couch tracking provides improved motion management of thoracic and abdominal tumors. A substantial body of work has been done in these technological advances for clinical translation. The proposed symposium will provide a timely review of these exciting opportunities. Learning Objectives: Recognize the potential of using digitally controlled linacs for clinically significant improvements in delivered dose distributions for various treatment sites. Identify existing approaches to treatment planning, optimization and delivery for treatment techniques utilizing the advanced functions of digital linacs and venues for further development and improvement. Understand methods for testing and validating delivery system performance. Identify tools available on current delivery systems for implementation and control for such treatments. Obtain the update in clinical applications, trials and regulatory approval. K. Sheng, NIH U19AI067769, NIH R43CA183390, NIH R01CA188300, Varian Medical Systems V. Yu, Varian Medical Systems, AAPM Summer Undergraduate Fellowship, NSF graduate fellowship S. Nill, Elekta AB. Cancer Research UK under Programme C33589/A19727, NIHR Biomedical Research Centre at The Royal Marsden and The Institute of Cancer Research.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

TH-AB-BRB-02: ET-Reloaded

Current state-of-the art digital C-arm medical linear accelerators are capable of delivering radiation treatments with high level of automation, which affords coordinated motions of gantry, couch, and multileaf collimator (MLC) with dose rate modulations. The new machine capacity has shown the potential to bring substantially improved radiation dosimetry and/or delivery efficiency to many challenging diseases. Combining an integrated beam orientation optimization algorithm with automated machine navigation, markedly improved dose conformity has been achieved using 4ρ therapy. Trajectory modulated radiation therapy (TMAT) can be used to deliver highly conformal dose to partial breast or to carve complex dose distribution for therapy involving extended volumes such as total marrow and total lymph node treatment. Dynamic electron arc radiotherapy (DEAR) not only overcomes the deficiencies of conventional electron therapy in dose conformity and homogeneity but also achieves so without patient-specific shields. The combination of MLC and couch tracking provides improved motion management of thoracic and abdominal tumors. A substantial body of work has been done in these technological advances for clinical translation. The proposed symposium will provide a timely review of these exciting opportunities. Learning Objectives: Recognize the potential of using digitally controlled linacs for clinically significant improvements in delivered dose distributions for various treatment sites. Identify existing approaches to treatment planning, optimization and delivery for treatment techniques utilizing the advanced functions of digital linacs and venues for further development and improvement. Understand methods for testing and validating delivery system performance. Identify tools available on current delivery systems for implementation and control for such treatments. Obtain the update in clinical applications, trials and regulatory approval. K. Sheng, NIH U19AI067769, NIH R43CA183390, NIH R01CA188300, Varian Medical Systems V. Yu, Varian Medical Systems, AAPM Summer Undergraduate Fellowship, NSF graduate fellowship S. Nill, Elekta AB. Cancer Research UK under Programme C33589/A19727, NIHR Biomedical Research Centre at The Royal Marsden and The Institute of Cancer Research.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2016

TH-AB-BRB-03: 4n Radiotherapy

Current state-of-the art digital C-arm medical linear accelerators are capable of delivering radiation treatments with high level of automation, which affords coordinated motions of gantry, couch, and multileaf collimator (MLC) with dose rate modulations. The new machine capacity has shown the potential to bring substantially improved radiation dosimetry and/or delivery efficiency to many challenging diseases. Combining an integrated beam orientation optimization algorithm with automated machine navigation, markedly improved dose conformity has been achieved using 4ρ therapy. Trajectory modulated radiation therapy (TMAT) can be used to deliver highly conformal dose to partial breast or to carve complex dose distribution for therapy involving extended volumes such as total marrow and total lymph node treatment. Dynamic electron arc radiotherapy (DEAR) not only overcomes the deficiencies of conventional electron therapy in dose conformity and homogeneity but also achieves so without patient-specific shields. The combination of MLC and couch tracking provides improved motion management of thoracic and abdominal tumors. A substantial body of work has been done in these technological advances for clinical translation. The proposed symposium will provide a timely review of these exciting opportunities. Learning Objectives: Recognize the potential of using digitally controlled linacs for clinically significant improvements in delivered dose distributions for various treatment sites. Identify existing approaches to treatment planning, optimization and delivery for treatment techniques utilizing the advanced functions of digital linacs and venues for further development and improvement. Understand methods for testing and validating delivery system performance. Identify tools available on current delivery systems for implementation and control for such treatments. Obtain the update in clinical applications, trials and regulatory approval. K. Sheng, NIH U19AI067769, NIH R43CA183390, NIH R01CA188300, Varian Medical Systems V. Yu, Varian Medical Systems, AAPM Summer Undergraduate Fellowship, NSF graduate fellowship S. Nill, Elekta AB. Cancer Research UK under Programme C33589/A19727, NIHR Biomedical Research Centre at The Royal Marsden and The Institute of Cancer Research.

60 APPLIED LIFE SCIENCES↗