Search NASA⌕ Search

SEARCH · Search NASA

Results for “learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Safety-Related Instrumentation and Control Pilot Upgrade: Conceptual - Detailed Design Phase Report and Lessons Learned

Historically, regulatory barriers have precluded nuclear plant safety-related (SR) Instrumentation and Control (I&C) system modernizations. These barriers have been largely addressed by industry and United States Nuclear Regulatory Commission (NRC) efforts, including development of the License Amendment Request (LAR) Alternate Review (AR) process for digital I&C upgrades. The U.S. Department of Energy (DOE) and the Idaho National Laboratory (INL) are supporting a SR I&C pilot upgrade at the Limerick Generating Station (LGS) to demonstrate this process and capturing associated lessons learned for the benefit of industry. In 2021, a paper summarizing lessons learned as captured in INL/EXT-20-59809, Safety Related Instrumentation & Control Pilot Upgrade Initial Scoping Phase Implementation Report and Lessons Learned was presented at the 12th NPIC & HMIT conference. It is recommended that readers of this paper review reference 1 or the associated Nuclear Journal paper for context. This paper summarizes activities performed and lessons learned for the conceptual design phase and portions of the detailed design phase as captured in INL/RPT-23-XXXX, “Safety-Related Instrumentation and Control Pilot Upgrade: Conceptual - Detailed Design Phase Report and Lessons Learned.” The portions of the effort covered by reference 2 include conceptual design phase activities from December 2020, through completion in November 2021, and detailed design activities through September 2022. These efforts and associated detailed documentation in reference 3 are intended to support similar industry upgrades as a foundation stone for a Digital Transformation that will improve plant safety, reliability, and operational performance while lowering plant total cost of ownership.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2023

pnnl/MBDRL

Model-based Deep Reinforcement Learning for Real-time Grid Emergency Voltage Control. A model-based Deep Reinforcement Learning (DRL) framework where a deep neural network (DNN)-based surrogate model is utilized within the control policy learning framework, making the learning process faster and more sample efficient

Yin, Tim↗
DOE OSTI2025

Surrogate Modelling of 3rd Integer Resonant Extraction at Fermilab Delivery Ring

We present an ongoing work in which a surrogate model is being developed to reproduce the response dynamics of the third-integer resonant extraction process in the Delivery Ring (DR) at Fermilab. This effort is in pursuit of smoothly extracting circulating beam to the Mu2e Experiment s production target, wherein the goal is to extract a uniform slice of the circulating $1e12$ protons in the DR over 25,000 turns (43~ms). The DR contains 3 harmonic sextupoles which excite a third-integer resonance as well as three fast, tune-ramping quadrupole magnets which drive the horizontal tune towards the $29/3$ resonance. In our initial work the surrogate model trains on a semi-analytical simulation provided in the same format as live data. Using Reinforcement Learning (and other potential ML methods), the trained surrogate acts as the environment in which a simple ML control agent could learn to dynamically adjust the quadrupole ramp at 430 break points within the 43 microsecond spill window. The control agent will be hosted on a dedicated Arria 10 FPGA, introducing its own requirements on control agent architecture. In this work we report the accuracy and fidelity of surrogate models in comparison to the response dynamics of the physics simulator.

Narayanan, Aakaash [Fermilab] (ORCID:0000000157944↗
DOE OSTI2021

Human-in-the-loop Sensing and Control for Commercial Building Energy Efficiency and Occupant Comfort

Most of the existing heating, ventilation and air conditioning (HVAC) systems in commercial buildings operate in a conservative manner by assuming maximum occupancy in each room during pre-specified periods of the week, leading to significant energy being wasted as rooms are over-conditioned compared to the actual requirements of the occupants. Though critical, our understanding of occupancy patterns and thermal comfort needs of the occupants in commercial buildings is lacking and it is well known that both of these quantities are stochastic and time-varying, thus requiring sensing solutions to estimate them. This project had the goal of designing, implementing and evaluating a hardware and software solution to ameliorate this challenge. In particular, a depth camera (one whose pixels reveal distance from the camera as opposed to color values) placed on doorways is used to detect entrance and exit events from thermal zones in the building, and thereby estimate their occupancy levels. This information is then fed to a novel control algorithm that can, through interactions with the HVAC system, learn how to provide control inputs that maximize comfort and minimize energy waste. The resulting system represents a significant improvement over existing controllers for commercial HVAC systems and allowed us to improve our understanding of the design of future human-in-the-loop control solutions. For this solution to be feasible, the project had target metrics for its performance and cost. In particular, entrance and exit events for occupants moving about the building would need to be detected with an accuracy higher than 97%; and the resulting control inputs derived from this information would need to lead to approximately 10% energy savings compared to a schedule-based controller. Furthermore, regarding the final hardware design, the project had a target bill of materials (BOM) cost for the sensing solution of less than US$200 per unit while using less than 25W of power on average. All of these target metrics were met or exceeded by our final proposed solution. We performed evaluations by deploying the system in over 20 rooms of different types across 6 commercial buildings in Pittsburgh, PA over the course of three years, and performing targeted controlled experiments to test its performance along the different metrics. The human-in-the-loop control solutions (both hardware and software) developed through this project are expected to lead to significant improvements in the comfort and energy efficiency of HVAC systems used in commercial buildings. The insights we developed through the project pave the way to HVAC systems that can condition interior spaces according to their real-time utilization and the thermal comfort needs of the occupants, thereby reducing energy use. They also open up a new learning-based way of configuring HVAC controllers without having to manually fine-tune them for each building. These innovations can significantly increase the adoption of novel control solutions by the industry and thereby save resources and reduce costs of operation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Machine learning-based ethylene concentration estimation, real-time optimization and feedback control of an experimental electrochemical reactor

With the increase in electricity supply from clean energy sources, electrochemical reduction of carbon dioxide (CO 2 ) has received increasing attention as an alternative source of carbon-based fuels. As CO 2 reduction is becoming a stronger alternative for the clean production of chemicals, the need to model, optimize and control the electrochemical reduction of the CO 2 process becomes inevitable. However, on one hand, a first-principles model to represent the electrochemical CO 2 reduction has not been fully developed yet because of the complexity of its reaction mechanism, which makes it challenging to define a precise state-space model for the control system. On the other hand, the unavailability of efficient concentration measurement sensors continues to challenge our ability to develop feedback control systems. Gas chromatography (GC) is the most common equipment for monitoring the gas product composition, but it requires a period of time to analyze the sample, which means that GC can provide only delayed measurements during the operation. Moreover, the electrochemical CO 2 reduction process is catalyzed by a fast-deactivating copper catalyst and undergoes a selectivity shift from the product-of-interest at the later stages of experiments, which can pose a challenge for conventional control methods. To this end, machine learning (ML) techniques provide a potential approach to overcome those difficulties due to their demonstrated ability to capture the dynamic behavior of a chemical process from data. Motivated by the above considerations, we propose a machine learning-based modeling methodology that integrates support vector regression and first-principles modeling to capture the dynamic behavior of an experimental electrochemical reactor; this model, together with limited gas chromatography measurements, is employed to predict the evolution of gas-phase ethylene concentration. The model prediction is directly used in a proportional-integral (PI) controller that manipulates the applied potential to regulate the gas-phase ethylene concentration at energy-optimal set-point values computed by a real-time process optimizer (RTO). Specifically, the RTO calculates the operation set-point by solving an optimization problem to maximize the economic benefit of the reactor. Finally, suitable compensation methods are introduced to further account for the experimental uncertainties and handle catalyst deactivation. The proposed modeling, optimization, and control approaches are the first demonstration of active control for a CO 2 electrolyzer and contribute to the automation and scale-up efforts for electrified manufacturing of fuels and chemicals starting from CO 2 .

42 ENGINEERING↗
DOE OSTI2020

Assessing three closed-loop learning algorithms by searching for high-quality quantum control pulses

Designing a high-quality control is crucial for reliable quantum computation. Among the existing approaches, closed-loop leaning control is an effective choice. Its efficiency depends on the learning algorithm employed, thus deserving algorithmic comparisons for its practical applications. Here we assess three representative learning algorithms, including GRadient Ascent Pulse Engineering (GRAPE), improved Nelder-Mead (NMplus), and Differential Evolution (DE), by searching for high-quality control pulses to prepare the Bell state. We first implement each algorithm experimentally in a nuclear magnetic resonance system and then conduct a numerical study considering the impact of some possible significant experimental uncertainties. The experiments report the successful preparation of the high-fidelity target state by the three algorithms, while NMplus converges fastest, and these results coincide with the numerical simulations when potential uncertainties are negligible. However, under certain significant uncertainties, these algorithms possess distinct performance with respect to their resulting precision and efficiency, and DE shows the best robustness. Finally, this study provides insight to aid in the practical application of different closed-loop learning algorithms in realistic physical scenarios.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Hierarchical Speed Planner for Automated Vehicles: A Framework for Lagrangian Variable Speed Limit in Mixed-Autonomy Traffic

Here, this article presents a novel hierarchical speed planning framework for variable speed limits in mixed-autonomy traffic environments, leveraging server-side macroscopic control and vehicle-side microscopic execution. The framework integrates real-time traffic state estimation (TSE) and reinforcement learning (RL)-based control to mitigate congestion and improve traffic flow. A TSE enhancement module combines macroscopic data from sources like INRIX with high-resolution observations from connected autonomous vehicles (CAVs), enabling predictive modeling to address latency and noise. The target speed design module employs kernel smoothing and a buffer zone strategy to optimize traffic density and flow around bottlenecks. The proposed system was validated in the largest open-road test to date with 100 CAVs, demonstrating an overall 8% traffic density decrease, with a specific decrease of 7% upstream, 10% downstream, and a 52% decrease during the congestion formation phase at bottlenecks.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Optimal Control of Differentially Private EV Charging: A Scalable Learning Approach Under Uncertainty

Internet of Things (IoT)-enabled electric vehicles (IoEVs) enable intelligent charging coordination that accounts for grid congestion. However, increased data exchange raises privacy concerns, as charging patterns can reveal sensitive driver behavior to grid operators. Here, we propose a differentially private (DP) EV charging framework that enables coordinated control while protecting driver data with theoretical privacy guarantees. Nevertheless, integrating DP inevitably introduces uncertainty into the control strategy for EVs, which can lead to infeasible solutions. To tackle this challenge, we develop a feasible and scalable control algorithm based on constrained reinforcement learning (CRL) and convex hulls. While our framework is designed to handle the uncertainty introduced by DP, it is general and also applicable to other sources of uncertainty in EV charging, such as the stochastic nature of driver behavior and renewable variability. This ensures feasible and privacy-preserving coordination of EV charging at scale. Our method constructs convex hulls within the action space to guarantee feasibility under stochastic constraints and incorporates constraint reduction techniques to improve scalability. Case studies based on IEEE benchmark systems demonstrate that the proposed approach effectively balances feasibility under uncertainty, scalability, and privacy in large-scale EV charging control.

Engineering - Power transmission and distribution↗
DOE OSTI2021

Design of nematic liquid crystals to control microscale dynamics

The dynamics of small particles, both living such as swimming bacteria and inanimate, such as colloidal spheres, has fascinated scientists for centuries. If one could learn how to control and streamline their chaotic motion, that would open technological opportunities in the transformation of stored or environmental energy into systematic motion, with applications in micro-robotics, transport of matter, guided morphogenesis. This review presents an approach to command microscale dynamics by replacing an isotropic medium with a liquid crystal. Orientational order and associated properties, such as elasticity, surface anchoring, and bulk anisotropy, enable new dynamic effects, ranging from the appearance and propagation of particle-like solitary waves to self-locomotion of an active droplet. By using photoalignment, the liquid crystal can be patterned into predesigned structures. In the presence of the electric field, these patterns enable the transport of solid and fluid particles through nonlinear electrokinetics rooted in anisotropy of conductivity and permittivity. Director patterns command the dynamics of swimming bacteria, guiding their trajectories, polarity of swimming, and distribution in space. This guidance is of a higher level of complexity than a simple following of the director by rod-like microorganisms. Namely, the director gradients mediate hydrodynamic interactions of bacteria to produce an active force and collective polar modes of swimming. The patterned director could also be engraved in a liquid crystal elastomer. When an elastomer coating is activated by heat or light, these patterns produce a deterministic surface topography. The director gradients define an activation force that shapes the elastomer in a manner similar to the active stresses triggering flows in active nematics. Finally, the patterned elastomer substrates could be used to define the orientation of cells in living tissues. The liquid-crystal guidance holds a major promise in achieving the goal of commanding microscale active flows.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Risk-Constrained Reinforcement Learning for Inverter-Dominated Power System Controls

Here, this paper develops a risk-aware controller for grid-forming inverters (GFMs) to minimize large frequency oscillations in GFM inverter-dominated power systems. To tackle the high variability from loads/renewables, we incorporate a mean-variance risk constraint into the classical linear quadratic regulator (LQR) formulation for this problem. The risk constraint aims to bound the time-averaged cost of state variability and thus can improve the worst-case performance for large disturbances. The resulting risk-constrained LQR problem is solved through the dual reformulation to a minimax problem, by using a reinforcement learning (RL) method termed as stochastic gradient-descent with max-oracle (SGDmax). In particular, the zero-order policy gradient (ZOPG) approach is used to simplify the gradient estimation using simulated system trajectories. Numerical tests conducted on the IEEE 68-bus system have validated the convergence of our proposed SGDmax for GFM model and corroborate the effectiveness of the risk constraint in improving the worst-case performance while reducing the variability of the overall control cost.

Frequency control↗
DOE OSTI2022

Safety-Related Instrumentation and Control Pilot Upgrade: Initial Scoping Phase Implementation and Lessons Learned

In May 2016, the U.S. Nuclear Regulatory Commission (NRC) staff provided a digital instrumentation and control (I&C) regulatory infrastructure integrated action plan to the NRC for approval. One of the objectives of that plan was to establish a clear regulatory structure with reduced regulatory uncertainty to enable the expanded safe use of digital I&C in commercial nuclear reactors while continuing to ensure safety and security. To achieve this end, the NRC, with collaboration from industry, developed a streamlined License Amendment Request Alternate Review (AR) process for safety-related (SR) digital I&C upgrades. In spite of this effort, the industry has remained reluctant to perform such I&C upgrades because of perceived regulatory and financial risks associated with being the first or an early adopter of the AR process for SR I&C upgrades. The U.S. Department of Energy Light Water Reactor Sustainability Program at the Idaho National Laboratory performed Initial Scoping Phase research to help break this impasse by supporting a SR I&C Pilot Upgrade, working with MPR Associates, ScottMadden Inc., and Exelon Generation. Exelon’s Limerick Generating Station (LGS) was selected as the target for this research. This paper summarizes the Initial Scoping Phase engineering and operations, licensing, and project management activities necessary to bound the scope, schedule, and estimated cost of the project sufficiently to enable utility management authorization of Conceptual Design Phase activities. These efforts and associated products are intended to provide a template to support larger industry efforts to perform similar upgrades as a foundation stone for a digital transformation that will improve plant safety, reliability, and operational performance while lowering plant total cost of ownership. As a result of the combined effort of Exelon Generation and research participants, Conceptual Design Phase activities for the subject upgrade at LGS were approved by Exelon. Further, the U.S. Department of Energy also awarded a $50 million cost share award to Exelon in order to pave the way for SR I&C modernization and associated control room upgrades across the U.S. nuclear fleet. Additional research reports are planned for the Conceptual Design Phase, Detailed Design Phase, and the Implementation Phase of the LGS project to document the process followed and promulgate lessons learned to industry.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Model-Free Voltage Control of Active Distribution System with PVs Using Surrogate Model-Based Deep Reinforcement Learning

Accurate knowledge of the distribution system topology and parameters is required to achieve good voltage control performance, but this is difficult to obtain in practice. This paper proposes a physical-model-free voltage control method based on a surrogate-model-enabled deep reinforcement learning approach. Specifically, a surrogate model is trained in a supervised manner using the recorded limited number of historical data to learn the relationship between the power injections and voltage fluctuations of each node. Then, the deep reinforcement learning algorithm is applied to learn an optimal control strategy from the experiences obtained by continuous interactions with the surrogate model. The proposed method can achieve physical-model-free control of unbalanced distribution network and inform real-time decisions to deal with fast voltage fluctuations caused by the rapid variation of PV generation. Simulation results on an unbalance IEEE 123-bus system show that the proposed method can achieve similar performance as that of perfect physical-model-based approaches while being advantageous over other traditional methods.

active distribution network↗
DOE OSTI2020

Uranium Mill Tailings Radiation Control Act Title II DOE Due Diligence and Lessons Learned from a Previous Site Transfer - 20352

Title II of the Uranium Mill Tailings Radiation Control Act (UMTRCA) established that a government agency will provide perpetual care for closed uranium and thorium ore-processing sites that were operating under an NRC source material license in 1978 or were licensed thereafter. Commercial owners (licensees) operating under an NRC or agreement state specific license when UMTRCA was passed are responsible for conducting reclamation of any byproduct material remaining from uranium-ore processing operations in accordance with an NRC or agreement state approved reclamation plan. Reclamation includes both surface and groundwater remedies. Upon completion of reclamation and approval by NRC, the site is required to be transferred to either the host state or the DOE for long-term surveillance and maintenance. Since UMTRCA's enactment, six Title II sites have been transferred to DOE; an additional 24 Title II sites are anticipated to be transferred before 2050. DoE's role mandated under UMTRCA Title II as the long-term care custodian is to perform 'monitoring, maintenance, and emergency measures necessary to protect the public health and safety.' UMTRCA requires that the licensee pay a long-term surveillance charge 'sufficient to cover the annual costs of site surveillance.' However, at some sites such as the Bluewater, New Mexico, Disposal Site, this mandate has required additional effort and expense by DOE, beyond the originally anticipated and intended scope within UMTRCA, but within the authority of DOE under UMTRCA. In 1997, the Bluewater site became the second UMTRCA Title II site to be transferred to DOE. The site was the location of a uranium mill operated from 1953 until 1982. The specific licensee began site reclamation in 1991, and by 1995 all tailings and contaminated materials were encapsulated in two tailings disposal cells and other disposal areas. In addition to surface contamination, milling activities impacted groundwater in the two upper aquifers. In 1989, the specific licensee attempted active groundwater remediation; however, no significant reduction in contaminant concentrations was observed. As a result, the specific licensee applied to NRC for alternate concentration limits (ACLs) in 1990, which were approved in 1996 as being protective, after additional corrective actions were performed. Since transfer of the Bluewater site to DOE, unforeseen challenges have occurred, requiring additional actions. The first challenge is the occurrence of surface depressions located on the northern section of the main tailings disposal cell. The depressions were first observed during DoE's initial inspection in 1998; however, evidence of these can be observed on satellite images taken prior to transfer. Since being first observed, the depressions have continued to grow both in depth and areal extent. Due to the design of the main tailings disposal cell, the depressions impede storm water from being effectively shed off the 101- hectare (250-acre) top slope of the main tailings disposal cell. Instead, storm water accumulates in the depressions, forming a large ephemeral pond that has stored up to 16.3 x 10{sup 6} liters (4.3 million gallons) of stormwater. The ponding poses a potential risk to the integrity of the main tailings disposal cell in the case of a large storm event, with the potential to cause the pond to overtop and erode the cover material and underlying waste. DOE has taken a number of short-term actions to monitor, measure, and reduce the ponding and is currently working with the US Army Corps of Engineers to design and construct a repair. Additional challenges are associated with groundwater at the Bluewater site. Nine wells were present on the 1335-hectare (3300-acre) site upon transfer. Groundwater compliance was called into question after the State of New Mexico reduced its uranium groundwater standard from 5.0 to 0.03 milligrams per liter in 2004, and when an ACL for uranium was exceeded in a site monitoring well in 2010. Acquiring historical groundwater data and subsequent evaluations as well as additional DOE groundwater monitoring led to installing 10 new monitoring wells and performing additional site hydrogeology recharacterization. DOE continues to evaluate groundwater conditions at the site and works with the NRC to determine regulatory requirements and a path forward. As a result of lessons learned at the Bluewater site and other Title II sites, improved processes have been implemented at a programmatic level to increase due diligence before site transfer and prevent similar issues from occurring at other UMTRCA Title II sites under long-term management. DoE's due diligence process is documented in the Process for Transition of UMTRCA Title II Disposal Sites to DOE for Long-Term Surveillance and Maintenance and is designed to ensure that DOE has no technical or compliance concerns with regulatory decisions that might compromise protectiveness following site transfer to DOE. Implementation of the due diligence process has increased DoE's role prior to site transfer and has been effective in identifying potential issues. Actions by NRC and specific licensees, in response to enhanced due diligence efforts by DOE, are expected to minimize, if not totally prevent, the need for unanticipated actions by DOE pertaining to the surface and groundwater remedies after site transfer. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2026

Steady Spills, Stronger Signals: Machine Learning for Slow Spill Analysis

Particle accelerator experiments rely on stable, consistent proton beams to maximize scientific discovery. This presentation introduces beam spills, duty factor, and beam stability using a meteor shower analogy before exploring how feedback control and machine learning, including recurrent neural networks (RNNs), can analyze spill data, identify patterns, and predict beam behavior. Together, these approaches support beam optimization and improve our understanding of accelerator performance for experiments such as Mu2e.

Prescott, Matthew J. [Fermilab]↗
DOE OSTI2021

Improve Learning from Crowds via Generative Augmentation

Crowdsourcing provides an efficient label collection schema for supervised machine learning. However, to control annotation cost, each instance in the crowdsourced data is typically annotated by a small number of annotators. This creates a sparsity issue and limits the quality of machine learning models trained on such data. In this paper, we study how to handle sparsity in crowdsourced data using data augmentation. Specifically, we propose to directly learn a classifier by augmenting the raw sparse annotations. We implement two principles of high-quality augmentation using Generative Adversarial Networks: 1) the generated annotations should follow the distribution of authentic ones, which is measured by a discriminator; 2) the generated annotations should have high mutual information with the ground-truth labels, which is measured by an auxiliary network. Extensive experiments and comparisons against an array of state-of-the-art learning from crowds methods on three real-world datasets proved the effectiveness of our data augmentation framework. It shows the potential of our algorithm for low-budget crowdsourcing in general.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2024

Resilient Control of Networked Microgrids Using Vertical Federated Reinforcement Learning: Designs and Real-Time Test-Bed Validations

Improving system-level resiliency of networked microgrids against adversarial cyber-attacks is an important aspect in the current regime of increased inverter-based resources (IBRs). To achieve that, this paper contributes in designing a hierarchical control layer, in conjunction with the existing control layers, resilient to adversarial attack signals. Considering model complexities, unknown dynamical behaviors of IBRs, and privacy issues regarding data sharing in multi-party-owned microgrids, designing such a control layer is non-trivial. Here, to tackle these issues, a novel federated reinforcement learning (Fed-RL) method is proposed. To grasp the interconnected dynamics of networked microgrids, the paper develops Federated Soft Actor-Critic (FedSAC) algorithm following the vertical structure of implementing Fed-RL. Next, utilizing the OpenAI Gym interface, we built a custom set-up in GridLAB-D/HELICS co-simulation platform, named Resilient RL Co-simulation (ResRLCoSIM), to train the RL agents with IEEE 123-bus benchmark comprising 3 interconnected microgrids. Finally, the learned policies in the simulation are transferred to the real-time hardware-in-the-loop (HIL) test-bed developed using the high-fidelity Hypersim platform. Finally, experiments show that the simulator-trained RL controllers achieve desirable performance with the test-bed platform, validating the minimization of the sim-to-real gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Traffic Control via Connected and Automated Vehicles (CAVs): An Open-Road Field Experiment with 100 CAVs

The CIRCLES project aims to reduce instabilities in traffic flow, which are naturally occurring phenomena due to human driving behavior. Also called “phantom jams” or “stop-and-go waves,” these instabilities are a significant source of wasted energy. Toward this goal, the CIRCLES project designed a control system, referred to as the MegaController by the CIRCLES team, that could be deployed in real traffic. Our field experiment, the MegaVanderTest (MVT), leveraged a heterogeneous fleet of 100 longitudinally controlled vehicles as Lagrangian traffic actuators, each of which ran a controller with the architecture described in this article. The MegaController is a hierarchical control architecture that consists of two main layers. The upper layer is called the Speed Planner and is a centralized optimal control algorithm. It assigns speed targets to the vehicles, conveyed through the LTE cellular network. The lower layer is a control layer, running on each vehicle. It performs local actuation by overriding the stock adaptive cruise controller, using the stock onboard sensors. The Speed Planner ingests live data feeds provided by third parties as well as data from our own control vehicles and uses both to perform the speed assignment. The architecture of the Speed Planner allows for the modular use of standard control techniques, such as optimal control, model predictive control (MPC), kernel methods, and others. The architecture of the local controller allows for the flexible implementation of local controllers. Corresponding techniques include deep reinforcement learning (RL), MPC, and explicit controllers. Depending on the vehicle architecture, all onboard sensing data can be accessed by the local controllers or only some. Likewise, control inputs vary across different automakers, with inputs ranging from torque or acceleration requests for some cars to electronic selection of adaptive cruise control (ACC) setpoints in others. The proposed architecture technically allows for the combination of all possible settings proposed previously, that is {Speed Planner algorithms} × {local Vehicle Controller algorithms} × {full or partial sensing} × {torque or speed control}. As a result, most configurations were tested throughout the ramp up to the MegaVandertest (MVT).

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗