Search NASA⌕ Search

SEARCH · Search NASA

Results for “actuated signal control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Traffic Signal Optimization by Integrating Reinforcement Learning and Digital Twins

Machine learning (ML) methods, especially reinforcement learning (RL), have been widely considered for traffic signal optimization in intelligent transportation systems. Most of these ML methods are centralized, lacking in scalability and adaptability in large traffic networks. Further, it is challenging to train such ML models due to the lack of training platforms and/or the cost of deploying and training in a real traffic networks. This paper presents an approach for the integration of decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin (DT) to optimize traffic signals for the reduction of traffic congestion and network-wide fuel consumption related to stopping. Specifically, the DGMARL agents learn traffic state patterns and make decisions regarding traffic signal control with assistance from a Digital Twin module, which simulates and replicates the traffic behaviors of a real traffic network. The proposed approach was evaluated using PTV-Vissim [1], a microscopic traffic simulation platform. PTV-Vissim is also the simulation engine of the DT, enabling emulation and optimization of the traffic signals on the MLK Smart Corridor in Chattanooga, Tennessee. Compared to an actuated signal control baseline approach, experiment results show that Eco_PI, a developed performance measure capturing the impact of stops on fuel consumption, was reduced by 44.27% in a 24-hour and an average of 29.88% in a PM-peak-hour scenario.

Multi-Agent Reinforcement Learning, Digital Twin, ↗

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

actuated signal control↗

Integration of Decentralized Graph-Based Multi-Agent Reinforcement Learning with Digital Twin for Traffic Signal Optimization

Machine learning (ML) methods, particularly Reinforcement Learning (RL), have gained widespread attention for optimizing traffic signal control in intelligent transportation systems. However, existing ML approaches often exhibit limitations in scalability and adaptability, particularly within large traffic networks. This paper introduces an innovative solution by integrating decentralized graph-based multi-agent reinforcement learning (DGMARL) with a Digital Twin to enhance traffic signal optimization, targeting the reduction of traffic congestion and network-wide fuel consumption associated with vehicle stops and stop delays. In this approach, DGMARL agents are employed to learn traffic state patterns and make informed decisions regarding traffic signal control. The integration with a Digital Twin module further facilitates this process by simulating and replicating the real-time asymmetric traffic behaviors of a complex traffic network. The evaluation of this proposed methodology utilized PTV-Vissim, a traffic simulation software, which also serves as the simulation engine for the Digital Twin. The study focused on the Martin Luther King (MLK) Smart Corridor in Chattanooga, Tennessee, USA, by considering symmetric and asymmetric road layouts and traffic conditions. Comparative analysis against an actuated signal control baseline approach revealed significant improvements. Experiment results demonstrate a remarkable 55.38% reduction in Eco_PI, a developed performance measure capturing the cumulative impact of stops and penalized stop delays on fuel consumption, over a 24 h scenario. In a PM-peak-hour scenario, the average reduction in Eco_PI reached 38.94%, indicating the substantial improvement achieved in optimizing traffic flow and reducing fuel consumption during high-demand periods. These findings underscore the effectiveness of the integrated DGMARL and Digital Twin approach in optimizing traffic signals, contributing to a more sustainable and efficient traffic management system.

42 ENGINEERING↗

Traffic Signal Control With Adaptive Online-Learning Scheme Using Multiple-Model Neural Networks

This article proposes a new traffic signal control algorithm to deal with unknown-traffic-system uncertainties and reduce delays in vehicle travel time. Unknown-traffic-system dynamics are approximated using a recurrent neural network (NN). To accurately identify the traffic system model, an online-learning scheme is developed to switch among a set of candidate NNs (i.e., multiple-model NNs) based on their estimation errors. Then, a bank of optimal signal-timing controllers is designed based on the online identification of the traffic system. Simulation studies have been carried out for the obtained control strategies using multiple-model NNs, and the desired results have been obtained. Moreover, compared with the widely used actuated traffic signal control schemes, it is shown that the proposed method can reduce vehicle travel delays and improve traffic system robustness.

99 GENERAL AND MISCELLANEOUS↗

NEMA-Phase Compliant Traffic Signal Controller Module in SUMO

The controller modules in SUMO use a stage-based control structure. A phase is defined as a stage of all allowed movements at a time instance. However, traffic signal controllers used in North America widely use National Electrical Manufacturers Association (NEMA) phase definition. A NEMA phase is defined by a certain flow movement at an intersection. At one time, more than one NEMA phase could happen together as long as they do not conflict with each other. We can visualize the NEMA phases and timings in Ring-and-Barrier structured NEMA diagrams. For one controller, only one phase from a ring can be activated at a time. Phases from different rings could be activated together as long as they are not from the different sides of a barrier. When a controller is operated in fixed-time control mode, we can model the NEMA phase timing as a corresponding stage-based control timing without any issues. When introducing actuation into the signal control, a Ring-and-Barrier structured traffic signal controller can be more flexible than stage-based controller by allowing different possible phase combinations. We made two efforts in modeling Ring-and-Barrier structured controllers in SUMO. One is to translate a NEMA phases timing into SUMO-readable phases and timings as an additional file for SUMO. This translation worked well for fixed-time control. To model actuated control and coordinated actuated control, we augmented the SUMO source code by adding a Ring-and-Barrier structured controller module. This module could implement traffic signal timing from controllers using NEMA phases. We also augmented TraCI to be able to set new NEMA phase timings during simulations. We examined the Ring-and-Barrier structured traffic signal controller module by both visually observing the simulation animations and the simulation records. The developed control module can model the generalized Ring-and-Barrier structured traffic signal timing that is used in North America. SEE: https://github.com/eclipse/sumo/blob/main/src/microsim/traffic_lights/NEMAController.cpp

Wang, Qichao↗

Automated Control for Nuclear Thermal Propulsion Start-Up using MOOSE-based Applications

This report presents a Griffin/Bison/RELAP-7 numerical model of a prototypical NTP system that features fuel assemblies arranged in rings, and which was designed to simulate rapid startup transients. The physics modeled include full-core neutronics, assembly-wise heat conduction, and conjugate heat transfer, with the balance of plant mainly imposed through boundary conditions. In addition, various forms of automated reactivity control were deployed by using the MOOSE to autonomously drive the model and simulate the reactor transitioning from assumed initial conditions to nominal power in a fraction of a minute. To generate the cross-sections of the neutronics model, and in an effort to simultaneously account for the tremendous axial temperature gradients in the reactor and to limit the number of state points required for cross-section generation, the average component temperatures and hydrogen densities in the cooling channels were correlated to the average fuel and moderator temperatures, and fixed axial profiles were derived for nominal conditions and then used during the transient. With this approximation, a tractable cross-section library tabulated with fuel/moderator temperatures and CD angles was generated using Serpent. The full-core SPH correction procedure and the CD decusping technology in Griffin, respectively, ensure preservation of the multiplication factor and reaction rates at state points, along with a reasonably accurate reactivity worth between tabulated CD angles, despite using a coarse mesh. Feedback from other physics was calculated by modeling one representative fuel assembly per ring, along with the corresponding fuel and moderator cooling channels. To limit power overshoots during startup, another layer of multiphysics coupling was added to the model in order to automatically control the drums. Two different technologies presented herein showed outstanding performance in this regard: (1) a novel hybrid PID controller based on both power and reactivity signals, and (2) a PGC that relies on kinetics parameters and reactivity coefficients to predict future behavior and adjust the desired signal accordingly. A challenging benchmark was devised, featuring a power demand curve that exponentially increases by a factor of 500 within 30 seconds, then levels out after that. Both control approaches create a simulated power curve that closely follows the power demand curve and limits power overshoots to 1% or less. While the former approach requires more tuning of the internal parameters, the latter requires additional knowledge of the reactivity feedback coefficients and rates of change of the corresponding variables, including fuel and moderator temperature, which could be difficult to dynamically measure for a real NTP system. Fortunately, some inaccuracy in these quantities will not drastically degrade the PGC performance. Subsequently, a more realistic startup sequence was considered, in which the mass flow rate and outlet pressures are ramped up to model bootstrap and thrust build-up phases prior to reaching steady-state conditions, demonstrating the ability of the hybrid PID and PGCs to handle such transients, with both types of controllers exhibiting very similar behavior. Nevertheless, a significant chamber temperature overshoot was observed, caused by the demanded power signal and assumed mass flow rate. This issue could be mitigated by deploying a reactor controller that follows the chamber temperature signal and actuates both the control valves and drums (rather than using a power signal based solely on the drums to control reactivity). Enhancement of the hydrogen fluid properties available in MOOSE, as well as a better understanding of prototypical initial conditions, are also needed to further enhance this startup model. Finally, a study was performed to model decay heat post-shutdown, and to prepare for extending this model to predict shutdown behavior and post-shutdown pulsed cooling requirements.

33 ADVANCED PROPULSION SYSTEMS↗

Cyber-Physical Smart Light Control System Integration with Smart Grid using Zigbee

This paper presents a hardware-in-the-loop cyber-physical system architecture design to monitor and control smart lights connected to the active distribution grid. The architecture uses Zigbee-based (IEEE 802.15.4) wireless sensor networks and publish-subscribe architecture to exchange monitoring and control signals between smart-light actuators (SLAs) and a smart-light central controller (SLCC). Each SLA integrated into a smart light consists of a Zigbee-based endpoint module to send and receive signals to and from the SLCC. The SLCC consists of a Zigbee-based coordinator module, which further exchanges the monitoring and control signals with the active distribution management system over the TCP/IP communication network. The monitoring signals from the SLAs include light status, brightness level, voltage, current, and power data, whereas, the control signals to the SLAs include light intensity, turn ON, turn OFF, standby, and default settings. We have used our existing hardware-in-the-loop (HIL) cyber-physical system (CPS) security SCADA testbed to process signals received from the SLCC and respond suitable control signals based on the smart light schedule requirements, system operation, and active distribution grid dynamic characteristics. We have integrated the proposed cyber-physical smart light control system (CPSLCS) testbed to our existing HIL CPS SCADA testbed. We use the integrated testbed to demonstrate the efficacy of the proposed algorithm by real-time performance and latency between the SLCC and SLAs. The experiments demonstrated significant results by 100% realtime performance and low latency while exchanging data between the SLCC and SLAs.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Demonstration of reconstruction-free static magnetic control of DIII-D plasma with deep reinforcement learning

This paper presents the development and experimental validation of a reinforcement learning (RL)-based magnetic controller on the DIII-D tokamak. The controller directly maps raw magnetic diagnostic signals to actuator commands, replacing the traditional isoflux control algorithm based on equilibrium reconstruction. Four RL controllers are trained using the Soft Actor–Critic algorithm with an asymmetric Actor–Critic architecture in the NSFsim simulator. All controllers are deployed in the DIII-D Plasma Control System and operated with a 4 kHz feedback loop. Two randomization strategies are evaluated during training: evolving kinetic profiles and fixed kinetic profiles within each episode. The latter approach is found to better capture experimental deviations in the current density profile and to provide overall improved control performance. Robust operation is demonstrated across heating power scans in both L- and H-mode plasmas, as well as during transient events such as L–H transitions and pellet injections. Control errors in plasma shape and radial position remained within 1.5–2.0 cm and 1 cm, respectively. A notable discrepancy was observed in the vertical X-point position, with errors of up to approximately 4 cm, attributed to the current density distribution mismatches between simulations and experiments.

DIII-D↗

Controlling optical-cavity locking using reinforcement learning

Abstract This study applies an effective methodology based on Reinforcement Learning to a control system. Using the Pound–Drever–Hall locking scheme, we match the wavelength of a controlled laser to the length of a Fabry-Pérot cavity such that the cavity length is an exact integer multiple of the laser wavelength. Typically, long-term drift of the cavity length and laser wavelength exceeds the dynamic range of this control if only the laser’s piezoelectric transducer is actuated, so the same error signal also controls the temperature of the laser crystal. In this work, we instead implement this feedback control grounded on Q-Learning. Our system learns in real-time, eschewing reliance on historical data, and exhibits adaptability to system variations post-training. This adaptive quality ensures continuous updates to the learning agent. This innovative approach maintains lock for eight days on average.

47 OTHER INSTRUMENTATION↗

Wide-Bandgap Semiconductor Amplifiers for Fusion Plasma Heating and Control

This paper discusses power electronics developed under the ARPA-E GAMOW program to support nuclear fusion power production. The goal of this project was to develop and assess the potential for wide-bandgap (WBG) semiconductor devices in power electronics to enable high-efficiency and high-voltage solid-state systems for fusion plasma generation, heating, and control. The power electronics use an architecture in which multiple high-power boards can be combined to produce megawatt-level power, where using multiple boards provides high reliability. Two main areas of power electronics boards are developed in this project for fusion plasma heating and control applications: (1) pulse generation and control and (2) radiofrequency generation. The first area is for boards capable of driving high-voltage millisecond pulses at high duty cycles. The envisioned application of these pulses is in plasma control of magnetohydrodynamic instabilities, plasma position, and edge-localized modes. Pulse-width modulation allows for the implementation of a wide variety of linear and nonlinear control systems. The boards developed for this project could actuate control coils based on digital input signals and can be parallelized to provide megawatts of output power. The design of the pulse generator is a low-side load switch. A load switch was designed and constructed that utilized 2-kV-rated field-effect transistor (FET)-based cascodes developed by Qorvo under this project to perform initial testing of these cascodes. The second area is being implemented using class E amplifiers with WBG devices and a reactance steering network to handle inductive or capacitive plasma loads. Applications include ion cyclotron resonance heating (ICRH) and high-harmonic fast-wave (HHFW) heating. A class E reactance steering network is demonstrated in modeling and experiment with a resistive-inductive load that models an inductively-coupled plasma. Power combining of boards with class E reactance steering networks is also simulated and demonstrated experimentally, to enable scaling up to high power. Modeling of high-power-density cooling and remaining useful life is conducted to enable reliable, effectively cooled high-power electronics for fusion applications.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Feedback Control Strategy for Transient Stability Application

Power systems are subjected to a wide range of disturbances during daily operations. Severe disturbances, such as a loss of a large generator, a three-phase bolted fault on a generator bus, or a loss of a transmission line, can lead to the loss of synchronism of a generator or group of generators. The ability of a power system to maintain synchronism during the few seconds after being subjected to a severe disturbance is known as transient stability. Most of the modern methods of controlling transient stability involve special protection schemes or remedial action schemes. These special protection schemes sense predetermined system conditions and take corrective actions, such as generator tripping or generation re-dispatch, in real time to maintain transient stability. Another method is the use of a real-time feedback control system to modulate the output of an actuator in response to a signal. This paper provides a fundamental evaluation of the use of feedback control strategies to improve transient stability in a power system. An optimal feedback control strategy that modulates the real power injected and absorbed by distributed energy-storage devices is proposed. Its performance is evaluated on a four-machine power system and on a 34-machine reduced-order model of the Western North American Power System. The result shows that the feedback control strategy can increase the critical fault clearing time by 60%, thereby improving the transient stability of the power system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Electrogenetic signaling and information propagation for controlling microbial consortia via programmed lysis

To probe signal propagation and genetic actuation in microbial consortia, we have coopted the components of both redox and quorum sensing (QS) signaling into a communication network for guiding composition by “programming” cell lysis. Here, in this paper, we use an electrode to generate hydrogen peroxide as a redox cue that determines consortia composition. The oxidative stress regulon of Escherichia coli, OxyR, is employed to receive and transform this signal into a QS signal that coordinates the lysis of a subpopulation of cells. We examine a suite of information transfer modalities including “monoculture” and “transmitter-receiver” models, as well as a series of genetic circuits that introduce time-delays for altering information relay, thereby expanding design space. A simple mathematical model aids in developing communication schemes that accommodate the transient nature of redox signals and the “collective” attributes of QS signals. We suggest this platform methodology will be useful in understanding and controlling synthetic microbial consortia for a variety of applications, including biomanufacturing and biocontainment.

59 BASIC BIOLOGICAL SCIENCES↗

Fuel-Based Nash Bargaining Approach for Adaptive Signal Control in an N -Player Cooperative Game

This paper presents a fuel-based game-theoretic approach for adaptive signal control. Our controller applies Nash bargaining (NB) in an n-player cooperative game to identify optimal phasing splits considering future traffic demands. The fuel-based NB controller utilizes an objective function that combines operational measures (delays and stops) with fuel consumption measures at intersections. The proposed controller was encoded in Python and then implemented and evaluated in a VISSIM microscopic traffic simulation model in an intersection with increasing volumes. The performance of the NB controller was compared to a traditional actuated control as the baseline. The results show that the NB controller was able to achieve superior environmental gains with a 17% saving in fuel consumption and a 17% reduction in CO emissions. In addition, the proposed controller was capable of maintaining acceptable operational conditions as it achieved a 20% reduction in delay, 8% reduction in the number of stops, and 8% reduction in queue lengths compared to the actuated controller. Compared to similar studies that applied NB for adaptive signal control, our fuel-based NB controller stands out as a promising approach to significantly improve fuel consumption at signalized intersections.

Engineering↗

Next-Generation Modular Flexible Low-Cost SiC-based High-Frequency-Link Transformer (Final Technical Report)

This project, led by the small business NextWatt LLC, led to the design and development of a novel high-frequency-link (HFL) transformer that yields reduce number of semiconductor devices compared to conventional solid-state transformers (SSTs). The HFL transformer is based on SiC device technology and nanocrystalline-core-based magnetics techniology. The control actuation of the SiC devices are algorithmically implemented in digital signal processor (DSP) platform. The overall HFL transformer technology is conceived and developed using a modular approach. During the course of the project the novel technology and its experimental performances at the module, multi-phase, and cascaded multi-phase levels have been demonstrated to the sponsor U.S. Department of Energy (DOE).

24 POWER TRANSMISSION AND DISTRIBUTION↗

Structural Controllability Assessment for Inverter-Based Microgrids

Enhanced inverter-based controls are considered for microgrids, which use additional actuation beyond a droop-like term. Shaping of the microgrid’s small-signal dynamics using such enhanced controls is posed as a structural controllability problem. A graph-theoretic characterization of structural controllability is obtained, in terms of the concept of zero-forcing sets.Two benchmark test systems are used to illustrate the selection of locations where enhanced controls should be applied, based on the graph-theoretic analysis. These examples indicate that small-signal characteristics can be shaped using a relatively small number of enhanced controls

microgrids, zero-forcing, structural controlabilit↗

Anomaly Detection and Mitigation for Wide-Area Damping Control using Machine Learning

In an interconnected multi-area power system, wide-area measurement based damping controllers are used to damp out inter-area oscillations, which jeopardize grid stability and constrain the power flows below to their transmission capacity. The effect of wide-area damping control (WADC) significantly depends on both power and cyber systems. At the cyber system layer, an adversary can inflict the WADC process by compromising either measurement signals, control signals or both. Stealthy and coordinated cyber-attacks may bypass the conventional cybersecurity measures to disrupt the seamless operation of WADC. This paper proposes an anomaly detection (AD) algorithm using supervised Machine Learning and a model-based logic for mitigation. The proposed AD algorithm considers measurement signals (input of WADC) and control signals (output of WADC) as input to evaluate the type of activity such as normal, perturbation (small or large signal faults), attack and perturbation-and-attack. Upon anomaly detection, the mitigation module tunes the WADC signal and sets the control status mode as either wide-area mode or local mode. The proposed anomaly detection and mitigation (ADM) module works inline with the WADC at the control center for attack detection on both measurement and control signals and eliminates the need for ADMs at the geographically distributed actuators. Here, we consider coordinated and primitive data-integrity attack vectors such as pulse, ramp, relay-trip and replay attacks. The performance of the proposed ADM algorithms was evaluated under these attack vector scenarios on a testbed environment for 2-area 4-machine power system. The ADM module shows effective performance with 96:5% accuracy to detect anomalies.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Dynamic Human-in-the-Loop Simulated Nuclear Power Plant Thermal Dispatch System Demonstration and Evaluation Study

An Idaho National Laboratory research team performed a human-in-the-loop study with two formerly licensed retired operators to evaluate thermal power dispatch operations supported by the modified GSE Systems GPWR plant simulator. Virtual representations of the analog control panels were presented on touch-screen bays configured to mimic the control room layout in the newly renovated Human Systems Simulation Laboratory. The operators performed 15 scenarios covering normal evolutions to transition the plant from full turbine operation to joint turbine and thermal power dispatch operations in addition to transient response scenarios induced with simulated faults to evaluate the impact of thermal power dispatch system on operator and plant responses. A prototype human-system interface (HSI) was developed and displayed in tandem with the virtual analog panels to support the operators executing the procedurally drive evolutions and transient responses. An interdisciplinary team of operations experts, nuclear engineers, and human factors experts observed the operators performing the scenarios to evaluate the operations. Only a preliminary analysis of the results has been performed. The full analysis will be shared in a milestone report scheduled for release at the end of September; however, some high-level conclusions were readily available. Two high level findings for the system design were captured in the study. The manual control supported by the HSI to transition from standard operations to thermal power dispatch operation imposed a considerable amount of workload on the operators due to tedious manual valve manipulations and system monitoring required to verify their intended effect. An additional operator would be required in the control room to support the daily evolution. Automatic control for the transition was deemed a requirement for plant adoption without imposing additional staffing costs. The second finding was the necessity for an automatic thermal power dispatch system trip isolation function linked to a turbine and reactor trip signal. The operators completed scenarios with automatic isolation functionality and manually required actuation of the thermal power dispatch system. The operator response was sufficiently slower in the manual trip condition, such that operators were unable to manually actuate key post trip safety functions an indicate a degraded control capability that should be avoided. With an automatic trip signal little to no impact of the thermal power dispatch system was identified on the primary plant response and therefore the system could be readily and safely adopted. Together these two findings represent the need to support the adoption of thermal power dispatch capability into existing operations by leveraging automation to augment any additional operator tasking required to control and monitor an additional system beyond existing operations.

99 GENERAL AND MISCELLANEOUS↗

Mold and method for molding a reinforced preform

A mold for molding a reinforced preform having at least two apertures therein includes first and second mold halves, first and second emitters disposed in the mold halves and configured to emit light therefrom, first and second receivers disposed in the mold halves and configured to receive light from the respective first and second emitters, and first and second moving members having couplings for connection with side portions of the reinforced preform and actuators for moving the couplings between respective first and second positions. A controller determines an alignment condition based on signals received from the receivers. If the alignment condition fails to meet predetermined criteria, then at least one of the actuators is caused to move its coupling from its respective first position to a respective adjusted position that is different from the respective second position.

Rodgers, William R.↗