Search NASA⌕ Search

SEARCH · Search NASA

Results for “Autonomous control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

"Reinforcement Learning Based Oscillation Dampening: Scaling up Single-Agent RL algorithms to a 100 AV highway field operational test"

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. We explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

Jang, Kathy↗
DOE OSTI2025

The AMACStar ASIC for the HL-LHC ATLAS ITk Strip detector: design, verification, testing, and quality assurance

For the high-luminosity upgrade to the LHC (HL-LHC), the ATLAS detector at CERN requires an all-new inner detector, the Inner Tracker (ITk). The ITk Strip subdetector is made up of silicon modules, which include three types of radiation-hard ASICs. One of these is the Autonomous Monitor and Control (AMAC). The AMAC is manufactured by Global Foundries using 130 nm CMOS8RF DM technology and is approximately 3 by 5 mm in size. This ASIC autonomously monitors the temperatures, voltages, and currents in the module components while controlling critical values in order to prevent these quantities from reaching dangerous levels. The final design, AMACStar, was verified, tested, and ensured to perform all necessary functions. A quality control procedure using an in-house probe station set-up was developed in order to ensure that every individual chip on each wafer of AMACStars required by the ITk Strip project met performance requirements. The average per wafer yield of usable AMACStars is 92.33%, exceeding the design-specific 90% yield estimated for project costing. This estimate was based on actual yields for similar designs in this process.

Analogue electronic circuits↗
DOE OSTI2020

Resilient Autonomous Wind Farms

With the advent of an increasing number of control strategies that seek to optimize wind turbine performance on a farm level, taking into account individual wind turbine information to achieve wind-farm-level objectives has become an increasingly important goal. Methods for controlling wind turbines on an individual and farm level have experienced significant development, and an abundance of new implementations for gathering and using data from turbines have created potential for novel control mechanisms that can further optimize the performance and delivery characteristics of a wind farm. A key element of making these wind farms more efficient is to develop reliable algorithms that use local sensor information that is already being collected, such as from local meteorological stations, nearby radars, sodars, and lidars, and supervisory control and data acquisition (SCADA) data. Making use of information from all wind turbines in a wind farm can enable such approaches as determining the atmospheric conditions across the farm, improving fault-finding, and ensuring more efficient overall control of farmwide optimizations through mechanisms such as wake steering. However, these approaches typically involve a centralized communications and control center. In order to ensure the resilient operation of the farm, it is necessary to develop an approach that distributes the calculation and communication amongst multiple nodes throughout the farm. In this fashion, a redundant, robust, and secure network can be created, which can tolerate faults in calculation, communication, and even external attacks that seek to disrupt the operation of the wind farm. This paper introduces the use of the Raft-Byzantine-Fault-Tolerant algorithm in the implementation of autonomous control of a wind farm. This implementation will allow for fault tolerance for malfunctioning nodes, sensors, transmitters, and connectors. This approach is equally extensible to account for malicious actors. It will...

fault tolerance↗
DOE OSTI2024

Real-Time Gear-Shift Optimization for an Autonomous Wheel Loader

Off-road vehicles, such as wheel loaders, consume a significant amount of fuel during the transportation of materials. The gear-shifting process is crucial in fuel savings for transportation, and therefore, optimization of gearshifts is important for vehicle control. For an autonomous off-road vehicle, there is potential for more fuel savings by proper coordination of gearshift optimization and optimization of other control inputs. This brief proposes a new method for integrating gear-shifting into transportation optimization for an autonomous wheel loader to minimize fuel consumption. Furthermore, tests conducted on short loading cycles show that this method can save around 10%–20% of fuel on average compared with a conventional gearshift-scheduling method.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

Autonomous Energy Systems: A Distributed Approach to Control Millions of Energy Devices

Energy systems of all sizes are becoming increasingly complex. The National Renewable Energy Laboratory has developed new controls that will support real-time operations and management of renewables, storage, electric vehicles and loads for grid efficiency and resilience. This fact sheet presents an overview of these autonomous energy systems.

AES↗
DOE OSTI2024

Enabling a Physical Twin for Control Methods Evaluation

Advanced nuclear reactors play an important role in the energy future of the United States and the rest of the world. They are designed and operated based on a different model than that of the current operating fleet, thus enabling deployment in remote locations and allowing for safe semi-autonomous or autonomous operations. Such characteristics require the development of a new reactor control paradigm. A significant factor in the development of control technologies and methods is integration of the various technologies and methods with each other and with hardware (both reactor system hardware and control hardware). A recent workshop on control of advanced reactors identified the lack of a flexible, expandable software/hardware infrastructure to enable such integration as a key gap. A previous phase of the current effort involved developing and demonstrating the Control and Optimization Modular Modeling Application for Nuclear Deployment (COMMAND) platform, which is capable of integrating autonomous-control-enabling technologies and methods, without the constraints imposed by existing software solutions. This platform will enable advanced reactor developers to deploy and test any developed technologies and methods by employing a common framework, and to couple them with their own models and hardware. The present phase of this effort entails using the Microreactor Automated Control System (MACS) platform, which was developed by the U.S. Department of Energy (DOE) Microreactor Program, to serve as a control method testbed. MACS can be used by advanced reactor developers to integrate their control related research activities with any reactor system. For the present effort, MACS was customized to mirror Idaho National Laboratory (INL)’s Microreactor Applications Research Validation and Evaluation (MARVEL) microreactor, and COMMAND was leveraged to enable MACS to emulate the physics of MARVEL, thus positioning MACS as a physical twin of MARVEL.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Evolutionary vs imitation learning for neuromorphic control at the edge*

Abstract Neuromorphic computing offers the opportunity to implement extremely low power artificial intelligence at the edge. Control applications, such as autonomous vehicles and robotics, are also of great interest for neuromorphic systems at the edge. It is not clear, however, what the best neuromorphic training approaches are for control applications at the edge. In this work, we implement and compare the performance of evolutionary optimization and imitation learning approaches on an autonomous race car control task using an edge neuromorphic implementation. We show that the evolutionary approaches tend to achieve better performing smaller network sizes that are well-suited to edge deployment, but they also take significantly longer to train. We also describe a workflow to allow for future algorithmic comparisons for neuromorphic hardware on control applications at the edge.

Schuman, Catherine↗
DOE OSTI2025

Autonomous Operations for Advanced Reactors Utilizing Supervisory Control

Automation is a critical tenet of reactor plant operations as reliance on nuclear energy increases. Nuclear power plants require a large workforce which does not scale with output; that is, the cost per megawatt increases as reactor output becomes smaller. The economic viability of advanced reactors, particularly small modular reactors (SMRs) and microreactors, requires a significantly reduced onsite workforce. The logical solution is establishing a systematic process of elimination of reliance on human operators, and to the extent possible, replacing these actions with automated functions. In this paper, we propose a method for such transformation to establish a robust technical basis to enable transition to autonomy. Our method is based on finite state automata (FSA)—also known as finite state machines (FSMs). Relying on this method allows us to exploit the rich set of mathematical proofs available in the field of regular languages. FSA are one of the mathematical tools to model discrete event systems (DES). These properties are applied to produce an automated startup controller for the Massachusetts Institute of Technology Research Reactor (MITR). The startup procedure is captured in terms of discrete changes from one state to another while an independent supervisory control system directs the sequence of states and alerts a human in the event of an abnormal operation. First, the design and behavior of the MITR rod control system were modeled in Simulink. Then, the startup procedure was applied to the rod control system and the DES performed a startup by procedurally withdrawing rods to the subcritical position. The simulation also stops rod motion in response to an uncontrollable event and restarts rod motion once the event has been cleared.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2025

Reinforcement Learning-Based Oscillation Dampening: Scaling Up Single-Agent Reinforcement Learning Algorithms to a 100-Autonomous-Vehicle Highway Field Operational Test

In this article, we explore the technical details of the reinforcement learning (RL) algorithms that were deployed in the largest field test of automated vehicles designed to smooth traffic flow in history as of 2023, uncovering the challenges and breakthroughs that come with developing RL controllers for automated vehicles. We delve into the fundamental concepts behind RL algorithms and their application in the context of self-driving cars, discussing the developmental process from simulation to deployment in detail, from designing simulators to reward function shaping. We present the results in both simulation and deployment, discussing the flow-smoothing benefits of the RL controller. From understanding the basics of Markov decision processes to exploring advanced techniques such as deep RL, our article offers a comprehensive overview and deep dive of the theoretical foundations and practical implementations driving this rapidly evolving field. We also showcase real-world case studies and alternative research projects that highlight the impact of RL controllers in revolutionizing autonomous driving. From tackling complex urban environments to dealing with unpredictable traffic scenarios, these intelligent controllers are pushing the boundaries of what automated vehicles can achieve. Furthermore, we examine the safety considerations and hardware-focused technical details surrounding deployment of RL controllers into automated vehicles. As these algorithms learn and evolve through interactions with the environment, ensuring their behavior aligns with safety standards becomes crucial. Here, we explore the methodologies and frameworks being developed to address these challenges, emphasizing the importance of building reliable control systems for automated vehicles.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2024

Advanced Computational Techniques for Improving Resilience of Critical Energy Infrastructure under Cyber-Physical Attacks

In this chapter, we present recent advances in improving the resilience of cyber-physical systems, especially with regards to energy systems. We provide discussions around various types of cyber-physical events that can cause disruptions and new advances in optimization, control, and reinforcement learning (RL) to deal with the challenges posed by such cyber-physical events. The presented methods range from distributed robust optimization, autonomous and coordinated control, reinforcement learning based resilient control and topology reconfiguration in Inter-System resilient control.

Nazir, Mohammad Nawaf [BATTELLE (PACIFIC NW LAB)]↗
DOE OSTI2022

Evaluating Efficiency and Security of Connected and Autonomous Vehicle Applications

Evaluating efficiency and security of Connected and Autonomous Vehicles (CAVs) requires an environment that can support applications and measurements under real-world conditions. This work introduces our implementation and evaluation of a Connected and Autonomous Vehicle Research Environment (CAVRE). We implement and evaluate an existing CAV application called Cooperative Adaptive Cruise Control (CACC) using physical Vehicle-to-Vehicle (V2V) communications between a virtual agent and a real autonomous vehicle operating on a steerable dynamometer. CAVRE allows the follower to autonomously control longitudinal behavior on the dynamometer in order to maintain a steady following time gap from the leader. The effects of a wireless jamming attack on CACC and fuel efficiency is also evaluated. By executing attacks in a controlled environment, we learn how compromised communications can degrade CAV applications. We show that jamming V2V communications can impact CACC’s string stability and decrease fuel efficiency by more than 50%.

Taylor, Curtis↗
DOE OSTI2023

Neuroevolution Application to Collaborative and Heuristics-Based Connected and Autonomous Vehicle Cohort Simulation at Uncontrolled Intersection

Artificial intelligence is gaining tremendous attractiveness and showing great success in solving various problems, such as simplifying optimal control derivation. This work focuses on the application of Neuroevolution to the control of Connected and Autonomous Vehicle (CAV) cohorts operating at uncontrolled intersections. The proposed method implementation’s simplicity, thanks to the inclusion of heuristics and effective real-time performance are demonstrated. The resulting architecture achieves nearly ideal operating conditions in keeping the average speeds close to the speed limit. It achieves twice as high mean speed throughput as a controlled intersection, hence enabling lower travel time and mitigating energy inefficiencies from stop-and-go vehicle dynamics. Low deviation from the road speed limit is hence continuously sustained for cohorts of at most 50 m long. This limitation can be mitigated with additional lanes that the cohorts can split into. The concept also allows the testing and implementation of fast-turning lanes by simply replicating and reconnecting the control architecture at each new road crossing, enabling high scalability for complex road network analysis. The controller is also successfully validated within a high-fidelity vehicle dynamic environment, showing its potential for driverless vehicle control in addition to offering a new traffic control simulation model for future autonomous operation studies.

Jacquelin, Frederic (ORCID:0000000183154344)↗
DOE OSTI2021

Enhancing the Operational Resilience of Advanced Reactors with Digital Twins by Recurrent Neural Networks

Because of a lack of operational data and uncertainty in evaluation model for abnormal and accident scenarios, the established operating procedures can be biased in characterizing the reactor states and ensuring operational resilience. To reduce uncertainty associated with actual plant conditions, digital twin (DT) technology is suggested to support operator’s decision-making by effectively extracting and using knowledge of the current and future plant states from the knowledge base. This study first builds a knowledge base based on the characterization of issue space and the simulation tool. Next, this study discusses diagnosis and prognosis DTs for enhancing operational resilience by recovering the complete states of reactors and by predicting the future reactor behaviors. Finally, the decision-making module of the control system can determine the optimal control strategy that meets operational goals during loss-of-flow scenarios. To demonstrate and evaluate the DTs capability for supporting the operations of nuclear reactors, this study develops and assesses both the diagnosis and prognosis DTs in a nearly autonomous management and control system for an Experimental Breeder Reactor-II simulator during different loss-of-flow scenarios.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Recent progress in atomic-scale controlled plasma processing

Atomic-scale control in plasma processing is becoming increasingly critical for fabricating of advanced semiconductor devices, particularly as the industry shifts toward three-dimensional (3D) architectures and high-aspect-ratio (HAR) structures. This review presents a comprehensive overview of recent developments in atomic-scale controlled plasma processes, organized along two key directions: the hierarchical structure of plasma–surface interactions and the generational evolution of atomic layer processing (ALP) technologies. We examined the gas phase, where molecular design enables selective generation of ions and radicals; the boundary layer, where transport phenomena govern species delivery into nanoscale features, and the surface, where temperature-dependent reactions and cyclic processing determine etching selectivity and precision. Building on this foundation, we outline five generations of ALP—from thermal atomic layer deposition to transport-aware, temporally and structurally decoupled processes—highlighting the increasing sophistication of process control. The review further explores the transition from empirical recipe development to science-based, data-driven methodologies. By integrating quantum-chemical modeling, advanced diagnostics, and machine learning, we demonstrated how predictive models can link plasma species composition to process outcomes, enabling autonomous and adaptive control strategies. Finally, this review discusses the broader societal implications of plasma process innovation through the E4 quartet: energy and resource efficiency, environmental sustainability, evolutionary advancement, and educational promotion. These principles guide the development of sustainable and intelligent atomic-scale manufacturing technologies that are not only technically advanced but also socially responsible.

Ishikawa, Kenji [Nagoya Univ. (Japan)] (ORCID:0000↗
DOE OSTI2022

Deploying Traffic Smoothing Cruise Controllers Learned from Trajectory Data

Autonomous vehicle-based traffic smoothing con- trollers are often not transferred to real-world use due to challenges in calibrating many-agent traffic simulators. We show a pipeline to sidestep such calibration issues by collecting trajectory data and learning controllers directly from trajectory data that are then deployed zero-shot onto the highway. We construct a dataset of 772.3 kilometers of recorded drives on the I-24. We then construct a simple simulator using the recorded drives as the lead vehicle in front of a simulated platoon consisting of one autonomous vehicle and five human followers. Using policy-gradient methods with an asymmetric critic to learn the controller, we show that we are able to improve average MPG by 11% in simulation on congested trajectories. We deploy this controller to a mixed platoon of 4 autonomous Toyota RAV-4’s and 7 human drivers in a validation experiment and demonstrate that the expected time-gap of the controller is maintained in the real world test. Finally, we release the driving dataset [1], the simulator, and the trained controller at https://github.com/nathanlct/trajectory-training-icra.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Artificial Intelligence and Machine Learning for Bioenergy Research: Opportunities and Challenges

The integration of artificial intelligence and machine learning (AI/ML) with automated experimentation, genomics, biosystems design, and bioprocessing technologies is poised to revolutionize scientific investigation and, particularly, bioenergy research. To identify the opportunities and challenges in this emerging research area, the U.S. Department of Energy’s (DOE) Biological and Environmental Research program (BER) and Bioenergy Technologies Office (BETO) held a joint virtual workshop on AI/ML for Bioenergy Research (AMBER) on August 23–25, 2022. These interests have since been amplified in a September 2022 Executive Order, “Advancing Biotechnology and Biomanufacturing Innovation for a Sustainable, Safe, and Secure U.S. Bioeconomy,” to promote a whole-of government approach to biotechnology development (White House 2022). Approximately 50 scientists with various backgrounds and expertise from academia, industry, and DOE national laboratories met to discuss the opportunities and challenges of AI/ML for bioenergy research. Workshop participants were tasked with assessing the potential for AI/ML and laboratory automation to advance biological understanding and engineering in general. They particularly examined how integrating AI/ML tools with laboratory automation could accelerate biosystems design and optimize biomanufacturing. Discussions included the data and computational infrastructure needed to augment biosystems design applications and the expertise and workforce development efforts urgently required to shift integrated systems toward bioenergy research more broadly. Participants discussed many existing and future applications of AI/ML for biosystems design ranging from enzymes to plants and microbes, microbiomes, and bioprocess development. They also identified three key categories of scientific and technical opportunities and challenges: high-quality data, AI/ML algorithms, and laboratory automation. Several main takeaways emerged from the workshop: 1. Numerous AI/ML and automated experimentation applications exist for a variety of DOE mission needs in energy and the environment; 2. Exemplary research grand challenges for which AI/ML could provide solutions include: building microbes and microbial communities to specifications, developing closed-loop autonomous design and control for biosystems design, and advancing scale-up and automation; 3. Lack of sufficient high-quality, annotated data hinders the development of AI/ML applications; 4. New and improved AI/ML tools are needed, particularly those meeting the specific needs of the BER and BETO research communities; 5. Trade-offs in performance, cost, and reliability exist between deploying commercially available versus building custom-developed instrumentation and software for automated or autonomous experimentation; translation of manual to automated or autonomous methods is often a nontrivial endeavor; 6. Training a new generation of young scientists who can develop and apply AI/ML tools is needed to solve long-standing scientific challenges in bioenergy research. The integration of AI/ML tools and automated experimentation represents a new data-driven research paradigm complementary to the traditional hypothesis-driven research paradigm. This paradigm accelerates design and optimization of biological systems and processes for a variety of DOE mission needs in energy and the environment. The AMBER workshop broadly explored the potential of this new paradigm for bioenergy research, of particular interest to BER and BETO, and identified key challenges and opportunities that DOE can address in the coming years by leveraging its unique capabilities and resources.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Model Based Predictive Control for Frequency Support in Multi-port Autonomous Reconfigurable Solar Plants

Primary frequency support functionality can be an added advantage for any new grid integrated power electronics resource. With increased penetration of power electronics resources, inertial and primary frequency response capabilities are decreasing. In this research paper, an advanced model-based predictive control (MBPC) algorithm for providing frequency support to a new topology of integrated photovoltaic(PV), batterybased energy storage systems (ESS), and high-voltage direct current (HV dc) systems called multi-port autonomus reconfigurable solar (MARS) plants is based on the synchronverter-based control. The simulation of MARS-HV dc system with proposed control strategy is simulated and validated for MARS connected to low short circuit ratio (SCR) grids in PSCAD/EMTDC simulation environment. The control algorithm proposed showcases better performance in terms of frequency nadir improvement and frequency steady state improvemnt with respect to no frequency control.

Marthi, Phani Ratna Vanamali↗
DOE OSTI2023

Designing a transactive electric vehicle agent with customer’s participation preference

The proliferation of electric vehicles (EVs) and their inherent flexibility in charging timings make them an asset to improve grid performance. In contrast to direct control by a utility or autonomous price-based charging, the transactive control framework not only provides benefits to both grid and customers but also ensures customer autonomy. In this work, we design a transactive electric vehicle (TEV) agent that incorporates the EV owner’s willingness to trade-off between savings and amenity in form of a slider, where the EV owner’s amenity is characterized as vehicle readiness. Further, a privacy-preserving bidding formulation is proposed that also represents the customer’s transactive preference. A transactive market mechanism is discussed that integrates the TEV Agents into the local retail market and reconciles with the current day-ahead and real-time market structure. It is demonstrated that the proposed slider is able to provide a preferred trade-off between savings and amenity to individual customers. At the same time, the market mechanism is shown to successfully reduce both peak prices and peak demand. A comparative investigation of V1G and V2G technologies with respect to the battery prices is also discussed. It reveals that the V2G does not offer significant additional benefits with current battery prices, but could be promising if battery costs decline in the future.

33 ADVANCED PROPULSION SYSTEMS↗