Search NASA⌕ Search

SEARCH · Search NASA

Results for “Learning control”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,801 records · Page 100
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Learning an intake oxygen concentration of an engine

System and method for learning an intake oxygen concentration of an engine. In one embodiment, the system includes an intake oxygen sensor and an electronic control unit. The intake oxygen sensor is configured to measure an oxygen concentration of intake air and output a signal indicative of an intake oxygen concentration value. The electronic control unit is configured to receive the signal indicative of the intake oxygen concentration value, determine whether fuel is being injected into an engine, determine whether the engine is operating in an over-run condition, control a valve to sweep an air pressure of an intake path of the engine across a plurality of air pressures, and store information indicative of intake oxygen concentration values across the plurality of air pressures to learn the intake oxygen concentration of the engine.

Allen, Justin↗
DOE OSTI2025

Employing MACS/ViBRANT as a Surrogate MARVEL Reactor for Startup Reactivity Tuning and Supervisory Control Processes

Advanced nuclear reactors are a key part of the future of nuclear energy both in the United States and globally. They offer unique benefits for various energy-demanding applications, including use in remote locations, compact size, modular manufacturing, remote monitoring, low and/or variable power rating operation, and reliance on novel technologies to enhance operational safety. To achieve economic feasibility, advanced reactors must significantly reduce their workforces in comparison with the current fleet. Achieving this reduction will occur through reducing staff workloads using technology to achieve autonomous or semi-autonomous operations, demonstrated by comprehensive testing and validation activities. These operations will require both software and hardware platforms during the design and testing phases. While simulations are useful during the design phase, their performance can significantly deviate during actual deployment on hardware. This report presents the outcomes of a collaborative technical initiative between the U.S. Department of Energy (DOE) Microreactor Program (MRP) and Advanced Sensors and Instrumentation (ASI) Program. The collaboration utilized the Microreactor Automated Control System (MACS) hardware platform to bridge the gap between theoretical reactor design and actual startup and control operations. Two key use cases were investigated: facilitating the startup testing period and demonstrating supervisory control. The first use case details the key Microreactor Applications Research Validation and Evaluation (MARVEL) reactor startup physics testing activities conducted using the MACS platform. These activities included drum worth measurements, shutdown margin assessment, temperature feedback analysis, and scram time evaluation, as well as unique testing that would apply to the MARVEL reactor to demonstrate the testing methodologies in a low-risk environment. The MACS platform, serving as a surrogate representation of the MARVEL reactor, proved instrumental in performing these tests. The exercise revealed aspects that led to optimized processes, refined hardware design, and enhanced base software capabilities. By maturing methods and technologies in this manner, the initiative promises to reduce wasted time in the actual on-site reactor deployment effort, thereby saving significant time and resources. The second use case focuses on the development and implementation of supervisory control methods aimed at managing core tilt, which can result from asymmetrical operations or manufacturing imperfections in fuel rods or reactivity control devices. A key objective was to assess and compare the use of artificial intelligence (AI) for supervisory control. The effort aimed to define the role of supervisory control to enhance performance without risking control instability. This effort explored three distinct approaches: rules-based (RB) methods, optimization techniques, and reinforcement learning (RL) algorithms. Each approach was evaluated for its ease of implementation, its usability, and its effectiveness in responding to asymmetries in neutron flux. Comparative analysis of these approaches provided valuable insights into their applicability and effectiveness, offering a robust framework for advanced reactor operations. Together, these two use cases highlight the potential of hardware test beds to help streamline the design, operation, and control of advanced nuclear reactors. This collaborative effort underscores the importance of continued innovation and experimentation in achieving the next generation of safe, reliable, and economically viable nuclear energy solutions.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1998

NASA/Mir Phase 1: A Lesson Learned in Long Duration Mission Planning and Operations

The NASA/Mir Phase I Program is a precursor to the assembly and operations of the International Space Station (ISS). The Phase I Program covers a four-year period and encompasses over ten shuttle flights to the Mir, culminating with the flight of STS-91 in June 1998. The Mir-Orbiting Station (Mir-OS) provides an excellent environment, both onboard and on the ground, for learning and studying the characteristics associated with crew-tended operations on long duration missions. By the end of the Phase I program, NASA astronauts will have accumulated over two years of continuous operations experience working and living onboard the Mir-OS. Ground controllers, working in Russia and at remote sites in the United States, will have gained valuable experience in the generation, maintenance, and coordination of the information and products required to support the long duration crew members onboard the Mir-OS. The experience gained by the crew and ground support personnel in the area of long duration mission operations has resulted in a number of factors that need to be considered in the definition and implementation of the processes and products for the ISS. While there are many areas in which experience has been gained, this paper focuses on the lessons learned regarding the level of detail and flexibility of crew timelines. Provided in this paper is an overview of the characteristics associated with long duration mission timeline development and execution, a summary of applicable lessons learned identified and documented through participation in the NASA/Mir Phase I Program, a detailed discussion of "Gross Timelines" as implementation which addresses the lessons learned. Also discussed are the challenges that must be overcome in getting the ISS program to recognize, evaluate, and learn from the crew and ground controller experience gained through participation in the NASA/Mir Phase I Program.

Hagopian, Jeff↗
DOE OSTI2015

Biomimetic catalysts responsive to specific chemical signals

Part 1. Design of Biomimetic Catalysts Based on Amphiphilic Systems The overall objective of our research is to create biomimetic catalysts from amphiphilic molecules. More specifically, we aim to create supramolecular systems that can be used to control the microenvironment around a catalytic center in a biomimetic fashion and apply the learning to construct supramolecular catalysts with novel functions found in enzymatic catalysts. We have prepared synthetic molecules (i.e., foldamers) that could fold into helical structures with nanometer-sized internal hydrophilic cavities. Cavities of this size are typically observed only in the tertiary and quaternary structures of proteins but were formed in our foldamer prepared in just a few steps from the monomer. Similar to many proteins, our foldamers displayed cooperativity in the folding/unfolding equilibrium and followed a two-state conformational transition. In addition, their conformational change could be triggered by solvent polarity, pH, or presence of metal ions and certain organic molecules. We studied their environmentally dependent conformational changes in solutions, surfactant micelles, and lipid bilayer membranes. Unlike conventional rigid supramolecular host, a foldamer undergoes conformational change during guest binding. Our study in the molecular recognition of an oligocholate host yielded some extremely exciting results. Cooperativity between host conformation and host–guest interactions was found to “magnify” weak binding interactions. In other words, since binding affinity is determined by the overall change of free energy during the binding, guest-induced conformational change of the host, whether near or far from the binding site, affects the binding. This study has strong implications in catalysis because enzymes have been hypothesized to harvest similar intramolecular forces to strengthen their binding with the transition state of an enzyme-catalyzed reaction. The supramolecular and amphiphilic principles used in the foldamer catalysts were extended to a few other systems, particularly to interfacially cross-linked reverse micelles and micelles. These features enabled unusual catalytic features such as basic/nucleophilic catalysis under acidic conditions. We were able to create highly active metal nanoclusters catalysts whose local environment could be tuned by the organic framework. We were even able to create a “catalytic nanomachine” that grabs the substrate to the encapsulated Au clusters, which efficiently convert the substrate to the product that is rapidly ejected due to its different binding properties. Our research has important impacts on fundamental and applied energy-related sciences. On the fundamental level, it tests important biocatalytic principles on relatively simple synthetic systems and is expected to afford deeper understanding of biological catalysis. On the practical level, the research is anticipated to lead to “smart” catalysts and open up exiting applications in chemical analysis, reaction control, and materials synthesis. Part 2. Electrochemical Reduction of CO₂ The primary objective of our research involving the electrochemical reduction of carbon dioxide is to apply a multidisciplinary approach toward developing a greater understanding of the problem of efficiently converting CO₂ to hydrocarbons through electrochemical routes. Our goal is to provide a better understanding of the principles that underlie the electrocatalytic reduction of CO₂ at electrode surfaces and the molecular pathways that lead to desired compounds. This understanding is essential for the design and development of new catalytic materials for the selective production of renewable feedstocks. The electrochemical reduction of CO₂ involves the formation of various reaction products and adsorbed intermediates whose distribution depends upon the nature of the electrode material and the electrochemical conditions, including applied potential, solvent, and electrolyte, used during reduction. Our efforts are focused on developing a detailed picture of how these various parameters impact the reaction pathway for CO₂ reduction and to tune this reaction to favor the formation of specific products by careful selection of electrode materials and reaction conditions. Our experimental plan includes a detailed examination of the electrode material as well as a quantitative evaluation of the reaction conditions using several in-situ analysis tools that give a direct measure of adsorbed and solution species that appear during CO₂ reduction. During the first year of funding for this aspect of the project, we have developed an on-line electrochemical measurement tool utilizing membrane-introduction mass spectrometry to detect reactions products directly during the electrochemical reduction of CO₂on various sputtered electrode materials. In addition, we have examined the role of a selection of different electrode compositions (including single component and binary electrodes) on product distributions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Load Control for Frequency Response - A Literature Review

As electricity grids employ greater fractions of renewable energy, which introduce additional variability and uncertainty in the net load, balancing electrical load and generation becomes more challenging. This paper reviews the literature documenting physical simulations and real systems that employ load control (LC) for frequency response and other grid services, which balance net load on the grid and prevent unwanted frequency excursions. Apart from academic and simulation studies, few sources exist on large-scale laboratory hardware testing or actual real-world systems that employ LC for frequency response, and we review them here. Four types of systems that we consider are: 1) Laboratory-based LC experiments, 2) Isolated microgrids that employ LC, 3) Larger grids that employ LC and 4) vehicle-to-grid (V2G) technology, using electric vehicles (EVs). In general, these systems have successfully used LC to meet their objectives, which are often keeping grid frequency within a required band. However, LC struggled to balance grid frequency in an isolated system powered by a single wind turbine, and V2G technology requires refinement in communication and control to provide optimal regulation that adheres to industry standards. As LC grows in the energy industry, we have three main recommendations: 1) encouraging system operators who use LC to publish system characteristics and lessons learned; 2) transitioning more LC theoretical/simulated systems to physical experiments, and physical experiments to real-world pilot systems; 3) demonstrating load control to support isolated, high-wind-contribution systems.

17 WIND ENERGY↗
DOE OSTI2024

Integration of scanning probe microscope with high-performance computing: Fixed-policy and reward-driven workflows implementation

The rapid development of computation power and machine learning algorithms has paved the way for automating scientific discovery with a scanning probe microscope (SPM). The key elements toward operationalization of the automated SPM are the interface to enable SPM control from Python codes, availability of high computing power, and development of workflows for scientific discovery. Here, we build a Python interface library that enables controlling an SPM from either a local computer or a remote high-performance computer, which satisfies the high computation power need of machine learning algorithms in autonomous workflows. We further introduce a general platform to abstract the operations of SPM in scientific discovery into fixed-policy or reward-driven workflows. Furthermore, our work provides a full infrastructure to build automated SPM workflows for both routine operations and autonomous scientific discovery with machine learning.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Quantifying Drivers of Methane Hydrobiogeochemistry in a Tidal River Floodplain System

The influence of coastal ecosystems on global greenhouse gas (GHG) budgets and their response to increasing inundation and salinization remains poorly constrained. In this study, we have integrated an uncertainty quantification (UQ) and ensemble machine learning (ML) framework to identify and rank the most influential processes, properties, and conditions controlling methane behavior in a freshwater floodplain responding to recently restored seawater inundation. Our unique multivariate, multiyear, and multi-site dataset comprises tidal creek and floodplain porewater observations encompassing water level, salinity, pH, temperature, dissolved oxygen (DO), dissolved organic carbon (DOC), total dissolved nitrogen (TDN), partial pressure of carbon dioxide (pCO 2 ), nitrous oxide (pN 2 O), methane (pCH 4 ), and the stable isotopic composition of methane (δ 13 CH 4 ). Additionally, we incorporated topographical data, soil porosity, hydraulic conductivity, and water retention parameters for UQ analysis using a previously developed 3D variably saturated flow and transport floodplain model for a physical mechanistic understanding of factors influencing groundwater levels and salinity and, therefore, CH 4 . Principal component analysis revealed that groundwater level and salinity are the most significant predictors of overall biogeochemical variability. The ensemble ML models and UQ analyses identified DO, water level, salinity, and temperature as the most influential factors for porewater methane levels and indicated that approximately 80% of the total variability in hourly water levels and around 60% of the total variability in hourly salinity can be explained by permeability, creek water level, and two van Genuchten water retention function parameters: the air-entry suction parameter α and the pore size distribution parameter m. These findings provide insights on the physicochemical factors in methane behavior in coastal ecosystems and their representation in local- to global-scale Earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

WHISPER: Wireless Home Identification and Sensing Platform for Energy Reduction

Many regions of the world benefit from heating, ventilating, and air-conditioning (HVAC) systems to provide productive, comfortable, and healthy indoor environments, which are enabled by automatic building controls. Due to climate change, population growth, and industrialization, HVAC use is globally on the rise. Unfortunately, these systems often operate in a continuous fashion without regard to actual human presence, leading to unnecessary energy consumption. As a result, the heating, ventilation, and cooling of unoccupied building spaces makes a substantial contribution to the harmful environmental impacts associated with carbon-based electric power generation, which is important to remedy. For our modern electric power system, transitioning to low-carbon renewable energy is facilitated by integration with distributed energy resources. Automatic engagement between the grid and consumers will be necessary to enable a clean yet stable electric grid, when integrating these variable and uncertain renewable energy sources. We present the WHISPER (Wireless Home Identification and Sensing Platform for Energy Reduction) system to address the energy and power demand triggered by human presence in homes. The presented system includes a maintenance-free and privacy-preserving human occupancy detection system wherein a local wireless network of battery-free environmental, acoustic energy, and image sensors are deployed to monitor homes, record empirical data for a range of monitored modalities, and transmit it to a base station. Several machine learning algorithms are implemented at the base station to infer human presence based on the received data, harnessing a hierarchical sensor fusion algorithm. Results from the prototype system demonstrate an accuracy in human presence detection in excess of 95%; ongoing commercialization efforts suggest approximately 99% accuracy. Using machine learning, WHISPER enables various applications based on its binary occupancy prediction, allowing situation-specific controls targeted at both personalized smart home and electric grid modernization opportunities.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
NASA NTRS2016

ROS Hexapod

As an intern project for NASA Johnson Space Center (JSC), my job was to familiarize myself and operate a Robotics Operating System (ROS). The project outcome converted existing software assets into ROS using nodes, enabling a robotic Hexapod to communicate to be functional and controlled by an existing PlayStation 3 (PS3) controller. Existing control algorithms and current libraries have no ROS capabilities within the Hexapod C++ source code when the internship started, but that has changed throughout my internship. Conversion of C++ codes to ROS enabled existing code to be compatible with ROS, and is now controlled using an existing PS3 controller. Furthermore, my job description was to design ROS messages and script programs that enabled assets to participate in the ROS ecosystem by subscribing and publishing messages. Software programming source code is written in directories using C++. Testing of software assets included compiling code within the Linux environment using a terminal. The terminal ran the code from a directory. Several problems occurred while compiling code and the code would not compile. So modifying code to where C++ can read the source code were made. Once the code was compiled and ran, the code was uploaded to Hexapod and then controlled by a PS3 controller. The project outcome has the Hexapod fully functional and compatible with ROS and operates using the PlayStation 3 controller. In addition, an open source software (IDE) Arduino board will be integrated into the ecosystem with designing circuitry on a breadboard to add additional behavior with push buttons, potentiometers and other simple elements in the electrical circuitry. Other projects with the Arduino will be a GPS module, digital clock that will run off 22 satellites to show accurate real time using a GPS signal and an internal patch antenna to communicate with satellites. In addition, this internship experience has led me to pursue myself to learn coding more efficiently and effectively to write, subscribe and publish my own source code in different programming languages. With some familiarity with software programming, it will enhance my skills in the electrical engineering field. In contrast, my experience here at JSC with the Simulation and Graphics Branch (ER7) has led me to take my coding skill to be more proficient to increase my knowledge in software programming, and also enhancing my skills in ROS. This knowledge will be taken back to my university to implement coding in a school project that will use source coding and ROS to work on the PR2 robot which is controlled by ROS software. My skills learned here will be used to integrate messages to subscribe and publish ROS messages to a PR2 robot. The PR2 robot will be controlled by an existing PS3 controller by changing C++ coding to subscribe and publish messages to ROS. Overall the skills that were obtained here will not be lost, but increased.

Davis, Kirsch↗
DOE OSTI2022

Deep learning potential of mean force between polymer grafted nanoparticles

Grafting polymer chains on the surfaces of nanoparticles is a well-known route to control their self-assembly and distribution in a polymer matrix. A wide variety of self-assembled structures are achieved by changing the grafting patterns on the surface of an individual nanoparticle. However, an accurate estimation of the effective potential of mean force between a pair of grafted nanoparticles that determines their assembly and distribution in a polymer matrix is an outstanding challenge in nanoscience. We address this problem via deep learning. As a proof of concept, here we report a deep learning framework that learns the interaction between a pair of single-chain grafted spherical nanoparticles from their molecular dynamics trajectory. Subsequently, we carry out the deep learning potential of mean force-based molecular simulation that predicts the self-assembly of a large number of single-chain grafted nanoparticles into various anisotropic superstructures, including percolating networks and bilayers depending on the nanoparticle concentration in three-dimensions. The deep learning potential of mean force-predicted self-assembled superstructures are consistent with the actual superstructures of single-chain polymer grafted spherical nanoparticles. This deep learning framework is very generic and extensible to more complex systems including multiple-chain grafted nanoparticles. Finally, we expect that this deep learning approach will accelerate the characterization and prediction of the self-assembly and phase behaviour of polymer-grafted and unfunctionalized nanoparticles in free space or a polymer matrix.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Deep learning model to detect various synchrophasor data anomalies

High-density synchrophasors provide valuable information for power grid situational awareness, operation and control. Unfortunately, due to factors including communication instability and hardware failure, their data quality can be greatly deteriorated by anomalies. Since the anomalies can impact the performance of the synchrophasor applications, it is of paramount significance to propose a model to detect anomalies in synchrophasor. In this study, a convolutional neural network model is established to detect and classify the anomalies in the synchrophasor measurements. Additionally, four types of anomalies observed in actual synchrophasors including erroneous patterns, random spikes, missing points and high-frequency interferences are considered in this study. The proposed model is extensively evaluated via field-collected measurements from the synchrophasor network in Jiangsu grid, China. The superior performance of the proposed model indicates the great potential of using deep learning for the detection of abnormal synchrophasor measurements.

42 ENGINEERING↗
DOE OSTI2020

On the Formalization of Development and Assessment Process for Digital Twins in the Nearly Autonomous Management and Control System

In recent years, the autonomous control system has been encouraged in advanced reactors for restoring economic viability, simplifying the operation and maintenance, and enabling remote-site power generations [1]. Since the reactor is expected to be operated for a long period of time with a limited number of individuals onsite, it is recommended that the autonomous control system should have access to very realistic models of the state of processes in the whole lifecycle, together with these process behaviors in interaction with their environment in the real world. As a result, digital twin (DT) technology is suggested in autonomous control systems. DT is defined as a digital representation of a physical object or system, which contains a record for the histories of loads, operation and maintenance status, predictions for the near-term transient of important state variables, and decision-making process [2]. Since machine learning (ML) can recognize patterns within a complex system in real-time applications, it has been used to build DTs in the autonomous control systems for advanced reactors. Meanwhile, due to the rareness of operation data in accident scenarios, the development and assessment of DTs is expected to be mainly driven by simulations. Although the capability and feasibility of ML-based DTs are recognized in improving the safety and efficiency of reactor control, a major concern from the regulatory commission and the nuclear industry is whether the information from a DT is developed and assessed in accordance with expectation and requirements by the target decision. Such concerns not only affect the acceptance criteria for DTs, but also values that can be extracted from DTs and autonomous control system during operations. Inspired by the success of formal methods in improving the reliability and robustness of computer programming and software development, it is suggested that the development and assessment process (DAP) for both separate DTs and integral control system should be formalized in a transparent, consistent, and improvable manner. In this study, a digital-twin development and assessment process (DT-DAP) is proposed by adapting the evaluation model development and assessment process (EMDAP) [3] to requirements by the autonomous control system, ML algorithms, and DT technology. To demonstrate the framework, a baseline nearly autonomous management and control (NAMAC) system with ML-based DTs for diagnosis and prognosis is developed and assessed based on the framework. It is found that with selected testing methods and techniques, the DT-DAP can help identify errors in DTs and NAMAC which would otherwise be left unverified. Meanwhile, it is found that the DT-DAP can improve the DTs and NAMAC by continuously learning and iterating through different elements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2018

Quantifying Uncertainty to Improve Decision Making in Machine Learning

Data-driven modeling, including machine learning methods, continue to play an increasing role in society. Data-driven methods impact decision making for applications ranging from everyday determinations about which news people see and control of self-driving cars to high-consequence national security situations related to cyber security and analysis of nuclear weapons reliability. Although modern machine learning methods have made great strides in model induction and show excellent performance in a broad variety of complex domains, uncertainty remains an inherent aspect of any data-driven model. In this report, we provide an update to the preliminary results on uncertainty quantification for machine learning presented in SAND2017-6776. Specifically, we improve upon the general problem definition and expand upon the experiments conducted for the earlier re- port. Most importantly, we summarize key lessons learned about how and when uncertainty quantification can inform decision making and provide valuable insights into the quality of learned models and potential improvements to them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Deep Learning–Assisted Multiobjective Optimization of Geological CO 2 Storage Performance under Geomechanical Risks

In geological CO 2 storage, designing the optimal well control strategy for CO 2 injection to maximize CO 2 storage while minimizing the associated geomechanical risks is not trivial. This challenge arises due to pressure buildup, CO 2 plume migration, the highly nonlinear nature of geomechanical responses to rock-fluid interaction, and the high computational cost associated with coupled flow and geomechanics simulations. In this paper, we introduce a novel optimization framework to address these challenges. The optimization problem is formulated as follows: maximize total CO 2 storage while minimizing geomechanical risks by adjusting the injection schedules within bounded constraints. The geomechanical risks are primarily driven by injection-induced pressure build-up, which is characterized by ground displacement and the induced microseismicity. We used the Fourier neural operator (FNO)-based deep learning model to construct surrogate models, replacing the time-consuming coupled flow and geomechanics simulations for evaluating the aforementioned objective functions. The developed surrogate models have been incorporated into a multiobjective optimization framework through a genetic algorithm to reduce the computational burden. The proposed optimization framework reduces the computational cost from approximately 2,400 hours, when using objective function evaluations based on physics-based simulations, to around 20 minutes. A set of Pareto-optimal solutions of the proposed workflow yields nontrivial optimal decisions, reducing the microseismicity potential and the vertical displacement. This Pareto front highlights the optimal trade-offs between CO 2 storage amount, safety, and ground displacement, emphasizing the need for careful optimization and management of injection strategies to achieve a balanced outcome. The novelty of this work is twofold. First, we demonstrate the importance of incorporating the minimization of the geomechanical risks as objective functions into the CO 2 storage optimization workflow to mitigate the potential risk of induced microseismicity and ground displacement. Second, we leverage the FNO-based surrogate models to optimize a real-field CO 2 storage operation.

42 ENGINEERING↗
DOE OSTI2024

User-Centric Communication With Aerial Network for 6G: A Reinforcement Learning Approach

Meeting the diverse needs of user verticals requires innovative cellular architectures that can offer additional degrees of freedom to provide on-demand services. The terrestrial user-centric radio access network (UC-RAN) stands out as an excellent choice for this purpose. However, a drawback of UC-RAN is its tendency to prioritize high-priority verticals, often resulting in a subpar quality of experience for low-priority verticals. This issue is particularly exacerbated in hotspot areas. Here, to address this problem, we introduce an aerial network integrated with terrestrial UC-RAN to provide coverage to users which are not served by the terrestrial network. Furthermore, we analyze the impact of key configuration and optimization parameters (COPs), such as location, transmit power, altitude, and beamwidth of aerial base stations (ABSs) on system key performance indicators (KPIs), such as coverage, latency satisfaction, average spectral efficiency, and energy efficiency. We formulate a robust multiobjective function to maximize these KPIs without biasing toward any specific KPI(s). Finally, we propose a deep reinforcement learning optimization framework based on the state-of-the-art soft actor-critic algorithm to control ABS COPs and optimize system KPIs. Experimental evaluations demonstrate that the proposed optimization framework can converge to near-optimal solutions derived from the pseudo brute force in a few thousand epochs.

6G↗
DOE OSTI2025

Automating Bug Report Classification with Few Shot Learning

Orthogonal defect classification (ODC) is a method used to categorize software defects, providing valuable insights into the development process. This study focuses on automating the classification of software bug reports into different ODC defect types using few shot learning, a machine learning approach that requires minimal labeled data. Previous research has manually classified bug reports or used traditional machine learning algorithms like linear support vector machine, achieving limited success. Our approach uses few shot learning to improve classification accuracy and efficiency. The results show a harmonic mean of recall and precision (i.e., the F1 score) of around 0.6 which is a performance improvement over previous methods. The results highlight the potential benefit of few shot learning techniques and their application in enhancing the safety and reliability of nuclear digital instrumentation and control (DI&C) systems. Future work will explore incorporating advanced techniques to supplement the model's training data and achieve better results.

42 - ENGINEERING↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗

Refine your search to explore more results.