Search NASA⌕ Search

SEARCH · Search NASA

Results for “Q learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Learning Based Edge Computing in Air-to-Air Communication Network

This paper studies learning-based edge computing and communication in a dynamic Air-to-Air Ad-hoc Network (AAAN). Due to spectrum scarcity, we assume the number of Air-to-Air (A2A) communication links is greater than that of the available frequency channels, such that some communication links have to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the aggregate spectrum utilization efficiency under resource and fairness constraints. A distributed deep Q learning-based edge computing and communication algorithm is proposed to find the optimal solution. In particular, we design two different neural network structures and each communication link can converge to the optimal operation by exploiting only the local information from its neighbors, making it scalable to large networks. Finally, experimental results demonstrate the effectiveness of the proposed solution in various AAAN scenarios.

Zhe Wang↗
DOE OSTI2024

Reinforcement Learning Control for Enhancing Marine Hydrokinetic Turbine Energy Generation

This paper proposes a reinforcement learning-based method to maximize power generation for a direct-drive marine hydrokinetic turbine. A high levelized cost of energy (LCOE) is preventative in the widespread adoption of many marine energy conversion technologies. A straightforward way to reduce LCOE is to increase conversion efficiency and ensure maximum energy generation. The proposed method utilizes a damping control methodology, varying applied generator torque via a linear relationship between the applied damping coefficient and rotor speed. A state-action-reward-state-action (SARSA) algorithm has been used to learn the optimal control action for a given flow velocity. The proposed SARSA methodology uses Gaussian radial basis functions to create a three-dimensional surface to estimate the relationship between damping coefficient, incoming flow velocity, and coefficient of power (C p ). Here, the SARSA algorithm was compared against a baseline optimal tip speed ratio controller over a year-long flow velocity case profile while considering the effects of biofouling on the turbine system, where the proposed RL method generated 0.92% more energy than the baseline.

Damp↗
DOE OSTI2025

Q-Cluster: Quantum Error Mitigation Through Noise-Aware Unsupervised Learning

Quantum error mitigation (QEM) is critical in reducing the impact of noise in the pre-fault-tolerant era, and is expected to complement error correction in fault-tolerant quantum computing (FTQC). In this work, we propose a novel QEM approach, Q-Cluster, that uses unsupervised learning (clustering) to reshape the measured bit-string distribution. Our approach starts with a simplified bit-flip noise model. It first performs clustering on noisy measurement results, i.e., bit-strings, based on the Hamming distance. The centroid of each cluster is calculated using a qubit-wise majority vote. Next, the noisy distribution is adjusted with the clustering outcomes and the bitflip error rates using Bayesian inference. Our simulation results show that Q-Cluster can mitigate high noise rates (up to 40% per qubit) with the simple bit-flip noise model. However, real quantum computers do not fit such a simple noise model. To address the problem, we (a) apply Pauli twirling to tailor the complex noise channels to Pauli errors, and (b) employ a machine learning model, ExtraTrees regressor, to estimate an effective bit-flip error rate using a feature vector consisting of machine calibration data (gate & measurement error rates), circuit features (number of qubits, numbers of different types of gates, etc.) and the shape of the noisy distribution (entropy). Our experimental results show that our proposed Q-Cluster scheme improves the fidelity by a factor of 1.46x, on average, compared to the unmitigated output distribution, for a set of low-entropy benchmarks on five different IBM quantum machines. Our approach outperforms the state-of-art QEM approaches RZNE [28], M3 [24], Hammer [35], and QBEEP [33] by 1.26x,1.29x,1.47x, and 2.65 x, respectively.

42 ENGINEERING↗
DOE OSTI2025

Bayesian inference analysis of jet quenching using inclusive jet and hadron suppression measurements

The JETSCAPE Collaboration reports a new determination of the jet transport parameter $\hat{q}$ in the quark-gluon plasma (QGP) using Bayesian inference, incorporating all available inclusive hadron and jet yield suppression data measured in heavy-ion collisions at the BNL Relativistic Heavy Ion Collider (RHIC) and the CERN Large Hadron Collider (LHC). This multi-observable analysis extends the previously published JETSCAPE Bayesian inference determination of $\hat{q}$, which was based solely on a selection of inclusive hadron suppression data. jetscape is a modular framework incorporating detailed dynamical models of QGP formation and evolution, and jet propagation and interaction in the QGP. Virtuality-dependent partonic energy loss in the QGP is modeled as a thermalized weakly coupled plasma, with parameters determined from Bayesian calibration using soft-sector observables. This Bayesian calibration of $\hat{q}$ utilizes active learning, a machine-learning approach, for efficient exploitation of computing resources. The experimental data included in this analysis span a broad range in collision energy and centrality, and in transverse momentum. In order to explore the systematic dependence of the extracted parameter posterior distributions, several different calibrations are reported, based on combined jet and hadron data; on jet or hadron data separately; and on restricted kinematic or centrality ranges of the jet and hadron data. Tension is observed in comparison of these variations, providing new insights into the physics of jet transport in the QGP and its theoretical formulation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS1997

Collaborating Fuzzy Reinforcement Learning Agents

Earlier, we introduced GARIC-Q, a new method for doing incremental Dynamic Programming using a society of intelligent agents which are controlled at the top level by Fuzzy Relearning and at the local level, each agent learns and operates based on ANTARCTIC, a technique for fuzzy reinforcement learning. In this paper, we show that it is possible for these agents to compete in order to affect the selected control policy but at the same time, they can collaborate while investigating the state space. In this model, the evaluator or the critic learns by observing all the agents behaviors but the control policy changes only based on the behavior of the winning agent also known as the super agent.

Berenji, Hamid R.↗
DOE OSTI2025

Integrated Routing and Traffic Signal Control for CAVs via Reinforcement Learning Approach

Incorporating Connected and Automated Vehicles (CAVs) into urban traffic networks presents opportunities and challenges for traffic management systems. This paper aims to develop an integrated routing and traffic signal control system designed explicitly for CAVs, utilizing a Reinforcement Learning (RL) approach. The objective is to enhance traffic flow and improve overall transportation efficiency in the controlled areas. We propose an innovative framework that employs the Deep Reinforcement Learning (DRL) algorithm, especially the Deep Q-network (DQN), to dynamically adjust the number of vehicles in the routes and the duration of traffic signals. Our simulation results demonstrate that a DQN agent successfully optimizes the number of vehicles in the routes and traffic signal timings of traffic signal controllers, eventually reducing total travel time. The study illustrates the potential usage of RL-based systems in managing routing and traffic signals for CAVs, offering a promising opportunity for future urban traffic management strategies.

Park, Jiho [New York University]↗
NASA NTRS1997

Learning and Tuning of Fuzzy Rules

In this chapter, we review some of the current techniques for learning and tuning fuzzy rules. For clarity, we refer to the process of generating rules from data as the learning problem and distinguish it from tuning an already existing set of fuzzy rules. For learning, we touch on unsupervised learning techniques such as fuzzy c-means, fuzzy decision tree systems, fuzzy genetic algorithms, and linear fuzzy rules generation methods. For tuning, we discuss Jang's ANFIS architecture, Berenji-Khedkar's GARIC architecture and its extensions in GARIC-Q. We show that the hybrid techniques capable of learning and tuning fuzzy rules, such as CART-ANFIS, RNN-FLCS, and GARIC-RB, are desirable in development of a number of future intelligent systems.

Berenji, Hamid R.↗
DOE OSTI2026

Search for heavy scalar resonances decaying to Lorentz-boosted Higgs and Higgs-like bosons in the $\mathrm{b}\overline{\mathrm{b}}4\mathrm{q}$ final state at $\sqrt{s}=13$ TeV

A search is performed for a heavy scalar resonance X decaying to a Higgs boson (H) and a Higgs-like scalar boson (Y) in the two bottom quark (H → $b\bar{b}$) and four quark (Y → VV → 4q) final state, where V denotes a W or Z boson. Masses of the X between 900 and 4000 GeV and the Y between 60 and 2800 GeV are considered. The search is performed in data collected by the CMS experiment at the CERN LHC from proton-proton collisions at 13 TeV center-of-mass energy, with a data set corresponding to a total integrated luminosity of 138 fb −1 . It targets the Lorentz-boosted regime, in which the products of the H → $b\bar{b}$ decay can be reconstructed as a single large-area jet, and those from the Y → VV → 4q decay as either one Y → 4q or two jets V → H → $q\bar{q}$. Jet identification and mass reconstruction exploit machine-learning tools, including a novel attention-based “particle transformer” for Y → 4q identification. No significant excess is observed in the data above the standard model background expectation. Upper limits on the product of production cross section and branching fraction as low as 0.2 fb are derived at 95% confidence level for various mass hypotheses. This is the first search at the LHC for scalar resonances in the all-hadronic H → $b\bar{b}$VV decay channel.

Beyond Standard Model↗
DOE OSTI2024

Machine learning BPS spectra and the gap conjecture

We explore statistical properties of Bogomol’nyi-Prasad-Sommerfield q-series for strongly coupled supersymmetric theories that correspond to a particular family of three-manifolds. We discover that gaps between exponents in the -series are statistically more significant at the beginning of the -series compared to gaps that appear in higher powers of. Our observations are obtained by calculating saliencies of -series features used as input data for principal component analysis, which is a standard example of an explainable machine learning technique that allows for a direct calculation and a better analysis of feature saliencies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Resolving the dynamic correlated disorder in KTa 1- x Nb x O 3

Understanding the complex temporal and spatial correlations of ions in disordered perovskite oxides is critical to rationalize their functional properties. Here, in this study, we provide new insights into the longstanding controversy regarding the off-centering of transition metal ions in the archetypal ferroelectric alloy KTa 1-x Nb x O 3 (KTN). By mapping the full energy (E) and wavevector (Q) dependence of the dynamical structure factor S(Q, E) using neutron scattering, and rationalizing our observations with atomistic simulations leveraging machine learning, we fully resolve the static vs dynamic nature of diffuse scattering sheets, as well as their composition (x) and temperature dependence. Our first-principles simulations, extended with machine-learning molecular dynamics, reproduce both inelastic neutron spectra and diffuse features, and establish how dynamically-correlated transition metal off-centerings couple to phonons, unifying local and collective viewpoints. This study sheds new light into an exemplary ferroelectric systems and shows the importance of mapping the full S(Q, E) to reveal critical spatio-temporal correlations of atomic disorder from which functional properties emerge.

42 ENGINEERING↗
DOE OSTI2026

Neural network potentials with effective charge separation for non-equilibrium dynamics of ionic solids: a ZnO case study

Developing neural network potentials (NNPs) accurate under non-equilibrium dynamics is challenging, as such systems require extensive sampling beyond equilibrium phases. Here we construct high-fidelity NNPs for zinc oxide (ZnO), a polymorphic ionic solid, using density functional theory (DFT) reference data. To efficiently capture transitional configurations, we combine enhanced-sampling molecular dynamics with empirical potentials, data distillation, and pretraining on short-range atomic energies (A-Train), followed by transfer learning with DFT-relabeled datasets. This hierarchical approach improves transferability across polymorphs and stress states. We further introduce effective charge separation, treating long-range Coulombic terms analytically while short-range residual interactions are learned by the NNP. The optimal effective charges fall in the range 0.5–1.0 q e , consistent with dielectric-screened values derived from formal charges but distinct from Bader estimates. Motivated by this observation, we propose a simple data-driven protocol in which effective charges are optimized by comparing DFT reference energies with explicit Coulomb calculations, without additional NNP training. This strategy improves accuracy and transferability in DFT-level predictions of energies, forces, and stress. Together, these results provide a practical charge-selection framework for robust NNP development in ionic solids, enabling reliable simulation of polymorphic phase transformations and non-equilibrium dynamics.

Chemistry↗
DOE OSTI2025

Phonon second harmonic generation in NaBr studied by inelastic neutron scattering and computer simulation

The phenomenon of second harmonic generation (SHG) was found for phonons in anharmonic NaBr by inelastic neutron scattering. The temperature dependence of this phonon SHG was measured from 300 K to 650 K. At 300 K the second harmonic (SH) is seen as a high-energy branch around 33 meV, nearly independent of $\overrightarrow{Q}$. The temperature effective potential (TDEP) method and classical molecular dynamics (MD) simulation with machine learning interatomic potential were able to reproduce the SH, and showed that SHG occurs with the flat transverse optical (TO) phonon branch. A classical model of a nonlinear medium explains the intensity and lifetime of the SH, compared to those of the TO modes. Also successful was a quantum model based on the Heisenberg-Langevin equation for interacting phonons coupled to a thermal bath, which also predicts a spectral distribution of the SH. In conclusion, the measured temperature dependence of the intensity of the second harmonic showed that it follows the Planck distribution of a one-phonon quasiparticle, and not two TO phonons.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Kernel methods for evolution of generalized parton distributions

Generalized parton distributions (GPDs) characterize the 3-dimensional structure of hadrons, combining information about their internal quark and gluon longitudinal momentum distributions and transverse position within the hadron. The dependence of GPDs on the factorization scale Q 2 allows one to connect hard exclusive processes involving GPDs at disparate energy and momentum scales, which is needed in global analyses of experimental data. Here, in this work, we explore how finite element methods can be used to construct fast and differentiable Q 2 evolution codes for GPDs in momentum space, which can be used in a machine learning framework. We show numerical benchmarks of the methods' accuracy, including a comparison to an existing evolution code from PARTONS/APFEL++, and provide a repository where the code can be accessed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1986

Investigation of porous polymer gas chromatographic packings for atmospheric analysis of extraterrestrial bodies

Measurement of the permanent gases in the atmospheres of mission targets is a major objective. A 16 meter long Porapak N column was used on the Venus probe and required a rather high carrier gas flow rate. The researchers have, therefore, surveyed commercial porous polymer types which had some ability to resolve nitrogen, oxygen, argon and carbon monoxide gases. Porapaks N and Q appeared superior to most. Batch to batch variation, however, was quite wide, so the researchers learned how to synthesize porous polymer and investigated some of the factors affecting the separations. A polymer was synthesized which was superior to all commercial products and allowed at least a 50% reduction in length and flow rate of carrier gas. Similar studies were made concerning the separation of hydrocarbons and new porous polymers have been synthesized which represent significant improvements in time of analysis, column, and carrier gas flow rate.

Pollock, G. E.↗
NASA NTRS1992

Using neural networks and Dyna algorithm for integrated planning, reacting and learning in systems

The traditional AI answer to the decision making problem for a robot is planning. However, planning is usually CPU-time consuming, depending on the availability and accuracy of a world model. The Dyna system generally described in earlier work, uses trial and error to learn a world model which is simultaneously used to plan reactions resulting in optimal action sequences. It is an attempt to integrate planning, reactive, and learning systems. The architecture of Dyna is presented. The different blocks are described. There are three main components of the system. The first is the world model used by the robot for internal world representation. The input of the world model is the current state and the action taken in the current state. The output is the corresponding reward and resulting state. The second module in the system is the policy. The policy observes the current state and outputs the action to be executed by the robot. At the beginning of program execution, the policy is stochastic and through learning progressively becomes deterministic. The policy decides upon an action according to the output of an evaluation function, which is the third module of the system. The evaluation function takes the following as input: the current state of the system, the action taken in that state, the resulting state, and a reward generated by the world which is proportional to the current distance from the goal state. Originally, the work proposed was as follows: (1) to implement a simple 2-D world where a 'robot' is navigating around obstacles, to learn the path to a goal, by using lookup tables; (2) to substitute the world model and Q estimate function Q by neural networks; and (3) to apply the algorithm to a more complex world where the use of a neural network would be fully justified. In this paper, the system design and achieved results will be described. First we implement the world model with a neural network and leave Q implemented as a look up table. Next, we use a lookup table for the world model and implement the Q function with a neural net. Time limitations prevented the combination of these two approaches. The final section discusses the results and gives clues for future work.

Lima, Pedro↗