Search NASA⌕ Search

SEARCH · Search NASA

Results for “Q learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Machine learning-based real-time kinetic profile reconstruction in DIII-D

Abstract Kinetic equilibrium reconstruction plays a vital role in the physical analysis of plasma stability and control in fusion tokamaks. However, the traditional approach is subjective and prone to human biases. To address this, the consistent automatic kinetic equilibrium reconstruction (CAKE) method was introduced, providing objective results. Nonetheless, its offline nature limits its application in real-time plasma control systems (PCSs). To address this limitation, we present RTCAKENN, a machine learning model that approximates 7 CAKE-level output profiles, namely pressure, inverse q , toroidal current density, electron temperature and density, carbon ion impurity temperature and rotation profiles, using real-time available inputs. The deep neural network consists of an encoder layer, where the scalars and interdependent inputs such as plasma boundary coordinates and motional Stark effect data are encoded using multi-layer perceptrons (MLPs), while profile inputs are encoded by 1D convolutional layers. The encoded data is passed through a MLP for latent feature extraction, before being decoded in the decoding layers, which consist of upsampling and convolutional layers. RTCAKENN has been implemented in the DIII-D PCS and our model achieves accuracy comparable to CAKE and surpasses existing real-time alternatives. Through clever dropout training, RTCAKENN exhibits robustness and can operate even in the absence of Thomson scattering data or charge exchange recombination data. It executes in under 8 ms in the real-time environment, enabling future application in real-time control and analysis.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

A fresh look at electron cyclotron current drive power requirements for stabilization of tearing modes in ITER

ITER is an international project to design and build an experimental fusion reactor based on the “tokamak” concept. ITER relies upon localized electron cyclotron current drive (ECCD) at the rational safety factor q=2 to suppress or stabilize the expected poloidal mode m=2, toroidal mode n=1 neoclassical tearing mode (NTM) islands. Such islands if unmitigated degrade energy confinement, lock to the resistive wall (stop rotating), cause loss of “H-mode” and induce disruption. The International Tokamak Physics Activity (ITPA) on MHD, Disruptions and Magnetic Control joint experiment group MDC-8 on Current Drive Prevention/Stabilization of Neoclassical Tearing Modes started in 2005, after which assessments were made for the requirements for ECCD needed in ITER, particularly that of rf power and alignment on q=2 [1]. Narrow well-aligned rf current parallel to and of order of one percent of the total plasma current is needed to replace the “missing” current in the island O-points and heal or preempt (avoid destabilization by applying ECCD on q=2 in absence of the mode) the island [2-4]. This paper updates the advances in ECCD stabilization on NTMs learned in DIII-D experiments and modeling during the last 5 to 10 years as applies to stabilization by localized ECCD of tearing modes in ITER. This includes the ECCD (inside the q=1 radius) stabilization of the NTM “seeding” instability known as sawteeth (m/n=1/1) [5]. Recent measurements in DIII-D show that the ITER-similar current profile is classically unstable, curvature stabilization must not be neglected, and the small island width stabilization effect from helical ion polarization currents is stronger than was previously thought [6]. The consequences of updated assumptions in ITER modeling of the minimum well-aligned ECCD power needed are all-in-all favorable (and well-within the ITER 24 gyrotron capability) when all effects are included. However, a “wild card” may be broadening of the localized ECCD by the presence of the island; various theories predict broadening could occur and there is experimental evidence for broadening in DIII-D. Wider than now expected ECCD in ITER would make alignment easier to do but weaken the stabilization and thus require more rf power. In addition to updated modeling for ITER, advances in the ITER-relevant DIII-D ECCD gyrotron launch mirror control system hardware and real-time plasma control system have been made [7] and there are plans for application in DIII-D ITER demonstration discharges.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Molecular fingerprint and machine learning to accelerate design of high-performance homochiral metal–organic frameworks

In this report computational screening was employed to calculate the enantioseparation capabilities of 45 functionalized homochiral metal–organic frameworks (FHMOFs), and machine learning (ML) and molecular fingerprint (MF) techniques were used to find new FHMOFs with high performance. With increasing temperature, the enantioselectivities for (R,S)-1,3-dimethyl-1,2-propadiene are improved. The “glove effect” in the chiral pockets was proposed to explain the correlations between the steric effect of functional groups and performance of FHMOFs. Moreover, the neighborhood component analysis and RDKit/MACCS MFs show the highest predictive effect on enantioselectivities among the four ML classification algorithms with nine MFs that were tested. Based on the importance of MF, 85 new FHMOFs were designed, and a newly designed FHMOF, NO 2 -NHOH-FHMOF, with high similarity to the optimal MFs achieved improved chiral separation performance, with enantioselectivities of 85%. The design principles and new chiral pockets obtained by ML and MFs could facilitate the development of new materials for chiral separation.

molecular fingerprint↗

An Approach to Evaluate Scientist Support in Abstract Workflows and Provenance Traces

Abstract workflows are useful to bridge the gap between scientists and technologists towards using computer systems to carry out scientific processes. Provenance traces provide evidence required to validate results and support their reuse. Assuming both technologies are based on formal semantics, a knowledge-based system that consistently merges both technologies is useful for scientists that produce data to document their data collecting and transformation processes; it is also useful for scientists that reuse data to assess scientific processes and resulting datasets produced by others. While evaluation of each technology is necessary for a given application, this work discusses their combined evaluation. The claim is that both technologies should complement each other and align consistently to a scientist’s perspective in order to be effective for science. Evaluation criteria are proposed based on lessons learned and exemplified for discussion.

Provenance↗

Soil Organic Matter Temperature Sensitivity Cannot be Directly Inferred From Spatial Gradients

Developing and testing decadal-scale predictions of soil response to climate change is difficult because there are few long-term warming experiments or other direct observations of temperature response. As a result, spatial variation in temperature is often used to characterize the influence of temperature on soil organic carbon (SOC) stocks under current and warmer temperatures. This approach assumes that the decadal-scale response of SOC to warming is similar to the relationship between temperature and SOC stocks across sites that are at quasi steady state; however, this assumption is poorly tested. We developed four variants of a Reaction-network-based model of soil organic matter and microbes using measured SOC stocks from a 4,000-km latitudinal transect. Each variant reflects different assumptions about the temperature sensitivities of microbial activity and mineral sorption. All four model variants predicted the same response of SOC to temperature at steady state, but different projections of transient warming responses. The relative importance of Q max , mean annual temperature, and net primary production, assessed using a machine-learning algorithm, changed depending on warming duration. When mineral sorption was temperature sensitive, the predicted average change in SOC after 100 years of 5 °C warming was –18% if warming decreased sorption or +9% if warming increased sorption. When microbial activity was temperature sensitive but mineral sorption was not, average site-level SOC loss was 5%. Here, we conclude that spatial climate gradients of SOC stocks are insufficient to constrain the transient response; measurements that distinguish process controls and/or observations from long-term warming experiments, especially mineral fractions, are needed.

54 ENVIRONMENTAL SCIENCES↗

Neural network potentials with effective charge separation for non-equilibrium dynamics of ionic solids: a ZnO case study

Developing neural network potentials (NNPs) accurate under non-equilibrium dynamics is challenging, as such systems require extensive sampling beyond equilibrium phases. Here we construct high-fidelity NNPs for zinc oxide (ZnO), a polymorphic ionic solid, using density functional theory (DFT) reference data. To efficiently capture transitional configurations, we combine enhanced-sampling molecular dynamics with empirical potentials, data distillation, and pretraining on short-range atomic energies (A-Train), followed by transfer learning with DFT-relabeled datasets. This hierarchical approach improves transferability across polymorphs and stress states. We further introduce effective charge separation, treating long-range Coulombic terms analytically while short-range residual interactions are learned by the NNP. The optimal effective charges fall in the range 0.5–1.0 q e , consistent with dielectric-screened values derived from formal charges but distinct from Bader estimates. Motivated by this observation, we propose a simple data-driven protocol in which effective charges are optimized by comparing DFT reference energies with explicit Coulomb calculations, without additional NNP training. This strategy improves accuracy and transferability in DFT-level predictions of energies, forces, and stress. Together, these results provide a practical charge-selection framework for robust NNP development in ionic solids, enabling reliable simulation of polymorphic phase transformations and non-equilibrium dynamics.

Chemistry↗

Phonon second harmonic generation in NaBr studied by inelastic neutron scattering and computer simulation

The phenomenon of second harmonic generation (SHG) was found for phonons in anharmonic NaBr by inelastic neutron scattering. The temperature dependence of this phonon SHG was measured from 300 K to 650 K. At 300 K the second harmonic (SH) is seen as a high-energy branch around 33 meV, nearly independent of $\overrightarrow{Q}$. The temperature effective potential (TDEP) method and classical molecular dynamics (MD) simulation with machine learning interatomic potential were able to reproduce the SH, and showed that SHG occurs with the flat transverse optical (TO) phonon branch. A classical model of a nonlinear medium explains the intensity and lifetime of the SH, compared to those of the TO modes. Also successful was a quantum model based on the Heisenberg-Langevin equation for interacting phonons coupled to a thermal bath, which also predicts a spectral distribution of the SH. In conclusion, the measured temperature dependence of the intensity of the second harmonic showed that it follows the Planck distribution of a one-phonon quasiparticle, and not two TO phonons.

36 MATERIALS SCIENCE↗

Studying Transient Phenomena in Thin Films with Reinforcement Learning

Neutron reflectometry has long been a powerful tool to study the interfacial properties of energy materials. Recently, time-resolved neutron reflectometry has been used to better understand transient phenomena in electrochemical systems. Those measurements often comprise a large number of reflectivity curves acquired over a narrow q range, with each individual curve having lower information content compared to a typical steady-state measurement. In this work, we present an approach that leverages existing reinforcement learning tools to model time-resolved data to extract the time evolution of structure parameters. Further, by mapping the reflectivity curves taken at different times as individual states, we use the Soft Actor-Critic algorithm to optimize the time series of structure parameters that best represent the evolution of an electrochemical system. We show that this approach constitutes an elegant solution to the modeling of time-resolved neutron reflectometry data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Kernel methods for evolution of generalized parton distributions

Generalized parton distributions (GPDs) characterize the 3-dimensional structure of hadrons, combining information about their internal quark and gluon longitudinal momentum distributions and transverse position within the hadron. The dependence of GPDs on the factorization scale Q 2 allows one to connect hard exclusive processes involving GPDs at disparate energy and momentum scales, which is needed in global analyses of experimental data. Here, in this work, we explore how finite element methods can be used to construct fast and differentiable Q 2 evolution codes for GPDs in momentum space, which can be used in a machine learning framework. We show numerical benchmarks of the methods' accuracy, including a comparison to an existing evolution code from PARTONS/APFEL++, and provide a repository where the code can be accessed.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Investigation of porous polymer gas chromatographic packings for atmospheric analysis of extraterrestrial bodies

Measurement of the permanent gases in the atmospheres of mission targets is a major objective. A 16 meter long Porapak N column was used on the Venus probe and required a rather high carrier gas flow rate. The researchers have, therefore, surveyed commercial porous polymer types which had some ability to resolve nitrogen, oxygen, argon and carbon monoxide gases. Porapaks N and Q appeared superior to most. Batch to batch variation, however, was quite wide, so the researchers learned how to synthesize porous polymer and investigated some of the factors affecting the separations. A polymer was synthesized which was superior to all commercial products and allowed at least a 50% reduction in length and flow rate of carrier gas. Similar studies were made concerning the separation of hydrocarbons and new porous polymers have been synthesized which represent significant improvements in time of analysis, column, and carrier gas flow rate.

Pollock, G. E.↗

Nanoindenting the Chelyabinsk Meteorite to Learn about Impact Deflection Effects in asteroids

The Chelyabinsk meteorite is a highly shocked, low porosity, ordinary chondrite, probably similar to S- or Q-type asteroids. Therefore, nanoindentation experiments on this meteorite allow us to obtain key data to understand the physical properties of near-Earth asteroids. Tests at different length scales provide information about the local mechanical properties of the minerals forming this meteorite: reduced Young’s modulus, hardness, elastic recovery, and fracture toughness. Those tests are also useful to understand the potential to deflect threatening asteroids using a kinetic projectile. We found that the differences in mechanical properties between regions of the meteorite, which increase or reduce the efficiency of impacts, are not a result of compositional differences. A low mean particle size, attributed to repetitive shock, can increase hardness, while low porosity promotes a higher momentum multiplication. Momentum multiplication is the ratio between the change in momentum of a target due to an impact, and the momentum of the projectile, and therefore, higher values imply more efficient impacts. In the Chelyabinsk meteorite, the properties of the light-colored lithology materials facilitate obtaining higher momentum multiplication values, compared to the other regions described for this meteorite. Also, we found a low value of fracture toughness in the shock-melt veins of Chelyabinsk, which would promote the ejection of material after an impact and therefore increase the momentum multiplication. These results are relevant to the growing interest in missions to test asteroid deflection, such as the recent collaboration between the European Space Agency and NASA, known as the Asteroid Impact and Deflection Assessment mission.

79 ASTRONOMY AND ASTROPHYSICS↗

Using neural networks and Dyna algorithm for integrated planning, reacting and learning in systems

The traditional AI answer to the decision making problem for a robot is planning. However, planning is usually CPU-time consuming, depending on the availability and accuracy of a world model. The Dyna system generally described in earlier work, uses trial and error to learn a world model which is simultaneously used to plan reactions resulting in optimal action sequences. It is an attempt to integrate planning, reactive, and learning systems. The architecture of Dyna is presented. The different blocks are described. There are three main components of the system. The first is the world model used by the robot for internal world representation. The input of the world model is the current state and the action taken in the current state. The output is the corresponding reward and resulting state. The second module in the system is the policy. The policy observes the current state and outputs the action to be executed by the robot. At the beginning of program execution, the policy is stochastic and through learning progressively becomes deterministic. The policy decides upon an action according to the output of an evaluation function, which is the third module of the system. The evaluation function takes the following as input: the current state of the system, the action taken in that state, the resulting state, and a reward generated by the world which is proportional to the current distance from the goal state. Originally, the work proposed was as follows: (1) to implement a simple 2-D world where a 'robot' is navigating around obstacles, to learn the path to a goal, by using lookup tables; (2) to substitute the world model and Q estimate function Q by neural networks; and (3) to apply the algorithm to a more complex world where the use of a neural network would be fully justified. In this paper, the system design and achieved results will be described. First we implement the world model with a neural network and leave Q implemented as a look up table. Next, we use a lookup table for the world model and implement the Q function with a neural net. Time limitations prevented the combination of these two approaches. The final section discusses the results and gives clues for future work.

Lima, Pedro↗

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗

Decentralised Reinforcement Learning for Dynamic Cyberattack Response in Microgrid Networks

Microgrids rely on communication networks for reliable operation, which makes them inherently vulnerable to cyberattacks. Such attacks can destabilise system dynamics and drive states away from their nominal operating trajectories. Although several physics-informed and machine learning-based strategies have been developed to counter these threats, the rapidly evolving cyber landscape enables adversaries to bypass static defences or rules-based mitigation approaches. This paper proposes a dynamic, online-trained and fully decentralised reinforcement learning (RL)-based cyberattack response framework to protect microgrids from evolving cyberattacks. The proposed framework deploys multiple deep Q-networks (DQNs), each associated with a distributed energy resource (DER), to enable localised and adaptive attack mitigation. In this framework, each DQN processes local voltage and frequency measurements—combined with intrusion detection system (IDS) alerts—as observations and rewards to guide decision-making. Extensive simulation studies demonstrate the robustness of the proposed framework under diverse attack scenarios and varying IDS-induced detection delays. Comparative analysis highlights its superiority over existing static or preexisting rules-based mitigation approaches. Finally, we present an analysis that shows the framework's scalability to real-life microgrids with more interacting agents.

24 POWER TRANSMISSION AND DISTRIBUTION↗

SU-E-I-41: Dictionary Learning Based Quantitative Reconstruction for Low-Dose Dual-Energy CT (DECT)

Purpose: DECT collects two sets of projection data under higher and lower energies. With appropriates composition methods on linear attenuation coefficients, quantitative information about the object, such as density, can be obtained. In reality, one of the important problems in DECT is the radiation dose due to doubled scans. This work is aimed at establishing a dictionary learning based reconstruction framework for DECT for improved image quality while reducing the imaging dose. Methods: In our method, two dictionaries were learned respectively from the high-energy and lowenergy image datasets of similar objects under normal dose in advance. The linear attenuation coefficient was decomposed into two basis components with material based composition method. An iterative reconstruction framework was employed. Two basis components were alternately updated with DECT datasets and dictionary learning based sparse constraints. After one updating step under the dataset fidelity constraints, both high-energy and low-energy images can be obtained from the two basis components. Sparse constraints based on the learned dictionaries were applied to the high- and low-energy images to update the two basis components. The iterative calculation continues until a pre-set number of iteration was reached. Results: We evaluated the proposed dictionary learning method with dual energy images collected using a DECT scanner. We re-projected the projection data with added Poisson noise to reflect the low-dose situation. The results obtained by the proposed method were compared with that obtained using FBP based method and TV based method. It was found that the proposed approach yield better results than other methods with higher resolution and less noise. Conclusion: The use of dictionary learned from DECT images under normal dose is valuable and leads to improved results with much lower imaging dose.

60 APPLIED LIFE SCIENCES↗

A planning quality evaluation tool for prostate adaptive IMRT based on machine learning

Purpose: To ensure plan quality for adaptive IMRT of the prostate, we developed a quantitative evaluation tool using a machine learning approach. This tool generates dose volume histograms (DVHs) of organs-at-risk (OARs) based on prior plans as a reference, to be compared with the adaptive plan derived from fluence map deformation. Methods: Under the same configuration using seven-field 15 MV photon beams, DVHs of OARs (bladder and rectum) were estimated based on anatomical information of the patient and a model learned from a database of high quality prior plans. In this study, the anatomical information was characterized by the organ volumes and distance-to-target histogram (DTH). The database consists of 198 high quality prostate plans and was validated with 14 cases outside the training pool. Principal component analysis (PCA) was applied to DVHs and DTHs to quantify their salient features. Then, support vector regression (SVR) was implemented to establish the correlation between the features of the DVH and the anatomical information. Results: DVH/DTH curves could be characterized sufficiently just using only two or three truncated principal components, thus, patient anatomical information was quantified with reduced numbers of variables. The evaluation of the model using the test data set demonstrated its accuracy {approx}80% in prediction and effectiveness in improving ART planning quality. Conclusions: An adaptive IMRT plan quality evaluation tool based on machine learning has been developed, which estimates OAR sparing and provides reference in evaluating ART.

60 APPLIED LIFE SCIENCES↗