Search NASA⌕ Search

SEARCH · Search NASA

Results for “model-free”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

14 SOLAR ENERGY↗
DOE OSTI2024

A safe reinforcement learning algorithm for supervisory control of power plants

Traditional control theory-based methods require tailored engineering for each system and constant fine-tuning. In power plant control, one often needs to obtain a precise representation of the system dynamics and carefully design the control scheme accordingly. Model-free Reinforcement learning (RL) has emerged as a promising solution for control tasks due to its ability to learn from trial-and-error interactions with the environment. It eliminates the need for explicitly modeling the environment’s dynamics, which is potentially inaccurate. However, the direct imposition of state constraints in power plant control raises challenges for standard RL methods. To address this, we propose a chance-constrained RL algorithm based on Proximal Policy Optimization for supervisory control. Our method employs Lagrangian relaxation to convert the constrained optimization problem into an unconstrained objective, where trainable Lagrange multipliers enforce the state constraints. In conclusion, our approach achieves the smallest distance of violation and violation rate in a load-follow maneuver for an advanced Nuclear Power Plant design.

constrained optimization↗
DOE OSTI2024

Unraveling the multi-step crystallization mechanism of polytetrafluoroethylene, modified polytetrafluoroethylene, and their nanocomposites with boron nitride nanobarbs: Experimental insights and theoretical analysis

The non-isothermal crystallization behavior and kinetics of polytetrafluoroethylene (PTFE) composites with boron nitride nanobarb (BNNB), a new generation nanostructure with unique surface morphology and mechanical “barbs” have been analyzed, understanding these properties is essential for their high-end applications as thermal interface materials (TIM) for microwave, 5G and microelectronic devices. The analysis of the crystallization parameters includes crystallization onset, peak and end temperatures, crystallization half-life and overall crystallinity of PTFE, modified PTFE and their BNNB composites. The results were further analyzed using theoretical models such as the combined Avrami-Ozawa model. It was found that BNNB supports crystallization in the modified PTFE but shows minimal effect on the crystallization of PTFE. Due to the limitation of the classical theoretical models used above in fully characterizing the multi-step crystallization process of PTFE, an in-depth analysis using the model-free advanced isoconversional computation was used to characterize the PTFE crystallization based on the evolution of activation energy with fractional crystallinity and for the first time with temperature. Three kinetic regions were identified in the crystallization mechanism. Here, this study investigated the molecular organization and microstructural evolution of PTFE, modified PTFE and their composites during non-isothermal crystallization using advanced X-ray scattering measurements. An insight into the changes undergone by the material's microstructural units including crystallite size and morphology, lamellar thickness and lamellar interfacial layer thickness, and crystallographic phase dynamics during non-isothermal cooling from the melt, was provided here in this work. The effect of copolymer modification of PTFE and the inclusion of pristine and functionalized BNNB (a thermally conductive and electrically insulating ceramic) are both new investigations that provide valuable knowledge for the development of materials with strong matrix-nanofiller interaction and guidance for optimizing sintering and cooling cycles, two key steps in PTFE processing that largely affect the material microstructural features. Overall, the result of the three-part investigation demonstrates that BNNB supports crystallization in the modified PTFE up to 20 wt% concentration and at low and high cooling rates typically used in the industrial processing of PTFE.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Generation-Storage Coordination Dispatch Strategy for Power System Based on Causal Reinforcement Learning

In the backdrop of global energy transformation, power systems integrating high proportions of renewable energy sources are facing unprecedented challenges in operational stability and dispatch efficiency. To address these challenges, this study introduces a generation-storage coordination real-time dispatch strategy based on Causal Power System Dynamic Reinforcement Learning (CPSDRL). Diverging from traditional reinforcement learning approaches, CPSDRL innovatively incorporates causal inference within the state prediction model - the crux of model-based reinforcement learning - thereby establishing the Power Causal Dynamic Model (PCDM). Assisted by the prior knowledge of power systems, the model significantly enhances prediction accuracy and reliability through a two-stage training process. Utilizing PCDM, this study further applies a direct policy search algorithm to optimize the real-time dispatch strategy. Experimental results indicate that the proposed method improves the stability of generation-storage coordination real-time dispatch and exhibits competitive advantages in sample efficiency and computational speed, compared to traditional model-based and model-free reinforcement learning algorithms. This method is expected to enhance the practicality and adaptability of causal reinforcement learning techniques in power system scheduling and control.

causal reinforcement learning↗
DOE OSTI2024

Ice Phase Classification Made Easy with Score-Based Denoising

Accurate identification of ice phases is essential for understanding various physicochemical phenomena. However, such classification for structures simulated with molecular dynamics is complicated by the complex symmetries of ice polymorphs and thermal fluctuations. For this purpose, both traditional order parameters and data-driven machine learning approaches have been employed, but they often rely on expert intuition, specific geometric information, or large training data sets. In this work, we present an unsupervised phase classification framework that combines a score-based denoiser model with a subsequent model-free classification method to accurately identify ice phases. Further, the denoiser model is trained on perturbed synthetic data of ideal reference structures, eliminating the need for large data sets and labeling efforts. The classification step utilizes the smooth overlap of atomic position (SOAP) descriptors as the atomic fingerprint, ensuring Euclidean symmetries and transferability to various structural systems. Our approach achieves a remarkable 100% accuracy in distinguishing ice phases of test trajectories using only seven ideal reference structures of ice phases as model inputs. This demonstrates the generalizability of the score-based denoiser model in facilitating phase identification for complex molecular systems. The proposed classification strategy can be broadly applied to investigate structural evolution and phase identification for a wide range of materials, offering new insights into the fundamental understanding of water and other complex systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Influence of Grafting Density and the Ionic Environment on the Structure of Zwitterionic Brushes

Zwitterionic polymer brushes possess a high potential for applications as surface coatings, e.g., in antifouling applications. Their complex association behavior due to the coexistence of oppositely charged groups on the same monomeric unit allows for a broad variation of polarity, hydrophilicity, and the antipolyelectrolyte effect with the variation of the surrounding environment. In this study, planar polysulfobetaine brushes were investigated with neutron reflectometry (NR) and neutron spin–echo spectroscopy under grazing incidence (GINSES) to explore the brush structure and its inner dynamics perpendicular to the substrate. In particular, the effects of the substrate–initiator system and ionic strength on the structure were investigated with neutron reflectometry and a swelling by a factor of 3–5 was found in the presence of aqueous NaCl and MgCl 2 solutions compared to the dry state. During the data analysis, the applicability of a model-free evaluation was also demonstrated for the investigated polysulfobetaine brushes. Furthermore, it was found with GINSES that in contrast to neutral brushes, these polysulfobetaine brushes do not show the typical concentration fluctuations in the nanosecond time range when partially swollen with salt-free water.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Kinetics of Pyrolysis and Thermal Evolution of Negev Desert Lithologies

The Negev desert in Israel is home to large quantities of organic-rich, shallow marine sedimentary lithologies that could potentially accommodate the disposal of spent nuclear fuel. Previous thermal analyses of Negev carbonates have focused on industrially relevant considerations such as natural gas and oil extraction or pyrolysis for recovering hydrocarbon fuels. Here, this study addresses thermal evolution of the Negev organic-rich carbonate, siliceous, and phosphorite rocks and associated chemical, mineralogical, and microstructural changes that may occur under prolonged thermal loading in the vicinity of spent nuclear fuel disposal systems. Our employed methods include high-temperature X-ray diffraction, high-temperature infrared spectroscopy, and thermal analysis integrating thermogravimetry, differential scanning calorimetry, and mass spectrometry. Further, we apply iterative iso-conversional model-free methods to derive kinetic parameters for thermal decomposition of the Negev organic-rich carbonate rocks from 200 to 550 °C. Our results have provided mechanistic insights into the thermal evolution encompassing water desorption, decomposition of organic matter, and decarbonation of carbonate phases.

58 GEOSCIENCES↗
DOE OSTI2024

X-ray Thomson scattering absolute intensity from the f-sum rule in the imaginary-time domain

We present a formally exact and simulation-free approach for the normalization of X-ray Thomson scattering (XRTS) spectra based on the f-sum rule of the imaginary-time correlation function (ITCF). Our method works for any degree of collectivity, over a broad range of temperatures, and is applicable even in nonequilibrium situations. In addition to giving us model-free access to electronic correlations, this new approach opens up the intriguing possibility to extract a plethora of physical properties from the ITCF based on XRTS experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

A multidimensional approach to quantum state tomography of photoelectron wavepackets

There is a growing interest in reconstructing the density matrix of photoelectron wavepackets, in particular in complex systems where decoherence can be introduced either by a partial measurement of the system or through coupling with a stochastic environment. To this end, several methods to reconstruct the density matrix, quantum state tomography protocols, have been developed and tested on photoelectrons ejected from noble gases following absorption of extreme ultraviolet (XUV) photons from attosecond pulses. It remains a challenge to obtain model-free, single scan protocols that can reconstruct the density matrix with high fidelities. Current methods require extensive measurements or involve complex fitting of the signal. Efficient single-scan reconstructions would be of great help to increase the number of systems that can be studied. We propose a new and more efficient protocol that is able to reconstruct the continuous variable density matrix of a photoelectron in a single time delay scan. It is based on measuring the coherences of a photoelectron created by absorption of an XUV pulse using a broadband infrared (IR) probe that is scanned in time and a narrowband IR reference that is temporally fixed to the XUV pulse. We illustrate its performance for a Fano resonance in He as well as mixed states in Ar arising from spin-orbit splitting. We show that the protocol results in excellent fidelities and near-perfect estimation of the purity.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Time irreversibility as an indicator of approaching tipping points in Earth subsystems

With shifting environmental trends, many Earth system elements may be poised to undergo critical transitions or ‘tipping’. Reliable anticipation of these tipping elements is vital to inform policy decisions. Many of the current methods for tipping point detection are based on loss of resilience or ‘critical slowdown’ of the system as it approaches a tipping point. However, these methods are prone to false alarms; the detected slowdown may be an artifact of nonstationary noise unrelated to tipping behavior. Here, we explore the efficacy of early warning signs based on a nonequilibrium thermodynamics framework. The model-free detection method relies on the increased intrinsic time-irreversibility due to detailed balance breaking, preceding the onset of tipping or instabilities. We demonstrate that these EWSs are effective for tipping point detection and robust against false alarms due to nonstationary noise, using idealized models for two key elements of the Earth system that are prone to tipping: the Atlantic Meridional Overturning Circulation and Arctic sea-ice loss.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Charges on a suspended silicon nitride membrane under a high-energy electron beam

Thin silicon nitride (SiN x ) membranes are widely used in gas and liquid phase transmission electron microscopy (TEM) and as phase plates to enhance imaging contrast. SiN x contains trap sites for both positive and negative charges, which can be manipulated by high-energy electron irradiation, external potential biasing, or light exposure. Charge accumulation on the membrane can significantly affect in situ TEM processes, including chemical and electrochemical reactions, nanoparticle dynamics, and catalytic activity, or introduce unwanted phase shifts when used as a phase plate. Here, in this study, charge accumulation on suspended SiN x membranes was investigated using off-axis electron holography combined with model-free charge analysis, supported by custom finite element analysis (FEA) simulations. An average residual positive charge density of approximately 2.8 × 10 −4 C m −2 was measured. Localized and stable regions of both positive and negative charges were identified on the membrane. The global positive and localized positive/negative charges give rise to strong electric fields and electroosmotic slip velocities at the membrane surface, which are sufficient to induce non-Brownian particle behavior and directional fluid flow, offering a physical explanation for previously observed anomalies in particle dynamics, nucleation, and growth during gas and liquid phase TEM experiments. These results provide a benchmark for understanding charge behavior at SiNx interfaces in gas and liquid phase TEM. Furthermore, the FEA simulations establish a framework for future investigations into charge distribution, electrostatic potentials, and electrical double layers at solid–liquid interfaces, particularly in complex geometries and chemically dynamic environments.

Suspended silicon↗
DOE OSTI2024

Effects of mosaic crystal instrument functions on x-ray Thomson scattering diagnostics

Mosaic crystals, with their high integrated reflectivities, are widely employed in spectrometers used to diagnose high energy density systems. X-ray Thomson scattering (XRTS) has emerged as a powerful diagnostic tool of these systems, providing in principle direct access to important properties such as the temperature via detailed balance. However, the measured XRTS spectrum is broadened by the spectrometer instrument function (IF), and without careful consideration of the IF one risks misdiagnosing system conditions. Here, we consider in detail the IF of 40 and 100 μm mosaic Highly Annealed Pyrolytic Graphite crystals, and how the broadening varies across the spectrometer in an energy range of 6.7–8.6 keV. Notably, we find a strong asymmetry in the shape of the IF toward higher energies. As an example, we consider the effect of the asymmetry in the IF on the temperature inferred via XRTS for simulated 80 eV CH plasmas and find that the temperature can be overestimated if an approximate symmetric IF is used. We, therefore, expect a detailed consideration of the full IF will have an important impact on system properties inferred via XRTS in both forward modeling and model-free approaches.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Correlation function metrology for warm dense matter: Recent developments and practical guidelines

X-ray Thomson scattering (XRTS) has emerged as a valuable diagnostic for matter under extreme conditions, as it captures the intricate many-body physics of the probed sample. Recent advances, such as the model-free temperature diagnostic of Dornheim et al. [Nat. Commun. 13 , 7911 (2022)], have demonstrated how much information can be extracted directly within the imaginary-time formalism. However, since the imaginary-time formalism is a concept often difficult to grasp, we provide here a systematic overview of its theoretical foundations and explicitly demonstrate its practical applications to temperature inference, including relevant subtleties. Furthermore, we present recent developments that enable the determination of the absolute normalization, Rayleigh weight, and density from XRTS measurements without reliance on uncontrolled model assumptions. Finally, we outline a unified workflow that guides the extraction of these key observables, offering a practical framework for applying the method to interpret experimental measurements.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Reinforcement learning pulses for transmon qubit entangling gates

The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor—in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Harnessing the power of gradient-based simulations for multi-objective optimization in particle accelerators

Abstract Particle accelerator operation requires simultaneous optimization of multiple objectives. Multi-objective optimization (MOO) is particularly challenging due to trade-offs between the objectives. Evolutionary algorithms, such as genetic algorithms (GAs), have been leveraged for many optimization problems, however, they do not apply to complex control problems by design. This paper demonstrates the power of differentiability for solving MOO problems in particle accelerators using a deep differentiable reinforcement learning (DDRL) algorithm. We compare the DDRL algorithm with model-free reinforcement learning (MFRL), GA, and Bayesian optimization (BO) for simultaneous optimization of heat load and trip rates in the continuous electron beam accelerator facility. The underlying problem enforces strict constraints on both individual states and actions as well as cumulative (global) constraints on energy requirements of the beam. Using historical accelerator data, we develop a physics-based surrogate model which is differentiable and allows for back-propagation of gradients. The results are evaluated in the form of a Pareto-front with two objectives. We show that the DDRL outperforms MFRL, BO, and GA on high dimensional problems.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Hamiltonian learning using machine-learning models trained with continuous measurements

Here, we build upon recent work on the use of machine-learning models to estimate Hamiltonian parameters using continuous weak measurement of qubits as input. We consider two settings for the training of our model: (1) supervised learning, where the weak-measurement training record can be labeled with known Hamiltonian parameters, and (2) unsupervised learning, where no labels are available. The first has the advantage of not requiring an explicit representation of the quantum state, thus potentially scaling very favorably to a larger number of qubits. The second requires the implementation of a physical model to map the Hamiltonian parameters to a measurement record, which we implement using an integrator of the physical model with a recurrent neural network to provide a model-free correction at every time step to account for small effects not captured by the physical model. We test our construction on a system of two qubits and demonstrate accurate prediction of multiple physical parameters in both the supervised context and the unsupervised context. We demonstrate that the model benefits from larger training sets, establishing that it is “learning,” and we show robustness regarding errors in the assumed physical model by achieving accurate parameter estimation in the presence of unanticipated single-particle relaxation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Desmearing two-dimensional small-angle neutron scattering data by central moment expansions

Resolution smearing is a critical challenge in the quantitative analysis of two-dimensional small-angle neutron scattering (SANS) data, particularly in studies of soft-matter flow and deformation using SANS. Here, we present a central moment expansion technique to address smearing in anisotropic scattering spectra, offering a model-free desmearing methodology. By accounting for directional variations in resolution smearing and enhancing computational efficiency, this approach reconstructs desmeared intensity distributions from smeared experimental data. Computational benchmarks using interacting hard-sphere fluids and Gaussian chain models validate the accuracy of the method, while simulated noise analyses confirm its robustness under experimental conditions. Experimental validation using rheological SANS data from shear-induced micellar structures demonstrates the practicality and effectiveness of the proposed algorithm. The desmearing technique provides a powerful tool for advancing the quantitative analysis of anisotropic scattering patterns, enabling precise insights into the interplay between material microstructure and macroscopic flow behavior.

anisotropic scattering spectra↗