Search NASA⌕ Search

SEARCH · Search NASA

Results for “Entropy reinforcement”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A Modified Maximum Entropy Inverse Reinforcement Learning Approach for Microgrid Energy Scheduling

Increasing popularity of integrating distributed energy resources (DERs) into the power system brings a challenge to optimize the microgrid dispatch policy. The reinforcement learning methods suffer from a long-time problem with the theoretical assumption of the objective/reward function for the microgrid system. Although the traditional inverse reinforcement learning (IRL) approaches can solve this problem to some extent, they encounter a limitation of complex computations for state visitation frequency in the large and continuous state space. To alleviate this limitation, we propose a modified maximum entropy IRL (MMIRL) method to extract the reward function from the expert demonstrations for solving the microgrid energy scheduling problem. The proposed MMIRL algorithm is promising in recovering the reward function and learning the dispatch policy compared to conventional approaches. Case studies are performed in an energy arbitrage problem and a microgrid system with DERs. Results substantiate that the proposed MMIRL approach can learn the dispatch policy with more than 99% efficiency and outperforms other comparative methods.

artificial intelligence, reinforcement learning, m↗
DOE OSTI2024

Halting Oxygen Evolution to Achieve Long Cycle Life in Sodium Layered Cathodes

Oxygen redox chemistries at high voltage have materialized as a revolutionary paradigm for cathodes with high-energy density; however, they are plagued by the challenges of labile oxygen loss and rapid degradations upon cycling, even after concerted endeavors from the research community. Here we propose a multi-concentration stratagem propelled by entropy reinforcement to enhance the electronic structure disorder (ESD) at high desodiation states for impeding undesired oxygen mobility and ensuring controlled oxygen activity, elucidated by density functional theory calculations. The increased disorder strengthens the reversible electrochemistry of lattice oxygen redox, leading to effectively suppressed P−O structural evolution and highly stable localized TMO 6 octahedral environments, as demonstrated by soft/hard X-ray absorption spectroscopy. Furthermore, through a comparative analysis of sodium-layered cathodes with different configuration entropy, we reveal that a high-entropy state induced by cationic disordering has the capacity to perturb cationic redox boundaries, significantly restraining the formation of detrimental O′3 phases. As a consequence, the high-voltage cycling stability has been greatly upgraded, up to 4.4 V versus Na + /Na, with an impressive 90.1 % capacity retention at 1 C over 100 cycles and 76.1 % capacity retention at 2 C over 300 cycles. In conclusion, the resilient oxygen redox, enabled through the control of ESD, broadens the horizons for entropy engineering and lays the foundation for advancements in high-energy, long-cycling, and safe batteries.

25 ENERGY STORAGE↗
DOE OSTI2026

Exploring the holographic entropy cone via reinforcement learning

We develop a reinforcement learning algorithm to study the holographic entropy cone. Given a target entropy vector, our algorithm searches for a graph realization whose min-cut entropies match the target vector. If the target vector does not admit such a graph realization, it must lie outside the cone, in which case the algorithm finds a graph whose corresponding entropy vector most nearly approximates the target and allows us to probe the location of the facets. For the N = 3 cone, we confirm that our algorithm successfully rediscovers monogamy of mutual information beginning with a target vector outside the holographic entropy cone. We then apply the algorithm to the N = 6 cone, analyzing the 6 mystery extreme rays of the subadditivity cone from [1] that satisfy all known holographic entropy inequalities yet lacked graph realizations. We found realizations for 3 of them, proving they are genuine extreme rays of the holographic entropy cone, while providing evidence that the remaining 3 are not realizable, implying unknown holographic inequalities exist for N = 6.

AdS-CFT correspondence↗
DOE OSTI2023

Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control

Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-critic architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

High Density of Strong yet Deformable Intermetallic Nanorods Leads to an Excellent Room Temperature Strength-Ductility Combination in a High Entropy Alloy

This paper introduces a new microstructural template for high entropy alloys (HEAs), where the face centered cubic (FCC) complex concentrated solid solution is reinforced with a high density of strong, yet deformable, nanorods of an ordered multi-component intermetallic L12 compound. Thermodynamic modeling has been employed to design this HEA with a large L12 volume fraction. Thermo-mechanical processing by isothermal annealing of the conventionally processed bulk cold-rolled alloy directly at precipitation temperatures, has been applied to produce a high density of uniformly distributed L12 nanorods within refined FCC grains, resulting from concomitant recrystallization and discontinuous precipitation processes. The nanorod morphology of the discontinuous L12 product has been established from three-dimensional atom probe tomography. The refined grains result in a complete coverage of the microstructure with discontinuously precipitated intermetallic nanorods. This nanorod strengthened HEA exhibits an exceptionally high room temperature yield strength of ~1630 MPa, good tensile ductility of ~15%, and an ultimate tensile strength of ~1720 MPa. Furthermore, a single L12 phase alloy, melted based on the precipitate composition in the two-phase FCC + L12 HEA, exhibits very high compressive deformability and strain hardenability, unusual for ordered intermetallic compounds. These results open a new strategy for design of fine-grained microstructures strengthened via ordered intermetallic phases, exploiting the beneficial effects of discontinuous precipitation, for achieving very high room temperature tensile strengths while maintaining good ductility.

Gwalani, Bharat↗
NASA NTRSDate not supplied

Adaptive Stress Testing of Trajectory Predictions in Flight Management Systems

To find failure events and their likelihoods in flight-critical systems, we investigate the use of an advanced black-box stress testing approach called adaptive stress testing. We analyze a trajectory predictor from a developmental commercial flight management system which takes as input a collection of lateral waypoints and en-route environmental conditions. Our aim is to search for failure events relating to inconsistencies in the predicted lateral trajectories. The intention of this work is to find likely failures and report them back to the developers so they can address and potentially resolve shortcomings of the system before deployment. To improve search performance, this work extends the adaptive stress testing formulation to be applied more generally to sequential decision-making problems with episodic reward by collecting the state transitions during the search and evaluating at the end of the simulated rollout. We use a modified Monte Carlo tree search algorithm with progressive widening as our adversarial reinforcement learner. The performance is compared to direct Monte Carlo simulations and to the cross-entropy method as an alternative importance sampling baseline. The goal is to find potential problems otherwise not found by traditional requirements-based testing. Results indicate that our adaptive stress testing approach finds more failures and finds failures with higher likelihood relative to the baseline approaches.

adaptive stress testing↗
DOE OSTI2025

Benchmarking image processing techniques for porosity measurement in polymer additive manufacturing: Review and experimental analysis

An image processing workflow is proposed for porosity measurement in polymer additive manufacturing. Various techniques, including global and local thresholding, region growing, and K-means clustering, were applied to microscopic images of carbon fiber reinforced acrylonitrile butadiene styrene (CF-ABS) and benchmarked for their ability to accurately measure porosity. Global methods included Otsu, minimum error, iterative, and entropy-based thresholding, while local methods included Niblack, Bernsen, Sauvola, and Bradley-Roth algorithms. Artificial uneven illumination was introduced to test local adaptive thresholds. Results showed significant differences in porosity values across methods. Otsu, region growing, and K-means clustering excelled under uniform illumination, while Sauvola and Bradley-Roth performed better with uneven illumination. Comparison with X-ray computed tomography (XCT) revealed slightly lower porosity values (2.55 %) than optimized methods (2.73–2.79 %) due to XCT's lower resolution excluding smaller pores. While XCT offers finer pore detection, it limits sample volume and underestimates porosity due to spatial variation. Validation using artificial grayscale images with 5 % porosity confirmed that Otsu, Bradley-Roth, region growing, and Sauvola algorithms produced accurate results. Although tested on a single material system, these methods can be adapted to others with optimization. In conclusion, given XCT's high computational and time costs, this study highlights suitable image processing techniques as cost-effective alternatives for porosity analysis in polymer composites.

Additive manufacturing↗
DOE OSTI2023

Glassy correlated state induced by disorder in the frustrated antiferromagnet Tb 2 Zr 2 O 7

Here, we study the low-temperature thermomagnetic properties of the polycrystalline material Tb 2 Zr 2 O 7 through ac susceptibility and specific heat measurements. This zirconate displays a defect-fluorite structure in which the magnetic Tb 3+ and nonmagnetic Zr 4+ cations sit randomly on the same metal sublattice. No long-range magnetic order is found down to 100 mK, although dominant antiferromagnetic interactions are observed and the spins remain dynamic down to the lowest temperatures investigated. We observed a frequency-dependent peak around 2.5 K which is well described by many models of a canonical spin-glass transition. In-field specific heat measurements and the recovered entropy of the system R ln(4) suggest a two doublets ground state separated by 7 K. Comparisons to the pyrochlores Tb 2 Ti 2 O 7 and Tb 2 Hf 2 O 7 reinforce the collective spin-glass-type behavior and the opening for discussions of an exotic Coulomb phase in this material.

36 MATERIALS SCIENCE↗
DOE OSTI2021

In situ synchrotron X-ray imaging and mechanical properties characterization of additively manufactured high-entropy alloy composites

Laser beam directed energy deposition has become an increasingly popular advanced manufacturing technique for materials discovery as a result of the in situ alloying capability. In this study, we leverage an additive manufacturing enabled high throughput materials discovery approach to explore the composition space of a graded W x (CoCrFeMnNi) 100–x sample spanning 0 ≤ x ≤ 21 at%. In addition to microstructural and mechanical characterization, synchrotron high speed x-ray computer aided tomography was conducted on a W 20 (CoCrFeMnNi) 80 composition to visualize melting dynamics, powder-laser interactions, and remelting effects of previously consolidated material. Results reveal the formation of the Fe 7 W 6 intermetallic phase at W concentrations> 6 at%, despite the high configurational entropy. Unincorporated W particles also occurred at W concentrations> 10 at% accompanied by a dissolution band of Fe 7 W 6 at the W/matrix interface and hardness values greater than 400 HV. In this work, the primary strengthening mechanism is attributed to the reinforcement of the Fe 7 W 6 and W phases as a metal matrix composite. The in situ high speed x-ray imaging during remelting showed that an additional laser pass did not promote further mixing of the Fe 7 W 6 or W phases suggesting that, despite the dissolution of the W into the Fe 7 W 6 phase being thermodynamically favored, it is kinetically limited by the thickness/diffusivity of the intermetallic phase, and the rapid solidification of the laser-based process.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Reinforcement Learning-Guided Long-Timescale Simulation of Hydrogen Transport in Metals

Diffusion in alloys is an important class of atomic processes. However, atomistic simulations of diffusion in chemically complex solids are confronted with the timescale problem: the accessible simulation time is usually far shorter than that of experimental interest. In this work, long-timescale simulation methods are developed using reinforcement learning (RL) that extends simulation capability to match the duration of experimental interest. Two special limits, RL transition kinetics simulator (TKS) and RL low-energy states sampler (LSS), are implemented and explained in detail, while the meaning of general RL are also discussed. As a testbed, hydrogen diffusivity is computed using RL TKS in pure metals and a medium entropy alloy, CrCoNi, and compared with experiments. The algorithm can produce counter-intuitive hydrogen-vacancy cooperative motion. We also demonstrate that RL LSS can accelerate the sampling of low-energy configurations compared to the Metropolis–Hastings algorithm, using hydrogen migration to copper (111) surface as an example.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Online Monitoring System for Concrete Structures Affected by Alkali-Silica Reaction

This report presents a comprehensive study of developing ultrasonic wave and acoustic emission (AE) techniques for long-term monitoring of alkali-silica reaction (ASR) development in concrete. Small, medium, and full-scale concrete specimens were cast, conditioned, and monitored for various periods, from at least one year to 2.4 years. The reactive concrete specimens contain coarse or fine reactive aggregates to study the effects of different types of reactive aggregates. Confinements were also designed to simulate the 2-dimensional confinement effects of reinforcement in the shield building of nuclear power plants. The concrete specimens were stored in a environmental chamber with high humidity and high temperature to accelerate the ASR development. The ultrasonic monitoring data shows high sensitivity to ASR development and could detect cracking initiation well before visible surface cracks occurred. However, the linear ultrasonic analysis based on wave velocity is strongly affected by temperature variation. Therefore, a nonlinear ultrasonic method was proposed to measure thermally induced nonlinear acoustic responses of concrete (thermal modulation of ultrasonic wave). The measured nonlinear acoustic parameter shows a high correlation with ASR expansion across specimens with different reactive aggregates and confinement conditions. The same conclusion was obtained from nonlinear resonance tests on small concrete prisms. Compared to the linear acoustic methods, the nonlinear tests show high sensitivities to ASR damage from internal microcracking initiation at the early stage to visible cracks at the late stage of ASR. The attributes of the thermal modulation of nonlinear ultrasonic method include high sensitivity, immunity to temperature effects, and strong correlation with ASR expansion, which demonstrate great potentials of the nonlinear ultrasonic method for diagnosis of ASR damage and prediction of concrete deterioration process. Acoustic emission is a passive sensing technique for damage assessment, and access to only one surface is needed even for thick and heavily reinforced elements such as the walls utilized for nuclear shield building. Additionally, relatively few sensors are required to monitor the progression of the damage process. Results indicate that the AE data can be related to the damage rating index, which is a petrography-based means of assessing damage due to ASR in reinforced concrete. Furthermore, AE is capable of detecting ASR damage long before surface cracking is visually noticeable. Boundary conditions play in important role in the progression of ASR damage and differences in boundary conditions are reflected in the AE data. Entropy based data assessment methods provide a means to assess the damage state, and convolutional neural network based data assessment provided a means to assess the damage state in real-time. Results indicate that artificial neural network models may be used as a means to predict volumetric expansion based on AE data.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Replace Human Intelligence with Fast and Smart Geometric Reasoning and Graph Neural Network to Accelerate Next Gen ModSim Workflows

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Closing the Loop between In Situ Stress Complexity and EGS Fracture Complexity

We present an agent-guided approach to CAD geometry decomposition that automates hex/hybrid meshing with graph neural networks (GNNs) to accelerate next-generation ModSim workflows. Our end-to-end pipeline (i) reduces 3D boundary-representation (B-Rep) models to a 2D chordal axis skeleton (CAT) and then to a 1D bipartite graph of surface and curve nodes, (ii) assigns per node labels as Cubit® WebCut actions, (iii) trains a multi-action GNN under supervised learning, and (iv) predicts five surface-node and three curve-node actions on out-of-distribution test geometries. Each graph node carries geometric, topological, and meshing attributes drawn from the B-Rep “skin” and CAT “skeleton,” with two-way mappings across 3D↔2D↔1D representations to maintain traceability back to 3D CAD. The supervised learning model exhibits stable convergence of the binary cross-entropy loss and achieves 98.7% accuracy on unseen lattice models. To operationalize decision-making, we rank predicted commands by geometric significance and prototyped the agent-guided workflow through the Cubit® Meshing PowerTool GUI. As a stretch goal, we explore reinforcement learning (RL) to reduce or remove label requirements and to learn policies for action sequences that maximize total reward (e.g., size of hex-meshable regions and resulting hex mesh quality). When all-hex meshing is not feasible, the agent assists in producing hybrid meshes—prioritizing hex in critical regions and transitioning to tetrahedral elements (tets) elsewhere—maintaining fidelity while ensuring robustness. The overarching objective is to replace manual, heuristics-based decomposition with data-driven, reproducible automation, cutting meshing turnaround time by orders of magnitude. We anticipate direct impact on simulation workflows through intelligent, scalable decomposition of complex CAD models into hex-meshable subdomains.

42 ENGINEERING↗
DOE OSTI2023

Adaptive Deep Reinforcement Learning Algorithm for Distribution System Cyber Attack Defense With High Penetration of DERs

With grid modernization, smart inverters are increasingly used to execute advanced controls for distribution network reliability. However, this also increases the cyber-attack space. Here this paper focuses on the defense approaches to restore the system to normal operation circumstances in the presence of cyber-attacks. A unique deep reinforcement learning (DRL) method is developed to minimize voltage violations and reduce power losses for impacted feeders. The defense problem is reformulated as a Markov decision-making process to dynamically control DERs while minimizing load shedding. This is achieved via an improved soft actor-critic (SAC)-based DRL algorithm, which can govern DER set points and load-shedding scenarios in discrete and continuous modes via the auto-tune entropy and Gaussian policy features. Numerical comparison results on the modified IEEE 123-node system with other control approaches, such as Volt-VAR (VV), Volt-Watt (VW), and model predictive control (MPC) show that the proposed method can eliminate voltage violations and provide feasible control actions that perform complete mitigation of cyber-threats.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Development of coherent-precipitate-hardened high-entropy alloys with hierarchical NiAl/Ni 2 TiAl precipitates in CrMnFeCoNiAl x Ti y alloys

Coherent precipitates hardening is currently emerging strengthening mechanism of the various high entropy alloys (HEAs). Recently, CrMnFeCoNiAl x HEAs have been studied to show a phase transition from face-centered-cubic (FCC) to body-centered-cubic (BCC) and formation of coherent precipitates (B2-NiAl) within the BCC matrix. The coherent precipitates in the CrMnFeCoNiAl x alloys could contribute to increase the strength but lead to considerable reduction of the ductility. The present work systematically investigated a series of CrMnFeCoNiAl 0.5 Ti x alloys to further improve the strength and plasticity, as compared to the previously reported CrMnFeCoNiAl x HEAs. As a result, an increase of Ti addition leads to the phase transition from FCC to BCC and formation of lamellar structure and hierarchical precipitates reinforced by B2-NiAl and L2 1 -Ni 2 TiAl phases. Excellent mechanical properties were achieved from CrMnFeCoNiAl 0.5 Ti 0.1 and CrMnFeCoNiAl 0.5 Ti 0.2 alloys. Herein, the mechanical properties of the CrMnFeCoNiAl 0.5 Ti x alloys were discussed via theoretical strengthening mechanisms.

36 MATERIALS SCIENCE↗
DOE OSTI2024

In Situ Cyclized Polyacrylonitrile Coating: Key to Stabilizing Porous High-Entropy Oxide Anodes for High-Performance Lithium-Ion Batteries

High-entropy oxides (HEOs) composed of multiple metal elements have attracted great attention as anode materials for lithium-ion batteries (LIBs) due to the synergistic effects of various metal species. However, the practical applications of HEOs are still plagued by poor conductivity, unstable solid electrolyte interphase (SEI) and poor cycling stability. In this work, nanosized (FeCoNiCrMn) 3 O 4 HEO (NHEO) is prepared successfully by the NaCl-assisted mechanical ball-milling strategy. Novelly, polyacrylonitrile (PAN) is used as the binder and then in situ thermochemically cyclized to construct a cyclized PAN (cPAN) outer layer onto NHEO (NHEO-cPAN). The in situ formed cPAN coating not only improves the electrical conductivity, but also reinforces the structural and interfacial stability, and thereby, the resulted NHEO-cPAN electrode exhibits significantly enhanced rate and cyclic performance. Specifically, NHEO-PAN500 electrode delivers a high reversible capacity of 560 mAh g –1 at 5 A g –1 and a high-capacity retention of 83% over 800 cycles at 3 A g –1 . Furthermore, the structural evolution and electrochemical behavior of NHEO-PAN electrode during discharge/charge is systematically investigated by operando X-ray diffraction, in situ impedance spectroscopy and ex situ high-resolution transmission electron microscopy. Therefore, this work provides new insights into the engineering of electrode and interphase for high-performance HEO electrode materials, potentially enlightening the practical applications of HEO-based LIBs.

25 ENERGY STORAGE↗
DOE OSTI2023

Real-time quantum error correction beyond break-even

The ambition of harnessing the quantum for computation is at odds with the fundamental phenomenon of decoherence. The purpose of quantum error correction (QEC) is to counteract the natural tendency of a complex system to decohere. This cooperative process, which requires participation of multiple quantum and classical components, creates a special type of dissipation that removes the entropy caused by the errors faster than the rate at which these errors corrupt the stored quantum information. Previous experimental attempts to engineer such a process faced an excessive generation of errors that overwhelmed the error-correcting capability of the process itself. Whether it is practically possible to utilize QEC for extending quantum coherence thus remains an open question. We answer it by demonstrating a fully stabilized and error-corrected logical qubit whose quantum coherence is significantly longer than that of all the imperfect quantum components involved in the QEC process, beating the best of them with a coherence gain of G=2.27±0.07. Here we achieve this performance by combining innovations in several domains including the fabrication of superconducting quantum circuits and model-free reinforcement learning.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗