Search NASA⌕ Search

SEARCH · Search NASA

Results for “Induction period”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Effect of ZnO on the whiteness of white Portland cement clinker

The addition of ZnO to significantly improve the Hunter whiteness of white Portland cement (WPC) clinkers is introduced in this paper. The effects of ZnO on the whiteness of WPC were studied by chroma analysis, X-ray diffraction (XRD), the Rietveld method, hydration heat measurements and SEM-EDS. The results showed that adding ZnO increased the visible light reflectance, increased the brightness (L{sup ⁎}), and reduced the yellowness (b{sup ⁎}) of WPC clinker with high Fe{sub 2}O{sub 3}. Therefore, ZnO can increase the whiteness of WPC clinker. Zn was mainly distributed in the amorphous phase and unquantifiable crystalline phase (ACn), and then dissolved in the silicate phase. ZnO stabilized R-C{sub 3}S, reduced the crystallization of C{sub 3}A, and increased the ACn content. ZnO increased the amount of Fe in the ACn and minerals, thereby reducing the formation of C{sub 4}AF and improving whiteness. ZnO increased the accumulated heat of hydration but prolonged the induction period. This method provides a new approach for the WPC industry to broaden sources of raw materials and improve product quality.

36 MATERIALS SCIENCE↗

Effect of seawater as mixing water on the hydration behaviour of tricalcium aluminate

This study explored the mechanism of the effect of seawater on tricalcium aluminate (C{sub 3}A) hydration. The results showed that seawater retarded C{sub 3}A hydration and reduced the reaction degree of C{sub 3}A. The co-existence or ion pairing of Ca{sup 2+} and SO{sub 4}{sup 2−} onto the surface of C{sub 3}A poisoning the reactive sites is the main reason for this retardation. Besides, the precipitation of Mg(OH){sub 2} on the surface of C{sub 3}A would prolong the induction period for another 30 min and then consequently decrease the dissolution rate of C{sub 3}A hydration. Trace amounts of MgSO{sub 4}{sup 0} and Mg{sup 2+} present in the alkaline solution had little retardation effect. It was also found that Cl{sup −} would preferentially react with C{sub 3}A to form Friedel's salt, rather than forming hydroxy-AFm as that in the C{sub 3}A-deionized water paste. The direct formation of Friedel's salt resulted in an accumulation of Al{sup 3+} in solution, which would also hinder the subsequent dissolution of C{sub 3}A.

36 MATERIALS SCIENCE↗

Infrared laser ablation of poly(vinylidene fluoride): The Loss of HF

The mechanism and kinetics of infrared laser ablation of poly(vinylidene fluoride) (PVDF) in vacuum have been investigated to understand how this can be used as a polymer processing technology. The laser heats the surface and initiates decomposition of macromolecules in the molten polymer layer which rapidly leads to production of an ablative flow of polymer decomposition products. There is only a short induction period of up to 2 s before ablation begins. After the initial period, the rate of mass loss increases linearly with the time of laser exposure. Spherical particles with an average diameter of about 100 nm are formed on the surface of the powder coating obtained by laser ablation. Here, the particle distribution ranges from 50 nm to 300 nm. The main gaseous product of ablation is HF from dehydrofluorination. A second important ablation process is the formation of gaseous carbenes generated during secondary reactions. Proposed ablation reactions due to laser-induced thermal dehydrofluorination occur in two stages. In the first stage, a double bond appears in PVDF due to direct loss of HF. Subsequent detachment of HF leads to the appearance of both triple bonds and cumulene bonds.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Enhanced Oxide Reduction by Hydrogen at Cuprous Oxide–Copper Interfaces near Ascending Step Edges

We report that understanding the dynamic processes involved in the interaction of hydrogen with oxides is of fundamental importance in catalysis. This paper probes the reduction of Cu 2 O-‘29’ surfaces by hydrogen at room temperature combining in situ ambient pressure scanning tunneling microscopy (STM), X-ray photoelectron spectroscopy (XPS), and density functional theory (DFT) calculations. Reduction of the atomic layer thin Cu 2 O film is observed to be preferentially initiated2 at step edges and terrace defects, where perfect Cu 2 O(111) terraces are found to be stable towards hydrogenation under the same conditions. After a long induction period, regions of partially reduced Cu 2 O-‘29’ and metallic Cu co-exist before the surface is fully reduced to Cu(111). The reduction rate strongly depends on the nature of nearby Cu step edges. We propose a mechanism for the reduction of Cu 2 O-‘29’ by hydrogen where free copper atoms from ascending metallic step edges facilitate the formation of active ensembles for H 2 dissociation and transfer H to the edges of Cu2O regions

08 HYDROGEN↗

Site-Selective Probes of Mixed-Node Metal Organic Frameworks for Photocatalytic Hydrogen Generation

Metal organic frameworks (MOFs) have emerged as a novel platform offering new promise for photocatalytic applications. In this work, we report a mixed-node MOF (MOF-74) based on earth-abundant metals (i.e., Co and Fe) as highly active and robust photocatalysts for visible light-driven H 2 generation. Using in situ X-ray absorption spectroscopy which allows us to selectively probe the individual metal node in MOFs, we unambiguously identified the catalytic active sites and captured the key intermediate species under the standard catalytic conditions. We show that both Co and Fe centers in MOF-74 are the reversible active sites, and the accumulation of their reduced states is responsible for the observed induction period for H 2 generation. Moreover, we captured the key intermediate species formed when the reaction reaches equilibrium, which is featured by an elongated Co–O bond at the Co center and a more distorted octahedral geometry at the Fe center with respect to their original geometry. These results not only demonstrate the potential of MOF-74 as a single site catalyst for H 2 generation but also provide new insight into the catalytic mechanism for further development.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Photocatalytic Transfer Hydrogenation in Water: Insight into Mechanism and Catalyst Speciation

Organometallic catalysts with strong metal-to-ligand charge-transfer (MLCT) optical transitions and reactive metal-hydride bonds provide opportunities for new excited-state reactivity. Toward this end, we have investigated photoinduced transfer hydrogenation reactions for a series of Cp*IrH complexes with functionalized bipyridyl ligands. Overcoming the baseline reactivity of this class of catalyst toward hydrogen evolution in aqueous media presents a major challenge for focusing this photochemical reactivity into desired transformations of more complex organic substrates. In this work, a photochemically induced aqueous ketone reduction catalyzed by [Cp*Ir(bpy)H][Cl] is reported and benchmarked to the competing hydrogen evolution reaction (HER). A detailed mechanistic study is conducted of the reduction of cyclohexanone with sodium formate delivering the reducing equivalent. Examining the kinetic profile of the cyclohexanone reduction revealed an unexpected induction period, suggesting a more complex underlying reaction mechanism where product selectivity was not simply a function of direct competition from a common excited-state reactive intermediate. The reaction kinetics were analyzed by variable time normalization analysis (VTNA), and the catalyst resting state was interrogated through operando light-emitting diode (LED) illumination H-1 NMR spectroscopy. Overall, these and other mechanistic studies provide insight into a photochemical reaction pathway to ketone reduction that is distinct from that previously reported for HER.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Microscopic Investigation of H 2 Reduced CuO x /Cu(111) and ZnO/CuO x /Cu(111) Inverse Catalysts: STM, AP-XPS, and DFT Studies

Understanding the reduction mechanism of ZnO/CuO x interfaces by hydrogen is of great importance for advancing the performance of industrial catalysts for CO 2 hydrogenation to methanol. Here, the reduction of pristine and ZnO-modified CuO x /Cu(111) by H 2 was investigated using ambient pressure scanning tunnelling microscopy (AP-STM), ambient pressure X-ray photoelectron spectroscopy (AP-XPS) and density functional theory (DFT). The morphological changes and reaction rates seen for the reduction of CuO x /Cu(111) and ZnO/CuO x /Cu(111) are very different. On CuO x /Cu(111), perfect "44" and "29" structures displayed a very low reactivity towards H 2 at room temperature. A long induction period associated with an autocatalytic process was observed to enable the reduction by the removal of chemisorbed non-lattice oxygen initially and lattice oxygen sequentially at the CuO x -Cu interface, which led to formation of oxygen deficient "5-7" hex and honeycomb structures. In the final stages of the reduction process, regions of residual oxygen species and metallic Cu were seen. The addition of ZnO particles to CuO x /Cu(111) opened new reaction channels. On the ZnO sites, the dissociation of H 2 was fast and H adatoms easily migrated to adjacent regions of copper oxide. This hydrogen spillover substantially enhanced the rate of oxygen removal, resulting in the rapid reduction of the copper oxide located in the periphery of the zinc oxide islands with no signs for the reduction of ZnO. The deposited ZnO completely modified the dynamics for H 2 dissociation and hydrogen migration, providing an excellent source for CO 2 hydrogenation processes on the inverse oxide/metal system.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Chemical Insights into the Formation of Colloidal Iridium Nanoparticles from In Situ X-ray Total Scattering: Influence of Precursors and Cations on the Reaction Pathway

Iridium nanoparticles are important catalysts for several chemical and energy conversion reactions. Studies of iridium nanoparticles have also been a key for the development of kinetic models of nanomaterial formation. However, compared to other metals such as gold or platinum, knowledge on the nature of prenucleation species and structural insights into the resultant nanoparticles are missing, especially for nanoparticles obtained from Ir$_x$Cl$_y$ precursors investigated here. We use in situ X-ray total scattering (TS) experiments with pair distribution function (PDF) analysis to study a simple, surfactant-free synthesis of colloidal iridium nanoparticles. The reaction is performed in methanol at 50 °C with only a base and an iridium salt as precursor. From different precursor salts–IrCl 3 , IrCl 4 , H 2 IrCl 6 , or Na 2 IrCl 6 –colloidal nanoparticles as small as Ir ~55 are obtained as the final product. The nanoparticles do not show the bulk iridium face-centered cubic ($fcc$) structure but show decahedral and icosahedral structures. The formation route is highly dependent on the precursor salt used. Using IrCl 3 or IrCl 4 , metallic iridium nanoparticles form rapidly from Ir$_x$Cl$_y$$^{n–}$ complexes, whereas using H 2 IrCl 6 or Na 2 IrCl 6 , the iridium nanoparticle formation follows a sudden growth after an induction period and the brief appearance of a crystalline phase. With H 2 IrCl 6 , the formation of different Ir$_n$ ($n$ = 55, 55, 85, and 116) nanoparticles depends on the nature of the cation in the base (LiOH, NaOH, KOH, or CsOH, respectively) and larger particles are obtained with larger cations. As the particles grow, the nanoparticle structure changes from partly icosahedral to decahedral. In conclusion, the results show that the synthesis of iridium nanoparticles from Ir$_x$Cl$_y$ is a valuable iridium nanoparticle model system, which can provide new compositional and structural insights into iridium nanoparticle formation and growth.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Why Seeding Works When Nucleation Barriers Vanish

Crystallization is a process governed by the interplay between nucleation and growth. While crystalline seeds are known to reduce nucleation barriers and accelerate crystallization under nucleation-limited conditions, their influence when nucleation is not the limiting step remains poorly understood. This creates a mechanistic puzzle in systems where nucleation barriers are already negligible, yet seeding still accelerates crystallization. Here, the synthesis of zeolites is a quintessential example of growth-limited crystallization in which seeds accelerate the process despite negligible homogeneous nucleation barriers. Using coarse-grained molecular dynamics simulations─validated across two zeolites and the unrelated case of ice crystallization─we establish that growth-limited crystallization produces many small, misoriented crystallites whose slow coarsening into larger domains controls the emergence of X-ray-detectable crystallinity. Local crystalline order, structural coherence, and X-ray detectability are therefore kinetically decoupled milestones: the first can be reached rapidly while the latter two lag significantly. Seeds resolve this lag by imposing a common orientational registry on nascent crystallites, enabling their coherent coalescence into large seed-bound domains. This early coherence-building step produces a crystallite-size asymmetry that accelerates subsequent coarsening, advancing the onset of X-ray-detectable crystallinity without necessarily increasing the nucleated fraction. We conclude that under growth-limited conditions, the apparent induction period observed in powder X-ray diffraction reflects the time required to build long-range coherence, not the time to form crystalline material. Accordingly, seeds function not primarily by reducing nucleation barriers but by enforcing spatial coherence, thereby shortening the time required to develop long-range order detectable by X-ray diffraction.

Growth-Limited Crystallization↗

Radiation-Induced Effects on the Extraction Properties of Hexa-n-octylnitrilo-triacetamide (HONTA) Complexes of Americium and Europium

The candidate An(III)/Ln(III) separation ligand hexa-n-octylnitrilo-triacetamide (HONTA) was irradiated under envisioned SELECT (Solvent Extraction from Liquid waste using Extractants of CHON-type for Transmutation) process conditions (n-dodecane/0.1 M HNO 3 ) using a solvent test loop in conjunction with cobalt-60 gamma irradiation. The extent of HONTA radiolysis and complimentary degradation product formation was quantified by HPLC-ESI-MS/MS. Further, the impact of HONTA radiolysis on process performance was evaluated by measuring the change in 243 Am and 154 Eu distribution ratios as a function of absorbed gamma dose. HONTA was found to decay exponentially with increasing dose, affording a dose coefficient of d = (4.48 ± 0.19) × 10 –3 kGy –1 . Multiple degradation products were detected by HPLC-ESI-MS/MS with dioctylamine being the dominant quantifiable species. Both 243 Am and 154 Eu distribution ratios exhibited an induction period of ~70 kGy for extraction (0.1 M HNO 3 ) and back-extraction (4.0 M HNO 3 ) conditions, after which both values decreased with absorbed dose. The decrease in distribution ratios was attributed to a combination of the destruction of HONTA and ingrowth of dioctylamine, which is capable of interfering in metal ion complexation. Furthermore, the loss of HONTA with absorbed gamma dose was predominantly attributed to its reaction with the n-dodecane radical cation (R˙ + ). These R˙ + reaction kinetics were measured for HONTA and its 241 Am and 154 Eu complexes using picosecond pulsed electron radiolysis techniques. All three second-order rate coefficients (k) were essentially diffusion limited in n-dodecane indicating a significant reaction pathway: k(HONTA + R˙ + ) = (7.6 ± 0.8) × 10 9 M –1 s –1 , k(Am(HONTA) 2 + R˙ + ) = (7.1 ± 0.7) × 10 10 M –1 s –1 , and k(Eu(HONTA) 2 + R˙ + ) = (9.5 ± 0.5) × 10 10 M –1 s –1 . HONTA-metal ion complexation afforded an order-of-magnitude increase in rate coefficient. Additional nanosecond time-resolved measurements showed that both direct and indirect HONTA radiolysis yielded the short-lived (< 100 ns) HONTA radical cation and a second long-lived (µs) species identified as the HONTA triplet excited state. The latter was confirmed by a series of oxygen quenching picosecond pulsed electron measurements, affording a quenching rate coefficient of k( 3 [HONTA]* + O 2 ) = 2.2 × 10 8 M –1 s –1 . Overall, both the HONTA radical cation and triplet excited state are important precursors to the suite of measured HONTA degradation products.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

The transformation of lepidocrocite (γ-FeOOH) with Fe($\tiny{II}$) (aq) in slightly acidic media: intermediate pathways and biomimetic behavior

Lepidocrocite (LP) is commonly found in natural or anthropogenic environments and oxidized alloy steel waste storage containers. Despite its importance, the end products formed and its mineral transformation pathways, including intermediate steps and underlying mechanisms under Fe(II) (aq) catalysis still need to be clarified due to decades of dispersed research. Here, in this work, we investigated LP's catalytic transformation with 10 mM and 0.2 mM Fe(II) (aq) at their natural solution pH's via bulk (X-ray Diffraction/XRD, Raman and Attenuated Total Reflectance Fourier Transform Infrared/ATR-FTIR) and micro/nano-scale (semi in situ Transmission Electron Microscopy/TEM) analysis. In general, we observed that goethite (GT) and LP were the main end products. However, a series of two major distinct intermediate events that were initiated by a dissolution type of reaction along with an “induction period” (lack of dissolution) on LP occurred. Fascinatingly, two of the intermediate steps along its mineral transformation presented novel types of non-classical mechanisms of crystallization via some type of guided oriented particle attachment. Furthermore, one of these intermediate steps is biomimetic in appearance, similar to what is observed during bacterial particle attachment. However, it uses inorganic nano-wire antennas that have a sensory-like function as observed with bacterial fimbriae and/or flagellum through an electron transparent film (similar to a bio-film matrix). Finally, this work leads us to comprehend the evolution of some well documented crystal morphologies for GT commonly observed in natural and anthropogenic settings.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Elucidating Norrish type I reactive pathways by ultrafast X-ray absorption spectroscopy

Norrish type I reactions selectively cleave carbon–carbon bonds directly adjacent to carbonyl groups. Despite their broad use in combination with aromatic carbonyls for additive manufacturing and dental UV curing applications, the nature of the photochemically active state and its population mechanism remain insufficiently understood. Detailed mechanistic insight requires mapping of the photoexcited population flow involving internal conversion and intersystem crossing. We present a time-domain study of gas phase acetophenone as a prototypical aromatic carbonyl combining soft X-ray time-resolved near-edge X-ray absorption fine structure (TR-NEXAFS) spectroscopy at the oxygen K-edge with ab initio multiple spawning (AIMS) simulations. Exploiting the specific sensitivity of TR-NEXAFS spectroscopy to states with nπ* character, we observe population transfer from the initially excited 1 ππ* state to the 1 nπ* state with a time constant of (0.13 ± 0.02) ps after an initial induction period of (0.12 ± 0.02) ps without population transfer, in quantitative agreement with the AIMS simulations. The population in the 1 nπ* state subsequently decays via intersystem crossing, likely mediated by a 3 ππ* state, within (3.17 ± 0.66) ps to a long-lived 3 nπ* state, which is presumed to be active towards Norrish type I chemistry.

Graßl, Martin [SLAC National Accelerator Laborator↗

Evaluation of Oxygen Consumption as a Sensitive Measure of Electrical Cable Polymer Insulation Degradation

Polymers used as dielectric materials in nuclear grade electrical cables, such as cross-linked polyethylene (XLPE) and ethylene-propylene rubber (EPR), are long-lived and durable. Estimating the decades-long useful service life of polymers experimentally in a feasible timeframe of weeks or months requires greatly accelerated aging in the laboratory at elevated temperatures and/or gamma dose rates. Many limitations of accelerated aging, such as due to artefacts of the process including diffusion limited oxidation and dose rate effects, have been identified that can make accurate lifetime prediction at service conditions challenging. This can be due to differences in degradation mechanisms between rapid and extended aging or to the presence of induction periods during which early aging is undetectable using common metrics of polymer insulation characterization such as tensile elongation at break. Conceptually, an ideal measure of aging would track clearly and predictably with material changes throughout the lifecycle of the evolving material state from early aging through end of useful life. With such a metric accelerated aging could be performed at much milder conditions, conditions closer to those found in long term service, and end of life calculated from the early data. Oxygen consumption has previously been proposed as a sensitive measure of insulation polymer oxidation that might meet such desirable characteristics. In this work, nuclear cable insulation samples of the two most common types, XLPE and EPR, from two of the most sourced manufacturers, RSCC and The Okonite Company, were subjected to gamma irradiation at dose rates of 100, 200, and 300 Gy/hr for up to 42 days at 26°C. The aged samples were used to evaluate oxygen consumption as a sensitive measure of aging for these materials and these conditions. For comparison and validation, the aged samples were also characterized using the more conventional tests of mass change, density, carbonyl index, and yellowness index. For both materials, oxygen consumption during irradiation was found to track consistently with applied dose—oxygen consumption rate was linear with time. Mass change and carbonyl index were also clearly proportional to exposure dose. Yellowness index, a surface sensitive measure, was observed to increase linearly with dose at low exposure levels before plateauing. Sample density was not observed to change under the conditions explored. Oxygen consumption was found to be a particularly sensitive metric to measure the degradation of polymeric cable insulation exposed to radiation. Oxygen consumption was compared to Arrhenius and time-temperature-superposition methods for activation energy calculation using literature date and found to be a useful and viable alternative for lifetime prediction. The techniques developed in this work are anticipated to be valuable in the estimation of insulation aging rates under mild conditions approaching service conditions. Direction comparison of useful life prediction by oxygen consumption with prediction via Arrhenius methodology for thermal aging and existing methods for radiation and combined thermal/radiation aging would serve to further validate this promising approach and refine our understanding of material-specific aging rates.

36 MATERIALS SCIENCE↗

Storage Stability of 50% Biodiesel Blends and Neat Biodiesel Produced in North America

This study investigated aging of two petroleum diesel, seven B100, three B20 blends, and 14 B50 blends representing the full range of fuels in the North American market for 12-weeks at 43 degrees Celsius under the ASTM D4625 accelerated aging protocol. Twelve-weeks under D4625 aging represents approximately 12 months of storage time in clean, dry conditions. Samples were taken every two weeks for measurement of Rancimat induction period (IP), peroxide content, and TAN; and total insoluble were measured at the end of the 12-week test.

09 BIOMASS FUELS↗

The Preparation and Characterization of Co–Ni Nanoparticles and the Testing of a Heterogenized Co–Ni/Alumina Catalyst for CO Hydrogenation

Samples of well-controlled nanoparticles consisting of alloys of cobalt and nickel of different atomic ratios were synthesized using wet chemical methods with oleylamine as the solvent and the reducing agent. These materials were characterized by a variety of techniques, including high-angle annular dark-field scanning transmission electron microscopy (HAADF-STEM), X-ray energy dispersive spectroscopy (EDS), and X-ray diffraction (XRD). Small amounts of heterogenized catalysts were prepared using alumina as the support. However, the potential for use of Co–Ni catalysts in CO hydrogenation was explored using a larger amount of Co–Ni/alumina catalyst prepared from standard aqueous impregnation methods and tested in a continuously stirred tank reactor (CSTR) for Fischer–Tropsch synthesis (FTS). Results are compared to a reference catalyst containing only cobalt. The heterogenized catalysts were characterized using synchrotron methods, including temperature programmed reduction with extended X-ray absorption fine structure spectroscopy and X-ray absorption near edge spectroscopy (TPR-EXAFS/XANES). The characterization results support intimate contact between Co and Ni, strongly suggesting alloy formation. In FTS testing, drawbacks of Ni addition included decreased CO conversion on a per gram catalyst basis, although Ni did not significantly impact the turnover number of cobalt, and produced slightly higher light gas selectivity. Benefits of Ni addition included an inverted induction period relative to undoped Co/Al2O3, where CO conversion increased with time on-stream in the initial period, and the stabilization of cobalt nanoparticles at a lower weight % of Co.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Transient Catalytic Reaction Analysis Through Signal Defragmentation

The Temporal Analysis of Products (TAP) pulse response technique provides valuable insights into catalytic function and reaction kinetics. However, complex fragmentation patterns in the TAP mass spectrometry signals can complicate precise quantification, particularly when analyzing transient gas flux data typical of TAP experiments. This work demonstrates a standard defragmentation method that deconvolves transient TAP signals while maintaining the temporal resolution of the experiment. First, the integrals of calibration gas fluxes are used to determine the fingerprint fragmentation pattern and construct a fragmentation matrix. This matrix is then used to defragment experimental flux data at each recorded time point via a non-negative least squares regression. The effectiveness of this method is demonstrated using virtual data and control experiments with a TAP reactor system. The defragmentation is then applied to the more complex propane dehydrogenation reaction on a chromia/alumina catalyst, which can contain up to ten significant gas species in the reactor outlet. Initial propane pulsing reveals an induction period during which propane is fully oxidized to CO2, followed by partial reduction to CO. Afterwards, there is a transition in chemistries towards coking and propylene production. Our example illustrates a practical method for the accurate determination of the time-dependent reactant/product concentrations and rates for a thorough analysis of the propane dehydrogenation kinetics. This approach can be broadly applied to any transient mass spectrometry experiment for a better understanding of catalyst-reaction dynamics.

36 - MATERIALS SCIENCE↗

Non-Stationary Policy Learning for Multi-Timescale Multi-Agent Reinforcement Learning

In multi-timescale multi-agent reinforcement learning (MARL), agents interact across different timescales. In general, policies for time-dependent behaviors, such as those induced by multiple timescales, are non-stationary. Learning non-stationary policies is challenging and typically requires sophisticated or inefficient algorithms. Motivated by the prevalence of this control problem in real-world complex systems, we introduce a simple framework for learning non-stationary policies for multi-timescale MARL. Our approach uses available information about agent timescales to define and learn periodic multi-agent policies. In detail, we theoretically demonstrate that the effects of non-stationarity introduced by multiple timescales can be learned by a periodic multi-agent policy. To learn such policies, we propose a policy gradient algorithm that parameterizes the actor and critic with phase-functioned neural networks, which provide an inductive bias for periodicity. The framework's ability to effectively learn multi-timescale policies is validated on a gridworld and building energy management environment.

control↗