SEARCH · Search NASA

Results for “TIME VARYING DELAYS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Streaming Analytics for Anomaly Detection in Large-Scale Data

Anomalous behavior poses serious risks to assured performance and reliability of complex, high-consequence systems. For spaceborne assets and their state-of-health (SOH) telemetry, the challenges of high-dimensional data of varying data types are compounded by computational limitations from size, weight, and power (SWaP) constraints as well as data availability. Automated anomaly detection methods tend to perform poorly under these constraints, while current operational approaches can introduce delays in response time due to the manual, retrospective processes for understanding system failures. As a result, presently deployed space systems, and those deployed in the near future, face situations where mission operations might be delayed or only be able to operate under degraded capabilities. Here, we examine a near-term lightweight solution that provides real-time detection capabilities for rare events and assess state-of-the-art anomaly detection techniques against real SOH telemetry from space platforms. This report describes our methodology and research, which could support more automated capabilities for comprehensive space operations as well as for other resource-constrained edge applications.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Design-Technology Co-Optimization for NVM-based Neuromorphic Processing Elements

An emerging use-case of machine learning (ML) is to train a model on a high-performance system and deploy the trained model on energy-constrained embedded systems. Neuromorphic hardware platforms, which operate on principles of the biological brain, can significantly lower the energy overhead of a machine learning inference task, making these platforms an attractive solution for embedded ML systems. In this paper, we present a design-technology tradeoff analysis to implement such inference tasks on the processing elements (PEs) of a Non-Volatile Memory (NVM)-based neuromorphic hardware. Through detailed circuit-level simulations at scaled process technology nodes, we show the negative impact of technology scaling on the information-processing latency, which impacts the quality-of-service (QoS) of an embedded ML system. At a finer granularity, the latency inside a PE depends on 1) the delay introduced by parasitic components on its current paths, and 2) the varying delay to sense different resistance states of its NVM cells. Based on these two observations, we make the following three contributions. First, on the technology front, we propose an optimization scheme where the NVM resistance state that takes the longest time to sense is set on current paths having the least delay, and vice versa, reducing the average PE latency, which improves the QoS. Second, on the architecture front, we introduce isolation transistors within each PE to partition it into regions that can be individually power-gated, reducing both latency and energy. Finally, on the system-software front, we propose a mechanism to leverage the proposed technological and architectural enhancements when implementing a machine-learning inference task on neuromorphic PEs of the hardware. Evaluations with a recent neuromorphic hardware architecture show that our proposed design-technology co-optimization approach improves both performance and energy efficiency of machine-learning inference tasks without incurring high cost-per-bit.

42 ENGINEERING↗

DOE OSTI2020

A geospatial model for the analysis of time-dependent land subsidence induced by reservoir depletion

Land subsidence due to fluid depletion is an outcome of physical processes operating across a wide range of time and length scales. Although geomechanical models are crucial to simulate reservoir compaction and predict its long-term fate, their use across large regions often bears prohibitive computational costs. To overcome this obstacle, this paper proposes a simplified modelling framework consisting of (i) a near-field numerical solver simulating the coupling between fluid flow and rock deformation with reference to a simplified one-dimensional geometry and (ii) a far-field geospatial algorithm mapping ground settlements across a region through the superposition of poroelastic computations at multiple wells. The model computes the delay between depletion history and reservoir compaction in proximity of a producing well by assuming basal depletion of a fluid-saturated deformable disk, while the Geertsma solution of nucleus of strain is used to extrapolate the impact of such time-varying reservoir compaction around the well. This approach has been used to back-analyze the spatio-temporal progression of subsidence at the Groningen gas field. The results are compared against measurements collected over 50 year of production at 25 benchmark locations scattered over an area of 900 km 2 . It is shown that coupled simulations based on average values of rock compressibility and permeability lead to nonlinear trends of subsidence evolution in good agreement with field measurements, while uncoupled analyses overpredict settlements by more than 70%. Lastly, synthetic forecasts based on different rates of depletion were provided. The results suggest that slower depletion rates lead to lower subsidence at a given time, and that residual subsidence may continue to develop for several decades after interruption of production activities.

58 GEOSCIENCES↗

DOE OSTI2020

Acceleration of Glass Alteration Rates Induced by Zeolite Seeds at Controlled pH

As the glass-water reaction proceeds, certain glass compositions undergo a delayed acceleration in the alteration rate – a phenomenon that is commonly referred to as Stage III behavior – which generally coincides with zeolite formation. However, studying Stage III in the laboratory is difficult because the time scale required to observe Stage III behavior can vary from months to several years, and the solution conditions that initiate this acceleration remain poorly understood. Consequently, the development of a corrosion test method capable of quantifying Stage III rates in a timely manner would be of great benefit to both the scientific community and policy makers concerned with nuclear waste disposal. In this work, we induced and measured the acceleration in the corrosion rate of the AFCI glass by seeding static corrosion tests with zeolite Na-P2 at fixed pH90°C values ranging from 9.5 to 11.5. The induced Stage III rates increased linearly with increasing pH and were measured to be between the forward and residual rates of AFCI. Tests designed to compare induced Stage III rates obtained for different seeding times showed that the induced Stage III rates were faster in the absence of an alteration layer. In summary, the use of the seeded test method allows for a conservative quantification of Stage III rates in a relatively short, one-month time period.

Neeway, James J.↗

DOE OSTI2025

Prediction of Complex Observed Shear Wave Splitting Patterns at Ryukyu Subduction Zone Using a Strong Intra-Slab Anisotropy Model

Complex shear wave splitting (SWS) patterns in subduction zones are often interpreted geodynamically as resulting from complex mantle flow; however, this may not always be necessary. We analyzed 7,093 high-quality SWS measurements from teleseismic S waves recorded by Hi-net stations across the Ryukyu arc in Japan. Our findings show a systematic rotation of the fast S polarization from trench-parallel to trench-perpendicular depending on the earthquake backazimuth. For the same earthquake, the measured splitting patterns also vary spatially across the southwest Japan. Using full-wave seismic modeling, we showed that a dipping slab with ~30% shear anisotropy of the tilted transverse isotropy (TTI) type, with a symmetry axis perpendicular to the slab interface, can predict the observed delay times and polarization rotation. Our results highlight the importance of considering dipping anisotropic slabs in interpreting SWS at subduction zones.

58 GEOSCIENCES↗

DOE OSTI2020

Mechano-Electrochemical Interaction in Solid-State Lithium Batteries

A solid-state lithium (Li) battery primarily consists of Li metal anode, solid electrolyte separator, and cathode. The asymmetric volume changes, originating from ion transport and interfacial Li growth during plating, lead to significant stresses in the layered architecture. In this study, we develop a coupled mechanics-electrochemistry formalism for polymer electrolyte based solid-state batteries, in particular, focusing on the stress effect on electrochemical performance. By means of a coupling coefficient, it is found that stress-assisted ion transport in the electrolyte results in a delayed Sand’s time and increased critical current density of unstable electrodeposition, and consequently, alleviates the propensity of dendrite formation. Stress at the Li metal-electrolyte interface affects the electrochemical reaction kinetics, and the influences from the deviatoric stress and hydrostatic pressure vary with Li plating time. In addition, a low restraint stiffness to the layered structure could elastically buffer the volumetric changes and thus reduce the stress during Li plating. This fundamental study provides guidance for the design of solid-state batteries, aimed at stable electrodeposition and mechanical integrity.

25 ENERGY STORAGE↗

DOE OSTI2024

Wave packet dynamics and control in excited states of molecular nitrogen

Wave packet interferometry with vacuum ultraviolet light has been used to probe a complex region of the electronic spectrum of molecular nitrogen, N 2 . Wave packets of Rydberg and valence states were excited by using double pulses of vacuum ultraviolet (VUV), free-electron-laser (FEL) light. These wave packets were composed of contributions from multiple electronic states with a moderate principal quantum number (n ~ 4-9) and a range of vibrational and rotational quantum numbers. The phase relationship of the two FEL pulses varied in time, but as demonstrated previously, a shot-by-shot analysis allows the spectra to be sorted according to the phase between the two pulses. The wave packets were probed by angle-resolved photoionization using an infrared pulse with a variable delay after the pair of excitation pulses. The photoelectron branching fractions and angular distributions display oscillations that depend on both the time delays and the relative phases of the VUV pulses. The combination of frequency, time delay, and phase selection provides significant control over the ionization process and ultimately improves the ability to analyze and assign complex molecular spectra.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2021

Use of continuous sample translation to reduce radiation damage for XPCS studies of protein diffusion

An experimental setup to measure X-ray photon correlation spectroscopy during continuous sample translation is presented and its effectiveness as a means to avoid sample damage in dynamics studies of protein diffusion is evaluated. X-ray damage from focused coherent synchrotron radiation remains below tolerable levels as long as the sample is translated through the beam sufficiently quickly. Here it is shown that it is possible to separate sample dynamics from the effects associated with the transit of the sample through the beam. By varying the sample translation rate, the damage threshold level, D thresh = 1.8 kGy, for when beam damage begins to modify the dynamics under the conditions used, is also determined. Signal-to-noise ratios, R sn ≥ 20, are obtained down to the shortest delay times of 20 µs. The applicability of this method of data collection to the next generation of multi-bend achromat synchrotron sources is discussed and it is shown that sub-microsecond dynamics should be obtainable on protein samples.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

UAE6 - Wind Tunnel Tests Data - UAE6 - Sequence Q - Raw Data

Sequence Q: Dynamic Inflow (P) This sequence was designed to characterize the dynamic inflow variation using the five-hole probes that extend upwind of the leading edge of the blade. This test sequence used an upwind, rigid turbine with a 0° cone angle. The wind speeds ranged from 5 m/s to 15 m/s, and data were collected at a 0° yaw angle. The rotor rotated at 72 RPM. Blade and probe pressure measurements were collected. The teeter dampers were replaced with rigid links, and these two channels were flagged as not applicable by setting the measured values in the data file to -99999.99 Nm. The teeter link load cell was pretensioned to 40,000 N. The blade pitch angle was changed from the initial tip pitch angle to the final tip pitch angle at the maximum rate of 67°/s. This angle was maintained for the specified delay time duration. Then the blade was pitched back to the initial pitch angle and held for the specified hold time. This was repeated 20 times. The campaign at 15 m/s was aborted early due to excessive low-speed shaft torque loads. The file length varied depending on the time required to obtain 20 pitch cycles for the specified range of angles. The name convention was the standard format, except for the short campaigns collected to ascertain the functionality of the instrumentation. These points were collected at a 3° pitch angle and use the characters REF in the four digits that normally represent yaw angle.

17 WIND ENERGY↗

DOE OSTI2025

Magneto-optical spectroscopy based on pump-probe strobe light

Here, we demonstrate a pump-probe strobe light spectroscopy for sensitive detection of magneto-optical dynamics in the context of hybrid magnonics. The technique uses a combinatorial microwave-optical pump-probe scheme, leveraging both the high-energy resolution of microwaves and high-efficiency detection using optical photons. In contrast with conventional stroboscopy using continuous-wave light, we apply microwave and optical pulses with varying pulse widths, and demonstrate magneto-optical detection of magnetization dynamics in Y 3 Fe 5 O 12 films. The detected magneto-optical signals depend strongly on the characteristics of both the microwave and the optical pulses, as well as their relative time delays. We show that good magneto-optical sensitivity and coherent stroboscopic character are maintained, even at a microwave pump pulse of 1.5 ns and an optical probe pulse of 80 ps, under a 7 MHz clock rate, corresponding to a pump-probe footprint of approximately 1% in one detection cycle. Our results show that time-dependent strobe light measurement of magnetization dynamics can be achieved in the gigahertz frequency range under a pump-probe detection scheme.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Dual-Fuel Ammonia Equivalence Ratio Sweep Data

Ammonia (NH3) has garnered significant interest as an alternative fuel for meeting international emissions reduction mandates in sectors with high weight and distance requirements, such as shipping. Technical barriers and unanswered questions remain on the combustion strategies that can maximize ammonia utilization and minimize emissions. Prior research studies at the US Department of Energy’s Oak Ridge National Laboratory have shown strong performance with NH3 under dual-fuel mode using conventional diesel combustion (CDC) manifold air pressure settings. Diesel airflow was initially used to simplify retrofitting (no turbocharger modification), which resulted in air-fuel equivalence ratios (λ) greater than 1.5. To characterize potential improvements in dual-fuel NH3 combustion performance at richer in-cylinder conditions, a global λ sweep compared the use of early (E-pilot) and late (L-pilot) single diesel injections. The experiments were conducted at 1200 RPM and 12.8 ± 0.2 bar (75 % load), and λ was varied by decreasing the commanded air flow to the engine at greater than 90 % ammonia energy substitution level. A diesel injection timing sweep was conducted for both the injection strategies at fixed λ, and the timing with the lowest engine-out N2O emissions was identified. The results indicated an optimal balance between CO2,eq and thermal efficiency benefits both E-pilot and L-pilot injection strategy cases compared with CDC at a λ of 1.4. The indicated nitrogen-based emissions exhibited a strong correlation to the ratio of CA5–50 and ignition delay for L-pilot, but no apparent trend emerged for the E-pilot injection strategy at the tested boundary conditions. This dataset includes the raw experimental data and documentation of the experiment conditions and methods.

30 DIRECT ENERGY CONVERSION↗

DOE OSTI2020

Effects of Turbulence and Temperature Fluctuations on Knock Development in an Ethanol/Air Mixture

The effects of turbulence on knock development and intensity for a thermally inhomogeneous stoichiometric ethanol/air mixture at a representative end-gas autoignition condition in internal combustion engines are investigated using direct numerical simulations with a skeletal reaction mechanism. Two- and three-dimensional simulations are performed by varying the most energetic length scale of temperature, $l_T$ , and its relative ratio with the most energetic length scale of turbulence, $l_T/l_e$ , together with two different levels of the turbulent velocity fluctuation, $u'$ . It is found that $l_T$ / $l_e$ and the ratio of ignition delay time to eddy-turnover time, $\tau _{ig}/\tau _t$ , are the key parameters that control the detonation development. An increase in either $l_T$ or $l_e$ enhances the detonation propensity by allowing a longer run-up distance for the detonation development. The characteristic length scale of the temperature field, $l_T$ , is significantly modified by high turbulence intensity achieved by a large $l_e$ and $u'$ . The intense turbulence mixing effectively distributes the initial temperature field to broader scales to support the developing detonation waves, thereby increasing the likelihood of the detonation formation. On the contrary, high turbulence intensity with a short mixing time scale, achieved by a small $l_e$ and a large $u'$ , reduces the super-knock intensity attributed to the finer broken-up structures of detonation waves. Either $\tau _{ig}/\tau _t$ less than unity or $l_e = l_T$ even with a large $u'$ is found to have no significant effect on super-knock mitigation. Finally, high turbulent intensity may induce high-pressure spikes comparable to the von Neumann spike. Overall, increased temperature and pressure by combustion heating, noticeably after the peak of heat release rate, significantly enhance the collision and interaction of multiple emerging autoignition fronts near the ending combustion process, resulting in localized high-pressure spikes.

42 ENGINEERING↗

DOE OSTI2022

FORTE Measurements of Global Optical Lightning Waveforms and Implications for Optical Lightning Detection

Lightning processes generate a diverse collection of optical pulses from current traversing the lightning channels. These signals are then broadened spatially and temporally via scattering in the clouds. The resulting waveforms measured from space with instruments like the photodiode detector (PDD) on the Fast On-orbit Recording of Transient Events (FORTE) satellite have a variety of shapes. In this study, we use coincident optical and Radio-Frequency measurements to document the properties of optical PDD waveforms associated with different types of lightning, estimate delays from scattering in the clouds, and comment on how pulse shape impacts optical lightning detection. We find that the attributes of optical pulses recorded by the PDD are consistent with prior studies, but vary globally and with event amplitude. The most powerful lightning tends to be single-peaked with faster rise times (median: ~100 µs) and shorter effective widths (median: ~400 µs) than normal lightning. Particularly dim events, meanwhile, include cases of broad optical waveforms with sustained optical emission throughout the PDD record, which the pixelated FORTE Lightning Location System (LLS) instrument has difficulty detecting. We propose that this is due to the optical signal being divided between individual LLS pixels that are each, individually, not bright enough to trigger. We use PDD waveforms and Monte Carlo radiative transfer modeling to demonstrate that increasing the temporal and spatial resolution of a pixelated lightning imager will make it more difficult to detect these broad/dim pulses as their energy becomes divided between additional pixels/integration frames.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2020

Measurement of neutron-proton capture in the $\mathrmSNO+ water phase

The SNO+ experiment collected data as a low-threshold water Cherenkov detector from September 2017 to July 2019. Measurements of the 2.2-MeV γ 's produced by neutron capture on hydrogen were made using an Am-Be calibration source, for which a large fraction of emitted neutrons are produced simultaneously with a 4.4-MeV γ. Analysis of the delayed coincidence between the 4.4-MeV γ and the 2.2-MeV capture γ revealed a neutron detection efficiency that is centered around 50% and varies at the level of 1% across the inner region of the detector, which to our knowledge is the highest efficiency achieved among pure water Cherenkov detectors. Finally, in addition, the neutron capture time constant was measured and converted to a thermal neutron-proton capture cross section of $336.3^{+1.2}_{-1.5}$ mb .

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Getting lost tracking the carbon footprint of hydropower

In the transition to low-carbon electricity, well-quantified estimates of carbon dynamics are needed to ensure that emissions reduction targets are achieved. Here we review the state of the science on carbon accounting for hydropower reservoirs and identify limitations and future solutions. Nearly all research on reservoir greenhouse-gas (GHG) emissions has focused on individual reservoirs in isolation without considering their position in a freshwater network draining organic matter from upstream watersheds or the coordinated operation of reservoir cascades. Second, carbon inventories have extrapolated from a small, non-probabilistic sample of highly variable measurements of GHG emissions to unsampled reservoirs. A stronger statistical foundation is needed to estimate a global inventory and its uncertainty. Third, attribution to hydropower is based on ranks assigned to reservoir purpose. Instead, the physical influence of hydropower on carbon dynamics could be directly measured. Fourth, current carbon-accounting practices neglect time. A time-varying approach would quantify variation in emissions for electricity portfolios from changes in the fuel mix at different times and account for ancillary services, i.e., the ability to support the grid when variable renewables are not available without using natural gas. Reservoirs also sequester a significant portion of inflowing carbon in sediments and slow the carbon cycle by delaying the return of carbon to the atmosphere for decades to centuries. Together, these refinements would help to illuminate pathways toward meeting energy demand with the longest-possible delay in returning carbon to the atmosphere and without adding ancient sources to the pool of carbon cycling through aquatic ecosystems.

13 HYDRO ENERGY↗

DOE OSTI2023

Validation of Octane Hyperboosting Phenomenon in Prenol and Structurally Related Olefinic Alcohols

Hyperboosting is a recently discovered phenomenon in which the research octane number (RON) of a blend is higher than both the neat blendstock and the neat fuel it was blended into. RON is a measure of a fuel's resistance to knock, and knock is a cause of engine inefficiency. Blends which exhibit hyperboosting are blends in which an overall improvement in engine efficiency may be expected. The first discovery of hyperboosting came from blending experiments in which prenol was blended into several different base fuels. Here, ignition delay time (IDT) measurements taken using a commercially available constant volume combustion chamber called the Advanced Fuel Ignition Delay Analyzer (AFIDA) are presented. The data show that some prenol blends have longer IDTs (lower reactivity) than either neat prenol or the base fuel, providing further evidence of hyperboosting. Additionally, more blending data is presented in which the base fuel is varied, which allows for a better understanding of hyperboosting sensitivity to chemical classes. The data indicate that aromatics may inhibit, and branched alkanes may enhance the magnitude of hyperboosting observed. Enthalpy of vaporization estimates are also given for several molecules which are blended into a 4-component surrogate. These estimates are derived from Equation of State simulations and reveal that there is no observable correlation between hyperboosting and enthalpy of vaporization. Blending data for molecules which share structural similarities with prenol are also presented. Structure property relationships are suggested, in which the double bond motif of prenol seems to play an important role in hyperboosting. This may help to understand hyperboosting and its underlying mechanism. Lastly, blending curves of surrogate blends with prenol experienced hyperboosting under lean (Homogeneous Charged Compression Ignition-HCCI) operating conditions, which validates that hyperboosting is not an artifact of the octane test methods, but inherent to the properties of prenol.

ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2023

Delayed Critical and Subcritical Experiments with Polyethylene Moderated Unreflected Thin 15 in. Diameter HEU Metal Plates

The thin ~15 in. diameter highly enriched uranium (HEU) metal plates were assembled to delayed criticality at the Oak Ridge Critical Experiments Facility (ORCEF) in 1969 with various thicknesses of polyethylene (varying from 1/16 to 2$\frac{3}{8}$ inches) between uranium metal plates. The average 235 U enrichment was 93.27 wt. %. These unreflected critical configurations contained 4$\frac{2}{3}$ to 20$\frac{5}{6}$ thin 15 in. diameter HEU metal plates (on loan from Los Alamos National Laboratory [LANL] and shipped to Oak Ridge National Laboratory [ORNL] on June 3, 1969). Depending on the thickness of polyethylene, the enriched uranium masses varying from 28,053 to 135,148 grams. Fractional plate sections consisted of the appropriate number of 60° pie sections. In addition to the measurement at delayed criticality, subcritical measurements were also performed by the inverse kinetic rod drop method. Prompt neutron decay constant measurements were also performed by the Rossi alpha and randomly pulsed neutron method using a time-tagged spontaneous fission californium neutron source; these are briefly reported here. At the time of these measurements in 1969, the thin HEU metal plates were in near-pristine condition with extremely little oxidation, allowing better descriptions of the uranium plates than the use of these plates in a heavily oxidized and deteriorated condition in recent reflected benchmark experiments at the LANL facility at the Nevada Test Site with these same thin highly enriched uranium metal plates. This report documents the experimental information for the measurements performed so that later researchers can perform the required uncertainty and calculational analyses and documentation to use these data for an International Nuclear Criticality Safety Benchmark Evaluation Program (ICSBEP) or a Nuclear Energy Agency (NEA) benchmark. Data from the experiments described should be acceptable for use as criticality safety benchmark experiments for the ICSBEP and the NEA nuclear criticality safety benchmark program once the uncertainty analysis on the measured neutron multiplication factors is completed. Additional data—such as the dimensional inspection reports, uranium isotopic information, and other relevant particulars—should be retrieved from the Y-12 Plant or LANL and incorporated in the final ICSBEP benchmark. Based on previous ICSBEP benchmarks with this enriched uranium metal at ORCEF, the uncertainties in $k_{eff}$ could be as low as ± 0.0002 for some configurations. Other experiments with smaller-diameter than 15 in. diameter HEU metal plates have been benchmarked in HEU-METFAST-001. The prompt neutron time decay measurements could be the basis for an International Reactor Physics Benchmark Program. Preparation of the present report is part of an effort at ORNL to document more than 15 undocumented critical and subcritical experiments enumerated in ORNL/TM-2019/18 and performed by ORNL at ORCEF and other US Department of Energy critical experiments facilities using more than 500 operational days of critical facility time. This work for this report publication was supported by the Nuclear Criticality, Radiation Transport and Safety NCSP Program at ORNL.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Deep reinforcement learning based optimization for a tightly coupled nuclear renewable integrated energy system

New ways to integrate energy systems to maximize efficiency are being sought to meet carbon emissions goals. Nuclear-renewable integrated energy system (NR-IES) concepts are a leading solution that couples a nuclear power plant with renewable energy, hydrogen generation plants, and energy storage systems, such that thermal and electrical power are dispatchable to fulfill grid-flexibility requirements while also producing hydrogen and maximizing revenue. Here, this paper introduces a deep reinforcement learning (DRL)-based framework to address the complex decision-making tasks for NR-IES. The objective is to maximize revenue by generating and selling hydrogen and electricity simultaneously according to their time-varying prices while keeping the energy flow in the subsystems in balance. A Python-based simulator for a NR-IES concept has been developed to integrate with OpenAI Gym and Ray/RLlib to enable an efficient and flexible computational framework for DRL research and development. Three state-of-the-art DRL algorithms have been investigated, including two-delayed deep deterministic policy gradient (TD3), soft-actor critic (SAC), proximal policy optimization (PPO), to illustrate DRL’s superiority for controlling NR-IES by comparing it with a conventional control approach, particle swarm optimization (PSO). In this effort, PPO has shown more-stable performance and also better generalization capability than SAC and TD3. Comparisons with PSO have demonstrated that, on average, PPO can achieve 13.9% more mean episode returns from the training process and 29.4% more mean episode returns from the testing process when different hydrogen-production targets are applied.

08 HYDROGEN↗