Search NASA⌕ Search

SEARCH · Search NASA

Results for “agent based modeling”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 721 records · Page 40
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Large-Scale Traveling Weather Systems in Mars Southern Extratropics

Between late fall and early spring, Mars' middle- and high-latitude atmosphere supports strong mean equator-to-pole temperature contrasts and an accompanying mean westerly polar vortex. Observations from both the MGS Thermal Emission Spectrometer (TES) and the MRO Mars Climate Sounder (MCS) indicate that a mean baroclinicity-barotropicity supports intense, large-scale eastward traveling weather systems (i.e., transient synoptic-period waves). Such extratropical weather disturbances are critical components of the global circulation as they serve as agents in the transport of heat and momentum, and generalized scalar/tracer quantities (e.g., atmospheric dust, water-vapor and ice clouds). The character of such traveling extratropical synoptic disturbances in Mars' southern hemisphere during late winter through early spring is investigated using a moderately high-resolution Mars global climate model (Mars GCM). This Mars GCM imposes interactively-lifted and radiatively-active dust based on a threshold value of the surface stress. The model exhibits a reasonable "dust cycle" (i.e., globally averaged, a dustier atmosphere during southern spring and summer occurs). Compared to the northern-hemisphere counterparts, the southern synoptic-period weather disturbances and accompanying frontal waves have smaller meridional and zonal scales, and are far less intense. Influences of the zonally asymmetric (i.e., east-west varying) topography on southern large-scale weather are investigated, in addition to large-scale up-slope/down-slope flows and the diurnal cycle. A southern storm zone in late winter and early spring presents in the western hemisphere via orographic influences from the Tharsis highlands, and the Argyre and Hellas impact basins. Geographically localized transient-wave activity diagnostics are constructed that illuminate dynamical differences amongst the simulations and these are presented.

Hollingsworth, Jeffery L.↗
NASA NTRS1996

Human-Centered Technologies and Procedures for Future Air Traffic Management: A Preliminary Overview of 1996 Studies and Results

In this project, we have been exploring the use of a general methodology to predict the impact of future Air Traffic Management (ATM) concepts and technologies. In applying this methodology, our emphasis has been on the importance of modeling coordination and cooperation among the multiple agents within this system, and on understanding how the interactions among these agents will be influenced as new roles, responsibilities, procedures and technologies are introduced. To accomplish this, we have been collecting data on performance under the current air traffic management system, trying to identify critical problem areas and looking for exemplars suggestive of general approaches for solving such problems. Based on the results of these field studies, we have developed a set of scenarios centered around potential future system designs, and have conducted studies using these scenarios involving a total 40 controllers, dispatchers, pilots and traffic managers. The purpose of this report is to provide NASA with an early summary of the major recommendations that have resulted from our research under the AATT Program thus far. Recommendations 1-3 deal with general approaches that our findings suggest should be incorporated in future AATT Program activities, while Recommendations 4-11 identify some specific topics and technologies that merit research and development activities. Detailed technical reports containing supporting data, as well as the results of our still ongoing analyses, will be provided at a later date. The remainder of this report is organized as follows. Section 1 briefly describes the general design philosophy supported by our empirical studies. Section 2 presents the research methods we have used for identifying requirements for future system designs and for evaluating alternative design solutions. Section 3 discusses preliminary results from an initial set of investigations that we have conducted using these research methods. Section 4 then provides an overall summary. An outline of the rest of this preliminary project summary is provided on the following page.

Smith, Philip↗
DOE OSTI2023

Understanding of three different polyvinylpyrrolidone (PVP) based battery binders blends on graphene surfaces from first principles via DFT simulations

Binders play a crucial role in binding electrodes, current collectors, and conductive agents during battery manufacturing. The selection of the binders and their blends considerably affect various properties such as mechanical durability, adhesion performance, ionic/electronic conductivities, and solid electrolyte interface stability for lithium-ion batteries. In this work, polyvinylidene di-fluoride (PVDF), polyacrylic acid (PAA), polyvinylpyrrolidone (PVP), and lithiated polyacrylic acid (Li-PAA) were selected as the model binders, and their blends were studied on graphene surfaces. Further, among them, PVDF is hardly soluble in water unlike the other binders, so in practice, the blend of PVP:PVDF is not thought to be a proper combination. However, the PVP:PVDF binder has also been included because the purpose of this work is to calculate several electronic properties such as binding energies, intermolecular interaction energies, bond critical points, electron density, and the Laplacian of electron density at critical bonding regions of all these blends at the molecular level via density functional theory (DFT) simulations in order to evaluate and compare how is the interaction strength and bonding type of three different lithium-ion batteries (LIBs) binder blends on graphene surfaces. Most stable binder pairs and their binding mechanisms on graphene surfaces were studied. The delocalization of lithium in Li-PAA was studied with the presence of water.

25 ENERGY STORAGE↗
DOE OSTI2021

A Real-time Agent Based Optimization and Control Approach for Residential Building Heating Ventilation and Air Conditioning Systems

The prevalence of the loT (Internet of Things) is fostering the development of new control options and decision making that was not previously available. This is creating a wealth of opportunities for real-time control approaches and systems that can optimize for a common goal. This paper presents a smart residential neighborhood with optimization at the residential level utilizing a system of agents. The optimization utilizes information and modeling to optimize variable speed HVAC real-time operation in actual residential buildings. Data is presented showing the performance of the optimization and conclusions are drawn on next steps for development.

Hall, Joni↗
DOE OSTI2017

SAHA-induced TRAIL-sensitisation of Multiple Myeloma cells is enhanced in 3D cell culture

Background: Multiple Myeloma (MM) is currently incurable despite many novel therapies. Tumour Necrosis Factor-Related Apoptosis-Inducing Ligand (TRAIL) is a potential anti-tumour agent although effects as a single agent are limited. In this study, we investigated whether the Histone Deacetylase (HDAC) inhibitor SAHA can enhance TRAIL-induced apoptosis and target TRAIL resistance in both suspension culture, and 3D cell culture as a model of disseminated MM lesions that form in bone. Methods: The effects of SAHA and/or TRAIL in 6 Multiple Myeloma cell lines were assessed in both suspension cultures and in an Alginate-based 3D cell culture model. The effect of SAHA and/or TRAIL was assessed on apoptosis by assessment of nuclear morphology using Hoechst 33342/Propidium Iodide staining. Viable cell number was assessed by CellTiter-Glo luminescence assay, Caspase-8 and -9 activities were measured by Caspase-Glo™ assay kit. TRAIL-resistant cells were generated by culture of RPMI 8226 and NCI-H929 by acute exposure to TRAIL followed by selection of TRAIL-resistant cells. Results: TRAIL significantly induced apoptosis in a dose-dependent manner in OPM-2, RPMI 8226, NCI-H929, U266, JJN-3 MM cell lines and ADC-1 plasma cell leukaemia cells. SAHA amplified TRAIL responses in all lines except OPM-2, and enhanced TRAIL responses were both via Caspase-8 and -9. SAHA treatment induced growth inhibition that further increased in the combination treatment with TRAIL in MM cells. The co-treatment of TRAIL and SAHA reduced viable cell numbers all cell lines. TRAIL responses were further potentiated by SAHA in 3D cell culture in NCI-H929, RPMI 8226 and U266 at lower TRAIL + SAHA doses than in suspension culture. However TRAIL responses in cells that had been selected for TRAIL resistance were not further enhanced by SAHA treatment. Conclusions: SAHA is a potent sensitizer of TRAIL responses in both TRAIL sensitive and resistant cell lines, in both suspension and 3D culture, however SAHA did not sensitise TRAIL-sensitive cell populations that had been selected for TRAIL-resistance from initially TRAIL-sensitive populations. SAHA may increase TRAIL sensitivity in insensitive cells, but not in cells that have specifically been selected for acquired TRAIL-resistance. - Highlights: • The HDAC inhibitor SAHA sensitised Myeloma cells to TRAIL-induced apoptosis. • TRAIL, SAHA and combination apoptotic responses are potentiated in 3D cell culture. • TRAIL-resistant cells are not further sensitised to TRAIL-induced apoptosis by SAHA.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Constrained Deep Reinforcement Learning for Energy Sustainable Multi-UAV Based Random Access IoT Networks With NOMA

In this paper, we apply the Non-Orthogonal Multiple Access (NOMA) technique to improve the massive channel access of a wireless IoT network where solar-powered Unmanned Aerial Vehicles (UAVs) relay data from IoT devices to remote servers. Specifically, IoT devices contend for accessing the shared wireless channel using an adaptive p-persistent slotted Aloha protocol; and the solar-powered UAVs adopt Successive Interference Cancellation (SIC) to decode multiple received data from IoT devices to improve access efficiency. To enable an energy-sustainable capacity-optimal network, we study the joint problem of dynamic multi-UAV altitude control and multi-cell wireless channel access management of IoT devices as a stochastic control problem with multiple energy constraints. We first formulate this problem as a Constrained Markov Decision Process (CMDP), and propose an online model-free Constrained Deep Reinforcement Learning (CDRL) algorithm based on Lagrangian primal-dual policy optimization to solve the CMDP. Extensive simulations demonstrate that our proposed algorithm learns a cooperative policy in which the altitude of UAVs and channel access probability of IoT devices are dynamically controlled to attain the maximal long-term network capacity while ensuring energy sustainability of UAVs, outperforming baseline schemes. The proposed CDRL agent can be trained on a small network, yet the learned policy can efficiently manage networks with a massive number of IoT devices and varying initial states, which can amortize the cost of training the CDRL agent.

42 ENGINEERING↗
DOE OSTI2025

Reinforcement Learning for In-Spill Optimization of the Mu2e Resonant Extraction: Compensating Non-Stationarity

We present design considerations and challenges for the fast machine learning component of a third-order resonant beam extraction regulation system being commissioned to deliver steady beam rates to the mu2e experiment at Fermilab. Dedicated quadrupoles drive the tune toward the 29/3 resonance each spill, extracting beam at kV multiwire septa. The overall Spill Regulation System consists of (1) a “slow” process using ~100-spill averages to adjust the base quad ramp infrequently, (2) a feedforward harmonic content compensator, and (3) the “fast” ML agent reacting during each ongoing spill with on-the-fly additive corrections to the sum of (1) and (2). We have demonstrated improved beam-rate steadying for a fast ML agent compared to a PID controller using a quasi-physical spill simulation, and demonstrated distillation of that simulation into a predictive surrogate model. Current work includes a data-and-training pipeline to generate data-aware surrogates with real-world dynamics, even as the dynamics shift unpredictably. The surrogates are to act as RL environments against which to train our fast ML control agents before deploying them on FPGA in the live system. Further current efforts focus on modeling and controlling beam loss around the storage ring, understanding additional available hardware inputs to the model, and the interplay of these with beam-steadying performance.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1999

High-Resolution Mapping of Lunar Crustal Magnetic Fields: Correlations with Albedo Markings of the Reiner Gamma Class

During the last eight months of the Lunar Prospector mission (December 1999-July 1999), the spacecraft was placed in a relatively low-altitude (15-30-km perapsis), near-polar orbit that allowed high-resolution mapping of crustal magnetic fields. We report here initial studies of the correlation of locally strong magnetic anomalies with unusual, swirl-like albedo markings of the Reiner Gamma class. Based on this correlation, which is known from earlier studies of Apollo subsatellite magnetometer data, it has been proposed that the swirls represent regions whose higher albedos have been preserved via deflection of the solar-wind ion bombardment by strong crustal fields. This model in turn depends on the hypothesis that solar-wind implanted H is at least one component of the process that optically matures exposed silicate surfaces in the inner solar system . Specifically, it is hypothesized that implanted H acts as an effective reducing agent to enhance the rate of production of nanophase metallic Fe particles from preexisting silicates during micrometeoroid impacts. According to the model, the curvilinear shapes of these albedo markings are caused, at least in part, by the geometry of ion deflections in a magnetic field. The improved resolution and coverage of the Prospector data allow more detailed mapping of the fields, especially on the lunar farside. This permits a more quantitative test of whether all albedo markings of this class are associated with strong local magnetic fields.Only if the latter condition is met can the solar-wind deflection hypothesis he valid. The basic procedure for mapping crustal magnetic fields using Lunar Prospector magnetometer data follows that developed for analysis of Apollo subsatellite magnetometer data. The specific mapping steps are (1) selection of mission time intervals suitable for mapping crustal fields; these are limited essentially either to times when the Moon is in a lobe of the geomagnetic tail or to times when the Moon is in the solar wind but the spacecraft is in the lunar wake; the data are transformed to a radial, east, and north coordinate system with measurements given as a function of spacecraft latitude, longitude, and altitude; (2) visual editing of individual orbit segments selected for minimal external field disturbances; (3) minimization of remaining low-frequency external fields for individual orbit data segments by quadratic detrending; and (4) two-dimensional filtering of individual orbit segments to produce a vector field map along the slightly curved surface defined by the spacecraft altitude; maps of the three field components (radial, east, and north), the field magnitude, and the spacecraft altitude are constructed. For data obtained at low to middle latitudes, the horizontal resolution of the field maps is limited by the orbit-track separation (about 30 km at the equator). Maps of the field magnitude have been constructed within limited selenographic regions based mainly on data acquired in March, April, and May of 1999. This was a time period when the orbit plane was nearly aligned with the Sun-Moon line so that field mapping was possible at times when the Moon was in the solar wind as well as when the Moon was in the geomagnetic tail. Most of the coverage is across the lunar farside. However, a shows an example of a field map produced from solar-wind wake data for a region including Reiner Gamm on western Oceanus Procellarum (location: 58.5W, 7.5N). The contour interval is 3 nT and the mean spacecraft altitude is 18 km to within the accuracy allowed by the resolution of the map (30 km or about 1 deg.); strong magnetic anomalies correlate closely with swirl locations. Individual orbit profiles (whose resolution along the orbit track is comparable to the spacecraft altitude of 18 km) also demonstrate a good correlation of field magnitude with surface albedo. In order to investigate the correlation of magnetic fields with the location of swirl features, we have reexamined available lunar imagery (Lunar Orbiter, Apollo, and Clementine) to identify and map swirl locations within regions where swirls have previously been mapped. In these images, swirls were distinguished from other high-albedo features such as crater rays by their curvilinear shapes and increased visibility in forward-scattered light. Digital maps of swirls identified by all available imagery were then superposed on maps of the field magnitude at the spacecraft altitude. Based upon analysis of these composite magnetic/geologic maps, we draw the preliminary conclusion that swirl features are associated with magnetic anomalies revealed by Lunar Prospector. Detailed maps of these swirl features are currently being constructed for the magnetically strong regions antipodal to the Imbrium, Serenitatis, and Crisium Basins. Additional information contained in the original,

Hood, L. L.↗
NASA NTRSDate not supplied

Use of Design of Experiments and Rule-Based Inference in Determining Neural Network Architectures for Loss of Control Detection

In this work, we describe methods for selecting the neural network architectures and input spaces to implement belief state inference on generic commercial transport aircraft. First, we highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). We present a structured method that can be used in a number of aerospace applications, to optimize the structure and training parameters of the CVAE for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from a NASA T-2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications. While theoretically, we could create a single CVAE with an input space the size of all measurable flight variables and environmental dynamics, it becomes intractable to use such a neural network in an in-situ intelligent multi-agent system. Using the recommendations from our case study, we introduce a technical approach for feasibly describing the belief space by (1) identifying significant statistical relationships among flight variables using rule induction, (2) using a set of rules that cover all features to define the input space of multiple CVAEs, and (3) forming a belief space based on the joint probability density of their collective latent spaces. This results in a series of relatively small matrix multiplications that can be performed in real time, as opposed to large matrix computations in a single CVAE. We demonstrate the application of this approach on the T-2 flight loss-of control experiments, using the architecture and hyperparameter recommendations from the case study. We compare the utilities of an individual CVAE trained on all flight variables and multiple CVAEs defined on subsets of flight variables for detecting qualitative changes in flight. We demonstrate that the use of multiple CVAEs with smaller input spaces permits the CVAE to capture more granular relationships in the latent space, permitting better state space characterization and loss-of-control detection.

Design of experiments↗
DOE OSTI2020

Structure, Hydration, and Interactions of Native and Hydrophobically Modified Phytoglycogen Nanoparticles

Phytoglycogen is a highly branched polymer of glucose produced as soft, compact nanoparticles by sweet corn. Properties such as softness, porosity, and mechanical integrity, combined with nontoxicity and biodegradability, make phytoglycogen nanoparticles ideal for applications involving the human body, ranging from skin moisturizing and rejuvenation agents in personal care formulations to functional therapeutics in biomedicine. To further broaden the range of applications, phytoglycogen nanoparticles can be chemically modified with hydrophobic species such as octenyl succinic anhydride (OSA). In this study, we present a self-consistent model of the particle structure, water content, and degree of chemical modification of the particles, as well as the emergence of well-defined interparticle spacings in concentrated dispersions, based on small-angle neutron scattering (SANS) measurements of aqueous dispersions of native phytoglycogen nanoparticles and particles that were hydrophobically modified using octenyl succinic anhydride (OSA) in both its protiated (pOSA) and deuterated (dOSA) forms. Measurements on native particles with reduced polydispersity have allowed us to refine the particle morphology, which is well described by a hairy particle (core-chain) geometry with short chains decorating the surface of the particles. The isotopic variants of OSA-modified particles enhanced the scattering contrast for neutrons, revealing lightly modified hairy chains for small degrees of substitution (DS) of OSA, and a raspberry particle geometry for the largest DS value, where the OSA-modified hairy chains collapse to form small seeds on the surface of the particles. This refined model of native and OSA-modified phytoglycogen nanoparticles establishes a quantitative basis for the development of new applications of this promising sustainable nanotechnology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Adsorptive behavior of poly (vinylidene fluoride) membranes for the recovery of lignin-derived hydrophobic deep eutectic solvents

Abstract Recently, membrane technology has gained significant traction as an energy-efficient alternative to traditional thermal processes for solvent recovery. Deep eutectic solvents (DESs) have emerged as sustainable alternatives to conventional organic solvents, yet a systematic methodology for selecting compatible membrane materials for their recovery remains underdeveloped. This study established a predictive framework for membrane material selection in hydrophobic DES applications using Hansen Solubility Parameters (HSP) with inverted criteria targeting materials with relative energy difference (RED) values greater than 1.0. Flat sheet membranes were fabricated via the non-solvent induced phase separation (NIPS) technique. Four NIPS fabricated polymer membranes were evaluated: polysulfone, cellulose acetate, polyvinylidene fluoride (PVDF) fabricated with polyethylene glycol (PEG) as a pore-forming agent, and polybenzimidazole (PBI). The HSP approach successfully predicted membrane-solvent compatibility, with polysulfone (RED = 0.6) and cellulose acetate (RED = 0.9) dissolving completely within 24 h, while PVDF (RED = 1.9) and PBI (RED = 1.1) maintained structural integrity throughout a 7-day exposure period. Furthermore, PVDF demonstrated superior performance with minimal weight gain (3.0%), hydrophobic surface characteristics (122° water contact angle), and enhanced mechanical properties following DES exposure. Comprehensive chemical and morphological characterization confirmed PVDF’s chemical stability and revealed a surface-selective interaction mechanism involving simultaneous PEG (pore-forming agent) extraction and DES component adsorption. Adsorption kinetics followed pseudo-first-order behavior with reversible characteristics, best described by the Temkin isotherm model (R² = 0.9987). PVDF membranes-maintained separation functionality with average lignin rejection (75.2 ± 7.69%) and demonstrated filtration permeability of 2.0 ± 0.34 LMH/bar. This methodology provides a rational approach for membrane selection in emerging solvent systems, contributing to the advancement of sustainable separation technologies for DES-based biomass processing applications.

Science & Technology - Other Topics↗
DOE OSTI2025

Reinforcement Learning‐Based Adaptation of Grid Following Inverter's Internal Controller to Networked Microgrids' Strengths

The varying topological configurations, generator commitments and dispatches, and dynamic load demand lead to changing system's strengths during the operations of networked microgrids. When the system's strengths significantly change, the fixed control gains at large devices may result in unsatisfactory system performance; this necessitates the tuning of the control gains at large devices to adapt to the changing system's strengths. In this paper, observer-based reinforcement learning (RL) is utilised to automatically tune the proportional-integral (PI) gains of phase lock loop (PLL) controller of grid-following (GFL) inverters to adapt to the changing strengths of microgrids and networked microgrids. The RL agent in this framework augments an observer predicting system's strengths, from which the RL control policy will adjust accordingly to tune the PLL controller's gains towards the system's strengths. Also, to enhance the control performance, the recently introduced Barrier function-based RL framework is leveraged for the design of reward function to prevent the high frequency nadir. An operational 26 kV electric distribution system, which is modelled as networked microgrids, is used to illustrate the need and effectiveness of the proposed RL-tuned control.

frequency response↗
DOE OSTI2024

Computational and Systems Biology Advances to Enable Bioagent Agnostic Signatures

Enumerated threat agent lists have long driven biodefense priorities. The global SARS-CoV-2 pandemic demonstrated the limitations of searching for known threat agents as compared to a more agnostic approach. Recent technological advances are enabling agent-agnostic biodefense, especially through the integration of multi-modal observations of host-pathogen interactions directed by a human immunological model. Although well-developed technical assays exist for many aspects of human-pathogen interaction, the analytic methods and pipelines to combine and holistically interpret the results of such assays are immature and require further investments to exploit new technologies. In this manuscript, we discuss potential immunologically based bioagent-agnostic approaches and the computational tool gaps the community should prioritize filling.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Multi-Agent Hierarchical Deep Reinforcement Learning for HVAC Control With Flexible DERs

As electricity consumption in commercial and residential buildings continues to rise, reducing energy costs presents an increasing challenge. Heating, ventilating, and air-conditioning (HVAC) systems, which typically account for 40%-50% of a building's energy use, are prime targets for energy savings. Intelligent control of HVAC temperature through the exploitation of HVAC load flexibility brings significant potential to reduce energy consumption and electricity expenses. The nonlinear models of HVAC systems challenge traditional control methods, while the uncertainty introduced by HVAC load flexibility complicates distributed energy resource (DER) management using conventional optimal dispatch techniques. In response to these challenges, we propose a hierarchical multi-agent deep reinforcement learning (DRL) approach. The lower-level agents focus on balancing comfort and energy conservation, while the upper-level DRL agents optimize the use of DERs to reduce peak demand based on the control outcomes of the HVAC by the lower-level agents. Here, in the upper-level agents, we incorporate a multi-agent structure based on ensemble learning, which acts based on historical and current data without relying on precise load forecasting to address the delayed rewarding issue in DRL. This allows for the effective reduction of energy costs. The proposed method is tested using a real-world microgrid comprising 413 buildings in Southern California, and the results demonstrate that our approach can significantly reduce overall electricity bills while ensuring the comfort of consumers and residents.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Evaluate the System-Level Impact of Connected and Automated Vehicles Coupled with Shared Mobility: An Agent-based Simulation Approach

With the rapid growth of information and communication technologies, Connected and Automated Vehicles (CAVs) are deemed to be disruptive with the potential to significantly improve overall transportation system efficiency, however, may bring Vehicle Miles Traveled (VMT) increase or other issues. Further, shared mobility systems are another disruptive force that is reshaping our travel patterns. To quantify the combined impact of CAV and shared mobility on travel behavior, traffic performance and energy efficiency, we develop a mesoscopic simulation-based framework for mobility and energy efficiency evaluation considering the disruptive transportation technologies. Under this framework, we develop novel models for energy intensity and modal activity, and evaluated a variety of energy scenarios for different combinations of CAV applications, various levels of automation, roadway characteristics, and traffic conditions, while also varying different vehicle types and fuel/powertrain technologies. Based applying this modeling suite to a calibrated BEAM simulation network in Riverside California, it was found that cooperative automated driving in general will improve mobility, but automated vehicles, even when deployed in a shared autonomous fleet, will likely bring an increase of VMT (up to 36%) due to mode shifts and deadheading. Ride-hailing vehicles typically have better energy efficiency and a higher share of electric vehicles, which helps offset the negative impact from VMT increases when estimating the system-level energy consumption. In general, simulation results show a 6% increase in energy consumption for the scenarios with an increasing shift to ride-hailing modes.

42 ENGINEERING↗
DOE OSTI2023

Deep Reinforcement Learning Based Smart Water Heater Control for Reducing Electricity Consumption and Carbon Emission

Water heating is the third largest electricity consumer in U.S. households, after space heating and cooling. Thus, water heaters represent a significant potential for reducing electricity consumption and associated CO2 emissions of residential buildings. To this end, this study proposes a model-free deep reinforcement learning (RL) approach that aims to minimize the electricity consumption and the CO2 emissions of a heat pump water heater without affecting user comfort. In this approach, a set of RL agents focusing on either electricity saving or emission reduction, with different look ahead periods, were trained using the deep Q-networks (DQN) algorithm and their performance was tested on different hot water usage and Marginal Operating Emissions Rate (MOER) profiles. The testing results showed that the RL agents that focus on electricity saving can save electricity in the range of 12–22% by operating the water heater with maximum heat pump efficiency and minimum electric element utilization. On the other hand, the RL agents that focus on emission reduction reduced emissions in the range of 18–37% by making use of the variable MOER values. These RL agents used the heat pump and/or an element when the MOER values are low due to the availability of renewable energy sources (e.g., solar and wind) and mostly avoided the periods of carbon-intensive periods. Overall, these results showed that the proposed RL approach can help minimize the electricity consumption and the CO2 emissions of a heat pump water heater without having any prior knowledge about the device.

Amasyali, Kadir↗
DOE OSTI2021

Decentralized Voltage Control of Large-Scale Distribution System with PVs Based on MADRL

This paper proposes a model-free decentralized control framework for the voltage regulation of large-scale distribution systems through the coordinated control of PV inverters. This is achieved by developing a novel interaction mechanism between the surrogate model and the centralized training and decentralized execution multiagent deep reinforcement learning framework. Specifically, the sparse Gaussian processes regression method is first utilized to develop the surrogate model of the original distribution system for reward calculation during the training stage, where each agent represents a sub-region in the centralized fashion for coordination strategy learning. After that, the learned control rules are used to inform controllers within each sub-region for real-time decisions with only local measurements. Comparative tests among various methods on the EPRI Ckt5 test system demonstrate the effectiveness of the proposed method.

distribution system↗