Search NASA⌕ Search

SEARCH · Search NASA

Results for “REDUCING AGENTS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

Considerations in Assuring Safety of Increasingly Autonomous Systems

Recent technological advances have accelerated the development and application of increasingly autonomous (IA) systems in civil and military aviation. IA systems can provide automation of complex mission tasks-ranging across reduced crew operations, air-traffic management, and unmanned, autonomous aircraft-with most applications calling for collaboration and teaming among humans and IA agents. IA systems are expected to provide benefits in terms of safety, reliability, efficiency, affordability, and previously unattainable mission capability. There is also a potential for improving safety by removal of human errors. There are, however, several challenges in the safety assurance of these systems due to the highly adaptive and non-deterministic behavior of these systems, and vulnerabilities due to potential divergence of airplane state awareness between the IA system and humans. These systems must deal with external sensors and actuators, and they must respond in time commensurate with the activities of the system in its environment. One of the main challenges is that safety assurance, currently relying upon authority transfer from an autonomous function to a human to mitigate safety concerns, will need to address their mitigation by automation in a collaborative dynamic context. These challenges have a fundamental, multidimensional impact on the safety assurance methods, system architecture, and V&V capabilities to be employed. The goal of this report is to identify relevant issues to be addressed in these areas, the potential gaps in the current safety assurance techniques, and critical questions that would need to be answered to assure safety of IA systems. We focus on a scenario of reduced crew operation when an IA system is employed which reduces, changes or eliminates a human's role in transition from two-pilot operations.

Alves, Erin E.↗
NASA NTRSDate not supplied

Bootstrapping Multi-Agent Unmanned Aerial Vehicle (UAV) System Integration Using Ground-Based Assets: Lessons Learned

In support of the Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project, a fleet of unmanned ground vehicles (UGVs) was developed as a test and evaluation (T\&E) platform to reduce system integration gaps between simulation and live flight hardware. While simulation and hardware-in-the-loop bench testing provide adequate environments for preliminary validation, differences in system deployment architecture, software interfaces, and hardware infrastructure increase the risks to safety, property, and the project. Given ATTRACTOR’s goal of establishing a basis of certification of trust and trustworthiness in multi-agent autonomous systems, bridging these gaps was critical to successful project execution and feasibility assessment. In this paper we present the UGV fleet and its role in speeding up system integration, smoothing the transition from simulation to flight, and providing researchers an easy-to-use hardware test bed. An overview of the hardware and software on-board the vehicles is provided along with supporting infrastructure. The system integration process is documented including results in supporting both the overarching design reference mission (DRM) of ATTRACTOR and individual research efforts conducted since the creation of the fleet. Finally, we discuss the practical lessons learned regarding the testing, deployment, and operation of multi-agent autonomous systems.

Matthew P. Vaughan↗
DOE OSTI2023

Elemental zoning enhances mass transport in zeolite catalysts for methanol to hydrocarbons

Mass transport limitations in zeolite catalysts pose major hurdles for their optimal performance in diverse chemical reactions. Most approaches to reduce these restrictions focus on the synthesis of either hierarchical or nanosized zeolites. Here we demonstrate that the existence of a siliceous, catalytically inactive exterior rim on ZSM-5 particles dramatically reduces the diffusion limitations, which leads to an enhanced catalyst lifetime for the methanol-to-hydrocarbon reaction. In this study, our findings reveal that binary inorganic and organic structure-directing agents enable a one-pot synthesis of Si-zoned ZSM-5 catalysts with diffusion properties that are characteristic of particles with a much smaller size. Operando ultraviolet–visible light diffuse reflectance spectroscopy reveals a marked reduction in external coking among Si-zoned samples. Molecular dynamics simulations to assess the diffusion of methanol and benzene in siliceous pores and in those with Brønsted acids reveal substantially reduced transport limitations in zoned regions, consistent with the improved catalyst activity of Si-zoned zeolites relative to that of ZSM-5 with a homogeneous acid-site distribution.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

A new approach of reduction of carbon dioxide emission and optimal use of carbon and hydrogen content for the desired syngas production from coal

We report this research is designed to make progress in overcoming the challenges through the development of a two-stage coal processing. Specifically, a two-stage process was used to maximize the use of the carbon in coal or increase the carbon monoxide yield or lower carbon dioxide and methane yields. Carbon dioxide-char gasification in the absence of water can generate carbon monoxide with near-zero methane, which is desired to produce high-carbon and low-hydrogen chemicals such as oxalic acid through catalytic carbon monoxide coupling and hydrolysis. The technology is applicable to any coal, although Power River Basin (PRB) coal is used as an example feedstock in this research Also, the sodium-iron catalyst can accelerate not only the reaction kinetics in both stages but also increase the hydrogen/carbon monoxide ratio in the syngas produced in the second stage. In other words, the catalyst is a multifunctional agent, which can not only intensify the overall coal process efficiency but also improve the qualities of the desired syngas products and reduce carbon dioxide emission. Thus, In the 1st stage, the catalysts can significantly reduce the CO 2 -char gasification by as high as 75.00%. In the 2nd stage, the H 2 /CO ratio of ideal syngas is ~2:1 with near-0 CH 4 generation and the CH4 production can be reduced as high as 61.29% for the CO 2 –H 2 O-Char coal gasification. The activation energy for the 2nd stage is reduced by as high as 35.43% than that of raw coal without use Na–Fe catalyst. The overall carbon footprint reduction for the study is reduced by 87.33% in the 1st stage and 96.77% compared to the direct coal combustion.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2021

Chlamydomonas as a model for reactive oxygen species signaling and thiol redox regulation in the green lineage

While the maintenance of redox balance is important for all organisms, the energy balance in the chloroplast is particularly tightly linked to the overall cellular redox status in photosynthetic organisms. Excess light can cause overreduction of the photosynthetic electron transport chain, which, when unchecked, results in the generation of reactive oxygen species (ROS) that can oxidize cellular components including nucleic acids, proteins, and lipids (Li et al., 2009). A photosynthetic cell possesses multiple layers of defense against oxidative stress, such as antioxidant enzymes (Foyer and Noctor, 2005) and alternative electron transfer pathways (Alric and Johnson, 2017). At the same time, ROS can function as important signals that mediate responses to biotic and abiotic stresses (Apel and Hirt, 2004). Reversibly oxidizable small molecules and proteins can serve as ROS sinks and signals and are re-reduced by multiple mechanisms. For example, the ascorbate–glutathione (GSH) cycle is central in controlling ROS levels and regenerating/reducing oxidized cellular redox agents (Figure 1A; reviewed by Foyer and Noctor, 2011). The functions of many proteins can be regulated by the redox state of cysteine residues, such as the formation or reduction of disulfides by thioredoxins (TRX) (Buchanan, 1991) and glutaredoxins (Michelet et al., 2013b; Figure 1, B and C).

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1993

Comparison of Gavage, Water Bottle, and a High-Moisture Diet Bolus as Dosing Methods for Quantitative D-xylose Administration to B6D2F1 (Mus musculus) Mice

Gavage, water bottle, and diet incorporation are 3 dosing methods used orally to administer test compounds to rodents. These 3 methods were compared in mice to determine which represented the most quantitative delivery system. For dietary incorporation, a high-moisture bolus form of NIH-31 rodent meal was developed using hydroxypropyl methylcellulose as an autoclave-stable binding agent. A high-moisture bolus were selected to increase the acceptability of the dosed diet and to promote quantitative consumption through reduced wastage. The test compound used was D-xylose, a pentose sugar that may be quantitatively detected, colorimetrically, in urine following oral dosing. Six male and 6 female B6D2FI mice were placed in metabolism cages and dosed with a known quantity of D-xylose by each of the 3 methods. Urine was collected before and after each method of administration and analysed for total D-xylose; the per cent recovery was based upon the amount of D-xylose consumed. Quantitative consumption was apparently greatest for water bottle dosing with an average recovery of 56.0% of the original D-xylose dose. High-moisture bolus incorporation ranked second with 50.0% D-xylose recovery, and gavage was third with 41.0% D-xylose recovery.

Zimmer, J. Paul↗
NASA NTRS1997

Evaluation of Thermal Control Coatings for Flexible Ceramic Thermal Protection Systems

This report summarizes the evaluation and testing of high emissivity protective coatings applied to flexible insulations for the Reusable Launch Vehicle technology program. Ceramic coatings were evaluated for their thermal properties, durability, and potential for reuse. One of the major goals was to determine the mechanism by which these coated blanket surfaces become brittle and try to modify the coatings to reduce or eliminate embrittlement. Coatings were prepared from colloidal silica with a small percentage of either SiC or SiB6 as the emissivity agent. These coatings are referred to as gray C-9 and protective ceramic coating (PCC), respectively. The colloidal solutions were either brushed or sprayed onto advanced flexible reusable surface insulation blankets. The blankets were instrumented with thermocouples and exposed to reentry heating conditions in the Ames Aeroheating Arc Jet Facility. Post-test samples were then characterized through impact testing, emissivity measurements, chemical analysis, and observation of changes in surface morphology. The results show that both coatings performed well in arc jet tests with backface temperatures slightly lower for the PCC coating than with gray C-9. Impact testing showed that the least extensive surface destruction was experienced on blankets with lower areal density coatings.

Kourtides, Demetrius↗
NASA NTRS2002

Differential effects of intermittent and continuous administration of parathyroid hormone on bone histomorphometry and gene expression

A mechanism explaining the differential skeletal effects of intermittent and continuous elevation of serum parathyroid hormone (PTH) remains elusive. Intermittent PTH increases bone formation and bone mass and is being investigated as a therapy for osteoporosis. By contrast, chronic hyperparathyroidism results in the metabolic bone disease osteitis fibrosa characterized by osteomalacia, focal bone resorption, and peritrabecular bone marrow fibrosis. Intermittent and continuous PTH have similar effects on the number of osteoblasts and bone-forming activity. Many of the beneficial as well as detrimental effects of the hormone appear to be mediated by osteoblast-derived growth factors. This hypothesis was tested using cDNA microgene arrays to compare gene expression in tibia of rats treated with continuous and pulsatile administration of PTH. These treatments result in differential expression of many genes, including growth factors. One of the genes whose steady-state mRNA levels was increased by continuous but not pulsatile administration was platelet-derived growth factor-A (PDGF-A). Administration of a PDGF-A antagonist greatly reduced bone resorption, osteomalacia, and bone marrow fibrosis in a rat model for hyperparathyroidism, suggesting that PDGF-A is a causative agent for this disease. These findings suggest that profiling changes in gene expression can help identify the metabolic pathways responsible for the skeletal responses to the hormone.

Review↗
NASA NTRS2008

Use of Potassium Citrate to Reduce the Risk of Renal Stone Formation During Spaceflight

Introduction: NASA s Vision for Space Exploration centers on exploration class missions including the goals of returning to the moon and landing on Mars. One of NASA s objectives is to focus research on astronaut health and the development of countermeasures that will protect crewmembers during long duration voyages. Exposure to microgravity affects human physiology and results in changes in the urinary chemical composition favoring urinary supersaturation and an increased risk of stone formation. Nephrolithiasis is a multifactorial disease and development of a renal stone is significantly influenced by both dietary and environmental factors. Previous results from long duration Mir and short duration Shuttle missions have shown decreased urine volume, pH, and citrate levels and increased calcium. Citrate, an important inhibitor of calcium-containing stones, binds with urinary calcium reducing the amount of calcium available to form stones. Citrate inhibits renal stone recurrence by preventing crystal growth, aggregation, and nucleation and is one of the most common therapeutic agents used to prevent stone formation. Methods: Thirty long duration crewmembers (29 male, 1 female) participated in this study. 24-hour urines were collected and dietary monitoring was performed pre, in, and postflight. Crewmembers in the treatment group received two potassium citrate (KCIT) pills, 10 mEq/pill, ingested daily beginning 3 days before launch, all inflight days and through 14 days postflight. Urinary biochemical and dietary analyses were completed. Results: KCIT treated subjects exhibited decreased urinary calcium excretion and maintained the levels of calcium oxalate supersaturation risk at their preflight levels. The increased urinary pH levels in these subjects reduced the risk of uric acid stones. Discussion: The current study investigated the use of potassium citrate as a countermeasure to minimize the risk of stone formation during ISS missions. Results suggest that supplementation with potassium citrate decreases the risk of stone formation during and immediately after spaceflight.

Whitson, P. A.↗
DOE OSTI2021

Role of Cation Ordering on Device Performance in (Ag,Cu)InSe 2 Solar Cells with KF Post-Deposition Treatment

CuInSe 2 (CIS) has been proposed as an attractive bottom cell candidate in tandem solar cells. However, to justify the coupling with high-performance top cells (e.g., perovskites, GaAs), significant work on improving the efficiency is required. To this extent, several authors have demonstrated the benefits of alkali post-deposition treatments (PDT) to increase device open-circuit voltage (V oc ) in CIS and how Ag alloying - (Ag,Cu)InSe 2 (ACIS) - reduces defect density and enhances current collection in devices. Herein, we present a detailed study of the role that KF-PDT plays on CIS and ACIS absorber composition and structure, and propose an explanation for the decreased V oc observed when silver and potassium coexist in the system (ACIS + KF). Through a suite of synchrotron-based techniques, we investigate the nanoscale chemical distribution of the films and the formation of secondary phases. Through photoluminescence imaging, we observed a high degree of passivation with the addition of KF, and synchrotron-based X-ray diffraction confirmed the absence of a KInSe 2 surface layer usually considered to be a passivating agent. Raman spectroscopy and synchrotron X-ray fluorescence show the increased presence of Cu- and Se-poor clusters in ACIS + KF, which are correlated to significantly reduced X-ray beam-induced current (XBIC). An increase in the intensity of the E/B 2 stretching mode of CIS is attributed to cation ordering near the junction and is found to track inversely to bulk V oc measurements. The cation ordering is hypothesized to arise from the formation and redistribution of defects that normally occur near the surfaces of CIS as a consequence of its polar character. Here, these defects compensate each other, and the overall inhomogeneity of the charge distribution generates electrostatic potential fluctuations that greatly increase the saturation current and hence reduce the open-circuit voltage of the device.

36 MATERIALS SCIENCE↗
NASA NTRS2009

Renal Stone Risk during Spaceflight: Assessment and Countermeasure Validation

NASA's Vision for Space Exploration centers on exploration class missions including the goals of returning to the moon and landing on Mars. One of NASA's objectives is to focus research on astronaut health and the development of countermeasures that will protect crewmembers during long duration voyages. Exposure to microgravity affects human physiology and results in changes in the urinary chemical composition favoring urinary supersaturation and an increased risk of stone formation. Nephrolithiasis is a multifactorial disease and development of a renal stone is significantly influenced by both dietary and environmental factors. Previous results from long duration Mir and short duration Shuttle missions have shown decreased urine volume, pH, and citrate levels and increased calcium. Citrate, an important inhibitor of calcium-containing stones, binds with urinary calcium reducing the amount of calcium available to form stones. Citrate inhibits renal stone recurrence by preventing crystal growth, aggregation, and nucleation and is one of the most common therapeutic agents used to prevent stone formation. Methods: Thirty long duration crewmembers (29 male, 1 female) participated in this study. 24-hour urines were collected and dietary monitoring was performed pre-, in-, and postflight. Crewmembers in the treatment group received two potassium citrate (KCIT) pills, 10 mEq/pill, ingested daily beginning 3 days before launch, all in-flight days and through 14 days postflight. Urinary biochemical and dietary analyses were completed. Results: KCIT treated subjects exhibited decreased urinary calcium excretion and maintained the levels of calcium oxalate supersaturation risk at their preflight levels. The increased urinary pH levels in these subjects reduced the risk of uric acid stones. Discussion: The current study investigated the use of potassium citrate as a countermeasure to minimize the risk of stone formation during ISS missions. Results suggest that supplementation with potassium citrate decreases the risk of stone formation during and immediately after spaceflight.

Whitson, Peggy A.↗
DOE OSTI2024

Multi-agent voltage control in distribution systems using GAN-DRL-based approach

Active distribution grids can experience voltage fluctuations and violations due to the high penetration of variable distributed energy resources (DERs). These problems might occur because of the uncertain and variable generation natures of these resources, especially solar photovoltaic resources, during panel shadowing scenarios. Volt-VAR control (VVC) is an efficient method that controls the reactive power set-points of the inverters to regulate the voltage of distribution grids. Although several VVC approaches have been proposed recently, the performance of these approaches degrades significantly if behind-the-meter solar generation data are unobservable/missing. Therefore, it is necessary to impute missing/unobservable PV data accurately to be utilized in VVC approaches. Further, this paper proposes a model-free, data-driven, centrally trained, and decentrally executed multi-agent deep reinforcement learning-based VVC architecture to regulate the voltage of distribution networks. A generative adversarial network (GAN) is incorporated to impute the unobservable PV data accurately, which improves the performance of the proposed control architecture. The proposed multi-agent-soft-actor–critic algorithm (MASAC)-based VVC technique utilizes the actual PV dataset as well as the imputed dataset from the GAN framework to learn the optimal coordinated control policy for controlling the optimal reactive power set-points of PV inverters. The effectiveness of the proposed approach is analyzed on a modified IEEE 34-bus test case with added PV inverters. The results are compared and analyzed with a base case model with no VVC and VVC with a local droop control approach, genetic algorithm optimization, and a centralized soft actor–critic-based approach. Moreover, the performance of the proposed approach is compared with that of a multi-agent VVC framework without using the PV generation data and load information as the system state. The results illustrate that the proposed method with more state input improves the voltage profile and reduces the power loss of the network across various loading and PV generation scenarios.

14 SOLAR ENERGY↗
DOE OSTI2026

Smart Charging of Fleet and Personal Electric Vehicles through Joint Vehicle-to-Grid Optimization

As electric vehicle (EV) adoption accelerates, vehicle-to-grid (V2G) strategies offer advantages over unmanaged charging (V0G) by enhancing grid stability, reducing fleet operation costs, and supporting integration of variable generation resources. This research develops a day-ahead optimization framework linked with agent-based simulations to evaluate coordinated V2G participation by fleet and personal EVs under 5 energy-pricing settings in Austin, Texas. Three scenarios (V0G, fleet-only V2G, and joint-V2G) are examined, considering real-time price and grid profiles, health-damage costs, and operational constraints for both fleet and personal EVs. Results show how V2G scenarios shift fleet EV charging to mid-day while enabling strategic battery-discharge during evening peaks, mitigating grid stress and lowering EV energy costs. V2G delivers close to 80% energy-cost savings for a 2000-EV fleet in Austin on grid-stressed days, with 55% lower charging pollutant outputs. Joint-V2G amplifies system-level benefits by complementing fleet discharge, but smart-charging equipment costs can offset those benefits.

Electric vehicle↗
DOE OSTI2021

Next Generation Chemical Agent Detection Architecture [Poster]

Sandia National Labs (SNL)-designed, portable chemical warfare agent (CWA) detection systems consist of three-stages: collection, separation, and detection. We use microfabrication technologies to miniaturize these stages and to reduce the overall size, weight, power, and (potentially) cost of the final system. Our newest system consists of a multi-dimensional separation stage and an miniature ion mobility spectrometer (IMS) detector for unprecedented system sensitivity, selectivity, and depth of target list.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRS2011

PMA-PhyloChip DNA Microarray to Elucidate Viable Microbial Community Structure

Since the Viking missions in the mid-1970s, traditional culture-based methods have been used for microbial enumeration by various NASA programs. Viable microbes are of particular concern for spacecraft cleanliness, for forward contamination of extraterrestrial bodies (proliferation of microbes), and for crew health/safety (viable pathogenic microbes). However, a "true" estimation of viable microbial population and differentiation from their dead cells using the most sensitive molecular methods is a challenge, because of the stability of DNA from dead cells. The goal of this research is to evaluate a rapid and sensitive microbial detection concept that will selectively estimate viable microbes. Nucleic acid amplification approaches such as the polymerase chain reaction (PCR) have shown promise for reducing time to detection for a wide range of applications. The proposed method is based on the use of a fluorescent DNA intercalating agent, propidium monoazide (PMA), which can only penetrate the membrane of dead cells. The PMA-quenched reaction mixtures can be screened, where only the DNA from live cells will be available for subsequent PCR reaction and microarray detection, and be identified as part of the viable microbial community. An additional advantage of the proposed rapid method is that it will detect viable microbes and differentiate from dead cells in only a few hours, as opposed to less comprehensive culture-based assays, which take days to complete. This novel combination approach is called the PMA-Microarray method. DNA intercalating agents such as PMA have previously been used to selectively distinguish between viable and dead bacterial cells. Once in the cell, the dye intercalates with the DNA and, upon photolysis under visible light, produces stable DNA adducts. DNA cross-linked in this way is unavailable for PCR. Environmental samples suspected of containing a mixture of live and dead microbial cells/spores will be treated with PMA, and then incubated in the dark. Thereafter, the sample is exposed to visible light for five minutes, so that the DNA from dead cells will be cross-linked. Following this PMA treatment step, the sample is concentrated by centrifugation and washed (to remove excessive PMA) before DNA is extracted. The 16S rRNA gene fragments will be amplified by PCR to screen the total microbial community using PhyloChip DNA microarray analysis. This approach will detect only the viable microbial community since the PMA intercalated DNA from dead cells would be unavailable for PCR amplification. The total detection time including PCR reaction for low biomass samples will be a few hours. Numerous markets may use this technology. The food industry uses spore detection to validate new alternative food processing technologies, sterility, and quality. Pharmaceutical and medical equipment companies also detect spores as a marker for sterility. This system can be used for validating sterilization processes, water treatment systems, and in various public health and homeland security applications.

Venkateswaran, Kasthuri J.↗
NASA NTRS2011

Contact Graph Routing

Contact Graph Routing (CGR) is a dynamic routing system that computes routes through a time-varying topology of scheduled communication contacts in a network based on the DTN (Delay-Tolerant Networking) architecture. It is designed to enable dynamic selection of data transmission routes in a space network based on DTN. This dynamic responsiveness in route computation should be significantly more effective and less expensive than static routing, increasing total data return while at the same time reducing mission operations cost and risk. The basic strategy of CGR is to take advantage of the fact that, since flight mission communication operations are planned in detail, the communication routes between any pair of bundle agents in a population of nodes that have all been informed of one another's plans can be inferred from those plans rather than discovered via dialogue (which is impractical over long one-way-light-time space links). Messages that convey this planning information are used to construct contact graphs (time-varying models of network connectivity) from which CGR automatically computes efficient routes for bundles. Automatic route selection increases the flexibility and resilience of the space network, simplifying cross-support and reducing mission management costs. Note that there are no routing tables in Contact Graph Routing. The best route for a bundle destined for a given node may routinely be different from the best route for a different bundle destined for the same node, depending on bundle priority, bundle expiration time, and changes in the current lengths of transmission queues for neighboring nodes; routes must be computed individually for each bundle, from the Bundle Protocol agent's current network connectivity model for the bundle s destination node (the contact graph). Clearly this places a premium on optimizing the implementation of the route computation algorithm. The scalability of CGR to very large networks remains a research topic. The information carried by CGR contact plan messages is useful not only for dynamic route computation, but also for the implementation of rate control, congestion forecasting, transmission episode initiation and termination, timeout interval computation, and retransmission timer suspension and resumption.

Burleigh, Scott C.↗
NASA NTRS2003

Agent Based Software for the Autonomous Control of Formation Flying Spacecraft

Distributed satellite systems is an enabling technology for many future NASA/DoD earth and space science missions, such as MMS, MAXIM, Leonardo, and LISA [1, 2, 3]. While formation flying offers significant science benefits, to reduce the operating costs for these missions it will be essential that these multiple vehicles effectively act as a single spacecraft by performing coordinated observations. Autonomous guidance, navigation, and control as part of a coordinated fleet-autonomy is a key technology that will help accomplish this complex goal. This is no small task, as most current space missions require significant input from the ground for even relatively simple decisions such as thruster burns. Work for the NMP DS1 mission focused on the development of the New Millennium Remote Agent (NMRA) architecture for autonomous spacecraft control systems. NMRA integrates traditional real-time monitoring and control with components for constraint-based planning, robust multi-threaded execution, and model-based diagnosis and reconfiguration. The complexity of using an autonomous approach for space flight software was evident when most of its capabilities were stripped off prior to launch (although more capability was uplinked subsequently, and the resulting demonstration was very successful).

How, Jonathan P.↗
DOE OSTI2023

Online transfer learning strategy for enhancing the scalability and deployment of deep reinforcement learning control in smart buildings

In recent years, advanced control strategies based on Deep Reinforcement Learning (DRL) proved to be effective in optimizing the management of integrated energy systems in buildings, reducing energy costs and improving indoor comfort conditions when compared to traditional reactive controllers. However, the scalability and implementation of DRL controllers are still limited since they require a considerable amount of time before converging to a near-optimal solution. This issue is currently addressed in literature through the offline pre-training of the DRL agent. However this solution results in two main critical issues: (1) the need to develop a building surrogate model to perform the training task, and (2) the need to perform a fine-tuning process over several training episodes to obtain a near-optimal control policy. In this context, this paper introduces an Online Transfer Learning (OTL) strategy that exploits two knowledge-sharing techniques, weight-initialization and imitation learning, to transfer a DRL control policy from a source office building to various target buildings in a simulation environment coupling EnergyPlus and Python. A DRL controller based on discrete Soft Actor–Critic (SAC) is trained on the source building to manage the operation of a cooling system consisting of a chiller and a thermal storage. Several target buildings are defined to benchmark the performance of the OTL strategy with that of a Rule-Based Controller (RBC) and two DRL-based control strategies, deployed in offline and online fashion. The strategy adopted for OTL emulates the real world implementation with a simulation process by implementing the transferred DRL agent for a single episode in the target buildings. Target buildings have the same geometrical features and are served by the same energy system as the source building, but differ in terms of weather conditions, electricity price schedules, occupancy patterns, and building envelope efficiency levels. The results show that the OTL strategy can reduce the cumulated sum of temperature violations on average by 50% and 80% respectively when compared to RBC and online DRL while enhancing the energy system operation with electricity cost savings ranging between 20% and 40%. Furthermore, the OTL agent performs slightly worse than the offline DRL controller but it does not require any modeling effort and can be implemented directly on target buildings emulating a real-world implementation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗