SEARCH · Search NASA

Results for “Sequentially optimized adaptation method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

23 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗

DOE OSTI2022

Machine learning with knowledge constraints for process optimization of open-air perovskite solar cell manufacturing

Perovskite photovoltaics (PV) have achieved rapid development in the past decade in terms of power conversion efficiency of small-area lab-scale devices; however, successful commercialization still requires further development of low-cost, scalable, and high-throughput manufacturing techniques. One of the critical challenges of developing a new fabrication technique is the high-dimensional parameter space for optimization, but machine learning (ML) can readily be used to accelerate perovskite PV scaling. Herein, we present an ML-guided framework of sequential learning for manufacturing process optimization. We apply our methodology to the Rapid Spray Plasma Processing (RSPP) technique for perovskite thin films in ambient conditions. With a limited experimental budget of screening 100 process conditions, we demonstrated an efficiency improvement to 18.5% as the best-in-our-lab device fabricated by RSPP, and we also experimentally found 10 unique process conditions to produce the top-performing devices of more than 17% efficiency, which is 5 times higher rate of success than the control experiments with pseudo-random Latin hypercube sampling. Our model is enabled by three innovations: (a) flexible knowledge transfer between experimental processes by incorporating data from prior experimental data as a probabilistic constraint; (b) incorporation of both subjective human observations and ML insights when selecting next experiments; (c) adaptive strategy of locating the region of interest using Bayesian optimization first, and then conducting local exploration for high-efficiency devices. Furthermore, in virtual benchmarking, our framework achieves faster improvements with limited experimental budgets than traditional design-of-experiments methods (e.g., one-variable-at-a-time sampling). This framework shows the capability of incorporating researchers’ domain knowledge into the ML-guided optimization loop; therefore, it has the potential to facilitate the wider adoption of ML in scaling to perovskite PV manufacturing.

14 SOLAR ENERGY↗

DOE OSTI2020

Using the Carbon Capture Simulation Initiative (CCSI) Tool to Design the Experiments in the Parametric Campaign of a Novel Compact Absorber for Carbon Capture

Gas absorption towers with structured packing and solvent have been used for Carbon Dioxide (CO 2 ) Capture for about many decades. To overcome process limitations and practical disadvantages for CO 2 capture from the stationary emitter (e.g. NG and coal power plant), many new designs have been proposed and explored at the various scales in the last decade with aim of either low energy penalty or low capital cost. To reduce the size of the absorption tower and hence the total cost of CO 2 capture, the University of Kentucky Center for Applied Energy Research Center (UK CAER) has designed and built a novel CO 2 capture absorption tower or Compact Absorber, integrated into an existing large-bench scale CO 2 capture unit. The Compact Absorber has three sections. The top of the column is a fogging section where the solvent is sprayed through a nozzle producing droplets flowing downward in a co-current fashion with the flue gas. The center of the column is a frothing section where the solvent and flue gas flow through regenerative frothing screens designed by Industrial Climate Solutions, Inc. The bottom of the column is a typical structured packing section were the flue gas and solvent flow in a counter-current fashion. The parametric campaign will be conducted in order to optimize the operating parameters for CO 2 capture including liquid/gas ratio, lean loading, and temperature, liquid residence time. A simulated flue gas with 14% CO 2 will be used along with a UK CAER developed proprietary solvent. The 100-hour parametric campaign is designed using a statistical approach of the Sequential Design of Experiments (sDOE). sDOE is one of the CCSI tools that provides an adaptive statistical approach for designing future experiments based on the results of previous experiments. Application of a typical DOE provides the user with the minimum number of experiments required to get the same data, but sDOE allows the user to make an informed choice of experiments based on the results of previous experiments. The complete absorption column has been constructed and has been partially commissioned. Initial data has been collected by operating using the fogging section and the frothing section. The fogging section produces solvent droplets of about 100 μm sauter mean diameter and as small as 25 μm using a hydraulic nozzle by BETE. The frothing section produces bubbles of about 5mm with high mixing of solvent promoting the higher mass transfer from gas to liquid. The absorber reaches the capture efficiency of about 50% with only two sections in operation. Based on the current results, it can be deduced that increasing the solvent feed temperature and including the packed section for absorption the capture efficiency will increase further. Initial data will be collected using all three sections of the absorber and will be used for sDOE. Non-Uniform Space Filling model of sDOE will be used to prioritize the input conditions resulting into maximum capture efficiency. sDOE is performed using the platform called Framework Optimization, Quantification of Uncertainty, and Surrogates (FOQUS). The method and results demonstrating the progress of the parametric campaign from the initial set of experiments to the final stage of obtaining optimized parameters using sDOE tool will be presented in detail.

20 FOSSIL-FUELED POWER PLANTS↗

DOE OSTI2025

Cardinal: Seismic and Geoacoustic Array Processing

Data collected via seismic and infrasound array deployments are leveraged in the geosciences to detect and characterize a myriad of natural and anthropogenic sources. These deployments consist of numerous sensors placed in a predetermined configuration to amplify signal strength and improve the efficacy of array processing techniques used to measure signal directionality and waveform coherence. High‐fidelity feature extraction is often predicated on interstation distance as well as the frequency content and wavelength of an incident signal. Numerous array processing softwares analyze data in sequential frequency bands to obtain a more detailed characterization of a signal. However, current algorithms are limited in their ability to determine optimal array configuration for each band. We introduce an open‐source Python code, called Cardinal, to process seismic and infrasound array data in discretized time–frequency space with the option of applying an adaptive array design to determine optimal subarray configuration for each frequency band. To reduce computational time, the array processing step can be run in parallel using multithreading. Furthermore, the software has the capability to aggregate array processing results from different time–frequency pixels to produce separate sets of detections, or families, with added utility via the application of an adaptive semblance threshold, which aids in isolating signals‐of‐interest from coherent background noise. Upon appropriate configuration, Cardinal exhibits the potential to combine distinct seismic and infrasound phases into separate families.

Adaptive Array↗

DOE OSTI2022

Reinforcement Learning for feedback-enabled cyber resilience

The rapid growth in the number of devices and their connectivity has enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain the critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an important gathering of algorithms that epitomize the feedback architectures for cyber resilience. It allows the CRM to provide dynamic and sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss the cyber-resilient defenses against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. Here we introduce moving target defense, defensive cyber deception, and assistive human security technologies as three application domains of CRMs to elaborate on their designs. The RL algorithms also have vulnerabilities themselves. We explain the major vulnerabilities of RL and present develop several attack models where the attacker target the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. The paper introduces several defense methods to secure the RL-enabled systems from these attacks. However, there is still a lack of works that focuses on the defensive mechanisms for RL-enabled systems. Last but not least, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.

97 MATHEMATICS AND COMPUTING↗