Search NASA⌕ Search

Engineering topics

Liu, Frank

Publications and source records attributed to Liu, Frank.

5 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Extreme Risk Mitigation in Reinforcement Learning using Extreme Value Theory

Risk-sensitive reinforcement learning (RL) has garnered significant attention in recent years due to the growing interest in deploying RL agents in real-world scenarios. A critical aspect of risk awareness involves modelling highly rare risk events (rewards) that could potentially lead to catastrophic outcomes. These infrequent occurrences present a formidable challenge for data-driven methods aiming to capture such risky events accurately. While risk-aware RL techniques do exist, they suffer from high variance estimation due to the inherent data scarcity. Our work proposes to enhance the resilience of RL agents when faced with very rare and risky events by focusing on refining the predictions of the extreme values predicted by the state-action value distribution. To achieve this, we formulate the extreme values of the state-action value function distribution as parameterized distributions, drawing inspiration from the principles of extreme value theory (EVT). We propose an extreme value theory based actor-critic approach, namely, Extreme Valued Actor-Critic (EVAC) which effectively addresses the issue of infrequent occurrence by leveraging EVT-based parameterization. Importantly, we theoretically demonstrate the advantages of employing these parameterized distributions in contrast to other risk-averse algorithms. Our evaluations show that the proposed method outperforms other risk averse RL algorithms on a diverse range of benchmark tasks, each encompassing distinct risk scenarios.

Wang, Yu↗
NASA NTRS2007

Lightweight, Flexible Solar Cells on Stainless Steel Foil and Polymer for Space and Stratospheric Applications

The availability of low-cost, lightweight and reliable photovoltaic (PV) modules is an important component in reducing the cost of satellites and spacecraft. In addition, future high-power spacecraft will require lightweight PV arrays with reduced stowage volume. In terms of the requirements for low mass, reduced stowage volume, and the harsh space environment, thin film amorphous silicon (a-Si) alloy cells have several advantages over other material technologies (1). The deposition process is relatively simple, inexpensive, and applicable to large area, lightweight, flexible substrates. The temperature coefficient has been found to be between -0.2 and -0.3 %/degC for high-efficiency triple-junction a-Si alloy cells, which is superior for high temperature operation compared to crystalline Si and triple-junction GaAs/InGaP/Ge devices at 0.53 %/degC and 0.45 %/degC, respectively (2). As a result, the reduction in efficiency at high temperature typical in space conditions is less for a-Si alloy cells than for their crystalline counterparts. Additionally, the a-Si alloy cells are relatively insensitive to electron and proton bombardment. We have shown that defects that are created by electrons with energies between 0.2 to 2 MeV with fluence up to 1x10(exp 15) e/sq cm and by protons with energy in the range 0.3 MeV to 5 MeV with fluence up to 1x10(exp 13) p/sq cm can be annealed out at 70 C in less than 50 hours (1). Further, modules incorporating United Solar s a-Si alloy cells have been tested on the MIR space station for 19 months with only minimal degradation (3). For stratospheric applications, such as the high altitude airship, the required PV arrays are typically of considerably higher power than current space arrays. Airships typically have a large area available for the PV, but weight is of critical importance. As a result, low cost and high specific power (W/kg) are key factors for airship PV arrays. Again, thin-film a-Si alloy solar cell technology is well suited to such applications.

Beernink, Kevin↗