Search NASA⌕ Search

SEARCH · Search NASA

Results for “model-based testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Preliminary Development of Heat Transfer Model-Based Control Algorithms of Liquid Sodium Purification System: Advanced Sensors and Instrumentation Advanced Controls

Monitoring the operation of sodium purification system is essential for efficient operation of sodium fast reactors. In this work, a heat transfer model has been developed for monitoring the plugging meter and cold trap systems at the Mechanisms Engineering Test Loop (METL) liquid sodium facility at Argonne National Laboratory. The model of the purification system was developed by treating the respective aspects of the cold trap purification loop and plugging meter diagnostic loop as two separate control volumes using information from the METL piping and instrumentation diagram (P&ID). A model predictive controller was designed using first order differential equations with the specified boundary conditions. The system behavior was studied with a tuned optimized procedure using the internal cold trap temperature and plugging meter outlet temperature as control variables, and the air blower temperature as an independent variable respectively. Results of computer simulations obtained in this study compared favorably with experimental data showing very good reference tracking response with negligible overshoot as both plugging meter and cold trap physical models approach the setpoint.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Motion Planning Algorithms for Safety and Quantum Computing Efficiency

Motion planning remains a fundamental problem in robotics. Sampling-based algorithms use randomization to allow efficient solutions to this complex problem. As mobile robots and autonomous vehicles become more prevalent in everyday life, motion planning must be applied to increasingly challenging scenarios. Safety has become a paramount concern in motion planning for ensuring robotic applications enrich human lives. To date, many motion planning techniques to increase safety in the face of uncertain and dynamic environments have been developed. This dissertation first addresses distributional safety of Rapidly-Exploring Random Trees (RRT) through our algorithm W-Safe RRT. To acknowledge distributional uncertainty and poor modeling, W-Safe RRT uses the Wasserstein metric to provide a probabilistic bound on the distributional distance between a robot and obstacles. Human-interpretable environmental agent classification allows online safety margin adaptation. We propose and analyze an integrating region method for online classification that increases actor labeling accuracy based on behavioral feature values when compared to state of the art methods. The method performs class assignments based on local maximum likelihood in a created behavioral feature-space, allowing a notion of classification uncertainty. Model-based methods with safety guarantees can quickly become computationally in tractable, especially with multiple agents, higher dimensions, and plentiful unknowns. Sampling based algorithms have been parallelized for computation with multi-core computers and GPUs. We consider the use of quantum algorithms and computers for sampling-based motion planning for the first time. Quantum computing performs operations on superpositions of states and can solve certain problems much more efficiently than classical computers, but introduces previously unseen challenges. With Quantum-RRT, we recast the motion planning problem into a database-search structure and use Quantum Amplitude Amplification to find reachable states in the database with a quadratic performance increase over classical methods. We address two error sources with this method: quantum measurement and quantum oracle errors. We then extend this method to Parallel Quantum-RRT, which uses a manager-worker architecture with multiple parallel quantum workers to increase database search efficiency. We compare algorithm architectures and characterize probabilities of multiple workers finding solutions. Lastly, we test in simulation the quantum algorithms against classical versions in a wide variety of scenarios, concluding that a similar parallelization improvement is to be found in the quantum case as was found in the parallelization of classical RRT.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

AI-Enabled Discovery and Physics-Based Optimization of Energy Efficient Processing Strategies for Advanced Turbine Alloys (Final Technical Report)

In this project, the multi-organizational team of academic and industrial researchers from the University of Kentucky an aerospace and energy generation OEM partner has leveraged novel Digital Process Twin (DPT) models of process/structure interactions (i.e., process-induced surface integrity) to advance a paradigm of fully integrated computational materials engineering (ICME). Using efficient process models as the core of a digital process simulator for a reinforcement learning algorithm, the team has integrated industrial data and metrics of structure/performance/energy relationships and manufacturing-related energy metrics to optimize dynamic processing parameters for significantly improved life-cycle energy efficiency of advanced γ-TiAl low-pressure turbine (LPT) alloys, as indicated by a set of design relevant parameters (e.g., residual stresses and scrap rate). The key objective and anticipated outcome of the project was at least a 10% reduction in life-cycle embodied energy for a recently developed, γ-TiAl low-pressure turbine (LPT) alloy and nickel-based superalloy Inconel 718, through the adoption of the proposed AI-enabled process optimization approach. The final project outcomes significantly exceeded this original target, realizing manufacturing-related energy efficiency improvements of more than 130% for TiAl and up to 80% for Inconel 718. Rather than following the prevailing and highly inefficient empirical paradigm, the proposed study demonstrated the feasibility of adopting a digital, physics-based process design and optimization paradigm. The recurring need for manual intervention, rework, reinspection causes significant production bottlenecks and unnecessary expense associated with delivering the requisite component quality. The OEM partner, and turbine industry in general, expect to reap significant cost and resource savings if an AI-optimized set of parameters can be applied to specific machining operations. The technical scope of the proposed project involved the paving of a realistic path towards model-based and AI-enabled Integrated Computational Materials Engineering (ICME), and away from inefficient empirical process optimization and legacy manufacturing practices, which are no longer able to efficiently process novel high-performance turbine alloy materials. The project team will address the fundamental knowledge gap that currently exists within the ICME paradigm with respect to the process/structure/performance/energy impacts of finishing processes. While significant resources have been devoted to the ‘early stages’ of manufacturing, such as alloy design, primary and secondary processing, finishing processes have not been adequately integrated within ICME. To provide an actionable path towards model-based finishing process design (e.g., machining, burnishing, grinding, polishing), we will employ a novel AI-enabled process optimization paradigm, based on a computationally efficient, physics-based process simulator. Through limited experimental work to calibrate and validate our process simulator model via an advanced in-situ characterization technique and process optimization via reinforcement learning, the project will seek to demonstrate a viable alternative to the inefficient ‘legacy’ processing strategies, empirical testing and broad scope machining learning approaches, all of which fail to adequately consider complex process physics. The project team has identified an intermetallic γ-TiAl LPT alloy, which is currently being used as part of the OEM partner’s advanced gas turbine designs. This particular alloy poses significant manufacturing challenges during finishing operations, which limit the degree to which the current turbine design can be manufactured in an energy- and cost-efficient manner. Empirical testing and numerical modeling efforts to optimize processing parameters for γ-TiAl have not been able to resolve these manufacturing challenges, so the proposed physics-based AI-enabled optimization technology would offer a truly novel and transformative capability. The multi-organizational team of academic and industry experts from the UKY and the OEM partner will work together closely to demonstrate the analytical and experimental critical function and characteristic proof of concept of this novel approach.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2024

Multilevel Cybersecurity for Photovoltaic Systems

The motivation behind this project is to protect critical infrastructure in electric power generation pertaining to solar photovoltaic (PV) systems. This growing renewable energy resource is becoming a more vital part of the nation’s energy portfolio, particularly since it has achieved grid-parity to existing generation methods in terms of cost. It is thus vital that steps be taken to ensure the cybersecurity of these assets. The project goal was to devise a multilevel cybersecurity solution to address PV security gaps at the inverter and system levels, and field test the solution under the supervision and review of a US-based solar inverter manufacturer and PV installer/operator. A two-level cyberattack defense approach was formulated whereby the first level, the solar inverter level, hardens individual devices and achieves a deeply cyber-secure inverter. The inverter level security involves a multi-layer defense-in-depth approach for securing the inverter while also providing data for the system level algorithms. The second level, the system level, addresses intrusion detection and restoration involving an ensemble of inverters and relevant systems.

14 SOLAR ENERGY↗
DOE OSTI2026

Battery Life Prediction Using Reduced-Order Physics Models and Machine Learning (CRADA Final Report)

Phase 1 (Original CRADA, plus no-cost extension modifications #1-3, 6/1/2017 to 3/13/2021): The Australian Department of Defence (AUDoD) is performing accelerated aging tests of Li-ion batteries to benchmark their reliability and degradation characteristics. Using its previously developed battery lifetime predictive model framework, the National Laboratory of the Rockies (NLR) will develop analytical models based the AUDoD data to predict lifetime of the multiple Li-ion battery chemistries under real-world use scenarios of interest to AUDoD. The NLR model is based on physical degradation mechanisms encountered by Li-ion batteries and has been previously validated. Phase 2 (CRADA modification #4, plus no-cost extension modification #5, 2/22/2021 to 3/30/2025): Train and support Australian Department of Defence personnel to use NLR software for model-based estimation of Li-ion battery lifetime using accelerated battery aging data collected by the Australian Department of Defence. Under separate DOE funding from 2019 to 2021, NLR enhanced its battery life-prediction software using machine learning algorithms to automate portions of the model-fitting process, requiring significantly less labor and expert judgment and also adding uncertainty quantification, increasing statistical rigor. Under Phase 2, NLR will customize NLR Software and provide it to AuDoD. NLR will enhance its NLR Model to capture aging modes of AuDoD's multi-cell modules, including cell-balancing effects. NLR will develop example single-cell and multi-cell models based on one AuDoD battery aging dataset. NLR will train AuDoD personnel on NLR Software. By the conclusion of the project, NLR will have provided AuDoD the training materials, a user manual and software needed to perform their own analysis of additional and/or future battery aging datasets.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2026

Captan+X Data Converter Integration

Fermi National Accelerator Laboratory's CAPTAN (Compact And Programmable daTa Acquisition Node) series provides a flexible hardware platform for data acquisition across a range of experiments and facilities. The latest iteration, CAPTAN+X, is built around a Kintex-7 FPGA supporting four FPGA Mezzanine Card (FMC) connections. As part of a broader laboratory effort to bring facility systems under a Model-Based Systems Engineering (MBSE) framework, CAPTAN+X is one of several systems slated to be incorporated into this modeling environment in the near term. A necessary step toward that goal is incorporating the platform's core functionality, which centers on integration with the LXD31K4 FMC, a data converter module combining dual AD9652 analog-to-digital converters and dual AD9142A digital-to-analog converters. Achieving compatibility required resolving pin-mapping conflicts between the LXD31K4's High Pin Count connector and the CAPTAN+X's available pin types, adapting a Board Support Project originally written for an UltraScale-class evaluation board to the Kintex-7 architecture, replacing incompatible primitives, restructuring clock distribution, and manually configuring chip initialization in place of an unsupported soft-processor-based approach. Functional verification of the ADC and DAC channels, followed by closed-loop testing combining both converters with real-time filtering, confirmed correct operation of the integrated system. These results establish a working hardware and firmware baseline for the CAPTAN+X platform, positioning it for future inclusion in the laboratory's growing MBSE modeling effort.

Espinoza, David [Illinois U., Urbana (main)]↗
DOE OSTI2022

Control of a Three-Phase Grid-Connected Voltage-Sourced Converter Using Long Short-Term Memory Networks

With the rise of inverter-based resources (IBRs) within the power system, the control of grid-connected converters (GCCs) has become pertinent due to the fact they interface IBRs to the grid. The conventional method of control for a GCC such as the voltage-sourced converter (VSC) is through a decoupled control loop in the synchronous reference frame. However, this model-based control method is sensitive to parameter changes causing deterioration in controller performance. Data-driven approaches such as machine learning can be utilized to design controllers that are capable of operating GCCs in various system conditions. This work explores a deep learning-based control method for a three-phase grid-connected VSC, specifically utilizing a long short-term memory (LSTM) network for robust control. Simulations of a conventional controlled VSC are conducted using Simulink to collect data for training the LSTM-based controller. The LSTM model is built and trained using the Keras and TensorFlow libraries in Python and tested in Simulink. The performance of the LSTM-based controller is evaluated under different case studies and compared to the conventional method of control. Simulation results demonstrate the effectiveness of this approach by outperforming the conventional controller and maintaining stability under different system parameter changes.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Experimental Comparison of Elastomeric Materials for Hydraulic Seal Durability under Reciprocating Conditions

Wave Energy Converters (WECs) rely on hydraulic Power Take-Off (PTO) systems to transform the kinetic energy of ocean waves into electricity, where hydraulic seals play a critical role in preventing fluid leakage and ensuring watertight operation under harsh marine conditions. However, seal degradation due to wear, fatigue, and corrosion remains a key challenge limiting system reliability and increasing maintenance costs. In this study, the tribological performance of elastomeric materials for hydraulic sealing applications that are based on commercial polyurethane and custom-compounded Ethylene Propylene Diene Monomer (EPDM) is investigated. Using a CSM tribometer, reciprocating wear tests were conducted in accordance with ASTM G133 under dry sliding conditions, with normal loads ranging from 3 to 10 N, and sliding speeds of 10 to 30 mm/s. Coefficient of friction (COF), specific wear rate, and worn surface morphology are evaluated to understand how load, velocity, and material composition affect seal durability. Improved wear stability and high Shore A hardness (93–94) were demonstrated by the commercial thermoplastic polyurethane (Duralast 4203 and 4758). EPDM-based compounds showed different filler-dependent behaviors, formulations filled with carbon black (E3) had the lowest wear rate (1.45 × 10?4 mm³ N?¹ m?¹) and increased load-bearing capacity, while samples modified with silica and plasticizer (E4, E5) had better conformability and damping but less hardness. Compounded EPDMs showed wider deformation zones because of their lower modulus and filler-matrix interfacial fluctuations, while commercial grades exhibited shallower wear scars (= 380 µm), according to optical profilometry. The Archard model-based lifetime estimation showed that, under 10 N and 30 mm/s, operational endurance ranged from 3.1 to 6.2 years. Because of its low wear rate and steady interfacial behavior, E3 had the longest anticipated life. For medium-load hydraulic sealing applications, these results demonstrate the possibility of customized EPDM formulations as affordable substitutes for commercial polyurethanes. They also offer a quantitative foundation for material selection and lifespan prediction.

Durability↗
DOE OSTI2021

Comparison of Deterministic and Statistical Models for Water Quality Compliance Forecasting in the San Joaquin River Basin, California

Model selection for water quality forecasting depends on many factors including analyst expertise and cost, stakeholder involvement and expected performance. Water quality forecasting in arid river basins is especially challenging given the importance of protecting beneficial uses in these environments and the livelihood of agricultural communities. In the agriculture-dominated San Joaquin River Basin of California, real-time salinity management (RTSM) is a state-sanctioned program that helps to maximize allowable salt export while protecting existing basin beneficial uses of water supply. The RTSM strategy supplants the federal total maximum daily load (TMDL) approach that could impose fines associated with exceedances of monthly and annual salt load allocations of up to $1 million per year based on average year hydrology and salt load export limits. The essential components of the current program include the establishment of telemetered sensor networks, a web-based information system for sharing data, a basin-scale salt load assimilative capacity forecasting model and institutional entities tasked with performing weekly forecasts of river salt assimilative capacity and scheduling west-side drainage export of salt loads. Web-based information portals have been developed to share model input data and salt assimilative capacity forecasts together with increasing stakeholder awareness and involvement in water quality resource management activities in the river basin. Two modeling approaches have been developed simultaneously. The first relies on a statistical analysis of the relationship between flow and salt concentration at three compliance monitoring sites and the use of these regression relationships for forecasting. The second salt load forecasting approach is a customized application of the Watershed Analysis Risk Management Framework (WARMF), a watershed water quality simulation model that has been configured to estimate daily river salt assimilative capacity and to provide decision support for real-time salinity management at the watershed level. Analysis of the results from both model-based forecasting approaches over a period of five years shows that the regression-based forecasting model, run daily Monday to Friday each week, provided marginally better performance. However, the regression-based forecasting model assumes the same general relationship between flow and salinity which breaks down during extreme weather events such as droughts when water allocation cutbacks among stakeholders are not evenly distributed across the basin. A recent test case shows the utility of both models in dealing with an exceedance event at one compliance monitoring site recently introduced in 2020.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Innovating the next generation of commercial smart building software

Nearly 30% of commercial building energy use is wasted due to equipment faults and HVAC controls problems. The result is increased emissions, compromised comfort and productivity, and less reliable coordination of building power needs with a clean grid. The energy impact alone represents $17 billion in potential savings. Today’s smart building software provides a robust solution to address these operational deficiencies. Energy management and information systems (EMIS) are saving up to 9% on average, with two-year paybacks. They are being incorporated into energy management processes, commissioning services, and utility programs. As effective as they are, two barriers prevent even deeper benefits; limited personnel to fix problems once they are identified, and the expense and time to manually implement changes in control systems. In partnership with the research community, the EMIS industry is developing new capabilities to overcome these barriers. Moving beyond siloed products for either fault detection and diagnostics, or optimal control, these new capabilities empower users to not only automatically identify faults, but also to push corrective action, and control improvements to their buildings. In this paper, several areas for enhancements are documented: ‘one-time’ correction of faults such as setpoints, schedules, and economizer lockouts; short-term active testing for automated proportional integral derivative (PID) loop tuning and functional testing; and continuous supervisory control for demand flexibility and year-round efficiency. Results are presented from a pair of partner implementations out of a dozen providers integrating these enhancements into their products, including field tests from across the country, and insights into operator acceptance and integration into operations and maintenance practices.

Casillas, Armando↗
DOE OSTI2020

Machine Learning-Based PV Reserve Determination Strategy for Frequency Control on the WECC System: Preprint

Frequency control from Photovoltaic (PV) plants has great potential to address the frequency response challenge of the power system with high renewable penetration. However, using model-based approaches to determine the optimal PV headroom reserve requires significant online computation and is intractable for an interconnection level system. This paper proposes a machine learning based strategy, that is suitable for real-time operation, to determine the optimal PV reserve for frequency control. The proposed machine learning algorithm is trained and tested on 1,987 offline simulations of a 60% renewable penetration Western Electricity Coordinating Council (WECC) system. Furthermore, the proposed reserve determination strategy is applied on a realistic one-day operation profile of the WECC system and demonstrates over 40% PV headroom saving compared to a conservative approach. It is evident that the proposed strategy can efficiently and effectively determine the optimal PV frequency control reserve for realistic interconnection systems.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2020

DECENTRALIZED APPROACH TO MULTI-ZONE GREY-BOX MODELING FOR MODEL-BASED PREDICTIVE CONTROL

This study aims to improve the easiness of utilizing the grey-box model (i.e., Resistance-Capacitance circuit) for Model-based Predictive Control (MPC). The primary barrier of implementing the MPC is estimating the control-oriented building model that needs to be computationally inexpensive and quick but reasonably precise in predicting building load and indoor conditions. The estimating of the model parameters becomes more complicated when the building scale is larger; e.g., multi-zone building. In this study, a decentralized approach is introduced; each zone is split and individually estimated with measured boundary temperature from adjacent zones integrated into one single system model. The proposed decentralized method is demonstrated with experimental data from a full-scale multi-zone test cell compared with the centralized reference case.

Joe, Jaewan↗
DOE OSTI2020

Model-Agnostic Algorithm for Real-Time Attack Identification in Power Grid using Koopman Modes

Malicious activities on measurements from sensors like Phasor Measurement Units (PMUs) can mislead the control center operator into taking wrong control actions resulting in disruption of operation, financial losses, and equipment damage. In particular, false data attacks initiated during power systems transients caused due to abrupt changes in load and generation can fool the conventional model-based detection methods relying on thresholds comparison to trigger an anomaly. In this paper, we propose a Koopman mode decomposition (KMD) based algorithm to detect and identify false data attacks in real-time. The Koopman modes (KMs) are capable of capturing the nonlinear modes of oscillation in the transient dynamics of the power networks and reveal the spatial embedding of both natural and anomalous modes of oscillations in the sensor measurements. The Koopman-based spatio-temporal nonlinear modal analysis is used to filter out the false data injected by an attacker. The performance of the algorithm is illustrated on the IEEE 68-bus test system using synthetic attack scenarios generated on GridSTAGE, a recently developed multivariate spatio-temporal data generation framework for simulation of adversarial scenarios in cyber-physical power systems.

Nandanoori, Sai Pushpak↗
DOE OSTI2022

Model-based Co-Simulation of Heat Pump Water Heater with Phase Change Materials Thermal Energy Storage

The study analysis the behavior of a new developed heat pump water heater technology which integrates a phase change materials storage with a standard heat pump water heater to maximize the performance parameters of the Unified Energy Factor (UEF) and First Hour Rating (FHR). A model-based control development co-simulation platform is developed to include equipment models, such as heat pump, standard water tank, phase change materials storage tank, and integrate them with control model to support controls design, analysis, verification, and validation. Simulation results are compared with lab test results to validate the accuracy of developed co-simulation platform

Sun, Jian↗
DOE OSTI2023

Transient Efficiency, Flexibility, and Reliability Optimization of Coal-Fired Power Plants - Final Program Review

This program developed an advanced model-based monitoring and model-predictive control algorithms for a coal fired power plant (CFPP), and deployed these algorithms in a real-time platform to demonstrate performance benefits for transient flexibility and plant operation efficiency. More specifically, the objectives were successfully achieved through a combination of (i) developing a high-fidelity transient plant model in Apros, which was used as a high-fidelity plant simulation between $100-50\% TMCR$ where TMCR denotes the turbine maximum continuous rating, i.e., baseload, (ii) developing a very fast physics-based reduced-order model (ROM) of the plant, which ran more than $100\times$ faster than real-time, enabling its use as real-time embedded model for model-based estimation (MBE) and model predictive control (MPC) (iii) implementing a real-time MBE based on ROM using a robust unscented Kalman filter (UKF) to continuously tune the ROM to match the measurements from high-fidelity Apros plant model despite significant plant-model mismatch, and thus, obtain a Digital Twin of the plant (iv) designing and implementing a real-time MPC with dual objectives of transient plant load tracking with high ramp rates and minimizing coal consumption, i.e., improving plant efficiency in the baseload-partload operation range of $100-50\% TMCR$. Each key element above was developed and tested individually, and has been reported in corresponding Topical Reports. Finally, all the individual elements were integrated in an overall closed-loop system, that was successfully tested in desktop Simulink test harness simulations with ROM or high-fidelity model as the plant. Thereafter, the Simulink implementation was used to auto-generate C-code and deploy as real-time Docker microservice containers in Linux, and validate that they can run in real-time in the hardware-in-the loop (HIL) setup and produce the same results as in Simulink. The results of the integrated simulation tests in Simulink as well as the real-time HIL deployment are documented in this final report, showing good load tracking for load ramps at $3-4\%/min$ ramp rates, and achieving up to $5.5\%$ reduction in coal relative to baseline operation at $50\% TMCR$ load. The desktop and HIL simulations show successful performance of the overall model based estimation and control solution and achieve the key objectives of the program for flexible, efficient and reliable operation of sub-critical coal fired power plants.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2024

BIM Interoperability Tool for Improving IFC-based Model Data Exchange Between Architectural Design and Structural Analysis for Linear Piping Components

There is a growing need in the AEC industry for the digitalization of model-based data exchange in BIM workflows. However, users continue to face difficulties exchanging data between BIM-based computer-aided design (CAD) and computer-aided engineering (CAE) software, even with the open, non-proprietary data exchange format called Industry Foundation Classes (IFC). Proposed solutions in academic research focus primarily on building systems, with comparatively little attention to piping models. Therefore, this paper introduces an interoperability tool for enabling piping model data exchange between architectural and structural analysis domains. The tool is tested using a piping model created in Autodesk Revit and shows marked improvement for IFC-based CAD-to-CAE interoperability over existing practice.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗