Reinforcement Learning for the Adaptive Construction of Surrogate Models
Explore the source record for details and available documents.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Robotic missions to planetary surfaces are becoming more ambitious and of longer duration. The nominal mission timeline for the MER called Spirit currently on the Martian surface is 90 days, with extensions to 180 days depending on rover health. The upcoming 2009 MSL mission is planned to be 300-500 days and will possibly involve traverses on the order of a kilometer or more.
Explore the source record for details and available documents.
Data center cooling typically involves non-uniform airflow and temperature distributions, which are affected by the IT workload distribution. It is helpful to simulate the airflow and temperature to optimize the workload distribution. Traditional computational fluid dynamics (CFD) simulation is usually time-consuming while conventional reduced order models (ROMs), though computationally fast, may generate inaccurate results even after being fully trained. In Situ Adaptive Tabulation (ISAT), contracting to conventional ROM, can make prediction with error lower than a user-specified tolerance. To demonstrate using of ISAT for optimal workload distribution in data center, this paper presents a preliminary study of an ISAT-based genetic algorithm optimization platform. The ISAT is trained offline by using the results from CFD simulations using a hypothetical simple data center. The optimal workload distribution determined by the platform leads to approximately 6.8% of energy savings when compared to the benchmark with a uniform workload distribution. We note that the time cost for the entire optimization process, including the training of ISAT is about 4 hours, which is acceptable in the design phase.
Explore the source record for details and available documents.
Adaptive control systems - combined optimization and adaptive control, analysis-synthesis and passive adaptive systems, learning systems, and measurement adaptive systems
The Deep Lynx Machine Learning (ML) Adapter is a generic adapter that programmatically runs the ML as continuous data is received. Then, Jupyter Notebooks can be customized according to the project for pre-processing the data, building the machine learning models, prediction analysis of incoming data using an existing model, and forecasting anomalies / failures of the physical asset.
Abstract There are a large number of cameras in modern transportation system that capture numerous vehicle images continuously. Therefore, automatic analysis of these vehicle images is helpful for traffic flow management, criminal investigations and vehicle inspections. Vehicle matching, which aims to determine whether two input images depict an identical vehicle, is one of the core tasks in vehicle analysis. Recent relevant studies have focused on local feature extraction instead of global extraction, since local details can provide crucial cues to distinguish between cars. However, these methods do not select local features; that is, they do not assign weights to local features. Therefore, in this research, we systematically study the vehicle matching task, and present a novel annotation‐free local‐based deep learning method called Adaptive super‐pixel discriminative feature‐selective learning (ASDFL) to address this issue. In ASDFL, vehicle images are segmented into clusters of super‐pixels of similar size by considering the location and colour similarities of pixels without using any component‐level annotation. These super‐pixels are deemed to be the virtual components of vehicles. Moreover, a convolutional neural network is used to extract the deep features of these virtual components. Thereafter, an instance‐specific mask generation module driven by the extracted global features is enhanced to produce a mask to select the most distinctive virtual components of each vehicle image pair in the feature space. Finally, the vehicle matching task is accomplished by classifying the selected virtual component features of each imaged vehicle pair. Extensive experiments on two popular vehicle identification benchmarks demonstrate that our method is 1.57% and 0.8% more accurate than the previous baselines in a vehicle matching task on the VeRi and VehicleID datasets, respectively, which demonstrates the effectiveness of our method.
Hedrick, E., K. Reynolds, D. Bhattacharyya, S.E. Zitney, and B.P. Omell, "Adaptive Control Using Reinforcement Learning Augmented Model Predictive Control," Advanced Manufacturing and Processing Conference 2022, Bethesda, MD, June 1-3 (2022).
To address grid variability caused by renewable energy integration and to maintain grid reliability and resilience, hydropower must quickly adjust its power generation over short time periods. This changing energy generation landscape requires advance technology integration and adaptive parameter optimization for hydropower systems via digital twin effort. However, this is difficult owing to the lack of characterization and modeling for the nonlinear nature of hydroturbines. To solve this issue, this paper first formulates a six-coefficient Kaplan hydroturbine model and then proposes a parametric optimization tuning framework based on the Nelder–Mead algorithm for adaptive dynamic learning of the six-coefficients so as to build models that describe the turbine. To assess the performance of the proposed optimal parametric tuning technique, operational data from a real-world Kaplan hydroturbine unit are collected and used to model the relationship between the gate opening and the generated power production. The findings show that the proposed technique can effectively and adaptively learn the unknown dynamics of the Kaplan hydroturbine while optimally tune the unknown coefficients to match the generated power output from the real hydroturbine unit with an inaccuracy of less than 5%. The method can be used to provides optimal tuning of parameters critical for controller design, operational optimization and daily maintenance for hydroturbines in general.
Residual-based adaptive strategies are widely used in scientific machine learning yet remain largely heuristic. We introduce a variational framework that formalizes these methods through convex transformations of the residual, where different transformations correspond to distinct objective functionals. For instance, exponential weights target uniform error minimization, while linear weights recover quadratic error minimization. This perspective reveals adaptive weighting as a means of selecting sampling distributions that optimize a primal objective, directly linking discretization choices to error metrics. This principled approach yields three key benefits: it enables systematic design of adaptive schemes, reduces discretization error by lowering estimator variance, and enhances learning dynamics by improving gradient signal-to-noise ratio. Extending the framework to operator learning, we demonstrate substantial performance gains across diverse optimizers and architectures. Our results provide a theoretical perspective for residual-based adaptivity and establish a foundation for principled discretization and training.
Abstract One major challenge in water resource management is to balance the uncertain and nonstationary water demands and supplies caused by the changing anthropogenic and hydroclimate conditions. To address this issue, we developed a reinforcement learning agent‐based modeling (RL‐ABM) framework where agents (agriculture water users) are able to learn and adjust water demands based on their interactions with the water systems. The intelligent agents are created by a reinforcement learning algorithm adapted from the Q‐learning algorithm. We illustrated this framework in a case study where the RL‐ABM is two‐way coupled with the Colorado River Simulation System (CRSS), a long‐term planning model used for the administration of the Colorado River Basin, for assessing agriculture water uses impacts on water scarcity. Seventy‐eight intelligent agents are simulated, which can be grouped into three categories based on their parameter values: the “aggressive” (swift actions; low regrets), the “forward‐looking conservative” (mild actions; high regrets; fast learning), and the “myopic conservative” (mild actions; median regrets; slow learning). The ABM‐CRSS results showed that the major reservoirs in the Upper Colorado Basin might experience more frequent water shortages due to the increasing water uses compared to the original CRSS results. If the drought continues, the case study also demonstrates that agents can learn and adjust their demands.
An adaptive controller is developed for a regolith excavation robot to determine the mass of excavated material and to account for the effects of gravity and friction while on the surface of other celestial bodies. A data-based integral concurrent learning (ICL) parameter update law accounts for and estimates the unknown mass, gravity, and friction parameters. A Lyapunov-based analysis proves that the trajectory tracking error and the parameter estimate errors exponentially converge to zero. An estimation of the mass of regolith excavated by the robot is calculated from the estimated parameters. A simulation study is performed to show the performance of the developed technique. Simulation results show that for 3 kilograms of excavated material, the mass estimate has an error of 4.9 grams.
Deep learning models have been shown to outperform methods that rely on summary statistics, like the power spectrum, in extracting information from complex cosmological data sets. However, due to differences in the subgrid physics implementation and numerical approximations across different simulation suites, models trained on data from one cosmological simulation show a drop in performance when tested on another. Similarly, models trained on any of the simulations would also likely experience a drop in performance when applied to observational data. Training on data from two different suites of the CAMELS hydrodynamic cosmological simulations, we examine the generalization capabilities of Domain Adaptive Graph Neural Networks (DA-GNNs). By utilizing GNNs, we capitalize on their capacity to capture structured scale-free cosmological information from galaxy distributions. Moreover, by including unsupervised domain adaptation via Maximum Mean Discrepancy (MMD), we enable our models to extract domain-invariant features. We demonstrate that DA-GNN achieves higher accuracy and robustness on cross dataset tasks (up to 28% better relative error and up to almost an order of magnitude better χ 2 ). Using data visualizations, we show the effects of domain adaptation on proper latent space data alignment. This shows that DA-GNNs are a promising method for extracting domain-independent cosmological information, a vital step toward robust deep learning for real cosmic survey data.
We adapt the significant advances achieved recently in the field of generative artificial intelligence/machine-learning to laser performance modeling in multipass, high-energy laser systems with application to high-shot-rate facilities relevant to inertial fusion energy. Advantages of neural-network architectures include rapid prediction capability, data-driven processing, and the possibility to implement such architectures within future low-latency, low-power consumption photonic networks. Four models were investigated that differed in their generator loss functions and utilized the U-Net encoder/decoder architecture with either a reconstruction loss alone or combined with an adversarial network loss. We achieved inference times of 1.3 ms for a 256 × 256 pixel near-field beam with errors in predicted energy of the order of 1% over most of the energy range. It is shown that prediction errors are significantly reduced by ensemble averaging the models with different weight initializations. These results suggest that including the temporal dimension in such models may provide accurate, real-time spatiotemporal predictions of laser performance in high-shot-rate laser systems.
Deep learning (DL) models are being deployed at medical centers to aid radiologists for diagnosis of lung conditions from chest radiographs. Such models are often trained on a large volume of publicly available labeled radiographs. These pre-trained DL models’ ability to generalize in clinical settings is poor because of the changes in data distributions between publicly available and privately held radiographs. In chest radiographs, the heterogeneity in distributions arises from the diverse conditions in X-ray equipment and their configurations used for generating the images. In the machine learning community, the challenges posed by the heterogeneity in the data generation source is known as domain shift, which is a mode shift in the generative model. In this work, we introduce a domain-shift detection and removal method to overcome this problem. Our experimental results show the proposed method’s effectiveness in deploying a pre-trained DL model for abnormality detection in chest radiographs in a clinical setting.
Adaptive digitalized learning control system for launch vehicles