Search NASA⌕ Search

SEARCH · Search NASA

Results for “online learning algorithm”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

A Learning-based Supervisory Control Architecture for Electric Vehicle Charging System Paired with Energy Storages

A Machine Learning-based predictive model is developed for optimal dispatching energy storage system integrated with Electric Vehicle battery charger. The model is effectively trained using transfer learning algorithm and successfully validated via measurement data to achieve high fidelity and accuracy. The study findings provide possibilities for future development of the presented approach to implement computational heavy predictive algorithms with multi-step predictions in emerging modular bidirectional charger topologies to realize vehicle-to-grid (V2G) technology. The online tuning can be explored using the new measurements obtained during the operation, henceforth, enhancing the controller performance for integrating multiple DERs into distribution power networks that shows huge interest for the utilities and regulatory agencies. Multiple operation scenatios have been evaluated and simulation results are discussed to verify intended performance of the proposed control solution.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Distributed Energy Management for Networked Microgrids Using Online ADMM With Regret

We propose a distributed algorithm for online energy management in networked microgrids with a high penetration of distributed energy resources (DERs). A high penetration of DERs introduces high uncertainty of power generation to the microgrids. In general, the state-of-the-art forecasting for non-dispatchable DERs such as solar energy is not sufficiently accurate, which results in inaccurate energy scheduling. To address the high uncertainty issue in the networked microgrids, we propose an online energy management based on the online alternating direction method of multipliers algorithm with the past power generation information from the DERs. The online algorithm provides less conservative schedule than the robust optimization-based approach. The effectiveness of the proposed algorithm is verified by various numerical examples.

alternating direction method of multipliers↗

Contextual Active Online Model Selection with Expert Advice

How can we collect the most useful labels to learn a model selection policy, when presented with arbitrary heterogeneous data streams? In this paper, we formulate this task as a contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. In particular, we focus on the task of selecting pre-trained classifiers, and propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection. In comparison to prior art, our algorithm does not assume a globally optimal model. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Our experiments on several benchmark classification datasets demonstrate the algorithm’s effectiveness in terms of both regret and query complexity. Notably, to achieve the same accuracy, CAMS incurs less than 10% of the label cost when compared to the best online model selection baselines on CIFAR10.

Liu, Xuefeng↗

A Predictive Prescription Framework for Stochastic Unit Commitment Using Boosting Ensemble Learning Algorithms

To take unit commitment (UC) decisions under uncertain load, most existing stochastic optimization (SO) frameworks adopt a generic representation of uncertainty. While load levels that materialize on a particular day are influenced by various covariates (such as the day of the week or temperature), SO frameworks typically disregard such side observations, wasting actionable information that could significantly enhance decision quality. Here, this article proposes a contextual SO (CSO) framework for UC under uncertain load, which can effectively exploit covariate observations in conjunction with a class of machine learning (ML) algorithms to improve the out-of-sample performance of UC decisions. It shows how three ML algorithms, adaptive boosting, gradient boosted trees, and extreme gradient boosting, can be used to this end, constituting the first application of these algorithms in any CSO framework. Using real-world data harvested from the New York ISO grid, we measure the out-of-sample performance of the framework in terms of total operation cost, shed load values, locational marginal prices, and total payments by the loads, against several benchmark methods proposed in the literature. The article has an online companion (Yurdakul et al.), wherein we present additional results and lay out further mathematical formulations used in this work.

42 ENGINEERING↗

Data Driven Online Learning of Power System Dynamics

With the advancement of sensing and communication in power networks, high-frequency real-time data from a power network can be used as a resource to develop better monitoring capabilities. In this work, a systematic approach based on data-driven operator theoretic methods involving Koopman operator is proposed for the online identification of power system dynamics. In particular, a new algorithm is provided, which unlike any previously existing algorithms, updates the Koopman operator iteratively as new data points are acquired. The proposed algorithm has three advantages: a) allows for real-time monitoring of the power system dynamics b) linear power system dynamics (this linear system is usually in a higher dimensional feature space and is not same as linearization of the underlying nonlinear dynamics) and c) computationally fast and less intensive when compared to the popular Extended Dynamic Mode Decomposition (EDMD) algorithm. The efficiency of the proposed algorithm is illustrated on an IEEE 9 bus system using synthetic data from the nonlinear model and on IEEE 39 bus system using synthetic data from the linearized model.

Sinha, Subhrajit↗

Data-Driven Robust M-LS-SVR-Based NARX Modeling for Estimation and Control of Molten Iron Quality Indices in Blast Furnace Ironmaking

Optimal operation of an industrial blast furnace (BF) ironmaking process largely depends on a reliable measurement of molten iron quality (MIQ), which is not feasible using the conventional sensors. This paper proposes a novel data-driven robust modeling method for online estimation and control of MIQ indices. First, to completely capture the nonlinear dynamics of the BF process, the nonlinear autoregressive exogenous (NARX) model is constructed for the MIQ indices. Then, considering that the standard least-squares support vector regression (LS-SVR) cannot directly cope with the multi-output problem, the multi- task transfer learning is proposed to design a novel multi-output LS-SVR (M-LS-SVR) for learning of the NARX model. Furthermore, a novel M-estimator is proposed to reduce the interference of outliers and improve the robustness of the M-LS- SVR model. Since the weights of different outlier data are properly given by the weight function, their corresponding contributions on modeling can be properly distinguished, thus a robust modeling result can be achieved. Finally, a novel multi- objective evaluation index on modeling performance is developed by comprehensively considering the root mean square error of modeling and the correlation coefficient on trend fitting, based on which the NSGA-II algorithm is used to globally optimize the model parameters. Both experiments using industrial data and industrial applications illustrate that the proposed method can eliminate the adverse effect caused by the fluctuation of data in BF process efficiently, which indicates its stronger robustness and higher accuracy. Moreover, control testing shows that the developed model can be well applied to realize data-driven control of the BF process.

blast cavities, Control↗

Attention Enabled Multi-Agent DRL for Decentralized Volt-VAR Control of Active Distribution System Using PV Inverters and SVCs

This paper proposes attention enabled multi-agent deep reinforcement learning (MADRL) framework for active distribution network decentralized Volt-VAR control. Using the unsupervised clustering, the whole distribution system can be decomposed into several sub-networks according to the voltage and reactive power sensitivity relationships. Then, the distributed control problem of each sub-network is modeled as Markov games and solved by the improved MADRL algorithm, where each sub-network is modeled as an adaptive agent. An attention mechanism is developed to help each agent focus on specific information that is mostly related to the reward. All agents are centrally trained offline to learn the optimal coordinated Volt-VAR control strategy and executed in a decentralized manner to make online decisions with only local information. Compared with other distributed control approaches, the proposed method can effectively deal with uncertainties, achieve fast decision makings, and significantly reduce the communication requirements. Comparison results with model-based and other data-driven methods on IEEE 33-bus and 123-bus systems demonstrate the benefits of the proposed approach.

distribution network↗

High-bandwidth image-based predictive laser stabilization via optimized Fourier filters

Controlling the delivery of kHz-class pulsed lasers is of interest in a variety of industrial and scientific applications, from next-generation laser-plasma acceleration to laser-based x-ray emission and high-precision manufacturing. The transverse position of the laser pulse train on the application target is often subject to fluctuations by external drivers (e.g., room cooling and heating systems, motorized optics stages and mounts, vacuum systems, chillers, and/or ground vibrations). For typical situations where the disturbance spectrum exhibits discrete peaks on top of a broad-bandwidth lower-frequency background, traditional PID (proportional-integral-derivative) controllers may struggle, since as a general rule PID controllers can be used to suppress vibrations up to only about 5%–10% of the sampling frequency. Here, a predictive feed-forward algorithm is presented that significantly enhances the stabilization bandwidth in such laser systems (up to the Nyquist limit at half the sampling frequency) by online identification and filtering of one or a few discrete frequencies using optimized Fourier filters. Furthermore, the system architecture demonstrated here uses off-the-shelf CMOS cameras and piezo-electric actuated mirrors connected to a standard PC to process the alignment images and implement the algorithm. To avoid high-end, high-cost components, a machine-learning-based model of the piezo mirror’s dynamics was integrated into the system, which enables high-precision positioning by compensating for hysteresis and other hardware-induced effects. A successful demonstration of the method was performed on a 1 kHz laser pulse train, where externally-induced vibrations of up to 400 Hz were attenuated by a factor of five, far exceeding what could be done with a standard PID scheme.

Natal, Joseph↗

Off-policy deep reinforcement learning with automatic entropy adjustment for adaptive online grid emergency control

Electric overloading conditions and contingencies put modern power systems at risk of voltage collapse and blackouts. Load shedding is crucial to maintain voltage stability for grid emergency control. However, the rule- or model-based schemes rely on accurate dynamic system models and face considerable challenges in adapting to various operating conditions and uncertain event occurrences. Here, to address these issues, this paper proposes a novel deep reinforcement learning (DRL)-based voltage stability control algorithm with automatic entropy adjustment (AEA) for grid emergency control. Various dynamic network components for complex system operations are modeled to construct the DRL environment. An off-policy soft actor-critic architecture is developed to maximize the expected reward and policy entropy simultaneously. The AEA mechanism is proposed to facilitate the policy maximum entropy procedure, and the proposed method can automatically provide effective discrete and continuous actions against various fault scenarios. Our approach accomplishes high sampling efficiency, scalability, and auto-adaptivity of the control policies under high uncertainties. Comparative studies with the existing DRL-based control methods in IEEE benchmarks indicate salient performance improvement of the proposed method for dynamic system emergency control.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Methodologies for Adaptive Flight Envelope Estimation and Protection

This paper reports the latest development of several techniques for adaptive flight envelope estimation and protection system for aircraft under damage upset conditions. Through the integration of advanced fault detection algorithms, real-time system identification of the damage/faulted aircraft and flight envelop estimation, real-time decision support can be executed autonomously for improving damage tolerance and flight recoverability. Particularly, a bank of adaptive nonlinear fault detection and isolation estimators were developed for flight control actuator faults; a real-time system identification method was developed for assessing the dynamics and performance limitation of impaired aircraft; online learning neural networks were used to approximate selected aircraft dynamics which were then inverted to estimate command margins. As off-line training of network weights is not required, the method has the advantage of adapting to varying flight conditions and different vehicle configurations. The key benefit of the envelope estimation and protection system is that it allows the aircraft to fly close to its limit boundary by constantly updating the controller command limits during flight. The developed techniques were demonstrated on NASA s Generic Transport Model (GTM) simulation environments with simulated actuator faults. Simulation results and remarks on future work are presented.

Tang, Liang↗

Hybrid Analytics Solution to Improve Coal Power Plant Operations

This project focused on developing advanced methods for thermal performance monitoring of a coal-fueled power plant. The specific goal was to develop and demonstrate a new thermal performance monitoring approach using a hybrid model that integrates a physics-based heat balance model with a machine learning-based pattern recognition model. The hybrid model enables increased accuracy and scope of the thermal analysis and an improved ability to monitor and detect changes in plant operation. This new approach takes full advantage of the individual model capabilities and creates an important new set of capabilities not previously possible using the two types of models separately. Using the heat balance model, a rich set of derived parameters (virtual sensors) are calculated from the measured plant operating data at each time point. The combined measured and derived data values are used by machine learning algorithms to create pattern recognition models over the range of normal unit operation. To create the monitoring models, historical data from normal operation of the plant is first processed by the heat balance model to compute the derived parameter data. The result is a greatly expanded set of normal operating data that can be used as input to create the pattern recognition model. Once the models are calibrated for normal operation, the hybrid model is suitable for use in continuous online monitoring. During online monitoring, new plant operating data is processed first by the heat balance model and then by the pattern recognition model. Results from the pattern recognition model quantify the deviation of each measured or derived parameter from its expected value in normal operation. The hybrid models can detect abnormal changes in plant operating data with very high accuracy and sensitivity. When abnormal behavior is detected, alerts are generated automatically for evaluation by the plant monitoring staff. The new hybrid solution product was developed and verified in the performance of the project. The hybrid solution was tested first in a simulation environment that mimicked the plant data systems and infrastructure used by U.S. power generating plants and utilities. The hybrid solution was then deployed for real-time, online monitoring of an operating coal-fueled power plant at a field test site. Field testing demonstrated that all hybrid solution development objectives were accomplished. The project work was based on combining the capabilities of two existing software products to create the new hybrid solution product. One of these was the existing MapEx® heat balance product and the other was the existing SureSense® advanced pattern recognition product. Each of these separate products was assessed to be at a Technology Readiness Level (TRL) of 9 at the start of the effort. The hybrid solution product was assessed to be at a TRL of 2 at the start of the project based on early feasibility work by the project team. At completion of the field testing performed in the project, the hybrid solution product was assessed to be at a TRL of 7. The project team expects that the hybrid solution product will be deployed commercially and will achieve a TRL of 9 within one year after completion of the project.

01 COAL, LIGNITE, AND PEAT↗

Robust Semantic Mapping and Localization on a Free-Flying Robot in Microgravity

We propose a system that uses semantic object detections to localize a microgravity free-flyer. Many applications require absolute localization in a known reference frame, such as the execution of waypoint trajectories defined by human operators. Classical geometric methods build a map of point features, which may not be able to be associated after lighting or environmental changes. By contrast, semantics remain invariant to changes up to the robustness of the detection algorithm and motion of the semantic objects. In this work, we describe our approaches for both offline semantic map generation as well as online localization against a semantic map, intended to run in real-time on the robot. We additionally demonstrate how our semantic localizer outperforms image-feature matching in some cases, and show the robustness of the algorithm to environmental changes. Crucially, we show in our experiments that when semantics are used to supplement point features, localization is always improved. To our knowledge, these experiments demonstrate the first use of learned semantics for localization on a free-flying robot in microgravity.

Localization↗

Evaluating Deception Detection Model Robustness To Linguistic Variation

With the increasing use of automated, machine learning-driven tools and the downstream impact that algorithmic judgements can have, it is critical to develop models that are robust to evolving or manipulated inputs. Evaluating the reliability of multimodal models across linguistic variations to understand model susceptibility to intentional linguistic adversarial attacks as well as natural linguistic variations is essential in this pursuit. We present extensive analysis of model robustness and susceptibility to linguistic variations in the setting of deceptive news detection, a difficult classification task that is an increasingly important problem to solve with the impact of misinformation spread online. We evaluate the effectiveness of incorporating adversarial defense strategies and measure model susceptibility to state-of-the-art adversarial attacks using two types of linguistic attacks — character and word perturbations. We consider two multiclass prediction tasks — a 3-way classification of tweets as trustworthy, propaganda, or disinformation; and a 4-way classification as clickbait, hoax, satire, or conspiracy — and compare the performance of three embeddings that have been state-of-the-art for several NLP tasks — GloVe, ELMo, and BERT — to highlight consistent trends in susceptibility, high confidence misclassifications, and high impact failures. We find that character or mixed ensemble models are the most effective defense mechanisms and that character perturbations are a more effective attack than word perturbations for deception classification.

adversarial evaluation↗

Intrinsic synaptic plasticity of ferroelectric field effect transistors for online learning

Nanoelectronic devices emulating neuro-synaptic functionalities through their intrinsic physics at low operating energies is imperative toward the realization of brain-like neuromorphic computers. In this work, we leverage the non-linear voltage dependent partial polarization switching of a ferroelectric field effect transistor to mimic plasticity characteristics of biological synapses. Here, we provide experimental measurements of the synaptic characteristics for a 28nm high-k metal gate technology based device and develop an experimentally calibrated device model for large-scale system performance prediction. Decoupled read-write paths, ultra-low programming energies and the possibility of arranging such devices in a cross-point architecture demonstrate the synaptic efficacy of the device. Our hardware-algorithm co-design analysis reveals that the intrinsic plasticity of the ferroelectric devices has potential to enable unsupervised local learning in edge devices with limited training data

42 ENGINEERING↗

The HEP.TrkX Project: deep neural networks for HL-LHC online and offline tracking

Particle track reconstruction in dense environments such as the detectors of the High Luminosity Large Hadron Collider (HL-LHC) is a challenging pattern recognition problem. Traditional tracking algorithms such as the combinatorial Kalman Filter have been used with great success in LHC experiments for years. However, these state-of-the-art techniques are inherently sequential and scale poorly with the expected increases in detector occupancy in the HL-LHC conditions. The HEP.TrkX project is a pilot project with the aim to identify and develop cross-experiment solutions based on machine learning algorithms for track reconstruction. Machine learning algorithms bring a lot of potential to this problem thanks to their capability to model complex non-linear data dependencies, to learn effective representations of high-dimensional data through training, and to parallelize easily on high-throughput architectures such as GPUs. This contribution will describe our initial explorations into this relatively unexplored idea space. Furthermore, we will discuss the use of recurrent (LSTM) and convolutional neural networks to find and fit tracks in toy detector data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Risk-Aware Reinforcement Learning Framework for User-Centric O-RAN

The evolution of Open Radio Access Networks (O-RAN) presents an opportunity to enhance network performance by enabling dynamic orchestration of configuration and optimization parameters (COPs) through online learning methods. However, leveraging this potential requires overcoming the limitations of traditional cell-centric RAN architectures, which lack the necessary flexibility. On the other hand, despite their recent popularity, the practical deployment of online learning frameworks, such as Deep Reinforcement Learning (DRL)-based COP optimization solutions, remains limited due to their risk of deteriorating network performance during the exploration phase. In this article, we propose and analyze a novel risk-aware DRL framework for user-centric RAN (UC-RAN), which offers both the architectural flexibility and COP optimization to exploit this flexibility. We investigate and identify UC-RAN COPs that can be optimized via a soft actor-critic algorithm implementable as an O-RAN application (rApp) to jointly maximize latency satisfaction, reliability satisfaction, area spectral efficiency, and energy efficiency. We use the offline learning on UC-RAN to reliably accelerate DRL training, thus minimizing the risk of DRL deteriorating cellular network performance. Results show that our proposed solution approaches near-optimal performance in just a few hundred iterations with a decrease in risk score by a factor of ten.

6G and beyond↗

Application of deep learning methods for beam size control during user operation at the Advanced Light Source

Past research at the Advanced Light Source (ALS) provided a proof-of-principle demonstration that deep learning methods could be effectively employed to compensate for the significant perturbations to the transverse electron beam size induced by user-controlled adjustments of the insertion devices. However, incorporating these methods into the ALS’ daily operations has faced notable challenges. The complexity of the system’s operational requirements and the significant upkeep demands has restricted their sustained application during user operation. Here, we introduce the development of a more robust neural network (NN)-based algorithm that utilizes a novel online fine-tuning approach and its systematic integration into the day-to-day machine operations. Our analysis emphasizes the process of NN model selection, demonstrates the superior performance of the NN-based method over traditional feedback methods, and examines the effectiveness and resilience of the new algorithm during user-operation scenarios. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗