Search NASA⌕ Search

SEARCH · Search NASA

Results for “regret analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

8 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

An Intelligent Distributed Ledger Construction Algorithm for IoT

Blockchain is the next generation of secure data management that creates near-immutable decentralized storage. Secure cryptography created a niche for blockchain to provide alternatives to well-known security compromises. However, design bottlenecks with traditional blockchain data structures scale poorly with increased network usage and are extremely computation-intensive. This made the technology difficult to combine with limited devices, like those in Internet of Things networks. In protocols like IOTA, replacement of blockchain's linked-list queue processing with a lightweight dynamic ledger showed remarkable throughput performance increase. However, current stochastic algorithms for ledger construction suffer distinct trade-offs between efficiency and security. This work proposed a machine-learning approach with a multi-arm bandit that resolved these issues and was designed for auditing on limited devices. This algorithm was tested in a reinforcement-learning environment simulating the IOTA ledger's construction with a decision tree. This study showed through regret analysis and experimentation that this approach was secure against impulse manipulation attacks while remaining energy-efficient. Although the IOTA protocol was a pioneer for lightweight distributed ledgers, it is expected that future blockchain protocols will adopt techniques similar to those presented in this work.

multi-arm bandit↗
DOE OSTI2026

Near-Optimal Performance of Stochastic Model Predictive Control

Here, this article presents a regret analysis for stochastic model predictive control (SMPC) in linear systems with quadratic performance index and additive and multiplicative uncertainties. Under a finite support assumption, the problem can be cast as a finite-dimensional quadratic program, but the problem becomes quickly intractable as the problem size grows exponentially in the horizon length. SMPC aims to compute approximate solutions by solving a sequence of problems with truncated prediction horizons and committing the solution in a receding-horizon fashion. Although this approach is widely used in practice, its performance relative to the optimal solution is not well understood. This article reports for the first time a rigorous near-optimal performance guarantee of SMPC: under stabilizability and detectability conditions, the regret of SMPC is exponentially small in the prediction horizon length, allowing SMPC to achieve near-optimal performance at a substantially reduced computational expense.

93E20, 93B45↗
DOE OSTI2022

Online Learning and Pricing for Network Revenue Management with Reusable Resources

We consider a price-based network revenue management problem with multiple products and multiple reusable resources. Each randomly arriving customer requests a product (service) that needs to occupy a sequence of reusable resources (servers). We adopt an incomplete information setting where the firm does not know the price-demand function for each product and the goal is to dynamically set prices of all products to maximize the total expected revenue of serving customers. We propose novel batched bandit learning algorithms for finding near-optimal pricing policies, and show that they admit a near-optimal cumulative regret bound of $O(J\sqrt{XT})$, where J, X, and T, are the numbers of products, candidate prices, and service periods, respectively. As part of our regret analysis, we develop the first finite-time mixing time analysis of an open network queueing system (i.e., the celebrated Jackson Network), which could be of independent interest. Our numerical studies show that the proposed approaches perform consistently well.

42 ENGINEERING↗
DOE OSTI2022

Contextual Active Online Model Selection with Expert Advice

How can we collect the most useful labels to learn a model selection policy, when presented with arbitrary heterogeneous data streams? In this paper, we formulate this task as a contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. In particular, we focus on the task of selecting pre-trained classifiers, and propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection. In comparison to prior art, our algorithm does not assume a globally optimal model. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Our experiments on several benchmark classification datasets demonstrate the algorithm’s effectiveness in terms of both regret and query complexity. Notably, to achieve the same accuracy, CAMS incurs less than 10% of the label cost when compared to the best online model selection baselines on CIFAR10.

Liu, Xuefeng↗
DOE OSTI2024

Enhancing Distribution System Resilience: A First-Order Meta-RL Algorithm for Critical Load Restoration

The increasing frequency of extreme events and the integration of distributed energy resources (DERs) into modern grids have elevated the need for resilient and efficient critical load restoration strategies in distribution systems. However, the stochastic nature of renewable DERs, limited energy resource availability and the intricate nonlinearities inherent in complex grid control problem make the problem challenging. Although reinforcement learning (RL) and warm-start RL methods have shown promising results, their performance often falls short in rapidly adapting to new, unseen situations and typically requires exhaustive problem-specific tuning. To address these gaps, we propose a First-Order Meta-based RL (FOM-RL) algorithm within an online framework for adaptive and robust critical load restoration. By harnessing local DERs as the enabling technology, FOM-RL allows the RL agent to swiftly adapt to new unseen scenarios by leveraging previously acquired knowledge of different tasks. Experimental results provide evidence that proposed algorithm learns more efficiently and showcases generalization capabilities across diverse set of operational scenarios. Moreover, a rigorous theoretical analysis yields a tight sublinear regret bound, sensitive to temporal variability, with a task-averaged optimality gap bounded by O(VM+D*/(Tsquare root(M))). These results suggest that optimality improves with task similarity and an increased number of tasks M, reaffirming the efficacy and scalability of the proposed approach in addressing the complexities of critical load restoration in distribution systems.

complexity theory↗
DOE OSTI2024

Corrigendum to “Cross-sectoral assessment of CO2 capture from U.S. industrial flue gases for fuels and chemicals manufacture” [International Journal of Greenhouse Gas Control 135 (2024) 1-20 / 104137]

The authors regret the inaccuracy in the vertical axis title of Fig. 4 and the distortion in the legend of Fig. 13. Corrections have been made to the vertical axis title and legend of Figs. 4 and 13, respectively. These changes do not affect the analysis, calculations, or results in any way. The authors apologize for any inconvenience caused. The corrected figures are as follows:

Zuberi, M Jibran S↗
DOE OSTI2022

Online Learning and Pricing for Service Systems with Reusable Resources

We consider a price-based revenue management problem with finite reusable resources over a finite time horizon T. Customers arrive following a price-dependent Poisson process, and each customer requests one unit of c homogeneous reusable resources. If there is an available unit, the customer gets served within a price-dependent exponentially distributed service time; otherwise, the customer waits in a queue until the next available unit. In this paper, we assume that the firm does not know how the arrival and service rates depend on posted prices, and thus it makes adaptive pricing decisions in each period based only on past observations to maximize the cumulative revenue. Given a discrete price set with cardinality P, we propose two online learning algorithms, termed batch upper confidence bound (BUCB) and batch Thompson sampling (BTS), and prove that the cumulative regret upper bound is O˜(√PT) , which matches the regret lower bound. In establishing the regret, we bound the transient system performance upon price changes via a novel coupling argument, and also generalize bandits to accommodate subexponential rewards. Here, we also extend our approach to models with balking and reneging customers and discuss a continuous price setting. Our numerical experiments demonstrate the efficacy of the proposed BUCB and BTS algorithms.

97 MATHEMATICS AND COMPUTING↗