Search NASA⌕ Search

SEARCH · Search NASA

Results for “controlled Markov chains”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

A Multiobjective Optimization Framework for Online Stochastic Optimal Control in Hybrid Electric Vehicles

The increasing urgency to extract additional efficiency from hybrid propulsion systems has led to the development of advanced power management control algorithms. In this paper we address the problem of online optimization of the supervisory power management control in parallel hybrid electric vehicles (HEVs). We model HEV operation as a controlled Markov chain and we show that the control policy yielding the Pareto optimal solution minimizes online the long-run expected average cost per unit time criterion. The effectiveness of the proposed solution is validated through simulation and compared to the solution derived with dynamic programming using the average cost criterion. Both solutions achieved the same cumulative fuel consumption demonstrating that the online Pareto control policy is an optimal control policy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2015

Centralized Stochastic Optimal Control of Complex Systems

In this paper we address the problem of online optimization of the supervisory power management control in parallel hybrid electric vehicles (HEVs). We model HEV operation as a controlled Markov chain using the long-run expected average cost per unit time criterion, and we show that the control policy yielding the Pareto optimal solution minimizes the average cost criterion online. The effectiveness of the proposed solution is validated through simulation and compared to the solution derived with dynamic programming using the average cost criterion.

complex systems↗
DOE OSTI2015

Pareto Efficient Policy for Supervisory Power Management Control

n this paper we address the problem of online optimization of the supervisory power management control in parallel hybrid electric vehicles (HEVs). We model HEV opera- tion as a controlled Markov chain using the long-run expected average cost per unit time criterion, and we show that the control policy yielding the Pareto optimal solution minimizes the average cost criterion online. The effectiveness of the proposed solution is validated through simulation and compared to the solution derived with dynamic programming using the average cost criterion.

Markov chain↗
DOE OSTI2018

Constructing the Pareto front for multi-objective Markov chains handling a strong Pareto policy approach

In this paper, we present a multi-objective solution of the Pareto front for a certain class of discrete-time ergodic controllable Markov chains. For solving the problem, we transform the original multi-objective problem into an equivalent nonlinear programming problem implementing the Lagrange principle. We then propose to adopt the Tikhonov’s regularization method to ensure the convergence of the cost functions to a unique point into the Pareto front. We show that Pareto policies are characterized as optimal policies. One of the fundamental problems related to the construction of the Pareto front is the existence and characterization of both, Pareto and strong Pareto policies. By introducing the Tikhonov’s regularizator, we ensure the existence of strong Pareto policies. This paper proposes a real solution to this problem. We formulate the original problem considering several constraints: (a) we employ the c-variable method for the introduction of linear constraints over the nonlinear problem and (b) restrict the cost functions, allowing points in the Pareto front to have a small distance from one another. The constraints imposed by the c-variable method make the problem computationally tractable and the restriction imposed by the small distance change ensures the continuation of the Pareto front. The resulting equation in this nonlinear system is an optimization problem for which the necessary and efficient condition of a minimum is solved using the projected gradient method. Moreover, we also prove the convergence conditions and compute the estimate rate of convergence of variables corresponding to the Lagrange principle and the Tikhonov’s regularization, respectively. We provide all the details needed to implement the proposed method in an efficient way. The usefulness of the method is successfully demonstrated by a numerical example.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

A Multiobjective Optimization Framework for Stochastic Control of Complex Systems

This paper addresses the problem of minimizing the long-run expected average cost of a complex system consisting of subsystems that interact with each other and the environment. We treat the stochastic control problem as a multiobjective optimization problem of the one-stage expected costs of the subsystems, and we show that the control policy yielding the Pareto optimal solution is an optimal control policy that minimizes the average cost criterion for the entire system. For practical situations with constraints consistent to those we study here, our results imply that the Pareto control policy may be of value in deriving online an optimal control policy in complex systems.

Complex systems↗
DOE OSTI2019

Data-based Aggregate Model of Refrigerators for Electric Power Grid Services

Electrical power system stability under high penetration of intermittent renewable energy generation, such as wind and solar energy, has become a high concern. It is unpredictable and hard to control due to little inertial behavior characteristic to these generation resources. To help mitigate possibly undesirable frequency excursions, demand-side frequency control presents an emerging solution. By observing the frequency deviation, devices could switch operating modes for a short time to rebalance generation. To analyze device level control algorithms, aggregate models for their behavior are needed. Refrigerators are household devices that could help with load modulation. This paper uses real measured data to analyze the refrigerator behavior, learn about its operating mode and power consumption ranges. A Markov chain process is proposed to model the transition of a refrigerator population from one operating mode to another, such that the overall load is modulated according to the control input. The Markov-chain model can be used to design control so that the overall load is modulated according to the control input.

Markov chain model, demand response, refrigerator,↗
DOE OSTI2017

Bayesian Inversion of Seismic and Electromagnetic Data for Marine Gas Reservoir Characterization Using Multi-chain Markov Chain Monte Carlo Sampling

In this study we developed an efficient Bayesian inversion framework for interpreting marine seismic amplitude versus angle (AVA) and controlled source electromagnetic (CSEM) data for marine reservoir characterization. The framework uses a multi-chain Markov-chain Monte Carlo (MCMC) sampler, which is a hybrid of DiffeRential Evolution Adaptive Metropolis (DREAM) and Adaptive Metropolis (AM) samplers. The inversion framework is tested by estimating reservoir-fluid saturations and porosity based on marine seismic and CSEM data. The multi-chain MCMC is scalable in terms of the number of chains, and is useful for computationally demanding Bayesian model calibration in scientific and engineering problems. As a demonstration, the approach is used to efficiently and accurately estimate the porosity and saturations in a representative layered synthetic reservoir. The results indicate that the seismic AVA and CSEM joint inversion provides better estimation of reservoir saturations than the seismic AVA-only inversion, especially for the parameters in deep layers. The performance of the inversion approach for various levels of noise in observational data was evaluated – reasonable estimates can be obtained with noise levels up to 25%. Sampling efficiency due to the use of multiple chains was also checked and was found to have almost linear scalability.

Ren, Huiying↗
DOE OSTI2025

Covariance-Free Bifidelity Control Variates Importance Sampling for Rare Event Reliability Analysis

Multifidelity modeling has been steadily gaining attention as a tool to address the problem of exorbitant model evaluation costs that makes the estimation of failure probabilities a significant computational challenge for complex real-world problems, particularly when failure is a rare event. To implement multifidelity modeling, estimators that efficiently combine information from multiple models/sources are necessary. In past works, the variance reduction techniques of control variates (CV) and importance sampling (IS) have been leveraged for this task. In this paper, we present the CVIS framework—a creative take on a coupled CV and IS estimator for bifidelity reliability analysis. The framework addresses some of the practical challenges of the CV method by using an estimator for the control variate mean and sidestepping the need to estimate the covariance between the original estimator and the control variate through a clever choice for the tuning constant. Furthermore, the task of selecting an efficient IS distribution is also considered, with a view towards maximally leveraging the bifidelity structure and maintaining expressivity. Additionally, a diagnostic is provided that indicates both the efficiency of the algorithm as well as the relative predictive quality of the models utilized. Finally, the behavior and performance of the framework is explored through analytical and numerical examples.

Markov chain Monte Carlo↗
DOE OSTI2015

Vehicle Energy Management Optimization through Digital Maps and Connectivity

There is an opportunity for using digital maps with real-time updates to improve the energy management of vehicles. In this paper, we present research on how this can be achieved in the particular case of plug-in hybrid electric vehicles. With perfect knowledge of the trip ahead, it is possible to find the optimal control of the vehicle. Digital maps, however, provide only averaged information about the future itinerary, such as locations of stops, types of intersection, and average speeds. First, we demonstrate how we can generate naturalistic speed profiles for a given itinerary. We do so by augmenting the information from digital maps by actual recorded speeds from a travel survey and Markov chains. Second, we detail how these speed profiles can be used for control optimization. We use the Pontryagin minimum principle to optimally tune the vehicle energy management and then evaluate the actual fuel savings using the Autonomie simulation tool.

Connectivity↗
DOE OSTI2023

Deep generative models for vehicle speed trajectories

Generating realistic vehicle speed trajectories is a crucial component in evaluating vehicle fuel economy and in predictive control of self-driving cars. Traditional generative models rely on Markov chain methods and can produce accurate synthetic trajectories but are subject to the curse of dimensionality. They do not allow to include conditional input variables into the generation process. In this paper, we show how extensions to deep generative models allow accurate and scalable generation. Proposed architectures involve recurrent and feed-forward layers and are trained using adversarial techniques. Our models are shown to perform well on generating vehicle trajectories using a model trained on GPS data from Chicago metropolitan area.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2019

An anomaly detection framework for dynamic systems using a Bayesian hierarchical framework

Complex systems are susceptible to many types of anomalies, faults, and abnormal behavior caused by a variety of off-nominal conditions that may ultimately result in major failures or catastrophic events. Early and accurate detection of these anomalies using system inputs and outputs collected from sensors and smart devices has become a challenging problem and an active area of research in many application domains. In this article, we present a new Bayesian hierarchical framework that is able to model the relationship between system inputs (sensor measurements) and outputs (response variables) without imposing strong distributional/parametric assumptions while using only a subset of training samples and sensor attributes. Then, an optimal cost-sensitive anomaly detection framework is proposed to determine whether a sample is an anomalous one taking into consideration the trade-off between misclassification errors and detection rates. The model can be used for both supervised and unsupervised settings depending on the availability of data regarding the behavior of the system under anomaly conditions. The unsupervised model is particularly useful when it is prohibitive to identify in advance the anomalies that a system may present and where no data are available regarding the behavior of the system under anomaly conditions. A Bayesian hierarchical setting is used to structure the proposed framework and help with accommodating uncertainty, imposing interpretability, and controlling the sparsity and complexity of the proposed anomaly detection framework. A Markov chain Monte Carlo algorithm is also developed for model training using past data. Furthermore, the numerical experiments conducted using a simulated data set and a wind turbine data set demonstrate the successful application of the proposed work for system response modeling and anomaly detection.

17 WIND ENERGY↗
DOE OSTI2025

Deep Learning without Global Optimization by Random Fourier Neural Networks

Here we introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Diffusion growth mechanism of penta-twinned Ag nanocrystals from decahedral seeds

Crystals with penta-twinned structures can be produced from diverse fcc metals, but the mechanisms that control the final product shapes are still not well understood. By using the theory of absorbing Markov chains to account for the growth of penta-twinned decahedral seeds via atom deposition and surface diffusion, we predicted the formation of various types of products: decahedra, nanorods, and nanowires. We showed that the type of product depends on the morphology of the seed and that small differences between various seed morphologies can lead to significantly different products. For the case of uncapped decahedra seeds, we compared predictions from our model to nanowire morphologies obtained in two different experiments and obtained favorable agreement. Finally, possible extensions of our model are indicated.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Real-Time Automated Hazard Detection Framework for Health Information Technology Systems

An increase in the reliability of Health Information Technology (HIT) will facilitate institutional trust and credibility of the systems. Here, we present an end-to-end framework for improving the reliability and performance of HIT systems. Specifically, we describe the system model, present some of the methods that drive the model, and discuss an initial implementation of two of the proposed methods using data from the Veterans Affairs HIT and Corporate Data Warehouse systems. The contributions of this work, thus, include (1) the design of a system model for monitoring and detecting hazards in HIT systems, (2) a data-driven approach for analysing the health care data warehouse, (3) analytical methods for characterising and analysing failures in HIT systems, and (4) a tool architecture for generating and reporting hazards in HIT systems. Our goal is to work towards an automated system that will help identify opportunities for improvements in HIT systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Quantification of early nonpharmaceutical interventions aimed at slowing transmission of Coronavirus Disease 2019 in the Navajo Nation and surrounding states (Arizona, Colorado, New Mexico, and Utah)

During an early period of the Coronavirus Disease 2019 (COVID-19) pandemic, the Navajo Nation, much like New York City, experienced a relatively high rate of disease transmission. Yet, between January and October 2020, it experienced only a single period of growth in new COVID-19 cases, which ended when cases peaked in May 2020. The daily number of new cases slowly decayed in the summer of 2020 until late September 2020. In contrast, the surrounding states of Arizona, Colorado, New Mexico, and Utah all experienced at least two periods of growth in the same time frame, with second surges beginning in late May to early June. Here, we investigated these differences in disease transmission dynamics with the objective of quantifying the contributions of non-pharmaceutical interventions (NPIs) (e.g., behaviors that limit disease transmission). We considered a compartmental model accounting for distinct periods of NPIs to analyze the epidemic in each of the five regions. We used Bayesian inference to estimate region-specific model parameters from regional surveillance data (daily reports of new COVID-19 cases) and to quantify uncertainty in parameter estimates and model predictions. Our results suggest that NPIs in the Navajo Nation were sustained over the period of interest, whereas in the surrounding states, NPIs were relaxed, which allowed for subsequent surges in cases. Our region-specific model parameterizations allow us to quantify the impacts of NPIs on disease incidence in the regions of interest.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Stochastic Adaptive Droop Control in Frequency Regulation of Power Systems With Intermittent Generators

Modern power systems (MPSs), including microgrids (MGs), are increasingly incorporating multiple renewable energy sources (RESs) such as wind and solar power, as well as battery storage and controllable loads. While environmentally beneficial, these sources pose challenges for control and management due to their intermittent and stochastic nature, especially in maintaining frequency stability with multiple interconnected generators of varying capacities. Traditional droop control methods are effective in systems with generators that are dispatchable and have fixed generation capacities, but they fall short when applied to systems with RESs, where generation capacities are dynamic and affected by unpredictable environmental conditions. To address these challenges, this paper introduces a novel stochastic adaptive droop control (SADC) method for load frequency control (LFC). The proposed method adapts droop coefficients in real time, based on the measured stochastic data of power generation capacities, enabling more effective frequency regulation in systems with variable and intermittent power generation. Unlike traditional adaptive control methods, which assume constant or slowly-varying system parameters, this approach accounts for stochastic processes by modeling them as Markov chains, enabling robust performance under highly dynamic and unpredictable conditions. The key contributions of this work include the development of real-time droop coefficient adaptation algorithms, derivation of their stability and convergence properties, and the demonstration of the advantages of the method through simulations. Case studies highlight the improved performance of frequency regulation, particularly in addressing the impact of stochastic weather conditions and the benefits of reducing dependence on battery reserves in dealing with intermittency of RESs. Finally, this paper provides a comprehensive analysis of the theoretical foundations of the method, as well as practical implementation insights for future power systems with high penetration of RESs.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2017

Robust Spectral Unmixing of Sparse Multispectral Lidar Waveforms using Gamma Markov Random Fields

Here, this paper presents a new Bayesian spectral un-mixing algorithm to analyse remote scenes sensed via sparse multispectral Lidar measurements. To a first approximation, in the presence of a target, each Lidar waveform consists of a main peak, whose position depends on the target distance and whose amplitude depends on the wavelength of the laser source considered (i.e, on the target reflectivity). Besides, these temporal responses are usually assumed to be corrupted by Poisson noise in the low photon count regime. When considering multiple wavelengths, it becomes possible to use spectral information in order to identify and quantify the main materials in the scene, in addition to estimation of the Lidar-based range profiles. Due to its anomaly detection capability, the proposed hierarchical Bayesian model, coupled with an efficient Markov chain Monte Carlo algorithm, allows robust estimation of depth images together with abundance and outlier maps associated with the observed 3D scene. The proposed methodology is illustrated via experiments conducted with real multispectral Lidar data acquired in a controlled environment. The results demonstrate the possibility to unmix spectral responses constructed from extremely sparse photon counts (less than 10 photons per pixel and band).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Growth Mechanism of Five-Fold Twinned Ag Nanowires from Multiscale Theory and Simulations

Five-fold twinned metal nanowires can be synthesized with high aspect ratios via solution-phase methods. The origins of their anisotropic growth, however, are poorly understood. Here, we combine atomic-scale, mesoscale, and continuum theoretical methods to predict growth morphologies of Ag nanowires from seeds and to demonstrate that high aspect ratio nanowires can originate from anisotropic surface diffusion induced by the strained nanowire structure. Nanowire seeds are similar to Marks decahedra, with {111} “notches” that accelerate diffusion along the nanowire axis to facilitate one-dimensional growth. The strain distribution on the {111} facets induces heterogeneous atom aggregation and leads to atom trapping at the nanowire ends. We predict that decahedral Ag seeds can grow to become nanowires with aspect ratios in the experimental range. Our studies show that there is a complex interplay between atom deposition, diffusion, seed architecture, and nanowire aspect ratio that could be manipulated experimentally to achieve controlled nanowire syntheses.

77 NANOSCIENCE AND NANOTECHNOLOGY↗