SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Optimal rules for controlling Markovian decision processes applied to solutions for problems dealing with ordering inventory supplies
Denumerable state Markovian decision processes - average cost criterion
Stochastic processes are used as a modeling tool in several sub-fields of physics, biology, and finance. Analytic understanding of the long term behavior of such processes is only tractable for very simple types of stochastic processes such as Markovian processes. However, in real world applications more complex stochastic processes often arise. In physics, the complicating factor might be nonlinearities; in biology it might be memory effects; and in finance is might be the non-random intentional behavior of participants in a market. In the absence of analytic insight, one is forced to understand these more complex stochastic processes via numerical simulation techniques. In this paper we present a quantum algorithm for performing such simulations. In particular, we show how a quantum algorithm can predict arbitrary descriptive statistics (moments) of N-step stochastic processes in just O(square root of N) time. That is, the quantum complexity is the square root of the classical complexity for performing such simulations. This is a significant speedup in comparison to the current state of the art.
A method for evaluating the probability of a Viable Earth Microorganism (VEM) contaminating a sample during the sample acquisition and handling (SAH) process of a potential future Mars Sample Return mission is developed. A scenario where multiple core samples would be acquired using a rotary percussive coring tool, deployed from an arm on a MER class rover is analyzed. The analysis is conducted in a structured way by decomposing sample acquisition and handling process into a series of discrete time steps, and breaking the physical system into a set of relevant components. At each discrete time step, two key functions are defined: The probability of a VEM being released from each component, and the transport matrix, which represents the probability of VEM transport from one component to another. By defining the expected the number of VEMs on each component at the start of the sampling process, these decompositions allow the expected number of VEMs on each component at each sampling step to be represented as a Markov chain. This formalism provides a rigorous mathematical framework in which to analyze the probability of a VEM entering the sample chain, as well as making the analysis tractable by breaking the process down into small analyzable steps.
The problem of formulating and analyzing Markov decision models having decentralized information and decision patterns is examined. Included are basic examples as well as the mathematical preliminaries needed to understand Markov decision models and, further, to superimpose decentralized decision structures on them. The notion of a variance admissible policy for the model is introduced and it is proved that there exist (possibly nondeterministic) optional policies from the class of variance admissible policies. Directions for further research are explored.
This modelization starts from the following hypotheses: pilot's behavior is a time discrete process, he can perform only one task at a time and his operating mode depends on the considered flight subphase. Pilot's behavior was observed using an electro oculometer and a simulator cockpit. A FORTRAN program has been elaborated using two strategies. The first one is a Markovian process in which the successive instrument readings are governed by a matrix of conditional probabilities. In the second one, strategy is an heuristic process and the concepts of mental load and performance are described. The results of the two aspects have been compared with simulation data.
Abstract Recurrent neural networks are used to forecast time series in finance, climate, language, and from many other domains. Reservoir computers are a particularly easily trainable form of recurrent neural network. Recently, a “next-generation” reservoir computer was introduced in which the memory trace involves only a finite number of previous symbols. We explore the inherent limitations of finite-past memory traces in this intriguing proposal. A lower bound from Fano’s inequality shows that, on highly non-Markovian processes generated by large probabilistic state machines, next-generation reservoir computers with reasonably long memory traces have an error probability that is at least $$\sim 60\%$$ ∼ 60 % higher than the minimal attainable error probability in predicting the next observation. More generally, it appears that popular recurrent neural networks fall far short of optimally predicting such complex processes. These results highlight the need for a new generation of optimized recurrent neural network architectures. Alongside this finding, we present concentration-of-measure results for randomly-generated but complex processes. One conclusion is that large probabilistic state machines—specifically, large $$\epsilon$$ ϵ -machines—are key to generating challenging and structurally-unbiased stimuli for ground-truthing recurrent neural network architectures.
Existing availability models of standby redundant systems consider only an operator's performance and its interaction with the hardware performance. In the case of operational data systems in the Deep Space Network (DSN), in addition to an operator system interface, a controller reconfigures the system and links a standby unit into the network data path upon failure of the operating unit. A stochastic (Markovian) process technique is used to model and analyze the availability performance and occurrence of degradation due to partial failures are quantitatively incorporated into the model. Exact expressions of the steady state availability and proportion degraded performance measures are derived for the systems under study. The interaction among the hardware, operator, and controller performance parameters and that interaction's effect on data availability are evaluated and illustrated for an operational data processing system.
We present preliminary results from a model that diffusively accelerates particles at multiple shocks. Our basic approach is related to box models (Protheroe and Stanev, 1998; Moraal and Axford, 1983; Ball and Kirk, 1992; Drury et al., 1999) in which a distribution of particles is diffusively accelerated inside the box while simultaneously experiencing decompression through adiabatic expansion and losses from the convection and diffusion of particles outside the box (Melrose and Pope, 1993; Zank et al., 2000). We adiabatically decompress the accelerated particle distribution between each shock by either the method explored in Melrose and Pope (1993) and Pope and Melrose (1994) or by the approach set forth in Zank et al. (2000) where we solve the transport equation by a method analogous to operator splitting. The second method incorporates the additional loss terms of convection and diffusion and allows for the use of a variable time between shocks. We use a maximum injection energy (Emax) appropriate for quasi-parallel and quasi-perpendicular shocks (Zank et al., 2000, 2006; Dosch and Shalchi, 2010) and provide a preliminary application of the diffusive acceleration of particles by multiple shocks with frequencies appropriate for solar maximum (i.e., a non-Markovian process).
Successful forecasting of energetic particle events in space weather models require algorithms for correctly predicting the spectrum of ions accelerated from a background population of charged particles. We present preliminary results from a model that diffusively accelerates particles at multiple shocks. Our basic approach is related to box models in which a distribution of particles is diffusively accelerated inside the box while simultaneously experiencing decompression through adiabatic expansion and losses from the convection and diffusion of particles outside the box. We adiabatically decompress the accelerated particle distribution between each shock by either the method explored in Melrose and Pope (1993) and Pope and Melrose (1994) or by the approach set forth in Zank et al. (2000) where we solve the transport equation by a method analogous to operator splitting. The second method incorporates the additional loss terms of convection and diffusion and allows for the use of a variable time between shocks. We use a maximum injection energy (E(sub max)) appropriate for quasi-parallel and quasi-perpendicular shocks and provide a preliminary application of the diffusive acceleration of particles by multiple shocks with frequencies appropriate for solar maximum (i.e., a non-Markovian process).
This paper proposes a risk-aware framework for Safe Multi-Agent Planning (SafeMAP) that unifies disparate models for multi-agent systems in a Markovian process that allows for simultaneous system health monitoring, decision making under uncertainty, and multi-agent system collaboration. As operations beyond low earth orbit mature, there is an increased need for autonomous cyber-physical systems with onboard decision making capabilities. Multi-agent cyber-physical systems in particular offer the potential of increased efficiency, resiliency, and mission capabilities for future applications such as multi-rover terrain operations, distributed satellite operations, and management of smart lunar habitats. SafeMAP utilizes physics-based models of each agent and the relevant components, probability models of the environment and component operational states, and reward models for mission-specific objectives such as scientific task completion or resource consumption. The output of SafeMAP is a set of mission plans that satisfy the mission objective under specified risk/reward constraints. A readable interpretation of each of these generated mission plans is provided as an additional output. SafeMAP has been demonstrated on a simulated case study involving a four-rover system performing surface mapping operations and science tasks. Results of this paper demonstrate SafeMAP’s ability to generate explainable mission plans that satisfy the mission objective while minimizing risk under nominal and off-nominal conditions.
This paper proposes a risk-aware framework for Safe Multi-Agent Planning (SafeMAP) that unifies disparate models for multi-agent systems in a Markovian process that allows for simultaneous system health monitoring, decision making under uncertainty, and multi-agent system collaboration. As operations beyond low earth orbit mature, there is an increased need for autonomous cyber-physical systems with onboard decision making capabilities. Multi-agent cyber-physical systems in particular offer the potential of increased efficiency, resiliency, and mission capabilities for future applications such as multi-rover terrain operations, distributed satellite operations, and management of smart lunar habitats. SafeMAP utilizes physics-based models of each agent and the relevant components, probability models of the environment and component operational states, and reward models for mission-specific objectives such as scientific task completion or resource consumption. The output of SafeMAP is a set of mission plans that satisfy the mission objective under specified risk/reward constraints. A readable interpretation of each of these generated mission plans is provided as an additional output. SafeMAP has been demonstrated on a simulated case study involving a four-rover system performing surface mapping operations and science tasks. Results of this paper demonstrate SafeMAP’s ability to generate explainable mission plans that satisfy the mission objective while minimizing risk under nominal and off-nominal conditions.
Highlights: • Non-Markovian master equation for open quantum system with power-law memory is proposed. • Non-Markovian dynamics of two-level quantum system with memory is described. • Solutions of non-Markovian quantum master equations are derived. • Complete positivity and bi-positivity in non-Markovian quantum dynamics with memory are described. • Exact solution of non-Markovian equations of system with memory is proposed. In this paper, non-Markovian generalization of master equation for open quantum system is proposed. Non-Markovian dynamics of two-level quantum system with memory and interaction with environment is described. To describe this system, the Gorini–Kossakowski–Sudarshan equation for open quantum states is generalized by taking into account power-law fading memory. The non-Markovian quantum processes with power-law memory are described by using integration and differentiation of non-integer orders. Complete positivity and bi-positivity in non-Markovian quantum dynamics with memory are described. An example of two-level quantum systems with power-law memory is suggested. Exact solution of the non-Markovian master equations of two-level open quantum systems with memory is derived.
Quantum algorithms for differential equation solving, data processing, and machine learning potentially offer an exponential speedup over all known classical algorithms. However, there also exist obstacles to obtaining this potential speedup in useful problem instances. The essential obstacle for quantum differential equation solving is that outputting useful information may require difficult postprocessing, and the essential obstacle for quantum data processing and machine learning is that inputting the data is a difficult task just by itself. In this study, we demonstrate that, when combined, these difficulties solve one another. We show how the output of quantum differential equation solving can serve as the input for quantum data processing and machine learning, allowing dynamical analysis in terms of principal components, power spectra, and wavelet decompositions. To illustrate this, we consider continuous-time Markov processes on epidemiological and social networks. These quantum algorithms provide an exponential advantage over existing classical Monte Carlo methods.
One of the most prominent platforms for demonstrating quantum sensing below the standard quantum limit is the spinor Bose–Einstein condensate. While a quantum advantage using several tens of thousands of atoms has been demonstrated in this platform, it faces an important challenge: atom loss. Atom loss is a Markovian error process modeled by Lindblad jump operators, and a no-go theorem, which we also show here, states that the loss of atoms in all spin components reduces the quantum advantage to a constant factor. Here, we show that this no-go theorem can be circumvented if we constrain atom losses to a single spin component. Moreover, we show that in this case, the maximum quantum Fisher information with N atoms scales as N 3/2 , establishing that a scalable quantum advantage can be achieved despite atom loss. Although Lindblad jump operators are generally non-Hermitian and non-invertible, we use their Moore–Penrose inverse to develop a framework for constructing several states with this scaling of Fisher information in the presence of losses. We use Hamiltonian engineering with realistic Hamiltonians to develop experimental protocols for preparing these states. Finally, we discuss possible experimental techniques to constrain the losses to a single spin mode.
The traditional approach to quantum parameter estimation focuses on the quantum state, deriving fundamental bounds on precision through the quantum Fisher information. In most experimental settings, however, performing arbitrary quantum measurements is highly unfeasible. In open quantum systems, an alternative approach to metrology involves the measurement of stochastic currents flowing from the system to its environment. However, the present understanding of current-based metrology is mostly limited to Markovian master equations. Considering a parameter estimation problem in a two-terminal mesoscopic conductor, we identify the key elements that determine estimation precision within the Landauer-Büttiker formalism. Crucially, this approach allows us to address arbitrary coupling and temperature regimes. Furthermore, we obtain analytical results for the precision in linear-response and zero-temperature regimes. For the specific parameter estimation task that we consider, we demonstrate that the boxcar transmission function is optimal for current-based metrology in all parameter regimes.
Entanglement is a resource to improve the sensitivity of quantum sensors. In an ideal case, using an entangled state as a probe to detect target fields, we can beat the standard quantum limit by which all classical sensors are bounded. However, since entanglement is fragile against decoherence, it is unclear whether entanglement-enhanced metrology is useful in a noisy environment. Its benefit is indeed limited when estimating the amplitude of dc magnetic fields under the effect of parallel Markovian decoherence, where the noise operator is parallel to the target field. In this paper, on the contrary, we show an advantage to using an entanglement over the classical strategy under the effect of parallel Markovian decoherence when we try to detect ac magnetic fields. We consider a scenario to induce a Rabi oscillation of the qubits with the target ac magnetic fields. Although we can, in principle, estimate the amplitude of the ac magnetic fields from the Rabi oscillation, the signal becomes weak if the qubit frequency is significantly detuned from the frequency of the ac magnetic field. We show that, by using the Greenberger-Horne-Zeilinger (GHZ) states, we can significantly enhance the signal of the detuned Rabi oscillation even under the effect of parallel Markovian decoherence. Further, our method is based on the fact that the interaction time between the GHZ states and ac magnetic fields scales as 1/L to mitigate the decoherence effect, where L is the number of qubits, which contributes to improving the bandwidth of the detectable frequencies of the ac magnetic fields. Our results pave the way for new applications of entanglement-enhanced ac magnetometry.
Here, we introduce a new framework to study the dynamics of open quantum systems with linearly coupled Gaussian baths. Our approach replaces the continuous bath with an auxiliary discrete set of pseudomodes with dissipative dynamics, but we further relax the complete positivity requirement in the Lindblad master equation and formulate a quasi-Lindblad pseudomode theory. We show that this quasi-Lindblad pseudomode formulation directly leads to a representation of the bath correlation function in terms of a complex weighted sum of complex exponentials, an expansion that is known to be rapidly convergent in practice and thus leads to a compact set of pseudomodes. The pseudomode representation is not unique and can differ by a gauge choice. When the global dynamics can be simulated exactly, the system dynamics is unique and independent of the specific pseudomode representation. However, the gauge choice may affect the stability of the global dynamics, and we provide an analysis of why and when the global dynamics can retain stability despite losing positivity. We showcase the performance of this formulation across various spectral densities in both bosonic and fermionic problems, finding significant improvements over conventional pseudomode formulations.