Search NASA⌕ Search

SEARCH · Search NASA

Results for “Value function”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
NASA NTRS1981

An approach to thermal-vacuum test optimization

An analytical model is presented that simulates failure rate characteristics of spacecraft equipment progressing through component and system level thermal-vacuum test programs and on into orbit. Costs for the test programs are part of the model as well as costs attributable to the mission. A lost value function combines the performance of the equipment with these costs and allows comparisons to be made among various test combinations and the resulting predicted mission performance. A minimum overall cost program is obtained by selecting that test program that yields the lowest lost value figure.

Kruger, R.↗
NASA NTRS1989

A high-order Lagrangian-decoupling method for the incompressible Navier-Stokes equations

A high-order Lagrangian-decoupling method is presented for the unsteady convection-diffusion and incompressible Navier-Stokes equations. The method is based upon: (1) Lagrangian variational forms that reduce the convection-diffusion equation to a symmetric initial value problem; (2) implicit high-order backward-differentiation finite-difference schemes for integration along characteristics; (3) finite element or spectral element spatial discretizations; and (4) mesh-invariance procedures and high-order explicit time-stepping schemes for deducing function values at convected space-time points. The method improves upon previous finite element characteristic methods through the systematic and efficient extension to high order accuracy, and the introduction of a simple structure-preserving characteristic-foot calculation procedure which is readily implemented on modern architectures. The new method is significantly more efficient than explicit-convection schemes for the Navier-Stokes equations due to the decoupling of the convection and Stokes operators and the attendant increase in temporal stability. Numerous numerical examples are given for the convection-diffusion and Navier-Stokes equations for the particular case of a spectral element spatial discretization.

Ho, Lee-Wing↗
NASA NTRS1999

The Analysis and Design of Low Boom Configurations Using CFD and Numerical Optimization Techniques

The use of computational fluid dynamics (CFD) for the analysis of sonic booms generated by aircraft has been shown to increase the accuracy and reliability of predictions. CFD takes into account important three-dimensional and nonlinear effects that are generally neglected by modified linear theory (MLT) methods. Up to the present time, CFD methods have been primarily used for analysis or prediction. Some investigators have used CFD to impact the design of low boom configurations using trial and error methods. One investigator developed a hybrid design method using a combination of Modified Linear Theory (e.g. F-functions) and CFD to provide equivalent area due to lift driven by a numerical optimizer to redesign or modify an existing configuration to achieve a shaped sonic boom signature. A three-dimensional design methodology has not yet been developed that completely uses nonlinear methods or CFD. Constrained numerical optimization techniques have existed for some time. Many of these methods use gradients to search for the minimum of a specified objective function subject to a variety of design variable bounds, linear and nonlinear constraints. Gradient based design optimization methods require the determination of the objective function gradients with respect to each of the design variables. These optimization methods are efficient and work well if the gradients can be obtained analytically. If analytical gradients are not available, the objective gradients or derivatives with respect to the design variables must be obtained numerically. To obtain numerical gradients, say, for 10 design variables, might require anywhere from 10 to 20 objective function evaluations. Typically, 5-10 global iterations of the optimizer are required to minimize the objective function. In terms of using CFD as a design optimization tool, the numerical evaluation of gradients can require anywhere from 100 to 200 CFD computations per design for only 10 design variables. If one CFD computation requires an hour of computational time on a Cray computer, one can see that the use of constrained numerical optimization quickly becomes impractical.Hence, in order to practically couple a numerical design optimization technique with a CFD method, the CFD method must be extremely efficient with running times on the order of only minutes. The CFD Euler code developed under NASA sponsorship and referred to as MIM3D-SB for the most part fulfills these efficiency requirements. Analysis of wing- body configurations can be computed in a matter of a few minutes. The present study will concentrate on the feasibility of the use of this CFD code in conjunction with a numerical design optimization technique for the sonic boom reduction of candidate HSCT configurations. A preliminary supersonic aircraft design system has been established that utilizes the numerical design optimization code NPSOL developed at Stanford University coupled with the supersonic NUM3D-SB CFD code. Many questions still need to be answered in regard to using CFD and numerical optimizers as design tools. There are difficulties related to both the CFD codes and the numerical optimizers. Numerical optimizers can converge to a local minima rather than a global minima. This behavior is largely a function of the initial guess in the design space. The optimizer also is searching for a minimum of the function in terms of its derivative without any regard to the actual function value. Numerically (i.e. CFD) determined gradients can also generate spurious numerical local minima. In addition, for the sonic boom problem, grid fineness will also determine the accuracy of the final design solution. Design optimization methods work well on problems defined by continuous objective functions. The sonic boom signature design problem is not necessarily defined by a continuous objective function. The signature can have a variety of shapes; i.e. from N-wave to multiple shocks. The far-field or ground signature may not transition continuously from one shape to another and hence, may exhibit discontinuous behavior. This is also a source of difficulty in using design optimization methods.In the following sections, several low boom and one reference aircraft configuration will be analyzed to predict their sonic boom signature characteristics. Modifications to some of these designs will also be presented to demonstrate the feasibility of using CFD as a design tool and to demonstrate the feasibility of designing shaped sonic boom signatures. Design modifications to some configurations will be presented to demonstrate the feasibility of achieving shaped signatures with reduced levels and not necessarily to represent realistic or aerodynamically efficient design modifications. Fuselage volume or camber are used as design variables in order to have a minimal effect on the primary wing aerodynamics. The paper will also seek to demonstrate whether a hybrid or ramped signature is feasible to achieve. For the low-boom configurations, the CFD predicted signatures will be compared qualitatively to their MLT design signatures.

Siclari, Michael J.↗
NASA NTRS2005

Quicker Q-Learning in Multi-Agent Systems

Multi-agent learning in Markov Decisions Problems is challenging because of the presence ot two credit assignment problems: 1) How to credit an action taken at time step t for rewards received at t' greater than t; and 2) How to credit an action taken by agent i considering the system reward is a function of the actions of all the agents. The first credit assignment problem is typically addressed with temporal difference methods such as Q-learning OK TD(lambda) The second credit assi,onment problem is typically addressed either by hand-crafting reward functions that assign proper credit to an agent, or by making certain independence assumptions about an agent's state-space and reward function. To address both credit assignment problems simultaneously, we propose the Q Updates with Immediate Counterfactual Rewards-learning (QUICR-learning) designed to improve both the convergence properties and performance of Q-learning in large multi-agent problems. Instead of assuming that an agent s value function can be made independent of other agents, this method suppresses the impact of other agents using counterfactual rewards. Results on multi-agent grid-world problems over multiple topologies show that QUICR-learning can achieve up to thirty fold improvements in performance over both conventional and local Q-learning in the largest tested systems.

Agogino, Adrian K.↗
DOE OSTI2026

A Stochastic Quasi-Newton Method in the Absence of Common Random Numbers

We present Q-SASS, a quasi-Newton method for unconstrained stochastic optimization that does not rely on common random numbers. Most existing quasi-Newton approaches leverage common random numbers to construct second-order updates. However, motivated by challenges in variational quantum algorithms—where such coordination is not possible—we consider the setting in which function values and gradients are accessible only through noisy probabilistic zeroth- and first-order oracles, and no common random numbers can be exploited. We derive high-probability tail bounds on the iteration complexity of our algorithm for nonconvex, convex, and strongly convex (more generally, those satisfying the PL condition) objective functions. Finally, we demonstrate the empirical benefits of our quasi-Newton updating scheme on both synthetic and quantum chemistry problems.

Complexity bound↗
DOE OSTI2022

A nonsmooth nonconvex optimization algorithm for two-stage optimization problems

An optimization algorithm for a group of nonsmooth nonconvex problems inspired by two-stage stochastic programming problems is proposed. The main challenges for these problems include (1) the problems lack the popular lower-type properties such as prox-regularity assumed in many nonsmooth nonconvex optimization algorithms, (2) the objective can not be analytically expressed and (3) the evaluation of function values and subgradients are computationally expensive. To address these challenges, this report first examines the properties that exist in many two-stage problems, specifically upper-C 2 objectives. Then, we show that quadratic penalty method for securityconstrained alternating current optimal power flow (SCACOPF) contingency problems can make the contingency solution functions upper-C 2 . Based on these observations, a simplified bundle algorithm that bears similarity to sequential quadratic programming (SQP) method is proposed. It is more efficient in implementation and computation compared to conventional bundle methods. Global convergence analysis of the algorithm is presented under novel and reasonable assumptions. The proposed algorithm therefore fills the gap of theoretical convergence for smoothed SCACOPF problems. The inconsistency that might arise in our treatment of the constraints are addressed through a penalty algorithm whose convergence analysis is also provided. Finally, theoretical capabilities and numerical performance of the algorithm are demonstrated through numerical examples.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2013

A Tool for the Automated Design and Evaluation of Habitat Interior Layouts

The objective of space habitat design is to minimize mass and system size while providing adequate space for all necessary equipment and a functional layout that supports crew health and productivity. Unfortunately, development and evaluation of interior layouts is often ignored during conceptual design because of the subjectivity and long times required using current evaluation methods (e.g., human-in-the-loop mockup tests and in-depth CAD evaluations). Early, more objective assessment could prevent expensive design changes that may increase vehicle mass and compromise functionality. This paper describes a new interior design evaluation method to enable early, structured consideration of habitat interior layouts. This interior layout evaluation method features a comprehensive list of quantifiable habitat layout evaluation criteria, automatic methods to measure these criteria from a geometry model, and application of systems engineering tools and numerical methods to construct a multi-objective value function measuring the overall habitat layout performance. In addition to a detailed description of this method, a C++/OpenGL software tool which has been developed to implement this method is also discussed. This tool leverages geometry modeling coupled with collision detection techniques to identify favorable layouts subject to multiple constraints and objectives (e.g., minimize mass, maximize contiguous habitable volume, maximize task performance, and minimize crew safety risks). Finally, a few habitat layout evaluation examples are described to demonstrate the effectiveness of this method and tool to influence habitat design.

Simon, Matthew A.↗
DOE OSTI2022

Interaction mechanism of transition metal phthalocyanines on transition metal nitride supports

Here, we investigated the electronic interactions between transition metal phthalocyanine (TMPc’s) on a refractory transition metal nitride support, specifically copper phthalocyanine (CuPc) on titanium nitride (TiN). X-ray Photoelectron Spectroscopy (XPS) results suggest a presence of a few nanometer native oxide layer on the surface of the TiN nanoparticles, which consists of TiN, TiO 2 , and Titanium oxynitrides (Ti x O y N z ). A TiNCuPc nanocomposite was synthesized via a simple mixing method due to the strong binding between CuPc and TiN confirmed by density functional theory (DFT) calculations. Both XPS data and DFT calculations revealed an electron transfer from TiN substrate to CuPc molecule. The nature of charge transfer is not influenced by the presence of an oxide layer on the surface of TiN. Substantial deviations are however found between photoelectron emission microscopy (PEEM) measured work function for TiN (4.68 eV) and theoretically calculated work function for pristine stoichiometric TiN (2.63 eV). This behavior is attributed to the presence of an oxide layer on the TiN surface. TiNCuPc composite system has a work function value between those of TiN and CuPc. Our studies open up an opportunity to apply a new class of materials based on transition metal phthalocyanine/transition metal nitride composites to catalysis and optoelectronic devices.

36 MATERIALS SCIENCE↗
NASA NTRS1982

Scattering characteristics of Martin Black at 118 microns

BRDF (bidirectional reflectance distribution function) values for 0.000118 m radiation at different angles of incidence and different scattering angles from the Infrared Astronomical Satellite telescope baffle coated with Martin Black are presented. Data from scatterometer experiments are collected and the BRDF and beta - beta sub 0 (sin theta sub s - sin theta sub 0) values are calculated based on the geometry, the voltage readings, the attenuators in the beam, and the calculated reference levels. A composite curve of forward and backward scattering data for several angles of incidence shows a peak near the specular direction (beta - beta sub 0 = 0), which is the instrument profile reduced by the 20% specular reflection of the Martin Black. The nonspecular part of the reflectivity indicates the slightly specular but largely Lambertian character of the coating. Data for the specular reflectivity as a function of the incidence angle unexpectedly shows a decrease in the specular reflectance with increasing angle of incidence.

Brooks, L. D.↗
NASA NTRS2004

Application of Simulated Annealing and Related Algorithms to TWTA Design

Simulated Annealing (SA) is a stochastic optimization algorithm used to search for global minima in complex design surfaces where exhaustive searches are not computationally feasible. The algorithm is derived by simulating the annealing process, whereby a solid is heated to a liquid state and then cooled slowly to reach thermodynamic equilibrium at each temperature. The idea is that atoms in the solid continually bond and re-bond at various quantum energy levels, and with sufficient cooling time they will rearrange at the minimum energy state to form a perfect crystal. The distribution of energy levels is given by the Boltzmann distribution: as temperature drops, the probability of the presence of high-energy bonds decreases. In searching for an optimal design, local minima and discontinuities are often present in a design surface. SA presents a distinct advantage over other optimization algorithms in its ability to escape from these local minima. Just as high-energy atomic configurations are visited in the actual annealing process in order to eventually reach the minimum energy state, in SA highly non-optimal configurations are visited in order to find otherwise inaccessible global minima. The SA algorithm produces a Markov chain of points in the design space at each temperature, with a monotonically decreasing temperature. A random point is started upon, and the objective function is evaluated at that point. A stochastic perturbation is then made to the parameters of the point to arrive at a proposed new point in the design space, at which the objection function is evaluated as well. If the change in objective function values (Delta)E is negative, the proposed new point is accepted. If (Delta)E is positive, the proposed new point is accepted according to the Metropolis criterion: rho((Delta)f) = exp((-Delta)E/T), where T is the temperature for the current Markov chain. The process then repeats for the remainder of the Markov chain, after which the temperature is decremented and the process repeats. Eventually (and hopefully), a near-globally optimal solution is attained as T approaches zero. Several exciting variants of SA have recently emerged, including Discrete-State Simulated Annealing (DSSA) and Simulated Tempering (ST). The DSSA algorithm takes the thermodynamic analogy one step further by categorizing objective function evaluations into discrete states. In doing so, many of the case-specific problems associated with fine-tuning the SA algorithm can be avoided; for example, theoretical approximations for the initial and final temperature can be derived independently of the case. In this manner, DSSA provides a scheme that is more robust with respect to widely differing design surfaces. ST differs from SA in that the temperature T becomes an additional random variable in the optimization. The system is also kept in equilibrium as the temperature changes, as opposed to the system being driven out of equilibrium as temperature changes in SA. ST is designed to overcome obstacles in design surfaces where numerous local minima are separated by high barriers. These algorithms are incorporated into the optimal design of the traveling-wave tube amplifier (TWTA). The area under scrutiny is the collector, in which it would be ideal to use negative potential to decelerate the spent electron beam to zero kinetic energy just as it reaches the collector surface. In reality this is not plausible due to a number of physical limitations, including repulsion and differing levels of kinetic energy among individual electrons. Instead, the collector is designed with multiple stages depressed below ground potential. The design of this multiple-stage collector is the optimization problem of interest. One remaining problem in SA and DSSA is the difficulty in determining when equilibrium has been reached so that the current Markov chain can be terminated. It has been suggested in recent literature that simulating the thermodynamic properties opecific heat, entropy, and internal energy from the Boltzmann distribution can provide good indicators of having reached equilibrium at a certain temperature. These properties are tested for their efficacy and implemented in SA and DSSA code with respect to TWTA collector optimization.

Radke, Eric M.↗
NASA NTRS2011

Existence and Stability of Menisci in Detached Bridgman Growth

Detached growth, also referred to as dewetted growth, is a Bridgman crystal growth process in which the melt is in contact with the crucible wall but the crystal is not. A meniscus bridges the gap between the top of the crystal and the crucible wall. The Young-Laplace capillary equation was used to calculate the crystal radii of detached states as a function of the pressure differential across the meniscus. The detached states depend on the contact angle of the melt with the crucible wall, the growth angle of the melt with respect to the solidifying crystal, and the Bond number. A static stability analysis was performed on the calculated detached states. The stability criterion was the sign of the second variation of the potential energy upon admissible meniscus shape perturbations. The conditions considered corresponded to the growth of Ge and InSb, in both terrestrial and microgravity conditions. Stability was found to depend significantly on whether the interior surface was considered to be microscopically rough or smooth, corresponding to pinned or unpinned states. It was also found that all meniscus shapes which are single-valued functions of the radius are statically stable in a microgravity environment.

Volz, M. P.↗
NASA NTRS2002

Evaluation of Standard Gear Metrics in Helicopter Flight Operation

Each false alarm made by a machine monitoring system carries a high price tag. The machine must be taken out of service, thoroughly inspected with possible disassembly, and then made ready for service. Loss of use of the machine and the efforts to inspect it are costly. In addition, if a monitoring system is prone to false alarms, the system will soon be turned off or ignored. For aircraft applications, one growing concern is that the dynamic flight environment differs from the laboratory environment where fault detection methods are developed and tested. Vibration measurements made in flight are less stationary than those made in a laboratory, or test facility, and thus a given fault detection method may produce more false alarms in flight than might be anticipated. In 1977. Stewart introduced several metrics, including FM0 and FM4, for evaluating the health of a gear. These metrics are single valued functions of the vibration signal that indicate if the signal deviates from an ideal model of the signal. FM0 is a measure of the ratio of the peak-to-peak level to the harmonic energy in the signal. FM4 is the kurtosis of the signal with the gear mesh harmonics and first order side bands removed. The underlying theory is that a vibration signal from a gear in good condition is expected to be dominated by a periodic signal at the gear mesh frequency. If one or a small number of gear teeth contain damage or faults, the signal will change, possibly showing increased amplitude, local phase changes or both near the damaged region of the gear. FM0 increases if a signal contains a local increase in amplitude. FM4 increases if a signal contains a local increase in amplitude or local phase change in a periodic signal. Over the years, other single value metrics were also introduced to detect the onset and growth of damage in gears. These various metrics have detected faults in several gear tests in experimental test rigs. Conditions in these tests have been steady state in the sense that the rpm, torque and forces on the gear have been held steady. For gears used in a dynamic environment such as that occurring in aircraft, the rpm, torque and forces on the gear are constantly changing. The authors have measured significant variation in rpm and torque in the transmissions of helicopters in controlled steady flight conditions flown by highly proficient test pilots. Statistical analyses of the data taken in flight show significant nonstationarity in the vibration measurements. These deviations from stationarity may increase false alarms in gear monitoring during aircraft flight. In the proposed paper, the authors will study vibration measurements made in flight on an AH- 1 Cobra and an OH-58C Kiowa helicopters. The primary focus will be the development of a methodology to assess the impact of nonstationarity on false alarms. Issues to be addressed include how time synchronous averages are constructed from raw data as well as how lack of stationarity effects the behavior of single value metrics. Emphasis will be placed on the occurrence of false alarms with the use of standard metrics. In order to maintain an acceptable level of false alarms in the flight environment, this study will also address the determination of appropriate threshold levels, which may need to be higher than for test rigs.

Mosher, M.↗
DOE OSTI2025

Realizing tunable Fermi level in SnTe by defect control

The tuning of the Fermi level in tin telluride, a topological crystalline insulator, is essential for accessing its unique surface states and optimizing its electronic properties for applications such as spintronics and quantum computing. In this study, we demonstrate that the Fermi level in tin telluride can be effectively modulated by controlling the tin concentration during chemical vapor deposition synthesis. By introducing tin-rich conditions, we observed a blue shift in the x-ray photoelectron spectroscopy core-level peaks of both tin and tellurium, indicating an upward shift in the Fermi level. Further, this shift is corroborated by a decrease in work function values measured via ultraviolet photoelectron spectroscopy, confirming the suppression of Sn vacancies. Our findings provide a low-cost, scalable method to achieve tunable Fermi levels in tin telluride, offering a significant advancement in the development of materials with tailored electronic properties for next-generation technological applications.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
NASA NTRS2007

Identification and Control of Aircrafts using Multiple Models and Adaptive Critics

We compared two possible implementations of local linear models for control: one approach is based on a self-organizing map (SOM) to cluster the dynamics followed by a set of linear models operating at each cluster. Therefore the gating function is hard (a single local model will represent the regional dynamics). This simplifies the controller design since there is a one to one mapping between controllers and local models. The second approach uses a soft gate using a probabilistic framework based on a Gaussian Mixture Model (also called a dynamic mixture of experts). In this approach several models may be active at a given time, we can expect a smaller number of models, but the controller design is more involved, with potentially better noise rejection characteristics. Our experiments showed that the SOM provides overall best performance in high SNRs, but the performance degrades faster than with the GMM for the same noise conditions. The SOM approach required about an order of magnitude more models than the GMM, so in terms of implementation cost, the GMM is preferable. The design of the SOM is straight forward, while the design of the GMM controllers, although still reasonable, is more involved and needs more care in the selection of the parameters. Either one of these locally linear approaches outperform global nonlinear controllers based on neural networks, such as the time delay neural network (TDNN). Therefore, in essence the local model approach warrants practical implementations. In order to call the attention of the control community for this design methodology we extended successfully the multiple model approach to PID controllers (still today the most widely used control scheme in the industry), and wrote a paper on this subject. The echo state network (ESN) is a recurrent neural network with the special characteristics that only the output parameters are trained. The recurrent connections are preset according to the problem domain and are fixed. In a nutshell, the states of the reservoir of recurrent processing elements implement a projection space, where the desired response is optimally projected. This architecture trades training efficiency by a large increase in the dimension of the recurrent layer. However, the power of the recurrent neural networks can be brought to bear on practical difficult problems. Our goal was to implement an adaptive critic architecture implementing Bellman s approach to optimal control. However, we could only characterize the ESN performance as a critic in value function evaluation, which is just one of the pieces of the overall adaptive critic controller. The results were very convincing, and the simplicity of the implementation was unparalleled.

Principe, Jose C.↗
DOE OSTI2021

Infinite-dimensional fermionic symmetry in supersymmetric gauge theories

We establish the existence of an infinite-dimensional fermionic symmetry in four-dimensional supersymmetric gauge theories by analyzing semiclassical photino dynamics in abelian $\mathscr{N}$ = 1 theories with charged matter. The symmetry is parametrized by a spinor-valued function on an asymptotic S2 at null infinity. It is not manifest at the level of the Lagrangian, but acts non-trivially on physical states, and its Ward identity is the soft photino theorem. The infinite-dimensional fermionic symmetry resides in the same $\mathscr{N}$ = 1 supermultiplet as the physically non-trivial large gauge symmetries associated with the soft photon theorem.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

A matheuristic for design and dispatch of a utility-connected distributed energy system

Modeling distributed power generation systems often requires complicated mathematical expressions that present challenges for commercial optimization solvers. Here, this paper presents a matheuristic to solve a mixed-integer optimization model that informs decisions regarding the design and dispatch of a utility-connected microgrid. We deploy a genetic algorithm to search the system design space and a linear program to solve the economic dispatch problem. The model is a component of a web tool that requires solutions within a few minutes. Our method yields objective function values within 5% of an exogenously produced optimal in fewer than 30 seconds for 90% of our test cases compared to only 10% of our test cases by a traditional optimization solver in the same amount of time.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Be Surface Structures on W(110) and W(211): A DFT Study

Beryllium and Tungsten are promising candidates for use as plasma facing materials (PFMs) in upcoming fusion reactors. Many complex and competing phenomena however complicate the understanding and development of materials that operate in the harsh conditions of the reactor. In particular, redeposition of Be onto the W divertor must be considered due to the expected erosion of the Be first wall under exposure to the plasma. It is known that a build up of Be on W allows for the formation of BeW alloys which can harm the longevity and performance of the fusion divertor. In an effort to understand the interaction of Be with W surfaces, a study of Be structures on W(110) and W(211) as a function of Be coverage has been carried out using Density Functional Theory. We have found that both W surfaces develop monolayers of Be characterized by a densely packed hexagonal structure. Below this monolayer coverage, Be structures are found to have two motifs of coverage, a densely packed hexagonal pattern punctuated by areas of low-density coverage. The structures found here produce work function values and trends in good agreement with experimental measurements.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗