Search NASA⌕ Search

SEARCH · Search NASA

Results for “Computational optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Open-loop control of noise amplification in a separated boundary layer flow

Linear optimal gains are computed for the subcritical two-dimensional separated boundary-layer flow past a bump. Very large optimal gain values are found, making it possible for small-amplitude noise to be strongly amplified and to destabilize the flow. The optimal forcing is located close to the summit of the bump, while the optimal response is the largest in the shear layer. The largest amplification occurs at frequencies corresponding to eigenvalues which first become unstable at higher Reynolds number. Nonlinear direct numerical simulations show that a low level of noise is indeed sufficient to trigger random flow unsteadiness, characterized here by large-scale vortex shedding. Next, a variational technique is used to compute efficiently the sensitivity of optimal gains to steady control (through source of momentum in the flow, or blowing/suction at the wall). A systematic analysis at several frequencies identifies the bump summit as the most sensitive region for control with wall actuation. Based on these results, a simple open-loop control strategy is designed, with steady wall suction at the bump summit. Linear calculations on controlled base flows confirm that optimal gains can be drastically reduced at all frequencies. Nonlinear direct numerical simulations also show that this control allows the flow to withstand a higher level of stochastic noise without becoming nonlinearly unstable, thereby postponing bypass transition. In the supercritical regime, sensitivity analysis of eigenvalues supports the choice of this control design. Full restabilization of the flow is obtained, as evidenced by direct numerical simulations and linear stability analysis.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Classical combinatorial optimization scaling for random Ising models on 2D heavy-hex graphs

Motivated by near term quantum computing hardware limitations, combinatorial optimization problems that can be addressed by current quantum algorithms and noisy hardware with little or no overhead are used to probe capabilities of quantum algorithms such as the quantum approximate optimization algorithm. In this study, a specific class of near term quantum computing hardware defined combinatorial optimization problems, Ising models on heavy-hex graphs both with and without geometrically local cubic terms, are examined for their classical computational hardness via empirical computation time scaling quantification. Specifically the time-to-solution (TTS) metric using the classical heuristic simulated annealing is measured for finding optimal variable assignments (ground states), as well as the time required for the optimization software Gurobi to find an optimal variable assignment. Because of the sparsity of these Ising models, the classical algorithms are able to find optimal solutions efficiently even for large instances (i.e. 100 000 spin variables). The Ising models both with and without geometrically local cubic terms exhibit average-case linear-time or weakly quadratic scaling when solved exactly using Gurobi, and the Ising models with no cubic terms show evidence of exponential-time TTS scaling when sampled using simulated annealing. These findings point to the necessity of developing and testing more complex, namely more densely connected, optimization problems in order for quantum computing to ever have a practical advantage over classical computing. Our results are another illustration that different classical algorithms can indeed have exponentially different running times, thus making the identification of the best practical classical technique important in any quantum computing vs. classical computing comparison.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Computational Tool Compatible with NEAMS Code Packages for Optimizing the Shape of Nuclear Reactor Components and of Whole Core Performance

We designed and implemented a shape optimization tool that functions with NEAMS codes, and that nuclear scientists and engineers can employ to optimize the shape of individual components and the whole core under the applicable single- or multi-physics model comprising the employed code(s). The shape-optimization tool enables varying the geometric shape itself as well as its dimensions to yield, potentially, new component designs that are not limited by the designer’s intuition and previous experience. In cases where the optimal-shape object is an individual component, we provide the capability for additional verification that the whole-core performance using the optimized component performs better, under the prescribed optimization criteria, than the initial design. Our shape-optimization tool couples to NEAMS codes via a flexible input- composer interface and enables the user to constrain the shape’s evolution to ensure the component’s manufacturability. Finally, we demonstrate our shape-optimization tool with single- and multi-physics NEAMS codes. This objective is motivated by the recent advances in manufacturing technology that, combined with rising interest in novel reactor concepts, are creating new opportunities for innovation in the design of individual components that affect the performance of the full reactor system. In particular, Additive Manufacturing (AM) enables mass production of highly precise, intricate and complex component shapes that are not feasible with traditional manufacturing techniques. To accomplish this goal we developed and implemented in MOOSE: (1) discrete shape optimization capability based on a state-space search that uses Artificial Intelligence strategies to find the optimal state/shape; (2) smooth shape optimization tool that employs PETSc’s toolkit for advanced optimization (TAO) to optimize node-displacement of the components’ model sidesets; (3) hierarchical core optimization workflow that recognizes the repeating patterns typical in a nuclear reactor and performs the optimization one level at a time with increasing length scale. Each of these tools is equipped with user-specified constraints to avoid optimal shapes that are not manufacturable. The developed shape optimization tool is verified and demonstrated on various nuclear reactor core components and models. The optimization process accounts for tightly coupled physics that govern the behavior of these target reactors, and exercises several NEAMS codes in a coupled multiphysics fashion. The impact of the delivered shape optimization tool will materialize in the optimal design, from the outset, of advanced reactors currently contemplated to regain the US’s leadership in nuclear energy R&D. Novel reactor concepts, e.g. Molten Salt Reactors, and sizes/capacities, e.g. micro- reactors, provide a unique opportunity to optimize performance from the early stages of development, before the investment in components’ production lines, validation experiments, and licensing regimes make future improvements in performance prohibitively expensive and force sub-optimal performance on the affected reactor concept in perpetuity. This benefit will be realized by the delivered shape optimization tool regardless of the applicable manufacturing process whether traditional or AM, thereby broadening the impact of this project on current and future reactor concepts and technologies

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Direct computation of magnetic surfaces in Boozer coordinates and coil optimization for quasisymmetry

We propose a new method to compute magnetic surfaces that are parametrized in Boozer coordinates for vacuum magnetic fields. We also propose a measure for quasisymmetry on the computed surfaces and use it to design coils that generate a magnetic field that is quasisymmetric on those surfaces. The rotational transform of the field and complexity measures for the coils are also controlled in the design problem. Using an adjoint approach, we are able to obtain analytic derivatives for this optimization problem, yielding an efficient gradient-based algorithm. Starting from an initial coil set that presents nested magnetic surfaces for a large fraction of the volume, our method converges rapidly to coil systems generating fields with excellent quasisymmetry and low particle losses. In particular for low complexity coils, we are able to significantly improve the performance compared with coils obtained from the standard two-stage approach, e.g. reduce losses of fusion-produced alpha particles born at half-radius from $17.7\,\%$ to $6.6\,\%$ . We also demonstrate 16-coil configurations with alpha loss ${<}1\,\%$ and neoclassical transport magnitude $\epsilon _{\text {eff}}^{3/2}$ less than approximately $5\times 10^{-9}$ .

Physics↗

On-line optimization scheme for HVAC demand response

A computer-implemented method of optimizing demand-response (DR) of a heating, ventilation, and air-conditioning (HVAC) system of a building, includes determining (30, 31, 32) a value of an objective function F.sub.ij of a HVAC system for each of a plurality of DR strategies j for each of a plurality of weather patterns i that is a weighted sum of an energy cost of the HVAC system and a thermal comfort loss of the HVAC system, assigning (33, 34, 35, 36) a likelihood score L.sub.i,j to each of a selected subset of near-optimal DR strategies j for each weather pattern i, and selecting (37, 38) those near-optimal DR strategies with large overall likelihood scores L.sub.j to create an optimal strategy pool of DR strategies. An optimal strategy pool can be searched (39) in real-time for an optimal DR strategy for a given weather pattern.

42 ENGINEERING↗

Computational and numerical analysis of AC optimal power flow formulations on large-scale power grids

We report that alternating current optimal power flow (AC-OPF) is a fundamental tool in electric utilities to determine optimal operation of the various resources. Typically, the AC-OPF problem uses power balance formulation containing voltages and power equations. Yet, there is no comprehensive comparison of the different AC-OPF formulations, especially for large-scale networks. This paper presents a detailed comparative evaluation of different formulations of the AC-OPF problem on networks ranging from 9-bus to 25,000 buses. Three different formulations: 1) power balance with polar voltages, 2) power balance with Cartesian voltages, and 3) current balance with Cartesian voltages are discussed in detail by comparing their characteristics, and numerical and computational performance.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Route-Optimized Energy Management of Connected and Automated Multi-Mode Plug-In Hybrid Electric Vehicle Using Dynamic Programming

This paper presents a methodology to optimize the blending of charge-depleting (CD) and charge-sustaining (CS) modes in a multi-mode plug-in hybrid electric vehicle (PHEV) that reduces overall energy consumption when the selected route cannot be completely driven in all-electric mode. The PHEV used in this investigation is the second-generation Chevrolet Volt and as many as four instrumented vehicles were utilized simultaneously on road to acquire validation data. The optimization method used is dynamic programming (DP) paired with a reduced-order powertrain model to enable onboard embedded controller compatibility and computational efficiency in optimally blending CD, CS modes over the entire drive route. The objective of the optimizer is to enable future Connected and Automated Vehicles (CAVs) to best utilize onboard energy for minimum overall energy consumption based on speed and elevation profile information from Intelligent Transportation Systems (ITS), Internet of Things (IoT), High-definition Mapping, and onboard sensing technologies. Emphasis is placed on runtime minimization to quickly react and plan an optimal mode scheme in highly dynamic road conditions with minimal computational resources. On-road performance of the optimizer paired with automated CD and CS mode selection is evaluated on a fleet of four instrumented Chevrolet Volts in a variety of driving scenarios. Here, the results indicate variable energy savings depending on the drive route and initial battery SOC with potential ranging between 2 to 12% and less than 7 seconds initial optimization for a 24-mile drive cycle.

33 ADVANCED PROPULSION SYSTEMS↗

Initial-State Dependent Optimization of Controlled Gate Operations with Quantum Computer

There is no unique way to encode a quantum algorithm into a quantum circuit. With limited qubit counts, connectivity, and coherence times, a quantum circuit optimization is essential to make the best use of near-term quantum devices. We introduce a new circuit optimizer called AQCEL, which aims to remove redundant controlled operations from controlled gates, depending on initial states of the circuit. Especially, the AQCEL can remove unnecessary qubit controls from multi-controlled gates in polynomial computational resources, even when all the relevant qubits are entangled, by identifying zero-amplitude computational basis states using a quantum computer. As a benchmark, the AQCEL is deployed on a quantum algorithm designed to model final state radiation in high energy physics. For this benchmark, we have demonstrated that the AQCEL-optimized circuit can produce equivalent final states with much smaller number of gates. Moreover, when deploying AQCEL with a noisy intermediate scale quantum computer, it efficiently produces a quantum circuit that approximates the original circuit with high fidelity by truncating low-amplitude computational basis states below certain thresholds. Our technique is useful for a wide variety of quantum algorithms, opening up new possibilities to further simplify quantum circuits to be more effective for real devices.

97 MATHEMATICS AND COMPUTING↗

Multi-cycle boiling water reactor fuel cycle optimization

In this work a new computer code, BWROPT (Boiling Water Reactor Optimization), is presented. BWROPT uses the Parallel Simulated Annealing (PSA) algorithm to solve the out-of-core optimization problem coupled with an in-core optimization that determines the optimum fuel loading pattern. However it uses a Haling power profile for the depletion instead of optimizing the operating strategy. The result of this optimization is the optimum new fuel inventory and the core loading pattern for the first cycle considered in the optimization. Several changes were made to the optimization algorithm with respect to other nuclear fuel cycle optimization codes that use PSA. Instead of using constant sampling probabilities for the solution perturbation types throughout the optimization as is usually done in PSA optimizations the sampling probabilities are varied to get a better solution and/or decrease runtime. The new fuel types available for use can be sorted into an array based on any number of parameters so that each parameter can be incremented or decremented, which allows for more precise fuel type selection compared to random sampling. Also, the results are sorted by the new fuel inventory of the first cycle for ease of comparing alternative solutions. (authors)

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Metal Big Area Additive Manufacturing: Process Modeling and Validation

Metal Big Area Additive Manufacturing (mBAAM) is a new additive manufacturing (AM) technology for printing large-scale 3D objects. mBAAM is based on the gas metal arc welding process and uses a continuous feed of welding wire to manufacture an object. An electric arc forms between the wire and the substrate, which melts the wire and deposits a bead of molten metal along the predetermined path. In general, the welding process parameters and local conditions determine the shape of the deposited bead. The sequence of the bead deposition and the corresponding thermal history of the manufactured object determine the long range effects, such as thermal-induced distortions and residual stresses. Therefore, the resulting performance or final properties of the manufactured object are dependent on its geometry and the deposition path, in addition to depending on the basic welding process parameters. Physical testing is critical for gaining the necessary knowledge for quality prints, but traversing the process parameter space in order to develop an optimized build strategy for each new design is impractical by pure experimental means. Computational modeling and optimization may accelerate development of a build process strategy and saves time and resources. Because computational modeling provides these opportunities, we have developed a physics-based Finite Element Method (FEM) simulation framework and numerical models to support the mBAAM process s development and design. In this paper, we performed a sequentially coupled heat transfer and stress analysis for predicting the final deformation of a small rectangular structure printed using the mild steel welding wire. Using the new simulation technologies, material was progressively added into the FEM simulation as the arc weld traversed the build path. In the sequentially coupled heat transfer and stress analysis, the heat transfer was performed to calculate the temperature evolution, which was used in a stress analysis to evaluate the residual stresses and distortions. In this formulation, we assume that physics is directionally coupled, i.e. the effect of stress of the component on the temperatures is negligible. The experiment instrumentation (measurement types, sensor types, sensor locations, sensor placements, measurement intervals) and the measurements are presented. The temperatures and distortions from the simulations show good correlation with experimental measurements. Ongoing modeling work is also briefly discussed.

Additive Manufacturing↗

Towards large-scale quantum optimization solvers with few qubits

Quantum computers hold the promise of more efficient combinatorial optimization solvers, which could be game-changing for a broad range of applications. However, a bottleneck for materializing such advantages is that, in order to challenge classical algorithms in practice, mainstream approaches require a number of qubits prohibitively large for near-term hardware. Here we introduce a variational solver for MaxCut problems over $m={{\mathcal{O}}}({n}^{k})$ binary variables using only n qubits, with tunable k > 1. The number of parameters and circuit depth display mild linear and sublinear scalings in m , respectively. Moreover, we analytically prove that the specific qubit-efficient encoding brings in a super-polynomial mitigation of barren plateaus as a built-in feature. Altogether, this leads to high quantum-solver performances. For instance, for m = 7000, numerical simulations produce solutions competitive in quality with state-of-the-art classical solvers. In turn, for m = 2000, experiments with n = 17 trapped-ion qubits feature MaxCut approximation ratios estimated to be beyond the hardness threshold 0.941. Our findings offer an interesting heuristics for quantum-inspired solvers as well as a promising route towards solving commercially-relevant problems on near-term quantum devices.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Breaking Computational Barriers: Real-time Analysis and Optimization with Large-scale Nonlinear Models via Model Reduction

Model reduction for dynamical systems is a promising approach for reducing the computational cost of large-scale physics-based simulations to enable high-fidelity models to be used in many- query (e.g., Bayesian inference) and near-real-time (e.g., fast-turnaround simulation) contexts. While model reduction works well for specialized problems such as linear time-invariant systems, it is much more difficult to obtain accurate, stable, and efficient reduced-order models (ROMs) for systems with general nonlinearities. This report describes several advances that enable nonlinear reduced-order models (ROMs) to be deployed in a variety of time-critical settings. First, we present an error bound for the Gauss-Newton with Approximated Tensors (GNAT) nonlinear model reduction technique. This bound allows the state-space error for the GNAT method to be quantified when applied with the backward Euler time-integration scheme. Second, we present a methodology for preserving classical Lagrangian structure in nonlinear model reduction. This technique guarantees that important properties--such as energy conservation and symplectic time-evolution maps--are preserved when performing model reduction for models described by a Lagrangian formalism (e.g., molecular dynamics, structural dynamics). Third, we present a novel technique for decreasing the temporal complexity --defined as the number of Newton-like iterations performed over the course of the simulation--by exploiting time-domain data. Fourth, we describe a novel method for refining projection-based reduced-order models a posteriori using a goal-oriented framework similar to mesh-adaptive h -refinement in finite elements. The technique allows the ROM to generate arbitrarily accurate solutions, thereby providing the ROM with a 'failsafe' mechanism in the event of insufficient training data. Finally, we present the reduced-order model error surrogate (ROMES) method for statistically quantifying reduced- order-model errors. This enables ROMs to be rigorously incorporated in uncertainty-quantification settings, as the error model can be treated as a source of epistemic uncertainty. This work was completed as part of a Truman Fellowship appointment. We note that much additional work was performed as part of the Fellowship. One salient project is the development of the Trilinos-based model-reduction software module Razor , which is currently bundled with the Albany PDE code and currently allows nonlinear reduced-order models to be constructed for any application supported in Albany. Other important projects include the following: 1. ROMES-equipped ROMs for Bayesian inference: K. Carlberg, M. Drohmann, F. Lu (Lawrence Berkeley National Laboratory), M. Morzfeld (Lawrence Berkeley National Laboratory). 2. ROM-enabled Krylov-subspace recycling: K. Carlberg, V. Forstall (University of Maryland), P. Tsuji, R. Tuminaro. 3. A pseudo balanced POD method using only dual snapshots: K. Carlberg, M. Sarovar. 4. An analysis of discrete v. continuous optimality in nonlinear model reduction: K. Carlberg, M. Barone, H. Antil (George Mason University). Journal articles for these projects are in progress at the time of this writing.

97 MATHEMATICS AND COMPUTING↗

A scalable matrix-free spectral element approach for unsteady PDE constrained optimization using PETSc/TAO

In this work, we provide a new approach for the efficient matrix-free application of the transpose of the Jacobian for the spectral element method for the adjoint-based solution of partial differential equation (PDE) constrained optimization. This results in optimizations of nonlinear PDEs using explicit integrators where the integration of the adjoint problem is not more expensive than the forward simulation. Solving PDE constrained optimization problems entails combining expertise from multiple areas, including simulation, computation of derivatives, and optimization. The Portable, Extensible Toolkit for Scientific computation (PETSc) together with its companion package, the Toolkit for Advanced Optimization (TAO), is an integrated numerical software library that contains an algorithmic/software stack for solving linear systems, nonlinear systems, ordinary differential equations, differential algebraic equations, and large-scale optimization problems and, as such, is an ideal tool for performing PDE-constrained optimization. This paper describes an efficient approach in which the software stack provided by PETSc/TAO can be used for large-scale nonlinear time-dependent problems. Time integration can involve a range of high-order methods, both implicit and explicit. The PDE-constrained optimization algorithm used is gradient-based and seamlessly integrated with the simulation of the physical problem.

97 MATHEMATICS AND COMPUTING↗

MO-G-18A-01: Radiation Dose Reducing Strategies in CT, Fluoroscopy and Radiography

Advances in medical x-ray imaging have provided significant benefits to patient care. According to NCRP 160, there are more than 400 million x-ray procedures performed annually in the United States alone that contributes to nearly half of all the radiation exposure to the US population. Similar growth trends in medical x-ray imaging are observed worldwide. Apparent increase in number of medical x-ray imaging procedures, new protocols and the associated radiation dose and risk has drawn considerable attention. This has led to a number of technological innovations such as tube current modulation, iterative reconstruction algorithms, dose alerts, dose displays, flat panel digital detectors, high efficient digital detectors, storage phosphor radiography, variable filters, etc. that are enabling users to acquire medical x-ray images at a much lower radiation dose. Along with these, there are number of radiation dose optimization strategies that users can adapt to effectively lower radiation dose in medical x-ray procedures. The main objectives of this SAM course are to provide information and how to implement the various radiation dose optimization strategies in CT, Fluoroscopy and Radiography. Learning Objectives: To update impact of technological advances on dose optimization in medical imaging. To identify radiation optimization strategies in computed tomography. To describe strategies for configuring fluoroscopic equipment that yields optimal images at reasonable radiation dose. To assess ways to configure digital radiography systems and recommend ways to improve image quality at optimal dose.

60 APPLIED LIFE SCIENCES↗

Development of Multiresolution Capabilities for the Holistic Energy Resource Optimization Network (HERON) tool A progress update

INL researchers work on technoeconomic analyses for integrated energy systems (IES) using the Framework for Optimization of ResourCes and Economics (FORCE). Within FORCE, researchers use the Holistic Energy Resource Optimization Network (HERON) tool to conduct optimization of grid portfolios under uncertain market conditions. These optimizations determine optimal capacities for all IES components and strategies for resource dispatch which maximize some economic metric (e.g., net present value). Resource dispatch occurs on finer timescales (typically hours) and thus are asked to respond to a given time series (e.g. hourly load demand profiles for a grid, or pre-determined electricity prices). Volatile and complex bidding dynamics as well as poorly forecasted weather events within deregulated markets add uncertainty to the time series; FORCE can address this uncertainty by training a reduced order model on historical time series and generate unique synthetic time series which represent individual scenarios or realizations of the market. The IES configuration can be simulated under these different sampled realizations and a stochastic optimization is conducted which optimizes the expected value of the desired economic metric. The training of a synthetic time series generator is limited by the chosen time resolution; dynamics can occur on different time scales. Seasonal demand trends can dominate faster dynamical events (such as power outages from certain sectors or severe weather events) which might not get captured correctly by the trained model. In this report, we investigate different ways of addressing the training and generation of time series on multiple time scales using three main algorithms: wavelet decomposition, dynamic mode decomposition, and generative adversarial networks for time series. We demonstrate a time series analysis that yields information on not just the frequency space but also temporal space: where a fast Fourier transform can provide what frequencies dominate, the new algorithms can provide when the frequencies dominate as well. These analyses can help improve IES optimization by allowing researchers to couple simulations at different timescales when it is most needed - seasonal, day-ahead, and real time optimization - with greater computational efficiency. Future work will include implementation of a subset of the proposed algorithms into the FORCE toolset and application of these analyses into multiple timescale optimization.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Efficient Sampling Allocation Procedures for Optimal Quantile Selection

We propose a dynamic sampling allocation and selection paradigm for finding the alternative with the optimal quantile in a Bayesian framework. Myopic allocation policies (MAPs), analogous to existing methods in classic ranking and selection for selecting the alternative with the optimal mean, and computationally efficient selection policies are derived for selecting the alternative with the optimal quantile. Under certain conditions, we prove that the proposed MAPs and selection procedures are consistent, which means that the best quantile would be eventually correctly selected as the sample size goes to infinity. Numerical experiments demonstrate that the proposed schemes can significantly improve the performance.

Computer Science↗

Design Optimization of Vena Cava Filters: An application to dual filtration devices

Pulmonary embolism (PE) is a significant medical problem that results in over 300,000 fatalities per year. A common preventative treatment for PE is the insertion of a metallic filter into the inferior vena cava that traps thrombi before they reach the lungs. The goal of this work is to use methods of mathematical modeling and design optimization to determine the configuration of trapped thrombi that minimizes the hemodynamic disruption. The resulting configuration has implications for constructing an optimally designed vena cava filter. Computational fluid dynamics is coupled with a nonlinear optimization algorithm to determine the optimal configuration of trapped model thrombus in the inferior vena cava. The location and shape of the thrombus are parameterized, and an objective function, based on wall shear stresses, determines the worthiness of a given configuration. The methods are fully automated and demonstrate the capabilities of a design optimization framework that is broadly applicable. Changes to thrombus location and shape alter the velocity contours and wall shear stress profiles significantly. For vena cava filters that trap two thrombi simultaneously, the undesirable flow dynamics past one thrombus can be mitigated by leveraging the flow past the other thrombus. Streamlining the shape of thrombus trapped along the cava wall reduces the disruption to the flow, but increases the area exposed to abnormal wall shear stress. Computer-based design optimization is a useful tool for developing vena cava filters. Characterizing and parameterizing the design requirements and constraints is essential for constructing devices that address clinical complications. In addition, formulating a well-defined objective function that quantifies clinical risks and benefits is needed for designing devices that are clinically viable.

42 ENGINEERING↗

Bayesian sequential optimal experimental design for nonlinear models using policy gradient reinforcement learning

We present a mathematical framework and computational methods for optimally designing a finite sequence of experiments. This sequential optimal experimental design (sOED) problem is formulated as a finite-horizon partially observable Markov decision process (POMDP) under a Bayesian setting and with information-theoretic utilities. The formulation is general and may accommodate continuous random variables, non-Gaussian posteriors, and nonlinear forward models. The sOED design policy incorporates elements of feedback and lookahead simultaneously, and we show it to generalize the commonly-used batch and greedy design strategies. We solve for the sOED policy using the policy gradient (PG) method from reinforcement learning, and provide a derivation for the PG expression in the sOED context. Adopting an actor-critic approach, the policy and value functions are parameterized using deep neural networks and improved via PG estimates produced from simulated episodes of designs and observations. The new PG-sOED algorithm is first validated on a linear-Gaussian benchmark, and then compared against other design baselines on a sensor movement problem for contaminant source inversion in a convection-diffusion field. As a result, we provide explanation for the policy behaviors using knowledge of the underlying physical process.

97 MATHEMATICS AND COMPUTING↗