Streamlining Gate Set Tomography Experiment Designs Using Optimal Design Theory
Explore the source record for details and available documents.
Ostrove, Corey Isaac↗
SEARCH · Search NASA
Results for “optimal experiment design”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Method for optimization of experiment design in aerosol size distribution measurement using multi-angle light scattering
Explore the source record for details and available documents.
Seetala, Kiran Sai↗
Optimal Experimental Design With Fast Neural Network Surrogate Models
Designing optimal experiments minimizes the uncertainty of results and maximizes the efficient use of resources. Herein, machine learning surrogate models and the approximate coordinate exchange (ACE) algorithm are used to determine optimum experimental designs over large or arbitrarily restrictive design spaces. Optimal experimental design is particularly salient in materials science where experiments are expensive and material properties must often be inferred indirectly. The proposed framework is demonstrated by finding optimal experiments with which the hidden constituent properties of composite materials can be most efficiently inferred from observable experimental outcomes. The optimum experimental design is given by an information-theoretic criteria, which maximizes the conditional mutual information between the hidden properties and the expected experimental outcomes. To perform tractable optimization a neural network is trained as a surrogate model to mimic a physics based simulation, which can calculate the expected experimental outcome based on a candidate experimental design and sampled constituent properties. The ACE algorithm is used to optimize over large design spaces with many tests and controlled parameters where an exhaustive search would be intractable even with the surrogate model. Using this approach, optimal experimental designs that are consistent with those produced by heuristic knowledge and established best practices are found; then optimal designs in larger design spaces where heuristic knowledge is unavailable are examined.
machine learning↗
Use of Taguchi design of experiments to optimize and increase robustness of preliminary designs
The research performed this summer includes the completion of work begun last summer in support of the Air Launched Personnel Launch System parametric study, providing support on the development of the test matrices for the plume experiments in the Plume Model Investigation Team Project, and aiding in the conceptual design of a lunar habitat. After the conclusion of last years Summer Program, the Systems Definition Branch continued with the Air Launched Personnel Launch System (ALPLS) study by running three experiments defined by L27 Orthogonal Arrays. Although the data was evaluated during the academic year, the analysis of variance and the final project review were completed this summer. The Plume Model Investigation Team (PLUMMIT) was formed by the Engineering Directorate to develop a consensus position on plume impingement loads and to validate plume flowfield models. In order to obtain a large number of individual correlated data sets for model validation, a series of plume experiments was planned. A preliminary 'full factorial' test matrix indicated that 73,024 jet firings would be necessary to obtain all of the information requested. As this was approximately 100 times more firings than the scheduled use of Vacuum Chamber A would permit, considerable effort was needed to reduce the test matrix and optimize it with respect to the specific objectives of the program. Part of the First Lunar Outpost Project deals with Lunar Habitat. Requirements for the habitat include radiation protection, a safe haven for occasional solar flare storms, an airlock module as well as consumables to support 34 extra vehicular activities during a 45 day mission. The objective for the proposed work was to collaborate with the Habitat Team on the development and reusability of the Logistics Modules.
Carrasco, Hector R.↗
Constrained optimization using design of experiment surfaces
An algorithm for solving constrained optimization problems is presented. First, design of experiment techniques are used to survey the design space. After evaluating the objective and constraint functions, as specified by Taguchi orthogonal arrays, analytical models of these functions are generated using a least-squares regression analysis. Next, a nonlinear programming package is used to optimize the analytical model. Based on the optimization information, the design space is reduced so as to close in around the minimum, and the entire procedure is repeated until convergence. An important feature of the algorithm is that function gradients are not required; therefore, for problems in which gradients would have to be estimated using finite-differences the number of function evaluations required for the optimization is significantly reduced, when compared with traditional nonlinear programming techniques. In addition, there is no requirement that the gradients must be smooth and continuous.
Bolt, Marvin Vance↗
Optimal experimental design using eigenvalue-based criteria with Pyomo.DoE
New developments in automated optimal experimental design within the PSE+ software ecosystem. Advancements in user experience (to reduce the time taken to perform optimal experiment design) and computational capabilities (allowing more diverse experimental design) are shown with an example relevant to critical minerals and materials. Also, a small tutorial on science-based optimal experimental design and novel contributions therein are presented.
97 MATHEMATICS AND COMPUTING↗
Bayesian OED for Seismic Monitoring
SAND2024-13870O The Bayesian OED (Optimal Experiment Design) for Seismic Monitoring code provides the tools to analyze and optimize seismic monitoring networks using Bayesian OED. This method designs a utility function for experiments (network designs) using network analysis and network optimization. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.
Catanach, Thomas↗
Modal identification experiment design for large space structures
This paper describes an on-orbit modal identification experiment design for large space structures. Space Station Freedom (SSF) systems design definition and structural dynamic models were used as representative large space structures for optimizing experiment design. Important structural modes of study models were selected to provide a guide for experiment design and used to assess the design performance. A pulsed random excitation technique using propulsion jets was developed to identify closely-spaced modes. A measuremenat location selection approach was developed to estimate accurate mode shapes as well as frequencies and damping factors. The data acquisition system and operational scenarios were designed to have minimal impacts on the SSF. A comprehensive simulation was conducted to assess the overall performance of the experiment design.
Kim, Hyoung M.↗
Neural message-passing for objective-based uncertainty quantification and optimal experimental design
Various real-world scientific applications involve the mathematical modeling of complex uncertain systems with numerous unknown parameters. Accurate parameter estimation is often practically infeasible in such systems, as the available training data may be insufficient and the cost of acquiring additional data may be high. In such cases, based on a Bayesian paradigm, we can design robust operators retaining the best overall performance across all possible models and design optimal experiments that can effectively reduce uncertainty to enhance the performance of such operators maximally. While objective-based uncertainty quantification (objective-UQ) based on MOCU (mean objective cost of uncertainty) provides an effective means for quantifying uncertainty in complex systems, the high computational cost of estimating MOCU has been a challenge in applying it to real-world scientific/engineering problems. In this work, we propose a novel scheme to reduce the computational cost for objective-UQ via MOCU based on a data-driven approach. We adopt a neural message-passing model for surrogate modeling, incorporating a novel axiomatic constraint loss that penalizes an increase in the estimated system uncertainty. As an illustrative example, we consider the optimal experimental design (OED) problem for uncertain Kuramoto models, where the goal is to predict the experiments that can most effectively enhance robust synchronization performance through uncertainty reduction. We show that our proposed approach can accelerate MOCU-based OED by four to five orders of magnitude, without any visible performance loss compared to the state-of-the-art. The proposed approach applies to general OED tasks, beyond the Kuramoto model.
97 MATHEMATICS AND COMPUTING↗
Numerical Optimization Using Computer Experiments
Engineering design optimization often gives rise to problems in which expensive objective functions are minimized by derivative-free methods. We propose a method for solving such problems that synthesizes ideas from the numerical optimization and computer experiment literatures. Our approach relies on kriging known function values to construct a sequence of surrogate models of the objective function that are used to guide a grid search for a minimizer. Results from numerical experiments on a standard test problem are presented.
Trosset, Michael W.↗
Optimal Design of Calibration Signals in Space Borne Gravitational Wave Detectors
Future space borne gravitational wave detectors will require a precise definition of calibration signals to ensure the achievement of their design sensitivity. The careful design of the test signals plays a key role in the correct understanding and characterization of these instruments. In that sense, methods achieving optimal experiment designs must be considered as complementary to the parameter estimation methods being used to determine the parameters describing the system. The relevance of experiment design is particularly significant for the LISA Pathfinder mission, which will spend most of its operation time performing experiments to characterize key technologies for future space borne gravitational wave observatories. Here we propose a framework to derive the optimal signals in terms of minimum parameter uncertainty to be injected to these instruments during its calibration phase. We compare our results with an alternative numerical algorithm which achieves an optimal input signal by iteratively improving an initial guess. We show agreement of both approaches when applied to the LISA Pathfinder case.
LISA Path_nder mission↗
Optimizing ablator thickness for laser shock experiments
In laser shock experiments, a well-defined, flat-top shock wave at the ablator/sample interface is important for accurately probing material response under uniaxial strain compression. However, the relationship between the ablator thickness and the resulting shock wave characteristics is insufficiently understood, limiting the ability to design optimal experiments. To address this need, we conducted a systematic experimental study using a 100 J laser to shock-compress polyimide ablators to peak stresses ranging from 20.4 to 111.6 GPa. Laser interferometry diagnostics measured the transmitted wave profiles at the ablator/sample interface, consistently showing a single jump followed by a constant peak state before the arrival of release waves. Here, by analyzing shock transit time, flat-top duration, and stress, our results establish a framework for selecting ablator thickness to maximize the flat-top duration, improving the precision and reproducibility of laser shock experiments.
design of experiments↗
Sensitivity Coefficients Calculated for the Prompt Neutron Decay Constant At or Near Delayed Critical
The derivation of a non-invasive prompt neutron decay constant sensitivity coefficient is provided in this work. The computation of the sensitivity coefficient derived in this work does not require modification of Monte Carlo source code and is based on capabilities available in Monte Carlo N-Particle R© Code Version 6.2. The prompt neutron decay constant sensitivity coefficients are calculated for 44-group and 252-group energy structures for specific nuclide-reaction pairs in the Jezebel benchmark experiment. The nuclide-reaction pairs investigated in this work include Pu- 239(n,f), Pu-240(n,f), and Pu-241(n,f). Physical explanations of the sensitivity profiles exhibited by the 252-group energy structure are investigated for the prompt neutron multiplication factor, mean neutron lifetime, and prompt neutron decay constant. The prompt neutron decay constant sensitivity coefficients calculated for the 44-group and 252-group energy structure of Pu-239(n,f) are compared. Lastly, the 44-group energy structure sensitivity coefficients calculated are used for nuclear-data induced uncertainty quantification of the neutron multiplication factor. This work shows that a reduction in the nuclear data-induced uncertainty of the neutron multiplication factor is possible for all nuclide-reaction pairs investigated when prompt neutron decay constant sensitivity coefficients are utilized. This is important for new critical experiment design optimization studies of measurement configurations. This work provides a basis for more detailed sensitivity analysis and uncertainty quantification of nuclide, reaction, and energy-specific cross section data.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Massively Parallel Bayesian Model Calibration and Uncertainty Quantification with Applications to Nuclear Fuels and Materials
The U.S. Department of Energy (DOE)’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop predictive capabilities by applying computational methods to the analysis and design of advanced reactor and fuel cycle systems. This program has been providing engineering-scale support for the development of BISON, a high-fidelity and high-resolution fuel performance tool. Fuel behavior in a nuclear reactor is governed by a complex network of mechanisms interacting with various other physics aspects in the reactor system. Any model developed to represent the fuel behavior will likely be idealized resulting in uncertainties in their predictions compared to the observed data. As such, this report was motivated by the need to identify the sources of uncertainties and quantify and propagate them through the fuel model outputs. Such quantification of uncertainties will establish a level of model trustworthiness, identify approaches to improve the model trustworthiness, and even guide optimal experiment design for maximal information gain. To accomplish the uncertainty quantification for computational models, this report has relied on the Bayesian framework which provides probabilistic treatment of models their inputs and outputs. The current state-of-the-art on performing Bayesian Uncertainty Quantification (UQ) for nuclear engineering models using High Performance Computing (HPC) resources have been reviewed. Implementation of capabilities for massively parallel Bayesian UQ in Multiphysics Object-Oriented Simulation Environment (MOOSE) is discussed. Several verification cases are discussed to verify the accuracy of the quantified uncertainties using the developed computational capabilities in MOOSE. Then, the problem of quantifying the uncertainties in TRI-Structural isOtropic (TRISO) fuel silver release is addressed. For the first time, the uncertainties arising from the TRISO Fission Gas Release (FGR) model due to model inadequacy and experimental noise are quantified. Also, the Bayesian capabilities are applied to the calibration of the MATPRO creep model, a widely used model in several fuel assessment cases. The impact of the prediction uncertainties in the MATPRO model on the fuel cladding behavior as part of the TRIBULATION assessment case (which is an integral effects case) is investigated. This report concludes with a discussion on the future work for the UQ for computational models.
11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
On-the-fly closed-loop materials discovery via Bayesian active learning
Active learning—the field of machine learning (ML) dedicated to optimal experiment design—has played a part in science as far back as the 18th century when Laplace used it to guide his discovery of celestial mechanics. In this work, we focus a closed-loop, active learning-driven autonomous system on another major challenge, the discovery of advanced materials against the exceedingly complex synthesis-processes-structure-property landscape. We demonstrate an autonomous materials discovery methodology for functional inorganic compounds which allow scientists to fail smarter, learn faster, and spend less resources in their studies, while simultaneously improving trust in scientific results and machine learning tools. This robot science enables science-over-the-network, reducing the economic impact of scientists being physically separated from their labs. The real-time closed-loop, autonomous system for materials exploration and optimization (CAMEO) is implemented at the synchrotron beamline to accelerate the interconnected tasks of phase mapping and property optimization, with each cycle taking seconds to minutes. We also demonstrate an embodiment of human-machine interaction, where human-in-the-loop is called to play a contributing role within each cycle. This work has resulted in the discovery of a novel epitaxial nanocomposite phase-change memory material.
36 MATERIALS SCIENCE↗
Quantifying the Multi-Objective Cost of Uncertainty
Various real-world applications involve modeling complex systems with immense uncertainty and optimizing multiple objectives based on the uncertain model. Quantifying the impact of the model uncertainty on the given operational objectives is critical for designing optimal experiments that can most effectively reduce the uncertainty that affect the objectives pertinent to the application at hand. In this paper, we propose the concept of mean multi-objective cost of uncertainty (multi-objective MOCU) that can be used for objective-based quantification of uncertainty for complex uncertain systems considering multiple operational objectives. We provide several illustrative examples that demonstrate the concept and strengths of the proposed multi-objective MOCU. Furthermore, we present a real-world example based on the mammalian cell cycle network to demonstrate how the multi-objective MOCU can be used for quantifying the operational impact of model uncertainty when there are multiple, possibly competing, objectives.
42 ENGINEERING↗
Uncertainty amplification due to density/refractive index gradients in background-oriented schlieren experiments
Here, we theoretically analyze the effect of density/refractive index gradients on the measurement precision of background-oriented schlieren (BOS) experiments by deriving the Cramer–Rao lower bound (CRLB) for the 2D centroid estimation process. A model is derived for the diffraction limited image of a dot viewed through a medium containing density gradients that includes the effect of the experimental parameters such as the magnification and f-number. It is shown using the model that nonlinearities in the density gradient field lead to blurring of the dot image. This blurring amplifies the effect of image noise on the centroid estimation process, leading to an increase in the CRLB and a decrease in the measurement precision. The ratio of position uncertainties of a dot in the reference and gradient images is shown to be a function of the ratio of the dot diameters and dot intensities. We termed this parameter the amplification ratio (A F ), and a methodology for reporting position uncertainties in tracking-based BOS measurements is proposed. The theoretical predictions of the dot position estimation variance from the CRLB are compared to ray tracing simulations, and agreement is obtained. The uncertainty amplification is also demonstrated on experimental BOS images of flow induced by a spark discharge, where it is seen that regions of high amplification ratio correspond to regions of density gradients. This analysis elucidates the dependence of the position uncertainty on density and refractive index gradient-induced distortion parameters, provides a methodology for accounting its effect on uncertainty quantification and provides a framework for optimizing experiment design.
42 ENGINEERING↗