Unlocking the potential of metagenomics through replicated experimental design
Abstract not provided
GILBERT, J G↗
SEARCH · Search NASA
Results for “Experimental design”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Robust A-Optimal Experimental Design for Sensor Placement in Bayesian Linear Inverse Problems
Optimal design of experiments for Bayesian inverse problems has recently gained wide popularity and attracted much attention, especially in the computational science and Bayesian inversion communities. An optimal design maximizes a predefined utility function that is formulated in terms of the elements of an inverse problem, an example being optimal sensor placement for parameter identification. The state-of-the-art algorithmic approaches following this simple formulation generally overlook misspecification of the elements of the inverse problem, such as the prior or the measurement uncertainties. This work presents an efficient algorithmic approach for designing optimal experimental design schemes for Bayesian linear inverse problems such that the optimal design is robust to misspecification of elements of the inverse problem. Specifically, we consider a worst-case scenario approach for the uncertain or misspecified parameters, formulate robust objectives, and propose an algorithmic approach for optimizing such objectives. Furthermore, both relaxation and stochastic solution approaches are discussed with detailed analysis and insight into the interpretation of the problem and the proposed algorithmic approach. Extensive numerical experiments to validate and analyze the proposed approach are carried out for sensor placement in a parameter identification problem.
Bayesian inverse problems↗
Part 1 - Electrode design. Part 2 - Experimental design Quarterly report, Nov. 1968 - Feb. 1969
Electrode development for long-term implantation and measurement of evoked response in rats
Marmarou, A.↗
The Tularosa Study: An Experimental Design and Implementation to Quantify the Effectiveness of Cyber Deception.
Abstract not provided.
Ferguson-Walter, Kimberly↗
Comparing D-optimal designs with common mixture experimental designs for logistic regression
Mixture experiments are used in applications where the proportion of mixing components affects a response variable, such as in studies involving chemical formulations. In many applications, the response is dichotomous or binary (e.g., pass or fail) and a concern for researchers is how to efficiently and informatively design such experiments. A naive approach is to use design recommendations derived from linear normal-theory models with constant variance. In this research, we investigate the potential risks of such designs by comparing them to D-optimal mixture designs for binary responses and evaluating the D-efficiency of these design alternatives for several parameter subspaces. Standard designs for normal theory models generally do not work well for binary responses due to the tendency of these designs to favor boundary points. In addition, D-optimal mixture designs for binary responses tend to locate design points in the region where the magnitude of predicted response probabilities are moderate. Lastly, we recommend that researchers pay close attention to what is known about the characteristics of the underlying process models in selecting appropriate mixture designs for binary-response applications.
D-optimality↗
Practical and Optimal Sequential Bayesian Experimental Design for Complex Systems Incorporating Human Experimenter Preferences (Final Scientific/Technical Report)
Experiments are indispensable for developing models of complex systems. Carefully designed experiments can provide substantial savings for these expensive data-acquisition opportunities. However, designs based on heuristics are often suboptimal for systems with multiphysics, nonlinear dynamics, and uncertain and noisy environments. Optimal experimental design, while leveraging predictive models, seeks to systematically quantify and maximize the value of experiments. In this project, we focused on the design of multiple experiments, where current approaches are largely suboptimal: batch-design does not adapt to new data acquired during the experiment campaign (no feedback), and greedy/myopic design ignores future dynamics and consequences (no lookahead). We developed the mathematical framework and computational methods for sequential optimal experimental design (sOED) for complex systems. We enabled tractable model-based sOED in a rigorous manner through novel algorithms based on reinforcement learning, and investigated the effects of human experimenters on the design process. Our methods are fully Bayesian, able to quantify and update uncertainty in a principled manner. The traits aimed by our approach—mathematical rigor and optimality, human effects and uncertainty quantification, computational practicality—are crucial for elevating the standards of artificial intelligence (AI) to support decision-making in scientific domains, and contribute toward trust and realistic adoption of AI in experimental design practice.
97 MATHEMATICS AND COMPUTING↗
Supercharging simulation-based inference for Bayesian optimal experimental design
Abstract Bayesian optimal experimental design (BOED) seeks to maximize the expected information gain (EIG) of experiments. This requires a likelihood estimate, which in many settings is intractable. Simulation-based inference (SBI) provides powerful tools for this regime. However, existing work explicitly connecting SBI and BOED is restricted to a single contrastive EIG bound. We show that the EIG admits multiple formulations which can directly leverage modern SBI density estimators, encompassing neural posterior, likelihood, and ratio estimation. Building on this perspective, we define a novel EIG estimator using neural likelihood estimation. Further, we identify optimization as a key bottleneck of gradient based EIG maximization and show that a simple multi-start parallel gradient ascent procedure can substantially improve reliability and performance. With these innovations, our SBI-based BOED methods are able to match or outperform by up to 22% existing state-of-the-art approaches across standard BOED benchmarks.
97 MATHEMATICS AND COMPUTING↗
SCEPTOR Power System Design: Experimental Electric Propulsion System Design and Qualification for Crewed Flight Testing
This presentation is the first in a series of demonstrators showcasing electric propulsion technologies in aircraft.
electric propulsion↗
Optimal Experimental Design With Fast Neural Network Surrogate Models
Designing optimal experiments minimizes the uncertainty of results and maximizes the efficient use of resources. Herein, machine learning surrogate models and the approximate coordinate exchange (ACE) algorithm are used to determine optimum experimental designs over large or arbitrarily restrictive design spaces. Optimal experimental design is particularly salient in materials science where experiments are expensive and material properties must often be inferred indirectly. The proposed framework is demonstrated by finding optimal experiments with which the hidden constituent properties of composite materials can be most efficiently inferred from observable experimental outcomes. The optimum experimental design is given by an information-theoretic criteria, which maximizes the conditional mutual information between the hidden properties and the expected experimental outcomes. To perform tractable optimization a neural network is trained as a surrogate model to mimic a physics based simulation, which can calculate the expected experimental outcome based on a candidate experimental design and sampled constituent properties. The ACE algorithm is used to optimize over large design spaces with many tests and controlled parameters where an exhaustive search would be intractable even with the surrogate model. Using this approach, optimal experimental designs that are consistent with those produced by heuristic knowledge and established best practices are found; then optimal designs in larger design spaces where heuristic knowledge is unavailable are examined.
machine learning↗
Analysis and Optimization of Seismo-Acoustic Monitoring Networks with Bayesian Optimal Experimental Design
The Bayesian optimal experimental design (OED) problem seeks to identify data, sensor configurations, or experiments which can optimally reduce uncertainty. The goal of OED is to find an experiment that maximizes the expected information gain (EIG) about quantities of interest given prior knowledge about expected data. Therefore, within the context of seismic monitoring, we can use Bayesian OED to configure sensor networks by choosing sensor locations, types, and fidelity in order to improve our ability to identify and locate seismic sources. In this work, we develop the framework necessary to use Bayesian OED to optimize the ability to locate seismic events from arrival time data of detected seismic phases. In order to do utilize Bayesian OED we must develop four elements:1. A likelihood function that describes the uncertainty of detection and travel times; 2. A Bayesian solver that takes a prior and likelihood to identify the posterior; 3. An algorithm to compute EIG; and, 4. An optimizer that finds a sensor network which maximizes EIG. Once we have developed this framework, we can explore many relevant questions to monitoring such as: how and what multiphenomenology data can be used to optimally reduce uncertainty, how to trade off sensor fidelity and earth model uncertainty, and how sensor types, number, and locations influence uncertainty
47 OTHER INSTRUMENTATION↗
Comparison of the experimental aerodynamic characteristics of theoretically and experimentally designed supercritical airfoils
A lifting airfoil theoretically designed for shockless supercritical flow utilizing a complex hodograph method has been evaluated in the Langley 8-foot transonic pressure tunnel at design and off-design conditions. The experimental results are presented and compared with those of an experimentally designed supercritical airfoil which were obtained in the same tunnel.
Harris, C. D.↗
Optimal experimental design using eigenvalue-based criteria with Pyomo.DoE
New developments in automated optimal experimental design within the PSE+ software ecosystem. Advancements in user experience (to reduce the time taken to perform optimal experiment design) and computational capabilities (allowing more diverse experimental design) are shown with an example relevant to critical minerals and materials. Also, a small tutorial on science-based optimal experimental design and novel contributions therein are presented.
97 MATHEMATICS AND COMPUTING↗
Perspectives on Prediction Variance and Bias in Developing, Assessing, and Comparing Experimental Designs
The vast majority of response surface methods used in practice to develop, assess, and compare experimental designs focus on variance properties of designs. Because response surface models only approximate the true unknown relationships, models are subject to bias errors as well as variance errors. Beginning with the seminal paper of Box and Draper (1959) and over the subsequent 50 years, methods that consider bias and mean-squared-error (variance and bias) properties of designs have been presented in the literature. However, these methods are not widely implemented in software and are not routinely used to develop, assess, and compare experimental designs in practice. Methods for developing, assessing, and comparing response surface designs that account for variance properties are reviewed. Brief synopses of publications that consider bias or mean-squared-error properties are provided. The difficulties and approaches for addressing bias properties of designs are summarized. Perspectives on experimental design methods that account for bias and/or variance properties and on future needs are presented.
97 MATHEMATICS AND COMPUTING↗
Comment: Spurious Correlation and Other Observations on Experimental Design for Engineering Dimensional Analysis
This article discusses the paper "Experimental Design for Engineering Dimensional Analysis" by Albrecht et al. (2013, Technometrics). That paper provides and overview of engineering dimensional analysis (DA) for use in developing DA models. The paper proposes methods for generating model-robust experimental designs to supporting fitting DA models. The specific approach is to develop a design that maximizes the efficiency of a specified empirical model (EM) in the original independent variables, subject to a minimum efficiency for a DA model expressed in terms of dimensionless groups (DGs). This discussion article raises several issues and makes recommendations regarding the proposed approach. Also, the concept of spurious correlation is raised and discussed. Spurious correlation results from the response DG being calculated using several independent variables that are also used to calculate predictor DGs in the DA model.
Dimensional analysis↗
Optimal Experimental Design for Inverse Problems in the Presence of Observation Correlations
Optimal experimental design (OED) is the general formalism of sensor placement and decisions on the data collection strategy for engineered or natural experiments. This problem is prevalent in many critical fields such as battery design, numerical weather prediction, geosciences, environmental and urban studies. State-of-the-art computational methods for experimental design do not accommodate correlation structure in observational errors produced by many expensive-to-operate devices such as X-ray machines, radars, and satellites. Discarding evident data correlations leads to biased results, higher expenses, and waste of valuable resources. We present a general formulation of the OED formalism for model-constrained large-scale Bayesian linear inverse problems, where measurement errors are generally correlated. The proposed approach utilizes the Hadamard product of matrices to formulate the weighted-likelihood, and is valid for both finite as well as infinite-dimensional Bayesian inverse problems. Extensive numerical experiments are carried out for empirical verification of the proposed approach using an advection-diffusion model, where the objective is to optimally place a small set of sensors, under a limited budget, to predict the concentration of a contaminant in a closed and bounded domain.
Correlated observations↗
A Scalable Gradient Free Method for Bayesian Experimental Design with Implicit Models
Bayesian experimental design (BED) is to answer the question that how to choose designs that maximize the information gathering. For implicit models, where the likelihood is intractable but sampling is possible, conventional BED methods have difficulties in efficiently estimating the posterior distribution and maximizing the mutual information (MI) between data and parameters. Recent work proposed the use of gradient ascent to maximize a lower bound on MI to deal with these issues. However, the approach requires a sampling path to compute the pathwise gradient of the MI lower bound with respect to the design variables, and such a pathwise gradient is usually inaccessible for implicit models. In this paper, we propose a novel approach that leverages recent advances in stochastic approximate gradient ascent incorporated with a smoothed variational MI estimator for efficient and robust BED. Without the necessity of pathwise gradients, our approach allows the design process to be achieved through a unified procedure with an approximate gradient for implicit models. Several experiments show that our approach outperforms baseline methods, and significantly improves the scalability of BED in high-dimensional problems.
Zhang, Jiaxin↗
The Effect of the Prior and the Experimental Design on the Inference of the Precision Matrix in Gaussian Chain Graph Models
Here, we investigate whether (and how) experimental design could aid in the estimation of the precision matrix in a Gaussian chain graph model, especially the interplay between the design, the effect of the experiment and prior knowledge about the effect. Estimation of the precision matrix is a fundamental task to infer biological graphical structures like microbial networks. We compare the marginal posterior precision of the precision matrix under four priors: flat, conjugate Normal-Wishart, Normal-MGIG and a general independent. Under the flat and conjugate priors, the Laplace-approximated posterior precision is not a function of the design matrix rendering useless any efforts to find an optimal experimental design to infer the precision matrix. In contrast, the Normal-MGIG and general independent priors do allow for the search of optimal experimental designs, yet there is a sharp upper bound on the information that can be extracted from a given experiment. We confirm our theoretical findings via a simulation study comparing (i) the KL divergence between prior and posterior and (ii) the Stein’s loss difference of MAPs between random and no experiment. Our findings provide practical advice for domain scientists conducting experiments to better infer the precision matrix as a representation of a biological network.
54 ENVIRONMENTAL SCIENCES↗
Experimental Design for Hanford Low-Activity Waste Glasses with High Waste Loading
This report discusses the development of an experimental design for the initial phase of the Hanford low-activity waste (LAW) enhanced glass study. This report is based on a manuscript written for an applied statistics journal. Appendices A, B, and E include additional information relevant to the LAW enhanced glass experimental design that is not included in the journal manuscript. The glass composition experimental region is defined by single-component constraints (SCCs), linear multiple-component constraints (MCCs), and a nonlinear MCC involving 15 LAW glass components. Traditional methods and software for designing constrained mixture experiments with SCCs and linear MCCs are not directly applicable because of the nonlinear MCC. A modification of existing methodology to account for the nonlinear MCC was developed and is described in this report. One of the glass components, SO 3 , has a solubility limit in glass that depends on the composition of the balance of the glass. A goal was to design the experiment so that SO 3 would not exceed its predicted solubility limit for any of the experimental glasses. The SO 3 solubility limit had previously been modeled by a partial quadratic mixture model expressed in the relative proportions of the 14 other components. The partial quadratic mixture model was used to construct a nonlinear MCC in terms of all 15 components. In addition, there were SCCs and linear MCCs. This report describes how a layered design was generated to (i) account for the SCCs, linear MCCs, and nonlinear MCC and (ii) meet the goals of the study. A layered design consists of points on an outer layer, and inner layer, and a center point. There were 18 outer-layer glasses chosen using optimal experimental design software to augment 147 existing glass compositions that were within the LAW glass composition experimental region. Then 13 inner-layer glasses were chosen with the software to augment the existing and outer-layer glasses. The experimental design was completed by a center-point glass, a Vitreous State Laboratory glass, and replicates of the center point and Vitreous State Laboratory glasses.
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗