Data-driven methods for batch data analysis – A critical overview and mapping on the complexity scale
Not Available
Rendall, Ricardo↗
SEARCH · Search NASA
Results for “Methods: data analysis”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Comprehensive Data Analysis and Analytic Method Development for PBX9501 Material (FY2019 Annual Report of Aging and Lifetimes Program)
In the polymer bonded explosive (PBX) 9501, a binder consisting of 2.5 wt% nitroplasticizer (NP) and 2.5 wt% Estane R 5703 (Estane) is combined with 94.9 wt% of the high-explosive HMX and 0.1 wt% stabilizer. Because of its flexibility and tensile strength, this binder lowers the sensitivity and improves the manufacturability of PBX 9501. However, like many plasticizers comprised of low molecular weight components, NP has a tendency to diffuse out of the PBX 9501 matrix and can decompose at moderate temperatures into reactive byproducts, such as NO, NO 2 , H 2 O, and HNO x . Through oxidation and hydrolysis, these molecules can further degrade NP and Estane, ultimately degrading the properties of PBX 9501. While gaseous molecules can readily diffuse out of the PBX 9501 charges, the intermediates with lower volatility are more likely trapped in the condensed phase inside the charges and, in a closed system, will co-exist with Estane for an extended period of time creating an ongoing reactive environment. Understanding the rates of formation for these volatiles and intermediates during the NP degradation is therefore a critical prerequisite for understanding the long-term stability of polymeric binders used in munition systems. To work toward this ultimate goal, in the past year, we have conducted comprehensive studies on the physical properties of NP and finished three-year long aging experiment to understand the aging behavior of NP under thermal treatment. Important results are summarized in this annual report. Although NP is widely used in the DOE complex, their physical properties are rather scattered and inconsistent in the open literature. For example, there are at least two widely different values, 14.5°C and -15°C, for the melting point of NP. Although it is known that NP is a eutectic 50:50 mixture of BDNPA and BDNPF, their eutectic phase diagram is not well documented. Furthermore, the effect of temperature on the miscibility between water and eutectic BDNPA/F mixture is rarely reported. To fill these knowledge gaps, a large set of BDNPA/F mixtures with BDNPA concentration ranging from 0 to 100 wt% was analyzed using DSC techniques. In addition to determining the eutectic melt point as -25°C, a phase diagram of the BDNPA/F system was constructed from -30°C to 45°C. With this phase diagram, the phase transition temperatures and composition can be readily found for the BDNPA/F mixtures with various mass ratios.
36 MATERIALS SCIENCE↗
Using numeric simulations to inform experimental data analysis: A new method to account for characteristic bending under dynamic loading and release.
Abstract not provided.
Alexander, C. Scott↗
Characterizing Reactor Operations from Realistic Simulated Environmental Samples: Combining High-Performance Computing and Data Analytics
Environmental sampling is a common technique employed by inspectors and facility operators in nuclear safeguards, proliferation detection, and process monitoring contexts. Interpreting measurements performed on samples or collections of samples and ensuring the information extracted is accurate and precise is difficult. To date, these analyses have relied on simulated data to enable systematic studies; however, these models are inherently limited by the fidelity of the models and the implicit spatial averaging of isotopic composition or other signatures of interest. To advance this capability, we have refined the spatial discretization and expanded the range of physics in the simulation codes we use to perform reactor simulations and depletion calculations. This allows us to generate data that are more representative of real environmental samples, especially for the length scale of the isotopic composition and associated variation. Accordingly, these new data allow a more realistic assessment of traditional and new data analytic analysis methods. Here we present motivation for developing reactor simulations using high-performance computing methods and resources, impacts of these new simulations on our assessment of data analysis and interpretation methods, and initial results of developing and systematically testing data analytic methods designed to overcome the challenges expected of real-world samples. We also quantify the performance of these analyses using defensible statistical methods.
Dayman, Ken J.↗
Planck 2015 results: VI. LFI mapmaking
This article describes the mapmaking procedure applied to Planck Low Frequency Instrument (LFI) data. The mapmaking step takes as input the calibrated timelines and pointing information. The main products are sky maps of I, Q, and U Stokes components. For the first time, we present polarization maps at LFI frequencies. The mapmaking algorithm is based on a destriping technique, which is enhanced with a noise prior. The Galactic region is masked to reduce errors arising from bandpass mismatch and high signal gradients. We apply horn-uniform radiometer weights to reduce the effects of beam-shape mismatch. The algorithm is the same as used for the 2013 release, apart from small changes in parameter settings. We validate the procedure through simulations. Special emphasis is put on the control of systematics, which is particularly important for accurate polarization analysis. We also produce low-resolution versions of the maps and corresponding noise covariance matrices. These serve as input in later analysis steps and parameter estimation. The noise covariance matrices are validated through noise Monte Carlo simulations. The residual noise in the map products is characterized through analysis of half-ring maps, noise covariance matrices, and simulations.
79 ASTRONOMY AND ASTROPHYSICS↗
A Bayesian approach to strong lens finding in the era of wide-area surveys
ABSTRACT The arrival of the Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST), Euclid-Wide and Roman wide-area sensitive surveys will herald a new era in strong lens science in which the number of strong lenses known is expected to rise from $\mathcal {O}(10^3)$ to $\mathcal {O}(10^5)$. However, current lens-finding methods still require time-consuming follow-up visual inspection by strong lens experts to remove false positives which is only set to increase with these surveys. In this work, we demonstrate a range of methods to produce calibrated probabilities to help determine the veracity of any given lens candidate. To do this we use the classifications from citizen science and multiple neural networks for galaxies selected from the Hyper Suprime-Cam survey. Our methodology is not restricted to particular classifier types and could be applied to any strong lens classifier which produces quantitative scores. Using these calibrated probabilities, we generate an ensemble classifier, combining citizen science, and neural network lens finders. We find such an ensemble can provide improved classification over the individual classifiers. We find a false-positive rate of 10−3 can be achieved with a completeness of 46 per cent, compared to 34 per cent for the best individual classifier. Given the large number of galaxy–galaxy strong lenses anticipated in LSST, such improvement would still produce significant numbers of false positives, in which case using calibrated probabilities will be essential for population analysis of large populations of lenses and to help prioritize candidates for follow-up.
79 ASTRONOMY AND ASTROPHYSICS↗
The structure of liquid water up to 360 MPa from x-ray diffraction measurements using a high Q-range and from molecular simulation
X-ray diffraction measurements of liquid water are reported at pressures up to 360 MPa corresponding to a density of 0.0373 molecules per Å{sup 3}. The measurements were conducted at a spatial resolution corresponding to Q{sub max} = 16 Å{sup −1}. The method of data analysis and measurement in this study follows the earlier benchmark results reported for water under ambient conditions having a density of 0.0333 molecules per Å{sup 3} and Q{sub max} = 20 Å{sup −1} [J. Chem. Phys. 138, 074506 (2013)] and at 70 °C having a density of 0.0327 molecules per Å{sup 3} and Q{sub max} = 20 Å{sup −1} [J. Chem. Phys. 141, 214507 (2014)]. The structure of water is very different at these three different T and P state points and thus they provide the basis for evaluating the fidelity of molecular simulation. Measurements show that at 360 MPa, the 4 waters residing in the region between 2.3 and 3 Å are nearly unchanged: the peak position, shape, and coordination number are nearly identical to their values under ambient conditions. However, in the region above 3 Å, large structural changes occur with the collapse of the well-defined 2nd shell and shifting of higher shells to shorter distances. The measured structure is compared to simulated structure using intermolecular potentials described by both first-principles methods (revPBE-D3) and classical potentials (TIP4P/2005, MB-pol, and mW). The DFT-based, revPBE-D3, method and the many-body empirical potential model, MB-pol, provide the best overall representation of the ambient, high-temperature, and high-pressure data. The revPBE-D3, MB-pol, and the TIP4P/2005 models capture the densification mechanism, whereby the non-bonded 5th nearest neighbor molecule, which partially encroaches the 1st shell at ambient pressure, is pushed further into the local tetrahedral arrangement at higher pressures by the more distant molecules filling the void space in the network between the 1st and 2nd shells.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Covariate Dependent Sparse Functional Data Analysis
This study proposes a method to incorporate covariate information into sparse functional data analysis. The method aims at cases where each subject has a limited number of longitudinal measurements and is associated with static covariates. This research is motivated by several use cases in practice. One representative example is void swelling, a nuclear-specific material degradation mechanism. Void swelling is affected by many covariates, including alloy composition and irradiation type. How to accurately model the complicated joint effects of such covariates on the swelling process is the key to mitigating the effect of swelling and ensuring safe operation. Unlike most of the existing methods, the proposed method can handle high-dimensional covariates with the informative covariate identification procedure and sparse and irregularly spaced measurements, that is, does not require complete or dense observations. The main innovation of the proposed method is that we model the variation coming from covariates and the variation left conditioned on covariates, such that the functional principal component analysis and Gaussian process can be conducted in a unified manner. Further, we also propose a systematic approach to identify important covariates in the hypothesis testing context. The methodology is demonstrated on applications in nuclear engineering and healthcare and simulation studies.
42 ENGINEERING↗
Method and system for real-time analysis of biosensor data
A method of biosensor-based detection of toxins includes the steps of providing a fluid to be analyzed having a plurality of photosynthetic organisms therein, wherein chemical, biological or radiological agents alter a nominal photosynthetic activity of the photosynthetic organisms. At a first time a measured photosynthetic activity curve is obtained from the photosynthetic organisms. The measured curve is automatically compared to a reference photosynthetic activity curve to determine differences therebetween. The presence of the chemical, biological or radiological agents, or precursors thereof, are then identified if present in the fluid using the differences.
47 OTHER INSTRUMENTATION↗
Small Angle Scattering Data Analysis Assisted by Machine Learning Methods
Small angle scattering (SAS) is a widely used technique for characterizing structures of wide ranges of materials. For such wide ranges of applications of SAS, there exist a large number of ways to model the scattering data. While such analysis models are often available from various suites of SAS data analysis software packages, selecting the right model to start with poses a big challenge for beginners to SAS data analysis. Here, we present machine learning (ML) methods that can assist users by suggesting scattering models for data analysis. A series of one-dimensional scattering curves have been generated by using different models to train the algorithms. The performance of the ML method is studied for various types of ML algorithms, resolution of the dataset, and the number of the dataset. The degree of similarities among selected scattering models is presented in terms of the confusion matrix. The scattering model suggestions with prediction scores provide a list of scattering models that are likely to succeed. Therefore, if implemented with extensive libraries of scattering models, this method can speed up the data analysis workflow by reducing search spaces for appropriate scattering models.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Vision 20/20: Automation and advanced computing in clinical radiation oncology
This Vision 20/20 paper considers what computational advances are likely to be implemented in clinical radiation oncology in the coming years and how the adoption of these changes might alter the practice of radiotherapy. Four main areas of likely advancement are explored: cloud computing, aggregate data analyses, parallel computation, and automation. As these developments promise both new opportunities and new risks to clinicians and patients alike, the potential benefits are weighed against the hazards associated with each advance, with special considerations regarding patient safety under new computational platforms and methodologies. While the concerns of patient safety are legitimate, the authors contend that progress toward next-generation clinical informatics systems will bring about extremely valuable developments in quality improvement initiatives, clinical efficiency, outcomes analyses, data sharing, and adaptive radiotherapy.
62 RADIOLOGY AND NUCLEAR MEDICINE↗
DeepMerge – II. Building robust deep learning algorithms for merging galaxy identification across domains
In astronomy, neural networks are often trained on simulation data with the prospect of being used on telescope observations. Unfortunately, training a model on simulation data and then applying it to instrument data leads to a substantial and potentially even detrimental decrease in model accuracy on the new target dataset. Simulated and instrument data represent different data domains, and for an algorithm to work in both, domain-invariant learning is necessary. Here we employ domain adaptation techniques— Maximum Mean Discrepancy (MMD) as an additional transfer loss and Domain Adversarial Neural Networks (DANNs)— and demonstrate their viability to extract domain-invariant features within the astronomical context of classifying merging and non-merging galaxies. Additionally, we explore the use of Fisher loss and entropy minimization to enforce better in-domain class discriminability. We show that the addition of each domain adaptation technique improves the performance of a classifier when compared to conventional deep learning algorithms. We demonstrate this on two examples: between two Illustris-1 simulated datasets of distant merging galaxies, and between Illustris-1 simulated data of nearby merging galaxies and observed data from the Sloan Digital Sky Survey. The use of domain adaptation techniques in our experiments leads to an increase of target domain classification accuracy of up to ~20%. With further development, these techniques will allow astronomers to successfully implement neural network models trained on simulation data to efficiently detect and study astrophysical objects in current and future large-scale astronomical surveys.
galaxies: interactions↗
Analysis Code - Data Analysis in 'Leveraging Multiple Statistical Methods for Inverse Prediction in Nuclear Forensics Applications' (LMSMIPNFA) v. 1.0
R code that performs the analysis of a data set presented in the paper ‘Leveraging Multiple Statistical Methods for Inverse Prediction in Nuclear Forensics Applications’ by Lewis, J., Zhang, A., Anderson-Cook, C. It provides functions for doing inverse predictions in this setting using several different statistical methods. The data set is a publicly available data set from a historical Plutonium production experiment.
Lewis, John↗
nbodykit: An Open-source, Massively Parallel Toolkit for Large-scale Structure
We present nbodykit, an open-source, massively parallel Python toolkit for analyzing large-scale structure (LSS) data. Using Python bindings of the Message Passing Interface, we provide parallel implementations of many commonly used algorithms in LSS. nbodykit is both an interactive and scalable piece of scientific software, performing well in a supercomputing environment while still taking advantage of the interactive tools provided by the Python ecosystem. Existing functionality includes estimators of the power spectrum, two- and three-point correlation functions, a friends-of-friends grouping algorithm, mock catalog creation via the halo occupation distribution technique, and approximate N-body simulations via the FastPM scheme. The package also provides a set of distributed data containers, insulated from the algorithms themselves, that enables nbodykit to provide a unified treatment of both simulation and observational data sets. nbodykit can be easily deployed in a high-performance computing environment, overcoming some of the traditional difficulties of using Python on supercomputers. We provide performance benchmarks illustrating the scalability of the software. The modular, component-based approach of nbodykit allows researchers to easily build complex applications using its tools. The package is extensively documented at http://nbodykit.readthedocs.io, which also includes an interactive set of example recipes for new users to explore. As open-source software, we hope nbodykit provides a common framework for the community to use and develop in confronting the analysis challenges of future LSS surveys.
79 ASTRONOMY AND ASTROPHYSICS↗
Functional Data Analysis in NTCP Modeling: A New Method to Explore the Radiation Dose-Volume Effects
Purpose/Objective(s): To describe a novel method to explore radiation dose-volume effects. Functional data analysis is used to investigate the information contained in differential dose-volume histograms. The method is applied to the normal tissue complication probability modeling of rectal bleeding (RB) for patients irradiated in the prostatic bed by 3-dimensional conformal radiation therapy. Methods and Materials: Kernel density estimation was used to estimate the individual probability density functions from each of the 141 rectum differential dose-volume histograms. Functional principal component analysis was performed on the estimated probability density functions to explore the variation modes in the dose distribution. The functional principal components were then tested for association with RB using logistic regression adapted to functional covariates (FLR). For comparison, 3 other normal tissue complication probability models were considered: the Lyman-Kutcher-Burman model, logistic model based on standard dosimetric parameters (LM), and logistic model based on multivariate principal component analysis (PCA). Results: The incidence rate of grade ≥2 RB was 14%. V{sub 65Gy} was the most predictive factor for the LM (P=.058). The best fit for the Lyman-Kutcher-Burman model was obtained with n=0.12, m = 0.17, and TD50 = 72.6 Gy. In PCA and FLR, the components that describe the interdependence between the relative volumes exposed at intermediate and high doses were the most correlated to the complication. The FLR parameter function leads to a better understanding of the volume effect by including the treatment specificity in the delivered mechanistic information. For RB grade ≥2, patients with advanced age are significantly at risk (odds ratio, 1.123; 95% confidence interval, 1.03-1.22), and the fits of the LM, PCA, and functional principal component analysis models are significantly improved by including this clinical factor. Conclusion: Functional data analysis provides an attractive method for flexibly estimating the dose-volume effect for normal tissues in external radiation therapy.
62 RADIOLOGY AND NUCLEAR MEDICINE↗
Merger identification through photometric bands, colours, and their errors
Aims. We present the application of a fully connected neural network (NN) for galaxy merger identification using exclusively photometric information. Our purpose is not only to test the method’s efficiency, but also to understand what merger properties the NN can learn and what their physical interpretation is. Methods. We created a class-balanced training dataset of 5860 galaxies split into mergers and non-mergers. The galaxy observations came from SDSS DR6 and were visually identified in Galaxy Zoo. The 2930 mergers were selected from known SDSS mergers and the respective non-mergers were the closest match in both redshift and r magnitude. The NN architecture was built by testing a different number of layers with different sizes and variations of the dropout rate. We compared input spaces constructed using: the five SDSS filters: u, g, r, i, and z; combinations of bands, colours, and their errors; six magnitude types; and variations of input normalization. Results. We find that the fibre magnitude errors contribute the most to the training accuracy. Studying the parameters from which they are calculated, we show that the input space built from the sky error background in the five SDSS bands alone leads to 92.64 ± 0.15% training accuracy. We also find that the input normalization, that is to say, how the data are presented to the NN, has a significant effect on the training performance. Conclusions. We conclude that, from all the SDSS photometric information, the sky error background is the most sensitive to merging processes. This finding is supported by an analysis of its five-band feature space by means of data visualization. Moreover, studying the plane of the g and r sky error bands shows that a decision boundary line is enough to achieve an accuracy of 91.59%.
79 ASTRONOMY AND ASTROPHYSICS↗
Better together: Elements of successful scientific software development in a distributed collaborative community
Many scientific disciplines rely on computational methods for data analysis, model generation, and prediction. Implementing these methods is often accomplished by researchers with domain expertise but without formal training in software engineering or computer science. This arrangement has led to underappreciation of sustainability and maintainability of scientific software tools developed in academic environments. Some software tools have avoided this fate, including the scientific library Rosetta. We use this software and its community as a case study to show how modern software development can be accomplished successfully, irrespective of subject area. Rosetta is one of the largest software suites for macromolecular modeling, with 3.1 million lines of code and many state-of-the-art applications. Since the mid 1990s, the software has been developed collaboratively by the RosettaCommons, a community of academics from over 60 institutions worldwide with diverse backgrounds including chemistry, biology, physiology, physics, engineering, mathematics, and computer science. Developing this software suite has provided us with more than two decades of experience in how to effectively develop advanced scientific software in a global community with hundreds of contributors. Here we illustrate the functioning of this development community by addressing technical aspects (like version control, testing, and maintenance), community-building strategies, diversity efforts, software dissemination, and user support. We demonstrate how modern computational research can thrive in a distributed collaborative community. The practices described here are independent of subject area and can be readily adopted by other software development communities
97 MATHEMATICS AND COMPUTING↗
BEYONDPLANCK II. CMB mapmaking through Gibbs sampling
We present a Gibbs sampling solution to the mapmaking problem for cosmic microwave background (CMB) measurements that builds on existing destriping methodology. Gibbs sampling breaks the computationally heavy destriping problem into two separate steps: noise filtering and map binning. Considered as two separate steps, both are computationally much cheaper than solving the combined problem. This provides a huge performance benefit as compared to traditional methods and it allows us, for the first time, to bring the destriping baseline length to a single sample. Here, we applied the Gibbs procedure to simulated Planck 30 GHz data. We find that gaps in the time-ordered data are handled efficiently by filling them in with simulated noise as part of the Gibbs process. The Gibbs procedure yields a chain of map samples, from which we are able to compute the posterior mean as a best-estimate map. The variation in the chain provides information on the correlated residual noise, without the need to construct a full noise covariance matrix. However, if only a single maximum-likelihood frequency map estimate is required, we find that traditional conjugate gradient solvers converge much faster than a Gibbs sampler in terms of the total number of iterations. The conceptual advantages of the Gibbs sampling approach lies in statistically well-defined error propagation and systematic error correction. This methodology thus forms the conceptual basis for the mapmaking algorithm employed in the BEYONDPLANCK framework, which implements the first end-to-end Bayesian analysis pipeline for CMB observations.
79 ASTRONOMY AND ASTROPHYSICS↗