Search NASA⌕ Search

SEARCH · Search NASA

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Methodology and performance comparison of statistical learning pulse shape classifiers as demonstrated with organic liquid scintillator [Performance comparison of statistical learning pulse shape classifiers]

In this study, we present novel methods for automated pulse shape discrimination. The classifiers are trained using simple radionuclide sources and do not require ground truth labeling. We test their performance using labels derived from time of flight experiments and present the results in terms of energy-dependent Receiver Operating Characteristic (ROC) curves. In addition, we also train and test standard pulse shape discrimination methods on the same data for comparison. We find multiple methods that can yield similar false neutron and true neutron rates at 24 keVee as tail-to-total or Gatti’s optimal linear filter yield at 54 keVee.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Uncovering the Forced Climate Response from a Single Ensemble Member Using Statistical Learning

Internal atmospheric variability fundamentally limits predictability of climate and obscures evidence of anthropogenic climate change regionally and on time scales of up to a few decades. Dynamical adjustment techniques estimate and subsequently remove the influence of atmospheric circulation variability on temperature or precipitation. The residual component is expected to contain the thermodynamical signal of the externally forced response but with less circulation-induced noise. Existing techniques have led to important insights into recent trends in regional (hydro-) climate and their drivers, but the variance explained by circulation is often low. Here, we develop a novel dynamical adjustment technique by implementing principles from statistical learning. We demonstrate in an ensemble of Community Earth System Model (CESM) simulations that statistical learning methods, such as regularized linear models, establish a clearer relationship between circulation variability and atmospheric target variables, and need relatively short periods of record for training (around 30 years). The method accounts for, on average, 83% and 78% of European monthly winter temperature and precipitation variability at gridcell level, and around 80% of global mean temperature and hemispheric precipitation variability. We show that the residuals retain forced thermodynamical contributions to temperature and precipitation variability. Accurate estimates of the total forced response can thus be recovered assuming that forced circulation changes are gradual over time. Overall, forced climate response estimates can be extracted at regional or global scales from approximately 3–5 times fewer ensemble members, or even a single realization, using statistical learning techniques. We anticipate the technique will contribute to reducing uncertainties around internal variability and facilitating climate change detection and attribution.

54 ENVIRONMENTAL SCIENCES↗

Physics-based statistical learning approach to mesoscopic model selection

In materials science and many other research areas, models are frequently inferred without considering their generalization to unseen data. Here, we apply statistical learning using cross-validation to obtain an optimally predictive coarse-grained description of a two-dimensional kinetic nearest-neighbor Ising model with Glauber dynamics (GD) based on the stochastic Ginzburg-Landau equation (sGLE). The latter is learned from GD “training” data using a log-likelihood analysis, and its predictive ability for various complexities of the model is tested on GD “test” data independent of the data used to train the model on. Using two different error metrics, we perform a detailed analysis of the error between magnetization time trajectories simulated using the learned sGLE coarse-grained description and those obtained using the GD model. We show that both for equilibrium and out-of-equilibrium GD training trajectories, the standard phenomenological description using a quartic free energy does not always yield the most predictive coarse-grained model. Moreover, increasing the amount of training data can shift the optimal model complexity to higher values. Our results are promising in that they pave the way for the use of statistical learning as a general tool for materials modeling and discovery.

36 MATERIALS SCIENCE↗

Statistical-learning strategies generate only modestly performing predictive models for urinary symptoms following external beam radiotherapy of the prostate: A comparison of conventional and machine-learning methods

Purpose: Given the paucity of available data concerning radiotherapy-induced urinary toxicity, it is important to ensure derivation of the most robust models with superior predictive performance. This work explores multiple statistical-learning strategies for prediction of urinary symptoms following external beam radiotherapy of the prostate. Methods: The performance of logistic regression, elastic-net, support-vector machine, random forest, neural network, and multivariate adaptive regression splines (MARS) to predict urinary symptoms was analyzed using data from 754 participants accrued by TROG03.04-RADAR. Predictive features included dose-surface data, comorbidities, and medication-intake. Four symptoms were analyzed: dysuria, haematuria, incontinence, and frequency, each with three definitions (grade ≥ 1, grade ≥ 2 and longitudinal) with event rate between 2.3% and 76.1%. Repeated cross-validations producing matched models were implemented. A synthetic minority oversampling technique was utilized in endpoints with rare events. Parameter optimization was performed on the training data. Area under the receiver operating characteristic curve (AUROC) was used to compare performance using sample size to detect differences of ≥0.05 at the 95% confidence level. Results: Logistic regression, elastic-net, random forest, MARS, and support-vector machine were the highest-performing statistical-learning strategies in 3, 3, 3, 2, and 1 endpoints, respectively. Logistic regression, MARS, elastic-net, random forest, neural network, and support-vector machine were the best, or were not significantly worse than the best, in 7, 7, 5, 5, 3, and 1 endpoints. The best-performing statistical model was for dysuria grade ≥ 1 with AUROC ± standard deviation of 0.649 ± 0.074 using MARS. For longitudinal frequency and dysuria grade ≥ 1, all strategies produced AUROC>0.6 while all haematuria endpoints and longitudinal incontinence models produced AUROC<0.6. Conclusions: Logistic regression and MARS were most likely to be the best-performing strategy for the prediction of urinary symptoms with elastic-net and random forest producing competitive results. The predictive power of the models was modest and endpoint-dependent. New features, including spatial dose maps, may be necessary to achieve better models.

60 APPLIED LIFE SCIENCES↗

Statistical learning framework for safety and failure analysis of a DNN-based autonomous aircraft system

Deep Neural Networks (DNNs) and Machine Learning technology is increasingly used for safety-critical applications in the Aerospace domain. To ensure safe operations, the DNN and the system must undergo rigorous verification and validation, including advanced statistical analyses. Performance and safety of the DNN and system behavior must not only be analyzed for the nominal case, but under numerous off-nominal and failure cases. In this paper we will describe how our statistical learning framework SYSAI can efficiently perform such analyses using the tool’s unique combination of advanced learning modeling and statistical analysis techniques. SYSAI can effectively explore the high-dimensional state and failure space of the system under test; geometrical shape detection of safety regions and boundaries support explainability of the results to the designer. In this paper, we report experiments and results obtained with a vision-based DNN control system (ACT) that is capable of autonomously steering an aircraft down a runway.

Yuning He↗

System and Safety Analysis with SysAI A Statistical Learning Framework

This is a tutorial on how to use the SYSAI (System Analysis using Statistical AI), a flexible statistical learning framework for the V&V and analysis of complex and high-dimensional Aerospace systems with DNN and AI components. SYSAI provides functionality for a variety of analyses and V&V tasks, including statistical data analysis, high dimensional safety-envelope and time-series analysis, property checking, as well as intelligent test-case generation. The tutorial will demonstrate SYSAI with our industrial partner’s Autonomous Centerline Tracking system, which uses a DNN to enable autonomous aircraft taxiing as an example. Video & Tutorial

Statistical V&V for Complex safety-critical system↗

Impact of Statistical Learning Methods on the Predictive Power of Multivariate Normal Tissue Complication Probability Models

Purpose: To study the impact of different statistical learning methods on the prediction performance of multivariate normal tissue complication probability (NTCP) models. Methods and Materials: In this study, three learning methods, stepwise selection, least absolute shrinkage and selection operator (LASSO), and Bayesian model averaging (BMA), were used to build NTCP models of xerostomia following radiotherapy treatment for head and neck cancer. Performance of each learning method was evaluated by a repeated cross-validation scheme in order to obtain a fair comparison among methods. Results: It was found that the LASSO and BMA methods produced models with significantly better predictive power than that of the stepwise selection method. Furthermore, the LASSO method yields an easily interpretable model as the stepwise method does, in contrast to the less intuitive BMA method. Conclusions: The commonly used stepwise selection method, which is simple to execute, may be insufficient for NTCP modeling. The LASSO method is recommended.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

A Statistical Learning Framework for Materials Science: Application to Elastic Moduli of k -nary Inorganic Polycrystalline Compounds

Materials scientists increasingly employ machine or statistical learning (SL) techniques to accelerate materials discovery and design. Such pursuits benefit from pooling training data across, and thus being able to generalize predictions over, k -nary compounds of diverse chemistries and structures. This work presents a SL framework that addresses challenges in materials science applications, where datasets are diverse but of modest size, and extreme values are often of interest. Our advances include the application of power or Hölder means to construct descriptors that generalize over chemistry and crystal structure, and the incorporation of multivariate local regression within a gradient boosting framework. The approach is demonstrated by developing SL models to predict bulk and shear moduli (K and G, respectively) for polycrystalline inorganic compounds, using 1,940 compounds from a growing database of calculated elastic moduli for metals, semiconductors and insulators. The usefulness of the models is illustrated by screening for superhard materials.

36 MATERIALS SCIENCE↗

Phospholipid fatty acid biomarkers in a freshwater periphyton community exposed to uranium: discovery by non-linear statistical learning

Phospholipid fatty acids (PLFA) have been widely used to characterize environmental microbial communities, generating community profiles that can distinguish phylogenetic or functional groups within the community. The poor specificity of organism groups with fatty acid biomarkers in the classic PLFA-microorganism associations is a confounding factor in many of the statistical classification/clustering approaches traditionally used to interpret PLFA profiles. In this paper we demonstrate that non-linear statistical learning methods, such as a support vector machine (SVM), can more accurately find patterns related to uranyl nitrate exposure in a freshwater periphyton community than linear methods, such as partial least squares discriminant analysis. In addition, probabilistic models of exposure can be derived from the identified lipid biomarkers to demonstrate the potential model-based approach that could be used in remediation. The SVM probability model separates dose groups at accuracies of ~87.0%, ~71.4%, ~87.5%, and 100% for the four groups; Control (non-amended system), low-dose (amended at 10 µg U L-1), medium dose (amended at 100 µg U L-1), and high dose (500 µg U L-1). The SVM model achieved an overall cross-validated classification accuracy of ~87% in contrast to ~59% for the best linear classifier.

54 ENVIRONMENTAL SCIENCES↗

Accelerating quantum optics experiments with statistical learning

Quantum optics experiments, involving the measurement of low-probability photon events, are known to be extremely time-consuming. We present a methodology for accelerating such experiments using physically motivated ansatzes together with simple statistical learning techniques such as Bayesian maximum a posteriori estimation based on few-shot data. We show that it is possible to reconstruct time-dependent data using a small number of detected photons, allowing for fast estimates in under a minute and providing a one-to-two order of magnitude speed-up in data acquisition time. We test our approach using real experimental data to retrieve the second order intensity correlation function, G (2) ($τ$), as a function of time delay τ between detector counts, for thermal light as well as anti-bunched light emitted by a quantum dot driven by periodic laser pulses. The proposed methodology has a wide range of applicability and has the potential to impact the scientific discovery process across a multitude of domains.

36 MATERIALS SCIENCE↗

Investigating phase transitions from local crystallographic analysis based on statistical learning of atomic environments in 2D MoS 2 -ReS 2

Traditionally, phase transitions are explored using a combination of macroscopic functional characterization and scattering techniques, providing insight into the average properties and symmetries of the lattice but local atomic level mechanisms during phase transitions generally remain unknown. Here, we explore the mechanisms of a phase transition between the trigonal prismatic and distorted octahedral phases of layered chalcogenides in the 2D MoS 2 - ReS 2 system from the observations of local degrees of freedom, namely atomic positions by Scanning Transmission Electron Microscopy (STEM). We employ local crystallographic analysis based on statistical learning of atomic environments to build a picture of the transition from the atomic level up and determine local and global variables controlling the local symmetry breaking. In particular, we argue that the dependence of the average symmetry breaking distortion amplitude on global and local concentration can be used to separate local chemical as well as global electronic effects on transition. This approach allows exploring atomic mechanisms beyond the traditional macroscopic descriptions, utilizing the imaging of compositional fluctuations in solids to explore phase transitions over a range of realized and observed local stoichiometries and atomic configurations.

25 ENERGY STORAGE↗

Examining the role of fluctuations in the early stages of homogenous polymer crystallization with simulation and statistical learning

Here, we propose a relationship between the dynamics in the amorphous and crystalline domains during polymer crystallization: the fluctuations of ordering-rate about a material-specific value in the amorphous phase drive those fluctuations associated with the increase in percent crystallinity. This suggests a differential equation that satisfies the three experimentally observed time regimes for the rate of crystal growth. To test this postulated expression, we applied a suite of statistical learning tools to molecular dynamics simulations to extract the relevant phenomenology. This study shows that the proposed relationship holds in the early time regime. It illustrates the effectiveness of soft computing tools in the analysis of coarse-grained simulations in which patterns exist, but may not easily yield to strict quantitative evaluation. This ability assists us in characterizing the critical early time molecular arrangement during the primary nucleation phase of polymer melt crystallization. In addition to supporting the validity of the proposed kinetics expression, the simulations show that (i) the classical nucleation and growth mechanism is active in the early stages of ordering; (ii) the number of nuclei and their masses grow linearly during this early time regime; and (iii) a fixed inter-nuclei distance is established.

36 MATERIALS SCIENCE↗

Generation Forecasting for Small Run-of-the-River Hydroelectric Systems Using Statistical Learning Models

With an increasing interest in constructing new run-of-the-river (ROR) hydroelectric generation over the more traditional reservoir-based hydroelectric systems, there is an increasing operational challenge due to the volatility of stream flow. The Snohomish County Public Utilities District (SnoPUD) has recently invested in the construction and operation of 3 new run-of-the-river projects in Northwestern Washington along Calligan Creek, Hancock Creek, and Youngs Creek. In order to effectively plan generation dispatch, SnoPUD has expressed interest in the development of an accurate forecasting tool to predict the generation capacity for these ROR systems. The following research project aims to use statistical learning models, namely Hidden Markov Models (HMMs), to predict day-ahead generation capacities for the aforementioned ROR systems. These models are constructed using 12 years of historical stream flow data collected at the intake sites and precipitation data recorded at the National Oceanic and Atmospheric Administration (NOAA) Alpine Meadows station. Four methods of constructing the models are studied for their forecast accuracies, and are compared with the persistence model. Despite using only one set of observable variables, the HMMs are shown to have slight improvements in accuracy over the persistence model approach, which shows great optimism for future work.

13 HYDRO ENERGY↗

Finite-Temperature Structures of Supported Subnanometer Catalysts Inferred via Statistical Learning and Genetic Algorithm-Based Optimization

Single-atom catalysts (SACs) minimize noble metal utilization and can alter the activity and selectivity of supported metal nanoparticles. However, the morphology of active centers, including single atoms and subnanometer clusters of a few atoms, remains elusive due to experimental challenges. The computational cost to describe numerous cluster shapes and sizes makes direct first-principles calculations impractical. We present a computational framework to enable structure determination for single-atom and subnanometer cluster catalysts. As a case study, we obtained the low energy structures of Pd n (n = 1-21) clusters supported on CeO 2 (111), which are critical components of automobile three-way catalysts. Trained on density functional theory data, a three-dimensional cluster expansion is established using statistical learning to describe the Hamiltonian and predict energies of supported Pdn clusters of any structure. Low energy stable and metastable structures are identified using a Metropolis Monte Carlo-based genetic algorithm in the canonical ensemble at 300 K. We observe that supported single atoms sinter to form bilayer clusters and large cluster isomers share similarities in both shape and energy, and elucidate the significance of the support and microstructure on cluster stability. We discovered a simple surrogate structure-energy model, where the energy per atom scales with the square root of the average first coordination number, which can be used to estimate energies and compare the stability of clusters. Our framework, applicable to any metal/support system, fills an important methodological gap to predict the stability of supported metal catalysts in the subnanometer regime.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

$\alpha$-Clustering in atomic nuclei from first principles with statistical learning and the Hoyle state character

A long-standing crucial question with atomic nuclei is whether or not α clustering occurs there. An α particle (helium-4 nucleus) comprises two protons and two neutrons, and may be the building block of some nuclei. This is a very beautiful and fascinating idea, and is indeed plausible because the α particle is particularly stable with a large binding energy. However, direct experimental evidence has never been provided. Here, we show whether and how α(-like) objects emerge in atomic nuclei, by means of state-of-the-art quantum many-body simulations formulated from first principles, utilizing supercomputers including K/Fugaku. The obtained physical quantities exhibit agreement with experimental data. The appearance and variation of the α clustering are shown by utilizing density profiles for the nuclei beryllium-8, -10 and carbon-12. With additional insight by statistical learning, an unexpected crossover picture is presented for the Hoyle state, a critical gateway to the birth of life.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Learning statistical correlation for fast prostate registration in image-guided radiotherapy

Purpose: In adaptive radiation therapy of prostate cancer, fast and accurate registration between the planning image and treatment images of the patient is of essential importance. With the authors' recently developed deformable surface model, prostate boundaries in each treatment image can be rapidly segmented and their correspondences (or relative deformations) to the prostate boundaries in the planning image are also established automatically. However, the dense correspondences on the nonboundary regions, which are important especially for transforming the treatment plan designed in the planning image space to each treatment image space, are remained unresolved. This paper presents a novel approach to learn the statistical correlation between deformations of prostate boundary and nonboundary regions, for rapidly estimating deformations of the nonboundary regions when given the deformations of the prostate boundary at a new treatment image. Methods: The main contributions of the proposed method lie in the following aspects. First, the statistical deformation correlation will be learned from both current patient and other training patients, and further updated adaptively during the radiotherapy. Specifically, in the initial treatment stage when the number of treatment images collected from the current patient is small, the statistical deformation correlation is mainly learned from other training patients. As more treatment images are collected from the current patient, the patient-specific information will play a more important role in learning patient-specific statistical deformation correlation to effectively reflect prostate deformation of the current patient during the treatment. Eventually, only the patient-specific statistical deformation correlation is used to estimate dense correspondences when a sufficient number of treatment images have been acquired from the current patient. Second, the statistical deformation correlation will be learned by using a multiple linear regression (MLR) model, i.e., ridge regression (RR) model, which has the best prediction accuracy than other MLR models such as canonical correlation analysis (CCA) and principal component regression (PCR). Results: To demonstrate the performance of the proposed method, we first evaluate its registration accuracy by comparing the deformation field predicted by our method with the deformation field estimated by the thin plate spline (TPS) based correspondence interpolation method on 306 serial prostate CT images of 24 patients. The average predictive error on the voxels around 5 mm of prostate boundary is 0.38 mm for our method of RR-based correlation model. Also, the corresponding maximum error is 2.89 mm. We then compare the speed for deformation interpolation by different methods. When considering the larger region of interest (ROI) with the size of 512 x 512 x 61, our method takes 24.41 seconds to interpolate the dense deformation field while TPS method needs 6.7 minutes; when considering a small ROI (surrounding prostate) with size of 112 x 110 x 93, our method takes 1.80 seconds, while TPS method needs 25 seconds. Conclusions: Experimental results show that the proposed method can achieve much faster registration speed yet with comparable registration accuracy, compared to the TPS-based correspondence (or deformation) interpolation approach.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

SIMPLIFIED PREDICTIVE MODELS FOR CO₂ SEQUESTRATION PERFORMANCE ASSESSMENT RESEARCH TOPICAL REPORT ON TASK #3 STATISTICAL LEARNING BASED MODELS

We compare two approaches for building a statistical proxy model (metamodel) for CO₂ geologic sequestration from the results of full-physics compositional simulations. The first approach involves a classical Box-Behnken or Augmented Pairs experimental design with a quadratic polynomial response surface. The second approach used a space-filling maxmin Latin Hypercube sampling or maximum entropy design with the choice of five different meta-modeling techniques: quadratic polynomial, kriging with constant and quadratic trend terms, multivariate adaptive regression spline (MARS) and additivity and variance stabilization (AVAS). Simulations results for CO₂ injection into a reservoir-caprock system with 9 design variables (and 97 samples) were used to generate the data for developing the proxy models. The fitted models were validated with using an independent data set and a cross-validation approach for three different performance metrics: total storage efficiency, CO₂ plume radius and average reservoir pressure. The Box-Behnken–quadratic polynomial metamodel performed the best, followed closely by the maximin LHS–kriging metamodel.

Mishra, Srikanta↗

Statistical Learning for Nonlinear Model Reduction from Local Simulations of Stochastic and Particle- and Agent-Based Systems

Stochastic physical systems across the sciences that have very high-dimensional state spaces, with a large number of fast degrees of freedom that force direct simulators to proceed by integration steps that are orders of magnitude smaller than events of interests (e.g., particle collisions). Examples range from molecular motion to dynamics of large populations of cells. A grand challenge in the simulation and understanding of such systems is the systematic construction of accurate, interpretable, reduced models, enabling faster simulations, revealing fundamental properties of the dynamics, and predicting phenomena of interest that the original simulator could not reached with sufficient accuracy or within a given computational budget. In this projected we developed novel statistical estimation/machine learning techniques for analyzing and building empirical reduced models for important families of high-dimensional stochastic systems, in particular: - we developed techniques for estimating interaction kernels in interacting particle- and agent-based systems, which are ubiquitous in Physics, Biology and many other sciences, given observed trajectories of the system; - we developed techniques for nonlinear model reduction for high-dimensional stochastic systems that have a small number of unknown, nonlinear slow variables, and a large number of fast modes, that are possibly of large magnitude, given observed short trajectories of the system in the form of bursts of trajectories from different initial conditions; - we developed novel techniques for estimating linear dynamical systems on graphs when both the dynamics and the underlying graph are unknown, and we have a sparse set of space-time observations; - we considered the problem of estimating an unknown nonlinear observation function of a standard process (e.g. Brownian motion), so that we can recognized if an observed dynamics is "just" a nonlinear version of a known dynamics; we also developed benchmarks for learning algorithms aimed at learning and classifying diffusion processes.

97 MATHEMATICS AND COMPUTING↗