Retrofitting Decision Tree Classifiers Using Kernel Density Estimation
A novel method for combining decision trees and kernel density estimators is proposed. Standard.
Tree Classifiers↗
SEARCH · Search NASA
Results for “kernel density estimation”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The effective number of parameters in kernel density estimation
We devise a new formula for measuring the effective degrees of freedom (EDoF) in kernel density estimation (KDE). Starting from the orthogonal polynomial sequence (OPS) expansion for the ratio of the empirical to the oracle density, we show how convolution with the kernel leads to a new OPS with respect to which one may express the resulting KDE. The expansion coefficients of the two OPS systems can then be related via a kernel sensitivity matrix, which leads to a natural oracle definition of EDoF through the trace operator. Asymptotic properties of the (empirical) plug-in EDoF are worked out through influence functions, and connections with other empirical EDoFs are established. Minimization of Kullback-Leibler divergence is investigated as an alternative to integrated squared error based bandwidth selection rules, yielding a new normal scale rule. The methodology, which arises from a proper oracle formulation and is not restricted to convolution kernels, suggests the possibility of a new bandwidth selection rule based on an information criterion such as AIC.
bandwidth selection↗
Characterization of a maximum-likelihood nonparametric density estimator of kernel type
Kernel type density estimators calculated by the method of sieves. Proofs are presented for the characterization theorem: Let x(1), x(2),...x(n) be a random sample from a population with density f(0). Let sigma 0 and consider estimators f of f(0) defined by (1).
Geman, S.↗
Density Estimation with Mercer Kernels
We present a new method for density estimation based on Mercer kernels. The density estimate can be understood as the density induced on a data manifold by a mixture of Gaussians fit in a feature space. As is usual, the feature space and data manifold are defined with any suitable positive-definite kernel function. We modify the standard EM algorithm for mixtures of Gaussians to infer the parameters of the density. One benefit of the approach is it's conceptual simplicity, and uniform applicability over many different types of data. Preliminary results are presented for a number of simple problems.
Macready, William G.↗
Efficient screening of rare large pit anomalies on polished surfaces using a minimalist sampling scheme
Lawrence Livermore National Laboratory (LLNL) has made significant strides in generating clean energy through its inertial confinement fusion (ICF) experiments. These experiments rely on high-density carbon (HDC) coated shells to encapsulate the fusion fuel. The success of these experiments is heavily dependent on the surface quality of these shells, as even minor imperfections, such as deep pits, can negatively impact fusion yield. Ensuring the required smoothness involves an extensive surface-finishing process that spans approximately 20 stages, making it both time-intensive and resource-demanding. A critical challenge in this process is the need for high-resolution scans to detect rare deep pits, which can be costly and impractical if performed on every shell. This highlights the necessity of developing more efficient scanning methods to optimize time and cost without compromising accuracy. To address these challenges, we introduce a novel approach that employs the multivariate Dvoretzky–Kiefer–Wolfowitz (DKW) inequality to provide a probabilistic upper bound on the error in estimating pit distribution characteristics via a Kernel Density Estimator (KDE). This error bound enables efficient and reliable estimation of pit distribution characteristics at a specified statistical confidence level using a minimal number of surface scans. The integrated DKW-KDE approach was validated through surface-finishing experiments across two batches of HDC-coated shells, demonstrating consistent and robust performance across multiple stages of the surface-finishing experiments. The validation studies suggest that the integrated DKW-KDE approach achieves comparable accuracy in estimating the risk of deleterious large pits with six scans, thus conserving time and resources. Further evaluations show that performance remains consistent across batches and over multiple polishing stages. In conclusion, based on these findings, one can leverage the minimal-scan insights to strategically improve the bottleneck inspection process, thus enhancing the productivity and quality of shell polishing and similar challenging manufacturing processes.
Inertial confinement fusion↗
NASA Langley's Approach to the Sandia's Structural Dynamics Challenge Problem
The objective of this challenge is to develop a data-based probabilistic model of uncertainty to predict the behavior of subsystems (payloads) by themselves and while coupled to a primary (target) system. Although this type of analysis is routinely performed and representative of issues faced in real-world system design and integration, there are still several key technical challenges that must be addressed when analyzing uncertain interconnected systems. For example, one key technical challenge is related to the fact that there is limited data on target configurations. Moreover, it is typical to have multiple data sets from experiments conducted at the subsystem level, but often samples sizes are not sufficient to compute high confidence statistics. In this challenge problem additional constraints are placed as ground rules for the participants. One such rule is that mathematical models of the subsystem are limited to linear approximations of the nonlinear physics of the problem at hand. Also, participants are constrained to use these models and the multiple data sets to make predictions about the target system response under completely different input conditions. Our approach involved initially the screening of several different methods. Three of the ones considered are presented herein. The first one is based on the transformation of the modal data to an orthogonal space where the mean and covariance of the data are matched by the model. The other two approaches worked solutions in physical space where the uncertain parameter set is made of masses, stiffnesses and damping coefficients; one matches confidence intervals of low order moments of the statistics via optimization while the second one uses a Kernel density estimation approach. The paper will touch on all the approaches, lessons learned, validation 1 metrics and their comparison, data quantity restriction, and assumptions/limitations of each approach. Keywords: Probabilistic modeling, model validation, uncertainty quantification, kernel density
Horta, Lucas G.↗
GalaxyFlow: upsampling hydrodynamical simulations for realistic mock stellar catalogues
ABSTRACT Cosmological N-body simulations of galaxies operate at the level of ‘star particles’ with a mass resolution on the scale of thousands of solar masses. Turning these simulations into stellar mock catalogues requires ‘upsampling’ the star particles into individual stars following the same phase-space density. In this paper, we introduce two new upsampling methods. First, we describe GalaxyFlow, a sophisticated upsampling method that utilizes normalizing flows to both estimate the stellar phase-space density and sample from it. Secondly, we improve on existing upsamplers based on adaptive kernel density estimation (KDE), using maximum likelihood estimation to fine-tune the bandwidth for such algorithms in a way that improves both the density estimation accuracy and upsampling results. We demonstrate our upsampling techniques on a neighbourhood of the Solar location in two simulated galaxies: Auriga 6 and h277. Both yield smooth stellar distributions that closely resemble the stellar densities seen in the Gaia DR3 catalogue. Furthermore, we introduce a novel multimodel classifier test to compare the accuracy of different upsampling methods quantitatively. This test confirms that GalaxyFlow more accurately estimates the density of the underlying star particles than methods based on KDE, at the cost of being more computationally intensive.
Lim, Sung Hak (ORCID:0000000330981092)↗
Experimental and Computational Evaluation of Lipidomic In-Source Fragmentation as a Result of Postionization with Matrix-Assisted Laser Desorption/Ionization
Matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI-MSI) can provide spatially resolved molecular information about a sample. Recently, a postionization approach (MALDI-2) has been commercially integrated with MALDI-MSI, allowing for bettered sensitivity and consequent improved spatial resolution. While advantages of MALDI-2 have previously been established, we demonstrate here statistically increased in-source fragmentation (ISF) results from postionization with a commercial instrument. Via lipid standard analyses, known MALDI ISF pathways (e.g., loss of trimethylamine) were statistically increased in MALDI-2 compared to MALDI-1 (65–172% increase in fragmentation). Gas phase molecular modeling with density functional theory estimated that the most-weighted virtual orbitals to excite within lipids involve ester and phosphate bonds. Protonated lipid excitation energies are furthermore red-shifted compared to those of other adduct types [e.g., 254 nm for protonated PC(16:0/18:1)] and approach the MALDI-2 laser energy (266 nm). Analysis of rat brain homogenate detected statistically more positive-ion mode peaks with MALDI-2 (1090) than that with MALDI-1 (719), where Kernel density estimations showed that the majority of this enhancement occurs with low m/z ions (i.e., m/z 75–500). Taken together with the lipid standard data, these observations may indicate ISF due to postionization. Finally, while artifact contributions from matrix blanks were also noted, both experimental and computational data sets suggest that the overall extent of ISF is statistically increased in MALDI-2 compared to MALDI-1.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Uncertainty Estimates for Sonic-Boom Pressure Signatures and Loudness Carpets
A non-intrusive uncertainty quantification method is applied to computational analysis of supersonic, low-boom aircraft. The mean and standard deviation statistics of the pressure waveforms and loudness metrics are evaluated through use of numerical quadrature. The probability density function (p.d.f.) of these outputs is evaluated via kernel density estimation. The simulations use an inviscid, embedded-boundary Cartesian-mesh flow solver in the nearfield combined with an augmented Burgers’ equation solver for propagation in the farfield. The results show that the p.d.f. of the waveform is bimodal at shocks, which makes the mean and standard deviation statistics inappropriate. Despite this limitation, we show that the moment statistics can provide effective assessment of discrepancies when comparing with experimental data. This is demonstrated by presenting uncertainty analysis of a wind-tunnel test and showing that we significantly improve the predictions when we include the test uncertainties in the simulation. Normal distributions are obtained for the ground signature and loudness metrics, which is primarily due to the careful shaping of the low-boom waveform. Separation of variables and error control are used to reduce computational cost. We demonstrate that this is an efficient approach in the sense of balancing numerical errors in the statistics quadrature with discretization errors in the solvers.
ARMD↗
Machine learning materials properties with accurate predictions, uncertainty estimates, domain guidance, and persistent online accessibility
One compelling vision of the future of materials discovery and design involves the use of machine learning (ML) models to predict materials properties and then rapidly find materials tailored for specific applications. However, realizing this vision requires both providing detailed uncertainty quantification (model prediction errors and domain of applicability) and making models readily usable. At present, it is common practice in the community to assess ML model performance only in terms of prediction accuracy (e.g. mean absolute error), while neglecting detailed uncertainty quantification and robust model accessibility and usability. Here, we demonstrate a practical method for realizing both uncertainty and accessibility features with a large set of models. We develop random forest ML models for 33 materials properties spanning an array of data sources (computational and experimental) and property types (electrical, mechanical, thermodynamic, etc). All models have calibrated ensemble error bars to quantify prediction uncertainty and domain of applicability guidance enabled by kernel-density-estimate-based feature distance measures. All data and models are publicly hosted on the Garden-AI infrastructure, which provides an easy-to-use, persistent interface for model dissemination that permits models to be invoked with only a few lines of Python code. We demonstrate the power of this approach by using our models to conduct a fully ML-based materials discovery exercise to search for new stable, highly active perovskite oxide catalyst materials.
domain of applicability↗
Enabling probabilistic learning on manifolds through double diffusion maps
Here, we present a generative learning framework for probabilistic sampling that extends Probabilistic Learning on Manifolds (PLoM), which is designed to generate statistically consistent realizations of a random vector in a finite-dimensional Euclidean space, informed by a (representative) set of observations. In its original form, PLoM constructs a reduced-order probabilistic model by combining three main components: (a) kernel density estimation to approximate the underlying probability measure, (b) Diffusion Maps to characterize the manifold of the data, and (c) a reduced-order Itô Stochastic Differential Equation (ISDE) to sample from the learned distribution. However, its sampling dynamics are posed in the ambient space and the retained number of reduced coordinates is chosen by projection-reconstruction error. In practice, this often (i) requires more coordinates than the data’s intrinsic dimension to achieve stable sampling and (ii) lacks a smooth, basis-independent lifting back to the data domain; moreover, standard Diffusion Maps emphasize harmonic eigenfunctions and can miss non-harmonic latent structure. We address these limitations by decoupling geometry learning from sampling: a first Diffusion Maps pass identifies non-harmonic coordinates on which we formulate a full-order ISDE directly in the latent space, while Double Diffusion Maps captures multiscale geometric features and Geometric Harmonics (GH) learns a smooth lifting map to the ambient variables that is independent of the particular diffusion basis. This hybrid design preserves the system’s dynamical richness with a compact geometric representation and enables principled out-of-sample inference. The effectiveness and robustness of the proposed method are illustrated through two numerical studies: one based on data generated from two-dimensional Hermite polynomial functions and another based on high-fidelity simulations of a detonation wave in a reactive flow.
Double diffusion maps↗
DONKEY: A Flexible and Accurate Algorithm for Clustering
We propose an accurate clustering algorithm suitable for the varied and multidimensional data sets that correspond to temporal snapshots from on-the-fly nonadiabatic trajectory-based simulations of photoexcited dynamics. The algorithm approximates the underlying probability density function using variable kernel density estimation, with local maxima corresponding to cluster centers. Each data point is then assigned to one of the maxima by employing a maximization procedure. Finally, clusters artificially separated by minor fluctuations in the probability density are merged. The algorithm does not require parameter tuning, which ensures flexibility and reduces the risk of bias. It is tested on several synthetic data sets, where it consistently outperforms conventional clustering algorithms. As a final example, the algorithm is applied to the excited dynamics of the norbornadiene ⇌ quadricyclane (C 7 H 8 ) molecular photoswitch, demonstrating how distinct reaction pathways can be identified.
algorithms↗
A meshless stochastic method for Poisson–Nernst–Planck equations
A plethora of biological, physical, and chemical phenomena involve transport of charged particles (ions). Its continuum-scale description relies on the Poisson–Nernst–Planck (PNP) system, which encapsulates the conservation of mass and charge. The numerical solution of these coupled partial differential equations is challenging and suffers from both the curse of dimensionality and difficulty in efficiently parallelizing. We present a novel particle-based framework to solve the full PNP system by simulating a drift–diffusion process with time- and space-varying drift. We leverage Green’s functions, kernel-independent fast multipole methods, and kernel density estimation to solve the PNP system in a meshless manner, capable of handling discontinuous initial states. The method is embarrassingly parallel, and the computational cost scales linearly with the number of particles and dimension. We use a series of numerical experiments to demonstrate both the method’s convergence with respect to the number of particles and computational cost vis-à-vis a traditional partial differential equation solver.
Chemistry↗
Polynomial Chaos Surrogate Construction for Random Fields with Parametric Uncertainty
Engineering and applied science rely on computational experiments to rigorously study physical systems. The mathematical models used to probe these systems are highly complex, and sampling-intensive studies often require prohibitively many simulations for acceptable accuracy. Surrogate models provide a means of circumventing the high computational expense of sampling such complex models. In particular, polynomial chaos expansions (PCEs) have been successfully used for uncertainty quantification studies of deterministic models where the dominant source of uncertainty is parametric. We discuss an extension to conventional PCE surrogate modeling to enable surrogate construction for stochastic computational models that have intrinsic noise in addition to parametric uncertainty. We develop a PCE surrogate on a joint space of intrinsic and parametric uncertainty, enabled by Rosenblatt transformations, which are evaluated via kernel density estimation of the associated conditional cumulative distributions. Furthermore, we extend the construction to random field data via the Karhunen–Loève expansion. We then take advantage of closed-form solutions for computing PCE Sobol indices to perform a global sensitivity analysis of the model which quantifies the intrinsic noise contribution to the overall model output variance. Additionally, the resulting joint PCE is generative in the sense that it allows generating random realizations at any input parameter setting that are statistically approximately equivalent to realizations from the underlying stochastic model. The method is demonstrated on a chemical catalysis example model and a synthetic example controlled by a parameter that enables a switch from unimodal to bimodal response distributions.
97 MATHEMATICS AND COMPUTING↗
PV-Finder: ML Based Algorithm for Primary Vertex Identification
he CMS detector at the High-Luminosity Large Hadron Collider (HL-LHC) will operate in challenging conditions with expected pile-up of up to 200 collisions per bunch crossing, necessitating the development of a more resilient primary vertex (PV) reconstruction method to ensure the integrity of data analysis and the efficiency of the CMS triggering system. This contribution describes preliminary studies on a new ML based PV-Finder method for PV identification. The method is based on a model trained using Kernel Density Estimations (KDEs) derived from the positions of reconstructed tracks at the beamline, incorporating uncertainties from track parameters. It also utilizes target histograms, modeled as Gaussian distributions centered on the actual ground truth values of specific primary vertices.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Learning User Preferences for Sets of Objects
Most work on preference learning has focused on pairwise preferences or rankings over individual items. In this paper, we present a method for learning preferences over sets of items. Our learning method takes as input a collection of positive examples--that is, one or more sets that have been identified by a user as desirable. Kernel density estimation is used to estimate the value function for individual items, and the desired set diversity is estimated from the average set diversity observed in the collection. Since this is a new learning problem, we introduce a new evaluation methodology and evaluate the learning method on two data collections: synthetic blocks-world data and a new real-world music data collection that we have gathered.
preferences↗
Pattern Recognition for a Flight Dynamics Monte Carlo Simulation
The design, analysis, and verification and validation of a spacecraft relies heavily on Monte Carlo simulations. Modern computational techniques are able to generate large amounts of Monte Carlo data but flight dynamics engineers lack the time and resources to analyze it all. The growing amounts of data combined with the diminished available time of engineers motivates the need to automate the analysis process. Pattern recognition algorithms are an innovative way of analyzing flight dynamics data efficiently. They can search large data sets for specific patterns and highlight critical variables so analysts can focus their analysis efforts. This work combines a few tractable pattern recognition algorithms with basic flight dynamics concepts to build a practical analysis tool for Monte Carlo simulations. Current results show that this tool can quickly and automatically identify individual design parameters, and most importantly, specific combinations of parameters that should be avoided in order to prevent specific system failures. The current version uses a kernel density estimation algorithm and a sequential feature selection algorithm combined with a k-nearest neighbor classifier to find and rank important design parameters. This provides an increased level of confidence in the analysis and saves a significant amount of time.
Restrepo, Carolina↗
Using Spatial Density to Characterize Volcanic Fields on Mars
We introduce a new tool to planetary geology for quantifying the spatial arrangement of vent fields and volcanic provinces using non parametric kernel density estimation. Unlike parametricmethods where spatial density, and thus the spatial arrangement of volcanic vents, is simplified to fit a standard statistical distribution, non parametric methods offer more objective and data driven techniques to characterize volcanic vent fields. This method is applied to Syria Planum volcanic vent catalog data as well as catalog data for a vent field south of Pavonis Mons. The spatial densities are compared to terrestrial volcanic fields.
Richardson, J. A.↗