Search NASA⌕ Search

SEARCH · Search NASA

Results for “Kernel methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Predicting carbon dioxide and energy fluxes across global FLUXNET sites with regression algorithms

Spatio-temporal fields of land–atmosphere fluxes derived from data-driven models can complement simulations by process-based land surface models. While a number of strategies for empirical models with eddy-covariance flux data have been applied, a systematic intercomparison of these methods has been missing so far. In this study, we performed a cross-validation experiment for predicting carbon dioxide, latent heat, sensible heat and net radiation fluxes across different ecosystem types with 11 machine learning (ML) methods from four different classes (kernel methods, neural networks, tree methods, and regression splines). We applied two complementary setups: (1) 8-day average fluxes based on remotely sensed data and (2) daily mean fluxes based on meteorological data and a mean seasonal cycle of remotely sensed variables. The patterns of predictions from different ML and experimental setups were highly consistent. There were systematic differences in performance among the fluxes, with the following ascending order: net ecosystem exchange ( R 2 < 0.5), ecosystem respiration ( R 2 > 0.6), gross primary production ( R 2 > 0.7), latent heat ( R 2 > 0.7), sensible heat ( R 2 > 0.7), and net radiation ( R 2 > 0.8). The ML methods predicted the across-site variability and the mean seasonal cycle of the observed fluxes very well ( R 2 > 0.7), while the 8-day deviations from the mean seasonal cycle were not well predicted ( R 2 < 0.5). Fluxes were better predicted at forested and temperate climate sites than at sites in extreme climates or less represented by training data (e.g., the tropics). Finally, the evaluated large ensemble of ML-based models will be the basis of new global flux products.

54 ENVIRONMENTAL SCIENCES↗

Application of Quantum Machine Learning to High Energy Physics Analysis at LHC Using Quantum Computer Simulators and Quantum Computer Hardware

Machine learning enjoys widespread success in High Energy Physics (HEP) analyses at LHC. However the ambitious HL-LHC program will require much more computing resources in the next two decades. Quantum computing may offer speed-up for HEP physics analyses at HL-LHC, and can be a new computational paradigm for big data analyses in High Energy Physics.We have successfully employed three methods (1) Variational Quantum Classifier (VQC) method, (2) Quantum Support Vector Machine Kernel (QSVM-kernel) method and (3) Quantum Neural Network (QNN) method for two LHC flagship analyses: ttH (Higgs production in association with two top quarks) and H->mumu (Higgs decay to two muons, the second generation fermions). We shall address the progressive improvements in performance from method (1) to method (3).We will present our experiences and results of a study on LHC High Energy Physics data analyses with IBM Quantum Simulator and Quantum Hardware (using IBM Qiskit framework), Google Quantum Simulator (using Google Cirq framework), and Amazon Quantum Simulator (using Amazon Braket cloud service). The work is in the context of a Qubit platform (a gate-model quantum computer). Taking into account the present limitation of hardware access, different quantum machine learning methods are studied on simulators and the results are compared with classical machine learning methods (BDT, classical Support Vector Machine and classical Neural Network). Furthermore, we do apply quantum machine learning on IBM quantum hardware to compare performance between quantum simulator and quantum hardware. The work is performed by an international and interdisciplinary collaboration with the Department of Physics and Department of Computer Sciences of University of Wisconsin, CERN Quantum Technology Initiative, IBM Research Zurich, IBM T.J. Watson Research Center, Fermilab Quantum Institute, BNL Computational Science Initiative, State University of New York at Stony Brook, and Quantum Computing and AI Research of Amazon Web Services. This work pioneers a close collaboration of academic institutions with industrial corporations in the High Energy Physics analyses effort. Though the size of event samples in future HL-LHC physics and the limited number of qubits pose some challenges to the Quantum Machine learning studies for High Energy Physics, more advanced quantum computers with larger number of qubits, reduced noise and improved running time (as envisioned by IBM and Google) may outperform classical machine learning in both classification power and in speed.Although the era of efficient quantum computing may still be years away, we have made promising progress and obtained preliminary results in applying quantum machine learning to High Energy Physics. A PROOF OF PRINCIPLE.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Systems and methods for customizing kernel machines with deep neural networks

A method including receiving an input data set. The input data set can include one of a feature domain set or a kernel matrix. The method also can include constructing dense embeddings using: (i) Nyström approximations on the input data set when the input data set comprises the kernel matrix, and (ii) clustered Nyström approximations on the input data set when the input data set comprises the feature domain set. The method additionally can include performing representation learning on each of the dense embeddings using a multi-layer fully-connected network for each of the dense embeddings to generate latent representations corresponding to each of the dense embeddings. The method further can include applying a fusion layer to the latent representations corresponding to the dense embeddings to generate a combined representation. The method additionally can include performing classification on the combined representation. Other embodiments of related systems and methods are also disclosed.

Song, Huan↗

Data assimilation in operator algebras

We develop an algebraic framework for sequential data assimilation of partially observed dynamical systems. In this framework, Bayesian data assimilation is embedded in a nonabelian operator algebra, which provides a representation of observables by multiplication operators and probability densities by density operators (quantum states). In the algebraic approach, the forecast step of data assimilation is represented by a quantum operation induced by the Koopman operator of the dynamical system. Moreover, the analysis step is described by a quantum effect, which generalizes the Bayesian observational update rule. Projecting this formulation to finite-dimensional matrix algebras leads to computational schemes that are i) automatically positivity-preserving and ii) amenable to consistent data-driven approximation using kernel methods for machine learning. Moreover, these methods are natural candidates for implementation on quantum computers. Applications to the Lorenz 96 multiscale system and the El Niño Southern Oscillation in a climate model show promising results in terms of forecast skill and uncertainty quantification.

97 MATHEMATICS AND COMPUTING↗

Magnetic reversal spurts: Rain gauges for comet showers

Abrupt increases in the rate of magnetic reversals (magnetic reversal spurts) were first studied by many others. They hypothesized that spurts result from increased turbulence in the earth's core dynamo during episodes of intense bolide bombardment of the earth. Mechanisms for creating episodes of intense bombardment of the earth involve gravitational perturbation of the Oort cloud of comets, either by a hidden planet, a solar companion, or massive matter in the galactic plane. Herein, the time variation in reversal rate is analyzed using methods of statistical density estimation. A smooth, continuous estimate of reversal rate is obtained using an adaptive kernel method, in which the kernel width is adjusted as a function of reversal rate. The estimates near the ends of the data series (at 165 my ago and the present) are obtained by extending the data by reflection. The results show that the reversal spurts are not associated demonstrably with extinctions or well-dated impacts. If the spurts do record episodes of intense bombardment of the earth, then the mass extinctions do not, in general, occur at times of impacts. Furthermore, the large impact craters seen are not obviously related to the spurts, suggesting that the craters may have been caused by bolides of a different nature and with a different temporal pattern. However, the most simple explanation seems to be that the spurts do not record comet showers, either because the recording mechanism suggested by Muller and Morris is not effective or because comet showers are not triggered in the ways considered by Hut et al.

Lutz, T. M.↗

Singularity Preserving Numerical Methods for Boundary Integral Equations

In the past twelve months (May 8, 1995 - May 8, 1996), under the cooperative agreement with Division of Multidisciplinary Optimization at NASA Langley, we have accomplished the following five projects: a note on the finite element method with singular basis functions; numerical quadrature for weakly singular integrals; superconvergence of degenerate kernel method; superconvergence of the iterated collocation method for Hammersteion equations; and singularity preserving Galerkin method for Hammerstein equations with logarithmic kernel. This final report consists of five papers describing these projects. Each project is preceeded by a brief abstract.

Kaneko, Hideaki↗

Optimizing Kernel Machines Using Deep Learning

Building highly non-linear and non-parametric models is central to several state-of-the-art machine learning systems. Kernel methods form an important class of techniques that induce a reproducing kernel Hilbert space (RKHS) for inferring non-linear models through the construction of similarity functions from data. These methods are particularly preferred in cases where the training data sizes are limited and when prior knowledge of the data similarities is available. Despite their usefulness, they are limited by the computational complexity and their inability to support end-to-end learning with a task specific objective. On the other hand, deep neural networks have become the de facto solution for end-to-end inference in several learning paradigms. In this article, we explore the idea of using deep architectures to perform kernel machine optimization, for both computational efficiency and end-to-end inferencing. To this end, we develop the DKMO (Deep Kernel Machine Optimization) framework, that creates an ensemble of dense embeddings using Nystrõm kernel approximations and utilizes deep learning to generate task-specific representations through the fusion of the embeddings. Intuitively, the filters of the network are trained to fuse information from an ensemble of linear subspaces in the RKHS. Furthermore, we introduce the kernel dropout regularization to enable improved training convergence. Finally, we extend this framework to the multiple kernel case, by coupling a global fusion layer with pre-trained deep kernel machines for each of the constituent kernels. Using case studies with limited training data, and lack of explicit feature sources, we demonstrate the effectiveness of our framework over conventional model inferencing techniques.

97 MATHEMATICS AND COMPUTING↗

A Study of Clustering Techniques and Hierarchical Matrix Formats for Kernel Ridge Regression

We present memory-efficient and scalable algorithms for kernel methods used in machine learning. Using hierarchical matrix approximations for the kernel matrix the memory requirements, the number of floating point operations, and the execution time are drastically reduced compared to standard dense linear algebra routines. We consider both the general H matrix hierarchical format as well as Hierarchically Semi-Separable (HSS) matrices. Furthermore, we investigate the impact of several preprocessing and clustering techniques on the hierarchical matrix compression. Effective clustering of the input leads to a ten-fold increase in efficiency of the compression. The algorithms are implemented using the STRUMPACK solver library. These results confirm that - with correct tuning of the hyperparameters - classification using kernel ridge regression with the compressed matrix does not lose prediction accuracy compared to the exact - not compressed - kernel matrix and that our approach can be extended to O(1M) datasets, for which computation with the full kernel matrix becomes prohibitively expensive. We present numerical experiments in a distributed memory environment up to 1,024 processors of the NERSC's Cori supercomputer using well-known datasets to the machine learning community that range from dimension 8 up to 784.

97 MATHEMATICS AND COMPUTING↗

Approximate l-fold cross-validation with Least Squares SVM and Kernel Ridge Regression

Kernel methods have difficulties scaling to large modern data sets. The scalability issues are based on computational and memory requirements for working with a large matrix. These requirements have been addressed over the years by using low-rank kernel approximations or by improving the solvers scalability. However, Least Squares Support VectorMachines (LS-SVM), a popular SVM variant, and Kernel Ridge Regression still have several scalability issues. In particular, the O(n^3) computational complexity for solving a single model, and the overall computational complexity associated with tuning hyperparameters are still major problems. We address these problems by introducing an O(n log n) approximate l-fold cross-validation method that uses a multi-level circulant matrix to approximate the kernel. In addition, we prove our algorithm s computational complexity and present empirical runtimes on data sets with approximately 1 million data points. We also validate our approximate method s effectiveness at selecting hyperparameters on real world and standard benchmark data sets. Lastly, we provide experimental results on using a multi-level circulant kernel approximation to solve LS-SVM problems with hyperparameters selected using our method.

Edwards, Richard E↗

Approximate l-Fold Cross-Validation with Least Squares SVM and Kernel Ridge Regression

Kernel methods have difficulties scaling to large modern data sets. The scalability issues are based on computational and memory requirements for working with a large matrix. These requirements have been addressed over the years by using low-rank kernel approximations or by improving the solvers' scalability. However, Least Squares Support Vector Machines (LS-SVM), a popular SVM variant, and Kernel Ridge Regression still have several scalability issues. In particular, the O(n^3) computational complexity for solving a single model, and the overall computational complexity associated with tuning hyper parameters are still major problems. We address these problems by introducing an O(nlog n) approximate l-fold cross-validation method that uses a multi-level circulant matrix to approximate the kernel. In addition, we prove our algorithm's computational complexity and present empirical runtimes on data sets with approximately one million data points. We also validate our approximate method's effectiveness at selecting hyper parameters on real world and standard benchmark data sets. Lastly, we provide experimental results on using a multi level circulant kernel approximation to solve LS-SVM problems with hyper parameters selected using our method.

Computer Science↗

Numerical evidence against advantage with quantum fidelity kernels on classical data

Quantum machine learning techniques are commonly considered one of the most promising candidates for demonstrating practical quantum advantage. In particular, quantum kernel methods have been demonstrated to be able to learn certain classically intractable functions efficiently if the kernel is well aligned with the target function. In the more general case, quantum kernels are known to suffer from exponential “flattening” of the spectrum as the number of qubits grows, preventing generalization and necessitating the control of the inductive bias by hyperparameters. We show that the general-purpose hyperparameter-tuning techniques proposed to improve the generalization of quantum kernels lead to the kernel becoming well approximated by a classical kernel, removing the possibility of quantum advantage. We provide extensive numerical evidence for this phenomenon utilizing multiple previously studied quantum feature maps and both synthetic and real data. Our results show that unless novel techniques are developed to control the inductive bias of quantum kernels, they are unlikely to provide a quantum advantage on classical data that lacks special structure.

Quantum Information, Science & Technology↗

(Active) Learning on Groups of Data with Information-Theoretic Estimators

A wide range of machine learning problems, including astronomical inference about galaxy clusters, scene classification, parametric statistical inference, and predictions of public opinion, can be well-modeled as learning a function on (samples from) distributions. This project explores problems in learning such functions via kernel methods, particularly for large-scale problems. When learning from large numbers of distributions, the computation of typical methods scales between quadratically and cubically, and so they are not amenable to large datasets. We investigate the approach of approximate embeddings into Euclidean spaces such that inner products in the embedding space approximate kernel values between the source distributions. We first improve the understanding of the workhorse methods of random Fourier features: we show that of the two approaches in common usage, one is strictly superior. We then present a new embedding for a class of information-theoretic distribution distances, and evaluate it and existing embeddings on several real-world applications.

97 MATHEMATICS AND COMPUTING↗

Convergence of Weak-SINDy Surrogate Models

In this paper, we give an in-depth error analysis for surrogate models generated by a variant of the Sparse Identification of Nonlinear Dynamics (SINDy) method. We start with an overview of a variety of nonlinear system identification techniques, namely SINDy, weak-SINDy, and the occupation kernel method. Under the assumption that the dynamics are a finite linear combination of a set of basis functions, these methods establish a linear system to recover coefficients. We illuminate the structural similarities between these techniques and establish a projection property for the weak-SINDy technique. Following the overview, we analyze the error of surrogate models generated by a simplified version of weak-SINDy. In particular, under the assumption of boundedness of a composition operator given by the solution, we show that (i) the surrogate dynamics converges towards the true dynamics and (ii) the solution of the surrogate model is reasonably close to the true solution. Finally, as an application, we discuss the use of a combination of weak-SINDy surrogate modeling and proper orthogonal decomposition (POD) to build a surrogate model for partial differential equations (PDEs).

97 MATHEMATICS AND COMPUTING↗

Gaussian Process Regression under Computational and Epistemic Misspecification

Gaussian process regression is a classical kernel method for function estimation and data interpolation. In large data applications, computational costs can be reduced using low-rank or sparse approximations of the kernel. This paper investigates the effect of such kernel approximations on the interpolation error. We introduce a unified framework to analyze Gaussian process regression under important classes of computational misspecification: Karhunen-Loève expansions that result in low-rank kernel approximations, multiscale wavelet expansions that induce sparsity in the covariance matrix, and finite element representations that induce sparsity in the precision matrix. Furthermore, our theory also accounts for epistemic misspecification in the choice of kernel parameters.

Gaussian process regression↗

Comparison of Zn accumulation and speciation in kernels of sweetcorn and maize differing in maturity

Abstract Background and Aims Understanding the speciation of Zn in edible portions of crops helps identify the most effective biofortification strategies to increase the supply of nutrients for improving the health and nutrition of consumers. Methods Kernels of 12 sweetcorn and three maize (Zea mays) varieties were analysed for Zn concentration and content. The speciation of the Zn in the embryos, endosperms and whole kernels at 21, 28 and 56 days after pollination (DAP) was then examined for one maize and one sweetcorn variety using synchrotron-based X-ray absorption spectroscopy (XAS). Key Results Averaged across all sweetcorn and maize varieties at 21 DAP, the embryo contributed 27–29% of the whole kernel Zn whilst the endosperm contributed 71–73 %. While sweetcorn embryos contributed a lower proportion to the total kernel Zn than those of maize, the proportion of total Zn in the embryo increased as kernels aged for both varieties, reaching 33 % for sweetcorn and 49% for maize at 28 DAP. Using XAS, it was predicted that an average of 90 % of the Zn in the embryos was present as Zn-phytate, while in the endosperm the Zn was primarily complexed with an N-containing ligand such as histidine and to a lesser extent with phytate. However, in maize endosperms, it was also observed that the proportion of Zn present as Zn-phytate increased as the kernel matured, thereby also probably decreasing its bioavailability in these mature maize kernels. Conclusions The apparent low bioavailability of Zn supplied in maize at its consumption stage (i.e. mature kernels) probably undermines the effectiveness of biofortification of this crop. Conversely, successful biofortification of Zn in sweetcorn and green maize consumed as immature kernels could potentially provide a good source of bioavailable Zn in human diets.

Cheah, Zhong Xiang↗

Evaluation of CloudSat Radiative Kernels Using ARM and CERES Observations and ERA5 Reanalysis

Despite the widespread use of the radiative kernel technique for studying radiative feedbacks and radiative forcings, there has not been any systematic, observation-based validation of the radiative kernel method. Here, we utilize observed and reanalyzed radiative fluxes and atmospheric profiles from the Atmospheric Radiation Measurement (ARM) program and ERA5 reanalysis to assess a set of observation-based radiative kernels from CloudSat for six ARM sites. The CloudSat radiative kernels, convoluted with the ERA5 state variables, can almost perfectly reconstruct the monthly anomalies of shortwave (SW) and longwave (LW) radiative fluxes in ERA5 at the surface (SFC) and top-of-atmosphere (TOA) with correlations significantly being greater than 0.95. The biases of kernel-estimated flux anomalies calculated using the ARM-observed state variables can be more than twice as large when compared with the ARM-observed surface flux anomalies and Clouds and Earth’s Radiant Energy System (CERES) observed anomalies at the TOA. Generally, clouds contribute to most (>60%) of the variance of flux anomalies at Southern Great Plain (SGP), Tropical Western Pacific (TWP), and Eastern North Atlantic (ENA), and surface albedo dominates (>69%) the variance of SW flux anomalies at North Slope of Alaska (NSA). Furthermore, the radiative kernels exhibit the lowest correlation (r ~ [0.55,0.85]) when reconstructing SFC LW flux anomalies at SGP, TWP, and ENA, whose biases are related to the possibility that the kernels may not fully capture the characteristics associated with MJO and ENSO at TWP and the presence of clouds at SGP and ENA.

54 ENVIRONMENTAL SCIENCES↗

Evaluation of CloudSat Radiative Kernels Using ARM and CERES Observations and ERA5 Reanalysis

Despite the widespread use of the radiative kernel technique for studying radiative feedbacks and radiative forcings, there has not been any systematic, observation-based validation of the radiative kernel method. Here, we utilize observed and reanalyzed radiative fluxes and atmospheric profiles from the Atmospheric Radiation Measurement (ARM) program and ERA5 reanalysis to assess a set of observation-based radiative kernels from CloudSat for six ARM sites. The CloudSat radiative kernels, convoluted with the ERA5 state variables, can almost perfectly reconstruct the monthly anomalies of shortwave (SW) and longwave (LW) radiative fluxes in ERA5 at the surface (SFC) and top-of atmosphere (TOA) with correlations significantly being greater than 0.95. The biases of kernel-estimated flux anomalies calculated using the ARM-observed state variables can be more than twice as large when compared with the ARM-observed surface flux anomalies and Clouds and Earth’s Radiant Energy System (CERES) observed anomalies at the TOA. Generally, clouds contribute to most (>60%) of the variance of flux anomalies at Southern Great Plain (SGP), Tropical Western Pacific (TWP), and Eastern North Atlantic (ENA), and surface albedo dominates (>69%) the variance of SW flux anomalies at North Slope of Alaska (NSA). The radiative kernels exhibit the lowest correlation (r~[0.55,0.85]) when reconstructing SFC LW flux anomalies at SGP, TWP, and ENA, whose biases are related to the possibility that the kernels may not fully capture the characteristics 47 associated with MJO and ENSO at TWP and the presence of clouds at SGP and ENA.

Radiative kernels↗

Recovering missing CFD data for high-order discretizations using deep neural networks and dynamics learning

Data I/O poses a significant bottleneck in large-scale CFD simulations; thus, practitioners would like to significantly reduce the number of times the solution is saved to disk, yet retain the ability to recover any field quantity (at any time instance) a posteriori . The objective of this work is therefore to accurately recover missing CFD data a posteriori at any time instance, given that the solution has been written to disk at only a relatively small number of time instances. We consider in particular high-order discretizations (e.g., discontinuous Galerkin), as such techniques are becoming increasingly popular for the simulation of highly separated flows. To satisfy this objective, this work proposes a methodology consisting of two stages: 1) dimensionality reduction and 2) dynamics learning. For dimensionality reduction, we propose a novel hierarchical approach. First, the method reduces the number of degrees of freedom within each element of the high-order discretization by applying autoencoders from deep learning. Second, the methodology applies principal component analysis to compress the global vector of encodings. This leads to a low-dimensional state, which associates with a nonlinear embedding of the original CFD data. For dynamics learning, we propose to apply regression techniques (e.g., kernel methods) to learn the discrete-time velocity characterizing the time evolution of this low-dimensional state. In conclusion, a numerical example on a large-scale CFD example characterized by nearly 13 million degrees of freedom illustrates the suitability of the proposed method in an industrial setting.

97 MATHEMATICS AND COMPUTING↗