Search NASA⌕ Search

SEARCH · Search NASA

Results for “sparse factorization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2019

A Novel Sparse Compositional Technique Reveals Microbial Perturbations

ABSTRACT The central aims of many host or environmental microbiome studies are to elucidate factors associated with microbial community compositions and to relate microbial features to outcomes. However, these aims are often complicated by difficulties stemming from high-dimensionality, non-normality, sparsity, and the compositional nature of microbiome data sets. A key tool in microbiome analysis is beta diversity, defined by the distances between microbial samples. Many different distance metrics have been proposed, all with varying discriminatory power on data with differing characteristics. Here, we propose a compositional beta diversity metric rooted in a centered log-ratio transformation and matrix completion called robust Aitchison PCA. We demonstrate the benefits of compositional transformations upstream of beta diversity calculations through simulations. Additionally, we demonstrate improved effect size, classification accuracy, and robustness to sequencing depth over the current methods on several decreased sample subsets of real microbiome data sets. Finally, we highlight the ability of this new beta diversity metric to retain the feature loadings linked to sample ordinations revealing salient intercommunity niche feature importance. IMPORTANCE By accounting for the sparse compositional nature of microbiome data sets, robust Aitchison PCA can yield high discriminatory power and salient feature ranking between microbial niches. The software to perform this analysis is available under an open-source license and can be obtained at https://github.com/biocore/DEICODE ; additionally, a QIIME 2 plugin is provided to perform this analysis at https://library.qiime2.org/plugins/q2-deicode .

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1995

A General Purpose Sparse Matrix Parallel Solvers Package

A general purpose solver package for constructing and solving a range of sparse linear systems arising from discretization of PDEs on unstructured meshes is developed. Once the sparse symmetric complex matrix is constructed, it can be solved by either a preconditioned bi-conjugate gradient solver, a two-stage Cholesky LDLT factorization solver, or a hybrid solver combining the above two methods.

solver sparse matrix solvers package PDE↗
DOE OSTI2022

A Novel Fault Diagnosis Approach for Rolling Bearing Based on CWT and Adaptive Sparse Representation

Extraction and enhancement of weak impulse signature is the key of rolling bearing fault prognostics in which case the features are often weak and covered by noise. Tunable Q-factor wavelet transform (TQWT), as an emerging wavelet construction theory developed in a frequency domain explicitly, has the advantages of matching with the specific oscillation behavior of signal components. In this article, an adaptive sparse representation (ASR) method is proposed, which integrates the sparse code shrinkage (SCS) and parameter optimization into TQWT. However, direct application of ASR is difficult to extract fault signatures at the early stage or low-speed operation due to weak fault symptoms and background noise. A novel fault diagnosis strategy based on continuous wavelet transform (CWT) and ASR is investigated. CWT owns significant advantages on multiscale subdivision and weak signal detection. The results of simulated and experimental vibration signal analyses verify the effectiveness of the proposed method in accurately extracting weak impulse features from the noise environment.

Yuan, Xing↗
DOE OSTI2016

HPC-NMF: A High-Performance Parallel Algorithm for Nonnegative Matrix Factorization

NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We propose a high-performance distributed-memory parallel algorithm that computes the factorization by iteratively solving alternating non-negative least squares (NLS) subproblems for $\WW$ and $\HH$. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). As opposed to previous implementation, our algorithm is also flexible: It performs well for both dense and sparse matrices, and allows the user to choose any one of the multiple algorithms for solving the updates to low rank factors $\WW$ and $\HH$ within the alternating iterations.

Kannan, Ramakrishnan↗
DOE OSTI2023

Distributed out-of-memory NMF on CPU/GPU architectures

We propose an efficient distributed out-of-memory implementation of the non-negative matrix factorization (NMF) algorithm for heterogeneous high-performance-computing systems. The proposed implementation is based on prior work on NMFk, which can perform automatic model selection and extract latent variables and patterns from data. In this work, we extend NMFk by adding support for dense and sparse matrix operation on multi-node, multi-GPU systems. The resulting algorithm is optimized for out-of-memory problems where the memory required to factorize a given matrix is greater than the available GPU memory. Memory complexity is reduced by batching/tiling strategies, and sparse and dense matrix operations are significantly accelerated with GPU cores (or tensor cores when available). Input/output latency associated with batch copies between host and device is hidden using CUDA streams to overlap data transfers and compute asynchronously, and latency associated with collective communications (both intra-node and inter-node) is reduced using optimized NVIDIA Collective Communication Library (NCCL) based communicators. Benchmark results show significant improvement, from 32X to 76x speedup, with the new implementation using GPUs over the CPU-based NMFk. Good weak scaling was demonstrated on up to 4096 multi-GPU cluster nodes with approximately 25,000 GPUs when decomposing a dense 340 Terabyte-size matrix and an 11 Exabyte-size sparse matrix of density 10 -6 .

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Low-Rank Correction Methods for Algebraic Domain Decomposition Preconditioners

This study presents a parallel preconditioning method for distributed sparse linear systems, based on an approximate inverse of the original matrix, that adopts a general framework of distributed sparse matrices and exploits domain decomposition (DD) and low-rank corrections. The DD approach decouples the matrix and, once inverted, a low-rank approximation is applied by exploiting the Sherman--Morrison--Woodbury formula, which yields two variants of the preconditioning methods. The low-rank expansion is computed by the Lanczos procedure with reorthogonalizations. Numerical experiments indicate that, when combined with Krylov subspace accelerators, this preconditioner can be efficient and robust for solving symmetric sparse linear systems. Comparisons with pARMS, a DD-based parallel incomplete LU (ILU) preconditioning method, are presented for solving Poisson's equation and linear elasticity problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

High-Resolution 2D Lg and Pg Attenuation Models in the Basin and Range Region with Implications for Frequency-Dependent Q

Here, we develop high–resolution, laterally varying attenuation models for the regional crustal phases of Pg and Lg in the area surrounding the Basin and Range Province in the western United States. The models are part of the characterization effort for the Source Physics Experiment (SPE), a series of chemical explosions at the Nevada National Security Site designed to improve our understanding of explosion source phenomenology. To aid in SPE modeling efforts, we focus on improving our ability to accurately predict amplitudes in a set of narrow frequency bands ranging from 0.5 to 16.0 Hz. To explore constraints at higher frequencies where data become more sparse, we test the robustness of the empirically observed power–law relationship between quality factor Q and frequency (Q=Q 0 f γ ). Our methodology uses a staged approach to consider attenuation, physics–based source terms, site terms, and geometrical spreading contributions to amplitude measurements. Tomographic inversion results indicate that the frequency dependence is a reasonable assumption as attenuation varies laterally for this region through all frequency bands considered. Our 2D Pg and Lg attenuation models correlate with underlying physiographic provinces, with the highest Q located in the Sierra Nevada Mountains and the Colorado plateau. Compared to a best–fitting 1D model for the region, the 2D model provides an 81% variance reduction overall for Lg residuals and a 75% reduction for Pg . These detailed attenuation maps at high frequencies will facilitate further study of local and regional distance P/S amplitude discriminants that are typically used to distinguish between earthquakes and underground explosions.

58 GEOSCIENCES↗
NASA NTRS1995

A General Purpose Sparse Matrix Parallel Solvers Package

A general purpose solver package for constructing and solving a range of sparse linear systems arising from discretization of PDEs on unstructured meshes is developed. Once the sparse symmetric complex matrix is constructed, it can be solved by either a preconditioned bi-conjugate gradient solver, a two-stage Cholesky LDLT factorization solver, or a hybrid solver combining the above two methods. (More detailed than 95-0127).

solver sparse matrix solvers package PDE↗
DOE OSTI2016

A high-performance parallel algorithm for nonnegative matrix factorization

Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A ≈ WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We propose a high-performance distributed-memory parallel algorithm that computes the factorization by iteratively solving alternating non-negative least squares (NLS) subproblems for W and H. It maintains the data and factor matrices in memory (distributed across processors), uses MPI for interprocessor communication, and, in the dense case, provably minimizes communication costs (under mild assumptions). As opposed to previous implementations, our algorithm is also flexible: (1) it performs well for both dense and sparse matrices, and (2) it allows the user to choose any one of the multiple algorithms for solving the updates to low rank factors W and H within the alternating iterations. We demonstrate the scalability of our algorithm and compare it with baseline implementations, showing significant performance improvements.

Kannan, Ramakrishnan↗
NASA NTRS1989

A one- and two-layer model for estimating evapotranspiration with remotely sensed surface temperature and ground-based meteorological data over partial canopy cover

Surface-air temperature differences are commonly used in a bulk resistance equation for estimating sensible heat flux (H), which is inserted in the one-dimensional energy balance equation to solve for the latent heat flux (LE) as a residual. Serious discrepancies between estimated and measured LE have been observed for partial-canopy-cover conditions, which are mainly attributed to inappropriate estimates of H. To improve the estimates of H over sparse canopies, one- and two-layer resistance models that account for some of the factors causing poor agreement are developed. The utility of the two models is tested with remotely sensed and micrometeorological data for a furrowed cotton field with 20 percent cover and a dry soil surface. It is found that the one-layer model performs better than the two-layer model when a theoretical bluff-body correction for heat transfer is used instead of an empirical adjustment; otherwise, the two-layer model is better.

Kustas, William P.↗
NASA NTRS2010

Analysis of Snow Bidirectional Reflectance from ARCTAS Spring-2008 Campaign

The spring 2008 Arctic Research of the Composition of the Troposphere from Aircraft and Satellites (ARCTAS) experiment was one of major intensive field campaigns of the International Polar Year aimed at detailed characterization of atmospheric physical and chemical processes in the Arctic region. A part of this campaign was a unique snow bidirectional reflectance experiment on the NASA P-3B aircraft conducted on 7 and 15 April by the Cloud Absorption Radiometer (CAR) jointly with airborne Ames Airborne Tracking Sunphotometer (AATS) and ground-based Aerosol Robotic Network (AERONET) sunphotometers. The CAR data were atmospherically corrected to derive snow bidirectional reflectance at high 1 degree angular resolution in view zenith and azimuthal angles along with surface albedo. The derived albedo was generally in good agreement with ground albedo measurements collected on 15 April. The CAR snow bidirectional reflectance factor (BRF) was used to study the accuracy of analytical Ross-Thick Li-Sparse (RTLS), Modified Rahman-Pinty-Verstraete (MRPV) and Asymptotic Analytical Radiative Transfer (AART) BRF models. Except for the glint region (azimuthal angles phi less than 40 degrees), the best fit MRPV and RTLS models fit snow BRF to within 0.05. The plane-parallel radiative transfer (PPRT) solution was also analyzed with the models of spheres, spheroids, randomly oriented fractal crystals, and with a synthetic phase function. The latter merged the model of spheroids for the forward scattering angles with the fractal model in the backscattering direction. The PPRT solution with synthetic phase function provided the best fit to measured BRF in the full range of angles. Regardless of the snow grain shape, the PPRT model significantly over-/underestimated snow BRF in the glint/backscattering regions, respectively, which agrees with other studies. To improve agreement with experiment, we introduced a model of macroscopic snow surface roughness by averaging the PPRT solution over the slope distribution function and by adding a simple model of shadows. With macroscopic roughness described by two parameters, the AART model achieved an accuracy of about plus or minus 0.05 with a possible bias of plus or minus 0.03 in the spectral range 0.4-2.2 micrometers. This high accuracy holds at view zenith angles below 55-60 degrees covering the practically important range for remote sensing applications, and includes both glint and backscattering directions.

Lyapustin, A.↗
NASA NTRS2012

Multi-Angle Implementation of Atmospheric Correction for MODIS (MAIAC). Part 3: Atmospheric Correction

This paper describes the atmospheric correction (AC) component of the Multi-Angle Implementation of Atmospheric Correction algorithm (MAIAC) which introduces a new way to compute parameters of the Ross-Thick Li-Sparse (RTLS) Bi-directional reflectance distribution function (BRDF), spectral surface albedo and bidirectional reflectance factors (BRF) from satellite measurements obtained by the Moderate Resolution Imaging Spectroradiometer (MODIS). MAIAC uses a time series and spatial analysis for cloud detection, aerosol retrievals and atmospheric correction. It implements a moving window of up to 16 days of MODIS data gridded to 1 km resolution in a selected projection. The RTLS parameters are computed directly by fitting the cloud-free MODIS top of atmosphere (TOA) reflectance data stored in the processing queue. The RTLS retrieval is applied when the land surface is stable or changes slowly. In case of rapid or large magnitude change (as for instance caused by disturbance), MAIAC follows the MODIS operational BRDF/albedo algorithm and uses a scaling approach where the BRDF shape is assumed stable but its magnitude is adjusted based on the latest single measurement. To assess the stability of the surface, MAIAC features a change detection algorithm which analyzes relative change of reflectance in the Red and NIR bands during the accumulation period. To adjust for the reflectance variability with the sun-observer geometry and allow comparison among different days (view geometries), the BRFs are normalized to the fixed view geometry using the RTLS model. An empirical analysis of MODIS data suggests that the RTLS inversion remains robust when the relative change of geometry-normalized reflectance stays below 15%. This first of two papers introduces the algorithm, a second, companion paper illustrates its potential by analyzing MODIS data over a tropical rainforest and assessing errors and uncertainties of MAIAC compared to conventional MODIS products.

Lyapustin, A.↗
DOE OSTI2018

The face of epicardial and endocardial derived cells in zebrafish

Highlights: • Growth factors cocktails can to lead a heart regeneration in ex-vivo culture. • Ex-vivo cultures have permitted to characterize the cells with mobility capability. • Epicardial- and endocardial-derived cells can be expanded for months in culture. • Whole heart or cells culture open a new field of translational studies. Zebrafish hearts can regenerate through activation of growth factors and trans-differentiation of fibroblasts, epicardial, myocardial and endocardial cells, all positive for GATA4 during the process. A possible model of regeneration of the whole heart and the regenerating cells in ex-vivo culture is presented here by a stimulation of cocktail of growth factors. In ex-vivo growth-factors-supplemented culture the heart regeneration was quite complete without signs of fibrosis. Epicardial- and endocardial-derived cells have been analyzed by electron microscopy evidencing two main types: 1) larger/prismatic and 2) small/rounded. Type (1) showed on the surface protein-sculptures, while type(2) was smooth with sparse globular proteins. To confirm their nature we have contemporarily analyzed their proliferative capability and markers-positivity. The cells treated by growth factors have at least two-fold more proliferation with GATA4-positivity. The type (1) cell evidenced WT1+(marker of embryonic epicardium); the type (2) showed NFTA2+(marker of embryonic endocardium); whereas cTNT-cardiotroponin was negative. Under growth factors stimulation, GATA4+/WT1+ and GATA4+/NFTA2+ could be suitable candidates to be the cells with capability to move in/out of the tissue, probably by using their integrins, and it opens the possibility to have long term selected culture to future characterization.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Inferring demographic and selective histories from population genomic data using a 2-step approach in species with coding-sparse genomes: an application to human data

Abstract The demographic history of a population, and the distribution of fitness effects (DFE) of newly arising mutations in functional genomic regions, are fundamental factors dictating both genetic variation and evolutionary trajectories. Although both demographic and DFE inference has been performed extensively in humans, these approaches have generally either been limited to simple demographic models involving a single population, or, where a complex population history has been inferred, without accounting for the potentially confounding effects of selection at linked sites. Taking advantage of the coding-sparse nature of the genome, we propose a 2-step approach in which coalescent simulations are first used to infer a complex multi-population demographic model, utilizing large non-functional regions that are likely free from the effects of background selection. We then use forward-in-time simulations to perform DFE inference in functional regions, conditional on the complex demography inferred and utilizing expected background selection effects in the estimation procedure. Throughout, recombination and mutation rate maps were used to account for the underlying empirical rate heterogeneity across the human genome. Importantly, within this framework it is possible to utilize and fit multiple aspects of the data, and this inference scheme represents a generalized approach for such large-scale inference in species with coding-sparse genomes.

Soni, Vivak (ORCID:0000000294969562)↗
DOE OSTI2022

Machine learning and shallow groundwater chemistry to identify geothermal prospects in the Great Basin, USA

This study discovers various geothermal prospects in the Great Basin, USA based on shallow groundwater chemical (geochemical) data. The geochemical data are expected to include hidden (latent) information that is a proxy for geothermal prospectivity. We processed the sparse geochemical data in the Great Basin at 14,341 locations including 18 attributes. Next, a non-negative matrix factorization with customized k-means clustering is applied to the geochemical data matrix that automatically finds three hidden geothermal signatures representing modestly, moderately, and highly confident geothermal prospects. The algorithm also evaluated the probability of occurrence of these types of resources through the studied region. There is a consistency between regional geothermal prospectivity as estimated by our ML methodology and the traditional play fairway analysis conducted over a portion of the study area. We also identify the dominant data attributes associated with each signature. Finally, our ML analyses allow us to reconstruct attributes from sparse into continuous over the study domain. The predicted continuous attributes can be used for future detailed geothermal explorations in the Great Basin.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Artificial intelligence-empowered cellular morphometric risk score improves prognostic stratification of cutaneous squamous cell carcinoma

Abstract Background Risk stratification of cutaneous squamous cell carcinoma (cSCC) is essential for managing patients. Objectives To determine if artificial intelligence and machine learning might help to stratify patients with cSCC by risk using more than solely clinical and histopathological factors. Methods We retrieved a retrospective cohort of 104 patients whose cSCCs had been excised with clear margins. Clinical and histopathological risk factors were evaluated. Haematoxylin and eosin-stained slides were scanned and analysed by an algorithm based on the stacked predictive sparse decomposition technique. Cellular morphometric biomarkers (CMBs) were identified via machine learning and used to derive a cellular morphometric risk score (CMRS) that classified cSCCs into clusters of differential prognoses. Concordance analysis, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV) and accuracy were calculated and compared with results obtained with the Brigham and Women’s Hospital (BWH) staging system. The performance of the combination of the BWH staging system and the CMBs was also analysed. Results There were no differences among the CMRS groups in terms of clinical and histopathological risk factors and T-stage assignment, but there were significant differences in prognosis. Combining the CMRS with BWH staging systems increased distinctiveness and improved prognostic performance. C-indices were 0.91 local recurrence and 0.91 for nodal metastasis when combining the two approaches. The NPV was 94.41% and 96.00%, the PPV was 36.36% and 41.67%, and accuracy reached 86.75% and 89.16%, respectively, with the combined approach. Conclusions CMRS is helpful for cSCC risk stratification beyond classic clinical and histopathological risk features. Combining the information from the CMRS and the BWH staging system offers outstanding prognostic performance for patients with high-risk cSCC.

Pérez-Baena, Manuel J.↗
NASA NTRS2001

Crystallization of bFGF-DNA Aptamer Complexes Using a Sparse Matrix Designed for Protein-Nucleic Acid Complexes

The Sparse Matrix approach for obtaining lead crystallization conditions has proven to be very fruitful for the crystallization of proteins and nucleic acids. Here we report a Sparse Matrix developed specifically for the crystallization of protein-DNA complexes. This method is rapid and economical, typically requiring 2.5 mg of complex to test 48 conditions. The method was originally developed to crystallize basic fibroblast growth factor (bFGF) complexed with DNA sequences identified through in vitro selection, or SELEX, methods. Two DNA aptamers that bind with approximately nanomolar affinity and inhibit the angiogenic properties of bFGF were selected for co-crystallization. The Sparse Matrix produced lead crystallization conditions for both bFGF-DNA complexes.

Cannone, Jaime J.↗