Search NASA⌕ Search

SEARCH · Search NASA

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

MixSim : An R Package for Simulating Data to Study Performance of Clustering Algorithms

The R package MixSim is a new tool that allows simulating mixtures of Gaussian distributions with different levels of overlap between mixture components. Pairwise overlap, defined as a sum of two misclassification probabilities, measures the degree of interaction between components and can be readily employed to control the clustering complexity of datasets simulated from mixtures. These datasets can then be used for systematic performance investigation of clustering and finite mixture modeling algorithms. Among other capabilities of MixSim, there are computing the exact overlap for Gaussian mixtures, simulating Gaussian and non-Gaussian data, simulating outliers and noise variables, calculating various measures of agreement between two partitionings, and constructing parallel distribution plots for the graphical display of finite mixture models. All features of the package are illustrated in great detail. The utility of the package is highlighted through a small comparison study of several popular clustering algorithms.

97 MATHEMATICS AND COMPUTING↗

Joint pattern recognition/data compression concept for ERTS multispectral imaging

This paper describes a new technique which jointly applies clustering and source encoding concepts to obtain data compression. The cluster compression technique basically uses clustering to extract features from the measurement data set which are used to describe characteristics of the entire data set. In addition, the features may be used to approximate each individual measurement vector by forming a sequence of scalar numbers which define each measurement vector in terms of the cluster features. This sequence, called the feature map, is then efficiently represented by using source encoding concepts. A description of a practical cluster compression algorithm is given and experimental results are presented to show trade-offs and characteristics of various implementations. Examples are provided which demonstrate the application of cluster compression to multispectral image data of the Earth Resources Technology Satellite.

Hilbert, E. E.↗

A study to explore the use of orbital remote sensing to determine native arid plant distribution

The author has identified the following significant results. Ground truth spectral signature data for various types of scenes, including ground with and without annuals, and various shrubs, were collected. When these signature data are plotted with infrared (MSS band 6 or 7) reflectivity on one axis and red (MSS band 5) reflectivity on the other axis, clusters of data from the various types of scenes are distinct. This method of expressing spectral signature data appears to be more useful for distinguishing types of scenes than a simple infrared to red reflectivity ration. Large areas of varnished desert pavement are visible and mappable on ERTS-1 and high altitude aircraft imagery. A large scale vegetation pattern was found to be correlated with the presence of the desert pavement. The large scale correlation was used in mapping the vegetation of the area. It was found that a distinctive soil type was associated with the presence of the varnished desert pavement. The high salinity and exchangeable sodium percentage of this soil type provide a basis for the explanation of both the large scale and small scale vegetation pattern.

Mcginnies, W. G.↗

Unified formula of half-lives for {alpha} decay and cluster radioactivity

In view of the fact that {alpha} decay and cluster radioactivity are physically analogical processes, we propose a general formula of half-lives and decay energies for {alpha} decay and cluster radioactivity. This new formula is directly deduced from the WKB barrier penetration probability with some approximations. It is not only simple in form and easy to see the physical meanings but also shows excellent agreement with the experimental values. Moreover, the difference between two sets of parameters to separately describe {alpha} decay and cluster radioactivity is small. Therefore, we use only one set of adjustable parameters to simultaneously describe the {alpha} decay and cluster radioactivity data for even-even nuclei. The results are also satisfactory. This indicates that this formula successfully combines the phenomenological laws of {alpha} decay and cluster radioactivity. We expect it to be a significant step toward a unified phenomenological law of {alpha} decay and cluster radioactivity.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

BiG-SLiCE 2 v1.0.0

BiG-SLiCE was originally an open source Python-based command line bioinformatics software that offers a highly scalable clustering analysis on biosynthetic gene clusters (BGC) data. It allows a simultaneous analysis of millions of BGCs, exceeding the capability of other existing tools (around one hundred thousands). As a tradeoff, the clustering accuracy is relatively lower and sometimes fall short in corner cases and specific BGC classes such as the RiPPs (Ribosomally-translated, Post-translationally modified Peptides). In BiG-SLiCE V2 (developed in LBNL), the clustering algorithm has been significantly improved to deliver a much accurate result even for RiPPs and other previous corner case classes. Moreover, the speed of the overall pipeline has been improved by 50-100%. Finally, additional features were implemented to support downstream analyses of BiG-SLiCE results, such as customized tabular (TSV/CSV) and columnar (Parquet) outputs.

Kautsar, Satria↗

Cooperative Data Sharing: Simple Support for Clusters of SMP Nodes

Libraries like PVM and MPI send typed messages to allow for heterogeneous cluster computing. Lower-level libraries, such as GAM, provide more efficient access to communication by removing the need to copy messages between the interface and user space in some cases. still lower-level interfaces, such as UNET, get right down to the hardware level to provide maximum performance. However, these are all still interfaces for passing messages from one process to another, and have limited utility in a shared-memory environment, due primarily to the fact that message passing is just another term for copying. This drawback is made more pertinent by today's hybrid architectures (e.g. clusters of SMPs), where it is difficult to know beforehand whether two communicating processes will share memory. As a result, even portable language tools (like HPF compilers) must either map all interprocess communication, into message passing with the accompanying performance degradation in shared memory environments, or they must check each communication at run-time and implement the shared-memory case separately for efficiency. Cooperative Data Sharing (CDS) is a single user-level API which abstracts all communication between processes into the sharing and access coordination of memory regions, in a model which might be described as "distributed shared messages" or "large-grain distributed shared memory". As a result, the user programs to a simple latency-tolerant abstract communication specification which can be mapped efficiently to either a shared-memory or message-passing based run-time system, depending upon the available architecture. Unlike some distributed shared memory interfaces, the user still has complete control over the assignment of data to processors, the forwarding of data to its next likely destination, and the queuing of data until it is needed, so even the relatively high latency present in clusters can be accomodated. CDS does not require special use of an MMU, which can add overhead to some DSM systems, and does not require an SPMD programming model. unlike some message-passing interfaces, CDS allows the user to implement efficient demand-driven applications where processes must "fight" over data, and does not perform copying if processes share memory and do not attempt concurrent writes. CDS also supports heterogeneous computing, dynamic process creation, handlers, and a very simple thread-arbitration mechanism. Additional support for array subsections is currently being considered. The CDS1 API, which forms the kernel of CDS, is built primarily upon only 2 communication primitives, one process initiation primitive, and some data translation (and marshalling) routines, memory allocation routines, and priority control routines. The entire current collection of 28 routines provides enough functionality to implement most (or all) of MPI 1 and 2, which has a much larger interface consisting of hundreds of routines. still, the API is small enough to consider integrating into standard os interfaces for handling inter-process communication in a network-independent way. This approach would also help to solve many of the problems plaguing other higher-level standards such as MPI and PVM which must, in some cases, "play OS" to adequately address progress and process control issues. The CDS2 API, a higher level of interface roughly equivalent in functionality to MPI and to be built entirely upon CDS1, is still being designed. It is intended to add support for the equivalent of communicators, reduction and other collective operations, process topologies, additional support for process creation, and some automatic memory management. CDS2 will not exactly match MPI, because the copy-free semantics of communication from CDS1 will be supported. CDS2 application programs will be free to carefully also use CDS1. CDS1 has been implemented on networks of workstations running unmodified Unix-based operating systems, using UDP/IP and vendor-supplied high- performance locks. Although its inter-node performance is currently unimpressive due to rudimentary implementation technique, it even now outperforms highly-optimized MPI implementation on intra-node communication due to its support for non-copy communication. The similarity of the CDS1 architecture to that of other projects such as UNET and TRAP suggests that the inter-node performance can be increased significantly to surpass MPI or PVM, and it may be possible to migrate some of its functionality to communication controllers.

DiNucci, David C.↗

Detection of Anomalies in Gamma Background Radiation Data with K-Means and Self-Organizing Map Clustering Algorithms (Consortium on Nuclear Security Technologies (CONNECT) Q1 Report)

Environmental screening of gamma radiation consists of detecting weak nuisance and anomaly signal in the presence of strong and highly varying background. In a typical scenario, a mobile detector-spectrometer continuously measures gamma radiation spectra in short, e.g., one-second, signal acquisition intervals. The measurement data is a 2D matrix, where one dimension is gamma ray energy, and the other dimension is the number of measurements or total time. In principle, gamma radiation sources can be detected and identified from the measured data by their unique spectral lines. Detecting sources from data measured in a search scenario is difficult due to the highly varying background because of naturally occurring radioactive material (NORM), and low signal-to-noise ratio (S/N) of spectral signal measured during one-second acquisition intervals. The objective of this work is to explore unsupervised machine learning (ML) algorithms for detection and identification of weak nuisances and anomalies events in the presence of highly fluctuating background. The challenge is that spectral lines of isotopes are difficult to observe in one-second measurements. Averaging over the entire measurement campaign data set reveals spectral lines of most common background isotopes. Spectral lines of orphan sources, which might appear only in a few measurements during the campaign, will be washed out if averaging is performed over the entire measurement data set. The approach we have explored consists of extracting one-second measurements containing weak spectral features through data clustering. Averaging one-second spectra in a cluster should reveal the presence of anomaly sources. We created two ML models using K-means clustering and Neural Network Self-organizing Map (SOM). Performance of these ML models was benchmarked using search data. One data set contained 137 Cs source, and another dataset contained 131 I source.

61 RADIATION PROTECTION AND DOSIMETRY↗

Methodology for Clustering High-Resolution Spatiotemporal Solar Resource Data

In this report, we introduce a methodology to achieve multiple levels of spatial resolution reduction of solar resource data, with minimal impact on data variability, for use in energy systems modeling. The selection of an appropriate clustering algorithm, parameter selection including cluster size, methods of temporal data segmentation, and methods of cluster evaluation are explored in the context of a repeatable process. In describing this process, we illustrate the steps in creating a reduced resolution, but still viable, dataset to support energy systems modeling, e.g. capacity expansion or production cost modeling. This process is demonstrated through the use of a solar resource dataset; however, the methods are applicable to other resource data represented through spatiotemporal grids, including wind data. In addition to energy modeling, the techniques demonstrated in this paper can be used in a novel top-down approach to assess renewable resources within many other contexts that leverage variability in resource data but require reduction in spatial resolution to accommodate modeling or computing constraints.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Scl-dE1 GC1: AN EXTENDED GLOBULAR CLUSTER IN A LOW-LUMINOSITY DWARF ELLIPTICAL GALAXY

We report the discovery from Hubble Space Telescope/ACS images of an extended globular cluster, denoted by Scl-dE1 GC1, in the Sculptor Group dwarf Elliptical galaxy Scl-dE1 (Sc22). The distance of the dE is determined as 4.3 {+-} 0.25 Mpc from the I magnitude of the tip of the red giant branch in the color-magnitude diagram. At this distance the half-light radius of Scl-dE1 GC1 is {approx}22 pc, placing it among the largest clusters known, particularly for globular clusters associated with dwarf galaxies. The absolute magnitude of Scl-dE1 GC1 is M{sub V} = -6.7 and, to within the photometric uncertainties of the data, the cluster stellar population appears indistinguishable from that of the dE. We suggest that there may be two modes of globular cluster formation in dwarf galaxies, a 'normal' mode with half-light radii of typically 3 pc, and an 'extended' mode with half-light radii of {approx}10 pc or more.

79 ASTRONOMY AND ASTROPHYSICS↗

Trends in solar radiation in NCEP/NCAR database and measurements in northeastern Brazil

The database from the National Center for Environmental Prediction/National Center for Atmospheric Research (NCEP/NCAR) re-analysis project available for the period from 1948 to 2009 was used for obtaining long-term solar radiation for northeastern Brazil. Measurements of global solar radiation (R{sub s}) from data collection platform (DCP) for four climatic zones of northeastern Brazil were compared to the re-analysis data. Applying cluster analysis to R{sub s} from database, homogeneous sub-regions in northeastern Brazil were determined. Long times series of R{sub s} and sunshine duration measurements data for two sites, Petrolina (09 09'S, 40 22'W) and Juazeiro (09 24'S, 40 26'W), exceeding 30 years, were analyzed. In order to exclude the decadal variations which are linked to the Pacific Decadal Oscillation, high-frequency cycles in the solar radiation and sunshine duration time series were eliminated by using a 14-year moving average, and the Mann-Kendall test was employed to assess the long-term variability of re-analysis and measured solar radiation. This study provides an overview of the decrease in solar radiation in a large area, which can be attributed to the global dimming effect. The global solar radiation obtained from the NCEP/NCAR re-analysis data overestimate that obtained from DCP measurements by 1.6% to 18.6%. Results show that there is a notable symmetry between R{sub s} from the re-analysis data and sunshine duration measurements. (author)

14 SOLAR ENERGY↗

Signature extension methods in crop area estimation

The Procedure B multispectral processing system is both multisegment and multistratum. It uses data from several LACIE-sized segments together and makes a proportion estimate for the entire group of segments as well as for the individual segments. In the clustering of data features, Procedure B produces multiple classes or strata rather than just two strata (as in Procedure 1), and performs stratified sampling on each of these mutliple strata in order to make a proportion estimate. Tests results for the components and for the overall performance of Procedure B are presented, and conclusions that can be drawn from these tests are discussed. The rationale for signature extension for crop area estimation is summarized.

Kauth, R. J.↗

Narrow-angle tail radio sources and evidence for radial orbits in Abell clusters

Published observational data on the tail orientations (TOs) of 60 narrow-angle-tail (NAT) radio sources in Abell clusters of galaxies are analyzed statistically using a maximum-likelihood approach. The results are presented in a table, and it is found that the observed TO distributions in the whole sample and in subsamples of morphologically regular NATs and NATs with pericentric distances d greater than 500 kpc are consistent with isotropic orbits, whereas the TOs for NATs with d less than 500 kpc are consistent with highly radial orbits. If radial orbits were observed near the centers of other types of cluster galaxies as well, it could be inferred that violent relaxation during cluster formation was incomplete, and that clusters form by spherical collapse and secondary infall, as proposed by Gunn (1977).

O'Dea, Christopher P.↗

Data-Driven Whole-Genome Clustering to Detect Geospatial, Temporal, and Functional Trends in SARS-CoV-2 Evolution

Current methods for defining SARS-CoV-2 lineages ignore the vast majority of the SARS-CoV-2 genome. We develop and apply an exhaustive vector comparison method that directly compares all known SARS-CoV-2 genome sequences to produce novel lineage classifications. We utilize data-driven models that (i) accurately capture the complex interactions across the set of all known SARS-CoV-2 genomes, (ii) scale to leadership-class computing systems, and (iii) enable tracking how such strains evolve geospatially over time. We show that during the height of the original Omicron surge, countries across Europe, Asia, and the Americas had a spatially asynchronous distribution of Omicron sub-strains. Moreover, neighboring countries were often dominated by either different clusters of the same variant or different variants altogether throughout the pandemic. Analyses of this kind may suggest a different pattern of epidemiological risk than was understood from conventional data, as well as produce actionable insights and transform our ability to prepare for and respond to current and future biological threats.

Jacobson, Daniel↗

Dark Energy Survey Year 1 Results: Multi-Probe Methodology and Simulated Likelihood Analyses

We present the methodology for and detail the implementation of the Dark Energy Survey (DES) 3x2pt DES Year 1 (Y1) analysis, which combines configuration-space two-point statistics from three different cosmological probes: cosmic shear, galaxy-galaxy lensing, and galaxy clustering, using data from the first year of DES observations. We have developed two independent modeling pipelines and describe the code validation process. We derive expressions for analytical real-space multi-probe covariances, and describe their validation with numerical simulations. We stress-test the inference pipelines in simulated likelihood analyses that vary 6-7 cosmology parameters plus 20 nuisance parameters and precisely resemble the analysis to be presented in the DES 3x2pt analysis paper, using a variety of simulated input data vectors with varying assumptions. We find that any disagreement between pipelines leads to changes in assigned likelihood $\Delta \chi^2 \le 0.045$ with respect to the statistical error of the DES Y1 data vector. We also find that angular binning and survey mask do not impact our analytic covariance at a significant level. We determine lower bounds on scales used for analysis of galaxy clustering (8 Mpc$~h^{-1}$) and galaxy-galaxy lensing (12 Mpc$~h^{-1}$) such that the impact of modeling uncertainties in the non-linear regime is well below statistical errors, and show that our analysis choices are robust against a variety of systematics. These tests demonstrate that we have a robust analysis pipeline that yields unbiased cosmological parameter inferences for the flagship 3x2pt DES Y1 analysis. We emphasize that the level of independent code development and subsequent code comparison as demonstrated in this paper is necessary to produce credible constraints from increasingly complex multi-probe analyses of current data.

79 ASTRONOMY AND ASTROPHYSICS↗

pnnl/apt

The current scripts utilize a well-known cluster analysis algorithm, called ordering points to identify the clustering structures (OPTICS), and an automatic cluster extraction algorithm for clustering analysis of 3D spatial data, specifically for the Atom Probe Tomography data. A hierarchical clustering scheme, together with density-based clustering, were implemented.

Wang, Jing↗

Alternatives to the 15% Rule

The third solicitation of the California Solar Initiative (CSI) Research, Development, Demonstration and Deployment (RD&D) Program established by the California Public Utility Commission (CPUC) is supporting the Electric Power Research Institute (EPRI), National Renewable Energy Laboratory (NREL), and Sandia National Laboratories (SNL) with collaboration from Pacific Gas and Electric (PG&E), Southern California Edison (SCE), and San Diego Gas and Electric (SDG&E), in research to improve the Utility Application Review and Approval process for interconnecting distributed energy resources to the distribution system. Currently this process is the most time - consuming of any step on the path to generating power on the distribution system. This CSI RD&D solicitation three project has completed the tasks of collecting data from the three utilities, clustering feeder characteristic data to attain representative feeders, detailed modeling of 16 representative feeders, analysis of PV impacts to those feeders, refinement of current screening processes, and validation of those suggested refinements. In this report each task is summarized to produce a final summary of all components of the overall project.

Broderick, Robert Joseph↗

OCEANIDS: Autonomous Data Acquisition, Management and Distribution System

OCEANIDS is a clearinghouse for mission essential and near-real-time satellite data streams. This viewgraph presentation describes this mission, and includes the following topics: 1) OCEANIDS Motivation; 2) High-Level Architecture; 3) OCEANIDS Features; 4) OCEANIDS GUI: Nodes; 5) OCEANIDS GUI: Cluster; 6) Data Streams; 7) Statistics; and 8) GHRSST-PP.

automated monitoring↗

Selection function of clusters in Dark Energy Survey year 3 data from cross-matching with South Pole Telescope detections

Context. Galaxy clusters selected based on overdensities of galaxies in photometric surveys provide the largest cluster samples. However, modeling the selection function of such samples is complicated by noncluster members projected along the line of sight (projection effects) and the potential detection of unvirialized objects (contamination). Aims. We empirically constrained the magnitude of these effects by cross-matching galaxy clusters selected in the Dark Energy Survey data with the redMaPPer algorithm with significant detections in three South Pole Telescope surveys (SZ, pol-ECS, pol-500d). Methods. For matched clusters, we augmented the redMaPPer catalog with the SPT detection significance. For unmatched objects we used the SPT detection threshold as an upper limit on the SZe signature. Using a Bayesian population model applied to the collected multiwavelength data, we explored various physically motivated models to describe the relationship between observed richness and halo mass. Results. Our analysis reveals a clear preference for models with an additional skewed scatter component associated with projection effects over a purely log-normal scatter model. We rule out significant contamination by unvirialized objects at the high-richness end of the sample. While dedicated simulations offer a well-fitting calibration of projection effects, our findings suggest the presence of redshift-dependent trends that these simulations may not have captured. Our findings highlight that modeling the selection function of optically detected clusters remains a complicated challenge that requires a combination of simulation and data-driven approaches.

79 ASTRONOMY AND ASTROPHYSICS↗