Search NASA⌕ Search

SEARCH · Search NASA

Results for “data clustering”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Combined Optical and Radio-Frequency Perspectives on a Hybrid Cloud-To-Ground Lightning Flash Observed by the FORTE Satellite

In this work, we use the coincident optical and radio-frequency (RF) measurements taken by the Fast On-orbit Recording of Transient Events (FORTE) satellite to shed light on common optical signatures recorded by NASA and NOAA lightning imagers during Cloud-to-Ground (CG) lightning. We build flash cluster data for FORTE using the same techniques as the NASA/NOAA instruments to document the optical/RF evolution of an oceanic hybrid -CG flash. The flash began with strong Very High Frequency (VHF)-band emission from a Narrow Bipolar Event (NBE) that initiated a period of normal bilevel intracloud (IC) activity in two vertical layers (8 and 12 km) that lasted for 490 ms. VHF waveforms show step leader activity ahead of the return stroke. All impulsive VHF sources after the stroke come from the lower (8 km) layer. K-changes are noted following the return stroke, but no subsequent strokes are detected. The optical flash began 136 ms after the NBE RF pulse. Twenty-two of the 33 optical groups were dim and occurred during the in-cloud phase of the flash. This activity included both isolated pulses and sustained periods of illumination over tens of milliseconds. Initial cloud pulses accounted for 23% of the total optical radiance from the flash. Illumination during the return stroke contributed 58% of the total radiance, and the K-changes and cloud pulses after the stroke supplied the remaining 19%. These results highlight the benefit of having RF alongside optical lightning measurements for clarifying signatures in the optical data and providing information on their physical origins.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Statistical moments of VIIRS night-time lights

Annual VIIRS global night-time lights traditionally report mean radiance levels after filtering to remove moonlit and cloudy data followed by outlier removal. The mean is the first statistical moment, but there are three other statistical moments (variance, skew and kurtosis) that are currently not produced globally. In this study, we generated VIIRS day/night band multiyear and annual statistical moments for a widely dispersed set of test areas. The moments were calculated from 15 arc second nightly temporal profiles spanning 2012-2020, filtered to exclude cloudy and sunlit data, with radiance adjustments to reduce view angle and lunar illuminance effects. The moment data were examined in two ways: (1) Geospatial grids-which reveal zonation and temporal changes present in urban areas and (2) Scattergrams of moment pairs. The variance versus mean scattergram exhibits several distinct data clusters used to define five zones: core lighting, dark-erratics, mid-erratics, bright-erratics and bright and seteady. The results indicate that the information content of VIIRS night-time lights can be augmented if all four statistical moments are calculated.

47 OTHER INSTRUMENTATION↗
DOE OSTI2016

The clustering of galaxies in the completed SDSS-III Baryon Oscillation Spectroscopic Survey: combining correlated Gaussian posterior distributions

The cosmological information contained in anisotropic galaxy clustering measurements can often be compressed into a small number of parameters whose posterior distribution is well described by a Gaussian. Here, we present a general methodology to combine these estimates into a single set of consensus constraints that encode the total information of the individual measurements, taking into account the full covariance between the different methods. We also illustrate this technique by applying it to combine the results obtained from different clustering analyses, including measurements of the signature of baryon acoustic oscillations and redshift-space distortions, based on a set of mock catalogues of the final SDSS-III Baryon Oscillation Spectroscopic Survey (BOSS). Our results show that the region of the parameter space allowed by the consensus constraints is smaller than that of the individual methods, highlighting the importance of performing multiple analyses on galaxy surveys even when the measurements are highly correlated. Our paper is part of a set that analyses the final galaxy clustering data set from BOSS. The methodology presented here is used in Alam et al. to produce the final cosmological constraints from BOSS.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Linear point and sound horizon as purely geometric standard rulers

The baryon acoustic oscillations feature (BAO) imprinted in the clustering correlation function is known to furnish us cosmic distance determinations that are independent of the cosmological-background model and the primordial perturbation parameters. These measurements can be accomplished rigorously by means of the purely geometric BAO methods. To date two different purely geometric BAO approaches have been proposed. The first exploits the linear-point standard ruler. The second, called correlation-function model-fitting, exploits the sound-horizon standard ruler. A key difference between them is that, when estimated from clustering data, the linear point makes use of a cosmological-model-independent procedure to extract the ratio of the ruler to the cosmic distance, while the correlation-function model-fitting relies on a phenomenological cosmological model for the correlation function. Nevertheless the two rulers need to be precisely defined independently of any specific observable (e.g., the BAO). We define the linear point and sound horizon and we fully characterize and compare the two rulers’ cosmological-parameter dependence. We find that they are both geometrical (i.e., independent of the primordial cosmological parameters) within the required accuracy, and that they have the same parameter dependence for a wide range of parameter values. We estimate the rulers’ best-fit values and errors, given the cosmological constraints obtained by the Planck Satellite team from their measurements of the cosmic microwave background temperature and polarization anisotropies. We do this for three different cosmological models encompassed by the purely geometric BAO methods. In each case we find that the relative errors of the two rulers coincide and they are insensitive to the assumed cosmological model. Interestingly both the linear point and the sound horizon shift by 0.5σ when we do not fix the spatial geometry to be flat in Λ CDM. Finally, this points toward a sensitivity of the rulers to different cosmological models when they are estimated from the cosmic microwave background.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Model-independent versus model-dependent interpretation of the SDSS-III BOSS power spectrum: Bridging the divide

The traditional clustering analyses of galaxy redshift surveys compress the clustering data into a set of late-time physical variables in a model-independent way. This approach has recently been extended by an additional shape variable encoding early-time physics information. Here we apply this new technique, ShapeFit, to SDSS-III BOSS data and show that it matches the constraining power of alternative, model-dependent approaches, which directly constrain the model’s parameters adopting a cosmological model ab initio. ShapeFit is ~30 times faster, model independent, naturally splits early- and late-time variables, and enables a better control of observational systematics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2018

The Source Physics Experiment Large N Array

To improve the understanding of S wave generation from an explosion, a temporary deployment of 996 geophones, including both one component (Z) and three component sensors (3C), was installed from April 15 to May 23, 2016 at the Nevada National Security Site (NNSS). Sensor spacing varied from 25 to 100 m and consisted of 500 Z and 496 3C 5-Hz geophones. Data were continuously recorded during the deployment at low-gain (0 db) from April 15 to April 28 and high-gain (36 db) from April 29 to May 23. A buried (76.5 m depth) 5035 kg TNT equivalent chemical explosion (SPE-5) was recorded on April 26. It was situated in a weathered granite body surrounded by volcanic tuffs, Paleozoic carbonates, and alluvium. The array was deployed approximately 400 to 3000 m from the explosion. A set of large weight-drop shots (13,000 kg source) at 53 locations both inside and outside the geophone array were also recorded, as were local, regional, and teleseismic earthquakes. Data recovery was good, with 95% of data recovered from the chemical explosion and up to 99% in the following weeks, including both the weightdrop shots campaign and the continuous data. Important initial results from the deployment include estimates of the spatial correlation length of velocity heterogeneities and a higher resolution velocity model. Observations of the data and synthetics, indicate that some far-field (elastic) S wave energy is generated by scattering and conversion outside the near-field (inelastic) region. Interferometric processing was conducted on a Hadoop ‘big-data’ cluster.

58 GEOSCIENCES↗
DOE OSTI2015

Enabling Exascale Hardware and Software Design through Scalable System Virtualization (Final Report)

The purpose of this project has been to extend the state of the art of systems software for high-end computing (HEC) platforms, and to use systems software to better enable the evaluation of potential future HEC platforms, for example exascale platforms. Such platforms, and their systems software, have the goal of providing scientific computation at new scales, thus enabling new research in the physical sciences and engineering. Over time, the innovations in systems software for such platforms also become applicable to more widely used computing clusters, data centers, and clouds. This was a five-institution project, centered on the Palacios virtual machine monitor (VMM) systems software, a project begun at Northwestern, and originally developed in a previous collaboration between Northwestern University and the University of New Mexico. In this project, Northwestern (including via our subcontract to the University of Pittsburgh) contributed to the continued development of Palacios, along with other team members. We took the leadership role in (1) continued extension of support for emerging Intel and AMD hardware, (2) integration and performance enhancement of overlay networking, (3) connectivity with architectural simulation, (4) binary translation, and (5) support for modern Non-Uniform Memory Access (NUMA) hosts and guests. We also took a supporting role in support for specialized hardware for I/O virtualization, profiling, configurability, and integration with configuration tools. The efforts we led (1-5) were largely successful and executed as expected, with code and papers resulting from them. The project demonstrated the feasibility of a virtualization layer for HEC computing, similar to such layers for cloud or datacenter computing. For effort (3), although a prototype connecting Palacios with the GEM5 architectural simulator was demonstrated, our conclusion was that such a platform was less useful for design space exploration than anticipated due to inherent complexity of the connection between the instruction set architecture level and the microarchitectural level. For effort (4), we found that a code injection approach proved to be more fruitful. The results of our efforts are publicly available in the open source Palacios codebase and published papers, all of which are available from the project web site, v3vee.org. Palacios is currently one of the two codebases (the other being Sandia’s Kitten lightweight kernel) that underlies the node operating system for the DOE Hobbes Project, one of two projects tasked with building a systems software prototype for the national exascale computing effort.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2015

Data Intensive Computing on Amazon Web Services

The Geophysical Monitoring Program (GMP) has spent the past few years building up the capability to perform data intensive computing using what have been referred to as “big data” tools. These big data tools would be used against massive archives of seismic signals (>300 TB) to conduct research not previously possible. Examples of such tools include Hadoop (HDFS, MapReduce), HBase, Hive, Storm, Spark, Solr, and many more by the day. These tools are useful for performing data analytics on datasets that exceed the resources of traditional analytic approaches. To this end, a research big data cluster (“Cluster A”) was set up as a collaboration between GMP and Livermore Computing (LC).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Optimal and robust reconstruction of BAO, redshift-space distortions and the Alcock-Paczynski effect

The goal of this project is to optimally reconstruct cosmological information that has been lost from large-scale clustering of galaxies due to cosmic structure growth. The outcome of our research will be used to improve dark energy and other cosmological constraints from the ongoing extended Baryon Oscillation Spectroscopic Survey (eBOSS) and the upcoming Dark Energy Spectroscopic Instrument (DESI). The large-scale galaxy clustering data contain two important features: the Baryon Acoustic Oscillations (BAO) and the overall shape from small to large scales.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2017

Coherent Structures at Ion Scales in Fast Solar Wind: Cluster Observations

We investigate the nature of magnetic turbulent fluctuations, around ion characteristic scales, in a fast solar wind stream, by using Cluster data. Contrarily to slow solar wind, where both Alfvénic (δb{sub ⊥}≫δb{sub ∥}) and compressive (δb{sub ∥}≫δb{sub ⊥}) coherent structures are observed, the turbulent cascade of fast solar wind is dominated by Alfvénic structures, namely, Alfvén vortices, with a small and/or finite compressive part, with the presence also of several current sheets aligned with the local magnetic field. Several examples of vortex chains are also recognized. Although an increase of magnetic compressibility around ion scales is observed also for fast solar wind, no strongly compressive structures are found, meaning that the nature of the slow and fast winds is intrinsically different. Multispacecraft analysis applied to this interval of fast wind indicates that the coherent structures are almost convected by the flow and aligned with the local magnetic field, i.e., their normal is perpendicular to B, which is consistent with a two-dimensional turbulence picture. Understanding intermittency and the related generation of coherent structures could provide a key insight into the nonlinear energy transfer and dissipation processes in magnetized and collisionless plasmas.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS1990

Cobweb/3: A portable implementation

An algorithm is examined for data clustering and incremental concept formation. An overview is given of the Cobweb/3 system and the algorithm on which it is based, as well as the practical details of obtaining and running the system code. The implementation features a flexible user interface which includes a graphical display of the concept hierarchies that the system constructs.

Mckusick, Kathleen↗
NASA NTRS1993

An expanded set of brown dwarf and very low mass star models

We present in this paper updated and improved theoretical models of brown dwarfs and late M dwarfs. The evolution and characteristics of objects between 0.01 and 0.2 solar mass are exhaustively investigated and special emphasis is placed on their properties at early ages. The dependence on the helium fraction, deuterium fraction, and metallicity of the masses, effective temperature and luminosities at the edge of the hydrogen main sequence are calculated. We derive luminosity functions for representative mass functions and compare our predictions to recent cluster data. We show that there are distinctive features in the theoretical luminosity functions that can serve as diagnostics of brown dwarf physics. A zero-metallicity model is presented as a bound to or approximation of a putative extreme halo population.

Burrows, A.↗
DOE OSTI2016

CLASH: JOINT ANALYSIS OF STRONG-LENSING, WEAK-LENSING SHEAR, AND MAGNIFICATION DATA FOR 20 GALAXY CLUSTERS

We present a comprehensive analysis of strong-lensing, weak-lensing shear and magnification data for a sample of 16 X-ray-regular and 4 high-magnification galaxy clusters at 0.19≲z≲0.69 selected from Cluster Lensing And Supernova survey with Hubble (CLASH). Our analysis combines constraints from 16-band Hubble Space Telescope observations and wide-field multi-color imaging taken primarily with Suprime-Cam on the Subaru Telescope, spanning a wide range of cluster radii (10″–16′). We reconstruct surface mass density profiles of individual clusters from a joint analysis of the full lensing constraints, and determine masses and concentrations for all of the clusters. We find the internal consistency of the ensemble mass calibration to be ≤5% ± 6% in the one-halo regime (200–2000 kpc h{sup −1}) compared to the CLASH weak-lensing-only measurements of Umetsu et al. For the X-ray-selected subsample of 16 clusters, we examine the concentration–mass (c–M) relation and its intrinsic scatter using a Bayesian regression approach. Our model yields a mean concentration of c|{sub z=0.34}=3.95±0.35 at M{sub 200c} ≃ 14 × 10{sup 14} M{sub ⊙} and an intrinsic scatter of σ(lnc{sub 200c})=0.13±0.06, which is in excellent agreement with Λ cold dark matter predictions when the CLASH selection function based on X-ray morphological regularity and the projection effects are taken into account. We also derive an ensemble-averaged surface mass density profile for the X-ray-selected subsample by stacking their individual profiles. The stacked lensing signal is detected at 33σ significance over the entire radial range ≤4000 kpc h{sup −1}, accounting for the effects of intrinsic profile variations and uncorrelated large-scale structure along the line of sight. The stacked mass profile is well described by a family of density profiles predicted for cuspy dark-matter-dominated halos in gravitational equilibrium, namely, the Navarro–Frenk–White (NFW), Einasto, and DARKexp models, whereas the single power-law, cored isothermal and Burkert density profiles are disfavored by the data. We show that cuspy halo models that include the large-scale two-halo term provide improved agreement with the data. For the NFW halo model, we measure a mean concentration of c{sub 200c}=3.79{sub −0.28}{sup +0.30} at M{sub 200c}=14.1{sub −1.0}{sup +1.0}×10{sup 14} M{sub ⊙}, demonstrating consistency between the complementary analysis methods.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Supporting data for climatic clustering and longitudinal analysis with impacts on food, bioenergy, and pandemics

This data supports the conclusions found in climatic clustering and longitudinal analysis with impacts on food, bioenergy, and pandemics. Included here are (i) the binarized geolocation vectors used for exhaustive vector comparisons, (ii) the resulting climatic networks, (iii) the results of applying Markov clustering to the climatic networks, and (iv) the results of applying Correlation-of-Correlations (cor-cor) to the climatic networks. The set of binarized geolocation vectors that are used as inputs for the Combinatorial Metrics library (CoMet) are of the form comet-UUUUUxVVVVV-XXXX-YYYY.shuffled.tped where UUUUU is the number of vectors, VVVVV is the length of each vector, XXXX is the starting year, and YYYY is the ending year. Each line corresponds to a geolocation vector of binary elements A (i.e., 0) and T (i.e., 1). The set of climatic networks that are used for downstream network analysis are of the form network-U-way-XXXX-YYYY.parsed.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, and YYYY is the ending year. Each line corresponds to an edge linking two geolocations (defined by latitude and longitude) with its corresponding edge weight (i.e., DUO score). The set of cluster results are of the form clusters-U-way-XXXX-YYYY-thresh-VVVV-inflation-WWW.clustered.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, YYYY is the ending year, VVVV is the similarity threshold, and WWW is the Markov clustering inflation rate. Each line corresponds to a single cluster and is composed of a number of corresponding geolocations (defined by latitude and longitude). The set of cor-cor results are of the form corcor-U-way-XXXX-YYYY.cumulative.txt where U is the order of the comparison (2-way or 3-way), XXXX is the starting year, and YYYY is the ending year. Each line corresponds to a single geolocation with it's corresponding cor-cor value.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2009

Application of a Self-Similar Pressure Profile to Sunyaev-Zeldovich Effect Data from Galaxy Clusters

We investigate the utility of a new, self-similar pressure profile for fitting Sunyaev-Zel'dovich (SZ) effect observations of galaxy clusters. Current SZ imaging instruments-such as the Sunyaev-Zel'dovich Array (SZA)- are capable of probing clusters over a large range in a physical scale. A model is therefore required that can accurately describe a cluster's pressure profile over a broad range of radii from the core of the cluster out to a significant fraction of the virial radius. In the analysis presented here, we fit a radial pressure profile derived from simulations and detailed X-ray analysis of relaxed clusters to SZA observations of three clusters with exceptionally high-quality X-ray data: A1835, A1914, and CL J1226.9+3332. From the joint analysis of the SZ and X-ray data, we derive physical properties such as gas mass, total mass, gas fraction and the intrinsic, integrated Compton y-parameter. We find that parameters derived from the joint fit to the SZ and X-ray data agree well with a detailed, independent X-ray-only analysis of the same clusters. In particular, we find that, when combined with X-ray imaging data, this new pressure profile yields an independent electron radial temperature profile that is in good agreement with spectroscopic X-ray measurements.

Mroczkowski, Tony↗
DOE OSTI2014

Parallel Optical Flow Processing of 4D Cardiac CT Data on Multicore Clusters, In: 2014 IEEE 17th International Conference on Computational Science and Engineering

Optical flow is the distribution of apparent velocities of movement of brightness patterns in a sequence of images. For large 3D image sequences, optical flow applications are time consuming and memory-bound. To cope with these problems, in this paper, we present parallel optical flow processing of 4D cardiac CT data on multicore cluster systems to significantly shorten the time for computing velocity fields of the heart in order to aid cardiologists in diagnosing heart disease such as myocardial infarction and cardiac dysrhythmia in time. First, we modify and extend two traditional 2D optical flow methods Horn-Schunck and Lucas-Kanade to three-dimensional cases to process the 4D cardiac CT data. Second, we extend Mat lab MPI to support parallel computing with Mat lab and Octave on these cluster systems. Then we develop the parallel Mat lab/Octave optical flow applications for the 4D cardiac CT data in detail. Our experimental results show that these parallel optical flow applications have good scalability with close to linear speedup, and are able to shorten the image processing time significantly from more than 5 hours on 4 cores to 1.5 minutes on 1024 cores.

Engineering↗
DOE OSTI2021

Parallelizing the Unpacking and Clustering of Detector Data for Reconstruction of Charged Particle Tracks on Multi-core CPUs and Many-core GPUs

We present results from parallelizing the unpacking and clustering steps of the raw data from the silicon strip modules for reconstruction of charged particle tracks. Throughput is further improved by concurrently processing multiple events using nested OpenMP parallelism on CPU or CUDA streams on GPU. The new implementation along with earlier work in developing a parallelized and vectorized implementation of the combinatoric Kalman filter algorithm has enabled efficient global reconstruction of the entire event on modern computer architectures. We demonstrate the performance of the new implementation on Intel Xeon and NVIDIA GPU architectures.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗