Search NASA⌕ Search

SEARCH · Search NASA

Results for “spatial data mining”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2005

NASA Remote Sensing Data in Earth Sciences: Processing, Archiving, Distribution, Applications at the GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is one of the major Distributed Active Archive Centers (DAACs) archiving and distributing remote sensing data from the NASA's Earth Observing System. In addition to providing just data, the GES DISC/DAAC has developed various value-adding processing services. A particularly useful service is data processing a t the DISC (i.e., close to the input data) with the users' algorithms. This can take a number of different forms: as a configuration-managed algorithm within the main processing stream; as a stand-alone program next to the on-line data storage; as build-it-yourself code within the Near-Archive Data Mining (NADM) system; or as an on-the-fly analysis with simple algorithms embedded into the web-based tools (to avoid downloading unnecessary all the data). The existing data management infrastructure at the GES DISC supports a wide spectrum of options: from data subsetting data spatially and/or by parameter to sophisticated on-line analysis tools, producing economies of scale and rapid time-to-deploy. Shifting processing and data management burden from users to the GES DISC, allows scientists to concentrate on science, while the GES DISC handles the data management and data processing at a lower cost. Several examples of successful partnerships with scientists in the area of data processing and mining are presented.

Leptoukh, Gregory G.↗
NASA NTRS1975

Utilizing Skylab data in on-going resources management programs in the state of Ohio

The author has identified the following significant results. The use of Skylab imagery for total area woodland surveys was found to be more accurate and cheaper than conventional surveys using aerial photo-plot techniques. Machine-aided (primarily density slicing) analyses of Skylab 190A and 190B color and infrared color photography demonstrated the feasibility of using such data for differentiating major timber classes including pines, hardwoods, mixed, cut, and brushland providing such analyses are made at scales of 1:24,000 and larger. Manual and machine-assisted image analysis indicated that spectral and spatial capabilities of Skylab EREP photography are adequate to distinguish most parameters of current, coal surface mining concern associated with: (1) active mining, (2) orphan lands, (3) reclaimed lands, and (4) active reclamation. Excellent results were achieved when comparing Skylab and aerial photographic interpretations of detailed surface mining features. Skylab photographs when combined with other data bases (e.g., census, agricultural land productivity, and transportation networks), provide a comprehensive, meaningful, and integrated view of major elements involved in the urbanization/encroachment process.

Baldridge, P. E.↗
DOE OSTI2018

A Big Data Analytics Framework for HPC Log Data: Three Case Studies Using the Titan Supercomputer Log

Reliability, availability and serviceability (RAS) logs of high performance computing (HPC) resources, when closely investigated in spatial and temporal dimensions, can provide invaluable information regarding system status, performance, and resource utilization. These data are often generated from multiple logging systems and sensors that cover many components of the system. The analysis of these data for finding persistent temporal and spatial insights faces two main difficulties: the volume of RAS logs makes manual inspection difficult and the unstructured nature and unique properties of log data produced by each subsystem adds another dimension of difficulty in identifying implicit correlation among recorded events. To address these issues, we recently developed a multi-user Big Data analytics framework for HPC log data at Oak Ridge National Laboratory (ORNL). This paper introduces three in-progress data analytics projects that leverage this framework to assess system status, mine event patterns, and study correlations between user applications and system events. We describe the motivation of each project and detail their workflows using three years of log data collected from ORNL's Titan supercomputer.

Park, Byung↗
NASA NTRS1994

An electromagnetic noncontacting sensor for thickness measurement in a dispersive medium

This paper describes a general purpose imaging technology developed by the U.S. Bureau of Mines (USBM) that, when fully implemented, will solve the general problem of 'seeing into the earth.' A first-generation radar coal thickness sensor, the RCTS-1, has been developed and field-tested in both underground and highwall mines. The noncontacting electromagnetic technique uses spatial modulation created by moving a simple sensor antenna in a direction along each axis to be measured while the complex reflection coefficient is measured at multiple frequencies over a two-to-one bandwidth. The antenna motion imparts spatial modulation to the data that enables signal processing to solve the problems of media, target, and antenna dispersion. Knowledge of the dielectric constant of the media is not necessary because the electrical properties of the media are determined automatically along with the distance to the target and thickness of each layer of the target. The sensor was developed as a navigation guidance sensor to accurately detect the coal/noncoal interface required for the USBM computer-assisted mining machine program. Other mining applications include the location of rock fractures, water-filled voids, and abandoned gas wells. These hazards can be detected in advance of the mining operation. This initiating technology is being expanded into a full three-dimensional (3-D) imaging system that will have applications in both the underground and surface environment.

Chufo, Robert L.↗
NASA NTRS2007

Efficient Implementation of an Optimal Interpolator for Large Spatial Data Sets

Interpolating scattered data points is a problem of wide ranging interest. A number of approaches for interpolation have been proposed both from theoretical domains such as computational geometry and in applications' fields such as geostatistics. Our motivation arises from geological and mining applications. In many instances data can be costly to compute and are available only at nonuniformly scattered positions. Because of the high cost of collecting measurements, high accuracy is required in the interpolants. One of the most popular interpolation methods in this field is called ordinary kriging. It is popular because it is a best linear unbiased estimator. The price for its statistical optimality is that the estimator is computationally very expensive. This is because the value of each interpolant is given by the solution of a large dense linear system. In practice, kriging problems have been solved approximately by restricting the domain to a small local neighborhood of points that lie near the query point. Determining the proper size for this neighborhood is a solved by ad hoc methods, and it has been shown that this approach leads to undesirable discontinuities in the interpolant. Recently a more principled approach to approximating kriging has been proposed based on a technique called covariance tapering. This process achieves its efficiency by replacing the large dense kriging system with a much sparser linear system. This technique has been applied to a restriction of our problem, called simple kriging, which is not unbiased for general data sets. In this paper we generalize these results by showing how to apply covariance tapering to the more general problem of ordinary kriging. Through experimentation we demonstrate the space and time efficiency and accuracy of approximating ordinary kriging through the use of covariance tapering combined with iterative methods for solving large sparse systems. We demonstrate our approach on large data sizes arising both from synthetic sources and from real applications.

Memarsadeghi, Nargess↗
NASA NTRS2002

Spatial Metadata for Global Change Investigations Using Remote Sensing

Satellite and aircraft-borne remote sensors have gathered petabytes of data over the past 30+ years. These images are an important resource for establishing cause and effect relationships between human-induced land cover changes and alterations in climate and other biophysical patterns at local to global scales. However, the spatial, temporal, and spectral characteristics of these datasets vary, thus complicating long-term studies involving several types of imagery. As the geographical and temporal coverage, the spectral and spatial resolution, and the number of individual sensors increase, the sheer volume and complexity of available data sets will complicate management and use of the rapidly growing archive of earth imagery. Mining this vast data resource for images that provide the necessary information for climate change studies becomes more difficult as more sensors are launched and more imagery is obtained.

Emerson, Charles W.↗
DOE OSTI2023

IRA Energy Community Data Layers

Data, geospatial data resources, and the linked mapping tool and web services reflect data for two types of potentially qualifying energy communities: 1) Census tracts and directly adjoining tracts that have had coal mine closures since 1999 or coal-fired electric generating unit retirements since 2009. These census tracts qualify as energy communities. 2) Metropolitan statistical areas (MSAs) and non-metropolitan statistical areas (non-MSAs) that are energy communities for 2023 and 2024, along with their fossil fuel employment (FFE) status. Additional information on energy communities and related tax credits can be accessed on the Interagency Working Group on Coal & Power Plant Communities & Economic Revitalization Energy Communities website (https://energycommunities.gov/energy-community-tax-credit-bonus/). Use limitations: these spatial data and mapping tool may not be relied upon by taxpayers to substantiate a tax return position or for determining whether certain penalties apply and will not be used by the IRS for examination purposes. The mapping tool does not reflect the application of the law to a specific taxpayer’s situation, and the applicable Internal Revenue Code provisions ultimately control.

Census Tract↗
NASA NTRS2012

Physics Mining of Multi-Source Data Sets

Powerful new parallel data mining algorithms can produce diagnostic and prognostic numerical models and analyses from observational data. These techniques yield higher-resolution measures than ever before of environmental parameters by fusing synoptic imagery and time-series measurements. These techniques are general and relevant to observational data, including raster, vector, and scalar, and can be applied in all Earth- and environmental science domains. Because they can be highly automated and are parallel, they scale to large spatial domains and are well suited to change and gap detection. This makes it possible to analyze spatial and temporal gaps in information, and facilitates within-mission replanning to optimize the allocation of observational resources. The basis of the innovation is the extension of a recently developed set of algorithms packaged into MineTool to multi-variate time-series data. MineTool is unique in that it automates the various steps of the data mining process, thus making it amenable to autonomous analysis of large data sets. Unlike techniques such as Artificial Neural Nets, which yield a blackbox solution, MineTool's outcome is always an analytical model in parametric form that expresses the output in terms of the input variables. This has the advantage that the derived equation can then be used to gain insight into the physical relevance and relative importance of the parameters and coefficients in the model. This is referred to as physics-mining of data. The capabilities of MineTool are extended to include both supervised and unsupervised algorithms, handle multi-type data sets, and parallelize it.

Helly, John↗
DOE OSTI2021

Public data from three US states provide new insights into well integrity

Significance There are over 900,000 active oil and gas wells in the United States. Estimates of the percentage of these wells that leak during their lifetime have been limited by data availability. We mined publicly available records from state regulatory databases and synthesized the results of various well testing methods into a uniform dataset that describes the integrity of 105,031 wells. Our analysis of the dataset provides insight into regional well leakage frequencies in three US states (Colorado, New Mexico, and Pennsylvania), spatial and temporal leakage trends among vertical and directional wells, and the characteristics of leaked fluids. Our findings demonstrate the value of statewide well testing programs and highlight the challenges of interpreting disparate interjurisdictional well testing data.

04 OIL SHALES AND TAR SANDS↗
DOE OSTI2013

Exploring Hardware Support For Scaling Irregular Applications on Multi-node Multi-core Architectures

With the recent emergence of large-scale knowledge dis- covery, data mining and social network analysis, irregular applications have gained renewed interest. Classic cache-based high-performance architectures do not provide optimal performances with such kind of workloads, mainly due to the very low spatial and temporal locality of the irregular control and memory access patterns. In this paper, we present a multi-node, multi-core, fine-grained multi-threaded shared-memory system architecture specifically designed for the execution of large-scale irregular applications, and built on top of three pillars, that we believe are fundamental to support these workloads. First, we offer transparent hardware support for Partitioned Global Address Space (PGAS) to provide a large globally-shared address space with no software library overhead. Second, we employ multi-threaded multi-core processing nodes to achieve the necessary latency tolerance required by accessing global memory, which potentially resides in a remote node. Finally, we devise hardware support for inter-thread synchronization on the whole global address space. We first model the performances by using an analytical model that takes into account the main architecture and application characteristics. We describe the hardware design of the proposed cus- tom architectural building blocks that provide support for the above- mentioned three pillars. Finally, we present a limited-scale evaluation of the system on a multi-board FPGA prototype with typical irregular kernels and benchmarks. The experimental evaluation demonstrates the architecture performance scalability for different configurations of the whole system.

FPGA↗
NASA NTRS2001

An Evaluation of Fractal Surface Measurement Methods for Characterizing Landscape Complexity from Remote-Sensing Imagery

The rapid increase in digital data volumes from new and existing sensors necessitates the need for efficient analytical tools for extracting information. We developed an integrated software package called ICAMS (Image Characterization and Modeling System) to provide specialized spatial analytical functions for interpreting remote sensing data. This paper evaluates the three fractal dimension measurement methods: isarithm, variogram, and triangular prism, along with the spatial autocorrelation measurement methods Moran's I and Geary's C, that have been implemented in ICAMS. A modified triangular prism method was proposed and implemented. Results from analyzing 25 simulated surfaces having known fractal dimensions show that both the isarithm and triangular prism methods can accurately measure a range of fractal surfaces. The triangular prism method is most accurate at estimating the fractal dimension of higher spatial complexity, but it is sensitive to contrast stretching. The variogram method is a comparatively poor estimator for all of the surfaces, particularly those with higher fractal dimensions. Similar to the fractal techniques, the spatial autocorrelation techniques are found to be useful to measure complex images but not images with low dimensionality. These fractal measurement methods can be applied directly to unclassified images and could serve as a tool for change detection and data mining.

Lam, Nina Siu-Ngan↗
NASA NTRS1998

Physics of Boundaries and their Interactions in Space Plasmas

This final report describes a brief summary of our accomplishments during the complete contract period. Traditionally, due to computational limitations, it has been impossible to obtain a global view of the magnetosphere on ion time and spatial scales. As a result, kinetic-simulations have concentrated on the local structure of different magnetospheric discontinuities and boundaries. However, due to the emergence of low cost supercomputers, as well as by taking full advantage of latest advances in data mining and visualization technology, we were able to bypass our planned (proposed) regional simulations and proceed to large-scale 3-D and 2-D global hybrid simulations of the magnetosphere. As a result, although we are only finishing the second year of the proposed activity, much of the original scientific objectives have been surpassed and new avenues of investigation have been opened. Such simulations have led us to possible explanations of some long-standing issues in magnetospheric physics. They have also enables us to make a number of important discoveries predictions, which need to be looked for in satellite data. Examples include the finding that the bow shock can become unstable to the Kelvin-Helmholtz (KH), (2) the discovery of a mechanism for intermittent reconnection due to ion physics which may be relevant to the explanation of the recurrence rate of flux transfer events (FTEs), and (3) this finding that the current sheet in the near-Earth magnetotail region can become unstable to KH with detectable, unique ionospheric signatures. Further, we demonstrated a viable mechanism for the onset of reconnection at the magnetopause, examined the detailed structure of the boundary layer incorporating curvature effects, and provided an explanation for the large core fields observed within FTEs as well as flux ropes in the magnetotail.

Omidi, Nojan↗
NASA NTRS1998

Physics of Boundaries and their Interactions in Space Plasmas

This final report describes a brief summary of our accomplishments during the complete contract period. Traditionally, due to computational limitations, it has been impossible to obtain a global view of the magnetosphere on ion time and spatial scales. As a result, kinetic simulations have concentrated on the local structure of different magnetospheric discontinuities and boundaries. However, due to the emergence of low cost desktop superconductors, as well as by taking full advantage of latest advances in data mining and visualization technology, we were able to bypass our planned (proposed) regional simulations and proceed to large-scale 3-D and 2-D global hybrid simulations of the magnetosphere. As a result, although we are only finishing the second year of the proposed activity, much of the original scientific objectives have been surpassed and new avenues of investigation have been opened. Such simulations have led us to possible explanations of some long-standing issues in magnetospheric physics. They have also enabled us to make a number of important discoveries/predictions, which need to be looked for in satellite data. Examples include: (1) the finding that the bow shock can become unstable to the Kelvin-Helmholtz (KH;) (2) the discovery of a mechanism for intermittent reconnection due to ion physics which may be relevant to the explanation of the recurrence rate of flux transfer events (FTEs;) and (3) the finding that the current sheet in the near-Earth magnetotail region can become unstable to KH with detectable, unique ionospheric signatures. Further, we demonstrated a viable mechanism for the onset of reconnection at the magnetopause, examined the detailed structure of the boundary layer incorporating curvature effects, and provided an explanation for the large core fields observed within FTEs as well as flux ropes in the magnetotail.

Omidi, Nojan↗
DOE OSTI2021

Pumped-Storage Hydropower using Abandoned Underground Mines (PSH-AUM) as an Innovative Energy Storage Technology for Fossil-Integrated Systems

Pumped-storage hydropower (PSH) provides around 95% of all utility-scale energy storage in the U.S. and globally. Additional deployment of PSH is hampered by excessively long permitting and commissioning requirements and is constrained to locations for which natural topography provides suitable elevation relief between the upper and lower reservoirs (the ΔH challenge). The purpose of this research was to evaluate Pumped-Storage Hydropower using Abandoned Underground Mines (PSH-AUM) as a means to solve the ΔH challenge and initiate the commercialization pathway for a promising new energy storage technology. Four primary tasks were conducted: Screening and ranking of candidate sites for project development; multiphase reservoir modeling to evaluate mine performance; techno-economic analysis and preliminary designs for PSH-AUM systems integrated with fossil-fuel power units; and stakeholder engagement to identify pathways to commercialization of this new technology. Key results were achieved in each of the four primary tasks. Candidate site screening determined that nearly 10,000 underground mines were spatially locatable, of which more than 100 sites appear suitable for integration with existing fossil power plants that are expected to remain in longer-term operation. Mine reservoir models were developed using PNNL’s STOMP simulator and parameterized using candidate site data to evaluate interactions with the surrounding groundwater system and confirm the potential for some mines to accommodate inflows and outflows on the order of 100 m3/s over 8-hour durations (1.6 GWh system) without excessive aqueous pressures. Techno-economic analysis resulted in a project cost optimization scheme to identify key sensitivities and the development of preliminary designs that can minimize overall costs of deployment. Finally, stakeholder engagement with industry, state government, and local economic development leaders confirmed the viability of PSH-AUM as a promising new technology. Our Phase I project results suggest that PSH-AUM technology has a domestic market potential on the order of $100+ Billion with ample space for technology development to commercialization within the next decade.

13 HYDRO ENERGY↗
DOE OSTI2023

Spatio-Temporal Surrogates for Interaction of a Jet with High Explosives: Part II - Clustering Extremely High-Dimensional Grid-Based Data

Building an accurate surrogate model for the spatio-temporal outputs of a computer simulation is a challenging task. A simple approach to improve the accuracy of the surrogate is to cluster the outputs based on similarity and build a separate surrogate model for each cluster. This clustering is relatively straightforward when the output at each time step is of moderate size. However, when the spatial domain is represented by a large number of grid points, numbering in the millions, the clustering of the data becomes more challenging. In this report, we consider output data from simulations of a jet interacting with high explosives. These data are available on spatial domains of different sizes, at grid points that vary in their spatial coordinates, and in a format that distributes the output across multiple files at each time step of the simulation. We first describe how we bring these data into a consistent format prior to clustering. Borrowing the idea of random projections from data mining, we reduce the dimension of our data by a factor of thousand, making it possible to use the iterative k-means method for clustering. We show how we can use the randomness of both the random projections, and the choice of initial centroids in k-means clustering, to determine the number of clusters in our data set. Our approach makes clustering of extremely high dimensional data tractable, generating meaningful cluster assignments for our problem, despite the approximation introduced in the random projections.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Reconstruction of global gridded monthly sectoral water withdrawals for 1971–2010 and analysis of their spatiotemporal patterns

Human water withdrawal has increasingly altered the global water cycle in past decades, yet our understanding of its driving forces and patterns is limited. Reported historical estimates of sectoral water withdrawals are often sparse and incomplete, mainly restricted to water withdrawal estimates available at annual and country scales, due to a lack of observations at seasonal and local scales. In this study, through collecting and consolidating various sources of reported data and developing spatial and temporal statistical downscaling algorithms, we reconstruct a global monthly gridded (0.5°) sectoral water withdrawal dataset for the period 1971–2010, which distinguishes six water use sectors, i.e., irrigation, domestic, electricity generation (cooling of thermal power plants), livestock, mining, and manufacturing. Based on the reconstructed dataset, the spatial and temporal patterns of historical water withdrawal are analyzed. Results show that total global water withdrawal has increased significantly during 1971–2010, mainly driven by the increase in irrigation water withdrawal. Regions with high water withdrawal are those densely populated or with large irrigated cropland production, e.g., the United States (US), eastern China, India, and Europe. Seasonally, irrigation water withdrawal in summer for the major crops contributes a large percentage of total annual irrigation water withdrawal in mid- and high-latitude regions, and the dominant season of irrigation water withdrawal is also different across regions. Domestic water withdrawal is mostly characterized by a summer peak, while water withdrawal for electricity generation has a winter peak in high-latitude regions and a summer peak in low-latitude regions. Despite the overall increasing trend, irrigation in the western US and domestic water withdrawal in western Europe exhibit a decreasing trend. Our results highlight the distinct spatial pattern of human water use by sectors at the seasonal and annual timescales. Here, the reconstructed gridded water withdrawal dataset is open access, and can be used for examining issues related to water withdrawals at fine spatial, temporal, and sectoral scales.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

STILGAR End-of-Project Report

The Subsurface Tunnel Imaging LeveraGed by Analysis of Rayleigh wave ellipticity (STILGAR) project demonstrated an integrated geophysical approach for detecting, locating, and characterizing underground structural changes using dense seismic arrays and advanced inversion techniques. Field campaigns were conducted at two operational mines—the Redmond salt mine (Utah) and Graymont Pleasant Gap limestone mine (Pennsylvania)—providing real-world testbeds for monitoring anthropogenic subsurface activity. At the Redmond salt mine, seismic interferometry combined with back-projection inversion successfully identified continuous, low-amplitude signals from mining operations. The approach differentiated stationary from migrating anthropogenic sources, captured daily operational cycles, and validated the potential of passive seismic monitoring for remote detection of underground activity. At the Graymont Pleasant Gap mine, two dense seismic deployments in the spring and fall of 2023 generated over 4 TB of high-resolution data. Key outcomes included the relocation of 199 underground and 8 surface explosions with accuracies within tens of meters and the development of a 3D P-wave velocity model using the triple-difference tomography algorithm (tomoTD) that resolved major structural features such as the mine entrance, low-velocity tunnels, and roof-collapse areas. Ambient noise cross-correlation and back-projection analyses revealed persistent sources linked to ongoing mining activity, whereas horizontal-to-vertical spectral ratio (HVSR) and ellipticity studies confirmed stable site responses across seasons and identified soil thickness trends consistent with regional erosional and depositional processes. Checkerboard and sensitivity tests further validated the robustness of the tomographic results. Overall, the findings emphasize that although significant progress has been made in subsurface imaging, further work is needed to enhance the detection and localization of underground structures. Accurate imaging requires higher frequencies, yet anthropogenic sources tend to dominate the seismic record at those frequencies, and high-frequency surface waves are affected by higher modes that complicate interpretation. The improved detection and localization of human-induced signals enabled detailed temporal and spatial mapping of daily mine operations, demonstrating the feasibility of continuous anthropogenic source monitoring. Sensitivity to signals from nontraditional sources, such as fan operations, highlights the broader applicability of this approach to other industrial environments in which continuous and impulsive signals are present. The field campaigns produced a substantial volume of high-quality seismic data, supporting the development and testing of new methods for seismic source characterization and subsurface imaging. Future deployments should include sensors capable of recording lower frequencies to probe deeper structures, increase bandwidth to enhance resolution and sensitivity to both shallow and deep targets, and collect additional large-scale datasets to refine imaging and source characterization techniques. Moreover, conducting 3D modeling studies of seismic wavefields at higher frequencies will provide a better understanding of wave scattering and cavity–wavefield interactions in complex underground environments. In conclusion, the STILGAR project demonstrated that integrated seismic monitoring can effectively characterize underground operations, capturing both natural and anthropogenic signals. The approaches developed provide a foundation for improved detection, localization, and imaging of subsurface structures and are directly transferable to broader industrial monitoring applications.

58 GEOSCIENCES↗
DOE OSTI2011

GX-Means: A model-based divide and merge algorithm for geospatial image clustering

One of the practical issues in clustering is the specification of the appropriate number of clusters, which is not obvious when analyzing geospatial datasets, partly because they are huge (both in size and spatial extent) and high dimensional. In this paper we present a computationally efficient model-based split and merge clustering algorithm that incrementally finds model parameters and the number of clusters. Additionally, we attempt to provide insights into this problem and other data mining challenges that are encountered when clustering geospatial data. The basic algorithm we present is similar to the G-means and X-means algorithms; however, our proposed approach avoids certain limitations of these well-known clustering algorithms that are pertinent when dealing with geospatial data. We compare the performance of our approach with the G-means and X-means algorithms. Experimental evaluation on simulated data and on multispectral and hyperspectral remotely sensed image data demonstrates the effectiveness of our algorithm.

99 GENERAL AND MISCELLANEOUS↗