Search NASA⌕ Search

SEARCH · Search NASA

Results for “correlation mining”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2015

Foundational Principles for Large-Scale Inference: Illustrations Through Correlation Mining

When can reliable inference be drawn in the ‘‘Big Data’’ context? This article presents a framework for answering this fundamental question in the context of correlation mining, with implications for general large-scale inference. In large-scale data applications like genomics, connectomics, and eco-informatics, the data set is often variable rich but sample starved: a regime where the number n of acquired samples (statistical replicates) is far fewer than the number p of observed variables (genes, neurons, voxels, or chemical constituents). Much of recent work has focused on understanding the computational complexity of proposed methods for ‘‘Big Data.’’ Sample complexity, however, has received relatively less attention, especially in the setting when the sample size n is fixed, and the dimension p grows without bound. To address this gap, we develop a unified statistical framework that explicitly quantifies the sample complexity of various inferential tasks. Sampling regimes can be divided into several categories: 1) the classical asymptotic regime where the variable dimension is fixed and the sample size goes to infinity; 2) the mixed asymptotic regime where both variable dimension and sample size go to infinity at comparable rates; and 3) the purely high-dimensional asymptotic regime where the variable dimension goes to infinity and the sample size is fixed. Each regime has its niche but only the latter regime applies to exa-scale data dimension. We illustrate this high-dimensional framework for the problem of correlation mining, where it is the matrix of pairwise and partial correlations among the variables that are of interest. Correlation mining arises in numerous applications and subsumes the regression context as a special case. We demonstrate various regimes of correlation mining based on the unifying perspective of high-dimensional learning rates and sample complexity for different structured covariance models and different inference tasks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Data mining graphene: Correlative analysis of structure and electronic degrees of freedom in graphenic monolayers with defects

The link between changes in the material crystal structure and its mechanical, electronic, magnetic, and optical functionalities known as the structure-property relationship is the cornerstone of the contemporary materials science research. The recent advances in scanning transmission electron and scanning probe microscopies (STEM and SPM) have opened an unprecedented path towards examining the materials structure property relationships on the single-impurity and atomic-configuration levels. Lacking, however, are the statistics-based approaches for cross-correlation of structure and property variables obtained in different information channels of the STEM and SPM experiments. Here we have designed an approach based on a combination of sliding window Fast Fourier Transform, Pearson correlation matrix, linear and kernel canonical correlation, to study a relationship between lattice distortions and electron scattering from the SPM data on graphene with defects. Our analysis revealed that the strength of coupling to strain is altered between different scattering channels which can explain coexistence of several quasiparticle interference patterns in the nanoscale regions of interest. In addition, the application of the kernel functions allowed us extracting a non-linear component of the relationship between the lattice strain and scattering intensity in graphene. Lastly, the outlined approach can be further utilized to analyzing correlations in various multi-modal imaging techniques where the information of interest is spatially distributed and has usually a complex multidimensional nature.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Applying Waveform Correlation to Mining Blasts Using a Global Sparse Network

Agencies that monitor for underground nuclear tests are interested in techniques that automatically characterize mining blasts to reduce the human analyst effort required to produce high-quality event bulletins. Waveform correlation is effective in finding similar waveforms from repeating seismic events, including mining blasts. We report the results of an experiment that uses waveform templates recorded by multiple International Monitoring System stations of the Comprehensive Nuclear-Test-Ban Treaty for up to 10 years prior to detect and identify mining blasts that occur during single weeks of study. We discuss approaches for template selection, threshold setting, and event detection that are specialized for mining blasts and a sparse, global network. We apply the approaches to two different weeks of study for each of two geographic regions, Wyoming and Scandinavia, to evaluate the potential for establishing a set of standards for waveform correlation processing of mining blasts that can be effective for operational monitoring systems with a sparse network. We compare candidate events detected with our processing methods to the Reviewed Event Bulletin of the International Data Centre to develop an intuition about potential reduction in analyst workload.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
DOE OSTI2020

Applying Waveform Correlation to Mining Blasts Using a Global Sparse Network

Agencies that monitor for underground nuclear tests are interested in techniques that automatically characterize mining blasts to reduce the human analyst effort required to produce high-quality event bulletins. Waveform correlation is effective in finding similar waveforms from repeating seismic events, including mining blasts. We report the results of an experiment that uses waveform templates recorded by multiple International Monitoring System stations of the Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization for up to 10 years prior to the time period of interest to detect and identify mining blasts that occur during single weeks of study. We discuss approaches for template selection, threshold setting, and event detection that are specialized for mining blasts and a sparse, global network. We apply the approaches to two different weeks of study for each of two geographic regions, Wyoming and Scandinavia, to evaluate the potential for establishing a set of standards for waveform correlation processing of mining blasts that can be effective for operational monitoring systems with a sparse network. We compare candidate events detected with our processing methods to the Reviewed Event Bulletin of the International Data Centre to develop an intuition about potential reduction in analyst workload.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗
DOE OSTI2018

Quickest Detection for Changes in Maximal kNN Coherence of Random Matrices

This paper addresses the problem of quickest detection of a change in the maximal coherence between columns of a n p random matrix based on a sequence of matrix observations having a single unknown change point. The random matrix is assumed to have identically distributed rows and the maximal coherence is defined as the largest of the ( 2 p) correlation coefficients associated with any row. Likewise the κ nearest neighbor (kNN) coherence is defined as the κ-th largest of these correlation coefficients. The forms of the preand postchange distributions of the observed matrices are assumed to belong to the family of elliptically contoured densities with sparse dispersion matrices but are otherwise unknown. A nonparametric stopping rule is proposed that is based on the maximal kNN sample coherence between columns of each observed random matrix. This is a summary statistic that is related to a test of existence of a hub vertex in a sample correlation graph having degree at least κ. Performance bounds on the delay and false alarm performance of the proposed stopping rule are obtained in the purely high-dimensional regime where p → ∞ and n is fixed. When the prechange dispersion matrix is diagonal it is shown that, among all functions of the proposed summary statistic, the proposed stopping rule is asymptotically optimal under a minimax quickest change detection model as the stopping threshold approaches infinity. Furthermore, the theory developed also applies to sequential hypothesis testing and fixed sample size tests.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2022

Applying Waveform Correlation and Waveform Template Metadata to Mining Blasts to Reduce Analyst Workload

Organizations that monitor for underground nuclear explosive tests are interested in techniques that automatically characterize mining blasts to reduce the human analyst effort required to produce high - quality event bulletins. Waveform correlation is effective in finding similar waveforms from repeating seismic events, including mining blasts. In this study we use waveform template event metadata to seek corroborating detections from multiple stations in the International Monitoring System of the Preparatory Commission for the Comprehensive Nuclear-Test-Ban Treaty Organization. We build upon events detected in a prior waveform correlation study of mining blasts in two geographic regions, Wyoming and Scandinavia. Using a set of expert analyst-reviewed waveform correlation events that were declared to be true positive detections, we explore criteria for choosing the waveform correlation detections that are most likely to lead to bulletin-worthy events and reduction of analyst effort.

47 OTHER INSTRUMENTATION↗
DOE OSTI2018

Exploring Properties and Correlations of Fatal Events in a Large-Scale HPC System

In this study, we explore potential correlations of fatal system events for one of the most powerful supercomputers-IBM Blue Gene/Q Mira, which is deployed at Argonne National Laboratory, based on its 5-year reliability, availability, and serviceability (RAS) log. Our contribution is two-fold. (1) We design an efficient log analysis tool, namely LogAider, with a novel filtering method to effectively extract fatal events from masses of system messages that are heavily duplicated in the log. LogAider exhibits a very precise detection of temporal-correlation with a high similarity (up to 95 percent) to the ground-truth (i.e., compared to the failure records reported by the administrators). The total number of fatal events can be reduced to about 1,255 compared with originally 2.6 million duplicated fatal messages. (2) We analyze the 5-year RAS log of the MIRA system using LogAider, and summarize six important "takeaways" which can help system vendors and administrators better understand an extreme-scale system's fatal events. Specifically, we find that the distribution or proportion of the fatal system events follow a Pareto-like principle in general. The temporal correlation among fatal events is much stronger than that of warn messages and info messages, and the correlated events tend to constitute a few clusters. The mean time between fatal events (MTBFE) of the Mira system is about 1.3 days from the perspective of the system, and the MTTI is 2-4 days from the perspective of users. The most error-prone item value with respect to any key attribute appears likely in the log every 2-10 days. Weibull, Gamma, and Pearson6 are the three best-fit distributions for the fatal event intervals. The overall correlation of fatal events on the 5D torus network is not prominent, whereas the small-region locality correlation (e.g., the fatal events inside racks) is relatively strong. We believe our work will be interesting to large-scale HPC system administrators and vendors and to fault tolerance researchers, enabling them to better understand fatal events and mitigate such events accordingly.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Automation of Data Collection Methods for Online Monitoring of Nuclear Power Plants

Maintenance activities in the United States nuclear power industry are mainly centered on preventive and corrective maintenance. Predictive maintenance, which is a variable time basis maintenance strategy that is dependent on the condition of the equipment, is less utilized but potentially more effective. In the current maintenance practices, the nuclear power plants rely on manual processes to supply information to a human decision making process. The plants have several processes that generate information that is not typically used beyond the intended target for collecting that information. This information is a source of data that is manually collected by labor intensive tasks. The data is therefore expensive to collect, yet limited in use. The industry has realized the opportunity present in reducing the labor intensive tasks by automating them and increasing the uses of the data collected by either developing advanced predictive maintenance methods using data-driven decision making that could span from trending to machine learning and artificial intelligence. This shift in maintenance strategy may result in high cost saving impact. In addition to reducing labor requirements and frequency of maintenance activities, this shift in strategy will reduce the materials portion of the operations and maintenance cost as well. The aim of this specific effort is to focus on the automation of data collection process. It is part of a series of planned efforts to target multiple elements of migrating the current preventive maintenance approach in nuclear power plants to a data-driven approach. These elements include data collection, management, analysis, visualization, in addition to value analysis and change enabling. The other five elements of migration were not considered because they are planned to be explicitly studied in multiple parallel efforts. Challenges including regulations requirements or limitations are planned in a future effort that focuses on change enabling. The risk and benefit/cost of the approach shift is planned in another effort targeted towards value. The storage, communication, computational power, data architecture, and protection are part of the information infrastructure and are planned in dedicated data management effort. The best means to present the data for assisted human decision making is planned in a data visualization effort. The data interpretation and patterns recognition for machine decision making is planned through multiple efforts in data analytics over the following years. This work identified seventeen data sources collection methods in nuclear power plants that could be automated for an integrated data platform that enables comprehensive and informed decision making. For each of the identified seventeen data sources, the base, modern, and state of the art states were described. The current state of data collection used by most utilities was identified as the "base state". The current achievable state was identified as the "modern state", a state that has either been achieved by at least some US utilities or in other industries. The future state that requires research and development to reach was identified as the "state of the art" state. The described states used an out-of-the-box thinking approach. This effort found that there is a hug potential for task automation and data fidelity improvements by evolving the current industry data collection methods. The majority of the processes analyzed were found to be manual and labor intensive and fell in the base state. As a result, the modern state often targeted reducing the manual logging efforts by using electronic and semi-intelligent means or complementing the work with sensors and mobile or fixed technologies that can reduce the demand of labor activities. The state of the art state involved applications advanced methods of data correlation, data mining, and machine learning along with increasing the spatial distribution and fidelity of data, coupling multiple data sources, and enabling the use of sophisticated mobile devices and machine comprehension methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Applying Waveform Correlation to Reduce Seismic Analyst Workload Due to Repeating Mining Blasts

Agencies that monitor for underground nuclear tests are interested in techniques that automatically characterize mining blasts to reduce the human analyst effort required to produce high–quality event bulletins. Waveform correlation is effective in finding similar waveforms from repeating seismic events, including mining blasts. We report the results of an experiment to detect and identify mining blasts for two regions, Wyoming (U.S.A.) and Scandinavia, using waveform templates recorded by multiple International Monitoring System stations of the Preparatory Commission for the Comprehensive Nuclear–Test–Ban Treaty Organization (CTBTO PrepCom) for up to 10 yr prior to the time of interest. We discuss approaches for template selection, threshold setting, and event detection that are specialized for characterizing mining blasts using a sparse, global network. We apply the approaches to one week of data for each of the two regions to evaluate the potential for establishing a set of standards for waveform correlation processing of mining blasts that can be generally applied to operational monitoring systems with a sparse network. Here, we compare candidate events detected with our processing methods to the Reviewed Event Bulletin of the International Data Centre to assess potential reduction in analyst workload.

58 GEOSCIENCES↗