Search NASA⌕ Search

SEARCH · Search NASA

Results for “Data Mining”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Aho-Corasick String Matching on Shared and Distributed Memory Parallel Architectures

String matching is at the core of many critical applications, including network intrusion detection systems, search engines, virus scanners, spam filters, DNA and protein sequencing, and data mining. For all of these applications string matching requires a combination of (sometimes all) the following characteristics: high and/or predictable performance, support for large data sets and flexibility of integration and customization. Many software based implementations targeting conventional cache-based microprocessors fail to achieve high and predictable performance requirements, while Field-Programmable Gate Array (FPGA) implementations and dedicated hardware solutions fail to support large data sets (dictionary sizes) and are difficult to integrate and customize. The advent of multicore, multithreaded, and GPU-based systems is opening the possibility for software based solutions to reach very high performance at a sustained rate. This paper compares several software-based implementations of the Aho-Corasick string searching algorithm for high performance systems. We discuss the implementation of the algorithm on several types of shared-memory high-performance architectures (Niagara 2, large x86 SMPs and Cray XMT), distributed memory with homogeneous processing elements (InfiniBand cluster of x86 multicores) and heterogeneous processing elements (InfiniBand cluster of x86 multicores with NVIDIA Tesla C10 GPUs). We describe in detail how each solution achieves the objectives of supporting large dictionaries, sustaining high performance, and enabling customization and flexibility using various data sets.

99 GENERAL AND MISCELLANEOUS↗

RNAV STAR Procedural Adherence

In this exploratory archival study we mined the performance of 24 major US airports area navigation standard terminal arrival routes (RNAV STARs) over the preceding three years. Overlaying radar track data on top of RNAV STAR routes provided a comparison between aircraft flight paths and the waypoint positions and altitude restrictions. NASA Ames Supercomputing resources were utilized to perform the data mining and processing. We investigated STARs by lateral transition path (full-lateral), vertical restrictions (full-lateral/full-vertical), and skipped waypoints (skips). In addition, we graphed altitudes and their frequencies of occurrence for altitude restrictions. Full-lateral compliance was generally greater than Full-lateral/full-vertical, but the delta between the rates was not always consistent. Full-lateral/full-vertical usage medians of the 2016 procedures ranged from 0 in KDEN (Denver) to 21 in KMEM (Memphis). Waypoint skips ranged from 0 to nearly 100 for specific waypoints. Altitudes restrictions were sometimes missed by systemic amounts in 1000 ft. increments from the restriction, creating multi-modal distributions. Other times, altitude misses looked to be more normally distributed around the restriction. This work is a preliminary investigation into the objective performance of instrument procedures and provides a framework to track how procedural concepts and design intervention function. In addition, this tool may aid in providing acceptability metrics as well as risk assessment information.

Stewart, Michael J.↗

Efficient Aho-Corasick String Matching on Emerging Multicore Architectures

String matching algorithms are critical to several scientific fields. Beside text processing and databases, emerging applications such as DNA protein sequence analysis, data mining, information security software, antivirus, ma- chine learning, all exploit string matching algorithms [3]. All these applica- tions usually process large quantity of textual data, require high performance and/or predictable execution times. Among all the string matching algorithms, one of the most studied, especially for text processing and security applica- tions, is the Aho-Corasick algorithm. 1 2 Book title goes here Aho-Corasick is an exact, multi-pattern string matching algorithm which performs the search in a time linearly proportional to the length of the input text independently from pattern set size. However, depending on the imple- mentation, when the number of patterns increase, the memory occupation may raise drastically. In turn, this can lead to significant variability in the performance, due to the memory access times and the caching effects. This is a significant concern for many mission critical applications and modern high performance architectures. For example, security applications such as Network Intrusion Detection Systems (NIDS), must be able to scan network traffic against very large dictionaries in real time. Modern Ethernet links reach up to 10 Gbps, and malicious threats are already well over 1 million, and expo- nentially growing [28]. When performing the search, a NIDS should not slow down the network, or let network packets pass unchecked. Nevertheless, on the current state-of-the-art cache based processors, there may be a large per- formance variability when dealing with big dictionaries and inputs that have different frequencies of matching patterns. In particular, when few patterns are matched and they are all in the cache, the procedure is fast. Instead, when they are not in the cache, often because many patterns are matched and the caches are continuously thrashed, they should be retrieved from the system memory and the procedure is slowed down by the increased latency. Efficient implementations of string matching algorithms have been the fo- cus of several works, targeting Field Programmable Gate Arrays [4, 25, 15, 5], highly multi-threaded solutions like the Cray XMT [34], multicore proces- sors [19] or heterogeneous processors like the Cell Broadband Engine [35, 22]. Recently, several researchers have also started to investigate the use Graphic Processing Units (GPUs) for string matching algorithms in security applica- tions [20, 10, 32, 33]. Most of these approaches mainly focus on reaching high peak performance, or try to optimize the memory occupation, rather than looking at performance stability. However, hardware solutions supports only small dictionary sizes due to lack of memory and are difficult to customize, while platforms such as the Cell/B.E. are very complex to program.

Tumeo, Antonino↗

The Application of Imaging to Very High Energy Gamma-ray Astronomy (Final Technical Report: VERITAS Site Operations 2013-2018)

VERITAS is the most-sensitive very-high-energy (VHE; E > 100 GeV) gamma-ray observatory in the world. This award partially supported its experimental operations, via the VERITAS Project Office, from 2013-18. Project operations were based at the Smithsonian’s F.L. Whipple Observatory (FLWO) near Tucson, AZ. During this time VERITAS completed its seventh through eleventh seasons of operations with its full four-telescope array. These were the second through the sixth seasons since the completion of a major detector upgrade which lowered the observatory’s energy threshold by 40% and enables it to detect objects twice as fast as when it began full-scale operations. The VERITAS Project Office operated within its budget throughout this period, as it has done throughout its history. Similarly the observatory met, and often exceeded, all of its performance metrics during the five-year period. In all, ~6600 h of data were acquired, of which nearly 5700 h are of publishable weather quality, and 93% contain the full four-telescope array. All categorical breakdowns of the data taking (e.g. hours devoted to dark matter research) match those envisioned in the funded operations proposal. As of April 2019, the VERITAS collaboration of ~80 scientists had reported the detection of 63 astrophysical sources of very high energy (VHE; E>100 GeV) gamma rays, belonging to ~8 different source classes. This marks an increase of 21 sources since the funding of this award. A total of 55 journal articles and 33 PhD theses were published by the VERITAS collaboration during the period covered by the award. The collaboration publication totals are now 108 articles and 63 theses, since project inception, and these publications have been cited ~3200 times. In addition, the collaboration released 22 Astronomer’s Telegrams between 2013-18 alerting the community to important phenomena in a timely manner. VERITAS remains fully operational and its most recent season (2017-18) was, in many respects, the best in VERITAS’s history in terms of technical performance. If funding is secured from various partners the collaboration desires to operate the instrument until at least 2023. Regardless of the future of VERITAS site-operations, the VERITAS data archive contains all previous observations and will remain accessible online for data mining efforts until at least 2024 (i.e. at least 5 years after the close of operations). This will enable the production of publications from all data taken during the period of performance of this award.

79 ASTRONOMY AND ASTROPHYSICS↗

Methodology for tomographic imaging ahead of mining using the shearer as a seismic source

Poor rock conditions in a coal long wall panel can result in roof collapse when a problematic zone is mined, significantly interrupting mine production. The ability to image rock conditions (stress and degree of fracturing) ahead of the face gives the miners the ability to respond proactively to such problems. This method uses the energy from mining machinery, in this case a coal shearer, to produce an image of the rock velocity ahead of the mining face without interrupting mining. Data from an experiment illustrates the concept. Geophones installed in gate-road roofs record the noise generated by the shearer after it has traversed the panel ahead of the mining face. A generalized crosscorrelation of the signals from pairs of sensors determines relative arrival times from the continuous seismic noise produced by the shearer. These relative times can then be inverted for a velocity structure. The crosscorrelations, performed in the frequency domain, are weighted by a confidence value derived from the spectral coherence between the traces. This produces stable crosscorrelation lags in the presence of noise. The errors in the time-domain data are propagated through to the relative traveltimes and then to the final tomographic velocity image, yielding an estimate of the uncertainty in velocity at each point. This velocity image can then be used to infer information about the stress and fracture state of the rock, providing advance warning of potentially hazardous zones.

01 COAL, LIGNITE, AND PEAT↗

Saturday Science with Geoff Fairchild

In this first episode, we sit down (on a chairlift) with Computer Scientist Geoff Fairchild to talk about how his team is using Los Alamos National Laboratory’s supercomputing capabilities to data mine social media and reveal trends relating to health and safety.

97 MATHEMATICS AND COMPUTING↗

Examining Runner’s Outdoor Heat Exposure Using Urban Microclimate Modeling and GPS Trajectory Mining

It is important to quantify human heat exposure in order to evaluate and mitigate the negative impacts of heat on human well-being in the context of global warming. This study proposed a human-centric framework to examine human personal heat exposure based on anonymous GPS trajectories data mining and urban microclimate modeling. The mean radiant temperature (Tmrt) that represents the human body’s energy balance was used to indicate human heat exposure. The meteorological data and high-resolution 3D urban model generated from multispectral remotely sensed images and LiDAR data were used as inputs in urban microclimate modeling to map the spatio-temporal distribution of the Tmrt in the Boston metropolitan area. The anonymous human GPS trajectory data collected from fitness Apps was used to map the spatiotemporal distribution of human outdoor activities. By overlaying the anonymous GPS trajectories on the generated spatio-temporal maps of Tmrt, this study further examined the heat exposure of runners in different age-gender groups in the Boston area. Results show that there is no significant difference in terms of heat exposure for female and male runners. The female runners in the age of 45-54 are exposed to more heat than female runners of 18-24 and 25-34, while there is no significant difference among male runners. This study proposed a novel method to estimate human heat exposure, which would shed new light on mitigating the negative impacts of heat on human health.

Personal heat exposure↗

A neural network for determination of latent dimensionality in Nonnegative Matrix Factorization

Non-negative Matrix Factorization (NMF) has proven to be a powerful unsupervised learning method for uncovering hidden features in complex and noisy datasets with applications in data mining, text recognition, dimension reduction, face recognition, anomaly detection, blind source separation, and many other fields. An important input for NMF is the latent dimensionality of the data, that is, the number of hidden features, K, present in the explored dataset. Unfortunately, and this quantity is rarely known a priori. The existing methods for determining latent dimensionality, such as Automatic Relevance Determination (ARD), are mostly heuristic and utilize different characteristics to estimate the number of hidden features. However, all of them require human presence to make a final determination of K. Here we utilize a supervised machine learning approach in combination with a recent method for model determination, called NMFk, to determine the number of hidden features automatically. NMFk performs a set of NMF simulations on an ensemble of matrices, obtained by bootstrapping the initial dataset, and estimates which K produces stable groups of latent features that reconstruct the initial dataset well. We then train a Multi-Layter Perceptron (MLP) classifier network to determine the correct number of latent features utilizing the statistics and characteristics of the NMF solution, obtained from NMFk. In order to train the MLP classifier, a training set of 58,660 matrices with predetermined latent features were factorized with NMFk. The MLP classifier in conjunction with NMFk maintains a greater than 95% success rate when applied to a held out test set. Additionally, when applied to two well-known benchmark datasets, the swimmer and MIT face data, NMFk/MLP correctly recovers the established number of hidden features. Finally, we compare the accuracy of our method to the ARD, AIC and Stability-based methods.

97 MATHEMATICS AND COMPUTING↗

MO-E-BRB-04: Real-Time Exit-Fluence Delivery Validation

Recent high profile reports of technical failures and human errors causing severe radiation- induced injuries and deaths come in support of the sustained efforts to ensure patient safety in the delivery of radiation treatments. In addition, highly conformal radiation therapies and escalated fraction doses mandate increased and sustained accuracy of the entire radiotherapy process. Consequently, and as a Result of AAPM and ASTRO led efforts patient specific quality assurance for specialized radiation treatments such as IMRT, SRS/SBRT and Arc Therapy had become a three-tier process: Pre-treatment, during treatment, and post treatment patient specific QA. Traditional patient QA consists of pre-treatment data transfer integrity dosimetric verifications and during-treatment geometric verifications. However, as treatment adaptation becomes closer to deployment in the clinics, during treatment validation via exit detectors had become a realistic QA option, permitting plan assessment in near real time. Post-treatment, machine logs allow comparisons of a range of mechanical parameters. A combination of these techniques could be used in evaluating inter-fraction, and intra-fraction delivery over a long time period such as an year, to evaluate the significant errors per site, per treatment technique. This type of data mining over longer periods of time provides the potential to recognize suboptimal radiation treatments, while allowing to identify systematic, possibly significant errors. This would allow creation of a data base of realized errors, small and large in dosimetry that could be for process or equipment improvement. This educational symposium will describe and review patient QA techniques, results, and strategies for patient specific quality assurance. Learning Objectives: review the goals of pre-treatment QA for various specialized procedures review methods and means for pre-treatment QA, limitations and tolerances review the scenarios where Varian/Tomo Log files could be used. review error detection via log file analysis Learn how exit detector signal recorded in log file could be used to improve patient safety understand real-time EPID-based transmission dosimetry methods understand the goals of real-time delivery monitoring CMS Innovation grant. Rapid response supportive and palliative care program for patients with advanced cancer, starts July, 2012; Accuray Inc. Exit detector based QA software evaluation, Sept, 2012-Sept, 2013; ROI GRant from ASTRO “Safety and Quality: IMRT treatment delivery accuracy”.

60 APPLIED LIFE SCIENCES↗

Analyzing Double Delays at Newark Liberty International Airport (EWR)

When weather or congestion impacts the National Airspace System, multiple different Traffic Management Initiatives can be implemented, sometimes with unintended consequences. One particular perceived inequity that is commonly identified is in the interaction between Ground Delay Programs (GDPs) and time based scheduling of internal departures by the Traffic Management Advisor (TMA) (now operationally superseded by the FAA's the Time-Based Flow Management system). Internal departures under TMA scheduling can take large GDP delays, followed by large TMA scheduling delays, because they cannot easily fit into the arrival flow at the runway. In this paper we examine the causes of these double delays through an analysis of arrival operations at Newark Liberty International Airport (EWR) from June to August 2010. TMA scheduling delays are found to be generally higher than TMA airborne metering delays, regardless of prior GDP delays. Depending on how the double delay is defined, between 42 and 62 of all internal departures in GDP and TMA scheduling experienced double delays in this period. A deep dive into the data reveals that contributors to double delays include upstream flights departing before their Expect Departure Clearance Times (EDCTs); differences in the rates used for setting EDCTs and TMA Scheduled Times of Arrival; differences in the arrival demand expected based on EDCTs and the arrival demand entering TMA; and shorter en route times between takeoff and entry into TMA than assumed in the calculation of flight EDCTs, all of which undermine the sequencing and spacing underlying flight EDCTs. Double delays are also found to coincide with periods in which the virtual runway arrival queue being served by a TMA is large, there are periods of high demand relative to capacity, and there are high airborne metering delays. Data mining techniques are used to confirm that each of these factors contribute to the occurrence of double delay andor high internal departure scheduling delay across three months of data from June to August 2010. Predictors of the occurrence of double delay and high TMA scheduling delay are built using logistic regression, providing prediction accuracies of 69 and 73, respectively.

traffic flow management↗

AMIA KDDM Working Group Collaborative Workshop: Enriching Electronic Health Records with Social Determinants of Health to Improve Outcomes and Health Equity

Prior research has demonstrated that social determinants of health (SDoH) are major drivers of health outcomes and contributors to widespread health inequities. It was estimated that, in the United States, SDoH could be responsible for up to 40% of all preventable deaths, significantly higher than the 10-15% for which better medical care is responsible. Public health interventions that target SDoH are instrumental for improving health outcomes and reducing long-standing health inequities. Currently, most mainstream EHR vendors have implemented SDoH screeners in their EHR systems. However, the utility of the screeners is low, rendering patient-level SDoH still widely unavailable in the structured fields. SDoH are sometimes mentioned in free-text clinical notes (e.g., social context section) where natural language processing (NLP) can be applied to extract relevant information. Contextual-level SDoH can be identified from multiple data sources, many of which are publicly available and spatiotemporally linked to EHR data. As such, there is an opportunity for the KDDM research community to create innovative solutions to draw meaningful insights by creating and using rich data with SDoH to improve health outcomes while reducing disparities. In this workshop organized by AMIA Knowledge Discovery and Data Mining Working Group (AMIA KDDM WG), we will invite world-leading experts from academia, national laboratories, and life science industry with varied backgrounds in biomedical informatics, epidemiology, data science, machine learning, natural language processing, and pediatric cardiology to discuss the best practice of capturing, standardizing, and using SDoH information in various applications aiming at improving outcomes and health equity.

He, Zhe↗

Automating Analysis of Neutron Scattering Time-of-Flight Single Crystal Phonon Data

This article introduces software called Phonon Explorer that implements a data mining workflow for large datasets of the neutron scattering function, S(Q, ω), measured on time-of-flight neutron spectrometers. This systematic approach takes advantage of all useful data contained in the dataset. It includes finding Brillouin zones where specific phonons have the highest scattering intensity, background subtraction, combining statistics in multiple Brillouin zones, and separating closely spaced phonon peaks. Using the software reduces the time needed to determine phonon dispersions, linewidths, and eigenvectors by more than an order of magnitude.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A framework for activity detection in wide-area motion imagery

Wide-area persistent imaging systems are becoming increasingly cost effective and now large areas of the earth can be imaged at relatively high frame rates (1-2 fps). The efficient exploitation of the large geo-spatial-temporal datasets produced by these systems poses significant technical challenges for image and video analysis and data mining. In recent years there has been significant progress made on stabilization, moving object detection and tracking and automated systems now generate hundreds to thousands of vehicle tracks from raw data, with little human intervention. However, the tracking performance at this scale, is unreliable and average track length is much smaller than the average vehicle route. This is a limiting factor for applications which depend heavily on track identity, i.e. tracking vehicles from their points of origin to their final destination. In this paper we propose and investigate a framework for wide-area motion imagery (W AMI) exploitation that minimizes the dependence on track identity. In its current form this framework takes noisy, incomplete moving object detection tracks as input, and produces a small set of activities (e.g. multi-vehicle meetings) as output. The framework can be used to focus and direct human users and additional computation, and suggests a path towards high-level content extraction by learning from the human-in-the-loop.

99 GENERAL AND MISCELLANEOUS↗

Big Data Analytics for Demand Response: Clustering Over Space and Time

The pervasive deployment of advanced sensing infrastructure in Cyber-Physical systems, such as the Smart Grid, has resulted in an unprecedented data explosion. Such data exhibit both large volumes and high velocity characteristics, two of the three pillars of Big Data, and have a time-series notion as datasets in this context typically consist of successive measurements made over a time interval. Time-series data can be valuable for data mining and analytics tasks such as identifying the “right” customers among a diverse population, to target for Demand Response programs. However, time series are challenging to mine due to their high dimensionality. In this paper, we motivate this problem using a real application from the smart grid domain. We explore novel representations of time-series data for BigData analytics, and propose a clustering technique for determining natural segmentation of customers and identification of temporal consumption patterns. Our method is generizable to large-scale, real-world scenarios, without making any assumptions about the data. We evaluate our technique using real datasets from smart meters, totaling ~ 18,200,000 data points, and show the efficacy of our technique in efficiency detecting the number of optimal number of clusters.

97 MATHEMATICS AND COMPUTING↗

International Space Station (ISS) Anomalies Trending Study: Appendices - Volume II

The NASA Engineering and Safety Center (NESC) set out to utilize data mining and trending techniques to review the anomaly history of the International Space Station (ISS) and provide tools for discipline experts not involved with the ISS Program to search anomaly data to aid in identification of areas that may warrant further investigation. Additionally, the assessment team aimed to develop an approach and skillset for integrating data sets, with the intent of providing an enriched data set for discipline experts to investigate that is easier to navigate, particularly in light of ISS aging and the plan to extend its life into the late 2020s. This document contains the Appendices to the Volume I report.

Beil, Robert J.↗

International Space Station (ISS) Anomalies Trending Study

The NASA Engineering and Safety Center (NESC) set out to utilize data mining and trending techniques to review the anomaly history of the International Space Station (ISS) and provide tools for discipline experts not involved with the ISS Program to search anomaly data to aid in identification of areas that may warrant further investigation. Additionally, the assessment team aimed to develop an approach and skillset for integrating data sets, with the intent of providing an enriched data set for discipline experts to investigate that is easier to navigate, particularly in light of ISS aging and the plan to extend its life into the late 2020s. This report contains the outcome of the NESC Assessment.

Beil, Robert J.↗

A Taxonomic Classification Approach for Global Spatio-temporal Data

The World Bank, World Health Organization, and other major vendors collectively provide thousands of global time series datasets that focus on issues of the environment, public health, economics, violence, education, and national security. Sorting these data into meaningful information requires the use of data mining techniques to cluster trends into an orderly and manageable number of cases. The World SpatioTemporal Analytics and Mapping (WSTAMP) project database (wstamp.ornl.gov) was developed to spatiotemporally harmonize global vendor data (23,300+ attributes, 200+ locations, 50+ years). Within the WSTAMP analytical environment, Dynamic Time Warping (DTW) has been a highly effective data-driven approach for clustering and mapping these time series into national spatiotemporal behavior maps. Two significant properties have surfaced from this work. First, several recognizable cluster patterns have emerged and persist across a range of locations, attributes, and time frames (e.g., increasing, decreasing, rebounding, peak, oscillating). Secondly, practitioners engaging WSTAMP have noted the explanatory and anticipatory value of these patterns and articulated particular interest in detecting them within the spatiotemporal cube. This need was addressed by shifting DTW-based clustering from an open ended, data-driven implementation to a taxonomic pattern matching approach. This paper presents the method including implementation strategies for visualization and human computer interaction and applies the approach to a sample data set and concludes with next steps.

Stewart, Robert↗

Phase Selection Rules of Multi‐Principal Element Alloys

Abstract Computational prediction of phase stability of multi‐principal element alloys (MPEAs) holds a lot of promise for rapid exploration of the enormous design space and autonomous discovery of superior structural and functional properties. Regardless of many plausible works that rely on phenomenological theory and machine learning, precise prediction is still limited by insufficient data and the lack of interpretability of some machine learning algorithms, e.g., convolutional neural network. In this work, a comprehensive approach is presented, encompassing the development of a complete dataset that contains 72 387 density functional theory calculations, as well as a predictive global phenomenological descriptor. The phase selection descriptor, based on atomic electronegativity and valence electron concentration, significantly outperforms the widely used valence electron concentration, excelling in both accuracy (with an f1 score of 63% compared to 47%) and its ability to predict the HCP phase (0.48 recall compared to 0). The comprehensive data mining on the global design space of 61 425 quaternary MPEAs made from 28 possible metals, together with the phenomenological theory and physical interpretation, will set up a solid computational science foundation for data‐driven exploration of MPEAs.

Chemistry↗