On data collection graph construction and sampling in Twitter.
Abstract not provided.
Wendt, Jeremy D↗
SEARCH · Search NASA
Results for “Twitter”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Mining Twitter Data to Augment NASA GPM Validation
The Twitter data stream is an important new source of real-time and historical global information for potentially augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. There have been other similar uses of Twitter, though mostly related to natural hazards monitoring and management. The validation of satellite precipitation estimates is challenging, because many regions lack data or access to data, especially outside of the U.S. and in remote and developing areas. The time-varying set of "precipitation" tweets can be thought of as an organic network of rain gauges, potentially providing a widespread view of precipitation occurrence. Twitter provides a large source of crowd for crowdsourcing. During a 24-hour period in the middle of the snow storm this past March in the U.S. Northeast, we collected more than 13,000 relevant precipitation tweets with exact geolocation. The overall objective of our project is to determine the extent to which processed tweets can provide additional information that improves the validation of GPM data. Though our current effort focuses on tweets and precipitation, our approach is general and applicable to other social media and other geophysical measurements. Specifically, we have developed an operational infrastructure for processing tweets, in a format suitable for analysis with GPM data; engaged with potential participants, both passive and active, to "enrich" the Twitter stream; and inter-compared "precipitation" tweet data, ground station data, and GPM retrievals. In this presentation, we detail the technical capabilities of our tweet processing infrastructure, including data abstraction, feature extraction, search engine, context-awareness, real-time processing, and high volume (big) data processing; various means for "enriching" the Twitter stream; and results of inter-comparisons. Our project should bring a new kind of visibility to Twitter and engender a new kind of appreciation of the value of Twitter by the science research communities.
validatio↗
Citizen Science Twitter Data Management for Earth Science Applications
Social media data can provide useful real-time and historical information relating to the natural world, but managing this data poses challenges. Scientists at GES DISC are exploring the potential of Twitter data to augment precipitation data from the Global Precipitation Measurement (GPM) mission. However, the format of Twitter data is unconventional in the context of NASA data centers, resulting in frustration for scientists who need to work with the data. This study investigated procedures and standards needed to properly manage Twitter data to make them compatible with these data centers. After comparing databases, the study found that the MongoDB database was best suited for the storage of raw Twitter data due to its flexibility, ability to be accessed by multiple users, and querying functionality. The study used the Python package Zarr to transform processed Twitter data into a gridded format similar to that of satellite data. Each Tweet was mapped onto a time-space grid; each grid location contained information about Tweet attributes and precipitation. The study developed a pipeline for downloading, storing, and gridding Twitter data and transformed Twitter data into an understandable format for users of NASA satellite data.
Li, Rachel↗
Cross-platform analysis of public responses to the 2019 Ridgecrest earthquake sequence on Twitter and Reddit
Online social networks (OSNs) have become a powerful tool to study collective human responses to extreme events such as earthquakes. Most previous research concentrated on a single platform and utilized users’ behaviors on a single platform to study people’s general responses. In this study, we explore the characteristics of people’s behaviors on different OSNs and conduct a cross-platform analysis of public responses to earthquakes. Our findings support the Uses and Gratification theory that users on Reddit and Twitter are engaging with platforms that they may feel best reflect their sense of self. Using the 2019 Ridgecrest earthquakes as our study cases, we collected 510,579 tweets and 45,770 Reddit posts (including 1437 submissions and 44,333 comments) to answer the following research questions: (1) What were the similarities and differences between public responses on Twitter and Reddit? (2) Considering the different mechanisms of Twitter and Reddit, what unique information of public responses can we learn from Reddit as compared with Twitter? By answering these research questions, we aim to bridge the gap of cross-platform public responses research towards natural hazards. Our study evinces that the users on the two different platforms have both different topics of interest and different sentiments towards the same earthquake, which indicates the necessity of investigating cross-platform OSNs to reveal a more comprehensive picture of people’s general public responses towards certain disasters. Our analysis also finds that r/conspiracy subreddit is one of the major venues where people discuss the 2019 Ridgecrest earthquakes on Reddit and different misinformation/conspiracies spread on Twitter and Reddit platforms (e.g., “Big one is coming” on Twitter and “Nuclear test” on Reddit).
58 GEOSCIENCES↗
Feasibility of Adding Twitter Data to Aid Drought Depiction: Case Study in Colorado
The use of social media, such as Twitter, has changed the information landscape for citizens’ participation in crisis response and recovery activities. Given that drought progression is slow and also spatially extensive, an interesting set of questions arise, such as how the usage of Twitter by a large population may change during the development of a major drought alongside how the changing usage facilitates drought detection. For this reason, contemporary analysis of how social media data, in conjunction with meteorological records, was conducted towards improvement in the detection of drought and its progression. The research utilized machine learning techniques applied over satellite-derived drought conditions in Colorado. Three different machine learning techniques were examined: the generalized linear model, support vector machines and deep learning, each applied to test the integration of Twitter data with meteorological records as a predictor of drought development. It is found that the integration of data resources is viable given that the Twitter-based model outperformed the control run which did not include social media input. Eight of the ten models tested showed quantifiable improvements in the performance over the control run model, suggesting that the Twitter-based model was superior in predicting drought severity. Future work lies in expanding this method to depict drought in the western U.S.
54 ENVIRONMENTAL SCIENCES↗
Methodology to Compare Twitter Reaction Trends between Disinformation Communities, to COVID related Campaign Events at Different Geospatial Granularities
With still ongoing COVID pandemic, there is an immediate need for a deeper understanding of how Twitter discussions (or chatters) in disinformation spreading communities get triggered. More specifically, the value is in monitoring how such trigger events in Twitter discussion do align with the timelines of relevant influencing events in the society (indicated in this work as campaign events). For campaign events in regards to COVID pandemic, we consider both NPI (Nonpharmaceutical Interventions) campaigns and disinformation spreading campaigns together. In this short paper we have presented a novel methodology to quantify, compare and relate two Twitter disinformation communities, in terms of their reaction patterns to the timelines of major campaign events. We have also analyzed these campaigns at their three geospatial granularity contexts: local county, state, and country/ federal. We have conducted a novel dataset collection on campaigns (NPI + Disinformation) at these different geospatial granularities. Then, with collected dataset on Twitter disinformation communities, we have performed a case study to validate our proposed methodology.
De, Debraj↗
Enriching the Twitter Stream Increasing Data Mining Yield and Quality Using Machine Learning
Social media data streams are important sources of real-time and historical global information for science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we are exploring the Twitter data stream for its potential in augmenting the validation program of NASA Earth science missions, specifically the Global Precipitation Measurement (GPM) mission. We have implemented a tweet processing infrastructure that outputs classified precipitation tweets. Inputs are "passive" tweets, along with a smaller number of tweets from "active" participants, i.e., those knowingly contributing to our effort. The "active" tweets, presumably of higher quality, enrich the Twitter stream. "Active" sources include data scraped from other social media (e.g., public Facebook posts) and data from existing crowdsourcing programs (e.g., mPING reports). In addition, there is likely relevant precipitation information in images and documents that are the end points of links often included in tweets. Information derived from these "active" sources could then be tweeted into the Twitter stream, thus enriching its quality. The objective of our current work is to mine these tweet linked images and documents, using neural networks, to increase the information content and quality related to precipitation. For images, we classified them as either precipitation-related or not. For training and validation, we used images obtained via the Google custom search API. We created two models: (1) by training a simple Convolutional Neural Network and (2) by using transfer learning principles to adapt a pre-trained object recognition model. For documents, both those linked to tweets and the tweet contents, we trained Hierarchical Attention Networks to determine precipitation occurrence, type, and intensity. For training and validation, we used a keyword-filtered tweet data set labelled with ground truth data from Dark Sky (an API to retrieve weather-related labels) and the National Severe Storms Laboratory's Multi Radar/Multi-Sensor (MRMS) system. Our results demonstrated the efficacy of our machine learning approaches for enriching the Twitter stream, to derive information potentially useful for validation of earth science satellite data.
Albayrak, Arif↗
Multiple social platforms reveal actionable signals for software vulnerability awareness: A study of GitHub, Twitter and Reddit
Software vulnerabilities are flaws in computer systems that leave users open to attack. In many cases, these vulnerabilities go unnoticed and remain unresolved in codebases. Thus, awareness of software vulnerabilities among the public is crucial to ensure effective cybersecurity practices, the development of high quality software, and ultimately national security. This awareness can be better understood by studying the spread and evolution of software vulnerability discussions in online communities. This work is the first to evaluate and contrast how discussions about software vulnerabilities spread on three social platforms -- Twitter, GitHub, and Reddit. To lay the groundwork, we showcase a novel fundamental framework for measuring information spread that identifies the spread mechanisms and observables across platforms, the units of information, and the groups of measurements that can be applied to focus on a specific phenomena e.g., information cascades. We then analyze and contrast social network topologies for three example social networks and measure the scale and speed of the spread of discussion of specific vulnerabilities to understand how far and how widely they spread, how many users participate in discussions, and the duration of their spread. To demonstrate the awareness of more impactful software vulnerabilities, a subset of our analysis focuses on vulnerabilities targeted during recent major cyber attacks as well as vulnerabilities exploited by advanced persistent threat groups. We discover that usually, vulnerability discussions start on GitHub, before occurring on Twitter and Reddit. While studying how some user-level and content-level characteristics influence vulnerability spread, we observe that Twitter discussions started by users predicted to be humans have larger size, breadth, depth, adoption rate, lifetime, and structural virality compared to those started by users predicted to be bots. On Reddit, we contrast the differences in thread structure that originate from posts with positive, negative and neutral polarity. We find that posts that are positive have larger, deeper and wider discussions compared to negative and neutral posts. We anticipate the results of our analysis to not only increase the understanding of software vulnerability awareness but also inform models for simulating information spread across multiple social environments online.
97 MATHEMATICS AND COMPUTING↗
Developing a Machine-Learning-Based Processing Framework for Twitter and Other Crowdsourced Data
Crowdsourced data streams such as Twitter and other social media are important sources of real-time and historical global information for Earth science applications. At the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), we have been exploring the Twitter data stream for its potential in augmenting the validation program of NASA's Global Precipitation Measurement (GPM) mission. To realize this potential, we need to increase the information density and enhance the quality of filtered precipitation tweets. We have implemented various components of a machine learning (ML)-based processing infrastructure for crowdsourced data that outputs, in this instance, useful and usable information derived from precipitation tweets. We have test enriched the Twitter stream with higher quality active tweets from those knowingly contributing to our effort and from existing crowdsourced programs (e.g., mPING, CoCoRaHS). We have experimented with various algorithms for processing tweets, including Naà ve Bayes, Convolutional Neural Network (CNN), Hierarchical Attention Network (HAN), and semi-supervised learning (with tri-training). Our current work focuses on (1) automated review of Earth science-related publications to determine relationships between discipline research needs and ML algorithms; (2) investigating Sequential Generative Adversarial Network (SeqGAN) for processing precipitation tweets for anomaly detection; and (3) managing crowdsourced data in a way that is compatible with existing NASA satellite data archives and using the data for ML applications. Key results include (1) network visualization of NLP-processed publications in various Earth science disciplines; (2) difference between GPM-linked, generated tweets and collected actual tweets that is small for GPM-determined light to moderate rain cases and high for GPM-determined heavy rain cases; and (3) identification of MongoDB for storing raw tweets and Zarr format for gridded tweets (compatible with GPM data). Our results have taken us a step closer to an operational ML-based tweet processing infrastructure and have already demonstrated that tweet-derived precipitation information is potentially useful for validation of Earth science satellite data.
Teng, William↗
Multimodal Event Detection in Twitter Hashtag Networks
In this study, event detection in a multimodal Twitter dataset is considered. We treat the hashtags in the dataset as instances with two modes: text and geolocation features. The text feature consists of a bag-of-words representation. The geolocation feature consists of geotags (i.e., geographical coordinates) of the tweets. Fusing the multimodal data we aim to detect, in terms of topic and geolocation, the interesting events and the associated hashtags. To this end, a generative latent variable model is assumed, and a generalized expectation-maximization (EM) algorithm is derived to learn the model parameters. The proposed method is computationally efficient, and lends itself to big datasets. Lastly, experimental results on a Twitter dataset from August 2014 show the efficacy of the proposed method.
97 MATHEMATICS AND COMPUTING↗
Twitter Geolocation: A Hybrid Approach
Geotagging Twitter messages is an important tool for event detection and enrichment. Despite the availability of both social media content and user network information, these two features are generally utilized separately in the methodology. In this article, we create a hybrid method that uses Twitter content and network information jointly as model features. We use Gaussian mixture models to map the raw spatial distribution of the model features to a predicted field. This approach is scalable to large datasets and provides a natural representation of model confidence. Our method is tested against other approaches and we achieve greater prediction accuracy. Furthermore, the model also improves both precision and coverage.
97 MATHEMATICS AND COMPUTING↗
AI-based Cyber Event OSINT via Twitter Data
Open-Source Intelligence (OSINT) is largely regarded as a necessary component for cybersecurity intelligence gathering to secure network systems. With the advancement of artificial intelligence (AI) and increasing usage of social media, like Twitter, we have a unique opportunity to obtain and aggregate information from social media. In this study, we propose an AI-based scheme capable of automatically pulling information from Twitter, filtering out security-irrelevant tweets, performing natural language analysis to correlate the tweets about each cybersecurity event (e.g., a malware campaign), and validating the information. This scheme has many applications, such as providing a means for security operators to gain insight into ongoing events and helping them prioritize vulnerabilities to deal with. To give examples of the possible uses, we present three case studies demonstrating the event discovery and investigation processes.
Dale, Dakota↗
Zika discourse in the Americas: A multilingual topic analysis of Twitter
Article Authors Metrics Comments Media Coverage Abstract Introduction Materials and methods Results Discussion Acknowledgments References Reader Comments (0) Media Coverage (0) Figures Abstract This work examines Twitter discussion surrounding the 2015 outbreak of Zika, a virus that is most often mild but has been associated with serious birth defects and neurological syndromes. We introduce and analyze a collection of 3.9 million tweets mentioning Zika geolocated to North and South America, where the virus is most prevalent. Using a multilingual topic model, we automatically identify and extract the key topics of discussion across the dataset in English, Spanish, and Portuguese. We examine the variation in Twitter activity across time and location, finding that rises in activity tend to follow to major events, and geographic rates of Zika-related discussion are moderately correlated with Zika incidence (ρ = .398).
60 APPLIED LIFE SCIENCES↗
Open Source Intelligence for Cybersecurity Events via Twitter Data
Open-Source Intelligence (OSINT) is largely regarded as a necessary component for cybersecurity intelligence gathering to secure network systems. With the advancement of artificial intelligence (AI) and increasing usage of social media, like Twitter, we have a unique opportunity to obtain and aggregate information from social media. In this study, we propose an AI-based scheme capable of automatically pulling information from Twitter, filtering out security-irrelevant tweets, performing natural language analysis to correlate the tweets about each cybersecurity event (e.g., a malware campaign), and validating the information. This scheme has many applications, such as providing a means for security operators to gain insight into ongoing events and helping them prioritize vulnerabilities to deal with. To give examples of the possible uses, we present three case studies demonstrating the event discovery and investigation processes. We also examine the potential of OSINT for identifying the network protocols associated with specific events, which can aid in the mitigation procedures by informing operators if the vulnerability is exploitable given their system’s network configurations.
Dale, Dakota↗
Disentangling the Lexicons of Disaster Response in Twitter
Abstract: People around the world use social media platforms such as Twitter heavily to express their opinion about various as- pects of daily life. In the same way social media changes communication in daily life, it also is transforming the way individuals communicate during disasters and emergencies. Emergency officials have come to rely on social media to communicate alerts and updates. How do users communi- cate risk on social media? We used a novel information- theoretic unsupervised learning tool, CorEx, to extract and classify highly relevant words used by the public on Twit- ter during known emergencies, such as fires, explosions, and hurricanes. By utilizing the resulting classification strategy, authorities can use the derived language to craft more rel- evant risk communication to maximize the effectiveness of short-message broadcasts such as the Wireless Emergency Alerts (WEA) service.
clustering↗
Crowdsourcing Dialect Characterization through Twitter
We perform a large-scale analysis of language diatopic variation using geotagged microblogging datasets. By collecting all Twitter messages written in Spanish over more than two years, we build a corpus from which a carefully selected list of concepts allows us to characterize Spanish varieties on a global scale. A cluster analysis proves the existence of well defined macroregions sharing common lexical properties. Remarkably enough, we find that Spanish language is split into two superdialects, namely, an urban speech used across major American and Spanish citites and a diverse form that encompasses rural areas and small towns. The latter can be further clustered into smaller varieties with a stronger regional character.
96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
Identifying Effective Signals to Predict Deleted and Suspended Accounts on Twitter across Languages
Social networks have an ephemerality to them where accounts and messages are constantly being edited, deleted, or marked as private. This continuous change comes from concerns around privacy, a potential desire for deception, and spam-like behavior. In this study we analyze multiple large datasets of thousands of active and deleted Twitter accounts to produce a series of predictive features for the removal or shutdown of an account. We have selected these accounts from speakers of three languages -- Russian, Spanish, and English to evaluate if speakers of various languages behave differently with regards to deleting accounts. We find that unlike previously used profile and network features, the discourse of deleted vs. active accounts forms the basis for highly accurate account deletion prediction. More precisely, we observed that the presence of a certain set of terms in user tweets leads to a higher likelihood for that user's account deletion. We show that the predictive power of profile, language, affect, and network features is not consistent across speakers of the three evaluated languages.
account deletion prediction↗
Identifying Effective Signals to Predict Deleted and Suspended Accounts on Twitter across Languages
Social networks have an ephemerality to them where accounts and messages are constantly being edited, deleted, or marked as private. This continuous change comes from concerns around privacy, a potential desire for deception, and spam-like behavior. In this study we analyze multiple large datasets of thousands of active and deleted Twitter accounts to produce a series of predictive features for the removal or shutdown of an account. We have selected these accounts from speakers of three languages -- Russian, Spanish, and English to evaluate if speakers of various languages behave differently with regards to deleting accounts. We find that unlike previously used profile and network features, the discourse of deleted vs. active accounts forms the basis for highly accurate account deletion prediction. More precisely, we observed that the presence of a certain set of terms in user tweets leads to a higher likelihood for that user's account deletion. We show that the predictive power of profile, language, affect, and network features is not consistent across speakers of the three evaluated languages.
social media, machine learning, natural language p↗