Learn how to visualize meaningful relationships between Twitter hashtags with Graph, a new Elastic X-Pack plugin for Elasticsearch, and Kibana.

<html><head></head><body>In this blog, I will discuss how I&apos;ve used <a href="https://www.elastic.co/products/graph">Graph</a>, a new Elastic X-Pack plugin for&#xA0;Elasticsearch and Kibana, to explore relationships in Twitter data based on Donald Trump&apos;s political campaign.A great source of free data to play with in Graph is Twitter and Logstash provides a Twitter input stream as standard.&#xA0;The Logstash Twitter input constantly watches the feed for a specific search term and in my example, I used the word &#x201C;Trump&#x201D;. Leaving Logstash running on my laptop for just an hour gives me a collection of thousands of tweets for Graph to analyse.&#xA0;A useful insight you might want to measure on Twitter is what hashtags are popular and which ones are frequently used together.&#xA0;As a business, gaining this insight can help maximise your social media presence. Looking at frequently used hashtags is trivial with Kibana, but what about hashtags frequently used together? What are people using these hashtags talking about? This is where Graph comes in. Using the Kibana Graph plugin to search for &#x201C;#Trump&#x201D;, instantly it produces a graph of interconnected hashtags. <img src="https://api.contentstack.io/v2/assets/575e4d66d8edd48f769382b9/download?uid=blt7f5d26dba62777b6?uid=blt7f5d26dba62777b6" data-sys-asset-uid="blt82e3ef3298cd2db9" alt="Trump-hastags-graph-dashboard.png" style="max-width:100%;">
Straight away Graph shows us popular hashtags associated with the Trump presidential campaign and links those which are frequently seen together. In contrast searching the hashtag &#x201C;#Cruz&#x201D; for the dataset gives a completely different impression. 
<img src="https://api.contentstack.io/v2/assets/575e4d660342dfd738265731/download?uid=bltde76d72cd6c4c2cd?uid=bltde76d72cd6c4c2cd" data-sys-asset-uid="bltcbe0b72e30e74570" alt="Cruz-hashtags-graph-dashboard.png" style="max-width:100%;">
Using the advanced features of Graph, not only can we assess links between hashtags and other hashtags but also links between hashtags and other data fields. For my case, a good example of this is to categorise America by time-zone, splitting our sample into three groups we can see what hashtags are most commonly associated with each of these time-zones.
<img src="https://api.contentstack.io/v2/assets/575e4d669e7a83165490e4e4/download?uid=blt526595be37ac3220?uid=blt526595be37ac3220" data-sys-asset-uid="blt8d58eb67f101bfbe" alt="timezones-graph-dashboard.png" style="max-width:100%;">
The data fields provided by Logstash have a vast potential for gaining insight on topics that may be very difficult to understand otherwise, like I have studied hashtags here we can also look at the raw content of the tweets and common associated words, as well as the words Twitter users use to describe themselves!
<img src="https://api.contentstack.io/v2/assets/575e4d660342dfd73826573e/download?uid=bltf0818e116a198bb1?uid=bltf0818e116a198bb1" data-sys-asset-uid="blt8bbbd68ddbd473b0" alt="political-views-graph-dashboard.png" style="max-width:100%;">
However, one place where the standard Logstash Twitter feed falls short is Twitter followers. What if I want to know what users are popular in a subject and who people who follow them also tend to follow? Fortunately, Logstash can read in pretty much any file, so it&#x2019;s pretty easy to use the Twitter API to bring down the data we&#x2019;re interested in and pass it to Logstash.
<img src="https://api.contentstack.io/v2/assets/575e4d6643e9adc538715f5b/download?uid=blt78ca41689f6a670b?uid=blt78ca41689f6a670b" data-sys-asset-uid="blt01fd0f29e529a94a" alt="twitter-followers-graph-dashboard.png" style="max-width:100%;">
Straight away, we can build up a graph showing common following trends, the thickness of the lines shows the strength of the links. Clearly Graph is a powerful tool for identifying key, associated and similar users quickly and easily.
A subtle issue that can come about in graph analysis is results being dominated by repeated data from one source. For example, if one user tweets one hundred times more than another then their description will be weighted one hundred times stronger, such as when I search for the term &#x201C;republican&#x201D; in user descriptions:
<img src="https://api.contentstack.io/v2/assets/575e4d663dc542cb38c0a7cc/download?uid=bltb00deca545b6e0d4?uid=bltb00deca545b6e0d4" data-sys-asset-uid="blt5b74c279dd38fb9f" alt="republican-significant-graph-dashboard.PNG" style="max-width:100%;">
Here at first glance the results seem good, but a couple of results seem a bit strange to see, are all Republicans cowgirls? I think it is more likely there is one Republican cowgirl who tweets very frequently. Fortunately, Graph has a solution to this issue, using the diversity field setting; one can limit the consideration given to content with the same value of a certain field. For our case it makes sense to limit this to one tweet per user ID. Repeating the analysis with this new setting gives a very different picture:
<img src="https://api.contentstack.io/v2/assets/575e4d669985d58976378ea3/download?uid=blt34294951ab08a430?uid=blt34294951ab08a430" data-sys-asset-uid="bltdf2604f6fdb919a6" alt="republican-diversity-graph-dashboard.PNG" style="max-width:100%;">
Another one of the key advantages of the Graph algorithm is its ability to filter out popular &#x201C;super-connected&#x201D; terms and identify only &#x201C;significant links&#x201D;. This feature can be disabled if needed. For example, repeating the search for &#x201C;republican&#x201D; without significant links, graph pulls back loads of connecting words, &#x201C;and&#x201D;, &#x201C;of&#x201D;, &#x201C;the&#x201D; etc. these words do appear frequently with the world &#x201C;republican&#x201D;, however they also appear frequently in descriptions without the word &#x201C;republican&#x201D; and are hence not reflective of descriptions featuring the word.<img src="https://api.contentstack.io/v2/assets/575e4d6743e9adc538715f68/download?uid=blt869f5ac94b080fca?uid=blt869f5ac94b080fca" data-sys-asset-uid="blt8478e24de87c48db" alt="republican-diversity-non-significant-graph dashboard.png" style="max-width:100%;">
All the relationships displayed in the Kibana app are as a result of querying the Elasticsearch Graph API. The app allows you to view the raw JSON, making it easy to integrate Graph into a custom application.To get an even better idea of the power of Graph watch&#xA0;<a href="https://www.elastic.co/webinars/sneak-peek-of-graph-capabilities-with-elasticsearch" target="_blank">this webinar</a>&#xA0;on the Elastic site, this video demonstrates the features of Graph as applied to other datasets such as Wikipedia and LastFM data.This blog appeared last month on our partner&#xA0;<a href="http://www.aiimi.com/gain-powerful-relationship-insights-elastic-graph/" target="_blank">Aiimi&#x2019;s blog</a>. <hr><img src="https://api.contentstack.io/v2/assets/575e4d669985d58976378e9f/download?uid=blt751272a4082ecf57?uid=blt751272a4082ecf57" data-sys-asset-uid="bltd9c264f9d77792e9" alt="Jack_headshot.jpg" width="146" height="150" style="line-height: 1.6em; float: left; width: 146px; height: 150px; background-color: initial;">
Jack Lawton is a Technical Consultant at Aiimi, an Elastic partner specialising in information management consultancy. After studying Physics at Manchester, Jack is now exploring cutting edge data science technology to evolve Aiimi&#x2019;s services and meet rising demands.
</body></html>

donald-trump-ted-cruz-elasticsearch-graph.jpg

Trump-hastags-graph-dashboard.png

Using Elastic Graph to Analyze #Trump Data

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

Jack Lawton

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_self">Terms of Service</a>, and you agree to receive the <a href="/legal/privacy-statement#how-we-use-the-information" target="_self">selected communications</a> at the contact details you provided above. See <a href="/legal/privacy-statement/" target="_self">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Communication Preferences - ONLY for /communication-preferences

送信することにより、Elasticの<a href="/jp/legal/elastic-cloud-account-terms" target="_self">利用規約</a>を読み、同意したものと見なされます。また、上記で提供された連絡先に<a href="/jp/legal/privacy-statement#how-we-use-the-information" target="_self">特定の通知</a>が送信されることを認めたことになります。詳細については、<a href="/jp/legal/privacy-statement/" target="_self">Elasticのプライバシーポリシー</a>をご覧ください。これらの同意についてはいつでもオプトアウトすることが可能です。

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.By submitting this form, I agree to Elastic sharing my contact information with Google Cloud. I also agree to Elastic giving my information to the vendor for the purpose of shipping my food package.

GDPR Text + Google Cloud

送信することにより、Elasticの<a href="/jp/legal/elastic-cloud-account-terms" target="_self">利用規約</a>を読み、同意したものと見なされます。また、関連する製品やサービスについて、上記で提供された情報を使用してElasticから<a href="/jp/legal/privacy-statement#how-we-use-the-information" target="_self">連絡する</a>場合があることを認めたことになります。詳細については、<a href="/jp/legal/privacy-statement/" target="_self">Elasticのプライバシー</a>ポリシーをご覧ください。これらの同意についてはいつでもオプトアウトすることが可能です。このフォームを送信することにより、Elasticが私の連絡先情報をGoogle Cloudに共有することに同意します。また、Elasticが私に食材パッケージを発送する目的で私の情報をベンダーに提供することにも同意します。

By submitting you acknowledge that you've read and agree to our <a href="/legal/serverless-preview-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Serverless GDPR Text

送信することにより、Elasticの<a href="/jp/legal/serverless-preview-terms" target="_self">利用規約</a>を読み、同意したものと見なされます。また、関連する製品やサービスについて、上記で提供された情報を使用してElasticから<a href="/jp/legal/privacy-statement#how-we-use-the-information" target="_self">連絡する</a>場合があることを認めたことになります。詳細については、<a href="/jp/legal/privacy-statement/" target="_self">Elasticのプライバシー</a>ポリシーをご覧ください。これらの同意についてはいつでもオプトアウトすることが可能です。

By submitting you agree to <a href="/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud Terms of Service</a>. Your personal data will be processed in accordance with <a href="/legal/privacy-statement">Elastic's Privacy Statement</a>.

Cloud Trial GDPR Text

送信することにより、<a href="/jp/agreements/global/cloud-services-monthly" rel="nofollow">Elastic Cloud利用規約</a>に同意したものとみなされます。個人情報は<a href="/jp/legal/privacy-statement">Elasticのプライバシーポリシー</a>に基づいて取り扱われます。

By submitting you acknowledge that you've read and agree to our <a href="/legal/terms-of-use" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

送信することにより、Elasticの<a href="/jp/legal/terms-of-use" target="_self">利用規約</a>を読み、同意したものと見なされます。また、関連する製品やサービスについて、上記で提供された情報を使用してElasticから<a href="/jp/legal/privacy-statement#how-we-use-the-information" target="_self">連絡する</a>場合があることを認めたことになります。詳細については、<a href="/jp/legal/privacy-statement/" target="_self">Elasticのプライバシー</a>ポリシーをご覧ください。これらの同意についてはいつでもオプトアウトすることが可能です。

Explore similar demos

Overview

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

このページのコンテンツは、選択された言語ではご利用いただけません。Elasticでは現在、コンテンツをさまざまな言語でご利用いただけるよう取り組んでいます。お使いの言語が準備されるまでお待ちくださるようお願いいたします。

Author

Articles by

Watch now (no PT)

Watch now

See all top stories

すべてのトップストーリーを見る

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Search Integrations

All Solutions

Video for

See when this webinar starts in my time zone

現在のタイムゾーンでこのウェビナー開始時間を見る

Register to Watch

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

他の予定と重なってしまった…という場合も、問題ありません。参加登録されると、録画配信をご利用いただけます。関連コンテンツのご案内もメールでお届けします。

Author

Articles by Jack Lawton

Using Elastic Graph to Analyze #Trump Data

Sign up for Elastic Cloud free trial