In this section, we will cover the end-to-end workflow of how to fetch tweets using Logstash and visualize tweets with Kibana.
The workflow for real-time Twitter data analysis will be as follows:
- Creating a Twitter developer account to get keys to fetch tweets
- Creating a Logstash configuration file that specifies the input and output
- Fetching tweets using the Logstash Twitter input
- Storing tweets in Elasticsearch using the Logstash Elasticsearch output
- Analyzing Twitter data and creating visualizations in Kibana for scenarios
This analysis can be performed for a wide range of activities, such as marketing, brand building, brand management, customer focus, and so on. In this chapter, we will focus on how a company can use Twitter to build their brand and get unique statistics related to it using Twitter. Also, this would be useful for a company to monitor their marketing campaigns.
At the end of this chapter, you will be able to answer questions such as these:
- How many times has my brand been tweeted about in a time interval?
- Which are the top languages in which people tweet about my brand?
- Which are the different geographical locations from where people are tweeting about my brand?
- From which devices are people mostly tweeting about my brand?
- In which languages, from different devices, are people tweeting about my brand?
- From which countries, using different devices, are people tweeting about my brand?
- What are the top retweeted user screen names related to my brand tweeting from different devices?
- What are the top user screen names tweeting about my brand?
- What are the most popular hashtags related to my brand?
After getting answers to the preceding questions, it will become easier for a company to analyze its brand and get crucial information such as the devices used by its customers, the top countries where its customers talk about its product, the top usernames, the top retweeted usernames, and the most popular hashtags used for its brand.