Birdwatcher: Data analysis and OSINT framework for Twitter

Birdwatcher: Data analysis and OSINT framework for Twitter

Yes, here again with another tool release. This time it's an OSINT framework for a different social network — Twitter.com.

Birdwatcher started out as a collection of small scripts to generate a classic weighted word cloud of Tweets from a group of users. As I thought about what else I could do with data from Twitter I decided to rewrite the scripts into a full-fledged, module based, console framework with a ton more functionality.

If you have any experience working with other frameworks such as Metasploit or Recon-ng, you will feel right at home with Birdwatcher as it's heavily inspired by these frameworks and has many of the same concepts and commands.

This blog post won't go over how to set up Birdwatcher, but you can have a look at the README to find out how to install and configure the framework.

Just like Metasploit and Recon-ng, Birdwatcher supports the concept of Workspaces. Workspaces enable you to segment and manage users and data stored in the underlying database. You can use workspaces to create logical separation between different users. For example, you may want to create a workspace for a company, a department or for a specific topic.

The command prompt will always show the currently active workspace inside the square brackets. Birdwatcher will always have a default workspace which might be all you need if you intend to use Birdwatcher on a single group of users. If you plan to use it on several different groups, it is recommended to create a workspace for each of them, to prevent cross contamination.

The core of the Birdwatcher framework is its commands and one of the most important ones is the command:

The command simply lists all available commands with short descriptions of what they do.

Again, just like Metasploit and Recon-ng, Birdwatcher ships with a bunch of modules that either enrich the raw Twitter data harvested by the commands or somehow present the data in interesting and useful ways. Here are some of the things the modules can currently do:

Birdwatcher's code is designed to make it pretty simple for anyone with a bit of Ruby knowledge to extend Birdwatcher with new modules. How to create one is out of scope for this blog post, but have a look at this Wiki article if you are interested in finding out more.

If you have been following the news around the Snowden documents, you might have heard of a program by the UK intelligence agency GCHQ called LOVELY HORSE. The program was made to simply monitor a smaller group of security related Twitter accounts to keep taps on what was being said and possibly more.

To demonstrate the capabilities and usage of Birdwatcher, I thought it would be fun to go through how we can create our own LOVELY HORSE program...

Instead of using the default workspace, let's create a dedicated one for our lovely horses to keep things neat and tidy:

The command created our new workspace and automatically made it the currently active one, as can be seen in the square brackets of the command prompt.

Now that we have our workspace we need to add some users to it so we have something to work with. The leaked PDF contains a list of 37 Twitter accounts that we will use for this example:

One way to add the users would be to execute but that would be a lot of typing and I don't really like that. Instead we can make use of our first module to easily import them into the workspace. We copy the usernames and save them to a file and load the User Importer module:

The command loads a module by its path. The path is determined simply by how the module files are placed in the directory stucture. Modules live inside at least one directory which can be seen as a namespace of the type of object they are working on. In this case the User Importer lives in the namespace which makes pretty good sense. When a module is loaded it is also indicated in the command prompt with another set of square brackets with the module's path in red text.

 

Share it:
Share it:

[Social9_Share class=”s9-widget-wrapper”]

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

You Might Be Interested In

Human error in data analytics, and how to fix it using artificial intelligence

27 Jan, 2022

The benefits of analytics are well-documented. Analytics has helped organisations transform retail experiences, map pathways for trains and trucks, discover …

Read more

Big Data, IoT And Blockchain: Benefits Of Merging The Trending Trio

4 Oct, 2018

As the advancement in the technologies across the globe have catch the lightning speed, technologies like big data, IoT and …

Read more

The importance of monitoring machine learning models

6 Oct, 2022

Agile development teams must ensure that microservices, applications, and databases are observable, have monitoring in place to identify operational issues, …

Read more

Do You Want to Share Your Story?

Bring your insights on Data, Visualization, Innovation or Business Agility to our community. Let them learn from your experience.

Get the 3 STEPS

To Drive Analytics Adoption
And manage change

3-steps-to-drive-analytics-adoption

Get Access to Event Discounts

Switch your 7wData account from Subscriber to Event Discount Member by clicking the button below and get access to event discounts. Learn & Grow together with us in a more profitable way!

Get Access to Event Discounts

Create a 7wData account and get access to event discounts. Learn & Grow together with us in a more profitable way!

Don't miss Out!

Stay in touch and receive in depth articles, guides, news & commentary of all things data.