Birdwatcher: Data analysis and OSINT framework for Twitter

Birdwatcher: Data analysis and OSINT framework for Twitter

Birdwatcher: Data analysis and OSINT framework for Twitter

Yes, here again with another tool release. This time it's an OSINT framework for a different social network — Twitter.com.

Birdwatcher started out as a collection of small scripts to generate a classic weighted word cloud of Tweets from a group of users. As I thought about what else I could do with data from Twitter I decided to rewrite the scripts into a full-fledged, module based, console framework with a ton more functionality.

If you have any experience working with other frameworks such as Metasploit or Recon-ng, you will feel right at home with Birdwatcher as it's heavily inspired by these frameworks and has many of the same concepts and commands.

This blog post won't go over how to set up Birdwatcher, but you can have a look at the README to find out how to install and configure the framework.

Just like Metasploit and Recon-ng, Birdwatcher supports the concept of Workspaces. Workspaces enable you to segment and manage users and data stored in the underlying database. You can use workspaces to create logical separation between different users. For example, you may want to create a workspace for a company, a department or for a specific topic.

The command prompt will always show the currently active workspace inside the square brackets. Birdwatcher will always have a default workspace which might be all you need if you intend to use Birdwatcher on a single group of users. If you plan to use it on several different groups, it is recommended to create a workspace for each of them, to prevent cross contamination.

Read Also:
Visualization Can Help Businesses Tame the Big Data Jungle

The core of the Birdwatcher framework is its commands and one of the most important ones is the command:

The command simply lists all available commands with short descriptions of what they do.

Again, just like Metasploit and Recon-ng, Birdwatcher ships with a bunch of modules that either enrich the raw Twitter data harvested by the commands or somehow present the data in interesting and useful ways. Here are some of the things the modules can currently do:

Birdwatcher's code is designed to make it pretty simple for anyone with a bit of Ruby knowledge to extend Birdwatcher with new modules. How to create one is out of scope for this blog post, but have a look at this Wiki article if you are interested in finding out more.

If you have been following the news around the Snowden documents, you might have heard of a program by the UK intelligence agency GCHQ called LOVELY HORSE. The program was made to simply monitor a smaller group of security related Twitter accounts to keep taps on what was being said and possibly more.

Read Also:
Are You Still in the Dark About the Quality of Your Data?

To demonstrate the capabilities and usage of Birdwatcher, I thought it would be fun to go through how we can create our own LOVELY HORSE program...

Instead of using the default workspace, let's create a dedicated one for our lovely horses to keep things neat and tidy:

The command created our new workspace and automatically made it the currently active one, as can be seen in the square brackets of the command prompt.

Now that we have our workspace we need to add some users to it so we have something to work with. The leaked PDF contains a list of 37 Twitter accounts that we will use for this example:

One way to add the users would be to execute but that would be a lot of typing and I don't really like that. Instead we can make use of our first module to easily import them into the workspace. We copy the usernames and save them to a file and load the User Importer module:

The command loads a module by its path. The path is determined simply by how the module files are placed in the directory stucture. Modules live inside at least one directory which can be seen as a namespace of the type of object they are working on. In this case the User Importer lives in the namespace which makes pretty good sense. When a module is loaded it is also indicated in the command prompt with another set of square brackets with the module's path in red text.

Read Also:
Can IT keep up with big data?

 



Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
The CEO of £1.4 billion software giant Xero says AI will be 'transformational' for finance

Chief Analytics Officer Spring 2017

2
May
2017
Chief Analytics Officer Spring 2017

15% off with code MP15

Read Also:
Visualization Can Help Businesses Tame the Big Data Jungle

Big Data and Analytics for Healthcare Philadelphia

17
May
2017
Big Data and Analytics for Healthcare Philadelphia

$200 off with code DATA200

Read Also:
Sisense wants to make every user a data scientist

SMX London

23
May
2017
SMX London

10% off with code 7WDATASMX

Read Also:
Airline Algorithms – To Delay or Not Delay? Big Data has the Answer

Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
The hidden value in imperfect big data

Leave a Reply

Your email address will not be published. Required fields are marked *