Birdwatcher: Data analysis and OSINT framework for Twitter

Birdwatcher: Data analysis and OSINT framework for Twitter

Birdwatcher: Data analysis and OSINT framework for Twitter
Yes, here again with another tool release. This time it’s an OSINT framework for a different social network — Twitter.com.

Birdwatcher started out as a collection of small scripts to generate a classic weighted word cloud of Tweets from a group of users. As I thought about what else I could do with data from Twitter I decided to rewrite the scripts into a full-fledged, module based, console framework with a ton more functionality.

If you have any experience working with other frameworks such as Metasploit or Recon-ng, you will feel right at home with Birdwatcher as it’s heavily inspired by these frameworks and has many of the same concepts and commands.

This blog post won’t go over how to set up Birdwatcher, but you can have a look at the README to find out how to install and configure the framework.

Just like Metasploit and Recon-ng, Birdwatcher supports the concept of Workspaces. Workspaces enable you to segment and manage users and data stored in the underlying database. You can use workspaces to create logical separation between different users. For example, you may want to create a workspace for a company, a department or for a specific topic.

Read Also:
What Spark's Structured Streaming really means

The command prompt will always show the currently active workspace inside the square brackets. Birdwatcher will always have a default workspace which might be all you need if you intend to use Birdwatcher on a single group of users. If you plan to use it on several different groups, it is recommended to create a workspace for each of them, to prevent cross contamination.

The core of the Birdwatcher framework is its commands and one of the most important ones is the command:

The command simply lists all available commands with short descriptions of what they do.

Again, just like Metasploit and Recon-ng, Birdwatcher ships with a bunch of modules that either enrich the raw Twitter data harvested by the commands or somehow present the data in interesting and useful ways. Here are some of the things the modules can currently do:

Birdwatcher’s code is designed to make it pretty simple for anyone with a bit of Ruby knowledge to extend Birdwatcher with new modules. How to create one is out of scope for this blog post, but have a look at this Wiki article if you are interested in finding out more.

Read Also:
Big Data vs. Little Data

If you have been following the news around the Snowden documents, you might have heard of a program by the UK intelligence agency GCHQ called LOVELY HORSE. The program was made to simply monitor a smaller group of security related Twitter accounts to keep taps on what was being said and possibly more.

To demonstrate the capabilities and usage of Birdwatcher, I thought it would be fun to go through how we can create our own LOVELY HORSE program…

Instead of using the default workspace, let’s create a dedicated one for our lovely horses to keep things neat and tidy:

The command created our new workspace and automatically made it the currently active one, as can be seen in the square brackets of the command prompt.

Now that we have our workspace we need to add some users to it so we have something to work with. The leaked PDF contains a list of 37 Twitter accounts that we will use for this example:

Read Also:
Big Data's Hidden Scourge: Data Drift

One way to add the users would be to execute but that would be a lot of typing and I don’t really like that. Instead we can make use of our first module to easily import them into the workspace. We copy the usernames and save them to a file and load the User Importer module:

The command loads a module by its path. The path is determined simply by how the module files are placed in the directory stucture. Modules live inside at least one directory which can be seen as a namespace of the type of object they are working on. In this case the User Importer lives in the namespace which makes pretty good sense. When a module is loaded it is also indicated in the command prompt with another set of square brackets with the module’s path in red text.

Read Full Story…

 

Leave a Reply

Your email address will not be published. Required fields are marked *