Your Garbage Data Is A Gold Mine

Your Garbage Data Is A Gold Mine

Your Garbage Data Is A Gold Mine

An increasingly diverse array of geospatial, network and time-series data is being used to generate new perspectives and insights about us.

One of the lesser understood aspects of what you can do with massive stockpiles of data is the ability to use data that would traditionally have been overlooked or in some cases even considered rubbish.

This whole new category of data is known as "exhaust" data—data generated as a by-product of some other process.
Much financial market data is a result of two parties agreeing on a price for the sale of an asset. The record of the price of the sale at that instant becomes a form of exhaust data.

Not that long ago, this kind of data wasn’t of much interest, except to economic historians and regulators.
A massive moment-by-moment archive of prices of shares and other securities sales prices is now key to many major banks and hedge funds as a "training ground" for their machine-learning algorithms. Their trading engines "learn" from that history and this learning now powers much of the world’s trading.
Traditional transactions such as house price sales history or share trading archives are one form of time-series data, but many other less conventional measures are being collected and traded too.

Read Also:
Spatial Data Heats Up Business Apps

There are also other categories of unconventional data that are not time-series-based. For example, network data outlines relationships and other signals from social networks, geospatial data lends itself to mapping, and survey data concerns itself with people’s viewpoints. Time series or longitudinal data is, however, the most common form and the easiest to integrate with other time-series data.

Location data from mobile phones means many companies now have people-movement data.[Photo: via The Conversation , Flickr user Andrew Hyde ]
Consistent Longitudinal Unconventional Exhaust Data or CLUE data sets, as I’m calling them, are many, varied and growing. They include:
foot traffic data
technology usage data
employee satisfaction data.

Say, for example, you are interested in the seasonal profitability of supermarkets over time. Foot traffic data may not be the cause of profitability, as more store visitors doesn’t necessarily correlate directly to profit or even sales. But it may be statistically related to volume of sales and so may be one useful clue, just as body temperature is a good clue or one signal to a person’s overall well-being. And when combined with massive amounts of other signals using data analytics techniques, this can provide valuable new insights.

Read Also:
How NASA and the UN are using location intelligence to build smart cities in developing countries

Rise Of "Quantamental" Investment Funds
Leading hedge fund Blackrock, for example, is using satellite images of China taken every five minutes to better understand industrial activity and to give it an independent reading on reported data.

Traditionally, there have been two main types of actors in the financial world—traders (including high-frequency traders ), who look to make money from massive volumes on many small transactions, and investors, who look to make money from a smaller number of larger bets over a longer time. Investors tend to care more about the underlying assets involved. In the case of company stocks, that usually means trying to understand the underlying or fundamental value of the company and future prospects based on its sales, costs, assets and liabilities and so on.

Aerial photography from drones and new low-cost satellites are one key new source of unconventional data.[Photo: Flickr user BxHxTxCx ]
A new type of fund is emerging that combines the speed and computational power of computer-based Quants with the fundamental analysis used by investors.

Read Also:
Six Lessons From Amsterdam’s Smart City Initiative


Data Science Congress 2017

5
Jun
2017
Data Science Congress 2017

20% off with code 7wdata_DSC2017

Read Also:
Turn Data Into Insight Into Action: Six Rules for Nailing Digital Analytics

AI Paris

6
Jun
2017
AI Paris

20% off with code AIP17-7WDATA-20

Read Also:
How NASA and the UN are using location intelligence to build smart cities in developing countries

Chief Data Officer Summit San Francisco

7
Jun
2017
Chief Data Officer Summit San Francisco

$200 off with code DATA200

Read Also:
Use Big Data to Create Value for Customers, Not Just Target Them

Customer Analytics Innovation Summit Chicago

7
Jun
2017
Customer Analytics Innovation Summit Chicago

$200 off with code DATA200

Read Also:
How technology advancements contribute to the democratization of data

Big Data and Analytics Marketing Summit London

12
Jun
2017
Big Data and Analytics Marketing Summit London

$200 off with code DATA200

Read Also:
Data Science Skills and the Improbable Unicorn

Leave a Reply

Your email address will not be published. Required fields are marked *