IDC and EMC estimate that by 2020 the world will have 40 zettabytes of data, and a 2016 Veritas Global Databerg Survey indicates that as much as 85% of this data will be “dark data.”
Gartner defines dark data as “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes (for example, analytics, business relationships and direct monetizing).”
From an organizational standpoint, dark data is the data that may or may not be in systems of record (SOR). It could be paper-based documents in file cabinets, photos, videos, or any number of information artifacts that are simply overlooked and/or neglected in the course of doing business because the initial take on them is that they are not essential.
However, organizations are not throwing this data out, so shouldn’t there be a way to put the data to use?
“If companies can learn how to harness this data, it can yield new insights,” said Mads C. Brink Hansen, product manager at TARGIT, a business intelligence and analytics solution provider. “In one case, a company wanted to assess the efficiency of its field-based salesforce. By looking at the travel expense reports submitted by its salespersons, it was able to determine the number of meetings that each salesperson had while in the field each day and then measure this against what should normally be expected in the way of meetings. This was one way in which an HR-based reporting function (travel and expense reports) was repurposed to provide insights into how many meetings per day an in-field salesperson was likely to have, and who was hitting those targets.”
Hansen says that companies should devote as much time to plumbing the depths of their dark data as they do for their big data—and his company has a toolset that facilitates the process. But for CIOs and others in charge of data analytics, selling dark data projects can be tough. After all, the data has already (by its inactivity) been declared as relatively useless.