The term “Big Data” and the relatively broad spectrum of tools, capabilities, and technologies that are associated with it have created sufficient hype amongst executive councils and board rooms across the nation. The relative simplicity of the term belies the complexity of what it entails, which is the promise of harnessing all of the data your organization possesses internally, or has access to through external channels. This data plays an instrumental role in driving business value and enabling the digital transformation of your enterprise.
If only it were that simple. The precept indeed begins simply enough, as all the analog processes and workflows of the world begin to move to digital realm. Businesses should be able to take the data being created, with its ever increasing volume, variety, and velocity, and leverage new high performance technology to extract the business value and lead to improved “data-driven” decision making. In a few specific and often touted instances, particularly in companies that were digital from the start (i.e. Google, Amazon, Netflix), this has been largely successful. This requires massive investments in these enablement technologies and relatively limited human capital with expertise in leveraging them.
There is little doubt that incredible value is buried in those mountains of data, and technology and enablement vendors are quick to promise the relative simplicity of leveraging their mining equipment and services to get to all the gold within. There's also the promise that these new tools are cheaper, faster, and better than the mining equipment most organizations have been using for the past twenty plus years to refine their data from raw materials to precious decision-driving value. The challenge is that while the tools and hardware themselves may (but not always) be cheaper, you also need people with the expertise to leverage them, while at the same time keep up with the existing demands of your business. For most organizations this juggling act proves to be quite difficult, with some likening it to changing the tires while your car is in motion. It’s actually a bit more like adding a rocket booster to your car along with the tires you already have—you still need your existing mechanics to keep the car running, but now also need rocket scientists (or, in this case, data scientists) to be able to take full advantage of your new capabilities.
The precept indeed begins simply enough, as all the analog processes and workflows of the world begin to move to digital realm.
Equipped with the information, to quote Mark Twain, “there’s gold in them thar hills,” how can organizations properly work towards realizing the promise of Big Data? The solution seems to boil down to addressing several elements critical to success, some of which are technology related. Perhaps, the most important aspect is where a majority of data and technology initiatives stagnate (or worse, die an expensive and painful death)—corporate culture.
