Most companies understand the importance of being able to accurately consolidate, blend, and analyze their data to better understand what is happening in their business and to help decide what to do next. But many business teams have only scratched the surface of what is possible with data and data analytics.
The reasons for this are varied. However, for most companies, it stems from the fact that, despite decades of product-development work from business intelligence (BI) vendors, setting up and running an end-to-end BI process is complex. While a new breed of agile, cloud-based tools on the market offer easier to use, slick-looking visualization tools, the truth is that taking raw data from multiple, disparate systems in the business, blending that data together, and turning it into something that can be visualized and reliably used as the basis for everyday business decisions is not a simple task. That task is made more difficult by the increasing volume and variety of data that is generated today from different business teams using a huge number of different apps to collect data on a seemingly endless number of new business touch points.
The hardest part of the BI process is undoubtedly data preparation – the initial connection, blending, and modeling of data. The complexity of preparing data for analysis is a challenge that has plagued the BI industry since its inception. While some vendors are trying to make data-preparation tools that are simpler to use, the truth remains that consolidating and relating data from different places requires a lot of math. To innovate in this space, BI vendors have to find new ways to separate the complexity of the underlying math from the user experience. Want to learn some Lambda calculus? Me neither.
Another uncomfortable truth about the BI industry is that most data analysis in businesses today still happens in spreadsheets. We see it time and time again when we start talking to companies about their existing processes for business reporting, diagnosis, and data discovery: Even though many businesses have a plethora of BI tools in place, spreadsheets are still rife.
This happens because BI initiatives are typically run by a centralized IT team with technical users. The standard corporate BI tools are purchased, controlled, and operated by IT, which leaves other business teams unable to access their data directly. As a result, business teams must endure long waits to obtain centrally generated reports from IT, with no room for ad-hoc data discovery or experimentation.
Eventually, each individual business team tries to circumvent the IT team by purchasing its own departmental analytics tools, which are typically cloud-based and easier to use. The challenge remains, however, that data preparation is inherently complex and, even with these newer tools, many business teams still struggle to get accurate, timely business answers from data.
Â
