Big data may be exploding all around us, but our brains can only handle so much information at a time. As this divergence continues to grow, it raises the bar on the tools we depend on to present us with the insights we need to get our jobs done.
Both business and technical people face this insight challenge. Business users require visibility into the key performance indicators (KPIs) that both indicate the health of their business as well as problems they must address.
Business intelligence tools of years past generated reports, which met the needs of business users at the time. Today, however, both business and technical users require more insight than reports can deliver, and they need it in real-time.
Dashboards solve some of these problems. They can display real-time information if it’s available, to be sure. But typically, an analyst must set up the desired dashboards ahead of time – both filtering the available information and limiting the ability of users to drill down into specific details depending upon the need at the time.
Furthermore, for operations personnel, dashboards account for only a small subset of the visualizations they work with on a day-to-day basis. Because dashboards display limited information, the real work of ops goes well beyond the scope of such displays – often leaving personnel with blind spots as the quantity of information continues to grow.
For all types of users, in fact, the explosion of available data drives the requirement for new types of visualizations – as well as increasingly powerful abilities to drill down into vast firehoses of data to discern the important insights within such torrents, whether they be business KPIs or critical anomalies that indicate urgent operational problems.
When we say ‘drill down’ into large quantities of data, we mean that we want to uncover just the information we require to gain the necessary insight.
For example, a dashboard may indicate a problem with ecommerce conversions (aka ‘purchases’) at a certain point in time. If we drill down, we might find that the real problem is with conversions using a particular kind of credit card for purchases taking place in a particular state, with errors occurring on servers at a particular content delivery network (CDN).
When we’re dealing with vast quantities of real-time data, it becomes increasingly difficult to find such nuggets of insight, given the vast number of permutations of the data that may be in place.
How would we ever have known to look for problems with one credit card type in one state on one CDN? No traditional dashboard would ever point us in the right direction.