I am fortunate enough to be on a voyage of discovery in the world of analytics courtesy of Teradata. When I joined Teradata nearly 18 months ago I had a good grasp of how analytics were used in IT audit and had used these regularly in engagements. I was even fortunate enough to have a data analytics team work with me in one organisation. These analytics proved to be excellent at investigating specific audit topics and applied very well to the usual HR and credit card work. The analytics we did was useful in investigating misuse of corporate networks, I often referred to as “trawling for porn”. However, while the audit tools available in audit were valuable and excellent at ensuring defensible evidence of findings, they were limited to traditional structured data sources, and to some degree so was my thinking.
Now on my journey, I have been fortunate enough to have been able to work alongside talented data scientists who have shown me what the new generation of tools, the power of integrated and/or federated data as well as the potential of open source and unstructured data can do. The Big Data technologies are opening up a new world of opportunity allowing the combination of traditional corporate structure data sets with non-traditional unstructured data.
While auditing I had believed the nirvana for data analytics was to be able to be able to conduct continuous monitoring of data feeds to detect events such as control breaches for investigation. While this is still desirable, I have learnt that this is only the beginning. Continuous monitoring is great at determining when an event occurs, however wouldn’t it be better to be able to predict and avoid an undesirable event where possible rather than taking action to resolve when it already has? This brings the concept of Predictive Analytics; a term Gartner uses in an optimisation path for Analytics.