Previously, only large entities such as governments and larger enterprises, could afford the massive infrastructure necessary for hosting and mining big data. Now as the technology become more powerful and affordable, big data use cases are expanding rapidly in numerous industries. Like many new technologies, big data presents numerous challenges as well as opportunities.
Many organizations are struggling with the question of what to do with the data. Analyzing big data in order to reveal insights that can improve decision making and business operations is a growing challenge. Rather than relying on humans analysts, machine learning and cognitive computing are increasingly being leveraged to make sense of big data.
The Cloud Security Alliance “Big Data Security and Privacy Handbook: 100 Best Practices in Big Data Security and Privacy” states that security vulnerabilities are compounded by the diversity of big data sources and formats, the streaming nature of data acquisition, and the transmission of data between distributed cloud infrastructures. The large volume of big datasets also results in large attack surfaces.
In other words, the very attributes that define big data are the same attributes that contribute to security vulnerabilities: volume, variety, and velocity.
Utility and privacy of data often work in opposition. Leaving data free and open for all certainly can enable all interested parties to access and utilize the data to its greatest advantage. But, of course, this is not an option. Fortunately a reasonable balance between enabling necessary access while restricting unauthorized access is possible.
Securing and encrypting big data is a big challenge. The Gemalto 2015 Breach Level Index showed a continuing failure among many organizations to prevent data breaches and actually protect their information assets of all sizes.
According to The Big Data Security and Privacy Handbook, “Traditional security mechanisms tailored to secure small-scale, static data on firewalled and semi-isolated networks are inadequate.”
Security shouldn’t hurt performance and cause lag times. After all, velocity is one of the characteristics that define big data.
Often big data use cases involve public data such as traffic patterns and residency stats. Anonymizing the data is a common solution. Unfortunately, it’s simply inadequate.
Much like perimeter security is no longer adequate to secure an organization’s IT assets, big data has also outgrown the tactics used at the dawn of its time.