Security data science is booming—reports indicate that the security analytics market is set to reach $8 billion dollars by 2023, with a growth rate of 26%, thanks to relentless cyber attacks. If you want to stay ahead of emerging security threats in 2017, it is important to invest in the right areas. In March 2016, I wrote a piece on the 4 trends to be aware of for 2016; for my 2017 trends post, Cody Rioux from Netflix joins me, bringing his platform perspective. Our goal is to help you formulate a plan for every quarter of 2017 (i.e., 4 trends for 4 quarters). For each of our trends, we provide a short rationale, why we think the time is right for investing, and how to capitalize on the investment, with pointers to specific tools and resources.
We believe the security industry is going to see an uptick in automated and autonomous responses in the form of chat bots that will provide information when a model deems the information relevant, as well as on demand responses. The responses will likely be integrated into the platform you’re currently using to communicate with teammates during incident response. This isn’t a new idea—chatbots have existed at least as long as internet relay chat (IRC), but they’ve seen a big uptick in popularity thanks to “ChatOps.” Shivon Zilis and James Cham refer to this as “the great chatbot explosion of 2016,” and their infographic lists 15 companies developing autonomous agents as of today.
Chris Messina(@chrismessina) , inventor of hashtag, recently penned that Chat Bots Aren’t a Fad. They’re a Revolution. Tech organizations are generally in a place where there is trust for autonomous systems within the production environment, and this opens the door for automating all types of menial tasks, including those in the security domain. Bot frameworks are prime for deployment for a wide array of communication platforms, including Slack, IRC, and Skype. You’re likely already using such a platform for communication both during security incidents and your day-to-day work, which makes a bot the ideal companion for both executing tasks quickly mid-incident, and performing and reporting on routine checks, such as rolling certificates and ensuring security standards compliance. Jason Chan (@chanjbs) also recently spoke about how Netflix uses bots in the context of security—from security consultation, to approving deployment changes, to having noticeable security keywords.
Threat intelligence (TI) feeds can be thought of as discrete instances of known bad actors—or rather, a collection of indicators of compromise. They can vary from hashes of known malicious files used by adversaries, IP addresses of the command and control servers of botnets, or even user agent strings used by persistent threats. Threat intelligence feeds have long been used by the security community as point-in-time checks for security monitoring, but we argue that the data science community should leverage them with the behavioral detection systems in 2017.
The Bayes error rate is the fundamental limit of any classifier with a given data set. The standard way to improve the error rate is to include new sources of information. We posit that TI feeds are an easy gateway, and a first step toward including new data sources.
Additionally, there’s surrogate interpretability—they also provide insight into explaining your alerts. For instance, if your ML system determines that the login is anomalous and the IP address of the login is present in a botnet feed, then we can surmise that the login is anomalous because it stems from a machine infected by botnet. Although hacky, and not a sure guarantee, this can provide a quick win for explaining alerts.
Before you start experimenting with TI feeds, keep in mind that the feeds have varying levels of confidence in their indicators and, thus, require some trial and error. Commercial TI vendors include Team Cymru, iSight, iDefense, and Webroot. Open source TI feeds include Project Honeypot.