Malicious hacking today largely consists of exploiting weaknesses in an applications stack, to gain access to private data that shouldn’t be public or corrupt/interfere with the operations of a given application. Sometimes this is to expose software weaknesses, other times this is done for hackers to generate income by trading private information which is of value.
Software vendors are now more focused on baking in security concepts into their code, rather than thinking of security as being an operational afterthought. Although breaches still happen. In fact, data science is being used in a positive way in the areas of intrusion, virus and malware detection to move use from reactive response to a more proactive and predictive approach to detecting breaches.
However, as we move forward into an era where aspects of human decision making are being replaced with data science combined with automation, I think it is of immense importance that we have the security aspects of this front of mind from the get go. Otherwise we are at risk of again falling into the trap of considering security as an afterthought. And to do this we really need to consider what aspects of data science open themselves up to security risk.
One key area that immediately springs to mind is “gaming the system” specifically in relation to machine learning. For example, banks may automate the approval of small bank loans and use machine learning prediction to determine if an applicant has the ability to service the loan and presents a suitable risk. The processing and approval of the loan may be performed in real-time without human involvement, and funds may immediately available to the applicant on approval.