What's to stop intelligent algorithms, programmed to make a profit, from learning to collude with one another in ways which bend market rules? Such a scenario would require regulatory oversight from the very cutting edge of computer science.
The idea of artificial intelligence manipulating outcomes in the real world and then exploiting these on the markets is bestseller material. But there's fascinating scope for this to actually happen as computing power increases and algorithms get smarter.
Someone who has thought about this a lot is Anthony Amicangioli, CEO and founder of Hyannis Port Research. His company developed Riskbot which has been described as "a supercomputer that watches supercomputers"; a literal box that sits between the trading firm and the exchange to prevent erroneous trades going through to market.
Amicangioli explains that the detection of excessive spoofing (placing orders to create the appearance of interest that the market is going to move in one direction or another) using artificial intelligence (AI) could potentially have some bewildering consequences.
He told IBTimes: "There are a few behaviours that are really interesting to consider. With machine learning (ML), you write code that acts on data. I distinguish advanced ML (from things like big data) where one begins to treat code in a manner similar to data; creating code that can change itself through intelligent morphing.
"Detecting spoofing is relatively easy. I'm looking for someone who sends a bunch of deceptive orders on one side of a given stock's book, abruptly pulls back, and benefits from trading the opposite side of the book shortly thereafter.
"But what about the case when an AI algorithm modifies itself so that rather than benefit for itself, maybe it gets even smarter than that and begins to collude with another trading entity with a similar algorithm.
"So I spoof but you benefit - we sort of have this unholy implicit contract to do the same in reverse. If that sort of hypothetical scenario is possible, then a lot of fascinating regulatory questions arise."
The regulator would have to adopt a scientifically rigorous approach to spot this kind of activity, notes Amicangioli. And then how would they deal with it?
"Is it possible that AI could be written where the author of said code is not intending to spoof or layer but an algorithm somehow inadvertently does? Or maybe even more complexly, begins to collude with other algorithms."
Amicangioli said the probability going forward that there could be unintended AI outcomes is high. From a regulatory perspective, the net behaviour as analysed, could be deemed to be bad, but the author could be deemed to be completely blameless.
Much was made of the fact that market regulators were eons behind the complexity that precipitated the 2010 flash crash. So what can regulators do to evolve in a ML markets environment?
"I don't believe the government or regulators could ever just hire teams of scientists and compete. These hypothetical problems I have outlined may be around the corner; I don't think they exist today; I could be wrong.