IBM’s debating AI just got a lot closer to being a useful tool
Computers have guided us to the moon and back but can’t help us with us with the biggest decisions we face today. Should Donald Trump be impeached? Should Britain leave the EU? Should Australia stop exporting fossil fuels? Questions like these do not have yes or no answers, however tempting it is to think otherwise.
We make decisions by weighing pros and cons. Artificial intelligence has the potential to help us with that by sifting through ever-increasing mounds of data. But to be truly useful, it needs to reason more like a human. “We make use of persuasive language and all sorts of background knowledge that is very difficult to model in AI,” says Jacky Visser of the Center for Argument Technology at the University of Dundee, UK. “This has been one of the holy grails since people started thinking about AI.”
A core technique used to help machines reason, known as argument mining, involves building software to analyze written documents and extract key sentences that provide evidence for or against a given claim. These can then be assembled into an argument. As well as helping us make better decisions, such tools could be used to catch fake news—undermining dodgy claims and backing up factual ones—or to filter online search results, returning relevant statements rather than whole documents.
Other groups’ work on argument mining has focused on specific types of texts, such as legal documents or student essays, which tend to contain a lot of structured argument to start with. That’s useful if you want a summary of all the evidence across lots of different documents in a legal case, for example. But the ultimate goal is to build a system that can trawl through as many sources of information as possible and build an argument using every bit of evidence it can find.
IBM has just taken a big step in that direction. The company’s Project Debater team has been developing an AI that can build arguments for several years. Last year IBM demonstrated its work-in-progress technology in a live debate against a world-champion human debater, the equivalent of Watson’s Jeopardy! showdown. Such stunts are fun, and it provided a proof of concept. Now IBM is turning its toy into a genuinely useful tool.
The version of Project Debater used in the live debates included the seeds of the latest system, such as the capability to search hundreds of millions of new articles. But in the months since, the team has extensively tweaked the neural networks it uses, improving the quality of the evidence the system can unearth. One important addition is BERT, a neural network Google built for natural-language processing, which can answer queries. The work will be presented at the Association for the Advancement of Artificial Intelligence conference in New York next month.
To train their AI, lead researcher Noam Slonim and his colleagues at IBM Research in Haifa, Israel, drew on 400 million documents taken from the LexisNexis database of newspaper and journal articles. This gave them some 10 billion sentences, a natural-language corpus around 50 times larger than Wikipedia. They paired this vast evidence pool with claims about several hundred different topics, such as “Blood donation should be mandatory” or “We should abandon Valentine’s Day.”
They then asked crowd workers on the Figure Eight platform to label sentences according to whether or not they provided evidence for or against particular claims. The labeled data was fed to a supervised learning algorithm.
