AI watchdog needed to regulate automated decision-making

AI watchdog needed to regulate automated decision-making, say experts

AI watchdog needed to regulate automated decision-making, say experts

An artificial intelligence watchdog should be set up to make sure people are not discriminated against by the automated computer systems making important decisions about their lives, say experts.

The rise of artificial intelligence (AI) has led to an explosion in the number of algorithms that are used by employers, banks, police forces and others, but the systems can, and do, make bad decisions that seriously impact people’s lives. But because technology companies are so secretive about how their algorithms work – to prevent other firms from copying them – they rarely disclose any detailed information about how AIs have made particular decisions.

In a new report, Sandra Wachter, Brent Mittelstadt, and Luciano Floridi, a research team at the Alan Turing Institute in London and the University of Oxford, call for a trusted third party body that can investigate AI decisions for people who believe they have been discriminated against.

“What we’d like to see is a trusted third party, perhaps a regulatory or supervisory body, that would have the power to scrutinise and audit algorithms, so they could go in and see whether the system is actually transparent and fair,” said Wachter.

It is not a new problem. Back in the 1980s, an algorithm used to sift student applications at St George’s Hospital Medial School in London was found to discriminate against women and people with non-European-looking names. More recently, a veteran American Airlines pilot described how he had been detained at airports on 80 separate occasions after an algorithm repeatedly confused him with an IRA leader. Others to fall foul of AI errors have lost their jobs, had car licences revoked, been kicked off the electoral register or mistakenly chased for child support bills.

Read Also:
How Microsoft's other machine learning tricks could make its bots even smarter

People who find themselves on the wrong end of a flawed AI can challenge the decision under national laws – but the report finds that the current laws to protect people are not now effective enough.

In Britain, the Data Protection Act allows automated decisions to be challenged. But UK firms, in common with those in other countries, do not have to release any information they consider a trade secret. In practice, this means that instead of releasing a full explanation for a specific AI decision, a company can simply describe how the algorithm works. For example, a person turned down for a credit card might be told that the algorithm took their credit history, age, and postcode into account, but not learn why their application was rejected.

In 2018, European member states, along with Britain, will adopt new legislation that governs how AIs can be challenged. Early drafts of the General Data Protection Regulation (GDPR) enshrined what is called a “right to explanation” in law. But the authors argue that the final version approved last year contains no legal guarantee.

Read Also:
Predictive analytics: Better than surveys?

“There is an idea that the GDPR will deliver accountability and transparency for AI, but that’s not at all guaranteed. It all depends on how it is interpreted in the future by national and European courts,” Mittelstadt said. The best the new regulation offers is a “right to be informed” compelling companies to reveal the purpose of an algorithm, the kinds of data it draws on to make its decisions, and other basic information. In their paper, the researchers argue for the regulation to be amended to make the “right to explanation” legally binding.

“We are already too dependent on algorithms to give up the right to question their decisions,” said Floridi. “The GDPR should be improved to ensure that such a right is fully and unambiguously supported”.

For the study, the authors reviewed legal cases in Austria and Germany which have some of the stricter laws around decision-taking algorithms. In many cases, they found that courts required companies to release only the most general information about the decisions algorithms had made.

Read Also:
Analytics for the Masses: Five Things to Consider


Data Innovation Summit 2017

30
Mar
2017
Data Innovation Summit 2017

30% off with code 7wData

Read Also:
Big data vs. smart data: Dun & Bradstreet chief data scientist breaks it down

Big Data Innovation Summit London

30
Mar
2017
Big Data Innovation Summit London

$200 off with code DATA200

Read Also:
Why Data Is Important for Companies and Why Innovation Is On the Way

Enterprise Data World 2017

2
Apr
2017
Enterprise Data World 2017

$200 off with code 7WDATA

Read Also:
How Microsoft's other machine learning tricks could make its bots even smarter

Data Visualisation Summit San Francisco

19
Apr
2017
Data Visualisation Summit San Francisco

$200 off with code DATA200

Read Also:
How big data is helping in implementation of demonetisation

Chief Analytics Officer Europe

25
Apr
2017
Chief Analytics Officer Europe

15% off with code 7WDCAO17

Read Also:
Analytics “at” the edge, on the device, in real time

Leave a Reply

Your email address will not be published. Required fields are marked *