How Causal Inference Can Lead To Real Intelligence In Machines

Last year, the machine learning community was thrown into disarray when its top minds Yann LeCun, Ali Rahimi and Judea Pearl had a faceoff on the state of artificial intelligence and machine learning.

While Rahimi and Pearl tried to tone down the hype around AI, LeCun was aghast over the scepticism around intelligence and causality of the models.

Pearl also went on record to say that deep learning was stuck with curve fitting and called it “sacrilege”. From the point of view of the mathematical hierarchy, Pearl said that no matter how well the data is manipulated, it’s still a curve-fitting exercise. 

This a very controversial accusation coming from Pearl, who was awarded the ACM Turing Award for fundamental contributions to artificial intelligence through the development of a calculus for probabilistic and causal reasoning.

“I think a lot of people from outside the field criticise the current status while ignoring that people actively work on fixing the very aspects they criticise. This includes causality, self-sup learning, reasoning, memory,” fired back LeCun in hisrecent post on Twitter, complementing the views of Max Welling, another noted AI researcher. 

To get a sense of what the critics of AI are suggesting, consider a reinforcement learning system that interacts and intervenes. This RL system only allows one to infer the consequences of those interventions, but ONLY those interventions. For a model to be causal, it has to go BEYOND — to actions not used in training.

However, in a recent work published by OpenAI, they show how the agents in a hide and seek game, do something astounding and break the game rules. The researchers at OpenAI came across agent’s new strategies to win the game, that was previously never thought of. This is indeed, a step in the right direction — emerging intelligence.

Deep learning, as the critics say, is just not all about curve fitting, the research that goes into causality doesn’t get the attention it deserves. In this next section, we list a few interesting works in this field.

Before we go any further, first let’s define what is causality and why so much talk about it in the machine learning circles:

Causality is the degree to which one can rule out plausible alternative explanations. The ability to rule out competing explanations is addressed by the design of the study (random assignment, sampling methods, etc). 

So, by defining causality in systems, one gets to ask or even answer why one needs or doesn’t need a certain feature in a model.