Canadians are known for modesty, often hesitant to trumpet their accomplishments to the world, and that goes for contributions in tech, too. While much of the focus is on Silicon Valley, when it comes to Artificial Intelligence and Machine Learning, many foundational breakthroughs happened above the 49th parallel.
So today we’re highlighting the groundbreaking work of Canadians in AI, specifically the Canadian Institute for Advanced Research, a Toronto lab that kept the fire alight through several long AI winters.
In particular, we’ll be looking at the accomplishments of three of CIFAR’s leaders, Geoffrey Hinton, Yann LeCun, and Yoshua Bengio, whose work would prove so fundamental that they’d become known as Canada’s AI Mafia and the Godfathers of AI. Although the Mafia members receive much of the credit today, it’s important to remember that their work was supported and accomplished by dozens of young researchers and students who aren’t as well recognized.
As proud Canadians who also push the boundaries of AI, we want to share their story.
Not only did the AI Mafia lay the groundwork for modern AI, along with their alumni they continue to drive those advancements today. We wouldn’t be where we are now without them.
The Long, Dark AI Winter
In 1982 the University of Toronto created a special program to focus on emerging fields that struggled to find support elsewhere. They called this new lab the Canadian Institute for Advanced Research (CIFAR).
The 1980s were a tough time for AI––those who read our History of AI post will know the decade marked one of the more frigid AI winters. Throughout the decade and into the 90s AI’s theories continuously ran up against the limits of the day’s computer’s. It wasn’t that researchers were short of ideas, they simply lacked the computational muscle to test them. Progress slowed, public interest waned, and funding dried up. The computer science community wrote off most of AI theory as a collection of interesting ideas but, ultimately, a dead end.
This did, however, make it a perfect candidate for CIFAR support. As one among just a handful of institutions still pushing the field, CIFAR became a magnet for global talent. Over the next decade the institute would recruit three renowned machine learning experts: Geoffrey Hinton, Yann LeCun, and Yoshua Bengio. Leading young and talented international teams, the three would undertake some of the most important work in AI and cement themselves in history as Canada’s AI Mafia.
The Mafia members collaborated often and also worked in their own spheres. Hinton became known early on for strengthening the recognition capabilities of the first neural networks, largely through the technique of backpropagation. Bengio made strides in machine translation, applying neural networks to natural language processing, as well as pioneering architectures that enabled more layers in deep neural networks. LeCun built some of the earliest convolutional neural networks that could recognize patterns in pixels. You can draw a straight line from his work at CIFAR to your ability to photograph and deposit a handwritten cheque through your phone’s banking app today. This paper LeCun wrote provides the technical details, if you want to get into the nitty gritty.
The CIFAR team made haltering progress but, like everyone else working in AI at the time, got held back by technical constrictions. Despite the roadblocks, the CIFAR team toiled away quietly for years, making incremental progress within their limited means, receiving little recognition or fanfare. In this way they kept AI alive through the long winter.
Big Data and the AI Renaissance
In the early 2000s AI began to build serious momentum again, largely thanks to the emergence of Big Data. You need enormous amounts of information to properly train a deep neural network. The prevalence of the Internet and social media allowed computer scientists to finally collect data sets large enough to feed their networks. Pair that with computers that had grown powerful enough to handle all that data, and the AI industry was primed to blast off. We’ve been climbing ever since.
AI may have been a fringe pursuit back in CIFAR’s early years, but the perseverance of many of the young researchers paid off. Today you’ll find CIFAR alumni filling many of the industry’s top positions. That goes for the AI Mafia, too. Hinton is a senior researcher at Google’s Alphabet Inc., LeCun heads Facebook’s AI division, and Bengio co-founded Element AI, a major Montreal-based company developing novel practical applications for machine learning. Last year the three jointly won the Turing Award, the highest accolade in AI (unofficially referred to as the Nobel Prize of computer science).
The Legacy Continues
Without the tireless work of the Canadian AI Mafia and their students through the down times, it’s difficult to say where AI would be. Without a doubt it would be years––perhaps even decades––behind its current state, and we certainly wouldn’t be seeing the exciting breakthroughs and benefits we do today.
As a Canadian AI company, we’re constantly inspired by the work the AI Mafia has and continues to do, as well as all the other AI innovators working in the Great White North. We feel honoured to take on that legacy and continue pushing the field forward as Canadians working in AI.