Researcher Profile: The quest for artificial intelligence
Dr. Marc Bellemare is working on the “holy grail” of artificial intelligence—computers that learn and think on their own.
Dr. Bellemare is a computer scientist working in one of the hottest research areas in artificial intelligence (AI). He is designing software agents that make machines—much like people and animals—capable of learning things for themselves through trial and error.
In August 2013, Dr. Bellemare joined some of the world’s top AI scientists at start-up DeepMind in London UK. Five months later, the company was bought by Google. Their mission: Solve Intelligence – Combining the best techniques from machine learning, reinforcement learning (RL), and systems neuroscience to build powerful general purpose learning algorithms.
Q: What was the focus of your research at the University of Alberta?
A: I led the development of the Arcade Learning Environment (ALE), a project initiated in 2008 by my supervisor, Dr. Michael Bowling. The ALE is an open-source interface through which artificial agents play Atari 2600 games. Most of my PhD argued that video games are great for studying RL—which combines data with decisions—and AI in general. There are now over 100 research groups around the world using the ALE, including DeepMind. My PhD project, along with DeepMind’s 2015 Nature paper on using deep learning to play Atari 2600 games, are probably the main reasons for worldwide interest in RL research. Today, I still act as the lead developer on the ALE project.
Q: What role did Compute Canada play in your Ph.D. research?
A: I mostly used two or three WestGrid compute clusters, typically running a large number of single-core programs in parallel. I also performed a number of experiments using Calcul Quebec’s then-new cluster, “Mammouth.” These resources were invaluable to my PhD and largely responsible for the success of my research career. Having access to sizeable computing resources enabled us to establish conclusive benchmarks on the ALE, instead of a small proof-of-concept. Without Compute Canada resources my PhD might have taken a year or two longer, and the results would not have been as compelling. I’ve had AI researchers from elsewhere in Canada and the United States express their “compute envy” regarding the results we generated during my PhD.
Q: What about the experts at Compute Canada? Were they able to help when questions arose?
A: When I did need their help, I found the Compute Canada teams resolved problems quickly and/or pointed me towards the relevant resources. I think one of their key strengths is they have a great understanding of the demands of scientific computing—how different problems require different kinds of resources. They were excellent both at dealing with time-critical requests and helping plan long-haul research projects.
Q: How did having access to Compute Canada resources prepare you for a career in industry?
A: Many of today’s most exciting AI applications involve billions of data points. There’s little hope of handling this kind of “big data” on a small compute cluster, let alone on a laptop computer. The same processes that I used during my PhD—automated parameter testing, agent benchmarking, etc.—are part of my day-to-day research at DeepMind.
Q: What are you currently working on?
A: I’m focusing mostly on integrating RL and information theory, which provides surprisingly elegant solutions to a number of applied RL problems. For example, if an agent has a finite memory and can only remember n pieces of information, can we devise an algorithm which automatically remembers the most important things? As it turns out, information theory allows us to come up with a practical solution to this problem.
Q: What are the potential real-world applications of RL?
A: At McGill University, Joelle Pineau (another Compute Canada user) is working on an intelligent robotic wheelchair for people with severe mobility impairments. For example, it might learn to navigate to the owner’s particular house layout, or learn to communicate with them. It has potential to give people with disabilities much more freedom. There’s also been work on applying RL to regulate power markets and smart grids and even help people with major depressive disorder through phased treatment. There’s a real potential here to inform the decisions doctors and psychiatrists make.
Q: Do you expect to return to Canada some day?
A: Yes, I’m certainly hoping one day to come back and contribute my expertise to solving Canada-specific problems.