Source: Macleans, Mar 2016
Q: So, why is it important that AI triumphed in the game of Go?
A: It relies on a lot of intuition. The really skilled players just sort of see where a good place to put a stone would be. They do a lot of reasoning as well, which they call reading, but they also have very good intuition about where a good place to go would be, and that’s the kind of thing that people just thought computes couldn’t do. But with these neural networks, computers can do that too. They can think about all the possible moves and think that one particular move seems a bit better than the others, just intuitively. That’s what the feed point neural network is doing: it’s giving the system intuitions about what might be a good move. It then goes off and tries all sorts of alternatives. The neural networks provides you with good intuitions, and that’s what the other programs were lacking, and that’s what people didn’t really understand computers could do.
Q: Beyond games, then—what might come next for AI?
A: It depends who you talk to. My belief is that we’re not going to get human-level abilities until we have systems that have the same number of parameters in them as the brain. So in the brain, you have connections between the neurons called synapses, and they can change. All your knowledge is stored in those synapses. You have about 1,000-trillion synapses—10 to the 15, it’s a very big number. So that’s quite unlike the neural networks we have right now. They’re far, far smaller, the biggest ones we have right now have about a billion synapses. That’s about a million times smaller than the brain.
Q: There’s also the fear that AI will render humanity obsolete, that there will come an inevitable loss of labour.
A: It’s hard to predict beyond five years. I’m pretty confident it won’t happen in the next five years, and I’m fairly confident that it won’t be something I’m going to have to deal with. But it’s something people should definitely be thinking about. But the main thing shouldn’t be, how do we cripple this technology so it can’t be harmful, it should be, how do we improve our political system so people can’t use it for bad purposes?
Q: How important is the power of computing to continued work in the deep learning field?
In deep learning, the algorithms we use now are versions of the algorithms we were developing in the 1980s, the 1990s. People were very optimistic about them, but it turns out they didn’t work too well. Now we know the reason is they didn’t work too well is that we didn’t have powerful enough computers, we didn’t have enough data sets to train them. If we want to approach the level of the human brain, we need much more computation, we need better hardware. We are much closer than we were 20 years ago, but we’re still a long way away. We’ll see something with proper common-sense reasoning.
Q: This kind of intellectual comeback story feels like it could only happen in science. The writer Thomas Kuhn talks about it when he talked about “paradigm shifts”—that these scientific revolutions don’t necessarily produce better ideas, just different ideas. Culture at large seems to have lost this concept. Is the comeback of deep learning the kind of thing that can only happen in science?
A: I think that’s what differentiates science from religion. In science, you can say things that seem crazy, but in the long run they can turn out to be right. We can get really good evidence, and in the end the community will come around. Probably the scientists you’re arguing with won’t come around, but the younger generation will defect, and that’s what’s happening with deep learning. It’s not so much the old conventional AI guys are believing in it, it’s the young graduate students all seeing which ways things are going.