Gary Marcus: Toward a Hybrid of Deep Learning and Symbolic AI #43

Transcript

00:00:00 The following is a conversation with Gary Marcus.

00:00:02 He’s a professor emeritus at NYU,

00:00:04 founder of Robust AI and Geometric Intelligence.

00:00:08 The latter is a machine learning company

00:00:10 that was acquired by Uber in 2016.

00:00:13 He’s the author of several books,

00:00:15 Unnatural and Artificial Intelligence,

00:00:18 including his new book, Rebooting AI,

00:00:20 Building Machines We Can Trust.

00:00:23 Gary has been a critical voice,

00:00:25 highlighting the limits of deep learning and AI in general

00:00:28 and discussing the challenges before our AI community

00:00:33 that must be solved in order to achieve

00:00:35 artificial general intelligence.

00:00:38 As I’m having these conversations,

00:00:40 I try to find paths toward insight, towards new ideas.

00:00:43 I try to have no ego in the process.

00:00:45 It gets in the way.

00:00:47 I’ll often continuously try on several hats, several roles.

00:00:52 One, for example, is the role of a three year old

00:00:54 who understands very little about anything

00:00:57 and asks big what and why questions.

00:01:00 The other might be a role of a devil’s advocate

00:01:02 who presents counter ideas with the goal of arriving

00:01:05 at greater understanding through debate.

00:01:08 Hopefully, both are useful, interesting,

00:01:11 and even entertaining at times.

00:01:13 I ask for your patience as I learn

00:01:15 to have better conversations.

00:01:17 This is the Artificial Intelligence Podcast.

00:01:20 If you enjoy it, subscribe on YouTube,

00:01:23 give it five stars on iTunes, support it on Patreon,

00:01:26 or simply connect with me on Twitter

00:01:28 at Lex Friedman, spelled F R I D M A N.

00:01:32 And now, here’s my conversation with Gary Marcus.

00:01:37 Do you think human civilization will one day have

00:01:40 to face an AI driven technological singularity

00:01:42 that will, in a societal way,

00:01:45 modify our place in the food chain

00:01:47 of intelligent living beings on this planet?

00:01:50 I think our place in the food chain has already changed.

00:01:54 So there are lots of things people used to do by hand

00:01:57 that they do with machine.

00:01:59 If you think of a singularity as like one single moment,

00:02:01 which is, I guess, what it suggests,

00:02:03 I don’t know if it’ll be like that,

00:02:04 but I think that there’s a lot of gradual change

00:02:07 and AI is getting better and better.

00:02:09 I mean, I’m here to tell you why I think it’s not nearly

00:02:11 as good as people think, but the overall trend is clear.

00:02:14 Maybe Rick Hertzweil thinks it’s an exponential

00:02:17 and I think it’s linear.

00:02:18 In some cases, it’s close to zero right now,

00:02:20 but it’s all gonna happen.

00:02:21 I mean, we are gonna get to human level intelligence

00:02:24 or whatever you want, artificial general intelligence

00:02:28 at some point, and that’s certainly gonna change

00:02:31 our place in the food chain,

00:02:32 because a lot of the tedious things that we do now,

00:02:35 we’re gonna have machines do,

00:02:36 and a lot of the dangerous things that we do now,

00:02:38 we’re gonna have machines do.

00:02:39 I think our whole lives are gonna change

00:02:41 from people finding their meaning through their work

00:02:45 through people finding their meaning

00:02:46 through creative expression.

00:02:48 So the singularity will be a very gradual,

00:02:53 in fact, removing the meaning of the word singularity.

00:02:56 It’ll be a very gradual transformation in your view.

00:03:00 I think that it’ll be somewhere in between,

00:03:03 and I guess it depends what you mean by gradual and sudden.

00:03:05 I don’t think it’s gonna be one day.

00:03:07 I think it’s important to realize

00:03:08 that intelligence is a multidimensional variable.

00:03:11 So people sort of write this stuff

00:03:14 as if IQ was one number, and the day that you hit 262

00:03:20 or whatever, you displace the human beings.

00:03:22 And really, there’s lots of facets to intelligence.

00:03:25 So there’s verbal intelligence,

00:03:26 and there’s motor intelligence,

00:03:28 and there’s mathematical intelligence and so forth.

00:03:32 Machines, in their mathematical intelligence,

00:03:34 far exceed most people already.

00:03:36 In their ability to play games,

00:03:38 they far exceed most people already.

00:03:40 In their ability to understand language,

00:03:41 they lag behind my five year old,

00:03:43 far behind my five year old.

00:03:44 So there are some facets of intelligence

00:03:46 that machines have grasped, and some that they haven’t,

00:03:49 and we have a lot of work left to do

00:03:51 to get them to, say, understand natural language,

00:03:54 or to understand how to flexibly approach

00:03:57 some kind of novel MacGyver problem solving

00:04:01 kind of situation.

00:04:03 And I don’t know that all of these things will come at once.

00:04:05 I think there are certain vital prerequisites

00:04:07 that we’re missing now.

00:04:09 So for example, machines don’t really have common sense now.

00:04:12 So they don’t understand that bottles contain water,

00:04:15 and that people drink water to quench their thirst,

00:04:18 and that they don’t wanna dehydrate.

00:04:19 They don’t know these basic facts about human beings,

00:04:22 and I think that that’s a rate limiting step

00:04:24 for many things.

00:04:25 It’s a great limiting step for reading, for example,

00:04:27 because stories depend on things like,

00:04:29 oh my God, that person’s running out of water.

00:04:31 That’s why they did this thing.

00:04:33 Or if they only had water, they could put out the fire.

00:04:37 So you watch a movie, and your knowledge

00:04:39 about how things work matter.

00:04:41 And so a computer can’t understand that movie

00:04:44 if it doesn’t have that background knowledge.

00:04:45 Same thing if you read a book.

00:04:47 And so there are lots of places where,

00:04:49 if we had a good machine interpretable set of common sense,

00:04:53 many things would accelerate relatively quickly,

00:04:56 but I don’t think even that is a single point.

00:04:59 There’s many different aspects of knowledge.

00:05:02 And we might, for example, find that we make a lot

00:05:05 of progress on physical reasoning,

00:05:06 getting machines to understand, for example,

00:05:09 how keys fit into locks, or that kind of stuff,

00:05:11 or how this gadget here works, and so forth and so on.

00:05:16 And so machines might do that long before they do

00:05:19 really good psychological reasoning,

00:05:21 because it’s easier to get kind of labeled data

00:05:24 or to do direct experimentation on a microphone stand

00:05:28 than it is to do direct experimentation on human beings

00:05:31 to understand the levers that guide them.

00:05:34 That’s a really interesting point, actually,

00:05:36 whether it’s easier to gain common sense knowledge

00:05:39 or psychological knowledge.

00:05:41 I would say the common sense knowledge

00:05:43 includes both physical knowledge and psychological knowledge.

00:05:46 And the argument I was making.

00:05:47 Well, you said physical versus psychological.

00:05:49 Yeah, physical versus psychological.

00:05:51 And the argument I was making is physical knowledge

00:05:53 might be more accessible, because you could have a robot,

00:05:55 for example, lift a bottle, try putting a bottle cap on it,

00:05:58 see that it falls off if it does this,

00:06:00 and see that it could turn it upside down,

00:06:02 and so the robot could do some experimentation.

00:06:04 We do some of our psychological reasoning

00:06:07 by looking at our own minds.

00:06:09 So I can sort of guess how you might react to something

00:06:11 based on how I think I would react to it.

00:06:13 And robots don’t have that intuition,

00:06:15 and they also can’t do experiments on people

00:06:18 in the same way or we’ll probably shut them down.

00:06:20 So if we wanted to have robots figure out

00:06:24 how I respond to pain by pinching me in different ways,

00:06:27 like that’s probably, it’s not gonna make it

00:06:29 past the human subjects board

00:06:31 and companies are gonna get sued or whatever.

00:06:32 So there’s certain kinds of practical experience

00:06:35 that are limited or off limits to robots.

00:06:39 That’s a really interesting point.

00:06:41 What is more difficult to gain a grounding in?

00:06:47 Because to play devil’s advocate,

00:06:49 I would say that human behavior is easier expressed

00:06:54 in data and digital form.

00:06:56 And so when you look at Facebook algorithms,

00:06:59 they get to observe human behavior.

00:07:01 So you get to study and manipulate even a human behavior

00:07:04 in a way that you perhaps cannot study

00:07:07 or manipulate the physical world.

00:07:09 So it’s true why you said pain is like physical pain,

00:07:14 but that’s again, the physical world.

00:07:16 Emotional pain might be much easier to experiment with,

00:07:20 perhaps unethical, but nevertheless,

00:07:22 some would argue it’s already going on.

00:07:25 I think that you’re right, for example,

00:07:27 that Facebook does a lot of experimentation

00:07:30 in psychological reasoning.

00:07:32 In fact, Zuckerberg talked about AI

00:07:36 at a talk that he gave in NIPS.

00:07:38 I wasn’t there, but the conference

00:07:40 has been renamed NeurIPS,

00:07:41 but he used to be called NIPS when he gave the talk.

00:07:43 And he talked about Facebook basically

00:07:45 having a gigantic theory of mind.

00:07:47 So I think it is certainly possible.

00:07:49 I mean, Facebook does some of that.

00:07:51 I think they have a really good idea

00:07:52 of how to addict people to things.

00:07:53 They understand what draws people back to things.

00:07:56 I think they exploit it in ways

00:07:57 that I’m not very comfortable with.

00:07:59 But even so, I think that there are only some slices

00:08:03 of human experience that they can access

00:08:05 through the kind of interface they have.

00:08:07 And of course, they’re doing all kinds of VR stuff,

00:08:08 and maybe that’ll change and they’ll expand their data.

00:08:11 And I’m sure that that’s part of their goal.

00:08:14 So it is an interesting question.

00:08:16 I think love, fear, insecurity,

00:08:21 all of the things that,

00:08:24 I would say some of the deepest things

00:08:26 about human nature and the human mind

00:08:28 could be explored through digital form.

00:08:30 It’s that you’re actually the first person

00:08:32 just now that brought up,

00:08:33 I wonder what is more difficult.

00:08:35 Because I think folks who are the slow,

00:08:40 and we’ll talk a lot about deep learning,

00:08:41 but the people who are thinking beyond deep learning

00:08:44 are thinking about the physical world.

00:08:46 You’re starting to think about robotics

00:08:48 in the home robotics.

00:08:49 How do we make robots manipulate objects,

00:08:52 which requires an understanding of the physical world

00:08:55 and then requires common sense reasoning.

00:08:57 And that has felt to be like the next step

00:08:59 for common sense reasoning,

00:09:00 but you’ve now brought up the idea

00:09:02 that there’s also the emotional part.

00:09:03 And it’s interesting whether that’s hard or easy.

00:09:06 I think some parts of it are and some aren’t.

00:09:08 So my company that I recently founded with Rod Brooks,

00:09:12 from MIT for many years and so forth,

00:09:15 we’re interested in both.

00:09:17 We’re interested in physical reasoning

00:09:18 and psychological reasoning, among many other things.

00:09:21 And there are pieces of each of these that are accessible.

00:09:26 So if you want a robot to figure out

00:09:28 whether it can fit under a table,

00:09:29 that’s a relatively accessible piece of physical reasoning.

00:09:33 If you know the height of the table

00:09:34 and you know the height of the robot, it’s not that hard.

00:09:36 If you wanted to do physical reasoning about Jenga,

00:09:39 it gets a little bit more complicated

00:09:41 and you have to have higher resolution data

00:09:43 in order to do it.

00:09:45 With psychological reasoning,

00:09:46 it’s not that hard to know, for example,

00:09:49 that people have goals and they like to act on those goals,

00:09:51 but it’s really hard to know exactly what those goals are.

00:09:54 But ideas of frustration.

00:09:56 I mean, you could argue it’s extremely difficult

00:09:58 to understand the sources of human frustration

00:10:01 as they’re playing Jenga with you, or not.

00:10:05 You could argue that it’s very accessible.

00:10:08 There’s some things that are gonna be obvious

00:10:09 and some not.

00:10:10 So I don’t think anybody really can do this well yet,

00:10:14 but I think it’s not inconceivable

00:10:16 to imagine machines in the not so distant future

00:10:20 being able to understand that if people lose in a game,

00:10:24 that they don’t like that.

00:10:26 That’s not such a hard thing to program

00:10:27 and it’s pretty consistent across people.

00:10:29 Most people don’t enjoy losing

00:10:31 and so that makes it relatively easy to code.

00:10:34 On the other hand, if you wanted to capture everything

00:10:36 about frustration, well, people can get frustrated

00:10:39 for a lot of different reasons.

00:10:40 They might get sexually frustrated,

00:10:42 they might get frustrated,

00:10:43 they can get their promotion at work,

00:10:45 all kinds of different things.

00:10:46 And the more you expand the scope,

00:10:48 the harder it is for anything like the existing techniques

00:10:51 to really do that.

00:10:53 So I’m talking to Garret Kasparov next week

00:10:55 and he seemed pretty frustrated

00:10:57 with his game against Deep Blue, so.

00:10:58 Yeah, well, I’m frustrated with my game

00:11:00 against him last year,

00:11:01 because I played him, I had two excuses,

00:11:03 I’ll give you my excuses up front,

00:11:04 but it won’t mitigate the outcome.

00:11:07 I was jet lagged and I hadn’t played in 25 or 30 years,

00:11:11 but the outcome is he completely destroyed me

00:11:13 and it wasn’t even close.

00:11:14 Have you ever been beaten in any board game by a machine?

00:11:19 I have, I actually played the predecessor to Deep Blue.

00:11:24 Deep Thought, I believe it was called,

00:11:27 and that too crushed me.

00:11:30 And that was, and after that you realize it’s over for us.

00:11:35 Well, there’s no point in my playing Deep Blue.

00:11:36 I mean, it’s a waste of Deep Blue’s computation.

00:11:40 I mean, I played Kasparov

00:11:41 because we both gave lectures this same event

00:11:44 and he was playing 30 people.

00:11:46 I forgot to mention that.

00:11:46 Not only did he crush me,

00:11:47 but he crushed 29 other people at the same time.

00:11:50 I mean, but the actual philosophical and emotional experience

00:11:55 of being beaten by a machine, I imagine is a,

00:11:59 I mean, to you who thinks about these things

00:12:01 may be a profound experience.

00:12:03 Or no, it was a simple mathematical experience.

00:12:07 Yeah, I think a game like chess particularly

00:12:10 where you have perfect information,

00:12:12 it’s two player closed end

00:12:14 and there’s more computation for the computer,

00:12:16 it’s no surprise the machine wins.

00:12:18 I mean, I’m not sad when a computer,

00:12:22 I’m not sad when a computer calculates

00:12:23 a cube root faster than me.

00:12:25 Like, I know I can’t win that game.

00:12:27 I’m not gonna try.

00:12:28 Well, with a system like AlphaGo or AlphaZero,

00:12:32 do you see a little bit more magic in a system like that

00:12:35 even though it’s simply playing a board game?

00:12:37 But because there’s a strong learning component?

00:12:39 You know, I find you should mention that

00:12:41 in the context of this conversation

00:12:42 because Kasparov and I are working on an article

00:12:45 that’s gonna be called AI is not magic.

00:12:47 And, you know, neither one of us thinks that it’s magic.

00:12:50 And part of the point of this article

00:12:51 is that AI is actually a grab bag of different techniques

00:12:55 and some of them have,

00:12:56 or they each have their own unique strengths and weaknesses.

00:13:00 So, you know, you read media accounts

00:13:02 and it’s like, ooh, AI, it must be magical

00:13:05 or it can solve any problem.

00:13:06 Well, no, some problems are really accessible

00:13:09 like chess and go and other problems like reading

00:13:11 are completely outside the current technology.

00:13:14 And it’s not like you can take the technology,

00:13:17 that drives AlphaGo and apply it to reading

00:13:20 and get anywhere.

00:13:21 You know, DeepMind has tried that a bit.

00:13:23 They have all kinds of resources.

00:13:24 You know, they built AlphaGo and they have,

00:13:26 you know, I wrote a piece recently that they lost

00:13:29 and you can argue about the word lost,

00:13:30 but they spent $530 million more than they made last year.

00:13:34 So, you know, they’re making huge investments.

00:13:36 They have a large budget

00:13:37 and they have applied the same kinds of techniques

00:13:40 to reading or to language.

00:13:43 It’s just much less productive there

00:13:45 because it’s a fundamentally different kind of problem.

00:13:47 Chess and go and so forth are closed end problems.

00:13:50 The rules haven’t changed in 2,500 years.

00:13:52 There’s only so many moves you can make.

00:13:54 You can talk about the exponential

00:13:56 as you look at the combinations of moves,

00:13:58 but fundamentally, you know, the go board has 361 squares.

00:14:01 That’s it.

00:14:02 That’s the only, you know, those intersections

00:14:04 are the only places that you can place your stone.

00:14:07 Whereas when you’re reading,

00:14:09 the next sentence could be anything.

00:14:11 You know, it’s completely up to the writer

00:14:13 what they’re gonna do next.

00:14:14 That’s fascinating that you think this way.

00:14:16 You’re clearly a brilliant mind

00:14:17 who points out the emperor has no clothes,

00:14:19 but so I’ll play the role of a person who says.

00:14:22 You’re gonna put clothes on the emperor?

00:14:23 Good luck with it.

00:14:24 It romanticizes the notion of the emperor, period,

00:14:27 suggesting that clothes don’t even matter.

00:14:30 Okay, so that’s really interesting

00:14:33 that you’re talking about language.

00:14:36 So there’s the physical world

00:14:37 of being able to move about the world,

00:14:39 making an omelet and coffee and so on.

00:14:41 There’s language where you first understand

00:14:46 what’s being written and then maybe even more complicated

00:14:48 than that, having a natural dialogue.

00:14:51 And then there’s the game of go and chess.

00:14:53 I would argue that language is much closer to go

00:14:57 than it is to the physical world.

00:14:59 Like it is still very constrained.

00:15:01 When you say the possibility of the number of sentences

00:15:04 that could come, it is huge,

00:15:06 but it nevertheless is much more constrained.

00:15:09 It feels maybe I’m wrong than the possibilities

00:15:12 that the physical world brings us.

00:15:14 There’s something to what you say

00:15:15 in some ways in which I disagree.

00:15:17 So one interesting thing about language

00:15:20 is that it abstracts away.

00:15:23 This bottle, I don’t know if it would be in the field of view

00:15:26 is on this table and I use the word on here

00:15:28 and I can use the word on here, maybe not here,

00:15:32 but that one word encompasses in analog space

00:15:36 sort of infinite number of possibilities.

00:15:39 So there is a way in which language filters down

00:15:43 the variation of the world and there’s other ways.

00:15:46 So we have a grammar and more or less

00:15:49 you have to follow the rules of that grammar.

00:15:51 You can break them a little bit,

00:15:52 but by and large we follow the rules of grammar

00:15:55 and so that’s a constraint on language.

00:15:57 So there are ways in which language is a constrained system.

00:15:59 On the other hand, there are many arguments

00:16:02 that say there’s an infinite number of possible sentences

00:16:04 and you can establish that by just stacking them up.

00:16:07 So I think there’s water on the table,

00:16:09 you think that I think there’s water on the table,

00:16:11 your mother thinks that you think that I think

00:16:13 that water’s on the table, your brother thinks

00:16:15 that maybe your mom is wrong to think

00:16:17 that you think that I think, right?

00:16:18 So we can make sentences of infinite length

00:16:21 or we can stack up adjectives.

00:16:23 This is a very silly example, a very, very silly example,

00:16:26 a very, very, very, very, very, very silly example

00:16:28 and so forth.

00:16:29 So there are good arguments

00:16:30 that there’s an infinite range of sentences.

00:16:32 In any case, it’s vast by any reasonable measure

00:16:35 and for example, almost anything in the physical world

00:16:37 we can talk about in the language world

00:16:40 and interestingly, many of the sentences that we understand,

00:16:43 we can only understand if we have a very rich model

00:16:46 of the physical world.

00:16:47 So I don’t ultimately want to adjudicate the debate

00:16:50 that I think you just set up, but I find it interesting.

00:16:54 Maybe the physical world is even more complicated

00:16:57 than language, I think that’s fair, but.

00:16:59 Language is really, really complicated.

00:17:03 It’s really, really hard.

00:17:04 Well, it’s really, really hard for machines,

00:17:06 for linguists, people trying to understand it.

00:17:08 It’s not that hard for children

00:17:09 and that’s part of what’s driven my whole career.

00:17:12 I was a student of Steven Pinker’s

00:17:14 and we were trying to figure out

00:17:15 why kids could learn language when machines couldn’t.

00:17:18 I think we’re gonna get into language,

00:17:20 we’re gonna get into communication intelligence

00:17:22 and neural networks and so on,

00:17:24 but let me return to the high level,

00:17:28 the futuristic for a brief moment.

00:17:32 So you’ve written in your book, in your new book,

00:17:37 it would be arrogant to suppose that we could forecast

00:17:39 where AI will be or the impact it will have

00:17:42 in a thousand years or even 500 years.

00:17:45 So let me ask you to be arrogant.

00:17:48 What do AI systems with or without physical bodies

00:17:51 look like 100 years from now?

00:17:53 If you would just, you can’t predict,

00:17:56 but if you were to philosophize and imagine, do.

00:18:00 Can I first justify the arrogance

00:18:02 before you try to push me beyond it?

00:18:04 Sure.

00:18:05 I mean, there are examples like,

00:18:07 people figured out how electricity worked,

00:18:09 they had no idea that that was gonna lead to cell phones.

00:18:13 I mean, things can move awfully fast

00:18:15 once new technologies are perfected.

00:18:17 Even when they made transistors,

00:18:19 they weren’t really thinking that cell phones

00:18:21 would lead to social networking.

00:18:23 There are nevertheless predictions of the future,

00:18:25 which are statistically unlikely to come to be,

00:18:28 but nevertheless is the best.

00:18:29 You’re asking me to be wrong.

00:18:31 Asking you to be statistically.

00:18:32 In which way would I like to be wrong?

00:18:34 Pick the least unlikely to be wrong thing,

00:18:37 even though it’s most very likely to be wrong.

00:18:39 I mean, here’s some things

00:18:40 that we can safely predict, I suppose.

00:18:42 We can predict that AI will be faster than it is now.

00:18:47 It will be cheaper than it is now.

00:18:49 It will be better in the sense of being more general

00:18:52 and applicable in more places.

00:18:56 It will be pervasive.

00:18:59 I mean, these are easy predictions.

00:19:01 I’m sort of modeling them in my head

00:19:03 on Jeff Bezos’s famous predictions.

00:19:05 He says, I can’t predict the future,

00:19:07 not in every way, I’m paraphrasing.

00:19:09 But I can predict that people

00:19:11 will never wanna pay more money for their stuff.

00:19:13 They’re never gonna want it to take longer to get there.

00:19:15 So you can’t predict everything,

00:19:17 but you can predict something.

00:19:18 Sure, of course it’s gonna be faster and better.

00:19:21 But what we can’t really predict

00:19:24 is the full scope of where AI will be in a certain period.

00:19:28 I mean, I think it’s safe to say that,

00:19:31 although I’m very skeptical about current AI,

00:19:35 that it’s possible to do much better.

00:19:37 You know, there’s no in principled argument

00:19:39 that says AI is an insolvable problem,

00:19:42 that there’s magic inside our brains

00:19:43 that will never be captured.

00:19:44 I mean, I’ve heard people make those kind of arguments.

00:19:46 I don’t think they’re very good.

00:19:48 So AI’s gonna come, and probably 500 years

00:19:54 is plenty to get there.

00:19:55 And then once it’s here, it really will change everything.

00:19:59 So when you say AI’s gonna come,

00:20:01 are you talking about human level intelligence?

00:20:03 So maybe I…

00:20:04 I like the term general intelligence.

00:20:06 So I don’t think that the ultimate AI,

00:20:09 if there is such a thing, is gonna look just like humans.

00:20:11 I think it’s gonna do some things

00:20:13 that humans do better than current machines,

00:20:16 like reason flexibly.

00:20:18 And understand language and so forth.

00:20:21 But it doesn’t mean they have to be identical to humans.

00:20:23 So for example, humans have terrible memory,

00:20:25 and they suffer from what some people

00:20:28 call motivated reasoning.

00:20:29 So they like arguments that seem to support them,

00:20:32 and they dismiss arguments that they don’t like.

00:20:35 There’s no reason that a machine should ever do that.

00:20:38 So you see that those limitations of memory

00:20:42 as a bug, not a feature.

00:20:43 Absolutely.

00:20:44 I’ll say two things about that.

00:20:46 One is I was on a panel with Danny Kahneman,

00:20:48 the Nobel Prize winner, last night,

00:20:50 and we were talking about this stuff.

00:20:51 And I think what we converged on

00:20:53 is that humans are a low bar to exceed.

00:20:56 They may be outside of our skill right now,

00:20:58 but as AI programmers, but eventually AI will exceed it.

00:21:04 So we’re not talking about human level AI.

00:21:06 We’re talking about general intelligence

00:21:07 that can do all kinds of different things

00:21:09 and do it without some of the flaws that human beings have.

00:21:12 The other thing I’ll say is I wrote a whole book,

00:21:13 actually, about the flaws of humans.

00:21:15 It’s actually a nice bookend to the,

00:21:17 or counterpoint to the current book.

00:21:19 So I wrote a book called Cluj,

00:21:21 which was about the limits of the human mind.

00:21:24 The current book is kind of about those few things

00:21:26 that humans do a lot better than machines.

00:21:28 Do you think it’s possible that the flaws

00:21:30 of the human mind, the limits of memory,

00:21:33 our mortality, our bias,

00:21:38 is a strength, not a weakness,

00:21:40 that that is the thing that enables,

00:21:43 from which motivation springs and meaning springs or not?

00:21:47 I’ve heard a lot of arguments like this.

00:21:49 I’ve never found them that convincing.

00:21:50 I think that there’s a lot of making lemonade out of lemons.

00:21:55 So we, for example, do a lot of free association

00:21:58 where one idea just leads to the next

00:22:00 and they’re not really that well connected.

00:22:02 And we enjoy that and we make poetry out of it

00:22:04 and we make kind of movies with free associations

00:22:07 and it’s fun and whatever.

00:22:08 I don’t think that’s really a virtue of the system.

00:22:12 I think that the limitations in human reasoning

00:22:15 actually get us in a lot of trouble.

00:22:16 Like, for example, politically we can’t see eye to eye

00:22:19 because we have the motivational reasoning I was talking

00:22:21 about and something related called confirmation bias.

00:22:25 So we have all of these problems that actually make

00:22:27 for a rougher society because we can’t get along

00:22:29 because we can’t interpret the data in shared ways.

00:22:34 And then we do some nice stuff with that.

00:22:36 So my free associations are different from yours

00:22:38 and you’re kind of amused by them and that’s great.

00:22:41 And hence poetry.

00:22:42 So there are lots of ways in which we take

00:22:45 a lousy situation and make it good.

00:22:47 Another example would be our memories are terrible.

00:22:50 So we play games like Concentration where you flip over

00:22:53 two cards, try to find a pair.

00:22:54 Can you imagine a computer playing that?

00:22:56 Computer’s like, this is the dullest game in the world.

00:22:58 I know where all the cards are, I see it once,

00:22:59 I know where it is, what are you even talking about?

00:23:02 So we make a fun game out of having this terrible memory.

00:23:07 So we are imperfect in discovering and optimizing

00:23:12 some kind of utility function.

00:23:13 But you think in general, there is a utility function.

00:23:16 There’s an objective function that’s better than others.

00:23:18 I didn’t say that.

00:23:20 But see, the presumption, when you say…

00:23:24 I think you could design a better memory system.

00:23:27 You could argue about utility functions

00:23:29 and how you wanna think about that.

00:23:32 But objectively, it would be really nice

00:23:34 to do some of the following things.

00:23:36 To get rid of memories that are no longer useful.

00:23:41 Objectively, that would just be good.

00:23:42 And we’re not that good at it.

00:23:43 So when you park in the same lot every day,

00:23:46 you confuse where you parked today

00:23:47 with where you parked yesterday

00:23:48 with where you parked the day before and so forth.

00:23:50 So you blur together a series of memories.

00:23:52 There’s just no way that that’s optimal.

00:23:55 I mean, I’ve heard all kinds of wacky arguments

00:23:56 of people trying to defend that.

00:23:58 But in the end of the day,

00:23:58 I don’t think any of them hold water.

00:24:00 It’s just above.

00:24:01 Or memories of traumatic events would be possibly

00:24:04 a very nice feature to have to get rid of those.

00:24:06 It’d be great if you could just be like,

00:24:08 I’m gonna wipe this sector.

00:24:10 I’m done with that.

00:24:12 I didn’t have fun last night.

00:24:13 I don’t wanna think about it anymore.

00:24:14 Whoop, bye bye.

00:24:15 I’m gone.

00:24:16 But we can’t.

00:24:17 Do you think it’s possible to build a system…

00:24:20 So you said human level intelligence is a weird concept, but…

00:24:23 Well, I’m saying I prefer general intelligence.

00:24:25 General intelligence.

00:24:26 I mean, human level intelligence is a real thing.

00:24:28 And you could try to make a machine

00:24:29 that matches people or something like that.

00:24:31 I’m saying that per se shouldn’t be the objective,

00:24:34 but rather that we should learn from humans

00:24:37 the things they do well and incorporate that into our AI,

00:24:39 just as we incorporate the things that machines do well

00:24:42 that people do terribly.

00:24:43 So, I mean, it’s great that AI systems

00:24:45 can do all this brute force computation that people can’t.

00:24:48 And one of the reasons I work on this stuff

00:24:50 is because I would like to see machines solve problems

00:24:53 that people can’t, that combine the strength,

00:24:56 or that in order to be solved would combine

00:24:59 the strengths of machines to do all this computation

00:25:02 with the ability, let’s say, of people to read.

00:25:04 So I’d like machines that can read

00:25:06 the entire medical literature in a day.

00:25:08 7,000 new papers or whatever the numbers,

00:25:10 comes out every day.

00:25:11 There’s no way for any doctor or whatever to read them all.

00:25:15 A machine that could read would be a brilliant thing.

00:25:17 And that would be strengths of brute force computation

00:25:21 combined with kind of subtlety and understanding medicine

00:25:24 that a good doctor or scientist has.

00:25:26 So if we can linger a little bit

00:25:28 on the idea of general intelligence.

00:25:29 So Yann LeCun believes that human intelligence

00:25:32 isn’t general at all, it’s very narrow.

00:25:35 How do you think?

00:25:36 I don’t think that makes sense.

00:25:38 We have lots of narrow intelligences for specific problems.

00:25:42 But the fact is, like, anybody can walk into,

00:25:45 let’s say, a Hollywood movie,

00:25:47 and reason about the content

00:25:49 of almost anything that goes on there.

00:25:51 So you can reason about what happens in a bank robbery,

00:25:55 or what happens when someone is infertile

00:25:58 and wants to go to IVF to try to have a child,

00:26:02 or you can, the list is essentially endless.

00:26:05 And not everybody understands every scene in the movie,

00:26:09 but there’s a huge range of things

00:26:11 that pretty much any ordinary adult can understand.

00:26:15 His argument is, is that actually,

00:26:18 the set of things seems large for us humans

00:26:20 because we’re very limited in considering

00:26:24 the kind of possibilities of experiences that are possible.

00:26:27 But in fact, the amount of experience that are possible

00:26:30 is infinitely larger.

00:26:32 Well, I mean, if you wanna make an argument

00:26:35 that humans are constrained in what they can understand,

00:26:38 I have no issue with that.

00:26:40 I think that’s right.

00:26:41 But it’s still not the same thing at all

00:26:44 as saying, here’s a system that can play Go.

00:26:47 It’s been trained on five million games.

00:26:49 And then I say, can it play on a rectangular board

00:26:52 rather than a square board?

00:26:53 And you say, well, if I retrain it from scratch

00:26:56 on another five million games, it can.

00:26:58 That’s really, really narrow, and that’s where we are.

00:27:01 We don’t have even a system that could play Go

00:27:05 and then without further retraining,

00:27:07 play on a rectangular board,

00:27:08 which any human could do with very little problem.

00:27:12 So that’s what I mean by narrow.

00:27:14 And so it’s just wordplay to say.

00:27:16 That is semantics, yeah.

00:27:18 Then it’s just words.

00:27:19 Then yeah, you mean general in a sense

00:27:21 that you can do all kinds of Go board shapes flexibly.

00:27:25 Well, that would be like a first step

00:27:28 in the right direction,

00:27:29 but obviously that’s not what it really meaning.

00:27:30 You’re kidding.

00:27:32 What I mean by general is that you could transfer

00:27:36 the knowledge you learn in one domain to another.

00:27:38 So if you learn about bank robberies in movies

00:27:43 and there’s chase scenes,

00:27:44 then you can understand that amazing scene in Breaking Bad

00:27:47 when Walter White has a car chase scene

00:27:50 with only one person.

00:27:51 He’s the only one in it.

00:27:52 And you can reflect on how that car chase scene

00:27:55 is like all the other car chase scenes you’ve ever seen

00:27:58 and totally different and why that’s cool.

00:28:01 And the fact that the number of domains

00:28:03 you can do that with is finite

00:28:04 doesn’t make it less general.

00:28:05 So the idea of general is you could just do it

00:28:07 on a lot of, don’t transfer it across a lot of domains.

00:28:09 Yeah, I mean, I’m not saying humans are infinitely general

00:28:11 or that humans are perfect.

00:28:12 I just said a minute ago, it’s a low bar,

00:28:15 but it’s just, it’s a low bar.

00:28:17 But right now, like the bar is here and we’re there

00:28:20 and eventually we’ll get way past it.

00:28:22 So speaking of low bars,

00:28:25 you’ve highlighted in your new book as well,

00:28:27 but a couple of years ago wrote a paper

00:28:29 titled Deep Learning, A Critical Appraisal

00:28:31 that lists 10 challenges faced

00:28:33 by current deep learning systems.

00:28:36 So let me summarize them as data efficiency,

00:28:40 transfer learning, hierarchical knowledge,

00:28:42 open ended inference, explainability,

00:28:46 integrating prior knowledge, cause of reasoning,

00:28:49 modeling on a stable world, robustness, adversarial examples

00:28:53 and so on.

00:28:54 And then my favorite probably is reliability

00:28:56 in the engineering of real world systems.

00:28:59 So whatever people can read the paper,

00:29:01 they should definitely read the paper,

00:29:02 should definitely read your book.

00:29:04 But which of these challenges is solved in your view

00:29:08 has the biggest impact on the AI community?

00:29:11 It’s a very good question.

00:29:13 And I’m gonna be evasive because I think that

00:29:16 they go together a lot.

00:29:17 So some of them might be solved independently of others,

00:29:21 but I think a good solution to AI

00:29:23 starts by having real,

00:29:25 what I would call cognitive models of what’s going on.

00:29:28 So right now we have a approach that’s dominant

00:29:31 where you take statistical approximations of things,

00:29:33 but you don’t really understand them.

00:29:35 So you know that bottles are correlated in your data

00:29:39 with bottle caps,

00:29:40 but you don’t understand that there’s a thread

00:29:42 on the bottle cap that fits with the thread on the bottle

00:29:45 and then that’s what tightens it.

00:29:46 If I tighten enough that there’s a seal

00:29:48 and the water won’t come out.

00:29:49 Like there’s no machine that understands that.

00:29:51 And having a good cognitive model

00:29:53 of that kind of everyday phenomena

00:29:55 is what we call common sense.

00:29:56 And if you had that,

00:29:57 then a lot of these other things start to fall

00:30:00 into at least a little bit better place.

00:30:02 Right now you’re like learning correlations between pixels

00:30:05 when you play a video game or something like that.

00:30:07 And it doesn’t work very well.

00:30:08 It works when the video game is just the way

00:30:10 that you studied it and then you alter the video game

00:30:12 in small ways,

00:30:13 like you move the paddle and break out a few pixels

00:30:15 and the system falls apart.

00:30:17 Because it doesn’t understand,

00:30:19 it doesn’t have a representation of a paddle,

00:30:20 a ball, a wall, a set of bricks and so forth.

00:30:23 And so it’s reasoning at the wrong level.

00:30:26 So the idea of common sense,

00:30:29 it’s full of mystery,

00:30:30 you’ve worked on it,

00:30:31 but it’s nevertheless full of mystery,

00:30:33 full of promise.

00:30:34 What does common sense mean?

00:30:36 What does knowledge mean?

00:30:38 So the way you’ve been discussing it now

00:30:40 is very intuitive.

00:30:40 It makes a lot of sense that that is something

00:30:42 we should have and that’s something

00:30:43 deep learning systems don’t have.

00:30:45 But the argument could be that we’re oversimplifying it

00:30:49 because we’re oversimplifying the notion of common sense

00:30:53 because that’s how it feels like we as humans

00:30:57 at the cognitive level approach problems.

00:30:59 So maybe.

00:31:00 A lot of people aren’t actually gonna read my book.

00:31:03 But if they did read the book,

00:31:05 one of the things that might come as a surprise to them

00:31:07 is that we actually say common sense is really hard

00:31:10 and really complicated.

00:31:11 So they would probably,

00:31:13 my critics know that I like common sense,

00:31:15 but that chapter actually starts by us beating up

00:31:18 not on deep learning,

00:31:19 but kind of on our own home team as it will.

00:31:21 So Ernie and I are first and foremost

00:31:25 people that believe in at least some

00:31:26 of what good old fashioned AI tried to do.

00:31:28 So we believe in symbols and logic and programming.

00:31:32 Things like that are important.

00:31:33 And we go through why even those tools

00:31:37 that we hold fairly dear aren’t really enough.

00:31:39 So we talk about why common sense is actually many things.

00:31:42 And some of them fit really well with those

00:31:45 classical sets of tools.

00:31:46 So things like taxonomy.

00:31:48 So I know that a bottle is an object

00:31:51 or it’s a vessel, let’s say.

00:31:52 And I know a vessel is an object

00:31:54 and objects are material things in the physical world.

00:31:57 So I can make some inferences.

00:32:00 If I know that vessels need to not have holes in them,

00:32:07 then I can infer that in order to carry their contents,

00:32:09 then I can infer that a bottle

00:32:10 shouldn’t have a hole in it in order to carry its contents.

00:32:12 So you can do hierarchical inference and so forth.

00:32:15 And we say that’s great,

00:32:17 but it’s only a tiny piece of what you need for common sense.

00:32:21 We give lots of examples that don’t fit into that.

00:32:23 So another one that we talk about is a cheese grater.

00:32:26 You’ve got holes in a cheese grater.

00:32:28 You’ve got a handle on top.

00:32:29 You can build a model in the game engine sense of a model

00:32:33 so that you could have a little cartoon character

00:32:35 flying around through the holes of the grater.

00:32:37 But we don’t have a system yet.

00:32:39 Taxonomy doesn’t help us that much

00:32:41 that really understands why the handle is on top

00:32:43 and what you do with the handle,

00:32:45 or why all of those circles are sharp,

00:32:47 or how you’d hold the cheese with respect to the grater

00:32:50 in order to make it actually work.

00:32:52 Do you think these ideas are just abstractions

00:32:55 that could emerge on a system

00:32:57 like a very large deep neural network?

00:32:59 I’m a skeptic that that kind of emergence per se can work.

00:33:03 So I think that deep learning might play a role

00:33:05 in the systems that do what I want systems to do,

00:33:08 but it won’t do it by itself.

00:33:09 I’ve never seen a deep learning system

00:33:13 really extract an abstract concept.

00:33:15 What they do, principled reasons for that

00:33:18 stemming from how back propagation works,

00:33:20 how the architectures are set up.

00:33:22 One example is deep learning people

00:33:25 actually all build in something called convolution,

00:33:29 which Jan Lacune is famous for, which is an abstraction.

00:33:33 They don’t have their systems learn this.

00:33:34 So the abstraction is an object that looks the same

00:33:37 if it appears in different places.

00:33:39 And what Lacune figured out and why,

00:33:41 essentially why he was a co winner of the Turing Award

00:33:44 was that if you programmed this in innately,

00:33:47 then your system would be a whole lot more efficient.

00:33:50 In principle, this should be learnable,

00:33:53 but people don’t have systems that kind of reify things

00:33:56 and make them more abstract.

00:33:58 And so what you’d really wind up with

00:34:00 if you don’t program that in advance is a system

00:34:02 that kind of realizes that this is the same thing as this,

00:34:05 but then I take your little clock there

00:34:06 and I move it over and it doesn’t realize

00:34:08 that the same thing applies to the clock.

00:34:10 So the really nice thing, you’re right,

00:34:12 that convolution is just one of the things

00:34:14 that’s like, it’s an innate feature

00:34:17 that’s programmed by the human expert.

00:34:19 We need more of those, not less.

00:34:21 Yes, but the nice feature is it feels like

00:34:24 that requires coming up with that brilliant idea,

00:34:28 can get you a Turing Award,

00:34:29 but it requires less effort than encoding

00:34:34 and something we’ll talk about, the expert system.

00:34:36 So encoding a lot of knowledge by hand.

00:34:40 So it feels like there’s a huge amount of limitations

00:34:43 which you clearly outline with deep learning,

00:34:46 but the nice feature of deep learning,

00:34:47 whatever it is able to accomplish,

00:34:49 it does a lot of stuff automatically

00:34:53 without human intervention.

00:34:54 Well, and that’s part of why people love it, right?

00:34:57 But I always think of this quote from Bertrand Russell,

00:34:59 which is it has all the advantages

00:35:02 of theft over honest toil.

00:35:04 It’s really hard to program into a machine

00:35:08 a notion of causality or even how a bottle works

00:35:11 or what containers are.

00:35:12 Ernie Davis and I wrote a, I don’t know,

00:35:14 45 page academic paper trying just to understand

00:35:17 what a container is,

00:35:18 which I don’t think anybody ever read the paper,

00:35:21 but it’s a very detailed analysis of all the things,

00:35:25 well, not even all of it,

00:35:26 some of the things you need to do

00:35:27 in order to understand a container.

00:35:28 It would be a whole lot nice,

00:35:30 and I’m a coauthor on the paper,

00:35:32 I made it a little bit better,

00:35:33 but Ernie did the hard work for that particular paper.

00:35:36 And it took him like three months

00:35:38 to get the logical statements correct.

00:35:40 And maybe that’s not the right way to do it,

00:35:42 it’s a way to do it.

00:35:44 But on that way of doing it,

00:35:46 it’s really hard work to do something

00:35:48 as simple as understanding containers.

00:35:50 And nobody wants to do that hard work,

00:35:52 even Ernie didn’t want to do that hard work.

00:35:55 Everybody would rather just like feed their system in

00:35:58 with a bunch of videos with a bunch of containers

00:36:00 and have the systems infer how containers work.

00:36:03 It would be like so much less effort,

00:36:05 let the machine do the work.

00:36:06 And so I understand the impulse,

00:36:08 I understand why people want to do that.

00:36:10 I just don’t think that it works.

00:36:11 I’ve never seen anybody build a system

00:36:14 that in a robust way can actually watch videos

00:36:18 and predict exactly which containers would leak

00:36:21 and which ones wouldn’t or something like,

00:36:23 and I know someone’s gonna go out and do that

00:36:25 since I said it, and I look forward to seeing it.

00:36:28 But getting these things to work robustly

00:36:30 is really, really hard.

00:36:32 So Yann LeCun, who was my colleague at NYU for many years,

00:36:37 thinks that the hard work should go into defining

00:36:40 an unsupervised learning algorithm

00:36:43 that will watch videos, use the next frame basically

00:36:46 in order to tell it what’s going on.

00:36:48 And he thinks that’s the Royal road

00:36:49 and he’s willing to put in the work

00:36:51 in devising that algorithm.

00:36:53 Then he wants the machine to do the rest.

00:36:55 And again, I understand the impulse.

00:36:57 My intuition, based on years of watching this stuff

00:37:01 and making predictions 20 years ago that still hold

00:37:03 even though there’s a lot more computation and so forth,

00:37:06 is that we actually have to do

00:37:07 a different kind of hard work,

00:37:08 which is more like building a design specification

00:37:11 for what we want the system to do,

00:37:13 doing hard engineering work to figure out

00:37:15 how we do things like what Yann did for convolution

00:37:18 in order to figure out how to encode complex knowledge

00:37:21 into the systems.

00:37:22 The current systems don’t have that much knowledge

00:37:25 other than convolution, which is again,

00:37:27 this objects being in different places

00:37:30 and having the same perception, I guess I’ll say.

00:37:34 Same appearance.

00:37:36 People don’t want to do that work.

00:37:38 They don’t see how to naturally fit one with the other.

00:37:41 I think that’s, yes, absolutely.

00:37:43 But also on the expert system side,

00:37:45 there’s a temptation to go too far the other way.

00:37:47 So we’re just having an expert sort of sit down

00:37:49 and encode the description,

00:37:51 the framework for what a container is,

00:37:54 and then having the system reason the rest.

00:37:56 From my view, one really exciting possibility

00:37:59 is of active learning where it’s continuous interaction

00:38:02 between a human and machine.

00:38:04 As the machine, there’s kind of deep learning type

00:38:07 extraction of information from data patterns and so on,

00:38:10 but humans also guiding the learning procedures,

00:38:14 guiding both the process and the framework

00:38:19 of how the machine learns, whatever the task is.

00:38:22 I was with you with almost everything you said

00:38:24 except the phrase deep learning.

00:38:26 What I think you really want there

00:38:28 is a new form of machine learning.

00:38:30 So let’s remember, deep learning is a particular way

00:38:32 of doing machine learning.

00:38:33 Most often it’s done with supervised data

00:38:36 for perceptual categories.

00:38:38 There are other things you can do with deep learning,

00:38:41 some of them quite technical,

00:38:42 but the standard use of deep learning

00:38:44 is I have a lot of examples and I have labels for them.

00:38:47 So here are pictures.

00:38:48 This one’s the Eiffel Tower.

00:38:50 This one’s the Sears Tower.

00:38:51 This one’s the Empire State Building.

00:38:53 This one’s a cat.

00:38:54 This one’s a pig and so forth.

00:38:55 You just get millions of examples, millions of labels,

00:38:58 and deep learning is extremely good at that.

00:39:01 It’s better than any other solution that anybody has devised,

00:39:04 but it is not good at representing abstract knowledge.

00:39:07 It’s not good at representing things

00:39:09 like bottles contain liquid and have tops to them

00:39:13 and so forth.

00:39:14 It’s not very good at learning

00:39:15 or representing that kind of knowledge.

00:39:17 It is an example of having a machine learn something,

00:39:21 but it’s a machine that learns a particular kind of thing,

00:39:23 which is object classification.

00:39:25 It’s not a particularly good algorithm for learning

00:39:28 about the abstractions that govern our world.

00:39:30 There may be such a thing.

00:39:33 Part of what we counsel in the book

00:39:34 is maybe people should be working on devising such things.

00:39:36 So one possibility, just I wonder what you think about it,

00:39:40 is that deep neural networks do form abstractions,

00:39:45 but they’re not accessible to us humans

00:39:48 in terms of we can’t.

00:39:49 There’s some truth in that.

00:39:50 So is it possible that either current or future

00:39:54 neural networks form very high level abstractions,

00:39:56 which are as powerful as our human abstractions

00:40:01 of common sense.

00:40:02 We just can’t get a hold of them.

00:40:04 And so the problem is essentially

00:40:06 we need to make them explainable.

00:40:09 This is an astute question,

00:40:10 but I think the answer is at least partly no.

00:40:13 One of the kinds of classical neural network architecture

00:40:16 is what we call an auto associator.

00:40:17 It just tries to take an input,

00:40:20 goes through a set of hidden layers,

00:40:21 and comes out with an output.

00:40:23 And it’s supposed to learn essentially

00:40:24 the identity function,

00:40:25 that your input is the same as your output.

00:40:27 So you think of it as binary numbers.

00:40:28 You’ve got the one, the two, the four, the eight,

00:40:30 the 16, and so forth.

00:40:32 And so if you want to input 24,

00:40:33 you turn on the 16, you turn on the eight.

00:40:35 It’s like binary one, one, and a bunch of zeros.

00:40:38 So I did some experiments in 1998

00:40:41 with the precursors of contemporary deep learning.

00:40:46 And what I showed was you could train these networks

00:40:50 on all the even numbers,

00:40:52 and they would never generalize to the odd number.

00:40:54 A lot of people thought that I was, I don’t know,

00:40:56 an idiot or faking the experiment,

00:40:58 or it wasn’t true or whatever.

00:41:00 But it is true that with this class of networks

00:41:03 that we had in that day,

00:41:04 that they would never ever make this generalization.

00:41:07 And it’s not that the networks were stupid,

00:41:09 it’s that they see the world in a different way than we do.

00:41:13 They were basically concerned,

00:41:14 what is the probability that the rightmost output node

00:41:18 is going to be one?

00:41:19 And as far as they were concerned,

00:41:21 in everything they’d ever been trained on, it was a zero.

00:41:24 That node had never been turned on,

00:41:27 and so they figured, why turn it on now?

00:41:28 Whereas a person would look at the same problem and say,

00:41:30 well, it’s obvious,

00:41:31 we’re just doing the thing that corresponds.

00:41:33 The Latin for it is mutatis mutandis,

00:41:35 we’ll change what needs to be changed.

00:41:38 And we do this, this is what algebra is.

00:41:40 So I can do f of x equals y plus two,

00:41:43 and I can do it for a couple of values,

00:41:45 I can tell you if y is three,

00:41:46 then x is five, and if y is four, x is six.

00:41:49 And now I can do it with some totally different number,

00:41:50 like a million, then you can say,

00:41:51 well, obviously it’s a million and two,

00:41:53 because you have an algebraic operation

00:41:55 that you’re applying to a variable.

00:41:57 And deep learning systems kind of emulate that,

00:42:00 but they don’t actually do it.

00:42:02 The particular example,

00:42:04 you could fudge a solution to that particular problem.

00:42:08 The general form of that problem remains,

00:42:10 that what they learn is really correlations

00:42:12 between different input and output nodes.

00:42:14 And they’re complex correlations

00:42:16 with multiple nodes involved and so forth.

00:42:18 Ultimately, they’re correlative,

00:42:20 they’re not structured over these operations over variables.

00:42:23 Now, someday, people may do a new form of deep learning

00:42:25 that incorporates that stuff,

00:42:27 and I think it will help a lot.

00:42:28 And there’s some tentative work on things

00:42:30 like differentiable programming right now

00:42:32 that fall into that category.

00:42:34 But the sort of classic stuff

00:42:35 like people use for ImageNet doesn’t have it.

00:42:38 And you have people like Hinton going around saying,

00:42:41 symbol manipulation, like what Marcus,

00:42:42 what I advocate is like the gasoline engine.

00:42:45 It’s obsolete.

00:42:46 We should just use this cool electric power

00:42:48 that we’ve got with the deep learning.

00:42:50 And that’s really destructive,

00:42:51 because we really do need to have the gasoline engine stuff

00:42:55 that represents, I mean, I don’t think it’s a good analogy,

00:42:59 but we really do need to have the stuff

00:43:02 that represents symbols.

00:43:03 Yeah, and Hinton as well would say

00:43:06 that we do need to throw out everything and start over.

00:43:08 Hinton said that to Axios,

00:43:12 and I had a friend who interviewed him

00:43:15 and tried to pin him down

00:43:16 on what exactly we need to throw out,

00:43:17 and he was very evasive.

00:43:19 Well, of course, because we can’t, if he knew.

00:43:22 Then he’d throw it out himself.

00:43:23 But I mean, you can’t have it both ways.

00:43:25 You can’t be like, I don’t know what to throw out,

00:43:27 but I am gonna throw out the symbols.

00:43:29 I mean, and not just the symbols,

00:43:32 but the variables and the operations over variables.

00:43:34 Don’t forget, the operations over variables,

00:43:36 the stuff that I’m endorsing

00:43:37 and which John McCarthy did when he founded AI,

00:43:41 that stuff is the stuff

00:43:42 that we build most computers out of.

00:43:44 There are people now who say,

00:43:45 we don’t need computer programmers anymore.

00:43:48 Not quite looking at the statistics

00:43:50 of how much computer programmers

00:43:51 actually get paid right now.

00:43:52 We need lots of computer programmers,

00:43:54 and most of them, they do a little bit of machine learning,

00:43:57 but they still do a lot of code, right?

00:43:59 Code where it’s like, if the value of X

00:44:02 is greater than the value of Y,

00:44:03 then do this kind of thing,

00:44:04 like conditionals and comparing operations over variables.

00:44:08 Like, there’s this fantasy you can machine learn anything.

00:44:10 There’s some things you would never wanna machine learn.

00:44:12 I would not use a phone operating system

00:44:14 that was machine learned.

00:44:16 Like, you made a bunch of phone calls

00:44:17 and you recorded which packets were transmitted

00:44:19 and you just machine learned it, it’d be insane.

00:44:22 Or to build a web browser by taking logs of keystrokes

00:44:27 and images, screenshots,

00:44:29 and then trying to learn the relation between them.

00:44:31 Nobody would ever,

00:44:32 no rational person would ever try to build a browser

00:44:35 that made, they would use symbol manipulation,

00:44:37 the stuff that I think AI needs to avail itself of

00:44:40 in addition to deep learning.

00:44:42 Can you describe your view of symbol manipulation

00:44:46 in its early days?

00:44:47 Can you describe expert systems

00:44:49 and where do you think they hit a wall

00:44:52 or a set of challenges?

00:44:53 Sure, so I mean, first I just wanna clarify,

00:44:56 I’m not endorsing expert systems per se.

00:44:58 You’ve been kind of contrasting them.

00:45:00 There is a contrast,

00:45:01 but that’s not the thing that I’m endorsing.

00:45:04 So expert systems tried to capture things

00:45:06 like medical knowledge with a large set of rules.

00:45:09 So if the patient has this symptom and this other symptom,

00:45:12 then it is likely that they have this disease.

00:45:15 So there are logical rules

00:45:16 and they were symbol manipulating rules

00:45:18 of just the sort that I’m talking about.

00:45:20 And the problem.

00:45:21 They encode a set of knowledge that the experts then put in.

00:45:24 And very explicitly so.

00:45:26 So you’d have somebody interview an expert

00:45:28 and then try to turn that stuff into rules.

00:45:31 And at some level I’m arguing for rules.

00:45:33 But the difference is those guys did in the 80s

00:45:37 was almost entirely rules,

00:45:40 almost entirely handwritten with no machine learning.

00:45:42 What a lot of people are doing now

00:45:44 is almost entirely one species of machine learning

00:45:47 with no rules.

00:45:48 And what I’m counseling is actually a hybrid.

00:45:50 I’m saying that both of these things have their advantage.

00:45:52 So if you’re talking about perceptual classification,

00:45:55 how do I recognize a bottle?

00:45:57 Deep learning is the best tool we’ve got right now.

00:45:59 If you’re talking about making inferences

00:46:00 about what a bottle does,

00:46:02 something closer to the expert systems

00:46:04 is probably still the best available alternative.

00:46:07 And probably we want something that is better able

00:46:09 to handle quantitative and statistical information

00:46:12 than those classical systems typically were.

00:46:14 So we need new technologies

00:46:16 that are gonna draw some of the strengths

00:46:18 of both the expert systems and the deep learning,

00:46:21 but are gonna find new ways to synthesize them.

00:46:23 How hard do you think it is to add knowledge at the low level?

00:46:27 So mine human intellects to add extra information

00:46:32 to symbol manipulating systems?

00:46:36 In some domains it’s not that hard,

00:46:37 but it’s often really hard.

00:46:40 Partly because a lot of the things that are important,

00:46:44 people wouldn’t bother to tell you.

00:46:46 So if you pay someone on Amazon Mechanical Turk

00:46:49 to tell you stuff about bottles,

00:46:52 they probably won’t even bother to tell you

00:46:55 some of the basic level stuff

00:46:57 that’s just so obvious to a human being

00:46:59 and yet so hard to capture in machines.

00:47:04 They’re gonna tell you more exotic things,

00:47:06 and they’re all well and good,

00:47:08 but they’re not getting to the root of the problem.

00:47:12 So untutored humans aren’t very good at knowing,

00:47:16 and why should they be,

00:47:18 what kind of knowledge the computer system developers

00:47:22 actually need?

00:47:23 I don’t think that that’s an irremediable problem.

00:47:26 I think it’s historically been a problem.

00:47:28 People have had crowdsourcing efforts,

00:47:31 and they don’t work that well.

00:47:32 There’s one at MIT, we’re recording this at MIT,

00:47:35 called Virtual Home, where,

00:47:37 and we talk about this in the book,

00:47:39 find the exact example there,

00:47:40 but people were asked to do things

00:47:42 like describe an exercise routine.

00:47:44 And the things that the people describe

00:47:47 are at a very low level

00:47:48 and don’t really capture what’s going on.

00:47:50 So they’re like, go to the room

00:47:52 with the television and the weights,

00:47:54 turn on the television,

00:47:56 press the remote to turn on the television,

00:47:59 lift weight, put weight down, whatever.

00:48:01 It’s like very micro level,

00:48:03 and it’s not telling you

00:48:04 what an exercise routine is really about,

00:48:06 which is like, I wanna fit a certain number of exercises

00:48:09 in a certain time period,

00:48:10 I wanna emphasize these muscles.

00:48:12 You want some kind of abstract description.

00:48:15 The fact that you happen to press the remote control

00:48:17 in this room when you watch this television

00:48:20 isn’t really the essence of the exercise routine.

00:48:23 But if you just ask people like, what did they do?

00:48:24 Then they give you this fine grain.

00:48:26 And so it takes a level of expertise

00:48:29 about how the AI works

00:48:31 in order to craft the right kind of knowledge.

00:48:34 So there’s this ocean of knowledge that we all operate on.

00:48:37 Some of them may not even be conscious,

00:48:39 or at least we’re not able to communicate it effectively.

00:48:43 Yeah, most of it we would recognize if somebody said it,

00:48:45 if it was true or not,

00:48:47 but we wouldn’t think to say that it’s true or not.

00:48:49 That’s a really interesting mathematical property.

00:48:53 This ocean has the property

00:48:54 that every piece of knowledge in it,

00:48:56 we will recognize it as true if we’re told,

00:48:59 but we’re unlikely to retrieve it in the reverse.

00:49:04 So that interesting property,

00:49:07 I would say there’s a huge ocean of that knowledge.

00:49:10 What’s your intuition?

00:49:11 Is it accessible to AI systems somehow?

00:49:14 Can we?

00:49:15 So you said this.

00:49:16 I mean, most of it is not,

00:49:18 well, I’ll give you an asterisk on this in a second,

00:49:20 but most of it has not ever been encoded

00:49:23 in machine interpretable form.

00:49:25 And so, I mean, if you say accessible,

00:49:27 there’s two meanings of that.

00:49:28 One is like, could you build it into a machine?

00:49:31 Yes.

00:49:32 The other is like, is there some database

00:49:34 that we could go download and stick into our machine?

00:49:38 But the first thing, could we?

00:49:40 What’s your intuition? I think we could.

00:49:42 I think it hasn’t been done right.

00:49:45 You know, the closest, and this is the asterisk,

00:49:47 is the CYC psych system tried to do this.

00:49:51 A lot of logicians worked for Doug Lennon

00:49:53 for 30 years on this project.

00:49:55 I think they stuck too closely to logic,

00:49:57 didn’t represent enough about probabilities,

00:50:00 tried to hand code it.

00:50:01 There are various issues,

00:50:02 and it hasn’t been that successful.

00:50:04 That is the closest existing system

00:50:08 to trying to encode this.

00:50:10 Why do you think there’s not more excitement

00:50:13 slash money behind this idea currently?

00:50:16 There was.

00:50:17 People view that project as a failure.

00:50:19 I think that they confuse the failure

00:50:22 of a specific instance that was conceived 30 years ago

00:50:25 for the failure of an approach,

00:50:26 which they don’t do for deep learning.

00:50:28 So in 2010, people had the same attitude

00:50:31 towards deep learning.

00:50:32 They’re like, this stuff doesn’t really work.

00:50:35 And all these other algorithms work better and so forth.

00:50:39 And then certain key technical advances were made,

00:50:41 but mostly it was the advent

00:50:43 of graphics processing units that changed that.

00:50:46 It wasn’t even anything foundational in the techniques.

00:50:50 And there was some new tricks,

00:50:51 but mostly it was just more compute and more data,

00:50:55 things like ImageNet that didn’t exist before

00:50:57 that allowed deep learning.

00:50:59 And it could be, to work,

00:51:00 it could be that CYC just needs a few more things

00:51:03 or something like CYC,

00:51:05 but the widespread view is that that just doesn’t work.

00:51:08 And people are reasoning from a single example.

00:51:11 They don’t do that with deep learning.

00:51:13 They don’t say nothing that existed in 2010,

00:51:16 and there were many, many efforts in deep learning

00:51:18 was really worth anything.

00:51:20 I mean, really, there’s no model from 2010

00:51:23 in deep learning or the predecessors of deep learning

00:51:26 that has any commercial value whatsoever at this point.

00:51:29 They’re all failures.

00:51:31 But that doesn’t mean that there wasn’t anything there.

00:51:33 I have a friend, I was getting to know him,

00:51:35 and he said, I had a company too,

00:51:38 I was talking about I had a new company.

00:51:40 He said, I had a company too, and it failed.

00:51:42 And I said, well, what did you do?

00:51:44 And he said, deep learning.

00:51:45 And the problem was he did it in 1986

00:51:47 or something like that.

00:51:48 And we didn’t have the tools then, or 1990,

00:51:51 we didn’t have the tools then, not the algorithms.

00:51:53 His algorithms weren’t that different from model algorithms,

00:51:56 but he didn’t have the GPUs to run it fast enough.

00:51:58 He didn’t have the data.

00:51:59 And so it failed.

00:52:01 It could be that symbol manipulation per se

00:52:06 with modern amounts of data and compute

00:52:09 and maybe some advance in compute

00:52:11 for that kind of compute might be great.

00:52:14 My perspective on it is not that we want to resuscitate

00:52:19 that stuff per se, but we want to borrow lessons from it,

00:52:21 bring together with other things that we’ve learned.

00:52:23 And it might have an ImageNet moment

00:52:25 where it would spark the world’s imagination

00:52:28 and there’ll be an explosion of symbol manipulation efforts.

00:52:31 Yeah, I think that people at AI2,

00:52:33 Paul Allen’s AI Institute, are trying to build data sets.

00:52:39 Well, they’re not doing it

00:52:39 for quite the reason that you say,

00:52:41 but they’re trying to build data sets

00:52:43 that at least spark interest in common sense reasoning.

00:52:45 To create benchmarks.

00:52:46 Benchmarks for common sense.

00:52:48 That’s a large part of what the AI2.org

00:52:50 is working on right now.

00:52:51 So speaking of compute,

00:52:53 Rich Sutton wrote a blog post titled Bitter Lesson.

00:52:56 I don’t know if you’ve read it,

00:52:57 but he said that the biggest lesson that can be read

00:52:59 from so many years of AI research

00:53:01 is that general methods that leverage computation

00:53:04 are ultimately the most effective.

00:53:06 Do you think that?

00:53:07 The most effective at what?

00:53:08 Right, so they have been most effective

00:53:11 for perceptual classification problems

00:53:14 and for some reinforcement learning problems.

00:53:18 And he works on reinforcement learning.

00:53:19 Well, no, let me push back on that.

00:53:20 You’re actually absolutely right.

00:53:22 But I would also say they have been most effective generally

00:53:28 because everything we’ve done up to…

00:53:31 Would you argue against that?

00:53:32 Is, to me, deep learning is the first thing

00:53:36 that has been successful at anything in AI.

00:53:41 And you’re pointing out that this success

00:53:45 is very limited, folks,

00:53:47 but has there been something truly successful

00:53:50 before deep learning?

00:53:51 Sure, I mean, I want to make a larger point,

00:53:54 but on the narrower point, classical AI is used,

00:54:00 for example, in doing navigation instructions.

00:54:03 It’s very successful.

00:54:06 Everybody on the planet uses it now,

00:54:07 like multiple times a day.

00:54:09 That’s a measure of success, right?

00:54:12 So I don’t think classical AI was wildly successful,

00:54:16 but there are cases like that.

00:54:17 They’re just used all the time.

00:54:19 Nobody even notices them because they’re so pervasive.

00:54:23 So there are some successes for classical AI.

00:54:26 I think deep learning has been more successful,

00:54:28 but my usual line about this, and I didn’t invent it,

00:54:32 but I like it a lot,

00:54:33 is just because you can build a better ladder

00:54:34 doesn’t mean you can build a ladder to the moon.

00:54:37 So the bitter lesson is if you have

00:54:39 a perceptual classification problem,

00:54:42 throwing a lot of data at it is better than anything else.

00:54:45 But that has not given us any material progress

00:54:49 in natural language understanding,

00:54:51 common sense reasoning,

00:54:53 like a robot would need to navigate a home.

00:54:56 Problems like that, there’s no actual progress there.

00:54:59 So flip side of that, if we remove data from the picture,

00:55:02 another bitter lesson is that you just have

00:55:05 a very simple algorithm,

00:55:10 and you wait for compute to scale.

00:55:12 It doesn’t have to be learning.

00:55:13 It doesn’t have to be deep learning.

00:55:14 It doesn’t have to be data driven,

00:55:16 but just wait for the compute.

00:55:18 So my question for you,

00:55:19 do you think compute can unlock some of the things

00:55:21 with either deep learning or symbol manipulation that?

00:55:25 Sure, but I’ll put a proviso on that.

00:55:29 I think more compute’s always better.

00:55:31 Nobody’s gonna argue with more compute.

00:55:33 It’s like having more money.

00:55:34 I mean, there’s the data.

00:55:36 There’s diminishing returns on more money.

00:55:37 Exactly, there’s diminishing returns on more money,

00:55:39 but nobody’s gonna argue

00:55:40 if you wanna give them more money, right?

00:55:42 Except maybe the people who signed the giving pledge,

00:55:44 and some of them have a problem.

00:55:46 They’ve promised to give away more money

00:55:47 than they’re able to.

00:55:49 But the rest of us, if you wanna give me more money, fine.

00:55:52 I’m saying more money, more problems, but okay.

00:55:54 That’s true too.

00:55:55 What I would say to you is your brain uses like 20 watts,

00:56:00 and it does a lot of things that deep learning doesn’t do,

00:56:02 or that symbol manipulation doesn’t do,

00:56:05 that AI just hasn’t figured out how to do.

00:56:07 So it’s an existence proof

00:56:09 that you don’t need server resources

00:56:12 that are Google scale in order to have an intelligence.

00:56:16 I built, with a lot of help from my wife,

00:56:18 two intelligences that are 20 watts each,

00:56:21 and far exceed anything that anybody else

00:56:25 has built at a silicon.

00:56:26 Speaking of those two robots,

00:56:30 what have you learned about AI from having?

00:56:33 Well, they’re not robots, but.

00:56:35 Sorry, intelligent agents.

00:56:36 Those two intelligent agents.

00:56:38 I’ve learned a lot by watching my two intelligent agents.

00:56:42 I think that what’s fundamentally interesting,

00:56:45 well, one of the many things

00:56:46 that’s fundamentally interesting about them

00:56:48 is the way that they set their own problems to solve.

00:56:51 So my two kids are a year and a half apart.

00:56:54 They’re both five and six and a half.

00:56:56 They play together all the time,

00:56:58 and they’re constantly creating new challenges.

00:57:00 That’s what they do, is they make up games,

00:57:03 and they’re like, well, what if this, or what if that,

00:57:05 or what if I had this superpower,

00:57:07 or what if you could walk through this wall?

00:57:10 So they’re doing these what if scenarios all the time,

00:57:14 and that’s how they learn something about the world

00:57:17 and grow their minds, and machines don’t really do that.

00:57:22 So that’s interesting, and you’ve talked about this,

00:57:24 you’ve written about it, you’ve thought about it,

00:57:26 nature versus nurture.

00:57:29 So what innate knowledge do you think we’re born with,

00:57:33 and what do we learn along the way

00:57:35 in those early months and years?

00:57:38 Can I just say how much I like that question?

00:57:41 You phrased it just right, and almost nobody ever does,

00:57:45 which is what is the innate knowledge

00:57:47 and what’s learned along the way?

00:57:49 So many people dichotomize it,

00:57:51 and they think it’s nature versus nurture,

00:57:53 when it is obviously has to be nature and nurture.

00:57:56 They have to work together.

00:57:58 You can’t learn this stuff along the way

00:58:00 unless you have some innate stuff,

00:58:02 but just because you have the innate stuff

00:58:03 doesn’t mean you don’t learn anything.

00:58:05 And so many people get that wrong, including in the field.

00:58:09 People think if I work in machine learning,

00:58:12 the learning side, I must not be allowed to work

00:58:15 on the innate side, or that will be cheating.

00:58:17 Exactly, people have said that to me,

00:58:19 and it’s just absurd, so thank you.

00:58:23 But you could break that apart more.

00:58:25 I’ve talked to folks who studied

00:58:26 the development of the brain,

00:58:28 and the growth of the brain in the first few days

00:58:32 in the first few months in the womb,

00:58:35 all of that, is that innate?

00:58:39 So that process of development from a stem cell

00:58:42 to the growth of the central nervous system and so on,

00:58:46 to the information that’s encoded

00:58:49 through the long arc of evolution.

00:58:52 So all of that comes into play, and it’s unclear.

00:58:55 It’s not just whether it’s a dichotomy or not.

00:58:57 It’s where most, or where the knowledge is encoded.

00:59:02 So what’s your intuition about the innate knowledge,

00:59:07 the power of it, what’s contained in it,

00:59:09 what can we learn from it?

00:59:11 One of my earlier books was actually trying

00:59:12 to understand the biology of this.

00:59:14 The book was called The Birth of the Mind.

00:59:15 Like how is it the genes even build innate knowledge?

00:59:18 And from the perspective of the conversation

00:59:21 we’re having today, there’s actually two questions.

00:59:23 One is what innate knowledge or mechanisms,

00:59:26 or what have you, people or other animals

00:59:29 might be endowed with.

00:59:30 I always like showing this video

00:59:32 of a baby ibex climbing down a mountain.

00:59:34 That baby ibex, a few hours after its birth,

00:59:37 knows how to climb down a mountain.

00:59:38 That means that it knows, not consciously,

00:59:40 something about its own body and physics

00:59:43 and 3D geometry and all of this kind of stuff.

00:59:47 So there’s one question about what does biology

00:59:49 give its creatures and what has evolved in our brains?

00:59:53 How is that represented in our brains?

00:59:54 The question I thought about in the book

00:59:56 The Birth of the Mind.

00:59:57 And then there’s a question of what AI should have.

00:59:59 And they don’t have to be the same.

01:00:01 But I would say that it’s a pretty interesting

01:00:06 set of things that we are equipped with

01:00:08 that allows us to do a lot of interesting things.

01:00:10 So I would argue or guess, based on my reading

01:00:13 of the developmental psychology literature,

01:00:15 which I’ve also participated in,

01:00:17 that children are born with a notion of space,

01:00:21 time, other agents, places,

01:00:25 and also this kind of mental algebra

01:00:27 that I was describing before.

01:00:30 No certain causation if I didn’t just say that.

01:00:33 So at least those kinds of things.

01:00:35 They’re like frameworks for learning the other things.

01:00:38 Are they disjoint in your view

01:00:40 or is it just somehow all connected?

01:00:42 You’ve talked a lot about language.

01:00:44 Is it all kind of connected in some mesh

01:00:47 that’s language like?

01:00:50 If understanding concepts all together or?

01:00:52 I don’t think we know for people how they’re represented

01:00:55 and machines just don’t really do this yet.

01:00:58 So I think it’s an interesting open question

01:01:00 both for science and for engineering.

01:01:03 Some of it has to be at least interrelated

01:01:06 in the way that the interfaces of a software package

01:01:10 have to be able to talk to one another.

01:01:12 So the systems that represent space and time

01:01:16 can’t be totally disjoint because a lot of the things

01:01:19 that we reason about are the relations

01:01:21 between space and time and cause.

01:01:22 So I put this on and I have expectations

01:01:26 about what’s gonna happen with the bottle cap

01:01:28 on top of the bottle and those span space and time.

01:01:32 If the cap is over here, I get a different outcome.

01:01:35 If the timing is different, if I put this here,

01:01:38 after I move that, then I get a different outcome.

01:01:41 That relates to causality.

01:01:43 So obviously these mechanisms, whatever they are,

01:01:47 can certainly communicate with each other.

01:01:50 So I think evolution had a significant role

01:01:53 to play in the development of this whole kluge, right?

01:01:57 How efficient do you think is evolution?

01:01:59 Oh, it’s terribly inefficient except that.

01:02:01 Okay, well, can we do better?

01:02:03 Well, I’ll come to that in a sec.

01:02:05 It’s inefficient except that.

01:02:08 Once it gets a good idea, it runs with it.

01:02:10 So it took, I guess, a billion years,

01:02:15 if I went roughly a billion years, to evolve

01:02:20 to a vertebrate brain plan.

01:02:24 And once that vertebrate brain plan evolved,

01:02:26 it spread everywhere.

01:02:28 So fish have it and dogs have it and we have it.

01:02:31 We have adaptations of it and specializations of it,

01:02:34 but, and the same thing with a primate brain plan.

01:02:37 So monkeys have it and apes have it and we have it.

01:02:41 So there are additional innovations like color vision

01:02:43 and those spread really rapidly.

01:02:45 So it takes evolution a long time to get a good idea,

01:02:48 but, and I’m being anthropomorphic and not literal here,

01:02:53 but once it has that idea, so to speak,

01:02:55 which cashes out into one set of genes or in the genome,

01:02:58 those genes spread very rapidly

01:03:00 and they’re like subroutines or libraries,

01:03:02 I guess the word people might use nowadays

01:03:04 or be more familiar with.

01:03:05 They’re libraries that get used over and over again.

01:03:08 So once you have the library for building something

01:03:11 with multiple digits, you can use it for a hand,

01:03:13 but you can also use it for a foot.

01:03:15 You just kind of reuse the library

01:03:17 with slightly different parameters.

01:03:19 Evolution does a lot of that,

01:03:20 which means that the speed over time picks up.

01:03:23 So evolution can happen faster

01:03:25 because you have bigger and bigger libraries.

01:03:28 And what I think has happened in attempts

01:03:32 at evolutionary computation is that people start

01:03:35 with libraries that are very, very minimal,

01:03:40 like almost nothing, and then progress is slow

01:03:44 and it’s hard for someone to get a good PhD thesis

01:03:46 out of it and they give up.

01:03:48 If we had richer libraries to begin with,

01:03:50 if you were evolving from systems

01:03:52 that had an rich innate structure to begin with,

01:03:55 then things might speed up.

01:03:56 Or more PhD students, if the evolutionary process

01:03:59 is indeed in a meta way runs away with good ideas,

01:04:04 you need to have a lot of ideas,

01:04:06 pool of ideas in order for it to discover one

01:04:08 that you can run away with.

01:04:10 And PhD students representing individual ideas as well.

01:04:13 Yeah, I mean, you could throw

01:04:14 a billion PhD students at it.

01:04:16 Yeah, the monkeys are typewriters with Shakespeare, yep.

01:04:20 Well, I mean, those aren’t cumulative, right?

01:04:22 That’s just random.

01:04:23 And part of the point that I’m making

01:04:24 is that evolution is cumulative.

01:04:26 So if you have a billion monkeys independently,

01:04:31 you don’t really get anywhere.

01:04:32 But if you have a billion monkeys,

01:04:33 and I think Dawkins made this point originally,

01:04:35 or probably other people, Dawkins made it very nice

01:04:37 and either a selfish gene or blind watchmaker.

01:04:40 If there is some sort of fitness function

01:04:44 that can drive you towards something,

01:04:45 I guess that’s Dawkins point.

01:04:47 And my point, which is a variation on that,

01:04:49 is that if the evolution is cumulative,

01:04:51 I mean, the related points,

01:04:53 then you can start going faster.

01:04:55 Do you think something like the process of evolution

01:04:57 is required to build intelligent systems?

01:05:00 So if we… Not logically.

01:05:01 So all the stuff that evolution did,

01:05:04 a good engineer might be able to do.

01:05:07 So for example, evolution made quadrupeds,

01:05:10 which distribute the load across a horizontal surface.

01:05:14 A good engineer could come up with that idea.

01:05:16 I mean, sometimes good engineers come up with ideas

01:05:18 by looking at biology.

01:05:19 There’s lots of ways to get your ideas.

01:05:22 Part of what I’m suggesting

01:05:23 is we should look at biology a lot more.

01:05:25 We should look at the biology of thought and understanding

01:05:30 and the biology by which creatures intuitively reason

01:05:33 about physics or other agents,

01:05:35 or like how do dogs reason about people?

01:05:37 Like they’re actually pretty good at it.

01:05:39 If we could understand, at my college we joked dognition,

01:05:44 if we could understand dognition well,

01:05:46 and how it was implemented, that might help us with our AI.

01:05:49 So do you think it’s possible

01:05:53 that the kind of timescale that evolution took

01:05:57 is the kind of timescale that will be needed

01:05:58 to build intelligent systems?

01:06:00 Or can we significantly accelerate that process

01:06:02 inside a computer?

01:06:04 I mean, I think the way that we accelerate that process

01:06:07 is we borrow from biology, not slavishly,

01:06:12 but I think we look at how biology has solved problems

01:06:15 and we say, does that inspire

01:06:16 any engineering solutions here?

01:06:18 Try to mimic biological systems

01:06:20 and then therefore have a shortcut.

01:06:22 Yeah, I mean, there’s a field called biomimicry

01:06:25 and people do that for like material science all the time.

01:06:28 We should be doing the analog of that for AI

01:06:32 and the analog for that for AI

01:06:34 is to look at cognitive science or the cognitive sciences,

01:06:37 which is psychology, maybe neuroscience, linguistics,

01:06:40 and so forth, look to those for insight.

01:06:43 What do you think is a good test of intelligence

01:06:45 in your view?

01:06:46 So I don’t think there’s one good test.

01:06:48 In fact, I tried to organize a movement

01:06:51 towards something called a Turing Olympics

01:06:53 and my hope is that Francois is actually gonna take,

01:06:56 Francois Chollet is gonna take over this.

01:06:58 I think he’s interested and I don’t,

01:06:59 I just don’t have place in my busy life at this moment,

01:07:03 but the notion is that there’d be many tests

01:07:06 and not just one because intelligence is multifaceted.

01:07:09 There can’t really be a single measure of it

01:07:12 because it isn’t a single thing.

01:07:15 Like just the crudest level,

01:07:17 the SAT has a verbal component and a math component

01:07:19 because they’re not identical.

01:07:21 And Howard Gardner has talked about multiple intelligences

01:07:23 like kinesthetic intelligence

01:07:25 and verbal intelligence and so forth.

01:07:27 There are a lot of things that go into intelligence

01:07:29 and people can get good at one or the other.

01:07:32 I mean, in some sense, like every expert has developed

01:07:35 a very specific kind of intelligence

01:07:37 and then there are people that are generalists

01:07:39 and I think of myself as a generalist

01:07:41 with respect to cognitive science,

01:07:43 which doesn’t mean I know anything about quantum mechanics,

01:07:45 but I know a lot about the different facets of the mind.

01:07:49 And there’s a kind of intelligence

01:07:51 to thinking about intelligence.

01:07:52 I like to think that I have some of that,

01:07:54 but social intelligence, I’m just okay.

01:07:57 There are people that are much better at that than I am.

01:08:00 Sure, but what would be really impressive to you?

01:08:04 I think the idea of a touring Olympics is really interesting

01:08:07 especially if somebody like Francois is running it,

01:08:09 but to you in general, not as a benchmark,

01:08:14 but if you saw an AI system being able to accomplish

01:08:17 something that would impress the heck out of you,

01:08:21 what would that thing be?

01:08:22 Would it be natural language conversation?

01:08:24 For me personally, I would like to see

01:08:28 a kind of comprehension that relates to what you just said.

01:08:30 So I wrote a piece in the New Yorker in I think 2015

01:08:34 right after Eugene Guestman, which was a software package,

01:08:39 won a version of the Turing test.

01:08:42 And the way that it did this is it be,

01:08:45 well, the way you win the Turing test,

01:08:46 so called win it, is the Turing test is you fool a person

01:08:50 into thinking that a machine is a person,

01:08:54 is you’re evasive, you pretend to have limitations

01:08:57 so you don’t have to answer certain questions and so forth.

01:09:00 So this particular system pretended to be a 13 year old boy

01:09:04 from Odessa who didn’t understand English

01:09:06 and was kind of sarcastic

01:09:08 and wouldn’t answer your questions and so forth.

01:09:09 And so judges got fooled into thinking briefly

01:09:12 with a very little exposure, it was a 13 year old boy,

01:09:14 and it docked all the questions

01:09:16 Turing was actually interested in,

01:09:17 which is like how do you make the machine

01:09:18 actually intelligent?

01:09:20 So that test itself is not that good.

01:09:22 And so in New Yorker, I proposed an alternative, I guess,

01:09:26 and the one that I proposed there

01:09:27 was a comprehension test.

01:09:30 And I must like Breaking Bad

01:09:31 because I’ve already given you one Breaking Bad example

01:09:32 and in that article, I have one as well,

01:09:35 which was something like if Walter,

01:09:37 you should be able to watch an episode of Breaking Bad

01:09:40 or maybe you have to watch the whole series

01:09:41 to be able to answer the question and say,

01:09:43 if Walter White took a hit out on Jesse,

01:09:45 why did he do that?

01:09:47 So if you could answer kind of arbitrary questions

01:09:49 about characters motivations, I would be really impressed

01:09:52 with that and he built software to do that.

01:09:55 They could watch a film or there are different versions.

01:09:58 And so ultimately, I wrote this up with Praveen Paritosh

01:10:01 in a special issue of AI Magazine

01:10:04 that basically was about the Turing Olympics.

01:10:05 There were like 14 tests proposed.

01:10:07 The one that I was pushing was a comprehension challenge

01:10:10 and Praveen who’s at Google was trying to figure out

01:10:12 like how we would actually run it

01:10:13 and so we wrote a paper together.

01:10:15 And you could have a text version too

01:10:17 or you could have an auditory podcast version,

01:10:19 you could have a written version.

01:10:20 But the point is that you win at this test

01:10:23 if you can do, let’s say human level or better than humans

01:10:27 at answering kind of arbitrary questions.

01:10:29 Why did this person pick up the stone?

01:10:31 What were they thinking when they picked up the stone?

01:10:34 Were they trying to knock down glass?

01:10:36 And I mean, ideally these wouldn’t be multiple choice either

01:10:38 because multiple choice is pretty easily gamed.

01:10:41 So if you could have relatively open ended questions

01:10:44 and you can answer why people are doing this stuff,

01:10:47 I would be very impressed.

01:10:48 And of course, humans can do this, right?

01:10:50 If you watch a well constructed movie

01:10:52 and somebody picks up a rock,

01:10:55 everybody watching the movie

01:10:56 knows why they picked up the rock, right?

01:10:59 They all know, oh my gosh,

01:11:01 he’s gonna hit this character or whatever.

01:11:03 We have an example in the book about

01:11:06 when a whole bunch of people say, I am Spartacus,

01:11:08 you know, this famous scene.

01:11:11 The viewers understand,

01:11:13 first of all, that everybody or everybody minus one

01:11:18 has to be lying.

01:11:19 They can’t all be Spartacus.

01:11:20 We have enough common sense knowledge

01:11:21 to know they couldn’t all have the same name.

01:11:24 We know that they’re lying

01:11:25 and we can infer why they’re lying, right?

01:11:27 They’re lying to protect someone

01:11:28 and to protect things they believe in.

01:11:30 You get a machine that can do that.

01:11:32 They can say, this is why these guys all got up

01:11:35 and said, I am Spartacus.

01:11:36 I will sit down and say, AI has really achieved a lot.

01:11:40 Thank you.

01:11:41 Without cheating any part of the system.

01:11:43 Yeah, I mean, if you do it,

01:11:45 there are lots of ways you could cheat.

01:11:46 You could build a Spartacus machine

01:11:48 that works on that film.

01:11:50 That’s not what I’m talking about.

01:11:51 I’m talking about, you can do this

01:11:52 with essentially arbitrary films

01:11:54 or from a large set. Even beyond films

01:11:56 because it’s possible such a system would discover

01:11:58 that the number of narrative arcs in film

01:12:02 is limited to 1930. Well, there’s a famous thing

01:12:04 about the classic seven plots or whatever.

01:12:07 I don’t care.

01:12:07 If you wanna build in the system,

01:12:09 boy meets girl, boy loses girl, boy finds girl.

01:12:11 That’s fine.

01:12:12 I don’t mind having some head stories on it.

01:12:13 And they acknowledge.

01:12:14 Okay, good.

01:12:16 I mean, you could build it in innately

01:12:17 or you could have your system watch a lot of films again.

01:12:20 If you can do this at all,

01:12:22 but with a wide range of films,

01:12:23 not just one film in one genre.

01:12:27 But even if you could do it for all Westerns,

01:12:28 I’d be reasonably impressed.

01:12:30 Yeah.

01:12:31 So in terms of being impressed,

01:12:34 just for the fun of it,

01:12:35 because you’ve put so many interesting ideas out there

01:12:38 in your book,

01:12:40 challenging the community for further steps.

01:12:43 Is it possible on the deep learning front

01:12:46 that you’re wrong about its limitations?

01:12:50 That deep learning will unlock,

01:12:52 Yann LeCun next year will publish a paper

01:12:54 that achieves this comprehension.

01:12:56 So do you think that way often as a scientist?

01:13:00 Do you consider that your intuition

01:13:03 that deep learning could actually run away with it?

01:13:06 I’m more worried about rebranding

01:13:09 as a kind of political thing.

01:13:11 So, I mean, what’s gonna happen, I think,

01:13:14 is the deep learning is gonna start

01:13:15 to encompass symbol manipulation.

01:13:17 So I think Hinton’s just wrong.

01:13:19 Hinton says we don’t want hybrids.

01:13:20 I think people will work towards hybrids

01:13:22 and they will relabel their hybrids as deep learning.

01:13:24 We’ve already seen some of that.

01:13:25 So AlphaGo is often described as a deep learning system,

01:13:29 but it’s more correctly described as a system

01:13:31 that has deep learning, but also Monte Carlo tree search,

01:13:33 which is a classical AI technique.

01:13:35 And people will start to blur the lines

01:13:37 in the way that IBM blurred Watson.

01:13:39 First, Watson meant this particular system,

01:13:41 and then it was just anything that IBM built

01:13:43 in their cognitive division.

01:13:44 But purely, let me ask, for sure,

01:13:45 that’s a branding question and that’s like a giant mess.

01:13:49 I mean, purely, a single neural network

01:13:51 being able to accomplish reasonable comprehension.

01:13:54 I don’t stay up at night worrying

01:13:55 that that’s gonna happen.

01:13:57 And I’ll just give you two examples.

01:13:59 One is a guy at DeepMind thought he had finally outfoxed me.

01:14:03 At Zergilord, I think is his Twitter handle.

01:14:06 And he said, he specifically made an example.

01:14:10 Marcus said that such and such.

01:14:12 He fed it into GP2, which is the AI system

01:14:16 that is so smart that OpenAI couldn’t release it

01:14:19 because it would destroy the world, right?

01:14:21 You remember that a few months ago.

01:14:22 So he feeds it into GPT2, and my example

01:14:27 was something like a rose is a rose,

01:14:28 a tulip is a tulip, a lily is a blank.

01:14:31 And he got it to actually do that,

01:14:32 which was a little bit impressive.

01:14:34 And I wrote back and I said, that’s impressive,

01:14:35 but can I ask you a few questions?

01:14:37 I said, was that just one example?

01:14:40 Can it do it generally?

01:14:41 And can it do it with novel words,

01:14:43 which was part of what I was talking about in 1998

01:14:45 when I first raised the example.

01:14:46 So a dax is a dax, right?

01:14:50 And he sheepishly wrote back about 20 minutes later.

01:14:53 And the answer was, well, it had some problems with those.

01:14:55 So I made some predictions 21 years ago that still hold.

01:15:00 In the world of computer science, that’s amazing, right?

01:15:02 Because there’s a thousand or a million times more memory

01:15:06 and computations a million times,

01:15:10 do million times more operations per second

01:15:13 spread across a cluster.

01:15:15 And there’s been advances in replacing sigmoids

01:15:20 with other functions and so forth.

01:15:23 There’s all kinds of advances,

01:15:25 but the fundamental architecture hasn’t changed

01:15:27 and the fundamental limit hasn’t changed.

01:15:28 And what I said then is kind of still true.

01:15:30 Then here’s a second example.

01:15:32 I recently had a piece in Wired

01:15:34 that’s adapted from the book.

01:15:35 And the book went to press before GP2 came out,

01:15:40 but we described this children’s story

01:15:42 and all the inferences that you make in this story

01:15:45 about a boy finding a lost wallet.

01:15:48 And for fun, in the Wired piece, we ran it through GP2.

01:15:52 GPT2, something called talktotransformer.com,

01:15:55 and your viewers can try this experiment themselves.

01:15:58 Go to the Wired piece that has the link

01:15:59 and it has the story.

01:16:01 And the system made perfectly fluent text

01:16:04 that was totally inconsistent

01:16:06 with the conceptual underpinnings of the story, right?

01:16:10 This is what, again, I predicted in 1998.

01:16:13 And for that matter, Chomsky and Miller

01:16:14 made the same prediction in 1963.

01:16:16 I was just updating their claim for a slightly new text.

01:16:19 So those particular architectures

01:16:22 that don’t have any built in knowledge,

01:16:24 they’re basically just a bunch of layers

01:16:27 doing correlational stuff.

01:16:28 They’re not gonna solve these problems.

01:16:31 So 20 years ago, you said the emperor has no clothes.

01:16:34 Today, the emperor still has no clothes.

01:16:36 The lighting’s better though.

01:16:38 The lighting is better.

01:16:39 And I think you yourself are also, I mean.

01:16:42 And we found out some things to do with naked emperors.

01:16:44 I mean, it’s not like stuff is worthless.

01:16:46 I mean, they’re not really naked.

01:16:48 It’s more like they’re in their briefs

01:16:49 than everybody thinks they are.

01:16:50 And so like, I mean, they are great at speech recognition,

01:16:54 but the problems that I said were hard.

01:16:56 I didn’t literally say the emperor has no clothes.

01:16:58 I said, this is a set of problems

01:17:00 that humans are really good at.

01:17:01 And it wasn’t couched as AI.

01:17:03 It was couched as cognitive science.

01:17:04 But I said, if you wanna build a neural model

01:17:07 of how humans do certain class of things,

01:17:10 you’re gonna have to change the architecture.

01:17:11 And I stand by those claims.

01:17:13 So, and I think people should understand

01:17:16 you’re quite entertaining in your cynicism,

01:17:19 but you’re also very optimistic and a dreamer

01:17:22 about the future of AI too.

01:17:23 So you’re both, it’s just.

01:17:25 There’s a famous saying about being,

01:17:27 people overselling technology in the short run

01:17:30 and underselling it in the long run.

01:17:34 And so I actually end the book,

01:17:37 Ernie Davis and I end our book with an optimistic chapter,

01:17:40 which kind of killed Ernie

01:17:41 because he’s even more pessimistic than I am.

01:17:44 He describes me as a contrarian and him as a pessimist.

01:17:47 But I persuaded him that we should end the book

01:17:49 with a look at what would happen

01:17:52 if AI really did incorporate, for example,

01:17:55 the common sense reasoning and the nativism

01:17:57 and so forth, the things that we counseled for.

01:17:59 And we wrote it and it’s an optimistic chapter

01:18:02 that AI suitably reconstructed so that we could trust it,

01:18:05 which we can’t now, could really be world changing.

01:18:09 So on that point, if you look at the future trajectories

01:18:13 of AI, people have worries about negative effects of AI,

01:18:17 whether it’s at the large existential scale

01:18:21 or smaller short term scale of negative impact on society.

01:18:25 So you write about trustworthy AI,

01:18:27 how can we build AI systems that align with our values,

01:18:31 that make for a better world,

01:18:32 that we can interact with, that we can trust?

01:18:34 The first thing we have to do

01:18:35 is to replace deep learning with deep understanding.

01:18:38 So you can’t have alignment with a system

01:18:42 that traffics only in correlations

01:18:44 and doesn’t understand concepts like bottles or harm.

01:18:47 So Asimov talked about these famous laws

01:18:51 and the first one was first do no harm.

01:18:54 And you can quibble about the details of Asimov’s laws,

01:18:56 but we have to, if we’re gonna build real robots

01:18:58 in the real world, have something like that.

01:19:00 That means we have to program in a notion

01:19:02 that’s at least something like harm.

01:19:04 That means we have to have these more abstract ideas

01:19:06 that deep learning is not particularly good at.

01:19:08 They have to be in the mix somewhere.

01:19:10 And you could do statistical analysis

01:19:12 about probabilities of given harms or whatever,

01:19:14 but you have to know what a harm is

01:19:15 in the same way that you have to understand

01:19:17 that a bottle isn’t just a collection of pixels.

01:19:20 And also be able to, you’re implying

01:19:24 that you need to also be able to communicate

01:19:25 that to humans so the AI systems would be able

01:19:29 to prove to humans that they understand

01:19:33 that they know what harm means.

01:19:35 I might run it in the reverse direction,

01:19:37 but roughly speaking, I agree with you.

01:19:38 So we probably need to have committees

01:19:42 of wise people, ethicists and so forth.

01:19:45 Think about what these rules ought to be

01:19:47 and we shouldn’t just leave it to software engineers.

01:19:49 It shouldn’t just be software engineers

01:19:51 and it shouldn’t just be people

01:19:53 who own large mega corporations

01:19:56 that are good at technology, ethicists

01:19:58 and so forth should be involved.

01:20:00 But there should be some assembly of wise people

01:20:04 as I was putting it that tries to figure out

01:20:07 what the rules ought to be.

01:20:08 And those have to get translated into code.

01:20:12 You can argue or code or neural networks or something.

01:20:15 They have to be translated into something

01:20:18 that machines can work with.

01:20:19 And that means there has to be a way

01:20:21 of working the translation.

01:20:23 And right now we don’t.

01:20:24 We don’t have a way.

01:20:25 So let’s say you and I were the committee

01:20:27 and we decide that Asimov’s first law is actually right.

01:20:29 And let’s say it’s not just two white guys,

01:20:31 which would be kind of unfortunate that we have abroad.

01:20:34 And so we’ve representative sample of the world

01:20:36 or however we wanna do this.

01:20:37 And the committee decides eventually,

01:20:40 okay, Asimov’s first law is actually pretty good.

01:20:42 There are these exceptions to it.

01:20:44 We wanna program in these exceptions.

01:20:46 But let’s start with just the first one

01:20:47 and then we’ll get to the exceptions.

01:20:48 First one is first do no harm.

01:20:50 Well, somebody has to now actually turn that into

01:20:53 a computer program or a neural network or something.

01:20:56 And one way of taking the whole book,

01:20:58 the whole argument that I’m making

01:21:00 is that we just don’t have to do that yet.

01:21:02 And we’re fooling ourselves

01:21:03 if we think that we can build trustworthy AI

01:21:05 if we can’t even specify in any kind of,

01:21:09 we can’t do it in Python and we can’t do it in TensorFlow.

01:21:13 We’re fooling ourselves in thinking

01:21:14 that we can make trustworthy AI

01:21:15 if we can’t translate harm into something

01:21:18 that we can execute.

01:21:19 And if we can’t, then we should be thinking really hard

01:21:22 how could we ever do such a thing?

01:21:24 Because if we’re gonna use AI

01:21:26 in the ways that we wanna use it,

01:21:27 to make job interviews or to do surveillance,

01:21:31 not that I personally wanna do that or whatever.

01:21:32 I mean, if we’re gonna use AI

01:21:33 in ways that have practical impact on people’s lives

01:21:36 or medicine, it’s gotta be able

01:21:38 to understand stuff like that.

01:21:41 So one of the things your book highlights

01:21:42 is that a lot of people in the deep learning community,

01:21:47 but also the general public, politicians,

01:21:50 just people in all general groups and walks of life

01:21:53 have different levels of misunderstanding of AI.

01:21:57 So when you talk about committees,

01:22:00 what’s your advice to our society?

01:22:05 How do we grow, how do we learn about AI

01:22:08 such that such committees could emerge

01:22:10 where large groups of people could have

01:22:13 a productive discourse about

01:22:15 how to build successful AI systems?

01:22:17 Part of the reason we wrote the book

01:22:19 was to try to inform those committees.

01:22:22 So part of the reason we wrote the book

01:22:23 was to inspire a future generation of students

01:22:25 to solve what we think are the important problems.

01:22:27 So a lot of the book is trying to pinpoint

01:22:29 what we think are the hard problems

01:22:31 where we think effort would most be rewarded.

01:22:34 And part of it is to try to train people

01:22:37 who talk about AI, but aren’t experts in the field

01:22:41 to understand what’s realistic and what’s not.

01:22:43 One of my favorite parts in the book

01:22:44 is the six questions you should ask

01:22:46 anytime you read a media account.

01:22:48 So like number one is if somebody talks about something,

01:22:51 look for the demo.

01:22:51 If there’s no demo, don’t believe it.

01:22:54 Like the demo that you can try.

01:22:55 If you can’t try it at home,

01:22:56 maybe it doesn’t really work that well yet.

01:22:58 So if, we don’t have this example in the book,

01:23:00 but if Sundar Pinchai says we have this thing

01:23:04 that allows it to sound like human beings in conversation,

01:23:08 you should ask, can I try it?

01:23:10 And you should ask how general it is.

01:23:11 And it turns out at that time,

01:23:13 I’m alluding to Google Duplex when it was announced,

01:23:15 it only worked on calling hairdressers,

01:23:18 restaurants and finding opening hours.

01:23:20 That’s not very general, that’s narrow AI.

01:23:22 And I’m not gonna ask your thoughts about Sophia,

01:23:24 but yeah, I understand that’s a really good question

01:23:27 to ask of any kind of hype top idea.

01:23:30 Sophia has very good material written for her,

01:23:32 but she doesn’t understand the things that she’s saying.

01:23:35 So a while ago you’ve written a book

01:23:38 on the science of learning, which I think is fascinating,

01:23:40 but the learning case studies of playing guitar.

01:23:43 That’s called Guitar Zero.

01:23:45 I love guitar myself, I’ve been playing my whole life.

01:23:47 So let me ask a very important question.

01:23:50 What is your favorite song, rock song,

01:23:53 to listen to or try to play?

01:23:56 Well, those would be different,

01:23:57 but I’ll say that my favorite rock song to listen to

01:23:59 is probably All Along the Watchtower,

01:24:01 the Jimi Hendrix version.

01:24:01 The Jimi Hendrix version.

01:24:02 It feels magic to me.

01:24:04 I’ve actually recently learned it, I love that song.

01:24:07 I’ve been trying to put it on YouTube, myself singing.

01:24:09 Singing is the scary part.

01:24:11 If you could party with a rock star for a weekend,

01:24:13 living or dead, who would you choose?

01:24:17 And pick their mind, it’s not necessarily about the partying.

01:24:21 Thanks for the clarification.

01:24:24 I guess John Lennon’s such an intriguing person,

01:24:26 and I think a troubled person, but an intriguing one.

01:24:31 Beautiful.

01:24:32 Well, Imagine is one of my favorite songs.

01:24:35 Also one of my favorite songs.

01:24:37 That’s a beautiful way to end it.

01:24:38 Gary, thank you so much for talking to me.

01:24:39 Thanks so much for having me.