Transcript
00:00:00 The following is a conversation with Jeff Hawkins.
00:00:02 He’s the founder of the Redwood Center
00:00:04 for Theoretical Neuroscience in 2002, and NuMenta in 2005.
00:00:08 In his 2004 book, titled On Intelligence,
00:00:11 and in the research before and after,
00:00:13 he and his team have worked to reverse engineer
00:00:16 the neural cortex, and propose artificial intelligence
00:00:19 architectures, approaches, and ideas
00:00:21 that are inspired by the human brain.
00:00:23 These ideas include Hierarchical Tupperware Memory,
00:00:25 HTM, from 2004, and new work,
00:00:28 the Thousand Brains Theory of Intelligence
00:00:30 from 2017, 18, and 19.
00:00:33 Jeff’s ideas have been an inspiration
00:00:36 to many who have looked for progress
00:00:38 beyond the current machine learning approaches,
00:00:40 but they have also received criticism
00:00:42 for lacking a body of empirical evidence
00:00:44 supporting the models.
00:00:46 This is always a challenge when seeking more
00:00:48 than small incremental steps forward in AI.
00:00:51 Jeff is a brilliant mind, and many of the ideas
00:00:54 he has developed and aggregated from neuroscience
00:00:56 are worth understanding and thinking about.
00:00:59 There are limits to deep learning,
00:01:00 as it is currently defined.
00:01:02 Forward progress in AI is shrouded in mystery.
00:01:05 My hope is that conversations like this
00:01:07 can help provide an inspiring spark for new ideas.
00:01:11 This is the Artificial Intelligence Podcast.
00:01:14 If you enjoy it, subscribe on YouTube, iTunes,
00:01:16 or simply connect with me on Twitter
00:01:18 at Lex Friedman, spelled F R I D.
00:01:21 And now, here’s my conversation with Jeff Hawkins.
00:01:26 Are you more interested in understanding the human brain
00:01:29 or in creating artificial systems
00:01:32 that have many of the same qualities
00:01:34 but don’t necessarily require that you actually understand
00:01:38 the underpinning workings of our mind?
00:01:41 So there’s a clear answer to that question.
00:01:44 My primary interest is understanding the human brain.
00:01:46 No question about it.
00:01:47 But I also firmly believe that we will not be able
00:01:53 to create fully intelligent machines
00:01:55 until we understand how the human brain works.
00:01:57 So I don’t see those as separate problems.
00:02:00 I think there’s limits to what can be done
00:02:01 with machine intelligence if you don’t understand
00:02:03 the principles by which the brain works.
00:02:05 And so I actually believe that studying the brain
00:02:07 is actually the fastest way to get to machine intelligence.
00:02:11 And within that, let me ask the impossible question,
00:02:14 how do you, not define, but at least think
00:02:17 about what it means to be intelligent?
00:02:19 So I didn’t try to answer that question first.
00:02:22 We said, let’s just talk about how the brain works
00:02:24 and let’s figure out how certain parts of the brain,
00:02:26 mostly the neocortex, but some other parts too.
00:02:29 The parts of the brain most associated with intelligence.
00:02:32 And let’s discover the principles by how they work.
00:02:35 Because intelligence isn’t just like some mechanism
00:02:39 and it’s not just some capabilities.
00:02:40 It’s like, okay, we don’t even know
00:02:42 where to begin on this stuff.
00:02:44 And so now that we’ve made a lot of progress on this,
00:02:49 after we’ve made a lot of progress
00:02:50 on how the neocortex works, and we can talk about that,
00:02:53 I now have a very good idea what’s gonna be required
00:02:55 to make intelligent machines.
00:02:57 I can tell you today, some of the things
00:02:59 are gonna be necessary, I believe,
00:03:02 to create intelligent machines.
00:03:03 Well, so we’ll get there.
00:03:04 We’ll get to the neocortex and some of the theories
00:03:07 of how the whole thing works.
00:03:09 And you’re saying, as we understand more and more
00:03:11 about the neocortex, about our own human mind,
00:03:14 we’ll be able to start to more specifically define
00:03:17 what it means to be intelligent.
00:03:18 It’s not useful to really talk about that until.
00:03:21 I don’t know if it’s not useful.
00:03:23 Look, there’s a long history of AI, as you know.
00:03:26 And there’s been different approaches taken to it.
00:03:28 And who knows, maybe they’re all useful.
00:03:32 So the good old fashioned AI, the expert systems,
00:03:37 the current convolutional neural networks,
00:03:38 they all have their utility.
00:03:40 They all have a value in the world.
00:03:43 But I would think almost everyone agree
00:03:45 that none of them are really intelligent
00:03:46 in a sort of a deep way that humans are.
00:03:49 And so it’s just the question of how do you get
00:03:53 from where those systems were or are today
00:03:56 to where a lot of people think we’re gonna go.
00:03:59 And there’s a big, big gap there, a huge gap.
00:04:02 And I think the quickest way of bridging that gap
00:04:06 is to figure out how the brain does that.
00:04:08 And then we can sit back and look and say,
00:04:10 oh, which of these principles that the brain works on
00:04:12 are necessary and which ones are not?
00:04:15 Clearly, we don’t have to build this in,
00:04:16 and intelligent machines aren’t gonna be built
00:04:18 out of organic living cells.
00:04:22 But there’s a lot of stuff that goes on the brain
00:04:24 that’s gonna be necessary.
00:04:25 So let me ask maybe, before we get into the fun details,
00:04:30 let me ask maybe a depressing or a difficult question.
00:04:33 Do you think it’s possible that we will never
00:04:36 be able to understand how our brain works,
00:04:38 that maybe there’s aspects to the human mind,
00:04:41 like we ourselves cannot introspectively get to the core,
00:04:46 that there’s a wall you eventually hit?
00:04:48 Yeah, I don’t believe that’s the case.
00:04:52 I have never believed that’s the case.
00:04:53 There’s not been a single thing humans have ever put
00:04:56 their minds to that we’ve said, oh, we reached the wall,
00:04:58 we can’t go any further.
00:04:59 It’s just, people keep saying that.
00:05:01 People used to believe that about life.
00:05:03 Alain Vital, right, there’s like,
00:05:05 what’s the difference between living matter
00:05:06 and nonliving matter, something special
00:05:07 that we never understand.
00:05:09 We no longer think that.
00:05:10 So there’s no historical evidence that suggests this
00:05:14 is the case, and I just never even consider
00:05:16 that’s a possibility.
00:05:17 I would also say, today, we understand so much
00:05:21 about the neocortex.
00:05:22 We’ve made tremendous progress in the last few years
00:05:25 that I no longer think of it as an open question.
00:05:30 The answers are very clear to me.
00:05:32 The pieces we don’t know are clear to me,
00:05:34 but the framework is all there, and it’s like,
00:05:36 oh, okay, we’re gonna be able to do this.
00:05:38 This is not a problem anymore, just takes time and effort,
00:05:41 but there’s no mystery, a big mystery anymore.
00:05:44 So then let’s get into it for people like myself
00:05:47 who are not very well versed in the human brain,
00:05:52 except my own.
00:05:54 Can you describe to me, at the highest level,
00:05:57 what are the different parts of the human brain,
00:05:59 and then zooming in on the neocortex,
00:06:02 the parts of the neocortex, and so on,
00:06:04 a quick overview.
00:06:05 Yeah, sure.
00:06:06 The human brain, we can divide it roughly into two parts.
00:06:10 There’s the old parts, lots of pieces,
00:06:14 and then there’s the new part.
00:06:15 The new part is the neocortex.
00:06:18 It’s new because it didn’t exist before mammals.
00:06:20 The only mammals have a neocortex,
00:06:22 and in humans, in primates, it’s very large.
00:06:24 In the human brain, the neocortex occupies
00:06:26 about 70 to 75% of the volume of the brain.
00:06:30 It’s huge.
00:06:32 And the old parts of the brain are,
00:06:34 there’s lots of pieces there.
00:06:36 There’s the spinal cord, and there’s the brain stem,
00:06:38 and the cerebellum, and the different parts
00:06:40 of the basal ganglia, and so on.
00:06:42 In the old parts of the brain,
00:06:42 you have the autonomic regulation,
00:06:44 like breathing and heart rate.
00:06:46 You have basic behaviors, so like walking and running
00:06:49 are controlled by the old parts of the brain.
00:06:51 All the emotional centers of the brain
00:06:53 are in the old part of the brain,
00:06:53 so when you feel anger or hungry, lust,
00:06:55 or things like that, those are all
00:06:56 in the old parts of the brain.
00:06:57 And we associate with the neocortex
00:07:02 all the things we think about as sort of
00:07:04 high level perception and cognitive functions,
00:07:08 anything from seeing and hearing and touching things
00:07:12 to language to mathematics and engineering
00:07:15 and science and so on.
00:07:16 Those are all associated with the neocortex,
00:07:19 and they’re certainly correlated.
00:07:21 Our abilities in those regards are correlated
00:07:23 with the relative size of our neocortex
00:07:25 compared to other mammals.
00:07:27 So that’s like the rough division,
00:07:30 and you obviously can’t understand
00:07:32 the neocortex completely isolated,
00:07:35 but you can understand a lot of it
00:07:37 with just a few interfaces to the old parts of the brain,
00:07:40 and so it gives you a system to study.
00:07:44 The other remarkable thing about the neocortex,
00:07:48 compared to the old parts of the brain,
00:07:49 is the neocortex is extremely uniform.
00:07:52 It’s not visibly or anatomically,
00:07:57 it’s very, I always like to say
00:07:59 it’s like the size of a dinner napkin,
00:08:01 about two and a half millimeters thick,
00:08:03 and it looks remarkably the same everywhere.
00:08:05 Everywhere you look in that two and a half millimeters
00:08:07 is this detailed architecture,
00:08:10 and it looks remarkably the same everywhere,
00:08:11 and that’s across species.
00:08:12 A mouse versus a cat and a dog and a human.
00:08:15 Where if you look at the old parts of the brain,
00:08:17 there’s lots of little pieces do specific things.
00:08:19 So it’s like the old parts of our brain evolved,
00:08:22 like this is the part that controls heart rate,
00:08:23 and this is the part that controls this,
00:08:24 and this is this kind of thing,
00:08:25 and that’s this kind of thing,
00:08:27 and these evolved for eons a long, long time,
00:08:30 and they have their specific functions,
00:08:31 and all of a sudden mammals come along,
00:08:33 and they got this thing called the neocortex,
00:08:35 and it got large by just replicating the same thing
00:08:38 over and over and over again.
00:08:39 This is like, wow, this is incredible.
00:08:42 So all the evidence we have,
00:08:46 and this is an idea that was first articulated
00:08:50 in a very cogent and beautiful argument
00:08:52 by a guy named Vernon Malcastle in 1978, I think it was,
00:08:56 that the neocortex all works on the same principle.
00:09:01 So language, hearing, touch, vision, engineering,
00:09:05 all these things are basically underlying,
00:09:06 are all built on the same computational substrate.
00:09:10 They’re really all the same problem.
00:09:11 So the low level of the building blocks all look similar.
00:09:14 Yeah, and they’re not even that low level.
00:09:16 We’re not talking about like neurons.
00:09:17 We’re talking about this very complex circuit
00:09:19 that exists throughout the neocortex.
00:09:21 It’s remarkably similar.
00:09:23 It’s like, yes, you see variations of it here and there,
00:09:26 more of the cell, less and less, and so on.
00:09:29 But what Malcastle argued was, he says,
00:09:32 you know, if you take a section of neocortex,
00:09:35 why is one a visual area and one is a auditory area?
00:09:38 Or why is, and his answer was,
00:09:41 it’s because one is connected to eyes
00:09:43 and one is connected to ears.
00:09:45 Literally, you mean just it’s most closest
00:09:47 in terms of number of connections
00:09:49 to the sensor. Literally, literally,
00:09:50 if you took the optic nerve and attached it
00:09:53 to a different part of the neocortex,
00:09:55 that part would become a visual region.
00:09:57 This actually, this experiment was actually done
00:10:00 by Merkankasur in developing, I think it was lemurs,
00:10:04 I can’t remember what it was, some animal.
00:10:06 And there’s a lot of evidence to this.
00:10:08 You know, if you take a blind person,
00:10:09 a person who’s born blind at birth,
00:10:12 they’re born with a visual neocortex.
00:10:15 It doesn’t, may not get any input from the eyes
00:10:18 because of some congenital defect or something.
00:10:21 And that region becomes, does something else.
00:10:24 It picks up another task.
00:10:27 So, and it’s, so it’s this very complex thing.
00:10:32 It’s not like, oh, they’re all built on neurons.
00:10:33 No, they’re all built in this very complex circuit
00:10:36 and somehow that circuit underlies everything.
00:10:40 And so this is the, it’s called
00:10:43 the common cortical algorithm, if you will.
00:10:45 Some scientists just find it hard to believe
00:10:47 and they just, I can’t believe that’s true,
00:10:50 but the evidence is overwhelming in this case.
00:10:52 And so a large part of what it means
00:10:54 to figure out how the brain creates intelligence
00:10:56 and what is intelligence in the brain
00:10:59 is to understand what that circuit does.
00:11:02 If you can figure out what that circuit does,
00:11:05 as amazing as it is, then you can,
00:11:06 then you understand what all these
00:11:08 other cognitive functions are.
00:11:10 So if you were to sort of put neocortex
00:11:13 outside of your book on intelligence,
00:11:15 you look, if you wrote a giant tome, a textbook
00:11:18 on the neocortex, and you look maybe
00:11:21 a couple of centuries from now,
00:11:23 how much of what we know now would still be accurate
00:11:26 two centuries from now?
00:11:27 So how close are we in terms of understanding?
00:11:30 I have to speak from my own particular experience here.
00:11:32 So I run a small research lab here.
00:11:35 It’s like any other research lab.
00:11:38 I’m sort of the principal investigator.
00:11:39 There’s actually two of us
00:11:40 and there’s a bunch of other people.
00:11:42 And this is what we do.
00:11:43 We study the neocortex and we publish our results
00:11:46 and so on.
00:11:46 So about three years ago,
00:11:49 we had a real breakthrough in this field.
00:11:52 Just tremendous breakthrough.
00:11:53 We’ve now published, I think, three papers on it.
00:11:56 And so I have a pretty good understanding
00:12:00 of all the pieces and what we’re missing.
00:12:02 I would say that almost all the empirical data
00:12:06 we’ve collected about the brain, which is enormous.
00:12:08 If you don’t know the neuroscience literature,
00:12:10 it’s just incredibly big.
00:12:13 And it’s, for the most part, all correct.
00:12:16 It’s facts and experimental results and measurements
00:12:21 and all kinds of stuff.
00:12:22 But none of that has been really assimilated
00:12:25 into a theoretical framework.
00:12:27 It’s data without, in the language of Thomas Kuhn,
00:12:32 the historian, would be a sort of a pre paradigm science.
00:12:35 Lots of data, but no way to fit it together.
00:12:38 I think almost all of that’s correct.
00:12:39 There’s just gonna be some mistakes in there.
00:12:42 And for the most part,
00:12:43 there aren’t really good cogent theories about it,
00:12:45 how to put it together.
00:12:47 It’s not like we have two or three competing good theories,
00:12:50 which ones are right and which ones are wrong.
00:12:51 It’s like, nah, people are just scratching their heads.
00:12:54 Some people have given up
00:12:55 on trying to figure out what the whole thing does.
00:12:57 In fact, there’s very, very few labs that we do
00:13:01 that focus really on theory
00:13:03 and all this unassimilated data and trying to explain it.
00:13:06 So it’s not like we’ve got it wrong.
00:13:08 It’s just that we haven’t got it at all.
00:13:11 So it’s really, I would say, pretty early days
00:13:15 in terms of understanding the fundamental theory’s forces
00:13:19 of the way our mind works.
00:13:20 I don’t think so.
00:13:21 I would have said that’s true five years ago.
00:13:25 So as I said,
00:13:26 we had some really big breakthroughs on this recently
00:13:29 and we started publishing papers on this.
00:13:30 So we’ll get to that.
00:13:34 But so I don’t think it’s,
00:13:35 I’m an optimist and from where I sit today,
00:13:38 most people would disagree with this,
00:13:39 but from where I sit today, from what I know,
00:13:43 it’s not super early days anymore.
00:13:44 We are, the way these things go
00:13:46 is it’s not a linear path, right?
00:13:48 You don’t just start accumulating
00:13:49 and get better and better and better.
00:13:50 No, all this stuff you’ve collected,
00:13:52 none of it makes sense.
00:13:53 All these different things are just sort of around.
00:13:55 And then you’re gonna have some breaking points
00:13:57 where all of a sudden, oh my God, now we got it right.
00:13:59 That’s how it goes in science.
00:14:01 And I personally feel like we passed that little thing
00:14:04 about a couple of years ago,
00:14:06 all that big thing a couple of years ago.
00:14:07 So we can talk about that.
00:14:09 Time will tell if I’m right,
00:14:11 but I feel very confident about it.
00:14:12 That’s why I’m willing to say it on tape like this.
00:14:15 At least very optimistic.
00:14:18 So let’s, before those few years ago,
00:14:20 let’s take a step back to HTM,
00:14:23 the hierarchical temporal memory theory,
00:14:26 which you first proposed on intelligence
00:14:27 and went through a few different generations.
00:14:29 Can you describe what it is,
00:14:31 how it evolved through the three generations
00:14:33 since you first put it on paper?
00:14:35 Yeah, so one of the things that neuroscientists
00:14:39 just sort of missed for many, many years,
00:14:42 and especially people who were thinking about theory,
00:14:45 was the nature of time in the brain.
00:14:49 Brains process information through time.
00:14:51 The information coming into the brain
00:14:52 is constantly changing.
00:14:55 The patterns from my speech right now,
00:14:57 if you were listening to it at normal speed,
00:15:00 would be changing on your ears
00:15:01 about every 10 milliseconds or so, you’d have a change.
00:15:04 This constant flow, when you look at the world,
00:15:06 your eyes are moving constantly,
00:15:08 three to five times a second,
00:15:09 and the input’s completely changing.
00:15:11 If I were to touch something like a coffee cup,
00:15:13 as I move my fingers, the input changes.
00:15:15 So this idea that the brain works on time changing patterns
00:15:19 is almost completely, or was almost completely missing
00:15:22 from a lot of the basic theories,
00:15:23 like fears of vision and so on.
00:15:25 It’s like, oh no, we’re gonna put this image
00:15:26 in front of you and flash it and say, what is it?
00:15:29 Convolutional neural networks work that way today, right?
00:15:32 Classify this picture.
00:15:34 But that’s not what vision is like.
00:15:35 Vision is this sort of crazy time based pattern
00:15:38 that’s going all over the place,
00:15:40 and so is touch and so is hearing.
00:15:41 So the first part of hierarchical temporal memory
00:15:43 was the temporal part.
00:15:45 It’s to say, you won’t understand the brain,
00:15:48 nor will you understand intelligent machines
00:15:50 unless you’re dealing with time based patterns.
00:15:52 The second thing was, the memory component of it was,
00:15:55 is to say that we aren’t just processing input,
00:16:00 we learn a model of the world.
00:16:02 And the memory stands for that model.
00:16:05 The point of the brain, the part of the neocortex,
00:16:07 it learns a model of the world.
00:16:08 We have to store things, our experiences,
00:16:11 in a form that leads to a model of the world.
00:16:14 So we can move around the world,
00:16:15 we can pick things up and do things and navigate
00:16:17 and know how it’s going on.
00:16:18 So that’s what the memory referred to.
00:16:19 And many people just, they were thinking about
00:16:22 like certain processes without memory at all.
00:16:25 They’re just like processing things.
00:16:26 And then finally, the hierarchical component
00:16:29 was a reflection to that the neocortex,
00:16:32 although it’s this uniform sheet of cells,
00:16:35 different parts of it project to other parts,
00:16:37 which project to other parts.
00:16:39 And there is a sort of rough hierarchy in terms of that.
00:16:43 So the hierarchical temporal memory is just saying,
00:16:45 look, we should be thinking about the brain
00:16:47 as time based, model memory based,
00:16:52 and hierarchical processing.
00:16:54 And that was a placeholder for a bunch of components
00:16:58 that we would then plug into that.
00:17:00 We still believe all those things I just said,
00:17:02 but we now know so much more that I’m stopping to use
00:17:06 the word hierarchical temporal memory yet
00:17:08 because it’s insufficient to capture the stuff we know.
00:17:11 So again, it’s not incorrect, but it’s,
00:17:13 I now know more and I would rather describe it
00:17:15 more accurately.
00:17:16 Yeah, so you’re basically, we could think of HTM
00:17:20 as emphasizing that there’s three aspects of intelligence
00:17:24 that are important to think about
00:17:25 whatever the eventual theory it converges to.
00:17:28 So in terms of time, how do you think of nature of time
00:17:32 across different time scales?
00:17:33 So you mentioned things changing,
00:17:36 sensory inputs changing every 10, 20 minutes.
00:17:39 What about every few minutes, every few months and years?
00:17:42 Well, if you think about a neuroscience problem,
00:17:44 the brain problem, neurons themselves can stay active
00:17:49 for certain periods of time, parts of the brain
00:17:52 where they stay active for minutes.
00:17:54 You could hold a certain perception or an activity
00:17:59 for a certain period of time,
00:18:01 but most of them don’t last that long.
00:18:04 And so if you think about your thoughts
00:18:07 are the activity of neurons,
00:18:09 if you’re gonna wanna involve something
00:18:10 that happened a long time ago,
00:18:11 even just this morning, for example,
00:18:14 the neurons haven’t been active throughout that time.
00:18:16 So you have to store that.
00:18:17 So if I ask you, what did you have for breakfast today?
00:18:20 That is memory, that is you’ve built into your model
00:18:23 the world now, you remember that.
00:18:24 And that memory is in the synapses,
00:18:27 is basically in the formation of synapses.
00:18:29 And so you’re sliding into what,
00:18:34 you know, it’s the different timescales.
00:18:36 There’s timescales of which we are like understanding
00:18:39 my language and moving about and seeing things rapidly
00:18:41 and over time, that’s the timescales
00:18:42 of activities of neurons.
00:18:44 But if you wanna get in longer timescales,
00:18:46 then it’s more memory.
00:18:47 And we have to invoke those memories to say,
00:18:49 oh yes, well now I can remember what I had for breakfast
00:18:51 because I stored that someplace.
00:18:54 I may forget it tomorrow, but I’d store it for now.
00:18:58 So does memory also need to have,
00:19:02 so the hierarchical aspect of reality
00:19:06 is not just about concepts, it’s also about time?
00:19:08 Do you think of it that way?
00:19:10 Yeah, time is infused in everything.
00:19:12 It’s like you really can’t separate it out.
00:19:15 If I ask you, what is your, you know,
00:19:18 how’s the brain learn a model of this coffee cup here?
00:19:21 I have a coffee cup and I’m at the coffee cup.
00:19:23 I say, well, time is not an inherent property
00:19:25 of the model I have of this cup,
00:19:28 whether it’s a visual model or a tactile model.
00:19:31 I can sense it through time,
00:19:32 but the model itself doesn’t really have much time.
00:19:34 If I asked you, if I said,
00:19:36 well, what is the model of my cell phone?
00:19:38 My brain has learned a model of the cell phone.
00:19:40 So if you have a smartphone like this,
00:19:43 and I said, well, this has time aspects to it.
00:19:45 I have expectations when I turn it on,
00:19:48 what’s gonna happen, what or how long it’s gonna take
00:19:50 to do certain things, if I bring up an app,
00:19:52 what sequences, and so I have,
00:19:54 and it’s like melodies in the world, you know?
00:19:57 Melody has a sense of time.
00:19:58 So many things in the world move and act,
00:20:01 and there’s a sense of time related to them.
00:20:03 Some don’t, but most things do actually.
00:20:08 So it’s sort of infused throughout the models of the world.
00:20:12 You build a model of the world,
00:20:13 you’re learning the structure of the objects in the world,
00:20:16 and you’re also learning how those things change
00:20:18 through time.
00:20:20 Okay, so it really is just a fourth dimension
00:20:23 that’s infused deeply, and you have to make sure
00:20:26 that your models of intelligence incorporate it.
00:20:30 So, like you mentioned, the state of neuroscience
00:20:34 is deeply empirical, a lot of data collection.
00:20:37 It’s, you know, that’s where it is.
00:20:41 You mentioned Thomas Kuhn, right?
00:20:43 Yeah.
00:20:44 And then you’re proposing a theory of intelligence,
00:20:48 and which is really the next step,
00:20:50 the really important step to take,
00:20:52 but why is HTM, or what we’ll talk about soon,
00:21:00 the right theory?
00:21:03 So is it more in the, is it backed by intuition?
00:21:07 Is it backed by evidence?
00:21:09 Is it backed by a mixture of both?
00:21:11 Is it kind of closer to where string theory is in physics,
00:21:15 where there’s mathematical components
00:21:18 which show that, you know what,
00:21:21 it seems that this, it fits together too well
00:21:24 for it not to be true, which is where string theory is.
00:21:28 Is that where you’re kind of seeing?
00:21:29 It’s a mixture of all those things,
00:21:30 although definitely where we are right now
00:21:32 is definitely much more on the empirical side
00:21:34 than, let’s say, string theory.
00:21:37 The way this goes about, we’re theorists, right?
00:21:39 So we look at all this data, and we’re trying to come up
00:21:41 with some sort of model that explains it, basically,
00:21:44 and there’s, unlike string theory,
00:21:46 there’s vast more amounts of empirical data here
00:21:50 that I think than most physicists deal with.
00:21:54 And so our challenge is to sort through that
00:21:57 and figure out what kind of constructs would explain this.
00:22:02 And when we have an idea,
00:22:04 you come up with a theory of some sort,
00:22:06 you have lots of ways of testing it.
00:22:08 First of all, there are 100 years of assimilated,
00:22:13 assimilated, unassimilated empirical data from neuroscience.
00:22:16 So we go back and read papers,
00:22:18 and we say, oh, did someone find this already?
00:22:20 We can predict X, Y, and Z,
00:22:23 and maybe no one’s even talked about it
00:22:25 since 1972 or something, but we go back and find that,
00:22:28 and we say, oh, either it can support the theory
00:22:31 or it can invalidate the theory.
00:22:33 And we say, okay, we have to start over again.
00:22:34 Oh, no, it’s supportive, let’s keep going with that one.
00:22:38 So the way I kind of view it, when we do our work,
00:22:42 we look at all this empirical data,
00:22:45 and what I call it is a set of constraints.
00:22:47 We’re not interested in something
00:22:48 that’s biologically inspired.
00:22:49 We’re trying to figure out how the actual brain works.
00:22:52 So every piece of empirical data
00:22:53 is a constraint on a theory.
00:22:55 In theory, if you have the correct theory,
00:22:57 it needs to explain every pin, right?
00:22:59 So we have this huge number of constraints on the problem,
00:23:03 which initially makes it very, very difficult.
00:23:05 If you don’t have many constraints,
00:23:07 you can make up stuff all the day.
00:23:08 You can say, oh, here’s an answer on how you can do this,
00:23:10 you can do that, you can do this.
00:23:11 But if you consider all biology as a set of constraints,
00:23:13 all neuroscience as a set of constraints,
00:23:15 and even if you’re working in one little part
00:23:17 of the neocortex, for example,
00:23:18 there are hundreds and hundreds of constraints.
00:23:20 These are empirical constraints
00:23:22 that it’s very, very difficult initially
00:23:24 to come up with a theoretical framework for that.
00:23:27 But when you do, and it solves all those constraints
00:23:30 at once, you have a high confidence
00:23:32 that you got something close to correct.
00:23:35 It’s just mathematically almost impossible not to be.
00:23:39 So that’s the curse and the advantage of what we have.
00:23:43 The curse is we have to solve,
00:23:45 we have to meet all these constraints, which is really hard.
00:23:48 But when you do meet them,
00:23:50 then you have a great confidence
00:23:53 that you’ve discovered something.
00:23:54 In addition, then we work with scientific labs.
00:23:58 So we’ll say, oh, there’s something we can’t find,
00:24:00 we can predict something,
00:24:01 but we can’t find it anywhere in the literature.
00:24:04 So we will then, we have people we’ve collaborated with,
00:24:06 we’ll say, sometimes they’ll say, you know what?
00:24:09 I have some collected data, which I didn’t publish,
00:24:11 but we can go back and look at it
00:24:13 and see if we can find that,
00:24:14 which is much easier than designing a new experiment.
00:24:17 You know, neuroscience experiments take a long time, years.
00:24:20 So, although some people are doing that now too.
00:24:23 So, but between all of these things,
00:24:27 I think it’s a reasonable,
00:24:30 actually a very, very good approach.
00:24:31 We are blessed with the fact that we can test our theories
00:24:35 out the yin yang here because there’s so much
00:24:37 unassimilar data and we can also falsify our theories
00:24:39 very easily, which we do often.
00:24:41 So it’s kind of reminiscent to whenever that was
00:24:44 with Copernicus, you know, when you figure out
00:24:47 that the sun’s at the center of the solar system
00:24:51 as opposed to earth, the pieces just fall into place.
00:24:54 Yeah, I think that’s the general nature of aha moments
00:24:59 is, and it’s Copernicus, it could be,
00:25:02 you could say the same thing about Darwin,
00:25:05 you could say the same thing about, you know,
00:25:07 about the double helix,
00:25:09 that people have been working on a problem for so long
00:25:12 and have all this data and they can’t make sense of it,
00:25:14 they can’t make sense of it.
00:25:15 But when the answer comes to you
00:25:17 and everything falls into place,
00:25:19 it’s like, oh my gosh, that’s it.
00:25:21 That’s got to be right.
00:25:23 I asked both Jim Watson and Francis Crick about this.
00:25:29 I asked them, you know, when you were working on
00:25:31 trying to discover the structure of the double helix,
00:25:35 and when you came up with the sort of the structure
00:25:39 that ended up being correct, but it was sort of a guess,
00:25:44 you know, it wasn’t really verified yet.
00:25:45 I said, did you know that it was right?
00:25:48 And they both said, absolutely.
00:25:50 So we absolutely knew it was right.
00:25:51 And it doesn’t matter if other people didn’t believe it
00:25:54 or not, we knew it was right.
00:25:55 They’d get around to thinking it
00:25:56 and agree with it eventually anyway.
00:25:59 And that’s the kind of thing you hear a lot with scientists
00:26:01 who really are studying a difficult problem.
00:26:04 And I feel that way too about our work.
00:26:07 Have you talked to Crick or Watson about the problem
00:26:10 you’re trying to solve, the, of finding the DNA of the brain?
00:26:15 Yeah, in fact, Francis Crick was very interested in this
00:26:19 in the latter part of his life.
00:26:21 And in fact, I got interested in brains
00:26:23 by reading an essay he wrote in 1979
00:26:26 called Thinking About the Brain.
00:26:28 And that was when I decided I’m gonna leave my profession
00:26:32 of computers and engineering and become a neuroscientist.
00:26:35 Just reading that one essay from Francis Crick.
00:26:37 I got to meet him later in life.
00:26:41 I spoke at the Salk Institute and he was in the audience.
00:26:44 And then I had a tea with him afterwards.
00:26:48 He was interested in a different problem.
00:26:50 He was focused on consciousness.
00:26:53 The easy problem, right?
00:26:54 Well, I think it’s the red herring.
00:26:58 And so we weren’t really overlapping a lot there.
00:27:02 Jim Watson, who’s still alive,
00:27:05 is also interested in this problem.
00:27:07 And he was, when he was director
00:27:09 of the Cold Spring Harbor Laboratories,
00:27:12 he was really sort of behind moving in the direction
00:27:15 of neuroscience there.
00:27:16 And so he had a personal interest in this field.
00:27:20 And I have met with him numerous times.
00:27:23 And in fact, the last time was a little bit over a year ago,
00:27:27 I gave a talk at Cold Spring Harbor Labs
00:27:30 about the progress we were making in our work.
00:27:34 And it was a lot of fun because he said,
00:27:39 well, you wouldn’t be coming here
00:27:41 unless you had something important to say.
00:27:42 So I’m gonna go attend your talk.
00:27:44 So he sat in the very front row.
00:27:46 Next to him was the director of the lab, Bruce Stillman.
00:27:50 So these guys are in the front row of this auditorium.
00:27:52 Nobody else in the auditorium wants to sit in the front row
00:27:54 because there’s Jim Watson and there’s the director.
00:27:56 And I gave a talk and then I had dinner with him afterwards.
00:28:03 But there’s a great picture of my colleague Subitai Amantak
00:28:07 where I’m up there sort of like screaming the basics
00:28:09 of this new framework we have.
00:28:11 And Jim Watson’s on the edge of his chair.
00:28:13 He’s literally on the edge of his chair,
00:28:15 like intently staring up at the screen.
00:28:17 And when he discovered the structure of DNA,
00:28:21 the first public talk he gave
00:28:23 was at Cold Spring Harbor Labs.
00:28:25 And there’s a picture, there’s a famous picture
00:28:27 of Jim Watson standing at the whiteboard
00:28:29 with an overrated thing pointing at something,
00:28:31 pointing at the double helix with his pointer.
00:28:33 And it actually looks a lot like the picture of me.
00:28:34 So there was a sort of funny,
00:28:36 there’s Arian talking about the brain
00:28:37 and there’s Jim Watson staring intently at it.
00:28:39 And of course there with, whatever, 60 years earlier,
00:28:41 he was standing pointing at the double helix.
00:28:44 That’s one of the great discoveries in all of,
00:28:47 whatever, biology, science, all science and DNA.
00:28:49 So it’s funny that there’s echoes of that in your presentation.
00:28:54 Do you think, in terms of evolutionary timeline and history,
00:28:58 the development of the neocortex was a big leap?
00:29:01 Or is it just a small step?
00:29:07 So like, if we ran the whole thing over again,
00:29:09 from the birth of life on Earth,
00:29:12 how likely would we develop the mechanism of the neocortex?
00:29:15 Okay, well those are two separate questions.
00:29:17 One is, was it a big leap?
00:29:18 And one was how likely it is, okay?
00:29:21 They’re not necessarily related.
00:29:22 Maybe correlated.
00:29:23 Maybe correlated, maybe not.
00:29:25 And we don’t really have enough data
00:29:26 to make a judgment about that.
00:29:28 I would say definitely it was a big leap.
00:29:29 And I can tell you why.
00:29:30 I don’t think it was just another incremental step.
00:29:34 I don’t get that at the moment.
00:29:35 I don’t really have any idea how likely it is.
00:29:38 If we look at evolution,
00:29:39 we have one data point, which is Earth, right?
00:29:42 Life formed on Earth billions of years ago,
00:29:45 whether it was introduced here or it created it here,
00:29:48 or someone introduced it, we don’t really know,
00:29:49 but it was here early.
00:29:51 It took a long, long time to get to multicellular life.
00:29:55 And then for multicellular life,
00:29:58 it took a long, long time to get the neocortex.
00:30:02 And we’ve only had the neocortex for a few 100,000 years.
00:30:05 So that’s like nothing, okay?
00:30:08 So is it likely?
00:30:09 Well, it certainly isn’t something
00:30:10 that happened right away on Earth.
00:30:13 And there were multiple steps to get there.
00:30:15 So I would say it’s probably not gonna be something
00:30:17 that would happen instantaneously
00:30:18 on other planets that might have life.
00:30:20 It might take several billion years on average.
00:30:23 Is it likely?
00:30:24 I don’t know, but you’d have to survive
00:30:25 for several billion years to find out.
00:30:27 Probably.
00:30:29 Is it a big leap?
00:30:30 Yeah, I think it is a qualitative difference
00:30:35 in all other evolutionary steps.
00:30:37 I can try to describe that if you’d like.
00:30:39 Sure, in which way?
00:30:41 Yeah, I can tell you how.
00:30:43 Pretty much, let’s start with a little preface.
00:30:47 Many of the things that humans are able to do
00:30:50 do not have obvious survival advantages precedent.
00:30:58 We could create music, is that,
00:31:00 is there a really survival advantage to that?
00:31:02 Maybe, maybe not.
00:31:03 What about mathematics?
00:31:04 Is there a real survival advantage to mathematics?
00:31:07 Well, you could stretch it.
00:31:09 You can try to figure these things out, right?
00:31:13 But most of evolutionary history,
00:31:14 everything had immediate survival advantages to it.
00:31:18 So, I’ll tell you a story, which I like,
00:31:22 may or may not be true, but the story goes as follows.
00:31:29 Organisms have been evolving for,
00:31:30 since the beginning of life here on Earth,
00:31:33 and adding this sort of complexity onto that,
00:31:35 and this sort of complexity onto that,
00:31:36 and the brain itself is evolved this way.
00:31:39 In fact, there’s old parts, and older parts,
00:31:42 and older, older parts of the brain
00:31:43 that kind of just keeps calming on new things,
00:31:45 and we keep adding capabilities.
00:31:47 When we got to the neocortex,
00:31:48 initially it had a very clear survival advantage
00:31:52 in that it produced better vision,
00:31:54 and better hearing, and better touch,
00:31:55 and maybe, and so on.
00:31:57 But what I think happens is that evolution discovered,
00:32:01 it took a mechanism, and this is in our recent theories,
00:32:05 but it took a mechanism evolved a long time ago
00:32:08 for navigating in the world, for knowing where you are.
00:32:10 These are the so called grid cells and place cells
00:32:13 of an old part of the brain.
00:32:15 And it took that mechanism for building maps of the world,
00:32:20 and knowing where you are on those maps,
00:32:22 and how to navigate those maps,
00:32:24 and turns it into a sort of a slimmed down,
00:32:27 idealized version of it.
00:32:29 And that idealized version could now apply
00:32:31 to building maps of other things.
00:32:32 Maps of coffee cups, and maps of phones,
00:32:35 maps of mathematics.
00:32:36 Concepts almost.
00:32:37 Concepts, yes, and not just almost, exactly.
00:32:40 And so, and it just started replicating this stuff, right?
00:32:44 You just think more, and more, and more.
00:32:45 So we went from being sort of dedicated purpose
00:32:48 neural hardware to solve certain problems
00:32:51 that are important to survival,
00:32:53 to a general purpose neural hardware
00:32:55 that could be applied to all problems.
00:32:58 And now it’s escaped the orbit of survival.
00:33:02 We are now able to apply it to things
00:33:04 which we find enjoyment,
00:33:08 but aren’t really clearly survival characteristics.
00:33:13 And that it seems to only have happened in humans,
00:33:16 to the large extent.
00:33:19 And so that’s what’s going on,
00:33:20 where we sort of have,
00:33:22 we’ve sort of escaped the gravity of evolutionary pressure,
00:33:26 in some sense, in the neocortex.
00:33:28 And it now does things which are not,
00:33:31 that are really interesting,
00:33:32 discovering models of the universe,
00:33:34 which may not really help us.
00:33:36 Does it matter?
00:33:37 How does it help us surviving,
00:33:38 knowing that there might be multiverses,
00:33:40 or that there might be the age of the universe,
00:33:42 or how do various stellar things occur?
00:33:46 It doesn’t really help us survive at all.
00:33:47 But we enjoy it, and that’s what happened.
00:33:50 Or at least not in the obvious way, perhaps.
00:33:53 It is required,
00:33:56 if you look at the entire universe in an evolutionary way,
00:33:58 it’s required for us to do interplanetary travel,
00:34:00 and therefore survive past our own sun.
00:34:03 But you know, let’s not get too.
00:34:04 Yeah, but evolution works at one time frame,
00:34:07 it’s survival, if you think of survival of the phenotype,
00:34:11 survival of the individual.
00:34:13 What you’re talking about there is spans well beyond that.
00:34:16 So there’s no genetic,
00:34:18 I’m not transferring any genetic traits to my children
00:34:23 that are gonna help them survive better on Mars.
00:34:26 Totally different mechanism, that’s right.
00:34:28 So let’s get into the new, as you’ve mentioned,
00:34:31 this idea of the, I don’t know if you have a nice name,
00:34:34 thousand.
00:34:35 We call it the thousand brain theory of intelligence.
00:34:37 I like it.
00:34:38 Can you talk about this idea of a spatial view of concepts
00:34:43 and so on?
00:34:44 Yeah, so can I just describe sort of the,
00:34:46 there’s an underlying core discovery,
00:34:49 which then everything comes from that.
00:34:51 That’s a very simple, this is really what happened.
00:34:55 We were deep into problems about understanding
00:34:58 how we build models of stuff in the world
00:35:00 and how we make predictions about things.
00:35:03 And I was holding a coffee cup just like this in my hand.
00:35:07 And my finger was touching the side, my index finger.
00:35:10 And then I moved it to the top
00:35:12 and I was gonna feel the rim at the top of the cup.
00:35:15 And I asked myself a very simple question.
00:35:18 I said, well, first of all, I say,
00:35:20 I know that my brain predicts what it’s gonna feel
00:35:22 before it touches it.
00:35:23 You can just think about it and imagine it.
00:35:26 And so we know that the brain’s making predictions
00:35:27 all the time.
00:35:28 So the question is, what does it take to predict that?
00:35:31 And there’s a very interesting answer.
00:35:33 First of all, it says the brain has to know
00:35:35 it’s touching a coffee cup.
00:35:36 It has to have a model of a coffee cup.
00:35:38 It needs to know where the finger currently is
00:35:41 on the cup relative to the cup.
00:35:43 Because when I make a movement,
00:35:44 it needs to know where it’s going to be on the cup
00:35:46 after the movement is completed relative to the cup.
00:35:50 And then it can make a prediction
00:35:51 about what it’s gonna sense.
00:35:53 So this told me that the neocortex,
00:35:54 which is making this prediction,
00:35:56 needs to know that it’s sensing it’s touching a cup.
00:35:59 And it needs to know the location of my finger
00:36:01 relative to that cup in a reference frame of the cup.
00:36:04 It doesn’t matter where the cup is relative to my body.
00:36:06 It doesn’t matter its orientation.
00:36:08 None of that matters.
00:36:09 It’s where my finger is relative to the cup,
00:36:10 which tells me then that the neocortex
00:36:13 has a reference frame that’s anchored to the cup.
00:36:17 Because otherwise I wouldn’t be able to say the location
00:36:19 and I wouldn’t be able to predict my new location.
00:36:21 And then we quickly, very instantly can say,
00:36:24 well, every part of my skin could touch this cup.
00:36:26 And therefore every part of my skin is making predictions
00:36:28 and every part of my skin must have a reference frame
00:36:30 that it’s using to make predictions.
00:36:33 So the big idea is that throughout the neocortex,
00:36:39 there are, everything is being stored
00:36:44 and referenced in reference frames.
00:36:46 You can think of them like XYZ reference frames,
00:36:48 but they’re not like that.
00:36:50 We know a lot about the neural mechanisms for this,
00:36:52 but the brain thinks in reference frames.
00:36:54 And as an engineer, if you’re an engineer,
00:36:56 this is not surprising.
00:36:57 You’d say, if I wanted to build a CAD model
00:37:00 of the coffee cup, well, I would bring it up
00:37:02 and some CAD software, and I would assign
00:37:04 some reference frame and say this features
00:37:05 at this locations and so on.
00:37:06 But the fact that this, the idea that this is occurring
00:37:09 throughout the neocortex everywhere, it was a novel idea.
00:37:14 And then a zillion things fell into place after that,
00:37:19 a zillion.
00:37:19 So now we think about the neocortex
00:37:21 as processing information quite differently
00:37:23 than we used to do it.
00:37:24 We used to think about the neocortex
00:37:25 as processing sensory data and extracting features
00:37:28 from that sensory data and then extracting features
00:37:30 from the features, very much like a deep learning network
00:37:33 does today.
00:37:34 But that’s not how the brain works at all.
00:37:36 The brain works by assigning everything,
00:37:39 every input, everything to reference frames.
00:37:41 And there are thousands, hundreds of thousands
00:37:44 of them active at once in your neocortex.
00:37:47 It’s a surprising thing to think about,
00:37:49 but once you sort of internalize this,
00:37:51 you understand that it explains almost every,
00:37:54 almost all the mysteries we’ve had about this structure.
00:37:57 So one of the consequences of that
00:38:00 is that every small part of the neocortex,
00:38:02 say a millimeter square, and there’s 150,000 of those.
00:38:06 So it’s about 150,000 square millimeters.
00:38:08 If you take every little square millimeter of the cortex,
00:38:11 it’s got some input coming into it
00:38:13 and it’s gonna have reference frames
00:38:14 where it’s assigned that input to.
00:38:16 And each square millimeter can learn
00:38:19 complete models of objects.
00:38:20 So what do I mean by that?
00:38:22 If I’m touching the coffee cup,
00:38:23 well, if I just touch it in one place,
00:38:25 I can’t learn what this coffee cup is
00:38:27 because I’m just feeling one part.
00:38:28 But if I move it around the cup
00:38:31 and touch it at different areas,
00:38:32 I can build up a complete model of the cup
00:38:34 because I’m now filling in that three dimensional map,
00:38:36 which is the coffee cup.
00:38:37 I can say, oh, what am I feeling
00:38:38 at all these different locations?
00:38:39 That’s the basic idea, it’s more complicated than that.
00:38:43 But so through time, and we talked about time earlier,
00:38:46 through time, even a single column,
00:38:48 which is only looking at, or a single part of the cortex,
00:38:50 which is only looking at a small part of the world,
00:38:52 can build up a complete model of an object.
00:38:55 And so if you think about the part of the brain,
00:38:57 which is getting input from all my fingers,
00:38:59 so they’re spread across the top of your head here.
00:39:01 This is the somatosensory cortex.
00:39:04 There’s columns associated
00:39:05 with all the different areas of my skin.
00:39:07 And what we believe is happening
00:39:10 is that all of them are building models of this cup,
00:39:12 every one of them, or things.
00:39:15 They’re not all building,
00:39:16 not every column or every part of the cortex
00:39:18 builds models of everything,
00:39:19 but they’re all building models of something.
00:39:21 And so you have, so when I touch this cup with my hand,
00:39:26 there are multiple models of the cup being invoked.
00:39:28 If I look at it with my eyes,
00:39:30 there are, again, many models of the cup being invoked,
00:39:32 because each part of the visual system,
00:39:34 the brain doesn’t process an image.
00:39:35 That’s a misleading idea.
00:39:38 It’s just like your fingers touching the cup,
00:39:40 so different parts of my retina
00:39:41 are looking at different parts of the cup.
00:39:42 And thousands and thousands of models of the cup
00:39:45 are being invoked at once.
00:39:47 And they’re all voting with each other,
00:39:48 trying to figure out what’s going on.
00:39:50 So that’s why we call it the thousand brains theory
00:39:51 of intelligence, because there isn’t one model of a cup.
00:39:54 There are thousands of models of this cup.
00:39:56 There are thousands of models of your cellphone
00:39:57 and about cameras and microphones and so on.
00:40:00 It’s a distributed modeling system,
00:40:02 which is very different
00:40:03 than the way people have thought about it.
00:40:04 And so that’s a really compelling and interesting idea.
00:40:07 I have two first questions.
00:40:08 So one, on the ensemble part of everything coming together,
00:40:12 you have these thousand brains.
00:40:14 How do you know which one has done the best job
00:40:17 of forming the…
00:40:18 Great question.
00:40:19 Let me try to explain it.
00:40:20 There’s a problem that’s known in neuroscience
00:40:23 called the sensor fusion problem.
00:40:25 Yes.
00:40:26 And so the idea is there’s something like,
00:40:27 oh, the image comes from the eye.
00:40:29 There’s a picture on the retina
00:40:30 and then it gets projected to the neocortex.
00:40:32 Oh, by now it’s all spread out all over the place
00:40:35 and it’s kind of squirrely and distorted
00:40:37 and pieces are all over the…
00:40:39 It doesn’t look like a picture anymore.
00:40:40 When does it all come back together again?
00:40:43 Or you might say, well, yes,
00:40:45 but I also have sounds or touches associated with the cup.
00:40:48 So I’m seeing the cup and touching the cup.
00:40:50 How do they get combined together again?
00:40:52 So it’s called the sensor fusion problem.
00:40:54 As if all these disparate parts
00:40:55 have to be brought together into one model someplace.
00:40:59 That’s the wrong idea.
00:41:01 The right idea is that you’ve got all these guys voting.
00:41:03 There’s auditory models of the cup.
00:41:05 There’s visual models of the cup.
00:41:06 There’s tactile models of the cup.
00:41:09 In the vision system,
00:41:10 there might be ones that are more focused on black and white
00:41:12 and ones focusing on color.
00:41:13 It doesn’t really matter.
00:41:14 There’s just thousands and thousands of models of this cup.
00:41:17 And they vote.
00:41:17 They don’t actually come together in one spot.
00:41:20 Just literally think of it this way.
00:41:21 Imagine you have these columns
00:41:24 that are like about the size of a little piece of spaghetti.
00:41:26 Like a two and a half millimeters tall
00:41:28 and about a millimeter in wide.
00:41:30 They’re not physical, but you could think of them that way.
00:41:33 And each one’s trying to guess what this thing is
00:41:35 or touching.
00:41:36 Now, they can do a pretty good job
00:41:38 if they’re allowed to move over time.
00:41:40 So I can reach my hand into a black box
00:41:41 and move my finger around an object.
00:41:43 And if I touch enough spaces, I go, okay,
00:41:45 now I know what it is.
00:41:46 But often we don’t do that.
00:41:48 Often I can just reach and grab something with my hand
00:41:49 all at once and I get it.
00:41:51 Or if I had to look through the world through a straw,
00:41:53 so I’m only invoking one little column,
00:41:55 I can only see part of something
00:41:56 because I have to move the straw around.
00:41:58 But if I open my eyes, I see the whole thing at once.
00:42:00 So what we think is going on
00:42:01 is all these little pieces of spaghetti,
00:42:03 if you will, all these little columns in the cortex,
00:42:05 are all trying to guess what it is that they’re sensing.
00:42:08 They’ll do a better guess if they have time
00:42:10 and can move over time.
00:42:11 So if I move my eyes, I move my fingers.
00:42:13 But if they don’t, they have a poor guess.
00:42:16 It’s a probabilistic guess of what they might be touching.
00:42:20 Now, imagine they can post their probability
00:42:22 at the top of a little piece of spaghetti.
00:42:24 Each one of them says,
00:42:25 I think, and it’s not really a probability distribution.
00:42:27 It’s more like a set of possibilities.
00:42:29 In the brain, it doesn’t work as a probability distribution.
00:42:31 It works as more like what we call a union.
00:42:34 So you could say, and one column says,
00:42:35 I think it could be a coffee cup,
00:42:37 a soda can, or a water bottle.
00:42:39 And another column says,
00:42:40 I think it could be a coffee cup
00:42:42 or a telephone or a camera or whatever, right?
00:42:46 And all these guys are saying what they think it might be.
00:42:49 And there’s these long range connections
00:42:51 in certain layers in the cortex.
00:42:53 So there’s in some layers in some cells types
00:42:56 in each column, send the projections across the brain.
00:43:00 And that’s the voting occurs.
00:43:01 And so there’s a simple associative memory mechanism.
00:43:04 We’ve described this in a recent paper
00:43:06 and we’ve modeled this that says,
00:43:09 they can all quickly settle on the only
00:43:11 or the one best answer for all of them.
00:43:14 If there is a single best answer,
00:43:16 they all vote and say, yep, it’s gotta be the coffee cup.
00:43:18 And at that point, they all know it’s a coffee cup.
00:43:21 And at that point, everyone acts as if it’s a coffee cup.
00:43:23 They’re like, yep, we know it’s a coffee,
00:43:24 even though I’ve only seen one little piece of this world,
00:43:26 I know it’s a coffee cup I’m touching
00:43:27 or I’m seeing or whatever.
00:43:28 And so you can think of all these columns
00:43:30 are looking at different parts in different places,
00:43:33 different sensory input, different locations,
00:43:35 they’re all different.
00:43:36 But this layer that’s doing the voting, it solidifies.
00:43:40 It’s just like it crystallizes and says,
00:43:42 oh, we all know what we’re doing.
00:43:44 And so you don’t bring these models together in one model,
00:43:46 you just vote and there’s a crystallization of the vote.
00:43:49 Great, that’s at least a compelling way
00:43:51 to think about the way you form a model of the world.
00:43:58 Now, you talk about a coffee cup.
00:44:00 Do you see this, as far as I understand,
00:44:03 you are proposing this as well,
00:44:04 that this extends to much more than coffee cups?
00:44:06 Yeah.
00:44:07 It does.
00:44:09 Or at least the physical world,
00:44:10 it expands to the world of concepts.
00:44:14 Yeah, it does.
00:44:15 And well, first, the primary thing is evidence for that
00:44:18 is that the regions of the neocortex
00:44:20 that are associated with language
00:44:22 or high level thought or mathematics
00:44:23 or things like that,
00:44:24 they look like the regions of the neocortex
00:44:26 that process vision, hearing, and touch.
00:44:28 They don’t look any different.
00:44:29 Or they look only marginally different.
00:44:32 And so one would say, well, if Vernon Mountcastle,
00:44:36 who proposed that all the parts of the neocortex
00:44:38 do the same thing, if he’s right,
00:44:41 then the parts that are doing language
00:44:42 or mathematics or physics
00:44:44 are working on the same principle.
00:44:45 They must be working on the principle of reference frames.
00:44:48 So that’s a little odd thought.
00:44:51 But of course, we had no prior idea
00:44:53 how these things happen.
00:44:55 So let’s go with that.
00:44:57 And we, in our recent paper,
00:44:59 we talked a little bit about that.
00:45:01 I’ve been working on it more since.
00:45:02 I have better ideas about it now.
00:45:05 I’m sitting here very confident
00:45:06 that that’s what’s happening.
00:45:08 And I can give you some examples
00:45:09 that help you think about that.
00:45:11 It’s not we understand it completely,
00:45:12 but I understand it better than I’ve described it
00:45:14 in any paper so far.
00:45:15 So, but we did put that idea out there.
00:45:17 It says, okay, this is,
00:45:18 it’s a good place to start, you know?
00:45:22 And the evidence would suggest it’s how it’s happening.
00:45:24 And then we can start tackling that problem
00:45:26 one piece at a time.
00:45:27 Like, what does it mean to do high level thought?
00:45:29 What does it mean to do language?
00:45:30 How would that fit into a reference frame framework?
00:45:34 Yeah, so there’s a,
00:45:35 I don’t know if you could tell me if there’s a connection,
00:45:37 but there’s an app called Anki
00:45:40 that helps you remember different concepts.
00:45:42 And they talk about like a memory palace
00:45:45 that helps you remember completely random concepts
00:45:47 by trying to put them in a physical space in your mind
00:45:51 and putting them next to each other.
00:45:52 It’s called the method of loci.
00:45:53 Loci, yeah.
00:45:54 For some reason, that seems to work really well.
00:45:57 Now, that’s a very narrow kind of application
00:45:59 of just remembering some facts.
00:46:00 But that’s a very, very telling one.
00:46:03 Yes, exactly.
00:46:04 So this seems like you’re describing a mechanism
00:46:06 why this seems to work.
00:46:09 So basically the way what we think is going on
00:46:11 is all things you know, all concepts, all ideas,
00:46:15 words, everything you know are stored in reference frames.
00:46:20 And so if you want to remember something,
00:46:24 you have to basically navigate through a reference frame
00:46:26 the same way a rat navigates through a maze
00:46:28 and the same way my finger navigates to this coffee cup.
00:46:31 You are moving through some space.
00:46:33 And so if you have a random list of things
00:46:35 you were asked to remember,
00:46:37 by assigning them to a reference frame,
00:46:39 you’ve already know very well to see your house, right?
00:46:42 And the idea of the method of loci is you can say,
00:46:43 okay, in my lobby, I’m going to put this thing.
00:46:45 And then the bedroom, I put this one.
00:46:47 I go down the hall, I put this thing.
00:46:48 And then you want to recall those facts
00:46:50 or recall those things.
00:46:51 You just walk mentally, you walk through your house.
00:46:54 You’re mentally moving through a reference frame
00:46:56 that you already had.
00:46:57 And that tells you,
00:46:59 there’s two things that are really important about that.
00:47:00 It tells us the brain prefers to store things
00:47:02 in reference frames.
00:47:03 And that the method of recalling things
00:47:06 or thinking, if you will,
00:47:08 is to move mentally through those reference frames.
00:47:11 You could move physically through some reference frames,
00:47:13 like I could physically move through the reference frame
00:47:15 of this coffee cup.
00:47:16 I can also mentally move through the reference frame
00:47:17 of the coffee cup, imagining me touching it.
00:47:19 But I can also mentally move my house.
00:47:22 And so now we can ask yourself,
00:47:24 or are all concepts stored this way?
00:47:26 There was some recent research using human subjects
00:47:31 in fMRI, and I’m going to apologize for not knowing
00:47:33 the name of the scientists who did this.
00:47:36 But what they did is they put humans in this fMRI machine,
00:47:41 which is one of these imaging machines.
00:47:42 And they gave the humans tasks to think about birds.
00:47:46 So they had different types of birds,
00:47:47 and birds that look big and small,
00:47:49 and long necks and long legs, things like that.
00:47:52 And what they could tell from the fMRI
00:47:55 was a very clever experiment.
00:47:57 You get to tell when humans were thinking about the birds,
00:48:00 that the birds, the knowledge of birds
00:48:03 was arranged in a reference frame,
00:48:05 similar to the ones that are used
00:48:07 when you navigate in a room.
00:48:08 That these are called grid cells,
00:48:10 and there are grid cell like patterns of activity
00:48:12 in the neocortex when they do this.
00:48:15 So it’s a very clever experiment.
00:48:18 And what it basically says,
00:48:20 that even when you’re thinking about something abstract,
00:48:22 and you’re not really thinking about it as a reference frame,
00:48:24 it tells us the brain is actually using a reference frame.
00:48:26 And it’s using the same neural mechanisms.
00:48:28 These grid cells are the basic same neural mechanism
00:48:30 that we propose that grid cells,
00:48:32 which exist in the old part of the brain,
00:48:34 the entorhinal cortex, that that mechanism
00:48:37 is now similar mechanism is used throughout the neocortex.
00:48:40 It’s the same nature to preserve this interesting way
00:48:43 of creating reference frames.
00:48:44 And so now they have empirical evidence
00:48:46 that when you think about concepts like birds,
00:48:49 that you’re using reference frames
00:48:51 that are built on grid cells.
00:48:53 So that’s similar to the method of loci,
00:48:55 but in this case, the birds are related.
00:48:56 So they create their own reference frame,
00:48:58 which is consistent with bird space.
00:49:01 And when you think about something, you go through that.
00:49:03 You can make the same example,
00:49:04 let’s take mathematics.
00:49:06 Let’s say you wanna prove a conjecture.
00:49:09 What is a conjecture?
00:49:10 A conjecture is a statement you believe to be true,
00:49:13 but you haven’t proven it.
00:49:15 And so it might be an equation.
00:49:16 I wanna show that this is equal to that.
00:49:19 And you have some places you start with.
00:49:21 You say, well, I know this is true,
00:49:22 and I know this is true.
00:49:23 And I think that maybe to get to the final proof,
00:49:25 I need to go through some intermediate results.
00:49:28 What I believe is happening is literally these equations
00:49:33 or these points are assigned to a reference frame,
00:49:36 a mathematical reference frame.
00:49:37 And when you do mathematical operations,
00:49:39 a simple one might be multiply or divide,
00:49:41 but you might be a little plus transform or something else.
00:49:44 That is like a movement in the reference frame of the math.
00:49:47 And so you’re literally trying to discover a path
00:49:50 from one location to another location
00:49:52 in a space of mathematics.
00:49:56 And if you can get to these intermediate results,
00:49:58 then you know your map is pretty good,
00:50:00 and you know you’re using the right operations.
00:50:02 Much of what we think about is solving hard problems
00:50:05 is designing the correct reference frame for that problem,
00:50:08 figuring out how to organize the information
00:50:11 and what behaviors I wanna use in that space
00:50:14 to get me there.
00:50:16 Yeah, so if you dig in an idea of this reference frame,
00:50:19 whether it’s the math, you start a set of axioms
00:50:21 to try to get to proving the conjecture.
00:50:25 Can you try to describe, maybe take a step back,
00:50:28 how you think of the reference frame in that context?
00:50:30 Is it the reference frame that the axioms are happy in?
00:50:36 Is it the reference frame that might contain everything?
00:50:38 Is it a changing thing as you?
00:50:41 You have many, many reference frames.
00:50:43 I mean, in fact, the way the theory,
00:50:44 the thousand brain theory of intelligence says
00:50:46 that every single thing in the world
00:50:47 has its own reference frame.
00:50:48 So every word has its own reference frames.
00:50:50 And we can talk about this.
00:50:52 The mathematics work out,
00:50:54 this is no problem for neurons to do this.
00:50:55 But how many reference frames does a coffee cup have?
00:50:58 Well, it’s on a table.
00:51:00 Let’s say you ask how many reference frames
00:51:03 could a column in my finger
00:51:06 that’s touching the coffee cup have?
00:51:07 Because there are many, many copy,
00:51:09 there are many, many models of the coffee cup.
00:51:10 So the coffee, there is no one model of a coffee cup.
00:51:13 There are many models of a coffee cup.
00:51:14 And you could say, well,
00:51:15 how many different things can my finger learn?
00:51:17 Is this the question you want to ask?
00:51:19 Imagine I say every concept, every idea,
00:51:21 everything you’ve ever know about that you can say,
00:51:23 I know that thing has a reference frame
00:51:27 associated with it.
00:51:28 And what we do when we build composite objects,
00:51:30 we assign reference frames
00:51:32 to point another reference frame.
00:51:33 So my coffee cup has multiple components to it.
00:51:37 It’s got a limb, it’s got a cylinder, it’s got a handle.
00:51:40 And those things have their own reference frames
00:51:42 and they’re assigned to a master reference frame,
00:51:45 which is called this cup.
00:51:46 And now I have this Numenta logo on it.
00:51:48 Well, that’s something that exists elsewhere in the world.
00:51:50 It’s its own thing.
00:51:51 So it has its own reference frame.
00:51:52 So we now have to say,
00:51:53 how can I assign the Numenta logo reference frame
00:51:56 onto the cylinder or onto the coffee cup?
00:51:59 So it’s all, we talked about this in the paper
00:52:01 that came out in December of this last year.
00:52:06 The idea of how you can assign reference frames
00:52:08 to reference frames, how neurons could do this.
00:52:10 So, well, my question is,
00:52:12 even though you mentioned reference frames a lot,
00:52:14 I almost feel it’s really useful to dig into
00:52:16 how you think of what a reference frame is.
00:52:20 I mean, it was already helpful for me to understand
00:52:22 that you think of reference frames
00:52:23 as something there is a lot of.
00:52:26 Okay, so let’s just say that we’re gonna have
00:52:28 some neurons in the brain, not many, actually,
00:52:31 10,000, 20,000 are gonna create
00:52:32 a whole bunch of reference frames.
00:52:34 What does it mean?
00:52:35 What is a reference frame?
00:52:37 First of all, these reference frames are different
00:52:40 than the ones you might be used to.
00:52:42 We know lots of reference frames.
00:52:43 For example, we know the Cartesian coordinates, X, Y, Z,
00:52:46 that’s a type of reference frame.
00:52:47 We know longitude and latitude,
00:52:50 that’s a different type of reference frame.
00:52:52 If I look at a printed map,
00:52:54 you might have columns A through M,
00:52:58 and rows one through 20,
00:53:00 that’s a different type of reference frame.
00:53:01 It’s kind of a Cartesian coordinate reference frame.
00:53:04 The interesting thing about the reference frames
00:53:06 in the brain, and we know this because these
00:53:08 have been established through neuroscience
00:53:10 studying the entorhinal cortex.
00:53:12 So I’m not speculating here, okay?
00:53:13 This is known neuroscience in an old part of the brain.
00:53:16 The way these cells create reference frames,
00:53:18 they have no origin.
00:53:20 So what it’s more like, you have a point,
00:53:24 a point in some space, and you,
00:53:27 given a particular movement,
00:53:29 you can then tell what the next point should be.
00:53:32 And you can then tell what the next point would be,
00:53:34 and so on.
00:53:35 You can use this to calculate
00:53:38 how to get from one point to another.
00:53:40 So how do I get from my house to my home,
00:53:43 or how do I get my finger from the side of my cup
00:53:44 to the top of the cup?
00:53:46 How do I get from the axioms to the conjecture?
00:53:50 So it’s a different type of reference frame,
00:53:54 and I can, if you want, I can describe in more detail,
00:53:57 I can paint a picture of how you might want
00:53:59 to think about that.
00:53:59 It’s really helpful to think it’s something
00:54:00 you can move through, but is there,
00:54:03 is it helpful to think of it as spatial in some sense,
00:54:08 or is there something that’s more?
00:54:09 No, it’s definitely spatial.
00:54:11 It’s spatial in a mathematical sense.
00:54:13 How many dimensions?
00:54:14 Can it be a crazy number of dimensions?
00:54:16 Well, that’s an interesting question.
00:54:17 In the old part of the brain, the entorhinal cortex,
00:54:20 they studied rats, and initially it looks like,
00:54:22 oh, this is just two dimensional.
00:54:24 It’s like the rat is in some box in the maze or whatever,
00:54:27 and they know where the rat is using
00:54:28 these two dimensional reference frames
00:54:30 to know where it is in the maze.
00:54:32 We said, well, okay, but what about bats?
00:54:35 That’s a mammal, and they fly in three dimensional space.
00:54:38 How do they do that?
00:54:39 They seem to know where they are, right?
00:54:41 So this is a current area of active research,
00:54:44 and it seems like somehow the neurons
00:54:46 in the entorhinal cortex can learn three dimensional space.
00:54:50 We just, two members of our team,
00:54:52 along with Elif Fett from MIT,
00:54:55 just released a paper this literally last week.
00:54:59 It’s on bioRxiv, where they show that you can,
00:55:03 if you, the way these things work,
00:55:05 and I won’t get, unless you want to,
00:55:06 I won’t get into the detail,
00:55:08 but grid cells can represent any n dimensional space.
00:55:12 It’s not inherently limited.
00:55:15 You can think of it this way.
00:55:16 If you had two dimensional, the way it works
00:55:18 is you had a bunch of two dimensional slices.
00:55:20 That’s the way these things work.
00:55:21 There’s a whole bunch of two dimensional models,
00:55:24 and you can just, you can slice up
00:55:26 any n dimensional space with two dimensional projections.
00:55:29 So, and you could have one dimensional models.
00:55:31 So there’s nothing inherent about the mathematics
00:55:34 about the way the neurons do this,
00:55:35 which constrain the dimensionality of the space,
00:55:39 which I think was important.
00:55:41 So obviously I have a three dimensional map of this cup.
00:55:44 Maybe it’s even more than that, I don’t know.
00:55:46 But it’s clearly a three dimensional map of the cup.
00:55:48 I don’t just have a projection of the cup.
00:55:50 But when I think about birds,
00:55:52 or when I think about mathematics,
00:55:53 perhaps it’s more than three dimensions.
00:55:55 Who knows?
00:55:56 So in terms of each individual column
00:56:00 building up more and more information over time,
00:56:04 do you think that mechanism is well understood?
00:56:06 In your mind, you’ve proposed a lot of architectures there.
00:56:09 Is that a key piece, or is it,
00:56:11 is the big piece, the thousand brain theory of intelligence,
00:56:16 the ensemble of it all?
00:56:17 Well, I think they’re both big.
00:56:18 I mean, clearly the concept, as a theorist,
00:56:20 the concept is most exciting, right?
00:56:23 The high level concept.
00:56:23 The high level concept.
00:56:24 This is a totally new way of thinking
00:56:26 about how the neocortex works.
00:56:27 So that is appealing.
00:56:28 It has all these ramifications.
00:56:30 And with that, as a framework for how the brain works,
00:56:33 you can make all kinds of predictions
00:56:34 and solve all kinds of problems.
00:56:36 Now we’re trying to work through
00:56:37 many of these details right now.
00:56:38 Okay, how do the neurons actually do this?
00:56:40 Well, it turns out, if you think about grid cells
00:56:42 and place cells in the old parts of the brain,
00:56:44 there’s a lot that’s known about them,
00:56:45 but there’s still some mysteries.
00:56:47 There’s a lot of debate about exactly the details,
00:56:49 how these work and what are the signs.
00:56:50 And we have that still, that same level of detail,
00:56:52 that same level of concern.
00:56:54 What we spend here most of our time doing
00:56:56 is trying to make a very good list
00:57:00 of the things we don’t understand yet.
00:57:02 That’s the key part here.
00:57:04 What are the constraints?
00:57:05 It’s not like, oh, this thing seems to work, we’re done.
00:57:07 No, it’s like, okay, it kind of works,
00:57:08 but these are other things we know it has to do
00:57:10 and it’s not doing those yet.
00:57:12 I would say we’re well on the way here.
00:57:15 We’re not done yet.
00:57:17 There’s a lot of trickiness to this system,
00:57:20 but the basic principles about how different layers
00:57:23 in the neocortex are doing much of this, we understand.
00:57:27 But there’s some fundamental parts
00:57:28 that we don’t understand as well.
00:57:30 So what would you say is one of the harder open problems
00:57:34 or one of the ones that have been bothering you,
00:57:37 keeping you up at night the most?
00:57:38 Oh, well, right now, this is a detailed thing
00:57:40 that wouldn’t apply to most people, okay?
00:57:42 Sure.
00:57:43 But you want me to answer that question?
00:57:44 Yeah, please.
00:57:46 We’ve talked about as if, oh,
00:57:48 to predict what you’re going to sense on this coffee cup,
00:57:50 I need to know where my finger is gonna be
00:57:52 on the coffee cup.
00:57:53 That is true, but it’s insufficient.
00:57:56 Think about my finger touches the edge of the coffee cup.
00:57:58 My finger can touch it at different orientations.
00:58:01 I can rotate my finger around here and that doesn’t change.
00:58:06 I can make that prediction and somehow,
00:58:08 so it’s not just the location.
00:58:10 There’s an orientation component of this as well.
00:58:13 This is known in the old parts of the brain too.
00:58:15 There’s things called head direction cells,
00:58:16 which way the rat is facing.
00:58:18 It’s the same kind of basic idea.
00:58:20 So if my finger were a rat, you know, in three dimensions,
00:58:23 I have a three dimensional orientation
00:58:25 and I have a three dimensional location.
00:58:27 If I was a rat, I would have a,
00:58:28 you might think of it as a two dimensional location,
00:58:30 a two dimensional orientation,
00:58:31 a one dimensional orientation,
00:58:32 like just which way is it facing?
00:58:35 So how the two components work together,
00:58:38 how it is that I combine orientation,
00:58:41 the orientation of my sensor,
00:58:43 as well as the location is a tricky problem.
00:58:49 And I think I’ve made progress on it.
00:58:52 So at a bigger version of that,
00:58:55 so perspective is super interesting, but super specific.
00:58:58 Yeah, I warned you.
00:59:00 No, no, no, that’s really good,
00:59:01 but there’s a more general version of that.
00:59:03 Do you think context matters,
00:59:06 the fact that we’re in a building in North America,
00:59:10 that we, in the day and age where we have mugs?
00:59:15 I mean, there’s all this extra information
00:59:19 that you bring to the table about everything else
00:59:22 in the room that’s outside of just the coffee cup.
00:59:24 How does it get connected, do you think?
00:59:27 Yeah, and that is another really interesting question.
00:59:30 I’m gonna throw that under the rubric
00:59:32 or the name of attentional problems.
00:59:35 First of all, we have this model,
00:59:36 I have many, many models.
00:59:38 And also the question, does it matter?
00:59:40 Well, it matters for certain things, of course it does.
00:59:42 Maybe what we think of that as a coffee cup
00:59:44 in another part of the world
00:59:45 is viewed as something completely different.
00:59:47 Or maybe our logo, which is very benign
00:59:50 in this part of the world,
00:59:51 it means something very different
00:59:52 in another part of the world.
00:59:53 So those things do matter.
00:59:57 I think the way to think about it is the following,
01:00:00 one way to think about it,
01:00:01 is we have all these models of the world, okay?
01:00:04 And we model everything.
01:00:06 And as I said earlier, I kind of snuck it in there,
01:00:08 our models are actually, we build composite structure.
01:00:12 So every object is composed of other objects,
01:00:15 which are composed of other objects,
01:00:16 and they become members of other objects.
01:00:18 So this room has chairs and a table and a room
01:00:20 and walls and so on.
01:00:21 Now we can just arrange these things in a certain way
01:00:24 and go, oh, that’s the nomenclature conference room.
01:00:26 So, and what we do is when we go around the world
01:00:31 and we experience the world,
01:00:33 by walking into a room, for example,
01:00:35 the first thing I do is I can say,
01:00:36 oh, I’m in this room, do I recognize the room?
01:00:38 Then I can say, oh, look, there’s a table here.
01:00:41 And by attending to the table,
01:00:43 I’m then assigning this table in the context of the room.
01:00:45 Then I can say, oh, on the table, there’s a coffee cup.
01:00:48 Oh, and on the table, there’s a logo.
01:00:49 And in the logo, there’s the word Nementa.
01:00:51 Oh, and look in the logo, there’s the letter E.
01:00:53 Oh, and look, it has an unusual serif.
01:00:55 And it doesn’t actually, but I pretended to serif.
01:00:59 So the point is your attention is kind of drilling
01:01:03 deep in and out of these nested structures.
01:01:07 And I can pop back up and I can pop back down.
01:01:09 I can pop back up and I can pop back down.
01:01:10 So when I attend to the coffee cup,
01:01:13 I haven’t lost the context of everything else,
01:01:15 but it’s sort of, there’s this sort of nested structure.
01:01:18 So the attention filters the reference frame information
01:01:22 for that particular period of time?
01:01:24 Yes, it basically, moment to moment,
01:01:26 you attend the sub components,
01:01:28 and then you can attend the sub components
01:01:29 to sub components.
01:01:30 And you can move up and down.
01:01:31 You can move up and down.
01:01:32 We do that all the time.
01:01:33 You’re not even, now that I’m aware of it,
01:01:35 I’m very conscious of it.
01:01:36 But until, but most people don’t even think about this.
01:01:39 You just walk in a room and you don’t say,
01:01:41 oh, I looked at the chair and I looked at the board
01:01:43 and looked at that word on the board
01:01:44 and I looked over here, what’s going on, right?
01:01:47 So what percent of your day are you deeply aware of this?
01:01:50 And what part can you actually relax and just be Jeff?
01:01:52 Me personally, like my personal day?
01:01:54 Yeah.
01:01:55 Unfortunately, I’m afflicted with too much of the former.
01:02:01 Well, unfortunately or unfortunately.
01:02:02 Yeah.
01:02:03 You don’t think it’s useful?
01:02:04 Oh, it is useful, totally useful.
01:02:06 I think about this stuff almost all the time.
01:02:09 And one of my primary ways of thinking
01:02:12 is when I’m in sleep at night,
01:02:13 I always wake up in the middle of the night.
01:02:15 And then I stay awake for at least an hour
01:02:17 with my eyes shut in sort of a half sleep state
01:02:20 thinking about these things.
01:02:21 I come up with answers to problems very often
01:02:23 in that sort of half sleeping state.
01:02:25 I think about it on my bike ride, I think about it on walks.
01:02:27 I’m just constantly thinking about this.
01:02:28 I have to almost schedule time
01:02:32 to not think about this stuff
01:02:34 because it’s very, it’s mentally taxing.
01:02:37 Are you, when you’re thinking about this stuff,
01:02:39 are you thinking introspectively,
01:02:41 like almost taking a step outside of yourself
01:02:43 and trying to figure out what is your mind doing right now?
01:02:45 I do that all the time, but that’s not all I do.
01:02:49 I’m constantly observing myself.
01:02:50 So as soon as I started thinking about grid cells,
01:02:53 for example, and getting into that,
01:02:55 I started saying, oh, well, grid cells
01:02:56 can have my place of sense in the world.
01:02:58 That’s where you know where you are.
01:02:59 And it’s interesting, we always have a sense
01:03:01 of where we are unless we’re lost.
01:03:03 And so I started at night when I got up
01:03:04 to go to the bathroom, I would start trying to do it
01:03:06 completely with my eyes closed all the time.
01:03:08 And I would test my sense of grid cells.
01:03:10 I would walk five feet and say, okay, I think I’m here.
01:03:13 Am I really there?
01:03:14 What’s my error?
01:03:15 And then I would calculate my error again
01:03:16 and see how the errors could accumulate.
01:03:17 So even something as simple as getting up
01:03:19 in the middle of the night to go to the bathroom,
01:03:20 I’m testing these theories out.
01:03:22 It’s kind of fun.
01:03:23 I mean, the coffee cup is an example of that too.
01:03:25 So I find that these sort of everyday introspections
01:03:30 are actually quite helpful.
01:03:32 It doesn’t mean you can ignore the science.
01:03:34 I mean, I spend hours every day
01:03:37 reading ridiculously complex papers.
01:03:40 That’s not nearly as much fun,
01:03:41 but you have to sort of build up those constraints
01:03:44 and the knowledge about the field and who’s doing what
01:03:46 and what exactly they think is happening here.
01:03:48 And then you can sit back and say,
01:03:50 okay, let’s try to piece this all together.
01:03:53 Let’s come up with some, I’m very,
01:03:56 in this group here, people, they know they do,
01:03:58 I do this all the time.
01:03:59 I come in with these introspective ideas and say,
01:04:01 well, have you ever thought about this?
01:04:02 Now watch, well, let’s all do this together.
01:04:04 And it’s helpful.
01:04:05 It’s not, as long as you don’t,
01:04:09 all you did was that, then you’re just making up stuff.
01:04:12 But if you’re constraining it by the reality
01:04:14 of the neuroscience, then it’s really helpful.
01:04:17 So let’s talk a little bit about deep learning
01:04:20 and the successes in the applied space of neural networks,
01:04:26 ideas of training model on data
01:04:29 and these simple computational units,
01:04:31 artificial neurons that with backpropagation,
01:04:36 statistical ways of being able to generalize
01:04:40 from the training set onto data
01:04:42 that’s similar to that training set.
01:04:44 So where do you think are the limitations
01:04:47 of those approaches?
01:04:48 What do you think are its strengths
01:04:50 relative to your major efforts
01:04:52 of constructing a theory of human intelligence?
01:04:56 Well, I’m not an expert in this field.
01:04:57 I’m somewhat knowledgeable.
01:04:59 So, but I’m not.
01:04:59 Some of it is in just your intuition.
01:05:01 What are your?
01:05:02 Well, I have a little bit more than intuition,
01:05:03 but I just want to say like,
01:05:05 you know, one of the things that you asked me,
01:05:07 do I spend all my time thinking about neuroscience?
01:05:09 I do.
01:05:10 That’s to the exclusion of thinking about things
01:05:11 like convolutional neural networks.
01:05:13 But I try to stay current.
01:05:15 So look, I think it’s great, the progress they’ve made.
01:05:17 It’s fantastic.
01:05:18 And as I mentioned earlier,
01:05:19 it’s very highly useful for many things.
01:05:22 The models that we have today are actually derived
01:05:26 from a lot of neuroscience principles.
01:05:28 There are distributed processing systems
01:05:30 and distributed memory systems,
01:05:31 and that’s how the brain works.
01:05:33 They use things that we might call them neurons,
01:05:35 but they’re really not neurons at all.
01:05:37 So we can just, they’re not really neurons.
01:05:39 So they’re distributed processing systems.
01:05:41 And that nature of hierarchy,
01:05:44 that came also from neuroscience.
01:05:47 And so there’s a lot of things,
01:05:48 the learning rules, basically,
01:05:49 not back prop, but other, you know,
01:05:51 sort of heavy on top of that.
01:05:52 I’d be curious to say they’re not neurons at all.
01:05:55 Can you describe in which way?
01:05:56 I mean, some of it is obvious,
01:05:57 but I’d be curious if you have specific ways
01:06:00 in which you think are the biggest differences.
01:06:02 Yeah, we had a paper in 2016 called
01:06:04 Why Neurons Have Thousands of Synapses.
01:06:06 And if you read that paper,
01:06:09 you’ll know what I’m talking about here.
01:06:11 A real neuron in the brain is a complex thing.
01:06:14 And let’s just start with the synapses on it,
01:06:17 which is a connection between neurons.
01:06:19 Real neurons can have everywhere
01:06:20 from five to 30,000 synapses on them.
01:06:25 The ones near the cell body,
01:06:27 the ones that are close to the soma of the cell body,
01:06:30 those are like the ones that people model
01:06:32 in artificial neurons.
01:06:33 There is a few hundred of those.
01:06:35 Maybe they can affect the cell.
01:06:37 They can make the cell become active.
01:06:39 95% of the synapses can’t do that.
01:06:43 They’re too far away.
01:06:44 So if you activate one of those synapses,
01:06:45 it just doesn’t affect the cell body enough
01:06:47 to make any difference.
01:06:48 Any one of them individually.
01:06:50 Any one of them individually,
01:06:50 or even if you do a mass of them.
01:06:54 What real neurons do is the following.
01:06:57 If you activate or you get 10 to 20 of them
01:07:03 active at the same time,
01:07:04 meaning they’re all receiving an input at the same time,
01:07:06 and those 10 to 20 synapses or 40 synapses
01:07:09 within a very short distance on the dendrite,
01:07:11 like 40 microns, a very small area.
01:07:13 So if you activate a bunch of these
01:07:14 right next to each other at some distant place,
01:07:17 what happens is it creates
01:07:19 what’s called the dendritic spike.
01:07:21 And the dendritic spike travels through the dendrites
01:07:24 and can reach the soma or the cell body.
01:07:27 Now, when it gets there, it changes the voltage,
01:07:31 which is sort of like gonna make the cell fire,
01:07:33 but never enough to make the cell fire.
01:07:36 It’s sort of what we call, it says we depolarize the cell,
01:07:38 you raise the voltage a little bit,
01:07:39 but not enough to do anything.
01:07:41 It’s like, well, what good is that?
01:07:42 And then it goes back down again.
01:07:44 So we propose a theory,
01:07:47 which I’m very confident in basics are,
01:07:50 is that what’s happening there is
01:07:52 those 95% of the synapses are recognizing
01:07:55 dozens to hundreds of unique patterns.
01:07:58 They can write about 10, 20 synapses at a time,
01:08:02 and they’re acting like predictions.
01:08:04 So the neuron actually is a predictive engine on its own.
01:08:07 It can fire when it gets enough,
01:08:09 what they call proximal input
01:08:10 from those ones near the cell fire,
01:08:11 but it can get ready to fire from dozens to hundreds
01:08:15 of patterns that it recognizes from the other guys.
01:08:18 And the advantage of this to the neuron
01:08:21 is that when it actually does produce a spike
01:08:23 in action potential,
01:08:24 it does so slightly sooner than it would have otherwise.
01:08:27 And so what could is slightly sooner?
01:08:29 Well, the slightly sooner part is it,
01:08:31 all the excitatory neurons in the brain
01:08:34 are surrounded by these inhibitory neurons,
01:08:36 and they’re very fast, the inhibitory neurons,
01:08:38 these basket cells.
01:08:40 And if I get my spike out
01:08:42 a little bit sooner than someone else,
01:08:44 I inhibit all my neighbors around me, right?
01:08:47 And what you end up with is a different representation.
01:08:49 You end up with a reputation that matches your prediction.
01:08:52 It’s a sparser representation,
01:08:53 meaning fewer neurons are active,
01:08:55 but it’s much more specific.
01:08:57 And so we showed how networks of these neurons
01:09:00 can do very sophisticated temporal prediction, basically.
01:09:04 So this, summarize this,
01:09:07 real neurons in the brain are time based prediction engines,
01:09:10 and there’s no concept of this at all
01:09:14 in artificial, what we call point neurons.
01:09:18 I don’t think you can build a brain without them.
01:09:20 I don’t think you can build intelligence without them,
01:09:21 because it’s where a large part of the time comes from.
01:09:26 These are predictive models, and the time is,
01:09:29 there’s a prior and a prediction and an action,
01:09:32 and it’s inherent through every neuron in the neocortex.
01:09:34 So I would say that point neurons sort of model
01:09:37 a piece of that, and not very well at that either.
01:09:40 But like for example, synapses are very unreliable,
01:09:46 and you cannot assign any precision to them.
01:09:49 So even one digit of precision is not possible.
01:09:52 So the way real neurons work is they don’t add these,
01:09:55 they don’t change these weights accurately
01:09:57 like artificial neural networks do.
01:09:59 They basically form new synapses,
01:10:01 and so what you’re trying to always do is
01:10:03 detect the presence of some 10 to 20
01:10:06 active synapses at the same time,
01:10:08 as opposed, and they’re almost binary.
01:10:11 It’s like, because you can’t really represent
01:10:12 anything much finer than that.
01:10:14 So these are the kind of,
01:10:16 and I think that’s actually another essential component,
01:10:18 because the brain works on sparse patterns,
01:10:20 and all that mechanism is based on sparse patterns,
01:10:24 and I don’t actually think you could build real brains
01:10:26 or machine intelligence without
01:10:29 incorporating some of those ideas.
01:10:30 It’s hard to even think about the complexity
01:10:32 that emerges from the fact that
01:10:34 the timing of the firing matters in the brain,
01:10:37 the fact that you form new synapses,
01:10:40 and I mean, everything you just mentioned
01:10:44 in the past couple minutes.
01:10:44 Trust me, if you spend time on it,
01:10:46 you can get your mind around it.
01:10:47 It’s not like, it’s no longer a mystery to me.
01:10:49 No, but sorry, as a function, in a mathematical way,
01:10:53 can you start getting an intuition about
01:10:56 what gets it excited, what not,
01:10:58 and what kind of representation?
01:10:59 Yeah, it’s not as easy as,
01:11:02 there’s many other types of neural networks
01:11:04 that are more amenable to pure analysis,
01:11:09 especially very simple networks.
01:11:10 Oh, I have four neurons, and they’re doing this.
01:11:12 Can we describe to them mathematically
01:11:14 what they’re doing type of thing?
01:11:16 Even the complexity of convolutional neural networks today,
01:11:19 it’s sort of a mystery.
01:11:20 They can’t really describe the whole system.
01:11:22 And so it’s different.
01:11:24 My colleague Subitai Ahmad, he did a nice paper on this.
01:11:31 You can get all this stuff on our website
01:11:32 if you’re interested,
01:11:34 talking about sort of the mathematical properties
01:11:36 of sparse representations.
01:11:37 And so what we can do is we can show mathematically,
01:11:40 for example, why 10 to 20 synapses to recognize a pattern
01:11:44 is the correct number, is the right number you’d wanna use.
01:11:47 And by the way, that matches biology.
01:11:49 We can show mathematically some of these concepts
01:11:53 about the show why the brain is so robust
01:11:58 to noise and error and fallout and so on.
01:12:01 We can show that mathematically
01:12:02 as well as empirically in simulations.
01:12:05 But the system can’t be analyzed completely.
01:12:07 Any complex system can’t, and so that’s out of the realm.
01:12:11 But there is mathematical benefits and intuitions
01:12:17 that can be derived from mathematics.
01:12:19 And we try to do that as well.
01:12:20 Most of our papers have a section about that.
01:12:23 So I think it’s refreshing and useful for me
01:12:25 to be talking to you about deep neural networks,
01:12:29 because your intuition basically says
01:12:30 that we can’t achieve anything like intelligence
01:12:34 with artificial neural networks.
01:12:35 Well, not in the current form.
01:12:36 Not in the current form.
01:12:37 I’m sure we can do it in the ultimate form, sure.
01:12:40 So let me dig into it
01:12:41 and see what your thoughts are there a little bit.
01:12:43 So I’m not sure if you read this little blog post
01:12:45 called Bitter Lesson by Rich Sutton recently.
01:12:49 He’s a reinforcement learning pioneer.
01:12:51 I’m not sure if you’re familiar with him.
01:12:53 His basic idea is that all the stuff we’ve done in AI
01:12:56 in the past 70 years, he’s one of the old school guys.
01:13:02 The biggest lesson learned is that all the tricky things
01:13:06 we’ve done, they benefit in the short term,
01:13:10 but in the long term, what wins out
01:13:12 is a simple general method that just relies on Moore’s law,
01:13:16 on computation getting faster and faster.
01:13:19 This is what he’s saying.
01:13:21 This is what has worked up to now.
01:13:23 This is what has worked up to now.
01:13:25 If you’re trying to build a system,
01:13:29 if we’re talking about,
01:13:30 he’s not concerned about intelligence.
01:13:31 He’s concerned about a system that works
01:13:34 in terms of making predictions
01:13:36 on applied narrow AI problems, right?
01:13:38 That’s what this discussion is about.
01:13:40 That you just try to go as general as possible
01:13:44 and wait years or decades for the computation
01:13:48 to make it actually.
01:13:50 Is he saying that as a criticism
01:13:51 or is he saying this is a prescription
01:13:53 of what we ought to be doing?
01:13:54 Well, it’s very difficult.
01:13:55 He’s saying this is what has worked
01:13:57 and yes, a prescription, but it’s a difficult prescription
01:14:00 because it says all the fun things
01:14:02 you guys are trying to do, we are trying to do.
01:14:05 He’s part of the community.
01:14:07 He’s saying it’s only going to be short term gains.
01:14:10 So this all leads up to a question, I guess,
01:14:13 on artificial neural networks
01:14:15 and maybe our own biological neural networks
01:14:19 is do you think if we just scale things up significantly,
01:14:23 so take these dumb artificial neurons,
01:14:27 the point neurons, I like that term.
01:14:30 If we just have a lot more of them,
01:14:33 do you think some of the elements
01:14:34 that we see in the brain may start emerging?
01:14:38 No, I don’t think so.
01:14:39 We can do bigger problems of the same type.
01:14:43 I mean, it’s been pointed out by many people
01:14:45 that today’s convolutional neural networks
01:14:46 aren’t really much different
01:14:47 than the ones we had quite a while ago.
01:14:50 They’re bigger and train more
01:14:51 and we have more labeled data and so on.
01:14:56 But I don’t think you can get to the kind of things
01:14:58 I know the brain can do and that we think about
01:15:01 as intelligence by just scaling it up.
01:15:03 So that may be, it’s a good description
01:15:06 of what’s happened in the past,
01:15:07 what’s happened recently with the reemergence
01:15:09 of artificial neural networks.
01:15:12 It may be a good prescription
01:15:14 for what’s gonna happen in the short term.
01:15:17 But I don’t think that’s the path.
01:15:19 I’ve said that earlier.
01:15:20 There’s an alternate path.
01:15:21 I should mention to you, by the way,
01:15:22 that we’ve made sufficient progress
01:15:25 on the whole cortical theory in the last few years
01:15:28 that last year we decided to start actively pursuing
01:15:35 how do we get these ideas embedded into machine learning?
01:15:40 Well, that’s, again, being led by my colleague,
01:15:41 Subed Tariman, and he’s more of a machine learning guy.
01:15:45 I’m more of a neuroscience guy.
01:15:46 So this is now, I wouldn’t say our focus,
01:15:51 but it is now an equal focus here
01:15:54 because we need to proselytize what we’ve learned
01:15:58 and we need to show how it’s beneficial
01:16:01 to the machine learning layer.
01:16:03 So we’re putting, we have a plan in place right now.
01:16:05 In fact, we just did our first paper on this.
01:16:07 I can tell you about that.
01:16:09 But one of the reasons I wanna talk to you
01:16:11 is because I’m trying to get more people
01:16:14 in the machine learning community to say,
01:16:15 I need to learn about this stuff.
01:16:17 And maybe we should just think about this a bit more
01:16:19 about what we’ve learned about the brain
01:16:20 and what are those team at Nimenta, what have they done?
01:16:23 Is that useful for us?
01:16:25 Yeah, so is there elements of all the cortical theory
01:16:28 that things we’ve been talking about
01:16:29 that may be useful in the short term?
01:16:31 Yes, in the short term, yes.
01:16:33 This is the, sorry to interrupt,
01:16:34 but the open question is,
01:16:37 it certainly feels from my perspective
01:16:39 that in the long term,
01:16:41 some of the ideas we’ve been talking about
01:16:42 will be extremely useful.
01:16:44 The question is whether in the short term.
01:16:46 Well, this is always what I would call
01:16:48 the entrepreneur’s dilemma.
01:16:50 So you have this long term vision,
01:16:53 oh, we’re gonna all be driving electric cars
01:16:55 or we’re all gonna have computers
01:16:56 or we’re all gonna, whatever.
01:16:59 And you’re at some point in time and you say,
01:17:01 I can see that long term vision,
01:17:02 I’m sure it’s gonna happen.
01:17:03 How do I get there without killing myself?
01:17:05 Without going out of business, right?
01:17:07 That’s the challenge.
01:17:08 That’s the dilemma.
01:17:09 That’s the really difficult thing to do.
01:17:11 So we’re facing that right now.
01:17:13 So ideally what you’d wanna do
01:17:14 is find some steps along the way
01:17:16 that you can get there incrementally.
01:17:17 You don’t have to like throw it all out
01:17:19 and start over again.
01:17:20 The first thing that we’ve done
01:17:22 is we focus on the sparse representations.
01:17:25 So just in case you don’t know what that means
01:17:28 or some of the listeners don’t know what that means,
01:17:31 in the brain, if I have like 10,000 neurons,
01:17:34 what you would see is maybe 2% of them active at a time.
01:17:36 You don’t see 50%, you don’t see 30%,
01:17:39 you might see 2%.
01:17:41 And it’s always like that.
01:17:42 For any set of sensory inputs?
01:17:44 It doesn’t matter if anything,
01:17:45 doesn’t matter any part of the brain.
01:17:47 But which neurons differs?
01:17:51 Which neurons are active?
01:17:52 Yeah, so let’s say I take 10,000 neurons
01:17:55 that are representing something.
01:17:56 They’re sitting there in a little block together.
01:17:57 It’s a teeny little block of neurons, 10,000 neurons.
01:18:00 And they’re representing a location,
01:18:01 they’re representing a cup,
01:18:02 they’re representing the input from my sensors.
01:18:04 I don’t know, it doesn’t matter.
01:18:05 It’s representing something.
01:18:07 The way the representations occur,
01:18:09 it’s always a sparse representation.
01:18:10 Meaning it’s a population code.
01:18:11 So which 200 cells are active tells me what’s going on.
01:18:14 It’s not, individual cells aren’t that important at all.
01:18:18 It’s the population code that matters.
01:18:20 And when you have sparse population codes,
01:18:23 then all kinds of beautiful properties come out of them.
01:18:26 So the brain uses sparse population codes.
01:18:28 We’ve written and described these benefits
01:18:30 in some of our papers.
01:18:32 So they give this tremendous robustness to the systems.
01:18:37 Brains are incredibly robust.
01:18:39 Neurons are dying all the time and spasming
01:18:41 and synapses are falling apart all the time.
01:18:43 And it keeps working.
01:18:45 So what Sibutai and Louise, one of our other engineers here
01:18:51 have done, have shown they’re introducing sparseness
01:18:55 into convolutional neural networks.
01:18:56 Now other people are thinking along these lines,
01:18:58 but we’re going about it in a more principled way, I think.
01:19:00 And we’re showing that if you enforce sparseness
01:19:04 throughout these convolutional neural networks
01:19:07 in both the act, which sort of,
01:19:09 which neurons are active and the connections between them,
01:19:12 that you get some very desirable properties.
01:19:15 So one of the current hot topics in deep learning right now
01:19:18 are these adversarial examples.
01:19:20 So, you know, you give me any deep learning network
01:19:23 and I can give you a picture that looks perfect
01:19:26 and you’re going to call it, you know,
01:19:27 you’re going to say the monkey is, you know, an airplane.
01:19:30 So that’s a problem.
01:19:32 And DARPA just announced some big thing.
01:19:34 They’re trying to, you know, have some contest for this.
01:19:36 But if you enforce sparse representations here,
01:19:40 many of these problems go away.
01:19:41 They’re much more robust and they’re not easy to fool.
01:19:44 So we’ve already shown some of those results,
01:19:48 just literally in January or February,
01:19:51 just like last month we did that.
01:19:53 And you can, I think it’s on bioRxiv right now,
01:19:57 or on iRxiv, you can read about it.
01:19:59 But, so that’s like a baby step, okay?
01:20:03 That’s taking something from the brain.
01:20:04 We know about sparseness.
01:20:05 We know why it’s important.
01:20:06 We know what it gives the brain.
01:20:08 So let’s try to enforce that onto this.
01:20:09 What’s your intuition why sparsity leads to robustness?
01:20:12 Because it feels like it would be less robust.
01:20:15 Why would you feel the rest robust to you?
01:20:17 So it just feels like if the fewer neurons are involved,
01:20:24 the more fragile the representation.
01:20:26 But I didn’t say there was lots of few neurons.
01:20:28 I said, let’s say 200.
01:20:29 That’s a lot.
01:20:31 There’s still a lot, it’s just.
01:20:32 So here’s an intuition for it.
01:20:35 This is a bit technical, so for engineers,
01:20:39 machine learning people, this will be easy,
01:20:41 but all the listeners, maybe not.
01:20:44 If you’re trying to classify something,
01:20:45 you’re trying to divide some very high dimensional space
01:20:48 into different pieces, A and B.
01:20:50 And you’re trying to create some point where you say,
01:20:52 all these points in this high dimensional space are A,
01:20:54 and all these points in this high dimensional space are B.
01:20:57 And if you have points that are close to that line,
01:21:01 it’s not very robust.
01:21:02 It works for all the points you know about,
01:21:04 but it’s not very robust,
01:21:07 because you can just move a little bit
01:21:08 and you’ve crossed over the line.
01:21:10 When you have sparse representations,
01:21:12 imagine I pick, I’m gonna pick 200 cells active
01:21:16 out of 10,000, okay?
01:21:19 So I have 200 cells active.
01:21:20 Now let’s say I pick randomly another,
01:21:22 a different representation, 200.
01:21:24 The overlap between those is gonna be very small,
01:21:26 just a few.
01:21:28 I can pick millions of samples randomly of 200 neurons,
01:21:32 and not one of them will overlap more than just a few.
01:21:36 So one way to think about it is,
01:21:39 if I wanna fool one of these representations
01:21:41 to look like one of those other representations,
01:21:43 I can’t move just one cell, or two cells,
01:21:45 or three cells, or four cells.
01:21:46 I have to move 100 cells.
01:21:49 And that makes them robust.
01:21:52 In terms of further, so you mentioned sparsity.
01:21:56 What would be the next thing?
01:21:57 Yeah.
01:21:58 Okay, so we have, we picked one.
01:22:00 We don’t know if it’s gonna work well yet.
01:22:02 So again, we’re trying to come up with incremental ways
01:22:04 to moving from brain theory to add pieces
01:22:07 to machine learning, current machine learning world,
01:22:10 and one step at a time.
01:22:12 So the next thing we’re gonna try to do
01:22:13 is sort of incorporate some of the ideas
01:22:15 of the thousand brains theory,
01:22:19 that you have many, many models that are voting.
01:22:22 Now that idea is not new.
01:22:23 There’s a mixture of models that’s been around
01:22:25 for a long time.
01:22:27 But the way the brain does it is a little different.
01:22:29 And the way it votes is different.
01:22:33 And the kind of way it represents uncertainty
01:22:36 is different.
01:22:37 So we’re just starting this work,
01:22:39 but we’re gonna try to see if we can sort of incorporate
01:22:42 some of the principles of voting,
01:22:43 or principles of the thousand brain theory.
01:22:45 Like lots of simple models that talk to each other
01:22:49 in a certain way.
01:22:53 And can we build more machines, systems that learn faster
01:22:57 and also, well mostly are multimodal
01:23:03 and robust to multimodal type of issues.
01:23:07 So one of the challenges there
01:23:09 is the machine learning computer vision community
01:23:13 has certain sets of benchmarks,
01:23:15 sets of tests based on which they compete.
01:23:18 And I would argue, especially from your perspective,
01:23:22 that those benchmarks aren’t that useful
01:23:24 for testing the aspects that the brain is good at,
01:23:28 or intelligence.
01:23:29 They’re not really testing intelligence.
01:23:31 They’re very fine.
01:23:32 And it’s been extremely useful
01:23:34 for developing specific mathematical models,
01:23:37 but it’s not useful in the long term
01:23:40 for creating intelligence.
01:23:41 So you think you also have a role in proposing
01:23:44 better tests?
01:23:47 Yeah, this is a very,
01:23:48 you’ve identified a very serious problem.
01:23:51 First of all, the tests that they have
01:23:53 are the tests that they want.
01:23:54 Not the tests of the other things
01:23:55 that we’re trying to do, right?
01:23:58 You know, what are the, so on.
01:24:01 The second thing is sometimes these,
01:24:04 to be competitive in these tests,
01:24:06 you have to have huge data sets and huge computing power.
01:24:10 And so, you know, and we don’t have that here.
01:24:13 We don’t have it as well as other big teams
01:24:15 that big companies do.
01:24:18 So there’s numerous issues there.
01:24:20 You know, we come out, you know,
01:24:22 where our approach to this is all based on,
01:24:24 in some sense, you might argue, elegance.
01:24:26 We’re coming at it from like a theoretical base
01:24:27 that we think, oh my God, this is so clearly elegant.
01:24:29 This is how brains work.
01:24:30 This is what intelligence is.
01:24:31 But the machine learning world has gotten in this phase
01:24:33 where they think it doesn’t matter.
01:24:35 Doesn’t matter what you think,
01:24:36 as long as you do, you know, 0.1% better on this benchmark,
01:24:39 that’s what, that’s all that matters.
01:24:40 And that’s a problem.
01:24:43 You know, we have to figure out how to get around that.
01:24:46 That’s a challenge for us.
01:24:47 That’s one of the challenges that we have to deal with.
01:24:50 So I agree, you’ve identified a big issue.
01:24:52 It’s difficult for those reasons.
01:24:55 But you know, part of the reasons I’m talking to you here
01:24:59 today is I hope I’m gonna get some machine learning people
01:25:01 to say, I’m gonna read those papers.
01:25:03 Those might be some interesting ideas.
01:25:04 I’m tired of doing this 0.1% improvement stuff, you know?
01:25:08 Well, that’s why I’m here as well,
01:25:10 because I think machine learning now as a community
01:25:13 is at a place where the next step needs to be orthogonal
01:25:18 to what has received success in the past.
01:25:21 Well, you see other leaders saying this,
01:25:23 machine learning leaders, you know,
01:25:25 Jeff Hinton with his capsules idea.
01:25:27 Many people have gotten up to say, you know,
01:25:29 we’re gonna hit road map, maybe we should look at the brain,
01:25:32 you know, things like that.
01:25:33 So hopefully that thinking will occur organically.
01:25:38 And then we’re in a nice position for people to come
01:25:40 and look at our work and say,
01:25:41 well, what can we learn from these guys?
01:25:43 Yeah, MIT is launching a billion dollar computing college
01:25:47 that’s centered around this idea, so.
01:25:49 Is it on this idea of what?
01:25:50 Well, the idea that, you know,
01:25:52 the humanities, psychology, and neuroscience
01:25:54 have to work all together to get to build the S.
01:25:58 Yeah, I mean, Stanford just did
01:26:00 this Human Centered AI Center.
01:26:02 I’m a little disappointed in these initiatives
01:26:04 because, you know, they’re focusing
01:26:08 on sort of the human side of it,
01:26:09 and it could very easily slip into
01:26:12 how humans interact with intelligent machines,
01:26:16 which is nothing wrong with that,
01:26:17 but that’s not, that is orthogonal
01:26:19 to what we’re trying to do.
01:26:20 We’re trying to say, like,
01:26:21 what is the essence of intelligence?
01:26:22 I don’t care.
01:26:23 In fact, I wanna build intelligent machines
01:26:25 that aren’t emotional, that don’t smile at you,
01:26:28 that, you know, that aren’t trying to tuck you in at night.
01:26:31 Yeah, there is that pattern that you,
01:26:34 when you talk about understanding humans
01:26:36 is important for understanding intelligence,
01:26:38 that you start slipping into topics of ethics
01:26:41 or, yeah, like you said,
01:26:43 the interactive elements as opposed to,
01:26:45 no, no, no, we have to zoom in on the brain,
01:26:47 study what the human brain, the baby, the…
01:26:51 Let’s study what a brain does.
01:26:52 Does.
01:26:53 And then we can decide which parts of that
01:26:54 we wanna recreate in some system,
01:26:57 but until you have that theory about what the brain does,
01:26:59 what’s the point, you know, it’s just,
01:27:01 you’re gonna be wasting time, I think.
01:27:02 Right, just to break it down
01:27:04 on the artificial neural network side,
01:27:05 maybe you could speak to this
01:27:06 on the biological neural network side,
01:27:09 the process of learning versus the process of inference.
01:27:13 Maybe you can explain to me,
01:27:15 is there a difference between,
01:27:18 you know, in artificial neural networks,
01:27:19 there’s a difference between the learning stage
01:27:21 and the inference stage.
01:27:22 Do you see the brain as something different?
01:27:24 One of the big distinctions that people often say,
01:27:29 I don’t know how correct it is,
01:27:30 is artificial neural networks need a lot of data.
01:27:32 They’re very inefficient learning.
01:27:34 Do you see that as a correct distinction
01:27:37 from the biology of the human brain,
01:27:40 that the human brain is very efficient,
01:27:41 or is that just something we deceive ourselves?
01:27:44 No, it is efficient, obviously.
01:27:45 We can learn new things almost instantly.
01:27:47 And so what elements do you think are useful?
01:27:50 Yeah, I can talk about that.
01:27:50 You brought up two issues there.
01:27:52 So remember I talked early about the constraints
01:27:54 we always feel, well, one of those constraints
01:27:57 is the fact that brains are continually learning.
01:28:00 That’s not something we said, oh, we can add that later.
01:28:03 That’s something that was upfront,
01:28:05 had to be there from the start,
01:28:08 made our problems harder.
01:28:11 But we showed, going back to the 2016 paper
01:28:14 on sequence memory, we showed how that happens,
01:28:16 how the brains infer and learn at the same time.
01:28:19 And our models do that.
01:28:21 And they’re not two separate phases,
01:28:24 or two separate sets of time.
01:28:26 I think that’s a big, big problem in AI,
01:28:29 at least for many applications, not for all.
01:28:33 So I can talk about that.
01:28:34 There are some, it gets detailed,
01:28:37 there are some parts of the neocortex in the brain
01:28:39 where actually what’s going on,
01:28:41 there’s these cycles of activity in the brain.
01:28:46 And there’s very strong evidence
01:28:49 that you’re doing more of inference
01:28:51 on one part of the phase,
01:28:52 and more of learning on the other part of the phase.
01:28:54 So the brain can actually sort of separate
01:28:55 different populations of cells
01:28:56 or going back and forth like this.
01:28:58 But in general, I would say that’s an important problem.
01:29:01 We have all of our networks that we’ve come up with do both.
01:29:05 And they’re continuous learning networks.
01:29:08 And you mentioned benchmarks earlier.
01:29:10 Well, there are no benchmarks about that.
01:29:12 So we have to, we get in our little soapbox,
01:29:17 and hey, by the way, this is important,
01:29:19 and here’s a mechanism for doing that.
01:29:20 But until you can prove it to someone
01:29:23 in some commercial system or something, it’s a little harder.
01:29:26 So yeah, one of the things I had to linger on that
01:29:28 is in some ways to learn the concept of a coffee cup,
01:29:33 you only need this one coffee cup
01:29:35 and maybe some time alone in a room with it.
01:29:37 Well, the first thing is,
01:29:39 imagine I reach my hand into a black box
01:29:41 and I’m reaching, I’m trying to touch something.
01:29:43 I don’t know upfront if it’s something I already know
01:29:46 or if it’s a new thing.
01:29:47 And I have to, I’m doing both at the same time.
01:29:50 I don’t say, oh, let’s see if it’s a new thing.
01:29:53 Oh, let’s see if it’s an old thing.
01:29:54 I don’t do that.
01:29:55 As I go, my brain says, oh, it’s new or it’s not new.
01:29:59 And if it’s new, I start learning what it is.
01:30:02 And by the way, it starts learning from the get go,
01:30:04 even if it’s gonna recognize it.
01:30:06 So they’re not separate problems.
01:30:08 And so that’s the thing there.
01:30:10 The other thing you mentioned was the fast learning.
01:30:13 So I was just talking about continuous learning,
01:30:15 but there’s also fast learning.
01:30:16 Literally, I can show you this coffee cup
01:30:18 and I say, here’s a new coffee cup.
01:30:20 It’s got the logo on it.
01:30:21 Take a look at it, done, you’re done.
01:30:23 You can predict what it’s gonna look like,
01:30:25 you know, in different positions.
01:30:27 So I can talk about that too.
01:30:29 In the brain, the way learning occurs,
01:30:34 I mentioned this earlier, but I’ll mention it again.
01:30:35 The way learning occurs,
01:30:36 imagine I am a section of a dendrite of a neuron,
01:30:40 and I’m gonna learn something new.
01:30:43 Doesn’t matter what it is.
01:30:44 I’m just gonna learn something new.
01:30:46 I need to recognize a new pattern.
01:30:48 So what I’m gonna do is I’m gonna form new synapses.
01:30:52 New synapses, we’re gonna rewire the brain
01:30:55 onto that section of the dendrite.
01:30:57 Once I’ve done that, everything else that neuron has learned
01:31:01 is not affected by it.
01:31:02 That’s because it’s isolated
01:31:04 to that small section of the dendrite.
01:31:06 They’re not all being added together, like a point neuron.
01:31:09 So if I learn something new on this segment here,
01:31:11 it doesn’t change any of the learning
01:31:13 that occur anywhere else in that neuron.
01:31:14 So I can add something without affecting previous learning.
01:31:18 And I can do it quickly.
01:31:20 Now let’s talk, we can talk about the quickness,
01:31:22 how it’s done in real neurons.
01:31:24 You might say, well, doesn’t it take time to form synapses?
01:31:26 Yes, it can take maybe an hour to form a new synapse.
01:31:30 We can form memories quicker than that,
01:31:32 and I can explain that how it happens too, if you want.
01:31:35 But it’s getting a bit neurosciencey.
01:31:39 That’s great, but is there an understanding
01:31:41 of these mechanisms at every level?
01:31:43 Yeah.
01:31:43 So from the short term memories and the forming.
01:31:48 So this idea of synaptogenesis, the growth of new synapses,
01:31:51 that’s well described, it’s well understood.
01:31:54 And that’s an essential part of learning.
01:31:55 That is learning.
01:31:56 That is learning.
01:31:58 Okay.
01:32:01 Going back many, many years,
01:32:03 people, you know, it was, what’s his name,
01:32:06 the psychologist who proposed, Hebb, Donald Hebb.
01:32:09 He proposed that learning was the modification
01:32:12 of the strength of a connection between two neurons.
01:32:15 People interpreted that as the modification
01:32:18 of the strength of a synapse.
01:32:19 He didn’t say that.
01:32:20 He just said there’s a modification
01:32:22 between the effect of one neuron and another.
01:32:24 So synaptogenesis is totally consistent
01:32:26 with what Donald Hebb said.
01:32:28 But anyway, there’s these mechanisms,
01:32:29 the growth of new synapses.
01:32:30 You can go online, you can watch a video
01:32:32 of a synapse growing in real time.
01:32:33 It’s literally, you can see this little thing going boop.
01:32:37 It’s pretty impressive.
01:32:38 So those mechanisms are known.
01:32:39 Now there’s another thing that we’ve speculated
01:32:42 and we’ve written about,
01:32:43 which is consistent with known neuroscience,
01:32:45 but it’s less proven.
01:32:48 And this is the idea, how do I form a memory
01:32:50 really, really quickly?
01:32:51 Like instantaneous.
01:32:52 If it takes an hour to grow a synapse,
01:32:54 like that’s not instantaneous.
01:32:56 So there are types of synapses called silent synapses.
01:33:01 They look like a synapse, but they don’t do anything.
01:33:04 They’re just sitting there.
01:33:04 It’s like if an action potential comes in,
01:33:07 it doesn’t release any neurotransmitter.
01:33:10 Some parts of the brain have more of these than others.
01:33:12 For example, the hippocampus has a lot of them,
01:33:14 which is where we associate most short term memory with.
01:33:18 So what we speculated, again, in that 2016 paper,
01:33:22 we proposed that the way we form very quick memories,
01:33:26 very short term memories, or quick memories,
01:33:28 is that we convert silent synapses into active synapses.
01:33:33 It’s like saying a synapse has a zero weight
01:33:36 and a one weight,
01:33:37 but the longterm memory has to be formed by synaptogenesis.
01:33:41 So you can remember something really quickly
01:33:43 by just flipping a bunch of these guys from silent to active.
01:33:46 It’s not from 0.1 to 0.15.
01:33:49 It’s like, it doesn’t do anything
01:33:50 till it releases transmitter.
01:33:52 And if I do that over a bunch of these,
01:33:53 I’ve got a very quick short term memory.
01:33:56 So I guess the lesson behind this
01:33:58 is that most neural networks today are fully connected.
01:34:01 Every neuron connects every other neuron
01:34:03 from layer to layer.
01:34:04 That’s not correct in the brain.
01:34:06 We don’t want that.
01:34:06 We actually don’t want that.
01:34:08 It’s bad.
01:34:09 You want a very sparse connectivity
01:34:10 so that any neuron connects to some subset of the neurons
01:34:14 in the other layer.
01:34:15 And it does so on a dendrite by dendrite segment basis.
01:34:18 So it’s a very some parcelated out type of thing.
01:34:21 And that then learning is not adjusting all these weights,
01:34:25 but learning is just saying,
01:34:26 okay, connect to these 10 cells here right now.
01:34:30 In that process, you know, with artificial neural networks,
01:34:32 it’s a very simple process of backpropagation
01:34:36 that adjusts the weights.
01:34:37 The process of synaptogenesis.
01:34:40 Synaptogenesis.
01:34:40 Synaptogenesis.
01:34:42 It’s even easier.
01:34:43 It’s even easier.
01:34:43 It’s even easier.
01:34:44 Backpropagation requires something
01:34:47 that really can’t happen in brains.
01:34:48 This backpropagation of this error signal,
01:34:51 that really can’t happen.
01:34:52 People are trying to make it happen in brains,
01:34:53 but it doesn’t happen in brains.
01:34:54 This is pure Hebbian learning.
01:34:56 Well, synaptogenesis is pure Hebbian learning.
01:34:58 It’s basically saying,
01:35:00 there’s a population of cells over here
01:35:01 that are active right now.
01:35:03 And there’s a population of cells over here
01:35:04 active right now.
01:35:05 How do I form connections between those active cells?
01:35:07 And it’s literally saying this guy became active.
01:35:11 These 100 neurons here became active
01:35:13 before this neuron became active.
01:35:15 So form connections to those ones.
01:35:17 That’s it.
01:35:17 There’s no propagation of error, nothing.
01:35:19 All the networks we do,
01:35:20 all the models we have work on almost completely on
01:35:25 Hebbian learning,
01:35:26 but on dendritic segments
01:35:30 and multiple synapses at the same time.
01:35:33 So now let’s sort of turn the question
01:35:34 that you already answered,
01:35:35 and maybe you can answer it again.
01:35:38 If you look at the history of artificial intelligence,
01:35:41 where do you think we stand?
01:35:43 How far are we from solving intelligence?
01:35:45 You said you were very optimistic.
01:35:47 Can you elaborate on that?
01:35:48 Yeah, it’s always the crazy question to ask
01:35:53 because no one can predict the future.
01:35:55 Absolutely.
01:35:55 So I’ll tell you a story.
01:35:58 I used to run a different neuroscience institute
01:36:01 called the Redwood Neuroscience Institute,
01:36:02 and we would hold these symposiums
01:36:04 and we’d get like 35 scientists
01:36:06 from around the world to come together.
01:36:08 And I used to ask them all the same question.
01:36:10 I would say, well, how long do you think it’ll be
01:36:11 before we understand how the neocortex works?
01:36:14 And everyone went around the room
01:36:15 and they had introduced the name
01:36:16 and they have to answer that question.
01:36:18 So I got, the typical answer was 50 to 100 years.
01:36:22 Some people would say 500 years.
01:36:24 Some people said never.
01:36:25 I said, why are you a neuroscientist?
01:36:27 It’s never gonna, it’s a good pay.
01:36:32 It’s interesting.
01:36:34 So, you know, but it doesn’t work like that.
01:36:36 As I mentioned earlier, these are not,
01:36:38 these are step functions.
01:36:39 Things happen and then bingo, they happen.
01:36:41 You can’t predict that.
01:36:43 I feel I’ve already passed a step function.
01:36:45 So if I can do my job correctly over the next five years,
01:36:50 then, meaning I can proselytize these ideas.
01:36:53 I can convince other people they’re right.
01:36:56 We can show that other people,
01:36:58 machine learning people should pay attention
01:37:00 to these ideas.
01:37:01 Then we’re definitely in an under 20 year timeframe.
01:37:04 If I can do those things, if I’m not successful in that,
01:37:07 and this is the last time anyone talks to me
01:37:09 and no one reads our papers and you know,
01:37:12 and I’m wrong or something like that,
01:37:13 then I don’t know.
01:37:15 But it’s not 50 years.
01:37:21 Think about electric cars.
01:37:22 How quickly are they gonna populate the world?
01:37:24 It probably takes about a 20 year span.
01:37:27 It’ll be something like that.
01:37:28 But I think if I can do what I said, we’re starting it.
01:37:31 And of course there could be other,
01:37:34 you said step functions.
01:37:35 It could be everybody gives up on your ideas for 20 years
01:37:40 and then all of a sudden somebody picks it up again.
01:37:42 Wait, that guy was onto something.
01:37:43 Yeah, so that would be a failure on my part, right?
01:37:47 Think about Charles Babbage.
01:37:49 Charles Babbage, he’s the guy who invented the computer
01:37:52 back in the 18 something, 1800s.
01:37:55 And everyone forgot about it until 100 years later.
01:37:59 And say, hey, this guy figured this stuff out
01:38:00 a long time ago.
01:38:02 But he was ahead of his time.
01:38:03 I don’t think, as I said,
01:38:06 I recognize this is part of any entrepreneur’s challenge.
01:38:09 I use entrepreneur broadly in this case.
01:38:11 I’m not meaning like I’m building a business
01:38:12 or trying to sell something.
01:38:13 I mean, I’m trying to sell ideas.
01:38:15 And this is the challenge as to how you get people
01:38:19 to pay attention to you, how do you get them
01:38:21 to give you positive or negative feedback,
01:38:24 how do you get the people to act differently
01:38:25 based on your ideas.
01:38:27 So we’ll see how well we do on that.
01:38:30 So you know that there’s a lot of hype
01:38:32 behind artificial intelligence currently.
01:38:34 Do you, as you look to spread the ideas
01:38:39 that are of neocortical theory, the things you’re working on,
01:38:43 do you think there’s some possibility
01:38:45 we’ll hit an AI winter once again?
01:38:47 Yeah, it’s certainly a possibility.
01:38:48 No question about it.
01:38:49 Is that something you worry about?
01:38:50 Yeah, well, I guess, do I worry about it?
01:38:54 I haven’t decided yet if that’s good or bad for my mission.
01:38:57 That’s true, that’s very true.
01:38:59 Because it’s almost like you need the winter
01:39:02 to refresh the palette.
01:39:04 Yeah, it’s like, I want, here’s what you wanna have it is.
01:39:07 You want, like to the extent that everyone is so thrilled
01:39:12 about the current state of machine learning and AI
01:39:15 and they don’t imagine they need anything else,
01:39:18 it makes my job harder.
01:39:19 If everything crashed completely
01:39:22 and every student left the field
01:39:24 and there was no money for anybody to do anything
01:39:26 and it became an embarrassment
01:39:27 to talk about machine intelligence and AI,
01:39:29 that wouldn’t be good for us either.
01:39:30 You want sort of the soft landing approach, right?
01:39:33 You want enough people, the senior people in AI
01:39:36 and machine learning to say, you know,
01:39:37 we need other approaches.
01:39:38 We really need other approaches.
01:39:40 Damn, we need other approaches.
01:39:42 Maybe we should look to the brain.
01:39:43 Okay, let’s look to the brain.
01:39:44 Who’s got some brain ideas?
01:39:45 Okay, let’s start a little project on the side here
01:39:47 trying to do brain idea related stuff.
01:39:49 That’s the ideal outcome we would want.
01:39:51 So I don’t want a total winter
01:39:53 and yet I don’t want it to be sunny all the time either.
01:39:57 So what do you think it takes to build a system
01:40:00 with human level intelligence
01:40:03 where once demonstrated you would be very impressed?
01:40:06 So does it have to have a body?
01:40:08 Does it have to have the C word we used before,
01:40:12 consciousness as an entirety in a holistic sense?
01:40:19 First of all, I don’t think the goal
01:40:20 is to create a machine that is human level intelligence.
01:40:23 I think it’s a false goal.
01:40:24 Back to Turing, I think it was a false statement.
01:40:27 We want to understand what intelligence is
01:40:29 and then we can build intelligent machines
01:40:30 of all different scales, all different capabilities.
01:40:34 A dog is intelligent.
01:40:35 I don’t need, that’d be pretty good to have a dog.
01:40:38 But what about something that doesn’t look
01:40:39 like an animal at all, in different spaces?
01:40:41 So my thinking about this is that
01:40:44 we want to define what intelligence is,
01:40:46 agree upon what makes an intelligent system.
01:40:48 We can then say, okay, we’re now gonna build systems
01:40:51 that work on those principles or some subset of them
01:40:54 and we can apply them to all different types of problems.
01:40:57 And the kind, the idea, it’s not computing.
01:41:00 We don’t ask, if I take a little one chip computer,
01:41:05 I don’t say, well, that’s not a computer
01:41:06 because it’s not as powerful as this big server over here.
01:41:09 No, no, because we know that what the principles
01:41:11 of computing are and I can apply those principles
01:41:12 to a small problem or into a big problem.
01:41:14 And same, intelligence needs to get there.
01:41:16 We have to say, these are the principles.
01:41:17 I can make a small one, a big one.
01:41:19 I can make them distributed.
01:41:19 I can put them on different sensors.
01:41:21 They don’t have to be human like at all.
01:41:23 Now, you did bring up a very interesting question
01:41:24 about embodiment.
01:41:25 Does it have to have a body?
01:41:27 It has to have some concept of movement.
01:41:30 It has to be able to move through these reference frames
01:41:33 I talked about earlier.
01:41:34 Whether it’s physically moving,
01:41:35 like I need, if I’m gonna have an AI
01:41:37 that understands coffee cups,
01:41:38 it’s gonna have to pick up the coffee cup
01:41:40 and touch it and look at it with its eyes and hands
01:41:43 or something equivalent to that.
01:41:45 If I have a mathematical AI,
01:41:48 maybe it needs to move through mathematical spaces.
01:41:51 I could have a virtual AI that lives in the internet
01:41:55 and its movements are traversing links
01:41:58 and digging into files,
01:42:00 but it’s got a location that it’s traveling
01:42:03 through some space.
01:42:04 You can’t have an AI that just take some flash thing input.
01:42:09 We call it flash inference.
01:42:10 Here’s a pattern, done.
01:42:12 No, it’s movement pattern, movement pattern,
01:42:15 movement pattern, attention, digging, building structure,
01:42:19 figuring out the model of the world.
01:42:20 So some sort of embodiment,
01:42:22 whether it’s physical or not, has to be part of it.
01:42:25 So self awareness and the way to be able to answer
01:42:28 where am I?
01:42:28 Well, you’re bringing up self,
01:42:29 that’s a different topic, self awareness.
01:42:31 No, the very narrow definition of self,
01:42:33 meaning knowing a sense of self enough to know
01:42:37 where am I in the space where it’s actually.
01:42:39 Yeah, basically the system needs to know its location
01:42:43 or each component of the system needs to know
01:42:46 where it is in the world at that point in time.
01:42:48 So self awareness and consciousness.
01:42:51 Do you think one, from the perspective of neuroscience
01:42:55 and neurocortex, these are interesting topics,
01:42:58 solvable topics.
01:42:59 Do you have any ideas of why the heck it is
01:43:02 that we have a subjective experience at all?
01:43:04 Yeah, I have a lot of thoughts on that.
01:43:05 And is it useful or is it just a side effect of us?
01:43:08 It’s interesting to think about.
01:43:10 I don’t think it’s useful as a means to figure out
01:43:13 how to build intelligent machines.
01:43:16 It’s something that systems do
01:43:20 and we can talk about what it is that are like,
01:43:22 well, if I build a system like this,
01:43:23 then it would be self aware.
01:43:25 Or if I build it like this, it wouldn’t be self aware.
01:43:28 So that’s a choice I can have.
01:43:30 It’s not like, oh my God, it’s self aware.
01:43:32 I can’t turn, I heard an interview recently
01:43:35 with this philosopher from Yale,
01:43:37 I can’t remember his name, I apologize for that.
01:43:39 But he was talking about,
01:43:39 well, if these computers are self aware,
01:43:41 then it would be a crime to unplug them.
01:43:42 And I’m like, oh, come on, that’s not,
01:43:45 I unplug myself every night, I go to sleep.
01:43:47 Is that a crime?
01:43:48 I plug myself in again in the morning and there I am.
01:43:51 So people get kind of bent out of shape about this.
01:43:56 I have very definite, very detailed understanding
01:43:59 or opinions about what it means to be conscious
01:44:02 and what it means to be self aware.
01:44:04 I don’t think it’s that interesting a problem.
01:44:06 You’ve talked to Christoph Koch.
01:44:08 He thinks that’s the only problem.
01:44:10 I didn’t actually listen to your interview with him,
01:44:12 but I know him and I know that’s the thing he cares about.
01:44:15 He also thinks intelligence and consciousness are disjoint.
01:44:18 So I mean, it’s not, you don’t have to have one or the other.
01:44:21 So he is.
01:44:21 I disagree with that.
01:44:22 I just totally disagree with that.
01:44:24 So where’s your thoughts and consciousness,
01:44:26 where does it emerge from?
01:44:27 Because it is.
01:44:28 So then we have to break it down to the two parts, okay?
01:44:30 Because consciousness isn’t one thing.
01:44:32 That’s part of the problem with that term
01:44:33 is it means different things to different people
01:44:35 and there’s different components of it.
01:44:37 There is a concept of self awareness, okay?
01:44:40 That can be very easily explained.
01:44:43 You have a model of your own body.
01:44:46 The neocortex models things in the world
01:44:48 and it also models your own body.
01:44:50 And then it has a memory.
01:44:53 It can remember what you’ve done, okay?
01:44:55 So it can remember what you did this morning,
01:44:57 can remember what you had for breakfast and so on.
01:44:59 And so I can say to you, okay, Lex,
01:45:03 were you conscious this morning when you had your bagel?
01:45:06 And you’d say, yes, I was conscious.
01:45:08 Now what if I could take your brain
01:45:10 and revert all the synapses back
01:45:12 to the state they were this morning?
01:45:14 And then I said to you, Lex,
01:45:15 were you conscious when you ate the bagel?
01:45:17 And you said, no, I wasn’t conscious.
01:45:18 I said, here’s a video of eating the bagel.
01:45:19 And you said, I wasn’t there.
01:45:22 That’s not possible
01:45:23 because I must’ve been unconscious at that time.
01:45:25 So we can just make this one to one correlation
01:45:27 between memory of your body’s trajectory through the world
01:45:31 over some period of time,
01:45:32 a memory and the ability to recall that memory
01:45:34 is what you would call conscious.
01:45:35 I was conscious of that, it’s a self awareness.
01:45:38 And any system that can recall,
01:45:41 memorize what it’s done recently
01:45:43 and bring that back and invoke it again
01:45:46 would say, yeah, I’m aware.
01:45:48 I remember what I did.
01:45:49 All right, I got it.
01:45:51 That’s an easy one.
01:45:52 Although some people think that’s a hard one.
01:45:54 The more challenging part of consciousness
01:45:57 is this one that’s sometimes used
01:45:59 going by the word of qualia,
01:46:01 which is, why does an object seem red?
01:46:04 Or what is pain?
01:46:06 And why does pain feel like something?
01:46:08 Why do I feel redness?
01:46:10 Or why do I feel painness?
01:46:12 And then I could say, well,
01:46:13 why does sight seems different than hearing?
01:46:15 It’s the same problem.
01:46:16 It’s really, these are all just neurons.
01:46:18 And so how is it that,
01:46:20 why does looking at you feel different than hearing you?
01:46:24 It feels different, but there’s just neurons in my head.
01:46:26 They’re all doing the same thing.
01:46:27 So that’s an interesting question.
01:46:29 The best treatise I’ve read about this
01:46:31 is by a guy named Oregon.
01:46:33 He wrote a book called,
01:46:35 Why Red Doesn’t Sound Like a Bell.
01:46:37 It’s a little, it’s not a trade book, easy to read,
01:46:42 but it, and it’s an interesting question.
01:46:46 Take something like color.
01:46:47 Color really doesn’t exist in the world.
01:46:49 It’s not a property of the world.
01:46:51 Property of the world that exists is light frequency.
01:46:54 And that gets turned into,
01:46:55 we have certain cells in the retina
01:46:57 that respond to different frequencies
01:46:59 different than others.
01:47:00 And so when they enter the brain,
01:47:01 you just have a bunch of axons
01:47:02 that are firing at different rates.
01:47:04 And from that, we perceive color.
01:47:06 But there is no color in the brain.
01:47:07 I mean, there’s no color coming in on those synapses.
01:47:10 It’s just a correlation between some axons
01:47:14 and some property of frequency.
01:47:17 And that isn’t even color itself.
01:47:18 Frequency doesn’t have a color.
01:47:20 It’s just what it is.
01:47:22 So then the question is,
01:47:24 well, why does it even appear to have a color at all?
01:47:27 Just as you’re describing it,
01:47:29 there seems to be a connection to those ideas
01:47:31 of reference frames.
01:47:32 I mean, it just feels like consciousness
01:47:37 having the subject,
01:47:38 assigning the feeling of red to the actual color
01:47:42 or to the wavelength is useful for intelligence.
01:47:47 Yeah, I think that’s a good way of putting it.
01:47:49 It’s useful as a predictive mechanism
01:47:51 or useful as a generalization idea.
01:47:53 It’s a way of grouping things together to say,
01:47:55 it’s useful to have a model like this.
01:47:57 So think about the well known syndrome
01:48:02 that people who’ve lost a limb experience
01:48:04 called phantom limbs.
01:48:06 And what they claim is they can have their arm is removed,
01:48:12 but they feel their arm.
01:48:13 That not only feel it, they know it’s there.
01:48:15 It’s there, I know it’s there.
01:48:17 They’ll swear to you that it’s there.
01:48:19 And then they can feel pain in their arm
01:48:20 and they’ll feel pain in their finger.
01:48:21 And if they move their non existent arm behind their back,
01:48:25 then they feel the pain behind their back.
01:48:27 So this whole idea that your arm exists
01:48:30 is a model of your brain.
01:48:31 It may or may not really exist.
01:48:34 And just like, but it’s useful to have a model of something
01:48:38 that sort of correlates to things in the world.
01:48:40 So you can make predictions about what would happen
01:48:41 when those things occur.
01:48:43 It’s a little bit of a fuzzy,
01:48:44 but I think you’re getting quite towards the answer there.
01:48:46 It’s useful for the model to express things certain ways
01:48:51 that we can then map them into these reference frames
01:48:53 and make predictions about them.
01:48:55 I need to spend more time on this topic.
01:48:57 It doesn’t bother me.
01:48:58 Do you really need to spend more time?
01:49:00 Yeah, I know.
01:49:01 It does feel special that we have subjective experience,
01:49:04 but I’m yet to know why.
01:49:07 I’m just personally curious.
01:49:09 It’s not necessary for the work we’re doing here.
01:49:11 I don’t think I need to solve that problem
01:49:13 to build intelligent machines at all, not at all.
01:49:15 But there is sort of the silly notion
01:49:17 that you described briefly
01:49:20 that doesn’t seem so silly to us humans is,
01:49:23 if you’re successful building intelligent machines,
01:49:27 it feels wrong to then turn them off.
01:49:30 Because if you’re able to build a lot of them,
01:49:33 it feels wrong to then be able to turn off the…
01:49:38 Well, why?
01:49:39 Let’s break that down a bit.
01:49:41 As humans, why do we fear death?
01:49:43 There’s two reasons we fear death.
01:49:47 Well, first of all, I’ll say,
01:49:47 when you’re dead, it doesn’t matter at all.
01:49:48 Who cares?
01:49:49 You’re dead.
01:49:50 So why do we fear death?
01:49:51 We fear death for two reasons.
01:49:53 One is because we are programmed genetically to fear death.
01:49:57 That’s a survival and pop beginning of the genes thing.
01:50:02 And we also are programmed to feel sad
01:50:05 when people we know die.
01:50:06 We don’t feel sad for someone we don’t know dies.
01:50:08 There’s people dying right now,
01:50:09 they’re only just gonna say,
01:50:10 I don’t feel bad about them,
01:50:11 because I don’t know them.
01:50:12 But if I knew them, I’d feel really bad.
01:50:13 So again, these are old brain,
01:50:16 genetically embedded things that we fear death.
01:50:19 It’s outside of those uncomfortable feelings.
01:50:24 There’s nothing else to worry about.
01:50:25 Well, wait, hold on a second.
01:50:27 Do you know the denial of death by Becker?
01:50:30 No.
01:50:31 There’s a thought that death is,
01:50:36 our whole conception of our world model
01:50:41 kind of assumes immortality.
01:50:43 And then death is this terror that underlies it all.
01:50:47 So like…
01:50:47 Some people’s world model, not mine.
01:50:50 But, okay, so what Becker would say
01:50:52 is that you’re just living in an illusion.
01:50:54 You’ve constructed an illusion for yourself
01:50:56 because it’s such a terrible terror,
01:50:59 the fact that this…
01:51:00 What’s the illusion?
01:51:01 The illusion that death doesn’t matter.
01:51:02 You’re still not coming to grips with…
01:51:04 The illusion of what?
01:51:05 That death is…
01:51:07 Going to happen.
01:51:08 Oh, like it’s not gonna happen?
01:51:10 You’re actually operating.
01:51:11 You haven’t, even though you said you’ve accepted it,
01:51:14 you haven’t really accepted the notion that you’re gonna die
01:51:16 is what you say.
01:51:16 So it sounds like you disagree with that notion.
01:51:21 Yeah, yeah, totally.
01:51:22 I literally, every night I go to bed, it’s like dying.
01:51:28 Like little deaths.
01:51:28 It’s little deaths.
01:51:29 And if I didn’t wake up, it wouldn’t matter to me.
01:51:32 Only if I knew that was gonna happen would it be bothersome.
01:51:35 If I didn’t know it was gonna happen, how would I know?
01:51:37 Then I would worry about my wife.
01:51:39 So imagine I was a loner and I lived in Alaska
01:51:43 and I lived out there and there was no animals.
01:51:45 Nobody knew I existed.
01:51:46 I was just eating these roots all the time.
01:51:48 And nobody knew I was there.
01:51:51 And one day I didn’t wake up.
01:51:54 What pain in the world would there exist?
01:51:57 Well, so most people that think about this problem
01:51:59 would say that you’re just deeply enlightened
01:52:01 or are completely delusional.
01:52:04 One of the two.
01:52:05 But I would say that’s a very enlightened way
01:52:10 to see the world.
01:52:13 That’s the rational one as well.
01:52:14 It’s rational, that’s right.
01:52:15 But the fact is we don’t,
01:52:19 I mean, we really don’t have an understanding
01:52:22 of why the heck it is we’re born and why we die
01:52:24 and what happens after we die.
01:52:25 Well, maybe there isn’t a reason, maybe there is.
01:52:27 So I’m interested in those big problems too, right?
01:52:30 You interviewed Max Tegmark,
01:52:32 and there’s people like that, right?
01:52:33 I’m interested in those big problems as well.
01:52:35 And in fact, when I was young,
01:52:38 I made a list of the biggest problems I could think of.
01:52:41 First, why does anything exist?
01:52:43 Second, why do we have the laws of physics that we have?
01:52:46 Third, is life inevitable?
01:52:49 And why is it here?
01:52:50 Fourth, is intelligence inevitable?
01:52:52 And why is it here?
01:52:53 I stopped there because I figured
01:52:55 if you can make a truly intelligent system,
01:52:57 that will be the quickest way
01:52:59 to answer the first three questions.
01:53:01 I’m serious.
01:53:04 And so I said, my mission, you asked me earlier,
01:53:07 my first mission is to understand the brain,
01:53:09 but I felt that is the shortest way
01:53:10 to get to true machine intelligence.
01:53:12 And I wanna get to true machine intelligence
01:53:13 because even if it doesn’t occur in my lifetime,
01:53:15 other people will benefit from it
01:53:17 because I think it’ll occur in my lifetime,
01:53:19 but 20 years, you never know.
01:53:23 But that will be the quickest way for us to,
01:53:26 we can make super mathematicians,
01:53:27 we can make super space explorers,
01:53:29 we can make super physicist brains that do these things
01:53:34 and that can run experiments that we can’t run.
01:53:37 We don’t have the abilities to manipulate things and so on,
01:53:40 but we can build intelligent machines that do all those things
01:53:42 with the ultimate goal of finding out the answers
01:53:46 to the other questions.
01:53:48 Let me ask you another depressing and difficult question,
01:53:51 which is once we achieve that goal of creating,
01:53:57 no, of understanding intelligence,
01:54:01 do you think we would be happier,
01:54:02 more fulfilled as a species?
01:54:04 The understanding intelligence
01:54:05 or understanding the answers to the big questions?
01:54:07 Understanding intelligence.
01:54:08 Oh, totally, totally.
01:54:11 It would be far more fun place to live.
01:54:13 You think so?
01:54:14 Oh yeah, why not?
01:54:15 I mean, just put aside this terminator nonsense
01:54:19 and just think about, you can think about,
01:54:24 we can talk about the risks of AI if you want.
01:54:26 I’d love to, so let’s talk about.
01:54:28 But I think the world would be far better knowing things.
01:54:30 We’re always better than know things.
01:54:32 Do you think it’s better, is it a better place to live in
01:54:35 that I know that our planet is one of many
01:54:37 in the solar system and the solar system’s one of many
01:54:39 in the galaxy?
01:54:40 I think it’s a more, I dread, I sometimes think like,
01:54:43 God, what would it be like to live 300 years ago?
01:54:45 I’d be looking up at the sky, I can’t understand anything.
01:54:47 Oh my God, I’d be like going to bed every night going,
01:54:49 what’s going on here?
01:54:50 Well, I mean, in some sense I agree with you,
01:54:52 but I’m not exactly sure.
01:54:54 So I’m also a scientist, so I share your views,
01:54:57 but I’m not, we’re like rolling down the hill together.
01:55:02 What’s down the hill?
01:55:03 I feel like we’re climbing a hill.
01:55:05 Whatever.
01:55:06 We’re getting closer to enlightenment
01:55:07 and you’re going down the hill.
01:55:10 We’re climbing, we’re getting pulled up a hill
01:55:12 by our curiosity.
01:55:13 Our curiosity is, we’re pulling ourselves up the hill
01:55:16 by our curiosity.
01:55:16 Yeah, Sisyphus was doing the same thing with the rock.
01:55:19 Yeah, yeah, yeah, yeah.
01:55:20 But okay, our happiness aside, do you have concerns
01:55:24 about, you talk about Sam Harris, Elon Musk,
01:55:29 of existential threats of intelligent systems?
01:55:31 No, I’m not worried about existential threats at all.
01:55:33 There are some things we really do need to worry about.
01:55:36 Even today’s AI, we have things we have to worry about.
01:55:38 We have to worry about privacy
01:55:39 and about how it impacts false beliefs in the world.
01:55:42 And we have real problems and things to worry about
01:55:47 with today’s AI.
01:55:48 And that will continue as we create more intelligent systems.
01:55:51 There’s no question, the whole issue
01:55:53 about making intelligent armaments and weapons
01:55:57 is something that really we have to think about carefully.
01:55:59 I don’t think of those as existential threats.
01:56:01 I think those are the kind of threats we always face
01:56:04 and we’ll have to face them here
01:56:05 and we’ll have to deal with them.
01:56:10 We could talk about what people think
01:56:12 are the existential threats,
01:56:13 but when I hear people talking about them,
01:56:16 they all sound hollow to me.
01:56:17 They’re based on ideas, they’re based on people
01:56:20 who really have no idea what intelligence is.
01:56:22 And if they knew what intelligence was,
01:56:24 they wouldn’t say those things.
01:56:26 So those are not experts in the field.
01:56:28 Yeah, so there’s two, right?
01:56:32 So one is like super intelligence.
01:56:33 So a system that becomes far, far superior
01:56:37 in reasoning ability than us humans.
01:56:43 How is that an existential threat?
01:56:46 Then, so there’s a lot of ways in which it could be.
01:56:49 One way is us humans are actually irrational, inefficient
01:56:54 and get in the way of, not happiness,
01:57:00 but whatever the objective function is
01:57:02 of maximizing that objective function.
01:57:04 Super intelligent.
01:57:05 The paperclip problem and things like that.
01:57:06 So the paperclip problem but with the super intelligent.
01:57:09 Yeah, yeah, yeah, yeah.
01:57:10 So we already face this threat in some sense.
01:57:15 They’re called bacteria.
01:57:17 These are organisms in the world
01:57:18 that would like to turn everything into bacteria.
01:57:21 And they’re constantly morphing,
01:57:23 they’re constantly changing to evade our protections.
01:57:26 And in the past, they have killed huge swaths
01:57:30 of populations of humans on this planet.
01:57:33 So if you wanna worry about something
01:57:34 that’s gonna multiply endlessly, we have it.
01:57:38 And I’m far more worried in that regard.
01:57:40 I’m far more worried that some scientists in the laboratory
01:57:43 will create a super virus or a super bacteria
01:57:45 that we cannot control.
01:57:47 That is a more of an existential threat.
01:57:49 Putting an intelligence thing on top of it
01:57:52 actually seems to make it less existential to me.
01:57:54 It’s like, it limits its power.
01:57:56 It limits where it can go.
01:57:57 It limits the number of things it can do in many ways.
01:57:59 A bacteria is something you can’t even see.
01:58:03 So that’s only one of those problems.
01:58:04 Yes, exactly.
01:58:05 So the other one, just in your intuition about intelligence,
01:58:09 when you think about intelligence of us humans,
01:58:12 do you think of that as something,
01:58:14 if you look at intelligence on a spectrum
01:58:16 from zero to us humans,
01:58:18 do you think you can scale that to something far,
01:58:22 far superior to all the mechanisms we’ve been talking about?
01:58:24 I wanna make another point here, Lex, before I get there.
01:58:28 Intelligence is the neocortex.
01:58:30 It is not the entire brain.
01:58:34 The goal is not to make a human.
01:58:36 The goal is not to make an emotional system.
01:58:38 The goal is not to make a system
01:58:39 that wants to have sex and reproduce.
01:58:41 Why would I build that?
01:58:42 If I wanna have a system that wants to reproduce
01:58:44 and have sex, make bacteria, make computer viruses.
01:58:47 Those are bad things, don’t do that.
01:58:49 Those are really bad, don’t do those things.
01:58:52 Regulate those.
01:58:53 But if I just say I want an intelligent system,
01:58:56 why does it have to have any of the human like emotions?
01:58:58 Why does it even care if it lives?
01:59:00 Why does it even care if it has food?
01:59:02 It doesn’t care about those things.
01:59:03 It’s just, you know, it’s just in a trance
01:59:06 thinking about mathematics or it’s out there
01:59:08 just trying to build the space for it on Mars.
01:59:14 That’s a choice we make.
01:59:15 Don’t make human like things,
01:59:17 don’t make replicating things,
01:59:18 don’t make things that have emotions,
01:59:19 just stick to the neocortex.
01:59:21 So that’s a view actually that I share
01:59:23 but not everybody shares in the sense that
01:59:25 you have faith and optimism about us as engineers of systems,
01:59:29 humans as builders of systems to not put in stupid, not.
01:59:34 So this is why I mentioned the bacteria one.
01:59:37 Because you might say, well, some person’s gonna do that.
01:59:40 Well, some person today could create a bacteria
01:59:42 that’s resistant to all the known antibacterial agents.
01:59:46 So we already have that threat.
01:59:49 We already know this is going on.
01:59:51 It’s not a new threat.
01:59:52 So just accept that and then we have to deal with it, right?
01:59:56 Yeah, so my point is nothing to do with intelligence.
01:59:59 Intelligence is a separate component
02:00:01 that you might apply to a system
02:00:03 that wants to reproduce and do stupid things.
02:00:06 Let’s not do that.
02:00:07 Yeah, in fact, it is a mystery
02:00:08 why people haven’t done that yet.
02:00:10 My dad is a physicist, believes that the reason,
02:00:14 he says, for example, nuclear weapons haven’t proliferated
02:00:18 amongst evil people.
02:00:19 So one belief that I share is that
02:00:21 there’s not that many evil people in the world
02:00:25 that would use, whether it’s bacteria or nuclear weapons
02:00:31 or maybe the future AI systems to do bad.
02:00:35 So the fraction is small.
02:00:36 And the second is that it’s actually really hard,
02:00:38 technically, so the intersection between evil
02:00:42 and competent is small in terms of, and that’s the.
02:00:45 And by the way, to really annihilate humanity,
02:00:47 you’d have to have sort of the nuclear winter phenomenon,
02:00:50 which is not one person shooting or even 10 bombs.
02:00:54 You’d have to have some automated system
02:00:56 that detonates a million bombs
02:00:58 or whatever many thousands we have.
02:01:00 So extreme evil combined with extreme competence.
02:01:03 And to start with building some stupid system
02:01:05 that would automatically, Dr. Strangelove type of thing,
02:01:08 you know, I mean, look, we could have
02:01:11 some nuclear bomb go off in some major city in the world.
02:01:14 I think that’s actually quite likely, even in my lifetime.
02:01:17 I don’t think that’s an unlikely thing.
02:01:18 And it’d be a tragedy.
02:01:20 But it won’t be an existential threat.
02:01:23 And it’s the same as, you know, the virus of 1917,
02:01:26 whatever it was, you know, the influenza.
02:01:30 These bad things can happen and the plague and so on.
02:01:33 We can’t always prevent them.
02:01:35 We always try, but we can’t.
02:01:37 But they’re not existential threats
02:01:38 until we combine all those crazy things together.
02:01:41 So on the spectrum of intelligence from zero to human,
02:01:45 do you have a sense of whether it’s possible
02:01:47 to create several orders of magnitude
02:01:51 or at least double that of human intelligence?
02:01:54 Talking about neuro context.
02:01:55 I think it’s the wrong thing to say double the intelligence.
02:01:59 Break it down into different components.
02:02:01 Can I make something that’s a million times fast
02:02:03 than a human brain?
02:02:04 Yes, I can do that.
02:02:06 Could I make something that is,
02:02:09 has a lot more storage than the human brain?
02:02:10 Yes, I could do that.
02:02:11 More common, more copies of common.
02:02:13 Can I make something that attaches
02:02:14 to different sensors than human brain?
02:02:16 Yes, I can do that.
02:02:17 Could I make something that’s distributed?
02:02:19 So these people, yeah, we talked early
02:02:21 about the departure of the neocortex voting.
02:02:23 They don’t have to be co located.
02:02:24 Like, you know, they can be all around the place.
02:02:25 I could do that too.
02:02:29 Those are the levers I have, but is it more intelligent?
02:02:32 Well, it depends what I train it on.
02:02:33 What is it doing?
02:02:34 If it’s.
02:02:35 Well, so here’s the thing.
02:02:36 So let’s say larger neocortex
02:02:39 and or whatever size that allows for higher
02:02:44 and higher hierarchies to form,
02:02:47 we’re talking about reference frames and concepts.
02:02:50 Could I have something that’s a super physicist
02:02:51 or a super mathematician?
02:02:52 Yes.
02:02:53 And the question is, once you have a super physicist,
02:02:56 will they be able to understand something?
02:03:00 Do you have a sense that it will be orders of math,
02:03:02 like us compared to ants?
02:03:03 Could we ever understand it?
02:03:04 Yeah.
02:03:06 Most people cannot understand general relativity.
02:03:11 It’s a really hard thing to get.
02:03:13 I mean, yeah, you can paint it in a fuzzy picture,
02:03:15 stretchy space, you know?
02:03:17 But the field equations to do that
02:03:19 and the deep intuitions are really, really hard.
02:03:23 And I’ve tried, I’m unable to do it.
02:03:25 Like it’s easy to get special relativity,
02:03:28 but general relativity, man, that’s too much.
02:03:32 And so we already live with this to some extent.
02:03:34 The vast majority of people can’t understand actually
02:03:37 what the vast majority of other people actually know.
02:03:40 We’re just, either we don’t have the effort to,
02:03:41 or we can’t, or we don’t have time,
02:03:43 or just not smart enough, whatever.
02:03:46 But we have ways of communicating.
02:03:48 Einstein has spoken in a way that I can understand.
02:03:51 He’s given me analogies that are useful.
02:03:54 I can use those analogies from my own work
02:03:56 and think about concepts that are similar.
02:04:01 It’s not stupid.
02:04:02 It’s not like he’s existing some other plane
02:04:04 and there’s no connection with my plane in the world here.
02:04:06 So that will occur.
02:04:07 It already has occurred.
02:04:09 That’s what my point of this story is.
02:04:10 It already has occurred.
02:04:11 We live it every day.
02:04:14 One could argue that when we create machine intelligence
02:04:17 that think a million times faster than us
02:04:18 that it’ll be so far we can’t make the connections.
02:04:20 But you know, at the moment,
02:04:23 everything that seems really, really hard
02:04:25 to figure out in the world,
02:04:26 when you actually figure it out, it’s not that hard.
02:04:29 You know, almost everyone can understand the multiverses.
02:04:32 Almost everyone can understand quantum physics.
02:04:34 Almost everyone can understand these basic things,
02:04:36 even though hardly any people could figure those things out.
02:04:39 Yeah, but really understand.
02:04:41 But you don’t need to really.
02:04:42 Only a few people really understand.
02:04:43 You need to only understand the projections,
02:04:47 the sprinkles of the useful insights from that.
02:04:50 That was my example of Einstein, right?
02:04:51 His general theory of relativity is one thing
02:04:53 that very, very, very few people can get.
02:04:56 And what if we just said those other few people
02:04:58 are also artificial intelligences?
02:05:00 How bad is that?
02:05:01 In some sense they are, right?
02:05:02 Yeah, they say already.
02:05:04 I mean, Einstein wasn’t a really normal person.
02:05:06 He had a lot of weird quirks.
02:05:07 And so did the other people who worked with him.
02:05:09 So, you know, maybe they already were sort of
02:05:11 this astral plane of intelligence that,
02:05:14 we live with it already.
02:05:15 It’s not a problem.
02:05:17 It’s still useful and, you know.
02:05:20 So do you think we are the only intelligent life
02:05:22 out there in the universe?
02:05:24 I would say that intelligent life
02:05:27 has and will exist elsewhere in the universe.
02:05:29 I’ll say that.
02:05:31 There was a question about
02:05:32 contemporaneous intelligence life,
02:05:34 which is hard to even answer
02:05:35 when we think about relativity and the nature of space time.
02:05:39 Can’t say what exactly is this time
02:05:41 someplace else in the world.
02:05:43 But I think it’s, you know,
02:05:44 I do worry a lot about the filter idea,
02:05:48 which is that perhaps intelligent species
02:05:52 don’t last very long.
02:05:54 And so we haven’t been around very long.
02:05:55 And as a technological species,
02:05:57 we’ve been around for almost nothing, you know.
02:05:59 What, 200 years, something like that.
02:06:02 And we don’t have any data,
02:06:04 a good data point on whether it’s likely
02:06:06 that we’ll survive or not.
02:06:08 So do I think that there have been intelligent life
02:06:10 elsewhere in the universe?
02:06:11 Almost certainly, of course.
02:06:13 In the past, in the future, yes.
02:06:16 Does it survive for a long time?
02:06:17 I don’t know.
02:06:18 This is another reason I’m excited about our work,
02:06:21 is our work meaning the general world of AI.
02:06:24 I think we can build intelligent machines
02:06:28 that outlast us.
02:06:32 You know, they don’t have to be tied to Earth.
02:06:34 They don’t have to, you know,
02:06:35 I’m not saying they’re recreating, you know,
02:06:38 aliens, I’m just saying,
02:06:40 if I asked myself,
02:06:41 and this might be a good point to end on here.
02:06:44 If I asked myself, you know,
02:06:45 what’s special about our species?
02:06:47 We’re not particularly interesting physically.
02:06:49 We don’t fly, we’re not good swimmers,
02:06:51 we’re not very fast, we’re not very strong, you know.
02:06:54 It’s our brain, that’s the only thing.
02:06:55 And we are the only species on this planet
02:06:57 that’s built the model of the world
02:06:58 that extends beyond what we can actually sense.
02:07:01 We’re the only people who know about
02:07:03 the far side of the moon, and the other universes,
02:07:05 and I mean, other galaxies, and other stars,
02:07:07 and about what happens in the atom.
02:07:09 There’s no, that knowledge doesn’t exist anywhere else.
02:07:12 It’s only in our heads.
02:07:13 Cats don’t do it, dogs don’t do it,
02:07:15 monkeys don’t do it, it’s just on.
02:07:16 And that is what we’ve created that’s unique.
02:07:18 Not our genes, it’s knowledge.
02:07:20 And if I asked me, what is the legacy of humanity?
02:07:23 What should our legacy be?
02:07:25 It should be knowledge.
02:07:25 We should preserve our knowledge
02:07:27 in a way that it can exist beyond us.
02:07:30 And I think the best way of doing that,
02:07:32 in fact you have to do it,
02:07:33 is it has to go along with intelligent machines
02:07:34 that understand that knowledge.
02:07:37 It’s a very broad idea, but we should be thinking,
02:07:41 I call it a state planning for humanity.
02:07:43 We should be thinking about what we wanna leave behind
02:07:46 when as a species we’re no longer here.
02:07:49 And that’ll happen sometime.
02:07:51 Sooner or later it’s gonna happen.
02:07:52 And understanding intelligence and creating intelligence
02:07:56 gives us a better chance to prolong.
02:07:58 It does give us a better chance to prolong life, yes.
02:08:01 It gives us a chance to live on other planets.
02:08:03 But even beyond that, I mean our solar system
02:08:06 will disappear one day, just given enough time.
02:08:08 So I don’t know, I doubt we’ll ever be able to travel
02:08:12 to other things, but we could tell the stars,
02:08:15 but we could send intelligent machines to do that.
02:08:17 So you have an optimistic, a hopeful view of our knowledge
02:08:23 of the echoes of human civilization
02:08:26 living through the intelligent systems we create?
02:08:29 Oh, totally.
02:08:30 Well, I think the intelligent systems we create
02:08:31 are in some sense the vessel for bringing them beyond Earth
02:08:36 or making them last beyond humans themselves.
02:08:39 How do you feel about that?
02:08:41 That they won’t be human, quote unquote?
02:08:43 Who cares?
02:08:45 Human, what is human?
02:08:46 Our species are changing all the time.
02:08:48 Human today is not the same as human just 50 years ago.
02:08:52 What is human?
02:08:53 Do we care about our genetics?
02:08:54 Why is that important?
02:08:56 As I point out, our genetics are no more interesting
02:08:58 than a bacterium’s genetics.
02:08:59 It’s no more interesting than a monkey’s genetics.
02:09:01 What we have, what’s unique and what’s valuable
02:09:04 is our knowledge, what we’ve learned about the world.
02:09:07 And that is the rare thing.
02:09:09 That’s the thing we wanna preserve.
02:09:11 It’s, who cares about our genes?
02:09:13 That’s not.
02:09:14 It’s the knowledge.
02:09:16 It’s the knowledge.
02:09:17 That’s a really good place to end.
02:09:19 Thank you so much for talking to me.
02:09:20 No, it was fun.