Transcript
00:00:00 The following is a conversation with Wojciech Zaremba, cofounder of OpenAI,
00:00:05 which is one of the top organizations in the world doing artificial intelligence
00:00:09 research and development.
00:00:12 Wojciech is the head of language and cogeneration teams, building and doing
00:00:17 research on GitHub Copilot, OpenAI Codex, and GPT 3, and who knows, 4, 5, 6,
00:00:27 and, and, and plus one, and he also previously led OpenAI’s robotic efforts.
00:00:34 These are incredibly exciting projects to me that deeply challenge and expand
00:00:39 our understanding of the structure and nature of intelligence.
00:00:43 The 21st century, I think, may very well be remembered for a handful of
00:00:49 revolutionary AI systems and their implementations.
00:00:52 GPT, Codex, and applications of language models and transformers in general
00:00:57 to the language and visual domains may very well be at the core of these AI
00:01:03 systems. To support this podcast, please check out our sponsors.
00:01:08 They’re listed in the description.
00:01:11 This is the Lex Friedman podcast, and here is my conversation
00:01:15 with Wojciech Zaremba.
00:01:16 You mentioned that Sam Altman asked about the Fermi Paradox, and the people
00:01:22 at OpenAI had really sophisticated, interesting answers, so that’s when you
00:01:27 knew this is the right team to be working with.
00:01:29 So let me ask you about the Fermi Paradox, about aliens.
00:01:34 Why have we not found overwhelming evidence for aliens visiting Earth?
00:01:39 I don’t have a conviction in the answer, but rather kind of probabilistic
00:01:42 perspective on what might be, let’s say, possible answers.
00:01:46 It’s also interesting that the question itself even can touch on the, you
00:01:51 know, your typical question of what’s the meaning of life, because if you
00:01:54 assume that, like, we don’t see aliens because they destroy themselves, that
00:01:58 kind of upweights the focus on making sure that we won’t destroy ourselves.
00:02:04 At the moment, the place where I am actually with my belief, and these
00:02:10 things also change over the time, is I think that we might be alone in
00:02:15 the universe, which actually makes life more, or let’s say, consciousness
00:02:19 life, more kind of valuable, and that means that we should more appreciate it.
00:02:24 Have we always been alone?
00:02:26 So what’s your intuition about our galaxy, our universe?
00:02:29 Is it just sprinkled with graveyards of intelligent civilizations, or are
00:02:34 we truly, is life, intelligent life, truly unique?
00:02:37 At the moment, my belief that it is unique, but I would say I could also,
00:02:42 you know, there was like some footage released with UFO objects, which makes
00:02:47 me actually doubt my own belief.
00:02:49 Yes.
00:02:51 Yeah, I can tell you one crazy answer that I have heard.
00:02:53 Yes.
00:02:55 So, apparently, when you look actually at the limits of computation, you
00:03:00 can compute more if the temperature of the universe would drop.
00:03:06 Temperature of the universe would drop down.
00:03:09 So one of the things that aliens might want to do if they are truly optimizing
00:03:15 to maximize amount of compute, which, you know, maybe can lead to, or let’s
00:03:18 say simulations or so, it’s instead of wasting current entropy of the
00:03:24 universe, because, you know, we, by living, we are actually somewhat
00:03:27 wasting entropy, then you can wait for the universe to cool down such that
00:03:32 you have more computation.
00:03:33 So that’s kind of a funny answer.
00:03:34 I’m not sure if I believe in it, but that would be one of the
00:03:37 reasons why you don’t see aliens.
00:03:39 It’s also possible to some people say that maybe there is not that much
00:03:44 point in actually going to other galaxies if you can go inwards.
00:03:49 So there is no limits of what could be an experience if we could, you
00:03:54 know, connect machines to our brains while there are still some limits
00:03:58 if we want to explore the universe.
00:03:59 Yeah, there could be a lot of ways to go inwards too.
00:04:04 Once you figure out some aspect of physics, we haven’t figured out yet.
00:04:08 Maybe you can travel to different dimensions.
00:04:11 I mean, travel in three dimensional space may not be the most fun kind of travel.
00:04:19 There may be like just a huge amount of different ways to travel and it
00:04:22 doesn’t require a spaceship going slowly in 3d space to space time.
00:04:28 It also feels, you know, one of the problems is that speed of light
00:04:31 is low and the universe is vast.
00:04:34 And it seems that actually most likely if we want to travel very far, then
00:04:42 we would, instead of actually sending spaceships with humans that weight a
00:04:46 lot, we would send something similar to what Yuri Miller is working on.
00:04:51 These are like a huge sail, which is at first powered or there is a shot of
00:04:56 laser from an air and it can propel it to quarter of speed of light and sail
00:05:02 itself contains a few grams of equipment.
00:05:07 And that might be the way to actually transport matter through universe.
00:05:12 But then when you think what would it mean for humans, it means that we would
00:05:16 need to actually put their 3d printer and, you know, 3d print the human on
00:05:20 other planet, I don’t know, play them YouTube or let’s say, or like a 3d
00:05:24 print like huge human right away, or maybe a womb or so, um, yeah.
00:05:28 With our current techniques of archeology, if, if, if a civilization
00:05:34 was born and died, uh, long, long enough ago on earth, we wouldn’t be able to
00:05:39 tell, and so that makes me really sad.
00:05:43 And so I think about earth in that same way.
00:05:45 How can we leave some remnants if we do destroy ourselves?
00:05:50 How can we leave remnants for aliens in the future to discover?
00:05:53 Like, here’s some nice stuff we’ve done, like Wikipedia and YouTube.
00:05:57 Do we have it like in a satellite orbiting earth with a hard drive?
00:06:02 Like, how, how do we say, how do we back up human civilization?
00:06:07 Uh, the good parts or all of it is good parts so that, uh, it can be
00:06:13 preserved longer than our bodies can.
00:06:15 That’s a, that’s kind of, um, it’s a difficult question.
00:06:19 It also requires the difficult acceptance of the fact that we may die.
00:06:24 And if we die, we may die suddenly as a civilization.
00:06:28 So let’s see, I think it kind of depends on the cataclysm.
00:06:32 We have observed in other parts of the universe that birds of gamma rays, uh,
00:06:38 these are, uh, high energy, uh, rays of light that actually can
00:06:42 apparently kill entire galaxy.
00:06:44 So there might be actually nothing, even to, nothing to protect us from it.
00:06:48 I’m also, and I’m looking actually at the past civilizations.
00:06:51 So it’s like Aztecs or so they disappear from the surface of the earth.
00:06:56 And one can ask, why is it the case?
00:07:00 And the way I’m thinking about it is, you know, that definitely they had some
00:07:06 problem that they couldn’t solve and maybe there was a flood and all of a
00:07:10 sudden they couldn’t drink, uh, there was no potable water and they all died.
00:07:15 And, um, I think that, uh, so far the best solution to such a problems is I
00:07:24 guess, technology, so, I mean, if they would know that you can just boil
00:07:27 water and then drink it after, then that would save their civilization.
00:07:31 And even now, when we look actually at the current pandemic, it seems
00:07:36 that there, once again, actually science comes to rest.
00:07:38 And somehow science increases size of the action space.
00:07:42 And I think that’s a good thing.
00:07:44 Yeah.
00:07:44 But nature has a vastly larger action space, but still it might be a good thing
00:07:51 for us to keep on increasing action space.
00:07:55 Okay.
00:07:56 Uh, looking at past civilizations.
00:07:58 Yes.
00:07:58 But looking at the destruction of human civilization, perhaps expanding the
00:08:04 action space will add, um, actions that are easily acted upon, easily executed
00:08:12 and as a result, destroy us.
00:08:15 So let’s see, I was pondering, uh, why actually even, uh, we have
00:08:21 negative impact on the, uh, globe.
00:08:24 Because, you know, if you ask every single individual, they
00:08:27 would like to have clean air.
00:08:29 They would like healthy planet, but somehow it’s not.
00:08:32 It’s not the case that as a collective, we are not going in this direction.
00:08:36 I think that there exists very powerful system to describe what we value.
00:08:41 That’s capitalism.
00:08:42 It assigns actually monetary values to various activities.
00:08:45 At the moment, the problem in the current system is that there’s
00:08:49 some things which we value.
00:08:50 There is no cost assigned to it.
00:08:52 So even though we value clean air, or maybe we also, uh, value, uh,
00:09:00 value lack of destruction on, let’s say internet or so at the moment, these
00:09:06 quantities, you know, companies, corporations can pollute them, uh, for free.
00:09:11 So in some sense, I wished or like, and that’s, I guess, purpose of politics
00:09:20 to, to align the incentive systems.
00:09:23 And we are kind of maybe even moving in this direction.
00:09:25 The first issue is even to be able to measure the things that we value.
00:09:28 Then we can actually assign the monetary value to them.
00:09:32 Yeah.
00:09:32 And that’s, so it’s getting the data and also probably through technology,
00:09:38 enabling people to vote and to move money around in a way that is aligned
00:09:44 with their values, and that’s very much a technology question.
00:09:48 So like having one president and Congress and voting that happens every four years
00:09:55 or something like that, that’s a very outdated idea that could be some
00:09:59 technological improvements to that kind of idea.
00:10:02 So I’m thinking from time to time about these topics, but it’s also feels to me
00:10:06 that it’s, it’s a little bit like, uh, it’s hard for me to actually make
00:10:10 correct predictions.
00:10:11 What is the appropriate thing to do?
00:10:13 I extremely trust, uh, Sam Altman, our CEO on these topics here, um, like, uh,
00:10:20 I’m more on the side of being, I guess, naive hippie.
00:10:24 That, uh, yeah, that’s your life philosophy.
00:10:29 Um, well, like I think self doubt and, uh, I think hippie implies optimism.
00:10:37 Those, those two things are pretty, pretty good way to operate.
00:10:41 I mean, still, it is hard for me to actually understand how the politics
00:10:46 works or like, uh, how this, like, uh, exactly how the things would play out.
00:10:51 And Sam is, uh, really excellent with it.
00:10:54 What do you think is rarest in the universe?
00:10:56 You said we might be alone.
00:10:58 What’s hardest to build is another engineering way to ask that life,
00:11:03 intelligence or consciousness.
00:11:05 So like you said that we might be alone, which is the thing that’s hardest to get
00:11:11 to, is it just the origin of life?
00:11:13 Is it the origin of intelligence?
00:11:15 Is it the origin of consciousness?
00:11:17 So, um, let me at first explain to you my kind of mental model, what I think
00:11:23 is needed for life to appear.
00:11:25 Um, so I imagine that at some point there was this primordial, uh, soup of, uh,
00:11:32 amino acids and maybe some proteins in the ocean and, uh, you know, some
00:11:38 proteins were turning into some other proteins through reaction and, uh, you
00:11:42 can also, uh, you know, you can, you know, you can, you know, you can
00:11:46 and, uh, you can almost think about this, uh, cycle of what, uh, turns into what
00:11:52 as there is a graph essentially describing which substance turns into
00:11:55 some other substance and essentially life means that all of a sudden in the graph
00:12:00 has been created that cycle such that the same thing keeps on happening over
00:12:04 and over again, that’s what is needed for life to happen.
00:12:07 And in some sense, you can think almost that you have this gigantic graph and it
00:12:12 needs like a sufficient number of edges for the cycle to appear.
00:12:15 Um, then, um, from perspective of intelligence and consciousness, uh, my
00:12:21 current intuition is that they might be quite intertwined.
00:12:26 First of all, it might not be that it’s like a binary thing that you
00:12:29 have intelligence or consciousness.
00:12:30 It seems to be, uh, uh, more, uh, continuous component.
00:12:36 Let’s see, if we look for instance on the event networks, uh, recognizing
00:12:41 images and people are able to show that the activations of these networks
00:12:46 correlate very strongly, uh, with activations in visual cortex, uh, of
00:12:51 some monkeys, the same seems to be true about language models.
00:12:56 Um, also if you, for instance, um, look, um, if you train agent in, um, 3d
00:13:04 world, um, at first, you know, it, it, it, it barely recognizes what is going
00:13:09 on over the time, it kind of recognizes foreground from a background over the
00:13:14 time, it kind of knows where there is a foot, uh, and it just follows it.
00:13:18 Um, over the time it actually starts having a 3d perception.
00:13:22 So it is possible for instance, to look inside of the head of an agent and ask,
00:13:27 what would it see if it looks to the right?
00:13:29 And the crazy thing is, you know, initially when the agents are barely
00:13:33 trained, that these predictions are pretty bad over the time they become
00:13:37 better and better, you can still see that if you ask what happens when the
00:13:42 head is turned by 360 degrees for some time, they think that the different
00:13:47 thing appears and then at some stage they understand actually that the same
00:13:51 thing supposed to appear.
00:13:52 So they get that understanding of 3d structure.
00:13:55 It’s also, you know, very likely that they have inside some level of, of like
00:14:01 a symbolic reasoning, like a particular, these symbols for other agents.
00:14:06 So when you look at DOTA agents, they collaborate together and, uh, and, uh,
00:14:13 no, they, they, they, they have some anticipation of, uh, if, if they would
00:14:17 win battle, they have some, some expectations with respect to other
00:14:21 agents.
00:14:22 I might be, you know, too much anthropomorphizing, um, the, the, the,
00:14:26 how the things look, look, look for me, but then the fact that they have a
00:14:31 symbol for other agents, uh, makes me believe that, uh, at some stage as the,
00:14:37 uh, you know, as they are optimizing for skills, they would have also symbol to
00:14:41 describe themselves.
00:14:43 Uh, this is like a very useful symbol to have.
00:14:46 And this particularity, I would call it like a self consciousness or self
00:14:50 awareness, uh, and, uh, still it might be different from the consciousness.
00:14:55 So I guess the, the way how I’m understanding the word consciousness,
00:14:59 I’d say the experience of drinking a coffee or let’s say experience of being
00:15:03 a bat, that’s the meaning of the word consciousness.
00:15:06 It doesn’t mean to be awake.
00:15:07 Uh, yeah, it feels, it might be also somewhat related to memory and
00:15:13 recurrent connections.
00:15:14 So, um, it’s kind of like, if you look at anesthetic drugs, they might be, uh,
00:15:21 uh, like, uh, that they essentially, they, they disturb, uh, uh, brainwaves, uh, such
00:15:30 that, um, maybe memories, not, not form.
00:15:33 And so there’s a lessening of consciousness when you do that.
00:15:37 Correct.
00:15:37 And so that’s the one way to intuit what is consciousness.
00:15:41 There’s also kind of another element here.
00:15:45 It could be that it’s, you know, this kind of self awareness
00:15:49 module that you described, plus the actual subjective experience is a
00:15:56 storytelling module that tells us a story about, uh, what we’re experiencing.
00:16:05 The crazy thing.
00:16:06 So let’s say, I mean, in meditation, they teach people not to speak
00:16:11 story inside of their head.
00:16:12 And there is also some fraction of population who doesn’t have actually
00:16:17 a narrator, I know people who don’t have a narrator and, you know, they have
00:16:22 to use external people in order to, um, kind of solve tasks that
00:16:27 require internal narrator.
00:16:30 Um, so it seems that it’s possible to have the experience without the talk.
00:16:37 What are we talking about when we talk about the internal narrator?
00:16:41 Is that the voice when you’re like, yeah, I thought that that’s what you are
00:16:44 referring to while I was referring more on the, like, not an actual voice.
00:16:51 I meant like, there’s some kind of like subjective experience feels like it’s.
00:17:00 It’s fundamentally about storytelling to ourselves.
00:17:04 It feels like, like the feeling is a story that is much, uh, much
00:17:13 simpler abstraction than the raw sensory information.
00:17:17 So there feels like it’s a very high level of abstraction that, uh, is useful
00:17:23 for me to feel like entity in this world.
00:17:27 M most useful aspect of it is that because I’m conscious, I think there’s
00:17:35 an intricate connection to me, not wanting to die.
00:17:39 So like, it’s a useful hack to really prioritize not dying, like those
00:17:46 seem to be somehow connected.
00:17:47 So I’m telling the story of like, it’s rich.
00:17:50 He feels like something to be me and the fact that me exists in this world.
00:17:55 I want to preserve me.
00:17:56 And so that makes it a useful agent hack.
00:17:59 So I will just refer maybe to that first part, as you said, about that kind
00:18:03 of story of describing who you are.
00:18:05 Um, I was, uh, thinking about that even, so, you know, obviously I’m, I, I like
00:18:13 thinking about consciousness, uh, I like thinking about AI as well, and I’m trying
00:18:18 to see analogies of these things in AI, what would it correspond to?
00:18:22 So, um, um, you know, open AI train, uh, uh, a model called GPT, uh, which, uh,
00:18:34 can generate, uh, pretty, I’m using texts on arbitrary topic and, um, um, and one
00:18:42 way to control GPT is, uh, by putting into prefix at the beginning of the text, some
00:18:49 information, what would be the story about, uh, you can have even chat with, uh, uh,
00:18:55 you know, with GPT by saying that the chat is with Lex or Elon Musk or so, and, uh,
00:19:01 GPT would just pretend to be you or Elon Musk or so, and, uh, uh, it almost feels
00:19:08 that this, uh, story that we give ourselves to describe our life, it’s almost like, uh,
00:19:15 things that you put into context of GPT.
00:19:17 Yeah.
00:19:17 The primary, it’s the, and so, but the context we provide to GPT is, uh, is multimodal.
00:19:25 It’s more so GPT itself is multimodal.
00:19:27 GPT itself, uh, hasn’t learned actually from experience of single human, but from the
00:19:33 experience of humanity, it’s a chameleon.
00:19:35 You can turn it into anything and in some sense, by providing context, um, it, you
00:19:42 know, behaves as the thing that you wanted it to be.
00:19:45 Um, it’s interesting that the, you know, people have a stories of who they are.
00:19:50 And, uh, as you said, these stories, they help them to operate in the world.
00:19:54 Um, but it’s also, you know, interesting, I guess, various people find it out through
00:19:59 meditation or so that, uh, there might be some patterns that you have learned when
00:20:05 you were a kid that actually are not serving you anymore.
00:20:08 And you also might be thinking that that’s who you are and that’s actually just a story.
00:20:13 Mm hmm.
00:20:15 Yeah.
00:20:15 So it’s a useful hack, but sometimes it gets us into trouble.
00:20:18 It’s a local optima.
00:20:19 It’s a local optima.
00:20:20 You wrote that Stephen Hawking, he tweeted, Stephen Hawking asked what
00:20:24 breathes fire into equations, which meant what makes given mathematical
00:20:29 equations realize the physics of a universe.
00:20:33 Similarly, I wonder what breathes fire into computation.
00:20:37 What makes given computation conscious?
00:20:40 Okay.
00:20:41 So how do we engineer consciousness?
00:20:44 How do you breathe fire and magic?
00:20:47 How do you breathe fire and magic into the machine?
00:20:51 So, um, it seems clear to me that not every computation is conscious.
00:20:57 I mean, you can, let’s say, just keep on multiplying one matrix over and over
00:21:01 again and might be gigantic matrix.
00:21:03 You can put a lot of computation.
00:21:05 I don’t think it would be conscious.
00:21:07 So in some sense, the question is, uh, what are the computations which could be
00:21:13 conscious, uh, I mean, so, so one assumption is that it has to do purely
00:21:18 with computation that you can abstract away matter and other possibilities
00:21:22 that it’s very important was the realization of computation that it has
00:21:25 to do with some, uh, uh, force fields or so, and they bring consciousness.
00:21:30 At the moment, my intuition is that it can be fully abstracted away.
00:21:33 So in case of computation, you can ask yourself, what are the mathematical
00:21:38 objects or so that could bring such a properties?
00:21:41 So for instance, if we think about the models, uh, AI models, the, what they
00:21:49 truly try to do, uh, or like a models like GPT is, uh, uh, you know, they try
00:21:57 to predict, uh, next word or so.
00:22:00 And this turns out to be equivalent to, uh, compressing, uh, text.
00:22:05 Um, and, uh, because in some sense, compression means that, uh, you learn
00:22:11 the model of reality and you have just to, uh, remember where are your mistakes.
00:22:16 The better you are in predicting the, and, and, and in some sense, when we
00:22:20 look at our experience, also, when you look, for instance, at the car driving,
00:22:24 you know, in which direction it will go, you are good like in prediction.
00:22:27 And, um, you know, it might be the case that the consciousness is intertwined
00:22:32 with, uh, compression, it might be also the case that self consciousness, uh,
00:22:38 has to do with compress or trying to compress itself.
00:22:41 So, um, okay.
00:22:43 I was just wondering, what are the objects in, you know, mathematics or
00:22:47 computer science, which are mysterious that could, uh, that, that, that could
00:22:52 have to do with consciousness.
00:22:53 And then I thought, um, you know, you, you see in mathematics, there is
00:22:59 something called Gadel theorem, uh, which means, okay, you have, if you have
00:23:03 sufficiently complicated mathematical system, it is possible to point the
00:23:08 mathematical system back on itself.
00:23:10 In computer science, there is, uh, something called helping problem.
00:23:14 It’s, it’s somewhat similar construction.
00:23:16 So I thought that, you know, if we believe that, uh, that, uh, that under
00:23:22 assumption that consciousness has to do with, uh, with compression, uh, then
00:23:28 you could imagine that the, that the, as you keep on compressing things, then
00:23:32 at some point, it actually makes sense for the compressor to compress itself.
00:23:36 Metacompression consciousness is metacompression.
00:23:40 That’s a, that’s an I, an, an, an idea.
00:23:44 And in some sense, you know, the crazy, thank you.
00:23:47 So, uh, but do you think if we think of a Turing machine, a universal
00:23:52 Turing machine, can that achieve consciousness?
00:23:55 So is there some thing beyond our traditional definition
00:24:00 of computation that’s required?
00:24:02 So it’s a specific computation.
00:24:03 And I said, this computation has to do with compression and, uh, the compression
00:24:08 itself, maybe other way of putting it is like, uh, you are internally creating
00:24:13 the model of reality in order, like, uh, it’s like a, you try inside to simplify
00:24:18 reality in order to predict what’s going to happen.
00:24:20 And, um, that also feels somewhat similar to how I think actually about my own
00:24:25 conscious experience, though clearly I don’t have access to reality.
00:24:29 The only access to reality is through, you know, cable going to my brain and my
00:24:33 brain is creating a simulation of reality and I have access to the simulation of
00:24:37 reality.
00:24:38 Are you by any chance, uh, aware of, uh, the Hutter prize, Marcus Hutter?
00:24:44 He, uh, he made this prize for compression.
00:24:48 Uh, Wikipedia pages, and, uh, there’s a few qualities to it.
00:24:53 One, I think has to be perfect compression, which makes, I think that
00:24:57 little cork makes it much less, um, applicable to the general task of
00:25:03 intelligence, because it feels like intelligence is always going to be messy.
00:25:07 Uh, like perfect compression is feels like it’s not the right goal, but
00:25:14 it’s nevertheless a very interesting goal.
00:25:19 So for him, intelligence equals compression.
00:25:22 And so the smaller you make the file, given a large Wikipedia page, the
00:25:29 more intelligent the system has to be.
00:25:31 Yeah, that makes sense.
00:25:31 So you can make perfect compression if you store errors.
00:25:34 And I think that actually what he meant is you have algorithm plus errors.
00:25:37 Uh, by the way, Hutter, Hutter is, uh, he was a PhD advisor of Sean
00:25:44 Leck, who is a DeepMind, uh, uh, DeepMind cofounder.
00:25:48 Yeah.
00:25:49 Yeah.
00:25:49 So there’s an interesting, uh, and now he’s a DeepMind, there’s an
00:25:53 interesting, uh, network of people.
00:25:55 And he’s one of the people that I think seriously took on the task of
00:26:02 what would an AGI system look like?
00:26:04 Uh, I think for a longest time, the question of AGI was not taken
00:26:12 seriously or rather rigorously.
00:26:15 And he did just that, like mathematically speaking, what
00:26:19 would the model look like if you remove the constraints of it, having to be,
00:26:23 uh, um, having to have a reasonable amount of memory, reasonable amount
00:26:31 of, uh, running time, complexity, uh, computation time, what would it look
00:26:36 like and essentially it’s, it’s a half math, half philosophical discussion
00:26:41 of, uh, how would it like a reinforcement learning type of
00:26:45 framework look like for an AGI?
00:26:47 Yeah.
00:26:47 So he developed the framework even to describe what’s optimal with
00:26:51 respect to reinforcement learning.
00:26:53 Like there is a theoretical framework, which is, as you said, under assumption,
00:26:57 there is infinite amount of memory and compute.
00:26:59 Um, there was actually one person before his name is Solomonov, who
00:27:03 there extended, uh, Solomonov work to reinforcement learning, but there
00:27:07 exists the, uh, theoretical algorithm, which is optimal algorithm to build
00:27:13 intelligence and I can actually explain you the algorithm.
00:27:16 Yes.
00:27:18 Let’s go.
00:27:18 Let’s go.
00:27:19 So the task itself, can I just pause how absurd it is for brain in a
00:27:26 skull, trying to explain the algorithm for intelligence, just go ahead.
00:27:31 It is pretty crazy.
00:27:32 It is pretty crazy that, you know, the brain itself is actually so
00:27:34 small and it can ponder, uh, how to design algorithms that optimally
00:27:40 solve the problem of intelligence.
00:27:42 Okay.
00:27:43 All right.
00:27:43 So what’s the algorithm?
00:27:44 So let’s see.
00:27:46 So first of all, the task itself is, uh, described as, uh, you have infinite
00:27:51 sequence of zeros and ones.
00:27:53 Okay.
00:27:53 Okay. You read, uh, N bits and they are about to predict N plus one bit.
00:27:59 So that’s the task.
00:28:00 And you could imagine that every task could be casted as such a task.
00:28:04 So if for instance, you have images and labels, you can just turn every image
00:28:08 into a sequence of zeros and ones, then label, you concatenate labels and
00:28:12 you, and that that’s actually the, the, and you could, you could start by
00:28:16 having training data first, and then afterwards you have test data.
00:28:20 So theoretically any problem could be casted as a problem of predicting
00:28:25 zeros and ones on this, uh, infinite tape.
00:28:28 So, um, so let’s say you read already N bits and you want to predict N plus
00:28:35 one bit, and I will ask you to write every possible program that generates
00:28:42 these N bits.
00:28:43 Okay.
00:28:43 So, um, and you can have, you, you choose programming language.
00:28:47 It can be Python or C plus plus.
00:28:49 And the difference between programming languages, uh, might be, there is
00:28:53 a difference by constant asymptotically, your predictions will be equivalent.
00:28:59 So you read N bits, you enumerate all the programs that produce
00:29:04 these N bits in their output.
00:29:06 And then in order to predict N plus one bit, you actually weight the programs
00:29:13 according to their length.
00:29:15 And there is like a, some specific formula, how you weight them.
00:29:18 And then the N plus, uh, one bit prediction is the prediction, uh, from each
00:29:24 of these program, according to that weight.
00:29:27 Like statistically, you pick, so the smaller the program, the more likely
00:29:31 you, you are to pick the, its output.
00:29:35 So, uh, that’s, that algorithm is grounded in the hope or the intuition
00:29:42 that the simple answer is the right one.
00:29:44 It’s a formalization of it.
00:29:46 Um, it also means like, if you would ask the question after how many years
00:29:52 would, you know, sun explode, uh, you can say, hmm, it’s more likely
00:29:58 the answer is due to some power because they’re shorter program.
00:30:02 Yeah.
00:30:02 Um, then other, well, I don’t have a good intuition about, uh, how different
00:30:08 the space of short programs are from the space of large programs.
00:30:12 Like, what is the universe where short programs, uh, like run things?
00:30:18 Uh, so, so I said, the things have to agree with N bits.
00:30:22 So even if you have, you, you need to start, okay.
00:30:25 If, if you have very short program and they’re like a steel, some has, if, if
00:30:29 it’s not perfectly prediction of N bits, you have to start errors.
00:30:33 What are the errors?
00:30:34 And that gives you the full program that agrees on N bits.
00:30:38 Oh, so you don’t agree with the N bits.
00:30:40 And you store, that’s like a longer, a longer program, slightly longer program
00:30:45 because it can take these extra bits of errors.
00:30:47 That’s fascinating.
00:30:48 What’s what’s your intuition about the, the programs that are able to do cool
00:30:55 stuff like intelligence and consciousness, are they, uh, perfectly like, is, is it,
00:31:02 uh, is there if then statements in them?
00:31:05 So like, is there a lot of a good, uh, if then statements in them?
00:31:08 So like, is there a lot of exceptions that they’re storing?
00:31:11 So, um, you could imagine if there would be tremendous amount of if statements,
00:31:16 then they wouldn’t be that short.
00:31:17 In case of neural networks, you could imagine that, um, what happens is, uh,
00:31:24 they, uh, when you start with an initialized neural network, uh, it stores
00:31:29 internally many possibilities, how the, uh, how the problem can be solved.
00:31:34 And SGD is kind of magnifying some, some, uh, some, uh, paths, which are slightly
00:31:42 similar to the correct answer.
00:31:44 So it’s kind of magnifying correct programs.
00:31:46 And in some sense, SGD is a search algorithm in the program space and the
00:31:50 program space is represented by, uh, you know, kind of the wiring inside of the
00:31:56 neural network and there’s like an insane number of ways how the features can be
00:32:00 computed.
00:32:01 Let me ask you the high level, basic question that’s not so basic.
00:32:05 What is deep learning?
00:32:08 Is there a way you’d like to think of it that is different than like
00:32:11 a generic textbook definition?
00:32:14 The thing that I hinted just a second ago is maybe that, uh, closest to how I’m
00:32:19 thinking these days about deep learning.
00:32:21 So, uh, now the statement is, uh, neural networks can represent some programs.
00:32:29 Uh, it seems that various modules that we are actually adding up to, or like, uh,
00:32:33 you know, we, we want networks to be deep because we, we want multiple
00:32:37 steps of the computation and, uh, uh, and deep learning provides the way to
00:32:45 represent space of programs, which is searchable and it’s searchable with,
00:32:48 uh, stochastic gradient descent.
00:32:50 So we have an algorithm to search over humongous number of programs and
00:32:56 gradient descent kind of bubbles up the things that are, uh, tend to give correct
00:33:01 answers.
00:33:01 So a neural network with a, with fixed weights that’s optimized, do you think
00:33:09 of that as a single program?
00:33:11 Um, so there is a, uh, work by Christopher Olaj where he, uh, so he works on
00:33:18 interpretability of neural networks and he was able to, uh, to identify the
00:33:24 neural network, for instance, a detector of a wheel for a car, or the detector of
00:33:29 a mask for a car, and then he was able to separate them out and assemble them, uh,
00:33:35 together using a simple program, uh, for the detector, for a car detector.
00:33:40 That’s like, uh, if you think of traditionally defined programs, that’s
00:33:44 like a function within a program that this particular neural network was able
00:33:48 to find and you can tear that out, just like you can copy and paste it into a
00:33:53 stack overflow that, so, uh, any program is a composition of smaller programs.
00:34:00 Yeah.
00:34:00 I mean, the nice thing about the neural networks is that it allows the things
00:34:04 to be more fuzzy than in case of programs.
00:34:07 Uh, in case of programs, you have this, like a branching this way or that way.
00:34:11 And the neural networks, they, they have an easier way to, to be somewhere in
00:34:16 between or to share things.
00:34:18 What is the most beautiful or surprising idea in deep learning and the utilization
00:34:23 of these neural networks, which by the way, for people who are not familiar,
00:34:27 neural networks is a bunch of, uh, what would you say it’s inspired by the human
00:34:32 brain, there’s neurons, there’s connection between those neurons, there’s inputs and
00:34:37 there’s outputs and there’s millions or billions of those neurons and the
00:34:41 learning happens in the neural network.
00:34:44 Neurons and the learning happens, uh, by adjusting the weights on the
00:34:52 edges that connect these neurons.
00:34:54 Thank you for giving definition that I supposed to do it, but I guess you have
00:34:58 enough empathy to listeners to actually know that the, that that might be useful.
00:35:02 No, that’s like, so I’m asking Plato of like, what is the meaning of life?
00:35:07 He’s not going to answer.
00:35:09 You’re being philosophical and deep and quite profound talking about the space
00:35:13 of programs, which is, which is very interesting, but also for people who
00:35:17 just not familiar with the hell we’re talking about when we talk about deep
00:35:20 learning anyway, sorry, what is the most beautiful or surprising idea to you in,
00:35:25 in, um, in all the time you’ve worked at deep learning and you worked on a lot of.
00:35:30 Fascinating projects, applications of neural networks.
00:35:35 It doesn’t have to be big and profound.
00:35:36 It can be a cool trick.
00:35:38 Yeah.
00:35:38 I mean, I’m thinking about the trick, but like, uh, it’s still, uh, I’m using
00:35:42 to me that it works at all that let’s say that the extremely simple algorithm
00:35:47 stochastic gradient descent, which is something that I would be able to derive
00:35:52 on the piece of paper to high school student, uh, when put at the, at the
00:35:58 scale of, you know, thousands of machines actually, uh, can create the.
00:36:03 Behaviors we, which we called kind of human like behaviors.
00:36:07 So in general, any application is stochastic gradient descent
00:36:11 to neural networks is, is amazing to you.
00:36:14 So that, or is there a particular application in natural language
00:36:20 reinforcement learning, uh, and also what do you attribute that success to?
00:36:29 Is it just scale?
00:36:31 What profound insight can we take from the fact that the thing works
00:36:36 for gigantic, uh, sets of variables?
00:36:39 I mean, the interesting thing is this algorithms, they were invented decades
00:36:44 ago and, uh, people actually, uh, gave up on the idea and, um, you know, back
00:36:52 then they thought that we need profoundly different algorithms and they spent a lot
00:36:58 of cycles on very different algorithms.
00:37:00 And I believe that, uh, you know, we have seen that various, uh, various innovations
00:37:05 that say like transformer or, or dropout or so they can, uh, you know, pass the
00:37:11 help, but it’s also remarkable to me that this algorithm from sixties or so, uh, or,
00:37:18 I mean, you can even say that the gradient descent was invented by Leibniz in, I
00:37:22 guess, 18th century or so that actually is the core of learning in the past.
00:37:29 In the past people are, it’s almost like a, out of the, maybe an ego, people are
00:37:35 saying that it cannot be the case that such a simple algorithm is there, you
00:37:39 know, uh, could solve complicated problems.
00:37:44 So they were in search for the other algorithms.
00:37:48 And as I’m saying, like, I believe that actually we are in the game where there
00:37:51 is, there are actually frankly three levers.
00:37:54 There is compute, there are algorithms and there is data.
00:37:56 And, uh, if we want to build intelligent systems, we have to pull, uh, all three
00:38:01 levers and they are actually multiplicative.
00:38:05 Um, it’s also interesting.
00:38:06 So you ask, is it only compute?
00:38:08 Uh, people internally, they did the studies to determine how much gains they
00:38:14 were coming from different levers.
00:38:16 And so far we have seen that more gains came from compute than algorithms, but
00:38:20 also we are in the world that in case of compute, there is a kind of, you know,
00:38:24 exponential increase in funding and at some point it’s impossible to, uh, invest
00:38:28 more, it’s impossible to, you know, invest $10 trillion as we are speaking about
00:38:32 the, let’s say all taxes in us.
00:38:36 Uh, but you’re talking about money that could be innovation in the compute.
00:38:42 That’s that’s true as well.
00:38:43 Uh, so I mean, they’re like a few pieces.
00:38:45 So one piece is human brain is an incredible supercomputer and they’re like
00:38:51 a, it, it, it has a hundred trillion parameters or like a, if you try to count
00:39:01 the various quantities in the brain, they’re like a neuron synapses that small
00:39:05 number of neurons, there is a lot of synapses it’s unclear even how to map, uh,
00:39:10 synapses to, uh, to parameters of neural networks, but it’s clear that there are
00:39:16 many more.
00:39:17 Yeah. Um, so it might be the case that our networks are still somewhat small.
00:39:22 Uh, it also might be the case that they are more efficient than brain or less
00:39:27 efficient by some, by some huge factor.
00:39:29 Um, I also believe that there will be like a, you know, at the moment we are at
00:39:33 the stage that the, these neural networks, they require thousand X or, or like a
00:39:39 huge factor of more data than humans do.
00:39:41 And it will be a matter of, uh, um, there will be algorithms that vastly decrease
00:39:48 sample complexity, I believe so, but that place where we are heading today is
00:39:53 there are domains which contains million X more data.
00:39:58 And even though computers might be 1000 times slower than humans in learning,
00:40:02 that’s not a problem.
00:40:03 Like, uh, for instance, uh, I believe that, uh, it should be possible to create
00:40:09 super human therapist, uh, by, uh, and, and the, the, like, uh, even simple
00:40:15 steps of, of, of doing what, of, of doing it.
00:40:18 And, you know, the, the core reason is there is just machine will be able to
00:40:23 read way more transcripts of therapies, and then it should be able to speak
00:40:27 simultaneously with many more people and it should be possible to optimize it,
00:40:31 uh, all in parallel.
00:40:33 And, uh, well, there’s now you’re touching on something I deeply care about
00:40:37 and think is way harder than we imagine.
00:40:40 Um, what’s the goal of a therapist?
00:40:43 What’s the goal of therapies?
00:40:45 So, okay, so one goal now this is terrifying to me, but there’s a lot of
00:40:50 people that, uh, contemplate suicide, suffer from depression, uh, and they
00:40:57 could significantly be helped with therapy and the idea that an AI algorithm
00:41:03 might be in charge of that, it’s like a life and death task.
00:41:08 It’s, uh, the stakes are high.
00:41:12 So one goal for a therapist, whether human or AI is to prevent suicide
00:41:19 ideation to prevent suicide.
00:41:21 How do you achieve that?
00:41:23 So let’s see.
00:41:25 So to be clear, I don’t think that the current models are good enough for such
00:41:31 a task because it requires insane amount of understanding, empathy, and the
00:41:35 models are far from this place, but it’s.
00:41:38 But do you think that understanding empathy, that signal is in the data?
00:41:43 Um, I think there is some signal in the data.
00:41:45 Yes.
00:41:45 I mean, there are plenty of transcripts of conversations and it is possible to,
00:41:51 it is possible from it to understand personalities.
00:41:54 It is possible from it to understand, uh, if conversation is, uh,
00:41:59 friendly, uh, amicable, uh, uh, antagonistic, it is, I believe that the,
00:42:05 you know, given the fact that the models that we train now, they can, uh, they
00:42:12 can have, they are chameleons that they can have any personality, they might
00:42:17 turn out to be better in understanding, uh, personality of other people than
00:42:21 anyone else and they empathetic to be empathetic.
00:42:24 Yeah.
00:42:25 Interesting.
00:42:26 Yeah, interesting. Uh, but I wonder if there’s some level of, uh, multiple
00:42:34 modalities required to be able to, um, be empathetic of the human experience,
00:42:42 whether language is not enough to understand death, to understand fear,
00:42:46 to understand, uh, childhood trauma, to understand, uh, wit and humor required
00:42:54 when you’re dancing with a person who might be depressed or suffering both
00:42:59 humor and hope and love and all those kinds of things.
00:43:02 So there’s another underlying question, which is self supervised versus
00:43:07 supervised.
00:43:09 So can you get that from the data by just reading a huge number of transcripts?
00:43:16 I actually, so I think that reading huge number of transcripts is a step one.
00:43:20 It’s like at the same way as you cannot learn to dance if just from YouTube by
00:43:25 watching it, you have to actually try it out yourself.
00:43:28 And so I think that here that’s a similar situation.
00:43:31 I also wouldn’t deploy the system in the high stakes situations right away, but
00:43:36 kind of see gradually where it goes.
00:43:39 And, uh, obviously initially, uh, it would have to go hand in hand with humans.
00:43:45 But, uh, at the moment we are in the situation that actually there is many
00:43:50 more people who actually would like to have a therapy or, or speak with, with
00:43:55 someone than there are therapies out there.
00:43:57 I can, you know, I was so, so fundamentally I was thinking, what are
00:44:02 the things that, uh, can vastly increase people’s well being therapy is one of
00:44:08 them being meditation is other one, I guess maybe human connection is a third
00:44:13 one, and I guess pharmacologically it’s also possible, maybe direct brain
00:44:17 stimulation or something like that.
00:44:19 But these are pretty much options out there.
00:44:21 Then let’s say the way I’m thinking about the AGI endeavor is by default,
00:44:26 that’s an endeavor to, uh, increase amount of wealth.
00:44:29 And I believe that we can invest the increase amount of wealth for everyone
00:44:34 and simultaneously.
00:44:35 So, I mean, there are like a two endeavors that make sense to me.
00:44:39 One is like essentially increase amount of wealth.
00:44:41 And second one is, uh, increase overall human wellbeing.
00:44:46 And those are coupled together and they, they can, like, uh, I would
00:44:49 say these are different topics.
00:44:51 One can help another and, uh, you know, therapist is a, is a funny word
00:44:57 because I see friendship and love as therapy.
00:44:59 I mean, so therapist broadly defined as just friendship as a friend.
00:45:04 So like therapist is, has a very kind of clinical sense to it, but what
00:45:10 is human connection you’re like, uh, not to get all Camus and Dostoevsky on you,
00:45:17 but you know, life is suffering and we draw, we seek connection with the
00:45:23 humans as we, uh, desperately try to make sense of this world in a deep
00:45:30 overwhelming loneliness that we feel inside.
00:45:34 So I think connection has to do with understanding.
00:45:36 And I think that almost like a lack of understanding causes suffering.
00:45:40 If you speak with someone and do you, do you feel ignored that actually causes pain?
00:45:45 If you are feeling deeply understood that actually they, they, they might
00:45:50 not even tell you what to do in life, but like a pure understanding
00:45:54 or just being heard, understanding is a kind of, that’s a lot, you know,
00:45:59 just being heard, feel like you’re being heard, like somehow that’s a
00:46:04 alleviation temporarily of the loneliness that if somebody knows
00:46:10 you’re here with their body language, with the way they are, with the way
00:46:15 they look at you, with the way they talk, do you feel less alone for a brief moment?
00:46:22 Yeah, very much agree.
00:46:23 So I thought in the past about, um, somewhat similar question to yours,
00:46:28 which is what is love, uh, rather what is connection.
00:46:31 Yes. And, um, and obviously I think about these things from AI perspective.
00:46:36 What would it mean?
00:46:37 Um, so I said that, um, you know, intelligence has to do with some compression,
00:46:43 which is more or less like I can say, almost understanding of what is going around.
00:46:47 It seems to me that, uh, other aspect is there seem to be reward functions and you
00:46:52 can have, uh, uh, you know, reward for, uh, food, for maybe human connection, for,
00:46:59 uh, let’s say warmth, uh, sex and so on.
00:47:03 And, um, and it turns out that the various people might be optimizing slightly
00:47:09 different, uh, reward functions.
00:47:11 They essentially might care about different things.
00:47:14 And, uh, uh, in case of, uh, love at least the love between two people, you can say
00:47:20 that the, um, you know, boundary between people dissolves to such extent that, uh,
00:47:25 they end up optimizing each other reward functions and yeah, oh, that’s interesting.
00:47:33 Um, celebrate the success of each other.
00:47:36 Yeah.
00:47:37 In some sense, I would say love means, uh, helping others to optimize their, uh,
00:47:42 reward functions, not your reward functions, not the things that you think are
00:47:45 important, but the things that the person cares about, you try to help them to,
00:47:51 uh, optimize it.
00:47:51 So love is, uh, if you think of two reward functions, you just, it’s a condition.
00:47:56 You combine them together, pretty much maybe like with a weight and it depends
00:48:00 like the dynamic of the relationship.
00:48:02 Yeah.
00:48:03 I mean, you could imagine that if you’re fully, uh, optimizing someone’s reward
00:48:06 function without yours, then, then maybe are creating codependency or something
00:48:10 like that, but I’m not sure what’s the appropriate weight, but the interesting
00:48:14 thing is I even, I even think that the, uh, individual reward function is
00:48:19 saying that the individual person, uh, uh, we ourselves, we are actually less
00:48:27 of a unified insight.
00:48:29 So for instance, if you look at, at the donut on the one level, you might think,
00:48:33 oh, this is like, it looks tasty.
00:48:35 I would like to eat it on other level.
00:48:36 You might tell yourself, I shouldn’t be doing it because I want to gain muscles.
00:48:42 So, and you know, you might do it regardless kind of against yourself.
00:48:45 So it seems that even within ourselves, they’re almost like a kind of intertwined
00:48:50 personas and, um, I believe that the self love means that, uh, the love between all
00:48:57 these personas, which also means being able to love, love yourself when we are
00:49:04 angry or stressed or so combining all those reward functions of the different
00:49:08 selves you have and accepting that they are there, like, uh, you know, often
00:49:12 people, they have a negative self talk or they say, I don’t like when I’m angry.
00:49:16 And like, I try to imagine, try to imagine if there would be like a small
00:49:23 baby Lex, like a five years old, angry, and then they are like, you shouldn’t
00:49:29 be angry.
00:49:30 Like stop being angry.
00:49:31 Yeah.
00:49:31 But like an instant, actually you want the Lex to come over, give him a hug and
00:49:35 just like, I say, it’s fine.
00:49:37 Okay.
00:49:37 It’s going to be angry as long as you want.
00:49:39 And then he would stop or, or maybe not, or maybe not, but you cannot expect it
00:49:45 even.
00:49:45 Yeah.
00:49:46 But still, that doesn’t explain the why of love.
00:49:49 Like why is love part of the human condition?
00:49:51 Why is it useful to combine the reward functions?
00:49:56 It seems like that doesn’t, I mean, I don’t think reinforcement learning
00:50:01 frameworks can give us answers to why even, even the Hutter framework has
00:50:06 an objective function that’s static.
00:50:08 So we came to existence as a consequence of evolutionary process.
00:50:13 And in some sense, the purpose of evolution is survival.
00:50:17 And then the, this complicated optimization objective baked into us, let’s
00:50:23 say compression, which might help us operate in the real world and it baked
00:50:27 into us various reward functions.
00:50:29 Yeah.
00:50:31 Then to be clear at the moment we are operating in the regime, which is somewhat
00:50:35 out of distribution, where they even evolution optimized us.
00:50:38 It’s almost like love is a consequence of a cooperation that we’ve discovered is
00:50:42 useful.
00:50:43 Correct.
00:50:43 In some way it’s even the case.
00:50:45 If you, I just love the idea that love is like the out of distribution.
00:50:50 Or it’s not out of distribution.
00:50:51 It’s like, as you said, it evolved for cooperation.
00:50:54 Yes.
00:50:55 And I believe that the cop, like in some sense, cooperation ends up helping each
00:50:58 of us individually, so it makes sense evolutionary and there is a, in some
00:51:03 sense, and, you know, love means there is this dissolution of boundaries that you
00:51:08 have a shared reward function and we evolve to actually identify ourselves with
00:51:12 larger groups, so we can identify ourselves, you know, with a family, we can
00:51:18 identify ourselves with a country to such extent that people are willing to give
00:51:22 away their life for country.
00:51:24 So there is, we are wired actually even for love.
00:51:29 And at the moment, I guess, the, maybe it would be somewhat more beneficial if you
00:51:36 will, if we would identify ourselves with all the humanity as a whole.
00:51:40 So you can clearly see when people travel around the world, when they run into
00:51:44 person from the same country, they say, oh, which CPR and all this, like all the
00:51:48 sudden they find all these similarities.
00:51:50 They find some, they befriended those folks earlier than others.
00:51:55 So there is like a sense, some sense of the belonging. And I would say, I think
00:51:58 it would be overall good thing to the world for people to move towards, I think
00:52:05 it’s even called open individualism, move toward the mindset of a larger and
00:52:11 larger groups.
00:52:12 So the challenge there, that’s a beautiful vision and I share it to expand
00:52:17 that circle of empathy, that circle of love towards the entirety of humanity.
00:52:21 But then you start to ask, well, where do you draw the line?
00:52:25 Because why not expand it to other conscious beings?
00:52:28 And then finally, for our discussion, something I think about is why not
00:52:34 expand it to AI systems?
00:52:37 Like we, we start respecting each other when the, the person, the entity on the
00:52:42 other side has the capacity to suffer.
00:52:45 Cause then we develop a capacity to sort of empathize.
00:52:49 And so I could see AI systems that are interacting with humans more and more
00:52:54 having conscious, like displays.
00:52:58 So like they display consciousness through language and through other means.
00:53:02 And so then the question is like, well, is that consciousness?
00:53:06 Because they’re acting conscious.
00:53:08 And so, you know, the reason we don’t like torturing animals is because
00:53:15 they look like they’re suffering when they’re tortured and if AI looks like
00:53:21 it’s suffering when it’s tortured, how is that not requiring of the same kind
00:53:30 of empathy from us and respect and rights that animals do and other humans do?
00:53:35 I think it requires empathy as well.
00:53:37 I mean, I would like, I guess us or humanity or so make a progress in
00:53:42 understanding what consciousness is, because I don’t want just to be speaking
00:53:46 about that, the philosophy, but rather actually make a scientific, uh, to have
00:53:50 a, like, you know, there was a time that people thought that there is a force of
00:53:56 life and, uh, the things that have this force, they are alive.
00:54:03 And, um, I think that there is actually a path to understand exactly what
00:54:08 consciousness is and how it works.
00:54:10 Understand exactly what consciousness is.
00:54:13 And, uh, um, in some sense, it might require essentially putting
00:54:19 probes inside of a human brain, uh, what Neuralink, uh, does.
00:54:23 So the goal there, I mean, there’s several things with consciousness
00:54:26 that make it a real discipline, which is one is rigorous
00:54:30 measurement of consciousness.
00:54:32 And then the other is the engineering of consciousness,
00:54:34 which may or may not be related.
00:54:36 I mean, you could also run into trouble.
00:54:38 Like, for example, in the United States for the department, DOT,
00:54:43 department of transportation, and a lot of different places
00:54:46 put a value on human life.
00:54:48 I think DOT is, uh, values $9 million per person.
00:54:54 Sort of in that same way, you can get into trouble.
00:54:57 If you put a number on how conscious of being is, because then you can start
00:55:01 making policy, if a cow is a 0.1 or like, um, 10% as conscious as a human,
00:55:12 then you can start making calculations and it might get you into trouble.
00:55:15 But then again, that might be a very good way to do it.
00:55:18 I would like, uh, to move to that place that actually we have scientific
00:55:23 understanding what consciousness is.
00:55:25 And then we’ll be able to actually assign value.
00:55:27 And I believe that there is even the path for the experimentation in it.
00:55:32 So, uh, you know, w we said that, you know, you could put the
00:55:37 probes inside of the brain.
00:55:39 There is actually a few other things that you could do with
00:55:42 devices like Neuralink.
00:55:44 So you could imagine that the way even to measure if AI system is conscious
00:55:49 is by literally just plugging into the brain.
00:55:52 Um, I mean, that, that seems like it’s kind of easy, but the plugging
00:55:56 into the brain and asking person if they feel that their consciousness
00:55:59 expanded, um, this direction of course has some issues.
00:56:02 You can say, you know, if someone takes a psychedelic drug, they might
00:56:05 feel that their consciousness expanded, even though that drug
00:56:08 itself is not conscious.
00:56:10 Right.
00:56:11 So like, you can’t fully trust the self report of a person saying their,
00:56:15 their consciousness is expanded or not.
00:56:20 Let me ask you a little bit about psychedelics is, uh, there’ve been
00:56:23 a lot of excellent research on, uh, different psychedelics, psilocybin,
00:56:26 MDMA, even DMT drugs in general, marijuana too.
00:56:33 Uh, what do you think psychedelics do to the human mind?
00:56:36 It seems they take the human mind to some interesting places.
00:56:41 Is that just a little, uh, hack, a visual hack, or is there some
00:56:46 profound expansion of the mind?
00:56:49 So let’s see, I don’t believe in magic.
00:56:52 I believe in, uh, I believe in, uh, in science in, in causality, um, still,
00:57:00 let’s say, and then as I said, like, I think that the brain, that the, our
00:57:06 subjective experience of reality is, uh, we live in the simulation run by our
00:57:12 brain and the simulation that our brain runs, they can be very pleasant or very
00:57:17 hellish drugs, they are changing some hyper parameters of the simulation.
00:57:23 It is possible thanks to change of these hyper parameters to actually look back
00:57:27 on your experience and even see that the given things that we took for
00:57:32 granted, they are changeable.
00:57:35 So they allow to have a amazing perspective.
00:57:39 There is also, for instance, the fact that after DMT people can see the
00:57:44 full movie inside of their head, gives me further belief that the brain can generate
00:57:51 that full movie, that the brain is actually learning the model of reality
00:57:57 to such extent that it tries to predict what’s going to happen next.
00:58:00 Yeah.
00:58:00 Very high resolution.
00:58:01 So it can replay reality.
00:58:03 Extremely high resolution.
00:58:05 Yeah.
00:58:05 It’s also kind of interesting to me that somehow there seems to be some similarity
00:58:11 between these, uh, drugs and meditation itself.
00:58:16 And I actually started even these days to think about meditation as a psychedelic.
00:58:22 Do you practice meditation?
00:58:24 I practice meditation.
00:58:26 I mean, I went a few times on the retreats and it feels after like after
00:58:31 second or third day of meditation, uh, there is a, there is almost like a
00:58:39 sense of, you know, tripping what, what does the meditation retreat entail?
00:58:44 So you w you wake up early in the morning and you meditate for extended
00:58:50 period of time, uh, and yeah, so it’s optimized, even though there are other
00:58:56 people, it’s optimized for isolation.
00:58:59 So you don’t speak with anyone.
00:59:01 You don’t actually look into other people’s eyes and, uh, you know, you sit
00:59:06 on the chair and say Vipassana meditation tells you, uh, to focus on the breath.
00:59:13 So you try to put, uh, all the, all attention into breathing and, uh,
00:59:18 breathing in and breathing out.
00:59:20 And the crazy thing is that as you focus attention like that, uh, after some
00:59:26 time, their stems starts coming back, like some memories that you completely
00:59:33 forgotten, it almost feels like, uh, that you’ll have a mailbox and then you know,
00:59:39 you are just like a archiving email one by one.
00:59:43 And at some point, at some point there is this like a amazing feeling of getting
00:59:48 to mailbox zero, zero emails.
00:59:51 And, uh, it’s very pleasant.
00:59:53 It’s, it’s kind of, it’s, it’s, it’s crazy to me that, um, that once you
01:00:02 resolve these, uh, inner store stories or like inner traumas, then once there is
01:00:08 nothing, uh, left that default, uh, state of human mind is extremely peaceful and
01:00:16 happy, extreme, like, uh, some sense it, it feels that the, it feels at least to
01:00:24 me that way, how, when I was a child that I can look at any object and it’s very
01:00:30 beautiful, I have a lot of curiosity about the simple things and that’s where
01:00:34 the usual meditation takes me.
01:00:37 Are you, what are you experiencing?
01:00:40 Are you just taking in simple sensory information and they’re just enjoying
01:00:45 the rawness of that sensory information?
01:00:48 So there’s no, there’s no memories or all that kind of stuff.
01:00:52 You’re just enjoying being.
01:00:54 Yeah, pretty much.
01:00:55 I mean, still there is, uh, that it’s, it’s thoughts are slowing down.
01:01:00 Sometimes they pop up, but it’s also somehow the extended meditation takes you
01:01:06 to the space that they are way more friendly, way more positive.
01:01:11 Um, there is also this, uh, this thing that, uh, we’ve, it almost feels that the.
01:01:19 It almost feels that the, we are constantly getting a little bit of a reward
01:01:24 function and we are just spreading this reward function on various activities.
01:01:28 But if you’ll stay still for extended period of time, it kind of accumulates,
01:01:33 accumulates, accumulates, and, uh, there is a, there is a sense, there is a sense
01:01:38 that some point it passes some threshold and it feels as drop is falling into kind
01:01:46 of ocean of love and this, and that’s like, uh, this is like a very pleasant.
01:01:49 And that’s, I’m saying like, uh, that corresponds to the subjective experience.
01:01:54 Some people, uh, I guess in spiritual community, they describe it that that’s
01:02:01 the reality, and I would say, I believe that they’re like, uh, all sorts of
01:02:04 subjective experience that one can have.
01:02:07 And, uh, I believe that for instance, meditation might take you to the
01:02:11 subjective experiences with the subject.
01:02:13 Vision might take you to the subjective experiences, which are
01:02:16 very pleasant, collaborative.
01:02:18 And I would like a word to move toward a more collaborative, uh, uh, place.
01:02:24 Yeah.
01:02:25 I would say that’s very pleasant and I enjoy doing stuff like that.
01:02:28 I, um, I wonder how that maps to your, uh, mathematical model of love with, uh,
01:02:35 the reward function, combining a bunch of things, it seems like our life then is
01:02:42 just, we have this reward function and we’re accumulating a bunch of stuff
01:02:46 in it with weights, it’s like, um, like multi objective and what meditation
01:02:55 is, is you just remove them, remove them until the weight on one, uh, or
01:03:01 just a few is very high and that’s where the pleasure comes from.
01:03:05 Yeah.
01:03:05 So something similar, how I’m thinking about this.
01:03:08 So I told you that there is this like, uh, that there is a story of who you are.
01:03:14 And I think almost about it as a, you know, text prepended to GPT.
01:03:20 Yeah.
01:03:21 And, uh, some people refer to it as ego.
01:03:24 Okay.
01:03:24 There’s like a story who, who, who you are.
01:03:27 Okay.
01:03:28 So ego is the prompt for GPT three or GPT.
01:03:31 Yes.
01:03:31 Yes.
01:03:31 And that’s description of you.
01:03:32 And then with meditation, you can get to the point that actually you experience
01:03:37 things without the prompt and you experience things like as they are, you
01:03:42 are not biased over the description, how they supposed to be, uh, that’s very
01:03:47 pleasant.
01:03:47 And then we’ve respected the reward function.
01:03:50 Uh, it’s possible to get to the point that the, there is the solution of self.
01:03:55 And therefore you can say that the, or you’re having a, your, or like a, your
01:03:59 brain attempts to simulate the reward function of everyone else or like
01:04:03 everything that’s that there is this like a love, which feels like a oneness with
01:04:07 everything.
01:04:08 And that’s also, you know, very beautiful, very pleasant.
01:04:11 At some point you might have a lot of altruistic thoughts during that moment.
01:04:16 And then the self, uh, always comes back.
01:04:19 How would you recommend if somebody is interested in meditation, like a big
01:04:23 thing to take on as a project, would you recommend a meditation retreat?
01:04:27 How many days, what kind of thing would you recommend?
01:04:30 I think that actually retreat is the way to go.
01:04:32 Um, it almost feels that, uh, um, as I said, like a meditation is a psychedelic,
01:04:39 but, uh, when you take it in the small dose, you might barely feel it.
01:04:43 Once you get the high dose, actually you’re going to feel it.
01:04:46 Um, so even cold turkey, if you haven’t really seriously meditated for a long
01:04:51 period of time, just go to a retreat.
01:04:53 Yeah.
01:04:54 How many days, how many days?
01:04:55 Start weekend one weekend.
01:04:57 So like two, three days.
01:04:58 And it’s like, uh, it’s interesting that first or second day, it’s hard.
01:05:03 And at some point it becomes easy.
01:05:06 There’s a lot of seconds in a day.
01:05:08 How hard is the meditation retreat just sitting there in a chair?
01:05:13 So the thing is actually, it literally just depends on your, uh, on the,
01:05:20 your own framing, like if you are in the mindset that you are waiting for it to
01:05:24 be over, or you are waiting for a Nirvana to happen, you are waiting
01:05:28 it will be very unpleasant.
01:05:30 And in some sense, even the difficulty, it’s not even in the lack of being
01:05:36 able to speak with others, like, uh, you’re sitting there, your legs
01:05:40 will hurt from sitting in terms of like the practical things.
01:05:44 Do you experience kind of discomfort, like physical discomfort of just
01:05:48 sitting, like your, your butt being numb, your legs being sore, all that kind of
01:05:53 stuff?
01:05:54 Yes.
01:05:54 You experience it.
01:05:55 And then the, the, they teach you to observe it rather.
01:05:59 And it’s like, uh, the crazy thing is you at first might have a feeling
01:06:03 toward trying to escape it and that becomes very apparent that that’s
01:06:07 extremely unpleasant.
01:06:09 And then you just, just observe it.
01:06:11 And then at some point it just becomes, uh, it just is, it’s like, uh, I remember
01:06:18 that we’ve, Ilya told me some time ago that, uh, you know, he takes a cold
01:06:22 shower and he’s the mindset of taking a cold shower was to embrace suffering.
01:06:28 Yeah.
01:06:28 Excellent.
01:06:29 I do the same.
01:06:30 This is your style?
01:06:31 Yeah, it’s my style.
01:06:32 I like this.
01:06:34 So my style is actually, I also sometimes take cold showers.
01:06:38 It is purely observing how the water goes through my body, like a purely being
01:06:43 present, not trying to escape from there.
01:06:46 Yeah.
01:06:46 And I would say then it actually becomes pleasant.
01:06:49 It’s not like, ah, well, that that’s interesting.
01:06:52 Um, I I’m also that mean that’s, that’s the way to deal with anything really
01:06:57 difficult, especially in the physical space is to observe it to say it’s pleasant.
01:07:04 Hmm.
01:07:05 It’s a D I would use a different word.
01:07:08 You’re, um, you’re accepting of the full beauty of reality.
01:07:14 I would say, cause say pleasant.
01:07:16 But yeah, I mean, in some sense it is pleasant.
01:07:19 That’s the only way to deal with a cold shower is to, to, uh, become an
01:07:24 observer and to find joy in it.
01:07:28 Um, same with like really difficult, physical, um, exercise or like running
01:07:32 for a really long time, endurance events, just anytime you’re, any kind of pain.
01:07:38 I think the only way to survive it is not to resist it is to observe it.
01:07:43 You mentioned, you mentioned, um, you mentioned, um, you mentioned
01:07:46 Ilya, Ilya says, it’s very, he’s our chief scientist, but also
01:07:51 he’s very close friend of mine.
01:07:53 He cofounded open air with you.
01:07:56 I’ve spoken with him a few times.
01:07:58 He’s brilliant.
01:07:59 I really enjoy talking to him.
01:08:02 His mind, just like yours works in fascinating ways.
01:08:06 Now, both of you are not able to define deep learning simply.
01:08:10 Uh, what’s it like having him as somebody you have technical discussions with on
01:08:15 in the space of machine learning, deep learning, AI, but also life.
01:08:21 What’s it like when these two, um, agents get into a self play situation in a room?
01:08:29 What’s it like collaborating with him?
01:08:30 So I believe that we have, uh, extreme, uh, respect to each other.
01:08:35 So, uh, in, I love Ilya’s insight, both like, uh, I guess about
01:08:43 consciousness, uh, life AI, but, uh, in terms of the, it’s interesting to
01:08:49 me, cause you’re a brilliant, uh, Thinker in the space of machine
01:08:56 learning, like intuition, like digging deep in what works, what doesn’t,
01:09:01 why it works, why it doesn’t, and so is Ilya.
01:09:05 I’m wondering if there’s interesting deep discussions you’ve had with him in the
01:09:09 past or disagreements that were very productive.
01:09:12 So I can say, I also understood over the time, where are my strengths?
01:09:18 So obviously we have plenty of AI discussions and, um, um, and do you
01:09:24 know, I myself have plenty of ideas, but like I consider Ilya, uh, what
01:09:29 of the most prolific AI scientists in the entire world.
01:09:33 And, uh, I think that, um, I realized that maybe my super skill, um, is, uh,
01:09:40 being able to bring people to collaborate together, that I have some level of
01:09:43 empathy that is unique in AI world.
01:09:46 And that might come, you know, from either meditation, psychedelics, or
01:09:50 let’s say I read just hundreds of books on this topic.
01:09:53 So, and I also went through a journey of, you know, I developed a
01:09:56 lot of, uh, algorithms, so I think that maybe I can, that’s my super human skill.
01:10:05 Uh, Ilya is, uh, one of the best AI scientists, but then I’m pretty
01:10:11 good in assembling teams and I’m also not holding to people.
01:10:14 Like I’m growing people and then people become managers at OpenAI.
01:10:18 I grew many of them, like a research managers.
01:10:20 So you, you find, you find places where you’re excellent and he finds like his,
01:10:27 his, his deep scientific insights is where he is and you find ways you can,
01:10:31 the puzzle pieces fit together.
01:10:33 Correct.
01:10:33 Like, uh, you know, ultimately, for instance, let’s say Ilya, he doesn’t
01:10:37 manage people, uh, that’s not what he likes or so.
01:10:42 Um, I like, I like hanging out with people.
01:10:45 By default, I’m an extrovert and I care about people.
01:10:48 Oh, interesting. Okay. All right. Okay, cool.
01:10:50 So that, that fits perfectly together.
01:10:52 But I mean, uh, I also just like your intuition about various
01:10:56 problems in machine learning.
01:10:58 He’s definitely one I really enjoy.
01:11:01 I remember talking to him about something I was struggling with, which
01:11:06 is coming up with a good model for pedestrians, for human beings across
01:11:12 the street in the context of autonomous vehicles, and I was like, okay,
01:11:16 in the context of autonomous vehicles.
01:11:19 And he immediately started to like formulate a framework within which you
01:11:24 can evolve a model for pedestrians, like through self play, all that kind of
01:11:29 mechanisms, the depth of thought on a particular problem, especially problems
01:11:35 he doesn’t know anything about is, is fascinating to watch.
01:11:38 It makes you realize like, um, yeah, the, the, the limits of the, that the human
01:11:46 intellect may be limitless, or it’s just impressive to see a descendant of
01:11:50 ape come up with clever ideas.
01:11:52 Yeah.
01:11:53 I mean, so even in the space of deep learning, when you look at various
01:11:56 people, there are people now who invented some breakthroughs once, but
01:12:03 there are very few people who did it multiple times.
01:12:06 And you can think if someone invented it once, that might be just a sheer luck.
01:12:11 And if someone invented it multiple times, you know, if a probability of
01:12:15 inventing it once is one over a million, then probability of inventing it twice
01:12:19 or three times would be one over a million square or, or to the power of
01:12:22 three, which, which would be just impossible.
01:12:25 So it literally means that it’s, it’s given that, uh, it’s not the luck.
01:12:30 Yeah.
01:12:30 And Ilya is one of these few people who, uh, uh, who have, uh, a lot of
01:12:36 these inventions in his arsenal.
01:12:38 It also feels that, um, you know, for instance, if you think about folks
01:12:42 like Gauss or Euler, uh, you know, at first they read a lot of books and then
01:12:49 they did thinking and then they figure out math and that’s how it feels with
01:12:55 Ilya, you know, at first he read stuff and then like he spent his thinking cycles.
01:13:01 And that’s a really good way to put it.
01:13:05 When I talk to him, I, I see thinking.
01:13:11 He’s actually thinking, like, he makes me realize that there’s like deep
01:13:15 thinking that the human mind can do.
01:13:18 Like most of us are not thinking deeply.
01:13:21 Uh, like you really have to put in a lot of effort to think deeply.
01:13:24 Like I have to really put myself in a place where I think deeply about a
01:13:29 problem, it takes a lot of effort.
01:13:30 It’s like, uh, it’s like an airplane taking off or something.
01:13:33 You have to achieve deep focus.
01:13:35 He he’s just, uh, he’s what is it?
01:13:38 He said, what does it, his brain is like a vertical takeoff in
01:13:43 terms of airplane analogy.
01:13:45 So it’s interesting, but it, I mean, Cal Newport talks about
01:13:49 this as ideas of deep work.
01:13:51 It’s, you know, most of us don’t work much at all in terms of like, like deeply
01:13:57 think about particular problems, whether it’s a math engineering, all that kind
01:14:01 of stuff, you want to go to that place often and that’s real hard work.
01:14:06 And some of us are better than others at that.
01:14:08 So I think that the big piece has to do with actually even engineering
01:14:13 your environment that says that it’s conducive to that.
01:14:15 Yeah.
01:14:16 So, um, see both Ilya and I, uh, on the frequent basis, we kind of disconnect
01:14:22 ourselves from the world in order to be able to do extensive amount of thinking.
01:14:26 Yes.
01:14:27 So Ilya usually, he just, uh, leaves iPad at hand.
01:14:33 He loves his iPad.
01:14:34 And, uh, for me, I’m even sometimes, you know, just going for a few days
01:14:39 to different location to Airbnb, I’m turning off my phone and there is no
01:14:44 access to me and, uh, that’s extremely important for me to be able to actually
01:14:51 just formulate new thoughts, to do deep work rather than to be reactive.
01:14:55 And the, the, the older I am, the more of these random tasks are at hand.
01:15:00 Before I go on to that, uh, thread, let me return to our friend, GPT.
01:15:06 And let me ask you another ridiculously big question.
01:15:09 Can you give an overview of what GPT three is, or like you say in
01:15:13 your Twitter bio, GPT N plus one, how it works and why it works.
01:15:21 So, um, GPT three is a humongous neural network.
01:15:25 Um, let’s assume that we know what is neural network, the definition, and it
01:15:30 is trained on the entire internet and just to predict next word.
01:15:36 So let’s say it sees part of the, uh, article and it, the only task that it
01:15:41 has at hand, it is to say what would be the next word and what would be the next
01:15:45 word and it becomes a really exceptional at the task of figuring out what’s the
01:15:51 next word. So you might ask, why would, uh, this be an important, uh, task?
01:15:57 Why would it be important to predict what’s the next word?
01:16:01 And it turns out that a lot of problems, uh, can be formulated, uh, as a text
01:16:07 completion problem.
01:16:08 So GPT is purely, uh, learning to complete the text.
01:16:13 And you could imagine, for instance, if you are asking a question, uh, who is
01:16:17 the president of the United States, then GPT can give you an answer to it.
01:16:22 It turns out that many more things can be formulated this way.
01:16:25 You can format text in the way that you have sentence in English.
01:16:30 You make it even look like some content of a website, uh, elsewhere, which would
01:16:35 be teaching people how to translate things between languages.
01:16:38 So it would be EN colon, uh, text in English, FR colon, and then you’ll
01:16:43 uh, uh, and then you’ll ask people and then you ask model to, to continue.
01:16:48 And it turns out that the, such a model is predicting translation from English
01:16:52 to French.
01:16:53 The crazy thing is that this model can be used for way more sophisticated tasks.
01:17:00 So you can format text such that it looks like a conversation between two people.
01:17:05 And that might be a conversation between you and Elon Musk.
01:17:08 And because the model read all the texts about Elon Musk, it will be able to
01:17:13 predict Elon Musk words as it would be Elon Musk.
01:17:16 It will speak about colonization of Mars, about sustainable future and so on.
01:17:22 And it’s also possible to, to even give arbitrary personality to the model.
01:17:29 You can say, here is a conversation that we’ve a friendly AI bot.
01:17:32 And the model, uh, will complete the text as a friendly AI bot.
01:17:37 So, I mean, how do I express how amazing this is?
01:17:43 So just to clarify, uh, a conversation, generating a conversation between me and
01:17:49 Elon Musk, it wouldn’t just generate good examples of what Elon would say.
01:17:56 It would get the same results as the conversation between Elon Musk and me.
01:18:01 Say it would get the syntax all correct.
01:18:04 So like interview style, it would say like Elon call and Lex call, like it,
01:18:09 it’s not just like, uh, inklings of, um, semantic correctness.
01:18:17 It’s like the whole thing, grammatical, syntactic, semantic, it’s just really,
01:18:25 really impressive, uh, generalization.
01:18:30 Yeah.
01:18:30 I mean, I also want to, you know, provide some caveats so it can generate
01:18:34 few paragraphs of coherent text, but as you go to, uh, longer pieces,
01:18:38 it, uh, it actually goes off the rails.
01:18:41 Okay.
01:18:41 If you try to write a book, it won’t work out this way.
01:18:45 What way does it go off the rails, by the way?
01:18:47 Is there interesting ways in which it goes off the rails?
01:18:50 Like what falls apart first?
01:18:54 So the model is trained on the, all the existing data, uh, that is out there,
01:18:58 which means that it is not trained on its own mistakes.
01:19:02 So for instance, if it would make a mistake, then, uh, I kept,
01:19:06 so to give you, give you an example.
01:19:08 So let’s say I have a conversation with a model pretending that is Elon Musk.
01:19:14 And then I start putting some, uh, I’m start actually making up
01:19:19 things which are not factual.
01:19:21 Um, I would say like Twitter, but I got you.
01:19:25 Sorry.
01:19:26 Yeah.
01:19:26 Um, like, uh, I don’t know.
01:19:28 I would say that Elon is my wife and the model will just keep on carrying it on.
01:19:35 And as if it’s true.
01:19:37 Yes.
01:19:38 And in some sense, if you would have a normal conversation with Elon,
01:19:41 he would be what the fuck.
01:19:43 Yeah.
01:19:43 There’ll be some feedback between, so the model is trained on things
01:19:48 that humans have written, but through the generation process, there’s
01:19:52 no human in the loop feedback.
01:19:54 Correct.
01:19:55 That’s fascinating.
01:19:56 Makes sense.
01:19:57 So it’s magnified.
01:19:57 It’s like the errors get magnified and magnified and it’s also interesting.
01:20:04 I mean, first of all, humans have the same problem.
01:20:06 It’s just that we, uh, we’ll make fewer errors and magnify the errors slower.
01:20:13 I think that actually what happens with humans is if you have a wrong
01:20:17 belief about the world as a kid, then very quickly we’ll learn that it’s
01:20:21 not correct because they are grounded in reality and they are learning
01:20:25 from your new experience.
01:20:26 Yes.
01:20:27 But do you think the model can correct itself too?
01:20:30 Won’t it through the power of the representation.
01:20:34 And so the absence of Elon Musk being your wife information on the
01:20:40 internet, won’t it correct itself?
01:20:43 There won’t be examples like that.
01:20:45 So the errors will be subtle at first.
01:20:48 Subtle at first.
01:20:49 And in some sense, you can also say that the data that is not out there is
01:20:54 the data, which would represent how the human learns and maybe model would
01:21:00 be learned, trained on such a data.
01:21:01 Then it would be better off.
01:21:03 How intelligent is GPT3 do you think?
01:21:06 Like when you think about the nature of intelligence, it
01:21:10 seems exceptionally impressive.
01:21:14 But then if you think about the big AGI problem, is this
01:21:18 footsteps along the way to AGI?
01:21:20 So let’s see, it seems that intelligence itself is, there are multiple axis of it.
01:21:25 And I would expect that the systems that we are building, they might end up being
01:21:33 superhuman on some axis and subhuman on some other axis.
01:21:37 It would be surprising to me on all axis simultaneously, they would become superhuman.
01:21:43 Of course, people ask this question, is GPT a spaceship that would take us to
01:21:48 the moon or are we putting a, building a ladder to heaven that we are just
01:21:52 building bigger and bigger ladder.
01:21:54 And we don’t know in some sense, which one of these two.
01:21:59 Which one is better?
01:22:02 I’m trying to, I like stairway to heaven.
01:22:04 It’s a good song.
01:22:04 So I’m not exactly sure which one is better, but you’re saying like the
01:22:08 spaceship to the moon is actually effective.
01:22:10 Correct.
01:22:11 So people who criticize GPT, they say, you guys just building a
01:22:17 taller, a ladder, and it will never reach the moon.
01:22:22 And at the moment, I would say the way I’m thinking is, is like a scientific question.
01:22:28 And I’m also in heart, I’m a builder creator and like, I’m thinking, let’s try out, let’s
01:22:35 see how far it goes.
01:22:36 And so far we see constantly that there is a progress.
01:22:40 Yeah.
01:22:41 So do you think GPT four, GPT five, GPT N plus one will, um, there’ll be a phase
01:22:52 shift, like a transition to a, to a place where we’ll be truly surprised.
01:22:56 Then again, like GPT three is already very like truly surprising.
01:23:00 The people that criticize GPT three as a stair, as a, what is it?
01:23:04 Ladder to heaven.
01:23:06 I think too quickly get accustomed to how impressive it is that they’re
01:23:09 impressive, it is that the prediction of the next word can achieve such depth of
01:23:15 semantics, accuracy of syntax, grammar, and semantics.
01:23:20 Um, do you, do you think GPT four and five and six will continue to surprise us?
01:23:28 I mean, definitely there will be more impressive models that there is a
01:23:31 question of course, if there will be a phase shift and, uh, the, also even the
01:23:38 way I’m thinking about the, about these models is that when we build these
01:23:42 models, you know, we see some level of the capabilities, but we don’t even fully
01:23:47 understand everything that the model can do.
01:23:50 And actually one of the best things to do is to allow other people to probe the
01:23:55 model to even see what is possible.
01:23:58 Hence the, the using GPT as an API and opening it up to the world.
01:24:05 Yeah.
01:24:05 I mean, so when I’m thinking from perspective of like, uh, obviously
01:24:10 various people are, that have concerns about AGI, including myself.
01:24:14 Um, and then when I’m thinking from perspective, what’s the strategy even to
01:24:18 deploy these things to the world, then the one strategy that I have seen many
01:24:23 times working is that iterative deployment that you deploy, um, slightly
01:24:29 better versions and you allow other people to criticize you.
01:24:32 So you actually, or try it out, you see where are their fundamental issues.
01:24:37 And it’s almost, you don’t want to be in that situation that you are holding
01:24:42 into powerful system and there’s like a huge overhang, then you deploy it and it
01:24:48 might have a random chaotic impact on the world.
01:24:50 So you actually want to be in the situation that they are
01:24:53 gradually deploying systems.
01:24:56 I asked this question of Illya, let me ask you, uh, you this question.
01:25:00 I’ve been reading a lot about Stalin and power.
01:25:09 If you’re in possession of a system that’s like AGI, that’s exceptionally
01:25:14 powerful, do you think your character and integrity might become corrupted?
01:25:21 Like famously power corrupts and absolute power corrupts.
01:25:23 Absolutely.
01:25:24 So I believe that the, you want at some point to work toward distributing the power.
01:25:31 I think that the, you want to be in the situation that actually AGI is not
01:25:36 controlled by a small number of people, uh, but, uh, essentially, uh, by a larger
01:25:42 collective.
01:25:43 So the thing is that requires a George Washington style move in the ascent to
01:25:50 power, there’s always a moment when somebody gets a lot of power and they
01:25:55 have to have the integrity and, uh, the moral compass to give away that power.
01:26:01 That humans have been good and bad throughout history at this particular
01:26:06 step.
01:26:07 And I wonder, I wonder we like blind ourselves in a, for example, between
01:26:13 nations, a race, uh, towards, um, they, yeah, AI race between nations, we might
01:26:20 blind ourselves and justify to ourselves the development of AI without distributing
01:26:25 the power because we want to defend ourselves against China, against Russia,
01:26:29 that kind of, that kind of logic.
01:26:32 And, um, I wonder how we, um, how we design governance mechanisms that, um,
01:26:40 prevent us from becoming power hungry and in the process, destroying ourselves.
01:26:46 So let’s see, I have been thinking about this topic quite a bit, but I also want
01:26:50 to admit that, uh, once again, I actually want to rely way more on Sam Altman on it.
01:26:55 He wrote an excellent blog on how even to distribute wealth.
01:27:01 Um, and he’s proper, he proposed in his blog, uh, to tax, uh, equity of the companies
01:27:08 rather than profit and to distribute it.
01:27:11 And this is, this is an example of, uh, Washington move.
01:27:17 I guess I personally have insane trust in some here already spent plenty of money
01:27:24 running, uh, universal basic income, uh, project.
01:27:28 That like, uh, gives me, I guess, maybe some level of trust to him, but I also,
01:27:34 I guess love him as a friend.
01:27:37 Yeah.
01:27:38 I wonder because we’re sort of summoning a new set of technologies.
01:27:44 I wonder if we’ll be, um, cognizant, like you’re describing the process of open AI,
01:27:50 but it could also be at other places like in the U S government, right?
01:27:54 Uh, both China and the U S are now full steam ahead on autonomous
01:28:00 weapons systems development.
01:28:03 And that’s really worrying to me because in the framework of something being a
01:28:09 national security danger or military danger, you can do a lot of pretty dark
01:28:14 things that blind our moral compass.
01:28:18 And I think AI will be one of those things, um, in some sense, the, the mission
01:28:24 and the work you’re doing in open AI is like the counterbalance to that.
01:28:28 So you want to have more open AI and less autonomous weapons systems.
01:28:33 I, I, I, I like these statements, like to be clear, like this interesting and I’m
01:28:37 thinking about it myself, but, uh, this is a place that I, I, I put my trust
01:28:43 actually in Sam’s hands, because it’s extremely hard for me to reason about it.
01:28:48 Yeah.
01:28:49 I mean, one important statement to make is, um, it’s good to think about this.
01:28:54 Yeah.
01:28:54 No question about it.
01:28:55 No question, even like low level quote unquote engineer, like there’s such a,
01:29:02 um, I remember I, I programmed a car, uh, our RC car, um, and it was, it was
01:29:10 programmed a car, uh, our RC car, they went really fast, like 30, 40 miles an hour.
01:29:18 And I remember I was like sleep deprived.
01:29:21 So I programmed it pretty crappily and it like, uh, the, the, the code froze.
01:29:26 So it’s doing some basic computer vision and it’s going around on track,
01:29:30 but it’s going full speed.
01:29:32 And, uh, there was a bug in the code that, uh, the car just went, it didn’t turn.
01:29:39 Went straight full speed and smash into the wall.
01:29:42 And I remember thinking the seriousness with which you need to approach the
01:29:49 design of artificial intelligence systems and the programming of artificial
01:29:53 intelligence systems is high because the consequences are high, like that
01:29:58 little car smashing into the wall.
01:30:00 For some reason, I immediately thought of like an algorithm that controls
01:30:04 nuclear weapons, having the same kind of bug.
01:30:07 And so like the lowest level engineer and the CEO of a company all need to
01:30:11 have the seriousness, uh, in approaching this problem and thinking
01:30:15 about the worst case consequences.
01:30:17 So I think that is true.
01:30:18 I mean, the, what I also recognize in myself and others even asking this
01:30:24 question is that it evokes a lot of fear and fear itself ends up being
01:30:29 actually quite debilitating.
01:30:31 The place where I arrived at the moment might sound cheesy or so, but it’s
01:30:38 almost to build things out of love rather than fear, like a focus on how, uh, I can,
01:30:48 you know, maximize the value, how the systems that I’m building might be, uh,
01:30:54 useful.
01:30:55 I’m not saying that the fear doesn’t exist out there and like it totally
01:31:00 makes sense to minimize it, but I don’t want to be working because, uh, I’m
01:31:04 scared, I want to be working out of passion, out of curiosity, out of the,
01:31:10 you know, uh, looking forward for the positive future.
01:31:13 With, uh, the definition of love arising from a rigorous practice of empathy.
01:31:19 So not just like your own conception of what is good for the world, but
01:31:23 always listening to others.
01:31:25 Correct.
01:31:25 Like the love where I’m considering reward functions of others.
01:31:29 Others to limit to infinity is like a sum of like one to N where N is, uh,
01:31:35 7 billion or whatever it is.
01:31:36 Not, not projecting my reward functions on others.
01:31:38 Yeah, exactly.
01:31:40 Okay.
01:31:41 Can we just take a step back to something else?
01:31:43 Super cool, which is, uh, OpenAI Codex.
01:31:47 Can you give an overview of what OpenAI Codex and GitHub Copilot is, how it works
01:31:53 and why the hell it works so well?
01:31:55 So with GPT tree, we noticed that the system, uh, you know, that system train
01:32:00 on all the language out there started having some rudimentary coding capabilities.
01:32:05 So we’re able to ask it, you know, to implement addition function between
01:32:10 two numbers and indeed it can write item or JavaScript code for that.
01:32:15 And then we thought, uh, we might as well just go full steam ahead and try to
01:32:20 create a system that is actually good at what we are doing every day ourselves,
01:32:25 which is programming.
01:32:27 We optimize models for proficiency in coding.
01:32:31 We actually even created models that both have a comprehension of language and code.
01:32:38 And Codex is API for these models.
01:32:42 So it’s first pre trained on language and then codex.
01:32:48 Then I don’t know if you can say fine tuned because there’s a lot of code,
01:32:54 but it’s language and code.
01:32:56 It’s language and code.
01:32:58 It’s also optimized for various things.
01:33:00 I can, let’s say low latency and so on.
01:33:02 Codex is the API, the similar to GPT tree.
01:33:06 We expect that there will be proliferation of the potential products that can use
01:33:10 coding capabilities and I can, I can speak about it in a second.
01:33:14 Copilot is a first product and developed by GitHub.
01:33:18 So as we’re building, uh, models, we wanted to make sure that these
01:33:22 models are useful and we work together with GitHub on building the first product.
01:33:27 Copilot is actually, as you code, it suggests you code completions.
01:33:32 And we have seen in the past, there are like a various tools that can suggest
01:33:36 how to like a few characters of the code or a line of code.
01:33:41 Then the thing about Copilot is it can generate 10 lines of code.
01:33:44 You, it’s often the way how it works is you often write in the comment
01:33:49 what you want to happen because people in comments, they describe what happens next.
01:33:53 So, um, these days when I code, instead of going to Google to search, uh, for
01:34:00 the appropriate code to solve my problem, I say, Oh, for this area, could you
01:34:06 smooth it and then, you know, it imports some appropriate libraries and say it
01:34:10 uses NumPy convolution or so I, that I was not even aware that exists and
01:34:15 it does the appropriate thing.
01:34:16 Um, so you, uh, you write a comment, maybe the header of a function
01:34:21 and it completes the function.
01:34:23 Of course, you don’t know what is the space of all the possible small
01:34:27 programs that can generate.
01:34:28 What are the failure cases?
01:34:30 How many edge cases, how many subtle errors there are, how many big errors
01:34:34 there are, it’s hard to know, but the fact that it works at all in a large
01:34:38 number of cases is incredible.
01:34:41 It’s like, uh, it’s a kind of search engine into code that’s
01:34:45 been written on the internet.
01:34:47 Correct.
01:34:48 So for instance, when you search things online, then usually you get to the,
01:34:53 some particular case, like if you go to stack overflow and people describe
01:34:58 that one particular situation, uh, and then they seek for a solution.
01:35:03 But in case of a copilot, it’s aware of your entire context and in
01:35:08 context is, Oh, these are the libraries that they are using.
01:35:10 That’s the set of the variables that is initialized.
01:35:14 And on the spot, it can actually tell you what to do.
01:35:17 So the interesting thing is, and we think that the copilot is one
01:35:21 possible product using codecs, but there is a place for many more.
01:35:25 So internally we tried out, you know, to create other fun products.
01:35:29 So it turns out that a lot of tools out there, let’s say Google
01:35:33 calendar or Microsoft word or so, they all have a internal API
01:35:38 to build plugins around them.
01:35:41 So there is a way in the sophisticated way to control calendar or Microsoft word.
01:35:47 Today, if you want, if you want more complicated behaviors from these
01:35:51 programs, you have to add the new button for every behavior.
01:35:55 But it is possible to use codecs and tell for instance, to calendar, uh,
01:36:00 could you schedule an appointment with Lex next week after 2 PM and it
01:36:06 writes corresponding piece of code.
01:36:08 And that’s the thing that actually you want.
01:36:10 So interesting.
01:36:11 So you figure out is there’s a lot of programs with which
01:36:15 you can interact through code.
01:36:17 And so there you can generate that code from natural language.
01:36:22 That’s fascinating.
01:36:23 And that’s somewhat like also closest to what was the promise of Siri or Alexa.
01:36:28 So previously all these behaviors, they were hard coded and it seems
01:36:33 that codecs on the fly can pick up the API of let’s say, given software.
01:36:39 And then it can turn language into use of this API.
01:36:42 So without hard coding, you can find, it can translate to machine language.
01:36:46 Correct.
01:36:47 To, uh, so for example, this would be really exciting for me, like for, um,
01:36:51 Adobe products, like Photoshop, uh, which I think action scripted, I think
01:36:57 there’s a scripting language that communicates with them, same with Premier.
01:37:00 And do you could imagine that that allows even to do coding by voice on your phone?
01:37:06 So for instance, in the past, okay.
01:37:09 As of today, I’m not editing Word documents on my phone because it’s
01:37:13 just the keyboard is too small.
01:37:15 But if I would be able to tell, uh, to my phone, you know, uh, make the
01:37:20 header large, then move the paragraphs around and that’s actually what I want.
01:37:25 So I can tell you one more cool thing, or even how I’m thinking about codecs.
01:37:29 So if you look actually at the evolution of, uh, of computers, we started with
01:37:36 a very primitive interfaces, which is a punch card and punch card.
01:37:40 So Charlie, you make a holes in the, in the plastic card to indicate zeros and ones.
01:37:47 And, uh, during that time, there was a small number of specialists
01:37:50 who were able to use computers.
01:37:52 And by the way, people even suspected that there is no need for many
01:37:55 more people to use computers.
01:37:56 Um, but then we moved from punch cards to at first assembly and see, and
01:38:03 at these programming languages, they were slightly higher level.
01:38:07 They allowed many more people to code and they also, uh, led to more
01:38:11 of a proliferation of technology.
01:38:14 And, uh, you know, further on, there was a jump to say from C++ to Java and Python.
01:38:19 And every time it has happened, more people are able to code
01:38:23 and we build more technology.
01:38:26 And it’s even, you know, hard to imagine now, if someone will tell you that you
01:38:31 should write code in assembly instead of let’s say, Python or Java or JavaScript.
01:38:37 And codecs is yet another step toward kind of bringing computers closer to
01:38:41 humans such that you communicate with a computer with your own language rather
01:38:47 than with a specialized language, and, uh, I think that it will lead to an
01:38:52 increase of number of people who can code.
01:38:55 Yeah.
01:38:55 And then, and the kind of technologies that those people will create is it’s
01:39:00 innumerable, it could, you know, it could be a huge number of technologies.
01:39:03 We’re not predicting at all because that’s less and less requirement
01:39:07 of having a technical mind, a programming mind, you’re not opening it to the world
01:39:13 of, um, other kinds of minds, creative minds, artistic minds, all that kind of stuff.
01:39:19 I would like, for instance, biologists who work on DNA to be able to program
01:39:23 and not to need to spend a lot of time learning it.
01:39:26 And I, I believe that’s a good thing to the world.
01:39:29 And I would actually add, I would add, so at the moment I’m a managing codecs
01:39:33 team and also language team, and I believe that there is like a plenty
01:39:37 of brilliant people out there and they should have a lot of experience.
01:39:41 There and they should apply.
01:39:44 Oh, okay.
01:39:45 Yeah.
01:39:45 Awesome.
01:39:45 So what’s the language and the codecs is, so those are kind of,
01:39:48 they’re overlapping teams.
01:39:50 It’s like GPT, the raw language, and then the codecs is like applied to programming.
01:39:57 Correct.
01:39:57 And they are quite intertwined.
01:40:00 There are many more things involved making this, uh, models,
01:40:03 uh, extremely efficient and deployable.
01:40:06 Okay.
01:40:06 For instance, there are people who are working to, you know, make our data
01:40:10 centers, uh, amazing, or there are people who work on putting these
01:40:14 models into production or, uh, or even pushing it at the very limit of the scale.
01:40:21 So all aspects from, from the infrastructure to the actual machine.
01:40:25 So I’m just saying there are multiple teams while the, and the team working
01:40:29 on codecs and language, uh, I guess I’m, I’m directly managing them.
01:40:33 I would like, I would love to hire more interested in machine learning.
01:40:37 This is probably one of the most exciting problems and like systems
01:40:41 to be working on is it’s actually, it’s, it’s, it’s pretty cool.
01:40:45 Like what, what, uh, the program synthesis, like generating a
01:40:48 programs is very interesting, very interesting problem that has echoes
01:40:53 of reasoning and intelligence in it.
01:40:57 It’s and I think there’s a lot of fundamental questions that you might
01:41:00 be able to sneak, uh, sneak up to by generating programs.
01:41:05 Yeah, that one more exciting thing about the programs is that, so I said
01:41:09 that the, um, you know, the, in case of language, that one of the travels
01:41:13 is even evaluating language.
01:41:15 So when the things are made up, you, you need somehow either a human to,
01:41:20 to say that this doesn’t make sense or so in case of program, there is one extra
01:41:25 lever that we can actually execute programs and see what they evaluate to.
01:41:29 So that process might be somewhat, uh, more automated in, in order to improve
01:41:35 the, uh, qualities of generations.
01:41:38 Oh, that’s fascinating.
01:41:39 So like the, wow, that’s really interesting.
01:41:42 So, so for the language, the, you know, the simulation to actually
01:41:45 execute it as a human mind.
01:41:47 Yeah.
01:41:48 For programs, there is a, there is a computer on which you can evaluate it.
01:41:53 Wow.
01:41:54 That’s a brilliant little insight.
01:41:58 Insight that the thing compiles and runs that’s first and second, you can evaluate
01:42:04 on a, like do automated unit testing and in some sense, it seems to me that we’ll
01:42:11 be able to make a tremendous progress.
01:42:12 You know, we are in the paradigm that there is way more data.
01:42:17 There is like a transcription of millions of, uh, of, uh, software engineers.
01:42:23 Yeah.
01:42:24 Yeah.
01:42:24 So, uh, I mean, you just mean, cause I was going to ask you about reliability.
01:42:29 The thing about programs is you don’t know if they’re going to, like a program
01:42:35 that’s controlling a nuclear power plant has to be very reliable.
01:42:39 So I wouldn’t start with controlling nuclear power plant maybe one day,
01:42:43 but that’s not actually, that’s not on the current roadmap.
01:42:46 That’s not the step one.
01:42:48 And you know, it’s the Russian thing.
01:42:50 You just want to go to the most powerful, destructive, most powerful
01:42:53 the most powerful, destructive thing right away run by JavaScript.
01:42:57 But I got you.
01:42:58 So this is a lower impact, but nevertheless, when you make me
01:43:01 realize it is possible to achieve some levels of reliability by doing testing.
01:43:06 And you could, you could imagine that, you know, maybe there are ways for
01:43:09 model to write event code for testing itself and so on, and there exists
01:43:15 a ways to create the feedback loops that the model could keep on improving.
01:43:19 Yeah. By writing programs that generate tests for the instance, for instance.
01:43:26 And that’s how we get consciousness, because it’s metacompression.
01:43:30 That’s what you’re going to write.
01:43:31 That’s the comment.
01:43:32 That’s the prompt that generates consciousness.
01:43:34 Compressor of compressors.
01:43:36 You just write that.
01:43:38 Do you think the code that generates consciousness will be simple?
01:43:42 So let’s see.
01:43:44 I mean, ultimately, the core idea behind will be simple,
01:43:48 but there will be also decent amount of engineering involved.
01:43:53 Like in some sense, it seems that, you know, spreading these models
01:43:58 on many machines, it’s not that trivial.
01:44:01 Yeah.
01:44:02 And we find all sorts of innovations that make our models more efficient.
01:44:08 I believe that first models that I guess are conscious or like a truly intelligent,
01:44:14 they will have all sorts of tricks, but then again, there’s a Richard Sutton
01:44:21 argument that maybe the tricks are temporary things that they might be
01:44:25 temporary things and in some sense, it’s also even important to, to know
01:44:32 that even the cost of a trick.
01:44:33 So sometimes people are eager to put the trick while forgetting that
01:44:38 there is a cost of maintenance or like a long term cost, long term cost
01:44:43 or maintenance, or maybe even flexibility of code to actually implement new ideas.
01:44:48 So even if you have something that gives you 2x, but it requires, you know,
01:44:53 1000 lines of code, I’m not sure if it’s actually worth it.
01:44:56 So in some sense, you know, if it’s five lines of code and 2x, I would take it.
01:45:02 And we see many of this, but also, you know, that requires some level of,
01:45:07 I guess, lack of attachment to code that we are willing to remove it.
01:45:12 Yeah.
01:45:14 So you led the OpenAI robotics team.
01:45:17 Can you give an overview of the cool things you were able to
01:45:20 accomplish, what are you most proud of?
01:45:22 So when we started robotics, we knew that actually reinforcement learning works
01:45:26 and it is possible to solve fairly complicated problems.
01:45:29 Like for instance, AlphaGo is an evidence that it is possible to build superhuman
01:45:36 Go players, DOTA2 is an evidence that it’s possible to build superhuman agents
01:45:44 playing DOTA, so I asked myself a question, you know, what about robots out there?
01:45:48 Could we train machines to solve arbitrary tasks in the physical world?
01:45:53 Our approach was, I guess, let’s pick a complicated problem that if we would
01:45:59 solve it, that means that we made some significant progress in the domain.
01:46:04 And if can progress the domain, and then we went after the problem.
01:46:08 So we noticed that actually the robots out there, they are kind of at the moment
01:46:13 optimized per task, so you can have a robot that it’s like, if you have a robot
01:46:18 opening a bottle, it’s very likely that the end factor is that bottle opener.
01:46:24 And the, and in some sense, that’s a hack to be able to solve a task,
01:46:27 which makes any task easier and ask myself, so what would be a robot that
01:46:33 can actually solve many tasks?
01:46:35 And we conclude that human hands have such a quality that indeed they are, you
01:46:42 know, you have five kind of tiny arms attached individually.
01:46:48 They can manipulate pretty broad spectrum of objects.
01:46:51 So we went after a single hand, like trying to solve Rubik’s cube single handed.
01:46:57 We picked this task because we thought that there is no way to hard code it.
01:47:01 And it’s also, we picked the robot on which it would be hard to hard code it.
01:47:05 And we went after the solution such that it could generalize to other problems.
01:47:11 And just to clarify, it’s one robotic hand solving the Rubik’s cube.
01:47:16 The hard part isn’t the solution to the Rubik’s cube is the manipulation of the,
01:47:21 of like having it not fall out of the hand, having it use the, uh, five baby
01:47:27 arms to, uh, what is it like rotate different parts of the Rubik’s cube to
01:47:32 achieve the solution.
01:47:33 Correct.
01:47:33 Yeah.
01:47:34 So what, uh, what was the hardest part about that?
01:47:38 What was the approach taken there?
01:47:40 What are you most proud of?
01:47:41 Obviously we have like a strong belief in reinforcement learning.
01:47:44 And, uh, you know, one path it is to do reinforcement learning, the real
01:47:49 world other path is to, uh, uh, that simulation in some sense, the tricky
01:47:55 part about the real world is at the moment, our models, they require a lot
01:47:59 of data and there is essentially no data.
01:48:02 And, uh, I did, we decided to go through the path of the simulation.
01:48:07 And in simulation, you can have infinite amount of data.
01:48:09 The tricky part is the fidelity of the simulation.
01:48:12 And also can you in simulation represent everything that you represent
01:48:16 otherwise in the real world.
01:48:18 And, you know, it turned out that, uh, that, you know, because there is
01:48:22 lack of fidelity, it is possible to what we, what we arrived at is training
01:48:29 a model that doesn’t solve one simulation, but it actually solves the
01:48:34 entire range of simulations, which, uh, uh, in terms of like, uh, what’s
01:48:39 the, exactly the friction of the cube or the weight or so, and the single AI
01:48:45 that can solve all of them ends up working well with the reality.
01:48:49 How do you generate the different simulations?
01:48:51 So, uh, you know, there’s plenty of parameters out there.
01:48:54 We just pick them randomly.
01:48:55 And, uh, and in simulation model just goes for thousands of years and keeps
01:49:01 on solving Rubik’s cube in each of them.
01:49:03 And the thing is that neural network that we used, it has a memory.
01:49:09 And as it presses, for instance, the side of the, of the cube, it can sense,
01:49:15 oh, that’s actually, this side was, uh, difficult to press.
01:49:19 I should press it stronger and throughout this process kind of, uh, learn it’s even
01:49:24 how to, uh, how to solve this particular instance of the Rubik’s cube, like even
01:49:29 mass, it’s kind of like, uh, you know, sometimes when you go to a gym and after,
01:49:34 um, after bench press, you try to leave the class and you kind of forgot, uh, and,
01:49:44 and your head goes like up right away because kind of you got used to maybe
01:49:48 different weight and it takes a second to adjust and this kind of, of a memory,
01:49:54 the model gained through the process of interacting with the cube in the
01:49:58 simulation, I appreciate you speaking to the audience with the bench press,
01:50:02 all the bros in the audience, probably working out right now.
01:50:05 There’s probably somebody listening to this actually doing bench press.
01:50:09 Um, so maybe, uh, put the bar down and pick up the water bottle and you’ll
01:50:13 know exactly what, uh, what Jack is talking about.
01:50:17 Okay.
01:50:17 Okay.
01:50:18 So what, uh, what was the hardest part of getting the whole thing to work?
01:50:24 So the hardest part is at the moment when it comes to, uh, physical work, when it
01:50:31 comes to robots, uh, they require maintenance, it’s hard to replicate a
01:50:36 million times it’s, uh, it’s also, it’s hard to replay things exactly.
01:50:41 I remember this situation that one guy at our company, he had like a model that
01:50:48 performs way better than other models in solving Rubik’s cube.
01:50:52 And, uh, you know, we kind of didn’t know what’s going on, why it’s that.
01:50:58 And, uh, it turned out, uh, that, you know, he was running it from his laptop
01:51:04 that had better CPU or better, maybe local GPU as well.
01:51:09 And, uh, because of that, there was less of a latency and the model was the same.
01:51:14 And that actually made solving Rubik’s cube more reliable.
01:51:18 So in some sense, there might be some subtle bugs like that when it comes
01:51:22 to running things in the real world.
01:51:24 Even hinting on that, you could imagine that the initial models you would like
01:51:29 to have models, which are insanely huge neural networks, and you would like to
01:51:34 give them even more time for thinking.
01:51:36 And when you have these real time systems, uh, then you might be constrained
01:51:41 actually by the amount of latency.
01:51:44 And, uh, ultimately I would like to build a system that it is worth for you to wait
01:51:50 five minutes because it gives you the answer that you’re willing to wait for
01:51:55 five minutes.
01:51:56 So latency is a very unpleasant constraint under which to operate.
01:51:59 Correct.
01:52:00 And also there is actually one more thing, which is tricky about robots.
01:52:04 Uh, there is actually, uh, no, uh, not much data.
01:52:08 So the data that I’m speaking about would be a data of, uh, first person
01:52:13 experience from the robot and like a gigabytes of data like that, if we would
01:52:17 have gigabytes of data like that, of robots solving various problems, it would
01:52:21 be very easy to make a progress on robotics.
01:52:24 And you can see that in case of text or code, there is a lot of data, like a
01:52:28 first person perspective, they don’t writing code.
01:52:31 Yeah. So you had this, you mentioned this really interesting idea that if you were
01:52:37 to build like a successful robotics company, so open as mission is much
01:52:42 bigger than robotics, this is one of the, one of the things you’ve worked on, but
01:52:46 if it was a robotics company, they, you wouldn’t so quickly dismiss supervised
01:52:51 learning, uh, correct that you would build a robot that, uh, was perhaps what
01:52:58 like, um, an empty shell, like dumb, and they would operate under teleoperation.
01:53:04 So you would invest, that’s just one way to do it, invest in human supervision,
01:53:09 like direct human control of the robots as it’s learning and over time, add
01:53:14 more and more automation.
01:53:16 That’s correct.
01:53:16 So let’s say that’s how I would build a robotics company today.
01:53:20 If I would be building a robotics company, which is, you know, spend 10
01:53:23 million dollars or so recording human trajectories, controlling a robot.
01:53:29 After you find a thing that the robot should be doing, that there’s a market
01:53:34 fit for, like you can make a lot of money with that product.
01:53:37 Correct.
01:53:37 Correct.
01:53:38 Yeah.
01:53:38 Uh, so I would record data and then I would essentially train supervised
01:53:43 learning model on it.
01:53:45 That might be the path today.
01:53:47 Long term.
01:53:47 I think that actually what is needed is to have a robot that can
01:53:52 train powerful models over video.
01:53:55 So, um, you have seen maybe a models that can generate images like Dali and people
01:54:02 are looking into models, generating videos, they’re like, uh, bodies,
01:54:06 algorithmic questions, even how to do it.
01:54:08 And it’s unclear if there is enough compute for this purpose, but, uh, I, I
01:54:13 suspect that the models that which would have a level of understanding of video,
01:54:19 same as GPT has a level of understanding of text, could be used, uh, to train
01:54:25 robots to solve tasks.
01:54:26 They would have a lot of common sense.
01:54:29 If one day, I’m pretty sure one day there will be a robotics company by robotics
01:54:36 company, I mean, the primary source of income is, is from robots that is worth
01:54:42 over $1 trillion.
01:54:44 What do you think that company will do?
01:54:49 I think self driving cars.
01:54:51 No, it’s interesting.
01:54:53 Cause my mind went to personal robotics, robots in the home.
01:54:57 It seems like there’s much more market opportunity there.
01:55:00 I think it’s very difficult to achieve.
01:55:04 I mean, this, this, this might speak to something important, which is I understand
01:55:09 self driving much better than understand robotics in the home.
01:55:12 So I understand how difficult it is to actually solve self driving to a, to a
01:55:17 level, not just the actual computer vision and the control problem and just the
01:55:22 basic problem of self driving, but creating a product that would undeniably
01:55:28 be, um, that will cost less money.
01:55:31 Like it will save you a lot of money, like orders of magnitude, less money
01:55:34 that could replace Uber drivers, for example.
01:55:36 So car sharing that’s autonomous, that creates a similar or better
01:55:41 experience in terms of how quickly you get from A to B or just whatever, the
01:55:46 pleasantness of the experience, the efficiency of the experience, the value
01:55:50 of the experience, and at the same time, the car itself costs cheaper.
01:55:55 I think that’s very difficult to achieve.
01:55:57 I think there’s a lot more, um, low hanging fruit in the home.
01:56:03 That, that, that could be, I also want to give you a perspective on like how
01:56:08 challenging it would be at home or like it maybe kind of depends on that exact
01:56:12 problem that you’d be solving.
01:56:14 Like if we’re speaking about these robotic arms and hands, these things,
01:56:20 they cost tens of thousands of dollars or maybe a hundred K and, um, you know,
01:56:27 maybe, obviously, maybe there would be economy of scale.
01:56:30 These things would be cheaper, but actually for any household to buy it,
01:56:34 the price would have to go down to maybe a thousand bucks.
01:56:37 Yeah.
01:56:38 I personally think that, uh, so self driving car, it provides a clear service.
01:56:44 I don’t think robots in the home, there’ll be a trillion dollar company
01:56:48 will just be all about service, meaning it will not necessarily be about like
01:56:53 a robotic arm that’s helps you.
01:56:56 I don’t know, open a bottle or wash the dishes or, uh, any of that kind of stuff.
01:57:02 It has to be able to take care of that whole, the therapist thing.
01:57:05 You mentioned, I think that’s, um, of course there’s a line between what
01:57:10 is a robot and what is not like, does it really need a body?
01:57:14 But you know, some, um, uh, AI system with some embodiment, I think.
01:57:20 So the tricky part when you think actually what’s the difficult part is,
01:57:24 um, when the robot has like, when there is a diversity of the environment
01:57:29 with which the robot has to interact, that becomes hard.
01:57:31 So, you know, on the one spectrum, you have, uh, industrial robots as they
01:57:36 are doing over and over the same thing, it is possible to some extent to
01:57:40 prescribe the movements and we’ve very small amount of intelligence, the, the
01:57:46 movement can be repeated millions of times.
01:57:48 Um, the, it, there are also, you know, various pieces of industrial robots
01:57:52 where it becomes harder and harder.
01:57:54 I can, for instance, in case of Tesla, it might be a matter of putting a, a
01:57:59 rack inside of a car and, you know, because the rack kind of moves around,
01:58:03 it’s, uh, it’s not that easy.
01:58:05 It’s not exactly the same every time.
01:58:08 That’s not being the case that you need actually humans to do it.
01:58:11 Uh, while, you know, welding cars together, it’s a very repetitive process.
01:58:16 Um, then in case of self driving itself, uh, that difficulty has to do with the
01:58:23 diversity of the environment, but still the car itself, um, the problem
01:58:27 that they are solving is you try to avoid even interacting with things.
01:58:32 You are not touching anything around because touching itself is hard.
01:58:36 And then if you would have in the home, uh, robot that, you know, has to
01:58:40 touch things and like if these things, they change the shape, if there is a huge
01:58:44 variety of things to be touched, then that’s difficult.
01:58:46 If you are speaking about the robot, which there is, you know, head that
01:58:50 is smiling in some way with cameras that either doesn’t, you know, touch things.
01:58:54 That’s relatively simple.
01:58:55 Okay. So to both agree and to push back.
01:59:00 So you’re referring to touch, like soft robotics, like the actual touch, but.
01:59:08 I would argue that you could formulate just basic interaction between, um, like
01:59:13 non contact interaction is also a kind of touch and that might be very difficult
01:59:18 to solve that’s the basic, this not disagreement, but that’s the basic open
01:59:22 question to me with self driving cars and this agreement with Elon, which
01:59:27 is how much interaction is required to solve self driving cars.
01:59:31 How much touch is required?
01:59:33 You said that in your intuition, touch is not required.
01:59:37 And my intuition to create a product that’s compelling to use, you’re going
01:59:41 to have to, uh, interact with pedestrians, not just avoid pedestrians,
01:59:46 but interact with them when we drive around.
01:59:49 In major cities, we’re constantly threatening everybody’s life with
01:59:54 our movements, um, and that’s how they respect us.
01:59:57 There’s a game to ready going out with pedestrians and I’m afraid you can’t
02:00:02 just formulate autonomous driving as a collision avoidance problem.
02:00:08 So I think it goes beyond like a collision avoidance is the
02:00:12 first order approximation.
02:00:14 Uh, but then at least in case of Tesla, you can’t just
02:00:18 at least in case of Tesla, they are gathering data from people driving their
02:00:22 cars and I believe that’s an example of supervised data that they can train
02:00:27 their models, uh, on, and they are doing it, uh, which, you know, can give
02:00:32 a model dislike, uh, another level of, uh, of, uh, behavior that is needed
02:00:38 to actually interact with the real world.
02:00:41 Yeah.
02:00:41 It’s interesting how much data is required to achieve that.
02:00:45 Um, w what do you think of the whole Tesla autopilot approach, the computer
02:00:49 vision based approach with multiple cameras and there’s a data engine.
02:00:53 It’s a multitask, multiheaded neural network, and it’s this fascinating
02:00:57 process of, uh, similar to what you’re talking about with the robotics
02:01:02 approach, uh, which is, you know, you deploy in your own network and
02:01:06 then there’s humans that use it and then it runs into trouble in a bunch
02:01:10 of places and that stuff is sent back.
02:01:12 So like the deployment discovers a bunch of edge cases and those edge
02:01:17 cases are sent back for supervised annotation, thereby improving the
02:01:22 neural network and that’s deployed again.
02:01:24 It goes over and over until the network becomes really good at the task of
02:01:29 driving becomes safer and safer.
02:01:31 What do you think of that kind of approach to robotics?
02:01:34 I believe that’s the way to go.
02:01:36 So in some sense, even when I was speaking about, you know, collecting
02:01:39 trajectories from humans, that’s like a first step and then you deploy
02:01:43 the system and then you have humans revising the, all the issues.
02:01:46 And in some sense, like at this approach converges to system that doesn’t make
02:01:51 mistakes because for the cases where there are mistakes, you got their
02:01:54 data, how to fix them and the system will keep on improving.
02:01:58 So there’s a very, to me, difficult question of how hard that, you know,
02:02:02 how long that converging takes, how hard it is.
02:02:04 The other aspect of autonomous vehicles, this probably applies to certain
02:02:09 robotics applications is society, right?
02:02:12 They put as, as the quality of the system converges.
02:02:18 So one, there’s a human factors perspective of psychology of humans being
02:02:21 able to supervise those even with teleoperation, those robots.
02:02:25 And the other is society willing to accept robots.
02:02:29 Currently society is much harsher on self driving cars than it is on human
02:02:32 driven cars in terms of the expectation of safety.
02:02:35 So the bar is set much higher than for humans.
02:02:39 And so if there’s a death in an autonomous vehicle, that’s seen as a much more,
02:02:47 much more dramatic than a death in the human driven vehicle.
02:02:50 Part of the success of deployment of robots is figuring out how to make robots
02:02:55 part of society, both on the, just the human side, on the media side, on the
02:03:01 media journalist side, and also on the policy government side.
02:03:04 And that seems to be, maybe you can put that into the objective function to
02:03:08 optimize, but that is, that is definitely a tricky one.
02:03:14 And I wonder if that is actually the trickiest part for self driving cars or
02:03:18 any system that’s safety critical.
02:03:21 It’s not the algorithm, it’s the society accepting it.
02:03:24 Yeah, I would say, I believe that the part of the process of deployment is actually
02:03:31 showing people that the given things can be trusted and, you know, trust is also
02:03:36 like a glass that is actually really easy to crack it and damage it.
02:03:43 And I think that’s actually very common with, with innovation, that there’s
02:03:52 some resistance toward it and it’s just the natural progression.
02:03:56 So in some sense, people will have to keep on proving that indeed these
02:04:00 systems are worth being used.
02:04:02 And I would say, I also found out that often the best way to convince people
02:04:09 is by letting them experience it.
02:04:11 Yeah, absolutely.
02:04:12 That’s the case with Tesla autopilot, for example, that’s the case with, yeah,
02:04:17 with basically robots in general.
02:04:18 It’s kind of funny to hear people talk about robots.
02:04:22 Like there’s a lot of fear, even with like legged robots, but when they
02:04:27 actually interact with them, there’s joy.
02:04:31 I love interacting with them.
02:04:32 And the same with the car, with a robot, if it starts being useful, I think
02:04:38 people immediately understand.
02:04:40 And if the product is designed well, they fall in love.
02:04:43 You’re right.
02:04:44 It’s actually even similar when I’m thinking about the car.
02:04:46 It’s actually even similar when I’m thinking about Copilot, the GitHub Copilot.
02:04:51 There was a spectrum of responses that people had.
02:04:54 And ultimately the important piece was to let people try it out.
02:05:00 And then many people just loved it.
02:05:02 Especially like programmers.
02:05:05 Yeah, programmers, but like some of them, you know, they came with a fear.
02:05:08 Yeah.
02:05:08 But then you try it out and you think, actually, that’s cool.
02:05:11 And, you know, you can try to resist the same way as, you know, you could
02:05:15 resist moving from punch cards to, let’s say, C++ or so.
02:05:20 And it’s a little bit futile.
02:05:23 So we talked about generation of program, generation of language, even
02:05:30 self supervised learning in the visual space for robotics and then
02:05:33 reinforcement learning.
02:05:35 What do you, in like this whole beautiful spectrum of AI, do you think is a
02:05:40 good benchmark, a good test to strive for to achieve intelligence?
02:05:47 That’s a strong test of intelligence.
02:05:49 You know, it started with Alan Turing and the Turing test.
02:05:53 Maybe you think natural language conversation is a good test.
02:05:57 So, you know, it would be nice if, for instance, machine would be able to
02:06:01 solve Riemann hypothesis in math.
02:06:04 That would be, I think that would be very impressive.
02:06:07 So theorem proving, is that to you, proving theorems is a good, oh, oh,
02:06:12 like one thing that the machine did, you would say, damn.
02:06:16 Exactly.
02:06:18 Okay.
02:06:19 That would be quite, quite impressive.
02:06:22 I mean, the tricky part about the benchmarks is, you know, as we are
02:06:26 getting closer with them, we have to invent new benchmarks.
02:06:29 There is actually no ultimate benchmark out there.
02:06:31 Yeah.
02:06:31 See, my thought with the Riemann hypothesis would be the moment the
02:06:36 machine proves it, we would say, okay, well then the problem was easy.
02:06:40 That’s what happens.
02:06:42 And I mean, in some sense, that’s actually what happens over the years
02:06:46 in AI that like, we get used to things very quickly.
02:06:50 You know something, I talked to Rodney Brooks.
02:06:52 I don’t know if you know who that is.
02:06:54 He called AlphaZero homework problem.
02:06:57 Cause he was saying like, there’s nothing special about it.
02:06:59 It’s not a big leap.
02:07:00 And I didn’t, well, he’s coming from one of the aspects that we referred
02:07:05 to is he was part of the founding of iRobot, which deployed now tens
02:07:10 of millions of robot in the home.
02:07:11 So if you see robots that are actually in the homes of people as the
02:07:18 legitimate instantiation of artificial intelligence, then yes, maybe an AI
02:07:23 that plays a silly game like go and chess is not a real accomplishment,
02:07:26 but to me it’s a fundamental leap.
02:07:29 But I think we as humans then say, okay, well then that that game of
02:07:33 chess or go wasn’t that difficult compared to the thing that’s currently
02:07:37 unsolved.
02:07:38 So my intuition is that from perspective of the evolution of these AI
02:07:44 systems will at first seen the tremendous progress in digital space.
02:07:49 And the, you know, the main thing about digital space is also that you
02:07:52 can, everything is that there is a lot of recorded data.
02:07:56 Plus you can very rapidly deploy things to billions of people.
02:07:59 While in case of a physical space, the deployment part takes multiple
02:08:05 years.
02:08:05 You have to manufacture things and, you know, delivering it to actual
02:08:10 people, it’s very hard.
02:08:13 So I’m expecting that the first and that prices in digital space of
02:08:19 goods, they would go, you know, down to the, let’s say marginal costs
02:08:24 are two zero.
02:08:25 And also the question is how much of our life will be in digital because
02:08:28 it seems like we’re heading towards more and more of our lives being in
02:08:31 the digital space.
02:08:33 So like innovation in the physical space might become less and less
02:08:37 significant.
02:08:38 Like why do you need to drive anywhere if most of your life is spent in
02:08:42 virtual reality?
02:08:44 I still would like, you know, to at least at the moment, my impression
02:08:47 is that I would like to have a physical contact with other people.
02:08:51 And that’s very important to me.
02:08:52 We don’t have a way to replicate it in the computer.
02:08:55 It might be the case that over the time it will change.
02:08:57 Like in 10 years from now, why not have like an arbitrary infinite number
02:09:02 of people you can interact with?
02:09:04 Some of them are real, some are not with arbitrary characteristics that
02:09:09 you can define based on your own preferences.
02:09:12 I think that’s maybe where we are heading and maybe I’m resisting the
02:09:15 future.
02:09:16 Yeah, I’m telling you, if I got to choose, if I could live in Elder
02:09:25 Scrolls Skyrim versus the real world, I’m not so sure I would stay with
02:09:29 the real world.
02:09:31 Yeah, I mean, the question is, so will VR be sufficient to get us there
02:09:35 or do you need to, you know, plug electrodes in the brain?
02:09:40 And it would be nice if these electrodes wouldn’t be invasive.
02:09:45 Or at least like provably non destructive.
02:09:49 But in the digital space, do you think we’ll be able to solve the
02:09:53 Turing test, the spirit of the Turing test, which is, do you think we’ll
02:09:57 be able to achieve compelling natural language conversation between
02:10:02 people, like have friends that are AI systems on the internet?
02:10:07 I totally think it’s doable.
02:10:08 Do you think the current approach of GPT will take us there?
02:10:12 So there is, you know, the part of at first learning all the content
02:10:16 out there and I think that Steel System should keep on learning as
02:10:20 it speaks with you.
02:10:21 Yeah.
02:10:21 Yeah, and I think that should work.
02:10:23 The question is how exactly to do it.
02:10:25 And, you know, obviously we have people at OpenAI asking these
02:10:29 questions and kind of at first pre training on all existing content
02:10:35 is like a backbone and is a decent backbone.
02:10:39 Do you think AI needs a body connecting to our robotics question to
02:10:45 truly connect with humans or can most of the connection be in the
02:10:49 digital space?
02:10:49 So let’s see, we know that there are people who met each other online
02:10:55 and they fell in love.
02:10:57 Yeah.
02:10:58 So it seems that it’s conceivable to establish connection, which is
02:11:03 purely through internet.
02:11:07 Of course, it might be more compelling the more modalities you add.
02:11:12 So it would be like you’re proposing like a Tinder, but for AI, you
02:11:16 like swipe right and left and half the systems are AI and the other is
02:11:21 humans and you don’t know which is which.
02:11:24 That would be our formulation of Turing test.
02:11:27 The moment AI is able to achieve more swipe right or left, whatever,
02:11:33 the moment it’s able to be more attractive than other humans, it
02:11:36 passes the Turing test.
02:11:38 Then you would pass the Turing test in attractiveness.
02:11:40 That’s right.
02:11:41 Well, no, like attractiveness just to clarify.
02:11:42 There will be conversation.
02:11:44 Not just visual.
02:11:44 Right, right.
02:11:45 It’s also attractiveness with wit and humor and whatever makes
02:11:51 conversation is pleasant for humans.
02:11:56 Okay.
02:11:56 All right.
02:11:58 So you’re saying it’s possible to achieve in the digital space.
02:12:02 In some sense, I would almost ask that question.
02:12:05 Why wouldn’t that be possible?
02:12:07 Well, I have this argument with my dad all the time.
02:12:11 He thinks that touch and smell are really important.
02:12:13 So they can be very important.
02:12:16 And I’m saying the initial systems, they won’t have it.
02:12:20 Still, there are people being born without these senses and I believe
02:12:28 that they can still fall in love and have meaningful life.
02:12:32 Yeah.
02:12:32 I wonder if it’s possible to go close to all the way by just training
02:12:37 on transcripts of conversations.
02:12:40 I wonder how far that takes us.
02:12:42 So I think that actually still you want images like I would like.
02:12:45 So I don’t have kids, but like I could imagine having AI Tutor.
02:12:50 It has to see, you know, kids drawing some pictures on the paper.
02:12:56 And also facial expressions, all that kind of stuff.
02:12:58 We use dogs and humans use their eyes to communicate with each other.
02:13:04 I think that’s a really powerful mechanism of communication.
02:13:07 Body language too, that words are much lower bandwidth.
02:13:12 And for body language, we still, you know, we kind of have a system
02:13:15 that displays an image of its or facial expression on the computer.
02:13:19 Doesn’t have to move, you know, mechanical pieces or so.
02:13:23 So I think that, you know, that there is like kind of a progression.
02:13:27 You can imagine that text might be the simplest to tackle.
02:13:31 But this is not a complete human experience at all.
02:13:36 You expand it to, let’s say images, both for input and output.
02:13:41 And what you describe is actually the final, I guess, frontier.
02:13:45 What makes us human, the fact that we can touch each other or smell or so.
02:13:50 And it’s the hardest from perspective of data and deployment.
02:13:54 And I believe that these things might happen gradually.
02:13:59 Are you excited by that possibility?
02:14:01 This particular application of human to AI friendship and interaction?
02:14:07 So let’s see.
02:14:09 Like would you, do you look forward to a world?
02:14:12 You said you’re living with a few folks and you’re very close friends with them.
02:14:16 Do you look forward to a day where one or two of those friends are AI systems?
02:14:19 So if the system would be truly wishing me well, rather than being in the situation
02:14:25 that it optimizes for my time to interact with the system.
02:14:28 The line between those is, it’s a gray area.
02:14:33 I think that’s the distinction between love and possession.
02:14:39 And these things, they might be often correlated for humans, but you might find that there are
02:14:46 some friends with whom you haven’t spoke for months.
02:14:49 Yeah.
02:14:50 And then you pick up the phone, it’s as the time hasn’t passed.
02:14:54 They are not holding to you.
02:14:55 And I will, I wouldn’t like to have AI system that, you know, it’s trying to convince me
02:15:02 to spend time with it.
02:15:03 I would like the system to optimize for what I care about and help me in achieving my own goals.
02:15:12 But there’s some, I mean, I don’t know, there’s some manipulation, there’s some possessiveness,
02:15:17 there’s some insecurities, this fragility, all those things are necessary to form a close
02:15:23 friendship over time, to go through some dark shit together, some bliss and happiness together.
02:15:29 I feel like there’s a lot of greedy self centered behavior within that process.
02:15:35 My intuition, but I might be wrong, is that human computer interaction doesn’t have to
02:15:41 go through a computer being greedy, possessive, and so on.
02:15:46 It is possible to train systems, maybe, that they actually
02:15:50 they are, I guess, prompted or fine tuned or so to truly optimize for what you care about.
02:15:57 And you could imagine that, you know, the way how the process would look like is at
02:16:01 some point, we as humans, we look at the transcript of the conversation or like an entire
02:16:08 interaction and we say, actually here, there was more loving way to go about it.
02:16:14 And we supervise system toward being more loving, or maybe we train the system such
02:16:20 that it has a reward function toward being more loving.
02:16:23 Yeah.
02:16:23 Or maybe the possibility of the system being an asshole and manipulative and possessive
02:16:29 every once in a while is a feature, not a bug.
02:16:33 Because some of the happiness that we experience when two souls meet each other, when two humans
02:16:40 meet each other, is a kind of break from the assholes in the world.
02:16:45 And so you need assholes in AI as well, because, like, it’ll be like a breath of fresh air
02:16:52 to discover an AI that the three previous AIs you had are too friendly or no, or cruel
02:17:00 or whatever.
02:17:01 It’s like some kind of mix.
02:17:03 And then this one is just right, but you need to experience the full spectrum.
02:17:07 Like, I think you need to be able to engineer assholes.
02:17:11 So let’s see.
02:17:14 Because there’s some level to us being appreciated to appreciate the human experience.
02:17:21 We need the dark and the light.
02:17:24 So that kind of reminds me.
02:17:27 I met a while ago at the meditation retreat, one woman, and she told me, you know,
02:17:35 beautiful, beautiful woman, and she had a she had a crutch.
02:17:41 Okay.
02:17:41 She had the trouble walking on one leg.
02:17:44 I asked her what has happened.
02:17:47 And she said that five years ago she was in Maui, Hawaii, and she was eating a salad and
02:17:55 some snail fell into the salad.
02:17:57 And apparently there are neurotoxic snails over there.
02:18:02 And she got into coma for a year.
02:18:04 Okay.
02:18:05 And apparently there is, you know, high chance of even just dying.
02:18:09 But she was in the coma.
02:18:10 At some point, she regained partially consciousness.
02:18:14 She was able to hear people in the room.
02:18:18 People behave as she wouldn’t be there.
02:18:21 You know, at some point she started being able to speak, but she was mumbling like a
02:18:25 barely able to express herself.
02:18:28 Then at some point she got into wheelchair.
02:18:30 Then at some point she actually noticed that she can move her toe and then she knew that
02:18:38 she will be able to walk.
02:18:40 And then, you know, that’s where she was five years after.
02:18:42 And she said that since then she appreciates the fact that she can move her toe.
02:18:48 And I was thinking, hmm, do I need to go through such experience to appreciate that I have
02:18:53 I can move my toe?
02:18:55 Wow, that’s a really good story and really deep example.
02:18:58 Yeah.
02:18:58 And in some sense, it might be the case that we don’t see light if we haven’t went through
02:19:05 the darkness.
02:19:06 But I wouldn’t say that we should.
02:19:08 We shouldn’t assume that that’s the case, which it may be able to engineer shortcuts.
02:19:14 Yeah.
02:19:15 Ilya had this, you know, belief that maybe one has to go for a week or six months to
02:19:22 do some challenging camp to just experience, you know, a lot of difficulties and then comes
02:19:29 back and actually everything is bright, everything is beautiful.
02:19:33 I’m with Ilya on this.
02:19:34 It must be a Russian thing.
02:19:35 Where are you from originally?
02:19:36 I’m Polish.
02:19:37 Polish.
02:19:39 Okay.
02:19:41 I’m tempted to say that explains a lot.
02:19:43 But yeah, there’s something about the Russian, the necessity of suffering.
02:19:47 I believe suffering or rather struggle is necessary.
02:19:52 I believe that struggle is necessary.
02:19:54 I mean, in some sense, you even look at the story of any superhero in the movie.
02:20:00 It’s not that it was like everything goes easy, easy, easy, easy.
02:20:03 I like how that’s your ground truth is the story of superheroes.
02:20:07 Okay.
02:20:09 You mentioned that you used to do research at night and go to bed at like 6 a.m.
02:20:13 or 7 a.m.
02:20:14 I still do that often.
02:20:18 What sleep schedules have you tried to make for a productive and happy life?
02:20:23 Like, is there is there some interesting wild sleeping patterns that you engaged that you
02:20:29 found that works really well for you?
02:20:31 I tried at some point decreasing number of hours of sleep like a gradually like a half
02:20:37 an hour every few days to this.
02:20:39 You know, I was hoping to just save time.
02:20:41 That clearly didn’t work for me.
02:20:43 Like at some point, there’s like a phase shift and I felt tired all the time.
02:20:50 You know, there was a time that I used to work during the nights.
02:20:53 The nice thing about the nights is that no one disturbs you.
02:20:57 And even I remember when I was meeting for the first time with Greg Brockman, his
02:21:04 CTO and chairman of OpenAI, our meeting was scheduled to 5 p.m.
02:21:09 And I overstepped for the meeting.
02:21:11 Over slept for the meeting at 5 p.m.
02:21:14 Yeah, now you sound like me.
02:21:15 That’s hilarious.
02:21:16 OK, yeah.
02:21:17 And at the moment, in some sense, my sleeping schedule also has to do with the fact that
02:21:23 I’m interacting with people.
02:21:26 I sleep without an alarm.
02:21:28 So, yeah, the the team thing you mentioned, the extrovert thing, because most humans operate
02:21:35 during a certain set of hours, you’re forced to then operate at the same set of hours.
02:21:42 But I’m not quite there yet.
02:21:46 I found a lot of joy, just like you said, working through the night because it’s quiet
02:21:51 because the world doesn’t disturb you.
02:21:53 And there’s some aspect counter to everything you’re saying.
02:21:57 There’s some joyful aspect to sleeping through the mess of the day because people are having
02:22:03 people are having meetings and sending emails and there’s drama meetings.
02:22:08 I can sleep through all the meetings.
02:22:09 You know, I have meetings every day and they prevent me from having sufficient amount of
02:22:14 time for focused work.
02:22:16 And then I modified my calendar and I said that I’m out of office Wednesday, Thursday
02:22:23 and Friday every day and I’m having meetings only Monday and Tuesday.
02:22:27 And that busty positively influenced my mood that I have literally like at three days for
02:22:33 fully focused work.
02:22:34 Yeah.
02:22:35 So there’s better solutions to this problem than staying awake all night.
02:22:39 OK, you’ve been part of development of some of the greatest ideas in artificial intelligence.
02:22:45 What would you say is your process for developing good novel ideas?
02:22:49 You have to be aware that clearly there are many other brilliant people around.
02:22:53 So you have to ask yourself a question, why the given idea, let’s say, wasn’t tried by
02:23:02 someone else and in some sense, it has to do with, you know, kind of simple.
02:23:10 It might sound simple, but like a thinking outside of the box.
02:23:12 And what do I mean here?
02:23:14 So, for instance, for a while, people in academia, they assumed that you have a feeling that
02:23:23 you have a fixed data set and then you optimize the algorithms in order to get the best performance.
02:23:31 And that was so in great assumption that no one thought about training models on
02:23:39 anti internet or like that.
02:23:42 Maybe some people thought about it, but it felt to many as unfair.
02:23:48 And in some sense, that’s almost like a it’s not my idea or so, but that’s an example of
02:23:53 breaking at the typical assumption.
02:23:55 So you want to be in the paradigm that you’re breaking at the typical assumption.
02:24:00 In the context of the community, getting to pick your data set is cheating.
02:24:06 Correct.
02:24:07 And in some sense, so that was that was assumption that many people had out there.
02:24:11 And then if you free yourself from assumptions, then they are likely to achieve something
02:24:19 that others cannot do.
02:24:20 And in some sense, if you are trying to do exactly the same things as others, it’s very
02:24:24 likely that you’re going to have the same results.
02:24:26 Yeah, I but there’s also that kind of tension, which is asking yourself the question, why
02:24:34 haven’t others done this?
02:24:35 Because, I mean, I get a lot of good ideas, but I think probably most of them suck when
02:24:44 they meet reality.
02:24:45 So so actually, I think the other big piece is getting into habit of generating ideas,
02:24:53 training your brain towards generating ideas and not even suspending judgment of the ideas.
02:25:00 So in some sense, I noticed myself that even if I’m in the process of generating ideas,
02:25:06 if I tell myself, oh, that was a bad idea, then that actually interrupts the process
02:25:12 and I cannot generate more ideas because I’m actually focused on the negative part, why
02:25:17 it won’t work.
02:25:17 Yes.
02:25:19 But I created also environment in the way that it’s very easy for me to store new ideas.
02:25:25 So, for instance, next to my bed, I have a voice recorder and it happens to me often
02:25:31 like I wake up during the night and I have some idea.
02:25:35 In the past, I was writing them down on my phone, but that means, you know, turning on
02:25:40 the screen and that wakes me up or like pulling a paper, which requires, you know, turning
02:25:45 on the light.
02:25:47 These days, I just start recording it.
02:25:49 What do you think, I don’t know if you know who Jim Keller is.
02:25:55 I know Jim Keller.
02:25:57 He’s a big proponent of thinking harder on a problem right before sleep so that he can
02:26:03 sleep through it and solve it in his sleep or like come up with radical stuff in his
02:26:08 sleep that’s trying to get me to do this.
02:26:11 So it happened from my experience perspective, it happened to me many times during the high
02:26:19 school days when I was doing mathematics that I had a solution to my problem as I woke up.
02:26:27 At the moment, regarding thinking hard about the given problem is I’m trying to actually
02:26:33 devote substantial amount of time to think about important problems, not just before
02:26:37 the sleep.
02:26:39 I’m organizing amount of the huge chunks of time such that I’m not constantly working
02:26:44 on the urgent problems, but I actually have time to think about the important one.
02:26:48 So you do it naturally.
02:26:49 But his idea is that you kind of prime your brain to make sure that that’s the focus.
02:26:56 Oftentimes people have other worries in their life that’s not fundamentally deep problems
02:27:00 like I don’t know, just stupid drama in your life and even at work, all that kind of stuff.
02:27:06 He wants to kind of pick the most important problem that you’re thinking about and go
02:27:12 to bed on that.
02:27:13 I think that’s wise.
02:27:14 I mean, the other thing that comes to my mind is also I feel the most fresh in the morning.
02:27:20 So during the morning, I try to work on the most important things rather than just being
02:27:25 pulled by urgent things or checking email or so.
02:27:29 What do you do with the…
02:27:30 Because I’ve been doing the voice recorder thing too, but I end up recording so many
02:27:35 messages it’s hard to organize.
02:27:37 I have the same problem.
02:27:38 Now I have heard that Google Pixel is really good in transcribing text and I might get
02:27:44 a Google Pixel just for the sake of transcribing text.
02:27:47 Yeah, people listening to this, if you have a good voice recorder suggestion that transcribe,
02:27:50 please let me know.
02:27:52 Some of it has to do with OpenAI codecs too.
02:27:57 Like some of it is simply like the friction.
02:28:01 I need apps that remove that friction between voice and the organization of the resulting
02:28:08 transcripts and all that kind of stuff.
02:28:11 But yes, you’re right.
02:28:12 Absolutely, like during, for me it’s walking, sleep too, but walking and running, especially
02:28:20 running, get a lot of thoughts during running and there’s no good mechanism for recording
02:28:25 thoughts.
02:28:25 So one more thing that I do, I have a separate phone which has no apps.
02:28:33 Maybe it has like audible or let’s say Kindle.
02:28:37 No one has this phone number, this kind of my meditation phone.
02:28:40 Yeah.
02:28:40 And I try to expand the amount of time that that’s the phone that I’m having.
02:28:47 It has also Google Maps if I need to go somewhere and I also use this phone to write down ideas.
02:28:52 Ah, that’s a really good idea.
02:28:55 That’s a really good idea.
02:28:57 Often actually what I end up doing is even sending a message from that phone to the other
02:29:01 phone.
02:29:02 So that’s actually my way of recording messages or I just put them into notes.
02:29:06 I love it.
02:29:07 What advice would you give to a young person, high school, college, about how to be successful?
02:29:15 You’ve done a lot of incredible things in the past decade, so maybe, maybe have some.
02:29:20 There’s something, there might be something.
02:29:22 There might be something.
02:29:25 I mean, might sound like a simplistic or so, but I would say literally just follow your
02:29:33 passion, double down on it.
02:29:34 And if you don’t know what’s your passion, just figure out what could be a, what could
02:29:38 be a passion.
02:29:39 So that might be an exploration.
02:29:41 When I was in elementary school was math and chemistry.
02:29:46 And I remember for some time I gave up on math because my school teacher, she told me
02:29:52 that I’m dumb.
02:29:54 And I guess maybe an advice would be just ignore people if they tell you that you’re
02:30:00 dumb.
02:30:00 You’re dumb.
02:30:01 You’re dumb. You mentioned something offline about chemistry and explosives.
02:30:08 What was that about?
02:30:09 So let’s see.
02:30:11 So a story goes like that.
02:30:16 I got into chemistry.
02:30:18 Maybe I was like a second grade of my elementary school, third grade.
02:30:23 I started going to chemistry classes.
02:30:27 I really love building stuff.
02:30:30 And I did all the experiments that they describe in the book, like, you know, how to create
02:30:35 oxygen with vinegar and baking soda or so.
02:30:39 Okay.
02:30:40 So I did all the experiments and at some point I was, you know, so what’s next?
02:30:45 What can I do?
02:30:47 And explosives, they also, it’s like a, you have a clear reward signal, you know, if the
02:30:53 thing worked or not.
02:30:54 So I remember at first I got interested in producing hydrogen.
02:31:00 That was kind of funny experiment from school.
02:31:03 You can just burn it.
02:31:04 And then I moved to nitroglycerin.
02:31:07 So that’s also relatively easy to synthesize.
02:31:11 I started producing essentially dynamite and detonating it with a friend.
02:31:16 I remember there was a, you know, there was at first like maybe two attempts that I went
02:31:20 with a friend to detonate what we built and it didn’t work out.
02:31:25 And like a third time he was like, ah, it won’t work.
02:31:27 Like, let’s don’t waste time.
02:31:30 And, you know, we were, I was carrying this, this, you know, that tube with dynamite, I
02:31:38 don’t know, pound or so, dynamite in my backpack, we’re like riding on the bike to the edges
02:31:45 of the city.
02:31:45 Yeah, and attempt number three, this was be attempt number three.
02:31:51 Attempt number three.
02:31:52 And now we dig a hole to put it inside.
02:31:57 It actually had the, you know, electrical detonator.
02:32:02 We draw a cable behind the tree.
02:32:05 I even, I never had, I haven’t ever seen like a explosion before.
02:32:10 So I thought that there would be a lot of, you know, a lot of, you know, a lot of, you
02:32:15 know, there will be a lot of sound.
02:32:17 But, you know, we’re like laying down and I’m holding the cable and the battery.
02:32:22 At some point, you know, we kind of like a three to one and I just connected it and it
02:32:28 felt like the ground shake.
02:32:30 It was like more like a sound.
02:32:32 And then the soil started kind of lifting up and started falling on us.
02:32:37 Yeah.
02:32:38 Wow.
02:32:39 And then, you know, the friend said, let’s make sure the next time we have helmets.
02:32:45 But also, you know, I’m happy that nothing happened to me.
02:32:48 It could have been the case that I lost the limbo or so.
02:32:52 Yeah, but that’s childhood of an engineering mind with a strong reward signal of an
02:33:01 explosion.
02:33:03 I love it.
02:33:04 My there’s some aspect of chemists that the chemists I know, like my dad with plasma
02:33:10 chemistry, plasma physics, he was very much into explosives, too.
02:33:13 It’s a worrying quality of people that work in chemistry that they love.
02:33:18 I think it is that exactly is the strong signal that the thing worked.
02:33:23 There is no doubt.
02:33:24 There’s no doubt.
02:33:25 There’s some magic.
02:33:26 It’s almost like a reminder that physics works, that chemistry works.
02:33:31 It’s cool.
02:33:32 It’s almost like a little glimpse at nature that you yourself engineer.
02:33:36 I that’s why I really like artificial intelligence, especially robotics, is you create a little
02:33:43 piece of nature and in some sense, even for me with explosives, the motivation was creation
02:33:49 rather than destruction.
02:33:50 Yes, exactly.
02:33:51 In terms of advice, I forgot to ask about just machine learning and deep learning for
02:33:57 people who are specifically interested in machine learning, how would you recommend
02:34:01 they get into the field?
02:34:03 So I would say re implement everything and also there is plenty of courses.
02:34:08 So like from scratch?
02:34:10 So on different levels of abstraction in some sense, but I would say re implement something
02:34:14 from scratch, re implement something from a paper, re implement something, you know,
02:34:19 from podcasts that you have heard about.
02:34:21 I would say that’s a powerful way to understand things.
02:34:23 So it’s often the case that you read the description and you think you understand, but you truly
02:34:30 understand once you build it, then you actually know what really matter in the description.
02:34:36 Is there a particular topics that you find people just fall in love with?
02:34:41 I’ve seen.
02:34:44 I tend to really enjoy reinforcement learning because it’s much more, it’s much easier
02:34:51 to get to a point where you feel like you created something special, like fun games
02:34:57 kind of things that are rewarding.
02:34:58 It’s rewarding.
02:34:59 Yeah.
02:35:01 As opposed to like re implementing from scratch, more like supervised learning kind of things.
02:35:07 It’s yeah.
02:35:08 So, you know, if someone would optimize for things to be rewarding, then it feels that
02:35:15 the things that are somewhat generative, they have such a property.
02:35:18 So you have, for instance, adversarial networks, or do you have just even generative language
02:35:23 models?
02:35:24 And you can even see, internally, we have seen this thing with our releases.
02:35:30 So we have, we released recently two models.
02:35:33 There is one model called Dali that generates images, and there is other model called Clip
02:35:39 that actually you provide various possibilities, what could be the answer to what is on the
02:35:45 picture, and it can tell you which one is the most likely.
02:35:48 And in some sense, in case of the first one, Dali, it is very easy for you to understand
02:35:56 that actually there is magic going on.
02:35:59 And in the case of the second one, even though it is insanely powerful, and you know, people
02:36:04 from a vision community, they, as they started probing it inside, they actually understood
02:36:12 how far it goes.
02:36:13 How far it goes, it’s difficult for a person at first to see how well it works.
02:36:21 And that’s the same, as you said, that in case of supervised learning models, you might
02:36:25 not kind of see, or it’s not that easy for you to understand the strength.
02:36:31 Even though you don’t believe in magic, to see the magic.
02:36:33 To see the magic, yeah.
02:36:35 It’s a generative.
02:36:36 That’s really brilliant.
02:36:37 So anything that’s generative, because then you are at the core of the creation.
02:36:42 You get to experience creation without much effort.
02:36:46 Unless you have to do it from scratch, but.
02:36:48 And it feels that, you know, humans are wired.
02:36:51 There is some level of reward for creating stuff.
02:36:54 Yeah.
02:36:56 Of course, different people have a different weight on this reward.
02:36:59 Yeah.
02:37:00 In the big objective function.
02:37:01 In the big objective function of a person.
02:37:03 Of a person.
02:37:05 You wrote that beautiful is what you intensely pay attention to.
02:37:10 Even a cockroach is beautiful.
02:37:12 If you look very closely, can you expand on this?
02:37:16 What is beauty?
02:37:18 So what I’m, I wrote here actually corresponds to my subjective experience that I had through
02:37:26 extended periods of meditation.
02:37:28 It’s, it’s pretty crazy that at some point the meditation gets you to the place that
02:37:34 you have really increased focus, increased attention.
02:37:39 Increased attention.
02:37:40 And then you look at the very simple objects that were all the time around you can look
02:37:45 at the table or on the pen or at the nature.
02:37:49 And you notice more and more details and it becomes very pleasant to look at it.
02:37:56 And it, once again, it kind of reminds me of my childhood.
02:38:01 Like I just pure joy of being.
02:38:03 It’s also, I have seen even the reverse effect that by default, regardless of what we possess,
02:38:11 we very quickly get used to it.
02:38:14 And you know, you can have a very beautiful house and if you don’t put sufficient effort,
02:38:21 you’re just going to get used to it and it doesn’t bring any more joy,
02:38:25 regardless of what you have.
02:38:27 Yeah.
02:38:27 Well, I actually, I find that material possessions get in the way of that experience of pure
02:38:36 joy.
02:38:38 So I’ve always, I’ve been very fortunate to just find joy in simple things.
02:38:45 Just, just like you’re saying, just like, I don’t know, objects in my life, just stupid
02:38:50 objects like this cup, like thing, you know, just objects sounds okay.
02:38:55 I’m not being eloquent, but literally objects in the world, they’re just full of joy.
02:39:00 Cause it’s like, I can’t believe when I can’t believe that I’m fortunate enough to be alive
02:39:07 to experience these objects.
02:39:09 And then two, I can’t believe humans are clever enough to have built these objects.
02:39:15 The hierarchy of pleasure that that provides is infinite.
02:39:19 I mean, even if you look at the cup of water, so, you know, you see first like a level of
02:39:24 like a reflection of light, but then you think, you know, man, there’s like a trillions upon
02:39:28 of trillions of particles bouncing against each other.
02:39:32 There is also the tension on the surface that, you know, if the back, back could like a stand
02:39:38 on it and move around.
02:39:40 And you think it also has this like a magical property that as you decrease temperature,
02:39:45 it actually expands in volume, which allows for the, you know, legs to freeze on the,
02:39:51 on the surface and at the bottom to have actually not freeze, which allows for life like a crazy.
02:39:58 Yeah.
02:39:58 You look in detail at some object and you think actually, you know, this table, that
02:40:03 was just a figment of someone’s imagination at some point.
02:40:06 And then there was like a thousands of people involved to actually manufacture it and put
02:40:10 it here.
02:40:11 And by default, no one cares.
02:40:15 And then you can start thinking about evolution, how it all started from single cell organisms
02:40:19 that led to this table.
02:40:21 And these thoughts, they give me life appreciation and even lack of thoughts, just the pure raw
02:40:27 signal also gives the life appreciation.
02:40:29 See, the thing is, and then that’s coupled for me with the sadness that the whole ride
02:40:37 ends and perhaps is deeply coupled in that the fact that this experience, this moment
02:40:43 ends, gives it, gives it an intensity that I’m not sure I would otherwise have.
02:40:50 So in that same way, I tried to meditate on my own death.
02:40:53 Often.
02:40:54 Do you think about your mortality?
02:40:58 Are you afraid of death?
02:41:01 So fear of death is like one of the most fundamental fears that each of us has.
02:41:07 We might be not even aware of it.
02:41:09 It requires to look inside, to even recognize that it’s out there and there is still, let’s
02:41:15 say, this property of nature that if things would last forever, then they would be also
02:41:22 boring to us.
02:41:24 The fact that the things change in some way gives any meaning to them.
02:41:29 I also, you know, found out that it seems to be very healing to people to have these
02:41:40 short experiences, like, I guess, psychedelic experiences in which they experience death
02:41:49 of self in which they let go of this fear and then maybe can even increase the intensity
02:41:58 can even increase the appreciation of the moment.
02:42:01 It seems that many people, they can easily comprehend the fact that the money is finite
02:42:12 while they don’t see that time is finite.
02:42:15 I have this like a discussion with Ilya from time to time.
02:42:18 He’s like, you know, man, like the life will pass very fast.
02:42:23 At some point I will be 40, 50, 60, 70 and then it’s over.
02:42:26 This is true, which also makes me believe that, you know, that every single moment it
02:42:33 is so unique that should be appreciated.
02:42:37 And this also makes me think that I should be acting on my life because otherwise it
02:42:44 will pass.
02:42:46 I also like this framework of thinking from Jeff Bezos on regret minimization that like
02:42:53 I would like if I will be at that deathbed to look back on my life and not regret that
02:43:01 I haven’t done something.
02:43:03 It’s usually you might regret that you haven’t tried.
02:43:07 I’m fine with failing.
02:43:10 I haven’t tried.
02:43:13 What’s the Nietzsche eternal occurrence?
02:43:15 Try to live a life that if you had to live it infinitely many times, that would be the
02:43:20 you’d be okay with that kind of life.
02:43:24 So try to live it optimally.
02:43:27 I can say that it’s almost like I’m.
02:43:33 I’m available to me where I am in my life.
02:43:36 I’m extremely grateful for actually people whom I met.
02:43:40 I would say I think that I’m decently smart and so on.
02:43:44 But I think that actually to a great extent where I am has to do with the people who I
02:43:50 met.
02:43:52 Would you be okay if after this conversation you died?
02:43:56 So if I’m dead, then it kind of I don’t have a choice anymore.
02:44:01 So in some sense, there’s like plenty of things that I would like to try out in my life.
02:44:07 I feel that I’m gradually going one by one and I’m just doing them.
02:44:10 I think that the list will be always infinite.
02:44:13 Yeah, so might as well go today.
02:44:16 Yeah, I mean, to be clear, I’m not looking forward to die.
02:44:20 I would say if there is no choice, I would accept it.
02:44:24 But like in some sense, I’m if there would be a choice, if there would be a possibility
02:44:30 to leave, I would fight for leaving.
02:44:33 I find it’s more.
02:44:37 I find it’s more honest and real to think about, you know, dying today at the end of
02:44:44 the day.
02:44:46 That seems to me, at least to my brain, more honest slap in the face as opposed to I still
02:44:52 have 10 years like today, then I’m much more about appreciating the cup and the table and
02:44:59 so on and less about like silly worldly accomplishments and all those kinds of things.
02:45:04 But we have in the company a person who say at some point found out that they have cancer
02:45:11 and that also gives, you know, huge perspective with respect to what matters now.
02:45:16 Yeah.
02:45:16 And, you know, often people in situations like that, they conclude that actually what
02:45:20 matters is human connection.
02:45:22 And love and that’s people conclude also if you have kids, kids as family.
02:45:28 You, I think, tweeted, we don’t assign the minus infinity reward to our death.
02:45:35 Such a reward would prevent us from taking any risk.
02:45:38 We wouldn’t be able to cross the road in fear of being hit by a car.
02:45:42 So in the objective function, you mentioned fear of death might be fundamental to the
02:45:46 human condition.
02:45:48 So, as I said, let’s assume that they’re like a reward functions in our brain.
02:45:52 And the interesting thing is even realization, how different reward functions can play with
02:46:01 your behavior.
02:46:03 As a matter of fact, I wouldn’t say that you should assign infinite negative reward to
02:46:09 anything because that messes up the math.
02:46:12 The math doesn’t work out.
02:46:13 It doesn’t work out.
02:46:14 And as you said, even, you know, government or some insurance companies, you said they
02:46:19 assign $9 million to human life.
02:46:22 And I’m just saying it with respect to, that might be a hard statement to ourselves, but
02:46:29 in some sense that there is a finite value of our own life.
02:46:34 I’m trying to put it from perspective of being less, of being more egoless and realizing
02:46:43 fragility of my own life.
02:46:44 And in some sense, the fear of death might prevent you from acting because anything can
02:46:53 cause death.
02:46:56 Yeah.
02:46:56 And I’m sure actually, if you were to put death in the objective function, there’s probably
02:47:00 so many aspects to death and fear of death and realization of death and mortality.
02:47:06 There’s just whole components of finiteness of not just your life, but every experience
02:47:13 and so on that you’re going to have to formalize mathematically.
02:47:18 And also, you know, that might lead to you spending a lot of compute cycles on this like
02:47:27 a deliberating this terrible future instead of experiencing now.
02:47:32 And then in some sense, it’s also kind of unpleasant simulation to run in your head.
02:47:36 Yeah.
02:47:39 Do you think there’s an objective function that describes the entirety of human life?
02:47:45 So, you know, usually the way you ask that is what is the meaning of life?
02:47:50 Is there a universal objective functions that captures the why of life?
02:47:55 So, yeah, I mean, I suspect that they will ask this question, but it’s also a question
02:48:03 that I ask myself many, many times.
02:48:06 See, I can tell you a framework that I have these days to think about this question.
02:48:10 So I think that fundamentally, meaning of life has to do with some of our reward actions
02:48:16 that we have in brain and they might have to do with, let’s say, for instance, curiosity
02:48:21 or human connection, which might mean understanding others.
02:48:27 It’s also possible for a person to slightly modify their reward function.
02:48:32 Usually they mostly stay fixed, but it’s possible to modify reward function and you can pretty
02:48:37 much choose.
02:48:38 So in some sense, the reward functions, optimizing reward functions, they will give you a life
02:48:42 satisfaction.
02:48:44 Is there some randomness in the function?
02:48:45 I think when you are born, there is some randomness.
02:48:48 You can see that some people, for instance, they care more about building stuff.
02:48:53 Some people care more about caring for others.
02:48:56 Some people, there are all sorts of default reward functions.
02:49:00 And then in some sense, you can ask yourself, what is the satisfying way for you to go after
02:49:08 this reward function?
02:49:09 And you just go after this reward function.
02:49:11 And, you know, some people also ask, are you satisfied with your life?
02:49:15 And, you know, some people also ask, are these reward functions real?
02:49:19 I almost think about it as, let’s say, if you would have to discover mathematics, in
02:49:27 mathematics, you are likely to run into various objects like complex numbers or differentiation,
02:49:34 some other objects.
02:49:35 And these are very natural objects that arise.
02:49:38 And similarly, the reward functions that we are having in our brain, they are somewhat
02:49:42 very natural, that, you know, there is a reward function for understanding, like a comprehension,
02:49:52 curiosity, and so on.
02:49:53 So in some sense, they are in the same way natural as their natural objects in mathematics.
02:49:59 Interesting.
02:49:59 So, you know, there’s the old sort of debate, is mathematics invented or discovered?
02:50:05 You’re saying reward functions are discovered.
02:50:07 So nature.
02:50:08 So nature provided some, you can still, let’s say, expand it throughout the life.
02:50:12 Some of the reward functions, they might be futile.
02:50:15 Like, for instance, there might be a reward function, maximize amount of wealth.
02:50:20 Yeah.
02:50:20 And this is more like a learned reward function.
02:50:25 But we know also that some reward functions, if you optimize them, you won’t be quite satisfied.
02:50:32 Well, I don’t know which part of your reward function resulted in you coming today, but
02:50:37 I am deeply appreciative that you did spend your valuable time with me.
02:50:40 Wojtek is really fun talking to you.
02:50:43 You’re brilliant.
02:50:45 You’re a good human being.
02:50:46 And it’s an honor to meet you and an honor to talk to you.
02:50:48 Thanks for talking today, brother.
02:50:50 Thank you, Lex a lot.
02:50:51 I appreciated your questions, curiosity.
02:50:54 I had a lot of time being here.
02:50:57 Thanks for listening to this conversation with Wojtek Zaremba.
02:51:00 To support this podcast, please check out our sponsors in the description.
02:51:04 And now, let me leave you with some words from Arthur C. Clarke, who is the author of
02:51:10 2001 A Space Odyssey.
02:51:12 It may be that our role on this planet is not to worship God, but to create him.
02:51:18 Thank you for listening, and I hope to see you next time.