Transcript
00:00:00 The following is a conversation with Jeff Schoenlein,
00:00:03 a scientist at NIST
00:00:04 interested in optoelectronic intelligence.
00:00:08 We have a deep technical dive into computing hardware
00:00:12 that will make Jim Keller proud.
00:00:14 I urge you to hop onto this rollercoaster ride
00:00:17 through neuromorphic computing
00:00:19 and superconducting electronics
00:00:21 and hold on for dear life.
00:00:24 Jeff is a great communicator of technical information
00:00:27 and so it was truly a pleasure to talk to him
00:00:30 about some physics and engineering.
00:00:33 To support this podcast,
00:00:34 please check out our sponsors in the description.
00:00:37 This is the Lex Friedman Podcast
00:00:39 and here is my conversation with Jeff Schoenlein.
00:00:44 I got a chance to read a fascinating paper you authored
00:00:48 called Optoelectronic Intelligence.
00:00:52 So maybe we can start by talking about this paper
00:00:55 and start with the basic questions.
00:00:57 What is optoelectronic intelligence?
00:01:00 Yeah, so in that paper,
00:01:02 the concept I was trying to describe
00:01:04 is sort of an architecture
00:01:06 for building brain inspired computing
00:01:10 that leverages light for communication
00:01:13 in conjunction with electronic circuits for computation.
00:01:17 In that particular paper,
00:01:18 a lot of the work we’re doing right now
00:01:20 in our project at NIST
00:01:22 is focused on superconducting electronics for computation.
00:01:25 I won’t go into why that is,
00:01:27 but that might make a little more sense in context
00:01:31 if we first describe what that is in contrast to,
00:01:35 which is semiconducting electronics.
00:01:37 So is it worth taking a couple minutes
00:01:39 to describe semiconducting electronics?
00:01:42 It might even be worthwhile to step back
00:01:45 and talk about electricity and circuits
00:01:49 and how circuits work
00:01:52 before we talk about superconductivity.
00:01:54 Right, okay.
00:01:56 How does a computer work, Jeff?
00:01:58 Well, I won’t go into everything
00:01:59 that makes a computer work,
00:02:01 but let’s talk about the basic building blocks,
00:02:05 a transistor, and even more basic than that,
00:02:08 a semiconductor material, silicon, say.
00:02:11 So in silicon, silicon is a semiconductor,
00:02:15 and what that means is at low temperature,
00:02:18 there are no free charges,
00:02:20 no free electrons that can move around.
00:02:22 So when you talk about electricity,
00:02:24 you’re talking about predominantly electrons
00:02:28 moving to establish electrical currents,
00:02:30 and they move under the influence of voltages.
00:02:33 So you apply voltages, electrons move around,
00:02:36 those can be measured as currents,
00:02:38 and you can represent information in that way.
00:02:40 So semiconductors are special
00:02:43 in the sense that they are really malleable.
00:02:46 So if you have a semiconductor material,
00:02:50 you can change the number of free electrons
00:02:52 that can move around by putting different elements,
00:02:56 different atoms in lattice sites.
00:02:58 So what is a lattice site?
00:03:00 Well, a semiconductor is a crystal,
00:03:02 which means all the atoms that comprise the material
00:03:06 are at exact locations
00:03:09 that are perfectly periodic in space.
00:03:10 So if you start at any one atom
00:03:12 and you go along what are called the lattice vectors,
00:03:14 you get to another atom and another atom and another atom,
00:03:17 and for high quality devices,
00:03:19 it’s important that it’s a perfect crystal
00:03:21 with very few defects,
00:03:23 but you can intentionally replace a silicon atom
00:03:27 with say a phosphorus atom,
00:03:29 and then you can change the number of free electrons
00:03:32 that are in a region of space
00:03:33 that has that excess of what are called dopants.
00:03:37 So picture a device that has a left terminal
00:03:40 and a right terminal,
00:03:42 and if you apply a voltage between those two,
00:03:44 you can cause electrical current to flow between them.
00:03:47 Now we add a third terminal up on top there,
00:03:52 and depending on the voltage
00:03:53 between the left and right terminal and that third voltage,
00:03:57 you can change that current.
00:03:58 So what’s commonly done in digital electronic circuits
00:04:02 is to leave a fixed voltage from left to right,
00:04:06 and then change that voltage
00:04:08 that’s applied at what’s called the gate,
00:04:10 the gate of the transistor.
00:04:11 So what you do is you make it to where
00:04:13 there’s an excess of electrons on the left,
00:04:15 excess of electrons on the right,
00:04:17 and very few electrons in the middle,
00:04:18 and you do this by changing the concentration
00:04:21 of different dopants in the lattice spatially.
00:04:24 And then when you apply a voltage to that gate,
00:04:27 you can either cause current to flow or turn it off,
00:04:30 and so that’s sort of your zero and one.
00:04:33 If you apply voltage, current can flow,
00:04:35 that current is representing a digital one,
00:04:38 and from that, from that basic element,
00:04:41 you can build up all the complexity
00:04:44 of digital electronic circuits
00:04:45 that have really had a profound influence on our society.
00:04:50 Now you’re talking about electrons.
00:04:51 Can you give a sense of what scale we’re talking about
00:04:54 when we’re talking about in silicon
00:04:57 being able to mass manufacture these kinds of gates?
00:05:01 Yeah, so scale in a number of different senses.
00:05:04 Well, at the scale of the silicon lattice,
00:05:07 the distance between two atoms there is half a nanometer.
00:05:10 So people often like to compare these things
00:05:14 to the width of a human hair.
00:05:16 I think it’s some six orders of magnitude smaller
00:05:20 than the width of a human hair, something on that order.
00:05:24 So remarkably small,
00:05:25 we’re talking about individual atoms here,
00:05:27 and electrons are of that length scale
00:05:29 when they’re in that environment.
00:05:31 But there’s another sense
00:05:32 that scale matters in digital electronics.
00:05:34 This is perhaps the more important sense,
00:05:36 although they’re related.
00:05:37 Scale refers to a number of things.
00:05:41 It refers to the size of that transistor.
00:05:44 So for example, I said you have a left contact,
00:05:47 a right contact, and some space between them
00:05:49 where the gate electrode sits.
00:05:52 That’s called the channel width or the channel length.
00:05:56 And what has enabled what we think of as Moore’s law
00:06:00 or the continued increased performance
00:06:03 in silicon microelectronic circuits
00:06:05 is the ability to make that size, that feature size,
00:06:08 ever smaller, ever smaller at a really remarkable pace.
00:06:14 I mean, that feature size has decreased consistently
00:06:20 every couple of years since the 1960s.
00:06:23 And that was what Moore predicted in the 1960s.
00:06:27 He thought it would continue for at least two more decades,
00:06:29 and it’s been much longer than that.
00:06:30 And so that is why we’ve been able to fit ever more devices,
00:06:35 ever more transistors, ever more computational power
00:06:39 on essentially the same size of chip.
00:06:41 So a user sits back and does essentially nothing.
00:06:44 You’re running the same computer program,
00:06:45 but those devices are getting smaller, so they get faster,
00:06:48 they get more energy efficient,
00:06:50 and all of our computing performance
00:06:51 just continues to improve.
00:06:53 And we don’t have to think too hard
00:06:56 about what we’re doing as, say,
00:06:59 a software designer or something like that.
00:07:00 I absolutely don’t mean to say
00:07:02 that there’s no innovation in software or the user side
00:07:05 of things, of course there is.
00:07:07 But from the hardware perspective,
00:07:09 we just have been given this gift
00:07:12 of continued performance improvement
00:07:14 through this scaling that is ever smaller feature sizes
00:07:19 with very similar, say, power consumption.
00:07:22 That power consumption has not continued to scale
00:07:25 in the most recent decades, but nevertheless,
00:07:29 we had a really good run there for a while.
00:07:31 And now we’re down to gates that are seven nanometers,
00:07:34 which is state of the art right now.
00:07:36 Maybe GlobalFoundries is trying to push it
00:07:38 even lower than that.
00:07:39 I can’t keep up with where the predictions are
00:07:42 that it’s gonna end.
00:07:43 But seven nanometer transistor has just a few tens of atoms
00:07:49 along the length of the conduction pathway.
00:07:51 So a naive semiconductor device physicist
00:07:56 would think you can’t go much further than that
00:07:58 without some kind of revolution in the way we think
00:08:02 about the physics of our devices.
00:08:03 Is there something to be said
00:08:04 about the mass manufacture of these devices?
00:08:08 Right, right, so that’s another thing.
00:08:09 So how have we been able
00:08:10 to make those transistors smaller and smaller?
00:08:13 Well, companies like Intel, GlobalFoundries,
00:08:17 they invest a lot of money in the lithography.
00:08:20 So how are these chips actually made?
00:08:22 Well, one of the most important steps
00:08:24 is this what’s called ion implantation.
00:08:27 So you start with sort of a pristine silicon crystal
00:08:31 and then using photolithography,
00:08:34 which is a technique where you can pattern
00:08:36 different shapes using light,
00:08:38 you can define which regions of space
00:08:41 you’re going to implant with different species of ions
00:08:45 that are going to change
00:08:46 the local electrical properties right there.
00:08:49 So by using ever shorter wavelengths of light
00:08:52 and different kinds of optical techniques
00:08:54 and different kinds of lithographic techniques,
00:08:56 things that go far beyond my knowledge base,
00:09:00 you can just simply shrink that feature size down.
00:09:03 And you say you’re at seven nanometers.
00:09:04 Well, the wavelength of light that’s being used
00:09:07 is over a hundred nanometers.
00:09:08 That’s already deep in the UV.
00:09:10 So how are those minute features patterned?
00:09:14 Well, there’s an extraordinary amount of innovation
00:09:16 that has gone into that,
00:09:18 but nevertheless, it stayed very consistent
00:09:20 in this ever shrinking feature size.
00:09:21 And now the question is, can you make it smaller?
00:09:24 And even if you do, do you still continue
00:09:26 to get performance improvements?
00:09:28 But that’s another kind of scaling
00:09:30 where these companies have been able to…
00:09:34 So, okay, you picture a chip that has a processor on it.
00:09:36 Well, that chip is not made as a chip.
00:09:38 It’s made on a wafer.
00:09:40 And using photolithography,
00:09:43 you basically print the same pattern on different dyes
00:09:47 all across the wafer, multiple layers,
00:09:49 tens, probably a hundred some layers
00:09:53 in a mature foundry process.
00:09:55 And you do this on ever bigger wafers too.
00:09:57 That’s another aspect of scaling
00:09:58 that’s occurred in the last several decades.
00:10:00 So now you have this 300 millimeter wafer.
00:10:02 It’s like as big as a pizza
00:10:04 and it has maybe a thousand processors on it.
00:10:06 And then you dice that up using a saw.
00:10:08 And now you can sell these things so cheap
00:10:11 because the manufacturing process was so streamlined.
00:10:14 I think a technology as revolutionary
00:10:17 as silicon microelectronics has to have
00:10:19 that kind of manufacturing scalability,
00:10:23 which I will just emphasize,
00:10:25 I believe is enabled by physics.
00:10:28 It’s not, I mean, of course there’s human ingenuity
00:10:31 that goes into it, but at least from my side where I sit,
00:10:35 it sure looks like the physics of our universe
00:10:38 allows us to produce that.
00:10:40 And we’ve discovered how more so than we’ve invented it,
00:10:45 although of course we have invented it,
00:10:47 humans have invented it,
00:10:48 but it’s almost as if it was there
00:10:50 waiting for us to discover it.
00:10:52 You mean the entirety of it
00:10:54 or are you specifically talking about
00:10:55 the techniques of photolithography,
00:10:58 like the optics involved?
00:10:59 I mean, the entirety of the scaling down
00:11:02 to the seven nanometers,
00:11:04 you’re able to have electrons not interfere with each other
00:11:08 in such a way that you could still have gates.
00:11:11 Like that’s enabled.
00:11:13 To achieve that scale, spatial and temporal,
00:11:16 it seems to be very special
00:11:18 and is enabled by the physics of our world.
00:11:21 All of the things you just said.
00:11:22 So starting with the silicon material itself,
00:11:25 silicon is a unique semiconductor.
00:11:28 It has essentially ideal properties
00:11:31 for making a specific kind of transistor
00:11:33 that’s extraordinarily useful.
00:11:35 So I mentioned that silicon has,
00:11:39 well, when you make a transistor,
00:11:40 you have this gate contact
00:11:42 that sits on top of the conduction channel.
00:11:44 And depending on the voltage you apply there,
00:11:47 you pull more carriers into the conduction channel
00:11:50 or push them away so it becomes more or less conductive.
00:11:53 In order to have that work
00:11:54 without just sucking those carriers right into that contact,
00:11:57 you need a very thin insulator.
00:11:59 And part of scaling has been to gradually decrease
00:12:03 the thickness of that gate insulator
00:12:06 so that you can use a roughly similar voltage
00:12:09 and still have the same current voltage characteristics.
00:12:12 So the material that’s used to do that,
00:12:14 or I should say was initially used to do that
00:12:16 was a silicon dioxide,
00:12:18 which just naturally grows on the silicon surface.
00:12:21 So you expose silicon to the atmosphere that we breathe
00:12:25 and well, if you’re manufacturing,
00:12:27 you’re gonna purify these gases,
00:12:29 but nevertheless,
00:12:30 that what’s called a native oxide will grow there.
00:12:33 There are essentially no other materials
00:12:36 on the entire periodic table
00:12:37 that have as good of a gate insulator
00:12:42 as that silicon dioxide.
00:12:43 And that has to do with nothing but the physics
00:12:46 of the interaction between silicon and oxygen.
00:12:49 And if it wasn’t that way,
00:12:51 transistors could not perform
00:12:54 in nearly the degree of capability that they have.
00:12:58 And that has to do with the way that the oxide grows,
00:13:02 the reduced density of defects there,
00:13:05 it’s insulation, meaning essentially it’s energy gaps.
00:13:08 You can apply a very large voltage there
00:13:10 without having current leak through it.
00:13:12 So that’s physics right there.
00:13:15 There are other things too.
00:13:16 Silicon is a semiconductor in an elemental sense.
00:13:19 You only need silicon atoms.
00:13:21 A lot of other semiconductors,
00:13:22 you need two different kinds of atoms,
00:13:24 like a compound from group three
00:13:26 and a compound from group five.
00:13:28 That opens you up to lots of defects that can occur
00:13:31 where one atom’s not sitting quite at the lattice site,
00:13:34 it is and it’s switched with another one
00:13:35 that degrades performance.
00:13:38 But then also on the side that you mentioned
00:13:40 with the manufacturing,
00:13:43 we have access to light sources
00:13:45 that can produce these very short wavelengths of light.
00:13:49 How does photolithography occur?
00:13:50 Well, you actually put this polymer on top of your wafer
00:13:54 and you expose it to light,
00:13:56 and then you use a aqueous chemical processing
00:14:00 to dissolve away the regions that were exposed to light
00:14:03 and leave the regions that were not.
00:14:05 And we are blessed with these polymers
00:14:08 that have the right property
00:14:09 where they can cause scission events
00:14:13 where the polymer splits where a photon hits.
00:14:16 I mean, maybe that’s not too surprising,
00:14:19 but I don’t know, it all comes together
00:14:21 to have this really complex,
00:14:24 manufacturable ecosystem
00:14:26 where very sophisticated technologies can be devised
00:14:30 and it works quite well.
00:14:33 And amazingly, like you said,
00:14:34 with a wavelength at like 100 nanometers
00:14:36 or something like that,
00:14:37 you’re still able to achieve on this polymer
00:14:39 precision of whatever we said, seven nanometers.
00:14:43 I think I’ve heard like four nanometers
00:14:45 being talked about, something like that.
00:14:48 If we could just pause on this
00:14:49 and we’ll return to superconductivity,
00:14:52 but in this whole journey from a history perspective,
00:14:56 what do you think is the most beautiful
00:14:59 at the intersection of engineering and physics
00:15:01 to you in this whole process
00:15:03 that we talked about with silicon and photolithography,
00:15:06 things that people were able to achieve
00:15:08 in order to push Moore’s law forward?
00:15:12 Is it the early days,
00:15:13 the invention of the transistor itself?
00:15:16 Is it some particular cool little thing
00:15:19 that maybe not many people know about?
00:15:21 Like, what do you think is most beautiful
00:15:24 in this whole process, journey?
00:15:26 The most beautiful is a little difficult to answer.
00:15:29 Let me try and sidestep it a little bit
00:15:32 and just say what strikes me about looking
00:15:35 at the history of silicon microelectronics is that,
00:15:42 so when quantum mechanics was developed,
00:15:44 people quickly began applying it to semiconductors
00:15:47 and it was broadly understood
00:15:49 that these are fascinating systems
00:15:50 and people cared about them for their basic physics,
00:15:52 but also their utility as devices.
00:15:55 And then the transistor was invented in the late forties
00:15:59 in a relatively crude experimental setup
00:16:02 where you just crammed a metal electrode
00:16:04 into the semiconductor and that was ingenious.
00:16:08 These people were able to make it work.
00:16:13 But so what I wanna get to that really strikes me
00:16:16 is that in those early days,
00:16:19 there were a number of different semiconductors
00:16:21 that were being considered.
00:16:22 They had different properties, different strengths,
00:16:23 different weaknesses.
00:16:24 Most people thought germanium was the way to go.
00:16:28 It had some nice properties related to things
00:16:33 about how the electrons move inside the lattice.
00:16:37 But other people thought that compound semiconductors
00:16:39 with group three and group five also had
00:16:42 really, really extraordinary properties
00:16:46 that might be conducive to making the best devices.
00:16:50 So there were different groups exploring each of these
00:16:52 and that’s great, that’s how science works.
00:16:54 You have to cast a broad net.
00:16:56 But then what I find striking is why is it that silicon won?
00:17:02 Because it’s not that germanium is a useless material
00:17:05 and it’s not present in technology
00:17:06 or compound semiconductors.
00:17:08 They’re both doing exciting and important things,
00:17:12 slightly more niche applications
00:17:14 whereas silicon is the semiconductor material
00:17:18 for microelectronics which is the platform
00:17:20 for digital computing which has transformed our world.
00:17:22 Why did silicon win?
00:17:24 It’s because of a remarkable assemblage of qualities
00:17:28 that no one of them was the clear winner
00:17:32 but it made these sort of compromises
00:17:34 between a number of different influences.
00:17:36 It had that really excellent gate oxide
00:17:40 that allowed us to make MOSFETs,
00:17:43 these high performance transistors,
00:17:45 so quickly and cheaply and easily
00:17:47 without having to do a lot of materials development.
00:17:49 The band gap of silicon is actually,
00:17:53 so in a semiconductor there’s an important parameter
00:17:56 which is called the band gap
00:17:57 which tells you there are sort of electrons
00:18:00 that fill up to one level in the energy diagram
00:18:04 and then there’s a gap where electrons aren’t allowed
00:18:06 to have an energy in a certain range
00:18:08 and then there’s another energy level above that.
00:18:11 And that difference between the lower sort of filled level
00:18:14 and the unoccupied level,
00:18:16 that tells you how much voltage you have to apply
00:18:19 in order to induce a current to flow.
00:18:22 So with germanium, that’s about 0.75 electron volts.
00:18:27 That means you have to apply 0.75 volts
00:18:29 to get a current moving.
00:18:32 And it turns out that if you compare that
00:18:34 to the thermal excitations that are induced
00:18:38 just by the temperature of our environment,
00:18:40 that gap’s not quite big enough.
00:18:42 You start to use it to perform computations,
00:18:45 it gets a little hot and you get all these accidental
00:18:47 carriers that are excited into the conduction band
00:18:50 and it causes errors in your computation.
00:18:53 Silicon’s band gap is just a little higher,
00:18:56 1.1 electron volts,
00:18:58 but you have an exponential dependence
00:19:01 on the number of carriers that are present
00:19:04 that can induce those errors.
00:19:06 It decays exponentially with that voltage.
00:19:08 So just that slight extra energy in that band gap
00:19:12 really puts it in an ideal position to be operated
00:19:17 in the conditions of our ambient environment.
00:19:20 It’s kind of fascinating that, like you mentioned,
00:19:22 errors decrease exponentially with the voltage.
00:19:27 So it’s funny because this error thing comes up
00:19:32 when you start talking about quantum computing.
00:19:34 And it’s kind of amazing that everything
00:19:36 we’ve been talking about, the errors,
00:19:37 as we scale down, seems to be extremely low.
00:19:41 Yes.
00:19:42 And like all of our computation is based
00:19:45 on the assumption that it’s extremely low.
00:19:47 Yes, well it’s digital computation.
00:19:49 Digital, sorry, digital computation.
00:19:51 So as opposed to our biological computation in our brain,
00:19:55 is like the assumption is stuff is gonna fail
00:19:58 all over the place and we somehow
00:19:59 have to still be robust to that.
00:20:01 That’s exactly right.
00:20:03 So this also, this is gonna be the most controversial part
00:20:05 of our conversation where you’re gonna make some enemies.
00:20:07 So let me ask,
00:20:09 because we’ve been talking about physics and engineering.
00:20:14 Which group of people is smarter
00:20:15 and more important for this one?
00:20:17 Let me ask the question in a better way.
00:20:20 Some of the big innovations,
00:20:22 some of the beautiful things that we’ve been talking about,
00:20:25 how much of it is physics?
00:20:26 How much of it is engineering?
00:20:28 My dad is a physicist and he talks down
00:20:31 to all the amazing engineering that we’re doing
00:20:34 in the artificial intelligence and the computer science
00:20:37 and the robotics and all that space.
00:20:39 So we argue about this all the time.
00:20:41 So what do you think?
00:20:42 Who gets more credit?
00:20:43 I’m genuinely not trying to just be politically correct here.
00:20:46 I don’t see how you would have any of the,
00:20:50 what we consider sort of the great accomplishments
00:20:52 of society without both.
00:20:54 You absolutely need both of those things.
00:20:55 Physics tends to play a key role earlier in the development
00:20:59 and then engineering optimization, these things take over.
00:21:04 And I mean, the invention of the transistor
00:21:09 or actually even before that,
00:21:10 the understanding of semiconductor physics
00:21:12 that allowed the invention of the transistor,
00:21:14 that’s all physics.
00:21:15 So if you didn’t have that physics,
00:21:17 you don’t even get to get on the field.
00:21:20 But once you have understood and demonstrated
00:21:24 that this is in principle possible,
00:21:26 more so as engineering.
00:21:28 Why we have computers more powerful
00:21:32 than old supercomputers in each of our phones,
00:21:36 that’s all engineering.
00:21:37 And I think I would be quite foolish to say
00:21:41 that that’s not valuable, that’s not a great contribution.
00:21:46 It’s a beautiful dance.
00:21:47 Would you put like Silicon,
00:21:49 the understanding of the material properties
00:21:52 in the space of engineering?
00:21:54 Like how does that whole process work?
00:21:55 To understand that it has all these nice properties
00:21:58 or even the development of photolithography,
00:22:02 is that basically,
00:22:03 would you put that in a category of engineering?
00:22:06 No, I would say that it is basic physics,
00:22:09 it is applied physics, it’s material science,
00:22:12 it’s X ray crystallography, it’s polymer chemistry,
00:22:17 it’s everything.
00:22:18 Chemistry even is thrown in there?
00:22:20 Absolutely, yes, absolutely.
00:22:22 Just no biology.
00:22:25 We can get to biology.
00:22:26 Or the biologies and the humans
00:22:28 that are engineering the system,
00:22:29 so it’s all integrated deeply.
00:22:31 Okay, so let’s return,
00:22:32 you mentioned this word superconductivity.
00:22:35 So what does that have to do with what we’re talking about?
00:22:38 Right, okay, so in a semiconductor,
00:22:40 as I tried to describe a second ago,
00:22:44 you can sort of induce currents by applying voltages
00:22:50 and those have sort of typical properties
00:22:52 that you would expect from some kind of a conductor.
00:22:55 Those electrons, they don’t just flow
00:22:59 perfectly without dissipation.
00:23:00 If an electron collides with an imperfection in the lattice
00:23:03 or another electron, it’s gonna slow down,
00:23:05 it’s gonna lose its momentum.
00:23:06 So you have to keep applying that voltage
00:23:09 in order to keep the current flowing.
00:23:11 In a superconductor, something different happens.
00:23:13 If you get a current to start flowing,
00:23:16 it will continue to flow indefinitely.
00:23:18 There’s no dissipation.
00:23:19 So that’s crazy.
00:23:21 How does that happen?
00:23:22 Well, it happens at low temperature and this is crucial.
00:23:26 It has to be a quite low temperature
00:23:30 and what I’m talking about there,
00:23:32 for essentially all of our conversation,
00:23:35 I’m gonna be talking about conventional superconductors,
00:23:39 sometimes called low TC superconductors,
00:23:42 low critical temperature superconductors.
00:23:45 And so those materials have to be at a temperature around,
00:23:50 say around four Kelvin.
00:23:51 I mean, their critical temperature might be 10 Kelvin,
00:23:54 something like that,
00:23:55 but you wanna operate them at around four Kelvin,
00:23:57 four degrees above absolute zero.
00:23:59 And what happens at that temperature,
00:24:01 at very low temperatures in certain materials
00:24:03 is that the noise of atoms moving around,
00:24:10 the lattice vibrating, electrons colliding with each other,
00:24:13 that becomes sufficiently low
00:24:15 that the electrons can settle into this very special state.
00:24:18 It’s sometimes referred to as a macroscopic quantum state
00:24:22 because if I had a piece of superconducting material here,
00:24:26 let’s say niobium is a very typical superconductor.
00:24:30 If I had a block of niobium here
00:24:33 and we cooled it below its critical temperature,
00:24:36 all of the electrons in that superconducting state
00:24:40 would be in one coherent quantum state.
00:24:42 The wave function of that state is described
00:24:47 in terms of all of the particles simultaneously,
00:24:49 but it extends across macroscopic dimensions,
00:24:52 the size of whatever block of that material
00:24:56 I have sitting here.
00:24:57 And the way this occurs is that,
00:25:01 let’s try to be a little bit light on the technical details,
00:25:03 but essentially the electrons coordinate with each other.
00:25:06 They are able to, in this macroscopic quantum state,
00:25:10 they’re able to sort of,
00:25:12 one can quickly take the place of the other.
00:25:14 You can’t tell electrons apart.
00:25:15 They’re what’s known as identical particles.
00:25:17 So if this electron runs into a defect
00:25:22 that would otherwise cause it to scatter,
00:25:25 it can just sort of almost miraculously avoid that defect
00:25:30 because it’s not really in that location.
00:25:32 It’s part of a macroscopic quantum state
00:25:34 and the entire quantum state
00:25:35 was not scattered by that defect.
00:25:37 So you can get a current that flows without dissipation
00:25:40 and that’s called a supercurrent.
00:25:42 That’s sort of just very much scratching the surface
00:25:47 of superconductivity.
00:25:49 There’s very deep and rich physics there,
00:25:52 just probably not the main subject
00:25:54 we need to go into right now.
00:25:55 But it turns out that when you have this material,
00:26:00 you can do usual things like make wires out of it
00:26:03 so you can get current to flow in a straight line on a chip,
00:26:06 but you can also make other devices
00:26:08 that perform different kinds of operations.
00:26:11 Some of them are kind of logic operations
00:26:14 like you’d get in a transistor.
00:26:16 The most common or the most,
00:26:21 I would say, diverse in its utility component
00:26:25 is a Josephson junction.
00:26:26 It’s not analogous to a transistor
00:26:28 in the sense that if you apply a voltage here,
00:26:31 it changes how much current flows from left to right,
00:26:33 but it is analogous in sort of a sense
00:26:36 of it’s the go to component
00:26:39 that a circuit engineer is going to use
00:26:42 to start to build up more complexity.
00:26:44 So these junctions serve as gates.
00:26:48 They can serve as gates.
00:26:50 So I’m not sure how concerned to be with semantics,
00:26:55 but let me just briefly say what a Josephson junction is
00:26:58 and we can talk about different ways that they can be used.
00:27:02 Basically, if you have a superconducting wire
00:27:05 and then a small gap of a different material
00:27:09 that’s not superconducting, an insulator or normal metal,
00:27:13 and then another superconducting wire on the other side,
00:27:15 that’s a Josephson junction.
00:27:17 So it’s sometimes referred to
00:27:18 as a superconducting weak link.
00:27:20 So you have this superconducting state on one side
00:27:24 and on the other side, and the superconducting wave function
00:27:27 actually tunnels across that gap.
00:27:30 And when you create such a physical entity,
00:27:35 it has very unusual current voltage characteristics.
00:27:41 In that gap, like weird stuff happens.
00:27:44 Through the entire circuit.
00:27:45 So you can imagine, suppose you had a loop set up
00:27:47 that had one of those weak links in the loop.
00:27:51 Current would flow in that loop independent,
00:27:53 even if you hadn’t applied a voltage to it,
00:27:55 and that’s called the Josephson effect.
00:27:57 So the fact that there’s this phase difference
00:28:00 in the quantum wave function from one side
00:28:02 of the tunneling barrier to the other
00:28:04 induces current to flow.
00:28:05 So how does you change state?
00:28:07 Right, exactly.
00:28:08 So how do you change state?
00:28:09 Now picture if I have a current bias coming down
00:28:13 this line of my circuit and there’s a Josephson junction
00:28:16 right in the middle of it.
00:28:18 And now I make another wire
00:28:19 that goes around the Josephson junction.
00:28:21 So I have a loop here, a superconducting loop.
00:28:24 I can add current to that loop by exceeding
00:28:28 the critical current of that Josephson junction.
00:28:30 So like any superconducting material,
00:28:34 it can carry this supercurrent that I’ve described,
00:28:37 this current that can propagate without dissipation
00:28:40 up to a certain level.
00:28:41 And if you try and pass more current than that
00:28:44 through the material, it’s going to become
00:28:47 a resistive material, normal material.
00:28:51 So in the Josephson junction, the same thing happens.
00:28:54 I can bias it above its critical current.
00:28:57 And then what it’s going to do,
00:28:58 it’s going to add a quantized amount of current
00:29:03 into that loop.
00:29:04 And what I mean by quantized is it’s going to come
00:29:07 in discrete packets with a well defined value of current.
00:29:11 So in the vernacular of some people working
00:29:15 in this community, you would say you pop a flux on
00:29:19 into the loop.
00:29:20 So a flux on.
00:29:21 You pop a flux on into the loop.
00:29:23 Yeah, so a flux on.
00:29:24 Sounds like skateboarder talk, I love it.
00:29:26 Okay, sorry, go ahead.
00:29:28 A flux on is one of these quantized sort of amounts
00:29:33 of current that you can add to a loop.
00:29:35 And this is a cartoon picture,
00:29:36 but I think it’s sufficient for our purposes.
00:29:38 So which, maybe it’s useful to say,
00:29:41 what is the speed at which these discrete packets
00:29:45 of current travel?
00:29:47 Because we’ll be talking about light a little bit.
00:29:49 It seems like the speed is important.
00:29:51 The speed is important, that’s an excellent question.
00:29:53 Sometimes I wonder where you, how you became so astute.
00:29:57 But so this.
00:30:00 Matrix 4 is coming out, so maybe that’s related.
00:30:04 I’m not sure.
00:30:05 I’m dressed for the job.
00:30:06 I was trying to get to become an extra on Matrix 4,
00:30:09 didn’t work out.
00:30:10 Anyway, so what’s the speed of these packets?
00:30:13 You’ll have to find another gig.
00:30:15 I know, I’m sorry.
00:30:16 So the speed of the pack is actually these flux ons,
00:30:19 these sort of pulses of current
00:30:24 that are generated by Joseph’s injunctions,
00:30:26 they can actually propagate very close
00:30:28 to the speed of light,
00:30:29 maybe something like a third of the speed of light.
00:30:31 That’s quite fast.
00:30:32 So one of the reasons why Joseph’s injunctions are appealing
00:30:37 is because their signals can propagate quite fast
00:30:40 and they can also switch very fast.
00:30:43 What I mean by switch is perform that operation
00:30:46 that I described where you add current to the loop.
00:30:49 That can happen within a few tens of picoseconds.
00:30:53 So you can get devices that operate
00:30:56 in the hundreds of gigahertz range.
00:30:58 And by comparison, most processors
00:31:02 in our conventional computers operate closer
00:31:04 to the one gigahertz range, maybe three gigahertz
00:31:08 seems to be kind of where those speeds have leveled out.
00:31:12 The gamers listening to this are getting really excited
00:31:15 to overclock their system to like, what is it?
00:31:18 Like four gigahertz or something,
00:31:19 a hundred sounds incredible.
00:31:21 Can I just as a tiny tangent,
00:31:24 is the physics of this understood well
00:31:26 how to do this stably?
00:31:28 Oh yes, the physics is understood well.
00:31:30 The physics of Joseph’s injunctions is understood well.
00:31:32 The technology is understood quite well too.
00:31:34 The reasons why it hasn’t displaced
00:31:37 silicon microelectronics in conventional digital computing
00:31:41 I think are more related to what I was alluding to before
00:31:45 about the myriad practical, almost mundane aspects
00:31:49 of silicon that make it so useful.
00:31:52 You can make a transistor ever smaller and smaller
00:31:55 and it will still perform its digital function quite well.
00:31:58 The same is not true of a Joseph’s injunction.
00:32:00 You really, they don’t, they just,
00:32:02 it’s not the same thing that there’s this feature
00:32:04 that you can keep making smaller and smaller
00:32:06 and it’ll keep performing the same operations.
00:32:08 This loop I described, any Joseph’s in circuit,
00:32:11 well, I wanna be careful, I shouldn’t say
00:32:13 any Joseph’s in circuit, but many Joseph’s in circuits,
00:32:17 the way they process information
00:32:19 or the way they perform whatever function it is
00:32:21 they’re trying to do,
00:32:22 maybe it’s sensing a weak magnetic field,
00:32:24 it depends on an interplay between the junction
00:32:27 and that loop.
00:32:28 And you can’t make that loop much smaller.
00:32:30 And it’s not for practical reasons
00:32:32 that have to do with lithography.
00:32:33 It’s for fundamental physical reasons
00:32:35 about the way the magnetic field interacts
00:32:38 with that superconducting material.
00:32:41 There are physical limits that no matter how good
00:32:44 our technology got, those circuits would,
00:32:47 I think would never be able to be scaled down
00:32:50 to the densities that silicon microelectronics can.
00:32:54 I don’t know if we mentioned,
00:32:55 is there something interesting
00:32:56 about the various superconducting materials involved
00:33:00 or is it all?
00:33:01 There’s a lot of stuff that’s interesting.
00:33:02 And it’s not silicon.
00:33:04 It’s not silicon, no.
00:33:05 So like it’s some materials that also required
00:33:09 to be super cold, four Kelvin and so on.
00:33:12 So let’s dissect a couple of those different things.
00:33:15 The super cold part,
00:33:16 let me just mention for your gamers out there
00:33:19 that are trying to clock it at four gigahertz
00:33:21 and would love to go to 400.
00:33:22 What kind of cooling system can achieve four Kelvin?
00:33:24 Four Kelvin, you need liquid helium.
00:33:26 And so liquid helium is expensive.
00:33:29 It’s inconvenient.
00:33:29 You need a cryostat that sits there
00:33:32 and the energy consumption of that cryostat
00:33:36 is impracticable for, it’s not going in your cell phone.
00:33:40 So you can picture holding your cell phone like this
00:33:42 and then something the size of a keg of beer or something
00:33:46 on your back to cool it.
00:33:47 Like that makes no sense.
00:33:49 So if you’re trying to make this in consumer devices,
00:33:54 electronics that are ubiquitous across society,
00:33:57 superconductors are not in the race for that.
00:33:59 For now, but you’re saying,
00:34:01 so just to frame the conversation,
00:34:03 maybe the thing we’re focused on
00:34:05 is computing systems that serve as servers, like large.
00:34:10 Yes, large systems.
00:34:11 So then you can contrast what’s going on in your cell phone
00:34:14 with what’s going on at one of the supercomputers.
00:34:19 Colleague Katie Schuman invited us out to Oak Ridge
00:34:22 a few years ago, so we got to see Titan
00:34:24 and that was when they were building Summit.
00:34:26 So these are some high performance supercomputers
00:34:29 out in Tennessee and those are filling entire rooms
00:34:32 the size of warehouses.
00:34:33 So once you’re at that level, okay,
00:34:36 there you’re already putting a lot of power into cooling.
00:34:39 Cooling is part of your engineering task
00:34:42 that you have to deal with.
00:34:43 So there it’s not entirely obvious
00:34:45 that cooling to four Kelvin is out of the question.
00:34:49 It has not happened yet and I can speak to why that is
00:34:53 in the digital domain if you’re interested.
00:34:55 I think it’s not going to happen.
00:34:57 I don’t think superconductors are gonna replace
00:35:01 semiconductors for digital computation.
00:35:05 There are a lot of reasons for that,
00:35:07 but I think ultimately what it comes down to
00:35:09 is all things considered cooling errors,
00:35:13 scaling down to feature sizes, all that stuff,
00:35:16 semiconductors work better at the system level.
00:35:19 Is there some aspect of just curious
00:35:22 about the historical momentum of this?
00:35:25 Is there some power to the momentum of an industry
00:35:28 that’s mass manufacturing using a certain material?
00:35:31 Is this like a Titanic shifting?
00:35:33 Like what’s your sense when a good idea comes along,
00:35:37 how good does that idea need to be
00:35:39 for the Titanic to start shifting?
00:35:42 That’s an excellent question.
00:35:44 That’s an excellent way to frame it.
00:35:46 And you know, I don’t know the answer to that,
00:35:51 but what I think is, okay,
00:35:53 so the history of the superconducting logic
00:35:56 goes back to the 70s.
00:35:58 IBM made a big push to do
00:35:59 superconducting digital computing in the 70s.
00:36:02 And they made some choices about their devices
00:36:06 and their architectures and things that in hindsight,
00:36:09 were kind of doomed to fail.
00:36:11 And I don’t mean any disrespect for the people that did it,
00:36:13 it was hard to see at the time.
00:36:14 But then another generation of superconducting logic
00:36:17 was introduced, I wanna say the 90s,
00:36:22 someone named Lykarev and Seminov,
00:36:25 they proposed an entire family of circuits
00:36:28 based on Joseph’s injunctions
00:36:29 that are doing digital computing based on logic gates
00:36:33 and or not these kinds of things.
00:36:37 And they showed how it could go hundreds of times faster
00:36:41 than silicon microelectronics.
00:36:43 And it’s extremely exciting.
00:36:45 I wasn’t working in the field at that time,
00:36:47 but later when I went back and read the literature,
00:36:49 I was just like, wow, this is so awesome.
00:36:53 And so you might think, well,
00:36:56 the reason why it didn’t display silicon
00:36:58 is because silicon already had so much momentum
00:37:00 at that time.
00:37:01 But that was the 90s.
00:37:02 Silicon kept that momentum
00:37:04 because it had the simple way to keep getting better.
00:37:06 You just make features smaller and smaller.
00:37:08 So it would have to be,
00:37:11 I don’t think it would have to be that much better
00:37:13 than silicon to displace it.
00:37:15 But the problem is it’s just not better than silicon.
00:37:17 It might be better than silicon in one metric,
00:37:19 speed of a switching operation
00:37:21 or power consumption of a switching operation.
00:37:24 But building a digital computer is a lot more
00:37:26 than just that elemental operation.
00:37:28 It’s everything that goes into it,
00:37:31 including the manufacturing, including the packaging,
00:37:33 including the various materials aspects of things.
00:37:38 So the reason why,
00:37:40 and even in some of those early papers,
00:37:42 I can’t remember which one it was,
00:37:44 Lykarev said something along the lines of,
00:37:47 you can see how we could build an entire family
00:37:49 of digital electronic circuits based on these components.
00:37:52 They could go a hundred or more times faster
00:37:55 than semiconductor logic gates.
00:37:59 But I don’t think that’s the right way
00:38:00 to use superconducting electronic circuits.
00:38:02 He didn’t say what the right way was,
00:38:04 but he basically said digital logic,
00:38:07 trying to steal the show from silicon
00:38:11 is probably not what these circuits
00:38:13 are most suited to accomplish.
00:38:16 So if we can just linger and use the word computation.
00:38:20 When you talk about computation, how do you think about it?
00:38:24 Do you think purely on just the switching,
00:38:28 or do you think something a little bit larger scale,
00:38:31 a circuit taken together,
00:38:32 performing the basic arithmetic operations
00:38:36 that are then required to do the kind of computation
00:38:40 that makes up a computer?
00:38:42 Because when we talk about the speed of computation,
00:38:44 is it boiled down to the basic switching,
00:38:46 or is there some bigger picture
00:38:48 that you’re thinking about?
00:38:49 Well, all right, so maybe we should disambiguate.
00:38:52 There are a variety of different kinds of computation.
00:38:55 I don’t pretend to be an expert
00:38:57 in the theory of computation or anything like that.
00:39:00 I guess it’s important to differentiate though
00:39:02 between digital logic,
00:39:05 which represents information as a series of bits,
00:39:09 binary digits, which you can think of them
00:39:13 as zeros and ones or whatever.
00:39:14 Usually they correspond to a physical system
00:39:17 that has two very well separated states.
00:39:21 And then other kinds of computation,
00:39:22 like we’ll get into more the way your brain works,
00:39:25 which it is, I think,
00:39:27 indisputably processing information,
00:39:30 but where the computation begins and ends
00:39:34 is not anywhere near as well defined.
00:39:36 It doesn’t depend on these two levels.
00:39:39 Here’s a zero, here’s a one.
00:39:41 There’s a lot of gray area
00:39:42 that’s usually referred to as analog computing.
00:39:45 Also in conventional digital computers
00:39:49 or digital computers in general,
00:39:54 you have a concept of what’s called arithmetic depth,
00:39:57 which is jargon that basically means
00:39:59 how many sequential operations are performed
00:40:03 to turn an input into an output.
00:40:07 And those kinds of computations in digital systems
00:40:10 are highly serial, meaning that data streams,
00:40:14 they don’t branch off too far to the side.
00:40:16 You do, you have to pull some information over there
00:40:18 and access memory from here and stuff like that.
00:40:20 But by and large, the computation proceeds
00:40:24 in a serial manner.
00:40:26 It’s not that way in the brain.
00:40:27 In the brain, you’re always drawing information
00:40:30 from different places.
00:40:31 It’s much more network based computing.
00:40:33 Neurons don’t wait for their turn.
00:40:35 They fire when they’re ready to fire.
00:40:37 And so it’s asynchronous.
00:40:39 So one of the other things about a digital system
00:40:41 is you’re performing these operations on a clock.
00:40:44 And that’s a crucial aspect of it.
00:40:46 Get rid of a clock in a digital system,
00:40:48 nothing makes sense anymore.
00:40:50 The brain has no clock.
00:40:51 It builds its own timescales based on its internal activity.
00:40:56 So you can think of the brain as kind of like this,
00:40:59 like network computation,
00:41:00 where it’s actually really trivial, simple computers,
00:41:05 just a huge number of them and they’re networked.
00:41:08 I would say it is complex, sophisticated little processors
00:41:12 and there’s a huge number of them.
00:41:14 Neurons are not, are not simple.
00:41:16 I don’t mean to offend neurons.
00:41:17 They’re very complicated and beautiful and yeah,
00:41:19 but we often oversimplify them.
00:41:21 Yes, they’re actually like there’s computation happening
00:41:24 within a neuron.
00:41:25 Right, so I would say to think of a transistor
00:41:29 as the building block of a digital computer is accurate.
00:41:32 You use a few transistors to make your logic gates.
00:41:34 You build up more, you build up processors
00:41:37 from logic gates and things like that.
00:41:39 So you can think of a transistor
00:41:40 as a fundamental building block,
00:41:42 or you can think of,
00:41:43 as we get into more highly parallelized architectures,
00:41:46 you can think of a processor
00:41:47 as a fundamental building block.
00:41:49 To make the analogy to the neuro side of things,
00:41:53 a neuron is not a transistor.
00:41:55 A neuron is a processor.
00:41:57 It has synapses, even synapses are not transistors,
00:42:00 but they are more,
00:42:02 they’re lower on the information processing hierarchy
00:42:04 in a sense.
00:42:05 They do a bulk of the computation,
00:42:08 but neurons are entire processors in and of themselves
00:42:13 that can take in many different kinds of inputs
00:42:16 on many different spatial and temporal scales
00:42:18 and produce many different kinds of outputs
00:42:20 so that they can perform different computations
00:42:23 in different contexts.
00:42:24 So this is where enters this distinction
00:42:27 between computation and communication.
00:42:30 So you can think of neurons performing computation
00:42:34 and the inter, the networking,
00:42:36 the interconnectivity of neurons
00:42:39 is communication between neurons.
00:42:40 And you see this with very large server systems.
00:42:43 I’ve been, I mentioned offline,
00:42:45 we’ve been talking to Jim Keller,
00:42:46 whose dream is to build giant computers
00:42:48 that, you know, the bottom like there
00:42:51 is often the communication
00:42:52 between the different pieces of computing.
00:42:54 So in this paper that we mentioned,
00:42:57 Optoelectronic Intelligence,
00:42:59 you say electrons excel at computation
00:43:03 while light is excellent for communication.
00:43:08 Maybe you can linger and say in this context,
00:43:11 what do you mean by computation and communication?
00:43:13 What are electrons, what is light
00:43:17 and why do they excel at those two tasks?
00:43:20 Yeah, just to first speak to computation
00:43:23 versus communication,
00:43:25 I would say computation is essentially taking in
00:43:30 some information, performing operations
00:43:33 on that information and producing new,
00:43:37 hopefully more useful information.
00:43:39 So for example, imagine you have a picture in front of you
00:43:45 and there is a key in it
00:43:48 and that’s what you’re looking for,
00:43:48 for whatever reason, you wanna find the key,
00:43:50 we all wanna find the key.
00:43:51 So the input is that entire picture
00:43:56 and the output might be the coordinates where the key is.
00:43:59 So you’ve reduced the total amount of information you have
00:44:01 but you found the useful information
00:44:03 for you in that present moment,
00:44:04 that’s the useful information.
00:44:05 And you think about this computation
00:44:07 as the controlled synchronous sequential?
00:44:10 Not necessarily, it could be,
00:44:12 that could be how your system is performing the computation
00:44:16 or it could be asynchronous,
00:44:19 there are lots of ways to find the key.
00:44:21 It depends on the nature of the data,
00:44:23 it depends on, that’s a very simplified example,
00:44:27 a picture with a key in it,
00:44:28 what about if you’re in the world
00:44:30 and you’re trying to decide the best way
00:44:32 to live your life?
00:44:35 It might be interactive,
00:44:37 it might be there might be some recurrence
00:44:38 or some weird asynchrony, I got it.
00:44:41 But there’s an input and there’s an output
00:44:43 and you do some stuff in the middle
00:44:44 that actually goes from the input to the output.
00:44:46 You’ve taken in information
00:44:47 and output different information,
00:44:49 hopefully reducing the total amount of information
00:44:51 and extracting what’s useful.
00:44:53 Communication is then getting that information
00:44:57 from the location at which it’s stored
00:44:59 because information is physical as Landauer emphasized
00:45:02 and so it is in one place
00:45:04 and you need to get that information to another place
00:45:07 so that something else can use it
00:45:10 for whatever computation it’s working on.
00:45:12 Maybe it’s part of the same network
00:45:13 and you’re all trying to solve the same problem
00:45:15 but neuron A over here just deduced something
00:45:20 based on its inputs
00:45:21 and it’s now sending that information across the network
00:45:25 to another location
00:45:26 so that would be the act of communication.
00:45:28 Can you linger on Landauer
00:45:29 and saying information is physical?
00:45:31 Rolf Landauer, not to be confused with Lev Landauer.
00:45:35 Yeah, and he made huge contributions
00:45:38 to our understanding of the reversibility of information
00:45:42 and this concept that energy has to be dissipated
00:45:46 in computing when the computation is irreversible
00:45:50 but if you can manage to make it reversible
00:45:52 then you don’t need to expend energy
00:45:55 but if you do expend energy to perform a computation
00:45:59 there’s sort of a minimal amount that you have to do
00:46:02 and it’s KT log two.
00:46:04 And it’s all somehow related
00:46:05 to the second law of thermodynamics
00:46:07 and that the universe is an information process
00:46:09 and then we’re living in a simulation.
00:46:11 So okay, sorry, sorry for that tangent.
00:46:13 So that’s the defining the distinction
00:46:17 between computation and communication.
00:46:19 Let me say one more thing just to clarify.
00:46:21 Communication ideally does not change the information.
00:46:27 It moves it from one place to another
00:46:28 but it is preserved.
00:46:30 Got it, okay.
00:46:32 All right, that’s beautiful.
00:46:33 So then the electron versus light distinction
00:46:38 and why are electrons good at computation
00:46:42 and light good at communication?
00:46:44 Yes, there’s a lot that goes into it I guess
00:46:48 but just try to speak to the simplest part of it.
00:46:54 Electrons interact strongly with one another.
00:46:56 They’re charged particles.
00:46:58 So if I pile a bunch of them over here
00:47:02 they’re feeling a certain amount of force
00:47:03 and they wanna move somewhere else.
00:47:05 They’re strongly interactive.
00:47:06 You can also get them to sit still.
00:47:08 You can, an electron has a mass
00:47:10 so you can cause it to be spatially localized.
00:47:15 So for computation that’s useful
00:47:18 because now I can make these little devices
00:47:20 that put a bunch of electrons over here
00:47:21 and then I change the state of a gate
00:47:25 like I’ve been describing,
00:47:26 put a different voltage on this gate
00:47:28 and now I move the electrons over here.
00:47:29 Now they’re sitting somewhere else.
00:47:31 I have a physical mechanism
00:47:33 with which I can represent information.
00:47:36 It’s spatially localized and I have knobs
00:47:38 that I can adjust to change where those electrons are
00:47:41 or what they’re doing.
00:47:42 Light by contrast, photons of light
00:47:45 which are the discrete packets of energy
00:47:48 that were identified by Einstein,
00:47:50 they do not interact with each other
00:47:54 especially at low light levels.
00:47:56 If you’re in a medium and you have a bright high light level
00:48:00 you can get them to interact with each other
00:48:02 through the interaction with that medium that they’re in
00:48:05 but that’s a little bit more exotic.
00:48:07 And for the purposes of this conversation
00:48:10 we can assume that photons don’t interact with each other.
00:48:13 So if you have a bunch of them
00:48:16 all propagating in the same direction
00:48:17 they don’t interfere with each other.
00:48:19 If I wanna send, if I have a communication channel
00:48:22 and I put one more photon on it,
00:48:24 it doesn’t screw up with those other ones.
00:48:26 It doesn’t change what those other ones were doing at all.
00:48:29 So that’s really useful for communication
00:48:31 because that means you can sort of allow
00:48:33 a lot of these photons to flow
00:48:37 without disruption of each other
00:48:38 and they can branch really easily and things like that.
00:48:41 But it’s not good for computation
00:48:42 because it’s very hard for this packet of light
00:48:46 to change what this packet of light is doing.
00:48:48 They pass right through each other.
00:48:50 So in computation you want to change information
00:48:53 and if photons don’t interact with each other
00:48:55 it’s difficult to get them to change the information
00:48:58 represented by the others.
00:48:59 So that’s the fundamental difference.
00:49:01 Is there also something about the way they travel
00:49:04 through different materials
00:49:07 or is that just a particular engineering?
00:49:10 No, it’s not, that’s deep physics I think.
00:49:12 So this gets back to electrons interact with each other
00:49:17 and photons don’t.
00:49:18 So say I’m trying to get a packet of information
00:49:22 from me to you and we have a wire going between us.
00:49:25 In order for me to send electrons across that wire
00:49:29 I first have to raise the voltage on my end of the wire
00:49:32 and that means putting a bunch of charges on it
00:49:34 and then that charge packet has to propagate along the wire
00:49:39 and it has to get all the way over to you.
00:49:41 That wire is gonna have something that’s called capacitance
00:49:44 which basically tells you how much charge
00:49:46 you need to put on the wire
00:49:48 in order to raise the voltage on it
00:49:49 and the capacitance is gonna be proportional
00:49:52 to the length of the wire.
00:49:54 So the longer the length of the wire is
00:49:56 the more charge I have to put on it
00:49:59 and the energy required to charge up that line
00:50:03 and move those electrons to you
00:50:04 is also proportional to the capacitance
00:50:06 and goes as the voltage squared.
00:50:08 So you get this huge penalty if you wanna send electrons
00:50:13 across a wire over appreciable distances.
00:50:16 So distance is an important thing here
00:50:19 when you’re doing communication.
00:50:20 Distance is an important thing.
00:50:22 So is the number of connections I’m trying to make.
00:50:25 Me to you, okay one, that’s not so bad.
00:50:27 If I want to now send it to 10,000 other friends
00:50:31 then all of those wires are adding tons
00:50:34 of extra capacitance.
00:50:35 Now not only does it take forever
00:50:37 to put the charge on that wire
00:50:39 and raise the voltage on all those lines
00:50:41 but it takes a ton of power
00:50:43 and the number 10,000 is not randomly chosen.
00:50:46 That’s roughly how many connections
00:50:49 each neuron in your brain makes.
00:50:50 So a neuron in your brain needs to send 10,000 messages
00:50:55 every time it has something to say.
00:50:56 You can’t do that if you’re trying to drive electrons
00:51:00 from here to 10,000 different places.
00:51:02 The brain does it in a slightly different way
00:51:03 which we can discuss.
00:51:04 How can light achieve the 10,000 connections
00:51:07 and why is it better?
00:51:09 In terms of like the energy use required
00:51:12 to use light for the communication of the 10,000 connections.
00:51:15 Right, right.
00:51:16 So now instead of trying to send electrons
00:51:17 from me to you, I’m trying to send photons.
00:51:19 So I can make what’s called a wave guide
00:51:21 which is just a simple piece of a material.
00:51:25 It could be glass like an optical fiber
00:51:27 or silicon on a chip.
00:51:29 And I just have to inject photons into that wave guide
00:51:34 and independent of how long it is,
00:51:35 independent of how many different connections I’m making,
00:51:39 it doesn’t change the voltage or anything like that
00:51:43 that I have to raise up on the wire.
00:51:45 So if I have one more connection,
00:51:47 if I add additional connections,
00:51:49 I need to add more light to the wave guide
00:51:51 because those photons need to split
00:51:53 and go to different paths.
00:51:55 That makes sense but I don’t have a capacitive penalty.
00:51:58 Sometimes these are called wiring parasitics.
00:52:01 There are no parasitics associated with light
00:52:03 in that same sense.
00:52:04 So this might be a dumb question
00:52:07 but how do I catch a photon on the other end?
00:52:11 Is it material?
00:52:12 Is it the polymer stuff you were talking about
00:52:15 for a different application for photolithography?
00:52:19 Like how do you catch a photon?
00:52:20 There’s a lot of ways to catch a photon.
00:52:22 It’s not a dumb question.
00:52:23 It’s a deep and important question
00:52:25 that basically defines a lot of the work
00:52:29 that goes on in our group at NIST.
00:52:31 One of my group leaders, Seywoon Nam,
00:52:34 has built his career around
00:52:35 these superconducting single photon detectors.
00:52:38 So if you’re going to try to sort of reach a lower limit
00:52:42 and detect just one particle of light,
00:52:45 superconductors come back into our conversation
00:52:47 and just picture a simple device
00:52:50 where you have current flowing
00:52:51 through a superconducting wire and…
00:52:54 A loop again or no?
00:52:56 Let’s say yes, you have a loop.
00:52:57 So you have a superconducting wire
00:52:59 that goes straight down like this
00:53:01 and on your loop branch, you have a little ammeter,
00:53:04 something that measures current.
00:53:05 There’s a resistor up there too.
00:53:07 Go with me here.
00:53:09 So your current biasing this,
00:53:12 so there’s current flowing
00:53:13 through that superconducting branch.
00:53:14 Since there’s a resistor over here,
00:53:16 all the current goes through the superconducting branch.
00:53:18 Now a photon comes in, strikes that superconductor.
00:53:22 We talked about this superconducting
00:53:24 macroscopic quantum state.
00:53:25 That’s going to be destroyed by the energy of that photon.
00:53:28 So now that branch of the circuit is resistive too.
00:53:32 And you’ve properly designed your circuit
00:53:33 so that the resistance on that superconducting branch
00:53:36 is much greater than the other resistance.
00:53:38 Now all of your current’s going to go that way.
00:53:40 Your ammeter says, oh, I just got a pulse of current.
00:53:43 That must mean I detected a photon.
00:53:45 Then where you broke that superconductivity
00:53:47 in a matter of a few nanoseconds,
00:53:49 it cools back off, dissipates that energy
00:53:51 and the current flows back
00:53:52 through that superconducting branch.
00:53:54 This is a very powerful superconducting device
00:53:59 that allows us to understand quantum states of light.
00:54:02 I didn’t realize a loop like that
00:54:04 could be sensitive to a single photon.
00:54:07 I mean, that seems strange to me because,
00:54:13 I mean, so what happens when you just barrage it
00:54:15 with photons?
00:54:16 If you put a bunch of photons in there,
00:54:18 essentially the same thing happens.
00:54:19 You just drive it into the normal state,
00:54:21 it becomes resistive and it’s not particularly interesting.
00:54:25 So you have to be careful how many photons you send.
00:54:27 Like you have to be very precise with your communication.
00:54:30 Well, it depends.
00:54:31 So I would say that that’s actually in the application
00:54:34 that we’re trying to use these detectors for.
00:54:37 That’s a feature because what we want is for,
00:54:41 if a neuron sends one photon to a synaptic connection
00:54:46 and one of these superconducting detectors is sitting there,
00:54:49 you get this pulse of current.
00:54:51 And that synapse says event,
00:54:54 then I’m gonna do what I do when there’s a synapse event,
00:54:56 I’m gonna perform computations, that kind of thing.
00:54:58 But if accidentally you send two there or three or five,
00:55:02 it does the exact same.
00:55:03 Got it.
00:55:04 And so this is how in the system that we’re devising here,
00:55:10 communication is entirely binary.
00:55:12 And that’s what I tried to emphasize a second ago.
00:55:15 Communication should not change the information.
00:55:17 You’re not saying, oh, I got this kind of communication
00:55:21 event for photons.
00:55:22 No, we’re not keeping track of that.
00:55:23 This neuron fired, this synapse says that neuron fired,
00:55:26 that’s it.
00:55:27 So that’s a noise filtering property of those detectors.
00:55:31 However, there are other applications
00:55:33 where you’d rather know the exact number of photons
00:55:36 that can be very useful in quantum computing with light.
00:55:39 And our group does a lot of work
00:55:41 around another kind of superconducting sensor
00:55:44 called a transition edge sensor that Adrian Alita
00:55:48 in our group does a lot of work on that.
00:55:49 And that can tell you based on the amplitude
00:55:53 of the current pulse you divert exactly how many photons
00:55:58 were in that pulse.
00:56:00 What’s that useful for?
00:56:02 One way that you can encode information
00:56:04 in quantum states of light is in the number of photons.
00:56:07 You can have what are called number states
00:56:09 and a number state will have a well defined number
00:56:11 of photons and maybe the output of your quantum computation
00:56:16 encodes its information in the number of photons
00:56:19 that are generated.
00:56:20 So if you have a detector that is sensitive to that,
00:56:23 it’s extremely useful.
00:56:24 Can you achieve like a clock with photons
00:56:29 or is that not important?
00:56:30 Is there a synchronicity here?
00:56:33 In general, it can be important.
00:56:36 Clock distribution is a big challenge
00:56:39 in especially large computational systems.
00:56:43 And so yes, optical clocks, optical clock distribution
00:56:47 is a very powerful technology.
00:56:51 I don’t know the state of that field right now,
00:56:53 but I imagine that if you’re trying to distribute a clock
00:56:55 across any appreciable size computational system,
00:56:58 you wanna use light.
00:57:00 Yeah, I wonder how these giant systems work,
00:57:04 especially like supercomputers.
00:57:07 Do they need to do clock distribution
00:57:09 or are they doing more ad hoc parallel
00:57:14 like concurrent programming?
00:57:15 Like there’s some kind of locking mechanisms or something.
00:57:18 That’s a fascinating question,
00:57:19 but let’s zoom in at this very particular question
00:57:23 of computation on a processor
00:57:28 and communication between processors.
00:57:31 So what does this system look like
00:57:36 that you’re envisioning?
00:57:38 One of the places you’re envisioning it
00:57:40 is in the paper on optoelectronic intelligence.
00:57:43 So what are we talking about?
00:57:44 Are we talking about something
00:57:46 that starts to look a lot like the human brain
00:57:48 or does it still look a lot like a computer?
00:57:51 What are the size of this thing?
00:57:52 Is it going inside a smartphone or as you said,
00:57:55 does it go inside something that’s more like a house?
00:57:58 Like what should we be imagining?
00:58:01 What are you thinking about
00:58:02 when you’re thinking about these fundamental systems?
00:58:05 Let me introduce the word neuromorphic.
00:58:07 There’s this concept of neuromorphic computing
00:58:09 where what that broadly refers to
00:58:12 is computing based on the information processing principles
00:58:17 of the brain.
00:58:19 And as digital computing seems to be pushing
00:58:23 towards some fundamental performance limits,
00:58:26 people are considering architectural advances,
00:58:29 drawing inspiration from the brain,
00:58:30 more distributed parallel network kind of architectures
00:58:33 and stuff.
00:58:34 And so there’s this continuum of neuromorphic
00:58:37 from things that are pretty similar to digital computers,
00:58:42 but maybe there are more cores
00:58:45 and the way they send messages is a little bit more
00:58:49 like the way brain neurons send spikes.
00:58:52 But for the most part, it’s still digital electronics.
00:58:56 And then you have some things in between
00:58:58 where maybe you’re using transistors,
00:59:02 but now you’re starting to use them
00:59:03 instead of in a digital way, in an analog way.
00:59:06 And so you’re trying to get those circuits
00:59:08 to behave more like neurons.
00:59:10 And then that’s a little bit,
00:59:12 quite a bit more on the neuromorphic side of things.
00:59:14 You’re trying to get your circuits,
00:59:17 although they’re still based on silicon,
00:59:19 you’re trying to get them to perform operations
00:59:22 that are highly analogous to the operations in the brain.
00:59:24 And that’s where a great deal of work is
00:59:26 in neuromorphic computing,
00:59:27 people like Giacomo Indoveri and Gert Kauenberg,
00:59:30 Jennifer Hasler, countless others.
00:59:32 It’s a rich and exciting field going back to Carver Mead
00:59:36 in the late 1980s.
00:59:39 And then all the way on the other extreme of the continuum
00:59:44 is where you say, I’ll give up anything related
00:59:48 to transistors or semiconductors or anything like that.
00:59:51 I’m not starting with the assumption
00:59:53 that I’m gonna use any kind
00:59:55 of conventional computing hardware.
00:59:57 And instead, what I wanna do is try and understand
00:59:59 what makes the brain powerful
01:00:00 at the kind of information processing it does.
01:00:03 And I wanna think from first principles
01:00:05 about what hardware is best going to enable us
01:00:10 to capture those information processing principles
01:00:14 in an artificial system.
01:00:16 And that’s where I live.
01:00:17 That’s where I’m doing my exploration these days.
01:00:21 So what are the first principles
01:00:25 of brain like computation communication?
01:00:29 Right, yeah, this is so important
01:00:32 and I’m glad we booked 14 hours for this because.
01:00:35 I only have 13, I’m sorry.
01:00:38 Okay, so the brain is notoriously complicated.
01:00:41 And I think that’s an important part
01:00:44 of why it can do what it does.
01:00:46 But okay, let me try to break it down.
01:00:49 Starting with the devices, neurons, as I said before,
01:00:54 they’re sophisticated devices in and of themselves
01:00:57 and synapses are too.
01:00:58 They can change their state based on the activity.
01:01:03 So they adapt over time.
01:01:04 That’s crucial to the way the brain works.
01:01:06 They don’t just adapt on one timescale,
01:01:09 they can adapt on myriad timescales
01:01:12 from the spacing between pulses,
01:01:16 the spacing between spikes that come from neurons
01:01:18 all the way to the age of the organism.
01:01:23 Also relevant, perhaps I think the most important thing
01:01:28 that’s guided my thinking is the network structure
01:01:32 of the brain, so.
01:01:33 Which can also be adjusted on different scales.
01:01:36 Absolutely, yes, so you’re making new,
01:01:39 you’re changing the strength of contacts,
01:01:41 you’re changing the spatial distribution of them,
01:01:44 although spatial distribution doesn’t change that much
01:01:46 once you’re a mature organism.
01:01:49 But that network structure is really crucial.
01:01:52 So let me dwell on that for a second.
01:01:55 You can’t talk about the brain without emphasizing
01:01:58 that most of the neurons in the neocortex
01:02:02 or the prefrontal cortex, the part of the brain
01:02:04 that we think is most responsible for high level reasoning
01:02:08 and things like that,
01:02:09 those neurons make thousands of connections.
01:02:11 So you have this network that is highly interconnected.
01:02:15 And I think it’s safe to say that one of the primary reasons
01:02:19 that they make so many different connections
01:02:23 is that allows information to be communicated very rapidly
01:02:26 from any spot in the network
01:02:28 to any other spot in the network.
01:02:30 So that’s a sort of spatial aspect of it.
01:02:33 You can quantify this in terms of concepts
01:02:38 that are related to fractals and scale invariants,
01:02:41 which I think is a very beautiful concept.
01:02:43 So what I mean by that is kind of,
01:02:46 no matter what spatial scale you’re looking at in the brain
01:02:50 within certain bounds, you see the same
01:02:53 general statistical pattern.
01:02:54 So if I draw a box around some region of my cortex,
01:02:59 most of the connections that those neurons
01:03:02 within that box make are gonna be within the box
01:03:04 to each other in their local neighborhood.
01:03:06 And that’s sort of called clustering, loosely speaking.
01:03:09 But a non negligible fraction
01:03:10 is gonna go outside of that box.
01:03:12 And then if I draw a bigger box,
01:03:14 the pattern is gonna be exactly the same.
01:03:16 So you have this scale invariants,
01:03:18 and you also have a non vanishing probability
01:03:22 of a neuron making connection very far away.
01:03:25 So suppose you wanna plot the probability
01:03:28 of a neuron making a connection as a function of distance.
01:03:32 If that were an exponential function,
01:03:34 it would go e to the minus radius
01:03:36 over some characteristic radius,
01:03:38 and it would drop off up to some certain radius,
01:03:41 the probability would be reasonably close to one,
01:03:44 and then beyond that characteristic length R zero,
01:03:49 it would drop off sharply.
01:03:51 And so that would mean that the neurons in your brain
01:03:53 are really localized, and that’s not what we observe.
01:03:58 Instead, what you see is that the probability
01:04:00 of making a longer distance connection, it does drop off,
01:04:03 but it drops off as a power law.
01:04:05 So the probability that you’re gonna have a connection
01:04:08 at some radius R goes as R to the minus some power.
01:04:13 And that’s more, that’s what we see with forces in nature,
01:04:16 like the electromagnetic force
01:04:18 between two particles or gravity
01:04:20 goes as one over the radius squared.
01:04:23 So you can see this in fractals.
01:04:24 I love that there’s like a fractal dynamics of the brain
01:04:28 that if you zoom out, you draw the box
01:04:31 and you increase that box by certain step sizes,
01:04:35 you’re gonna see the same statistics.
01:04:36 I think that’s probably very important
01:04:40 to the way the brain processes information.
01:04:41 It’s not just in the spatial domain,
01:04:43 it’s also in the temporal domain.
01:04:45 And what I mean by that is…
01:04:48 That’s incredible that this emerged
01:04:50 through the evolutionary process
01:04:52 that potentially somehow connected
01:04:54 to the way the physics of the universe works.
01:04:57 Yeah, I couldn’t agree more that it’s a deep
01:05:00 and fascinating subject that I hope to be able
01:05:02 to spend the rest of my life studying.
01:05:04 You think you need to solve, understand this,
01:05:07 this fractal nature in order to understand intelligence
01:05:10 and communication. I do think so.
01:05:11 I think they’re deeply intertwined.
01:05:13 Yes, I think power laws are right at the heart of it.
01:05:16 So just to push that one through,
01:05:19 the same thing happens in the temporal domain.
01:05:21 So suppose your neurons in your brain
01:05:26 were always oscillating at the same frequency,
01:05:28 then the probability of finding a neuron oscillating
01:05:31 as a function of frequency
01:05:32 would be this narrowly peaked function
01:05:34 around that certain characteristic frequency.
01:05:36 That’s not at all what we see.
01:05:37 The probability of finding neurons oscillating
01:05:40 or producing spikes at a certain frequency
01:05:43 is again a power law,
01:05:45 which means there’s no defined scale
01:05:49 of the temporal activity in the brain.
01:05:53 At what speed do your thoughts occur?
01:05:56 Well, there’s a fastest speed they can occur
01:05:58 and that is limited by communication and other things,
01:06:01 but there’s not a characteristic scale.
01:06:03 We have thoughts on all temporal scales
01:06:06 from a few tens of milliseconds,
01:06:10 which is physiologically limited by our devices,
01:06:13 compare that to tens of picoseconds
01:06:15 that I talked about in superconductors,
01:06:17 all the way up to the lifetime of the organism.
01:06:19 You can still think about things
01:06:20 that happened to you when you were a kid.
01:06:22 Or if you wanna be really trippy
01:06:24 then across multiple organisms
01:06:25 in the entirety of human civilization,
01:06:27 you have thoughts that span organisms, right?
01:06:29 Yes, taking it to that level, yes.
01:06:31 If you’re willing to see the entirety of the human species
01:06:34 as a single organism with a collective intelligence
01:06:37 and that too on a spatial and temporal scale,
01:06:39 there’s thoughts occurring.
01:06:41 And then if you look at not just the human species,
01:06:44 but the entirety of life on earth
01:06:46 as an organism with thoughts that are occurring,
01:06:49 that are greater and greater sophisticated thoughts,
01:06:51 there’s a different spatial and temporal scale there.
01:06:54 This is getting very suspicious.
01:06:57 Well, hold on though, before we’re done,
01:06:58 I just wanna just tie the bow
01:07:00 and say that the spatial and temporal aspects
01:07:04 are intimately interrelated with each other.
01:07:06 So activity between neurons that are very close to each other
01:07:10 is more likely to happen on this faster timescale
01:07:13 and information is gonna propagate
01:07:15 and encompass more of the brain,
01:07:17 more of your cortices, different modules in the brain
01:07:20 are gonna be engaged in information processing
01:07:23 on longer timescales.
01:07:25 So there’s this concept of information integration
01:07:27 where neurons are specialized.
01:07:31 Any given neuron or any cluster of neuron
01:07:33 has its specific purpose,
01:07:35 but they’re also very much integrated.
01:07:39 So you have neurons that specialize,
01:07:41 but share their information.
01:07:43 And so that happens through these fractal nested oscillations
01:07:47 that occur across spatial and temporal scales.
01:07:49 I think capturing those dynamics in hardware,
01:07:53 to me, that’s the goal of neuromorphic computing.
01:07:57 So does it need to look,
01:07:58 so first of all, that’s fascinating.
01:08:00 We stated some clear principles here.
01:08:03 Now, does it have to look like the brain
01:08:08 outside of those principles as well?
01:08:09 Like what other characteristics
01:08:11 have to look like the human brain?
01:08:13 Or can it be something very different?
01:08:15 Well, it depends on what you’re trying to use it for.
01:08:18 And so I think a lot of the community
01:08:21 asks that question a lot.
01:08:23 What are you gonna do with it?
01:08:24 And I completely get it.
01:08:26 I think that’s a very important question.
01:08:28 And it’s also sometimes not the most helpful question.
01:08:31 What if what you wanna do with it is study it?
01:08:33 What if you just wanna see,
01:08:37 what do you have to build into your hardware
01:08:39 in order to observe these dynamical principles?
01:08:43 And also, I ask myself that question every day
01:08:47 and I’m not sure I’m able to answer that.
01:08:49 So like, what are you gonna do
01:08:51 with this particular neuromorphic machine?
01:08:53 So suppose what we’re trying to do with it
01:08:55 is build something that thinks.
01:08:56 We’re not trying to get it to make us any money
01:08:59 or drive a car.
01:09:00 Maybe we’ll be able to do that, but that’s not our goal.
01:09:02 Our goal is to see if we can get the same types of behaviors
01:09:07 that we observe in our own brain.
01:09:08 And by behaviors in this sense,
01:09:10 what I mean the behaviors of the components,
01:09:14 the neurons, the network, that kind of stuff.
01:09:16 I think there’s another element that I didn’t really hit on
01:09:19 that you also have to build into this.
01:09:21 And those are architectural principles.
01:09:22 They have to do with the hierarchical modular construction
01:09:26 of the network.
01:09:27 And without getting too lost in jargon,
01:09:30 the main point that I think is relevant there,
01:09:33 let me try and illustrate it with a cartoon picture
01:09:35 of the architecture of the brain.
01:09:38 So in the brain, you have the cortex,
01:09:41 which is sort of this outer sheet.
01:09:44 It’s actually, it’s a layered structure.
01:09:46 You can, if you could take it out of your brain,
01:09:48 you could unroll it on the table
01:09:50 and it would be about the size of a pizza sitting there.
01:09:53 And that’s a module.
01:09:56 It does certain things.
01:09:57 It processes as Yogi Buzaki would say,
01:10:00 it processes the what of what’s going on around you.
01:10:03 But you have another really crucial module
01:10:06 that’s called the hippocampus.
01:10:08 And that network is structured entirely differently.
01:10:10 First of all, this cortex that had described
01:10:12 10 billion neurons in there.
01:10:14 So numbers matter here.
01:10:16 And they’re organized in that sort of power law distribution
01:10:20 where the probability of making a connection drops off
01:10:22 as a power law in space.
01:10:24 The hippocampus is another module that’s important
01:10:26 for understanding how, where you are,
01:10:30 when you are keeping track of your position
01:10:36 in space and time.
01:10:37 And that network is very much random.
01:10:39 So the probability of making a connection,
01:10:41 it almost doesn’t even drop off as a function of distance.
01:10:44 It’s the same probability that you’ll make it here
01:10:46 to over there, but there are only about 100 million neurons
01:10:50 there, so you can have that huge densely connected module
01:10:54 because it’s not so big.
01:10:57 And the neocortex or the cortex and the hippocampus,
01:11:02 they talk to each other constantly.
01:11:04 And that communication is largely facilitated
01:11:07 by what’s called the thalamus.
01:11:09 I’m not a neuroscientist here.
01:11:10 I’m trying to do my best to recite things.
01:11:12 Cartoon picture of the brain, I gotcha.
01:11:14 Yeah, something like that.
01:11:15 So this thalamus is coordinating the activity
01:11:18 between the neocortex and the hippocampus
01:11:20 and making sure that they talk to each other
01:11:23 at the right time and send messages
01:11:25 that will be useful to one another.
01:11:26 So this all taken together is called
01:11:29 the thalamocortical complex.
01:11:31 And it seems like building something like that
01:11:34 is going to be crucial to capturing the types of activity
01:11:39 we’re looking for because those responsibilities,
01:11:43 those separate modules, they do different things,
01:11:45 that’s gotta be central to achieving these states
01:11:51 of efficient information integration across space and time.
01:11:55 By the way, I am able to achieve this state
01:11:58 by watching simulations, visualizations
01:12:01 of the thalamocortical complex.
01:12:03 There’s a few people I forget from where.
01:12:06 They’ve created these incredible visual illustrations
01:12:09 of visual stimulation from the eye or something like that.
01:12:14 And this image flowing through the brain.
01:12:18 Wow, I haven’t seen that, I gotta check that out.
01:12:20 So it’s one of those things,
01:12:22 you find this stuff in the world,
01:12:24 and you see on YouTube, it has 1,000 views,
01:12:26 these visualizations of the human brain
01:12:30 processing information.
01:12:32 And because there’s chemistry there,
01:12:36 because this is from actual human brains,
01:12:38 I don’t know how they’re doing the coloring,
01:12:40 but they’re able to actually trace
01:12:42 the different, the chemical and the electrical signals
01:12:46 throughout the brain, and the visual thing,
01:12:48 it’s like, whoa, because it looks kinda like the universe,
01:12:51 I mean, the whole thing is just incredible.
01:12:53 I recommend it highly, I’ll probably post a link to it.
01:12:56 But you can just look for, one of the things they simulate
01:13:00 is the thalamocortical complex and just visualization.
01:13:05 You can find that yourself on YouTube, but it’s beautiful.
01:13:09 The other question I have for you is,
01:13:11 how does memory play into all of this?
01:13:14 Because all the signals sending back and forth,
01:13:17 that’s computation and communication,
01:13:20 but that’s kinda like processing of inputs and outputs,
01:13:26 to produce outputs in the system,
01:13:27 that’s kinda like maybe reasoning,
01:13:29 maybe there’s some kind of recurrence.
01:13:30 But is there a storage mechanism that you think about
01:13:33 in the context of neuromorphic computing?
01:13:35 Yeah, absolutely, so that’s gotta be central.
01:13:37 You have to have a way that you can store memories.
01:13:41 And there are a lot of different kinds
01:13:43 of memory in the brain.
01:13:45 That’s yet another example of how it’s not a simple system.
01:13:49 So there’s one kind of memory,
01:13:53 one way of talking about memory,
01:13:56 usually starts in the context of Hopfield networks.
01:13:59 You were lucky to talk to John Hopfield on this program.
01:14:02 But the basic idea there is working memory
01:14:05 is stored in the dynamical patterns
01:14:07 of activity between neurons.
01:14:10 And you can think of a certain pattern of activity
01:14:14 as an attractor, meaning if you put in some signal
01:14:19 that’s similar enough to other
01:14:22 previously experienced signals like that,
01:14:26 then you’re going to converge to the same network dynamics
01:14:29 and you will see these neurons
01:14:31 participate in the same network patterns of activity
01:14:36 that they have in the past.
01:14:37 So you can talk about the probability
01:14:39 that different inputs will allow you to converge
01:14:42 to different basins of attraction
01:14:44 and you might think of that as,
01:14:46 oh, I saw this face and then I excited
01:14:49 this network pattern of activity
01:14:50 because last time I saw that face,
01:14:53 I was at some movie and that’s a famous person
01:14:56 that’s on the screen or something like that.
01:14:58 So that’s one memory storage mechanism.
01:15:00 But crucial to the ability to imprint those memories
01:15:04 in your brain is the ability to change
01:15:07 the strength of connection between one neuron and another,
01:15:11 that synaptic connection between them.
01:15:13 So synaptic weight update is a massive field of neuroscience
01:15:18 and neuromorphic computing as well.
01:15:19 So there are two poles on that spectrum.
01:15:26 Okay, so more in the language of machine learning,
01:15:28 we would talk about supervised and unsupervised learning.
01:15:32 And when I’m trying to tie that down
01:15:33 to neuromorphic computing,
01:15:35 I will use a definition of supervised learning,
01:15:38 which basically means the external user,
01:15:42 the person who’s controlling this hardware
01:15:45 has some knob that they can tune
01:15:48 to change each of the synaptic weights,
01:15:50 depending on whether or not the network
01:15:52 is doing what you want it to do.
01:15:53 Whereas what I mean in this conversation
01:15:56 when I say unsupervised learning
01:15:57 is that those synaptic weights
01:15:59 are dynamically changing in your network
01:16:03 based on nothing that the user is doing,
01:16:05 nothing that there’s no wire from the outside
01:16:07 going into any of those synapses.
01:16:09 The network itself is reconfiguring those synaptic weights
01:16:12 based on physical properties
01:16:15 that you’ve built into the devices.
01:16:17 So if the synapse receives a pulse from here
01:16:21 and that causes the neuron to spike,
01:16:23 some circuit built in there with no help from me
01:16:27 or anybody else adjust the weight
01:16:29 in a way that makes it more likely
01:16:31 to store the useful information
01:16:34 and excite the useful network patterns
01:16:36 and makes it less likely that random noise,
01:16:39 useless communication events
01:16:41 will have an important effect on the network activity.
01:16:45 So there’s memory encoded in the weights,
01:16:48 the synaptic weights.
01:16:49 What about the formation of something
01:16:51 that’s not often done in machine learning,
01:16:53 the formation of new synaptic connections?
01:16:56 Right, well, that seems to,
01:16:57 so again, not a neuroscientist here,
01:17:00 but my reading of the literature
01:17:01 is that that’s particularly crucial
01:17:04 in early stages of brain development
01:17:06 where a newborn is born
01:17:09 with tons of extra synaptic connections
01:17:11 and it’s actually pruned over time.
01:17:13 So the number of synapses decreases
01:17:16 as opposed to growing new long distance connections.
01:17:19 It is possible in the brain to grow new neurons
01:17:22 and assign new synaptic connections
01:17:26 but it doesn’t seem to be the primary mechanism
01:17:29 by which the brain is learning.
01:17:31 So for example, like right now,
01:17:34 sitting here talking to you,
01:17:35 you say lots of interesting things
01:17:37 and I learn from you
01:17:38 and I can remember things that you just said
01:17:41 and I didn’t grow new axonal connections
01:17:44 down to new synapses to enable those.
01:17:47 It’s plasticity mechanisms
01:17:50 in the synaptic connections between neurons
01:17:52 that enable me to learn on that timescale.
01:17:55 So at the very least,
01:17:57 you can sufficiently approximate that
01:17:59 with just weight updates.
01:18:01 You don’t need to form new connections.
01:18:02 I would say weight updates are a big part of it.
01:18:05 I also think there’s more
01:18:06 because broadly speaking,
01:18:08 when we’re doing machine learning,
01:18:10 our networks, say we’re talking about feed forward,
01:18:12 deep neural networks,
01:18:14 the temporal domain is not really part of it.
01:18:16 Okay, you’re gonna put in an image
01:18:18 and you’re gonna get out a classification
01:18:20 and you’re gonna do that as fast as possible.
01:18:22 So you care about time
01:18:23 but time is not part of the essence of this thing really.
01:18:27 Whereas in spiking neural networks,
01:18:30 what we see in the brain,
01:18:31 time is as crucial as space
01:18:33 and they’re intimately intertwined
01:18:34 as I’ve tried to say.
01:18:36 And so adaptation on different timescales
01:18:40 is important not just in memory formation,
01:18:44 although it plays a key role there,
01:18:45 but also in just keeping the activity
01:18:48 in a useful dynamic range.
01:18:50 So you have other plasticity mechanisms,
01:18:52 not just weight update,
01:18:54 or at least not on the timescale
01:18:56 of many action potentials,
01:18:58 but even on the shorter timescale.
01:19:00 So a synapse can become much less efficacious.
01:19:04 It can transmit a weaker signal
01:19:07 after the second, third, fourth,
01:19:08 that can second, third, fourth action potential
01:19:11 to occur in a sequence.
01:19:13 So that’s what’s called short term synaptic plasticity,
01:19:15 which is a form of learning.
01:19:17 You’re learning that I’m getting too much stimulus
01:19:19 from looking at something bright right now.
01:19:21 So I need to tone that down.
01:19:24 There’s also another really important mechanism
01:19:28 in learning that’s called metoplasticity.
01:19:30 What that seems to be is a way
01:19:33 that you change not the weights themselves,
01:19:37 but the rate at which the weights change.
01:19:40 So when I am in say a lecture hall and my,
01:19:45 this is a potentially terrible cartoon example,
01:19:48 but let’s say I’m in a lecture hall
01:19:49 and it’s time to learn, right?
01:19:51 So my brain will release more,
01:19:54 perhaps dopamine or some neuromodulator
01:19:57 that’s gonna change the rate
01:20:00 at which synaptic plasticity occurs.
01:20:02 So that can make me more sensitive
01:20:03 to learning at certain times,
01:20:05 more sensitive to overriding previous information
01:20:08 and less sensitive at other times.
01:20:10 And finally, as long as I’m rattling off the list,
01:20:13 I think another concept that falls in the category
01:20:16 of learning or memory adaptation is homeostasis
01:20:20 or homeostatic adaptation,
01:20:22 where neurons have the ability
01:20:24 to control their firing rate.
01:20:27 So if one neuron is just like blasting way too much,
01:20:31 it will naturally tone itself down.
01:20:33 Its threshold will adjust
01:20:35 so that it stays in a useful dynamical range.
01:20:38 And we see that that’s captured in deep neural networks
01:20:41 where you don’t just change the synaptic weights,
01:20:43 but you can also move the thresholds of simple neurons
01:20:46 in those models.
01:20:47 And so to achieve the spiking neural networks,
01:20:53 you want to use,
01:20:58 you want to implement the first principles
01:21:01 that you mentioned of the temporal
01:21:03 and the spatial fractal dynamics here.
01:21:07 So you can communicate locally,
01:21:09 you can communicate across much greater distances
01:21:13 and do the same thing in space
01:21:16 and do the same thing in time.
01:21:18 Now, you have like a chapter called
01:21:21 Superconducting Hardware for Neuromorphic Computing.
01:21:24 So what are some ideas that integrate
01:21:27 some of the things we’ve been talking about
01:21:29 in terms of the first principles of neuromorphic computing
01:21:32 and the ideas that you outline
01:21:34 in optoelectronic intelligence?
01:21:38 Yeah, so let me start, I guess,
01:21:40 on the communication side of things,
01:21:42 because that’s what led us down this track
01:21:46 in the first place.
01:21:47 By us, I’m talking about my team of colleagues at NIST,
01:21:51 Saeed Han, Bryce Brimavera, Sonia Buckley,
01:21:54 Jeff Chiles, Adam McCallum to name,
01:21:57 Alex Tate to name a few,
01:21:58 our group leaders, Saewoo Nam and Rich Mirren.
01:22:01 We’ve all contributed to this.
01:22:02 So this is not me saying necessarily
01:22:05 just the things that I’ve proposed,
01:22:07 but sort of where our team’s thinking
01:22:09 has evolved over the years.
01:22:11 Can I quickly ask, what is NIST
01:22:14 and where is this amazing group of people located?
01:22:18 NIST is the National Institute of Standards and Technology.
01:22:23 The larger facility is out in Gaithersburg, Maryland.
01:22:26 Our team is located in Boulder, Colorado.
01:22:31 NIST is a federal agency under the Department of Commerce.
01:22:36 We do a lot with, by we, I mean other people at NIST,
01:22:40 do a lot with standards,
01:22:43 making sure that we understand the system of units,
01:22:46 international system of units, precision measurements.
01:22:49 There’s a lot going on in electrical engineering,
01:22:53 material science.
01:22:54 And it’s historic.
01:22:56 I mean, it’s one of those, it’s like MIT
01:22:58 or something like that.
01:22:59 It has a reputation over many decades
01:23:00 of just being this really a place
01:23:04 where there’s a lot of brilliant people have done
01:23:06 a lot of amazing things.
01:23:07 But in terms of the people in your team,
01:23:10 in this team of people involved
01:23:12 in the concept we’re talking about now,
01:23:14 I’m just curious,
01:23:15 what kind of disciplines are we talking about?
01:23:17 What is it?
01:23:18 Mostly physicists and electrical engineers,
01:23:20 some material scientists,
01:23:23 but I would say,
01:23:24 yeah, I think physicists and electrical engineers,
01:23:27 my background is in photonics,
01:23:29 the use of light for technology.
01:23:31 So coming from there, I tend to have found colleagues
01:23:36 that are more from that background.
01:23:38 Although Adam McConn,
01:23:40 more of a superconducting electronics background,
01:23:42 we need a diversity of folks.
01:23:44 This project is sort of cross disciplinary.
01:23:46 I would love to be working more
01:23:48 with neuroscientists and things,
01:23:50 but we haven’t reached that scale yet.
01:23:53 But yeah.
01:23:54 You’re focused on the hardware side,
01:23:56 which requires all the disciplines that you mentioned.
01:23:59 And then of course,
01:24:00 neuroscientists may be a source of inspiration
01:24:02 for some of the longterm vision.
01:24:04 I would actually call it more than inspiration.
01:24:06 I would call it sort of a roadmap.
01:24:11 We’re not trying to build exactly the brain,
01:24:15 but I don’t think it’s enough to just say,
01:24:17 oh, neurons kind of work like that.
01:24:19 Let’s kind of do that thing.
01:24:20 I mean, we’re very much following the concepts
01:24:25 that the cognitive sciences have laid out for us,
01:24:27 which I believe is a really robust roadmap.
01:24:30 I mean, just on a little bit of a tangent,
01:24:33 it’s often stated that we just don’t understand the brain.
01:24:36 And so it’s really hard to replicate it
01:24:37 because we just don’t know what’s going on there.
01:24:40 And maybe five or seven years ago,
01:24:43 I would have said that,
01:24:44 but as I got more interested in the subject,
01:24:47 I read more of the neuroscience literature
01:24:50 and I was just taken by the exact opposite sense.
01:24:53 I can’t believe how much they know about this.
01:24:55 I can’t believe how mathematically rigorous
01:24:59 and sort of theoretically complete
01:25:02 a lot of the concepts are.
01:25:04 That’s not to say we understand consciousness
01:25:06 or we understand the self or anything like that,
01:25:08 but what is the brain doing
01:25:11 and why is it doing those things?
01:25:13 Neuroscientists have a lot of answers to those questions.
01:25:16 So if you’re a hardware designer
01:25:17 that just wants to get going,
01:25:19 whoa, it’s pretty clear which direction to go in, I think.
01:25:23 Okay, so I love the optimism behind that,
01:25:28 but in the implementation of these systems
01:25:32 that uses superconductivity, how do you make it happen?
01:25:39 So to me, it starts with thinking
01:25:41 about the communication network.
01:25:43 You know for sure that the ability of each neuron
01:25:47 to communicate to many thousands of colleagues
01:25:50 across the network is indispensable.
01:25:52 I take that as a core principle of my architecture,
01:25:56 my thinking on the subject.
01:25:58 So coming from a background in photonics,
01:26:02 it was very natural to say,
01:26:03 okay, we’re gonna use light for communication.
01:26:05 Just in case listeners may not know,
01:26:08 light is often used in communication.
01:26:10 I mean, if you think about radio, that’s light,
01:26:12 it’s long wavelengths, but it’s electromagnetic radiation.
01:26:15 It’s the same physical phenomenon
01:26:17 obeying exactly the same Maxwell’s equations.
01:26:20 And then all the way down to fiber, fiber optics.
01:26:24 Now you’re using visible
01:26:26 or near infrared wavelengths of light,
01:26:27 but the way you send messages across the ocean
01:26:30 is now contemporary over optical fibers.
01:26:33 So using light for communication is not a stretch.
01:26:37 It makes perfect sense.
01:26:38 So you might ask, well, why don’t you use light
01:26:41 for communication in a conventional microchip?
01:26:45 And the answer to that is, I believe, physical.
01:26:49 If we had a light source on a silicon chip
01:26:53 that was as simple as a transistor,
01:26:55 there would not be a processor in the world
01:26:58 that didn’t use light for communication,
01:26:59 at least above some distance.
01:27:01 How many light sources are needed?
01:27:04 Oh, you need a light source at every single point.
01:27:06 A light source per neuron.
01:27:08 Per neuron, per little,
01:27:09 but then if you could have a really small
01:27:13 and nice light source,
01:27:15 your definition of neuron could be flexible.
01:27:17 Could be, yes, yes.
01:27:19 Sometimes it’s helpful to me to say,
01:27:21 in this hardware, a neuron is that entity
01:27:24 which has a light source.
01:27:25 That, and I can explain.
01:27:27 And then there was light.
01:27:29 I mean, I can explain more about that, but.
01:27:32 Somehow this like rhymes with consciousness
01:27:34 because people will often say the light of consciousness.
01:27:38 So that consciousness is that which is conscious.
01:27:41 I got it.
01:27:43 That’s not my quote.
01:27:44 That’s me, that’s my quote.
01:27:47 You see, that quote comes from my background.
01:27:49 Yours is in optics, mine in light, mine’s in darkness.
01:27:55 So go ahead.
01:27:56 So the point I was making there is that
01:27:59 if it was easy to manufacture light sources
01:28:02 along with transistors on a silicon chip,
01:28:05 they would be everywhere.
01:28:07 And it’s not easy.
01:28:08 People have been trying for decades
01:28:10 and it’s actually extremely difficult.
01:28:11 I think an important part of our research
01:28:14 is dwelling right at that spot there.
01:28:16 So.
01:28:17 Is it physics or engineering?
01:28:18 It’s physics.
01:28:19 So, okay, so it’s physics, I think.
01:28:22 So what I mean by that is, as we discussed,
01:28:26 silicon is the material of choice for transistors
01:28:29 and it’s very difficult to imagine
01:28:33 that that’s gonna change anytime soon.
01:28:35 Silicon is notoriously bad at emitting light.
01:28:39 And that has to do with the immutable properties
01:28:43 of silicon itself.
01:28:44 The way that the energy bands are structured in silicon,
01:28:47 you’re never going to make silicon efficient
01:28:50 as a light source at room temperature
01:28:53 without doing very exotic things
01:28:55 that degrade its ability to interface nicely
01:28:58 with those transistors in the first place.
01:28:59 So that’s like one of these things where it’s,
01:29:02 why is nature dealing us that blow?
01:29:05 You give us these beautiful transistors
01:29:07 and you give us all the motivation
01:29:08 to use light for communication,
01:29:10 but then you don’t give us a light source.
01:29:11 So, well, okay, you do give us a light source.
01:29:14 Compound semiconductors,
01:29:15 like we talked about back at the beginning,
01:29:16 an element from group three and an element from group five
01:29:19 form an alloy where every other lattice site
01:29:21 switches which element it is.
01:29:23 Those have much better properties for generating light.
01:29:27 You put electrons in, light comes out.
01:29:30 Almost 100% of the electron hold,
01:29:33 it can be made efficient.
01:29:36 I’ll take your word for it, okay.
01:29:37 However, I say it’s physics, not engineering,
01:29:39 because it’s very difficult
01:29:41 to get those compound semiconductor light sources
01:29:45 situated with your silicon.
01:29:47 In order to do that ion implantation
01:29:49 that I talked about at the beginning,
01:29:50 high temperatures are required.
01:29:52 So you gotta make all of your transistors first
01:29:55 and then put the compound semiconductors on top of there.
01:29:58 You can’t grow them afterwards
01:30:00 because that requires high temperature.
01:30:02 It screws up all your transistors.
01:30:04 You try and stick them on there.
01:30:05 They don’t have the same lattice constant.
01:30:07 The spacing between atoms is different enough
01:30:10 that it just doesn’t work.
01:30:11 So nature does not seem to be telling us that,
01:30:15 hey, go ahead and combine light sources
01:30:17 with your digital switches
01:30:19 for conventional digital computing.
01:30:22 And conventional digital computing
01:30:24 will often require smaller scale, I guess,
01:30:27 in terms of like smartphone.
01:30:30 So in which kind of systems does nature hint
01:30:35 that we can use light and photons for communication?
01:30:40 Well, so let me just try and be clear.
01:30:42 You can use light for communication in digital systems,
01:30:46 just the light sources are not intimately integrated
01:30:49 with the silicon.
01:30:50 You manufacture all the silicon,
01:30:52 you have your microchip, plunk it down.
01:30:54 And then you manufacture your light sources,
01:30:56 separate chip, completely different process
01:30:58 made in a different foundry.
01:31:00 And then you put those together at the package level.
01:31:03 So now you have some,
01:31:06 I would say a great deal of architectural limitations
01:31:09 that are introduced by that sort of
01:31:13 package level integration
01:31:15 as opposed to monolithic on the same chip integration,
01:31:18 but it’s still a very useful thing to do.
01:31:19 And that’s where I had done some work previously
01:31:23 before I came to NIST.
01:31:24 There’s a project led by Vladimir Stoyanovich
01:31:27 that now spun out into a company called IR Labs
01:31:30 led by Mark Wade and Chen Sun
01:31:33 where they’re doing exactly that.
01:31:34 So you have your light source chip,
01:31:36 your silicon chip, whatever it may be doing,
01:31:39 maybe it’s digital electronics,
01:31:40 maybe it’s some other control purpose, something.
01:31:43 And the silicon chip drives the light source chip
01:31:47 and modulates the intensity of the lights.
01:31:49 You can get data out of the package on an optical fiber.
01:31:52 And that still gives you tremendous advantages in bandwidth
01:31:56 as opposed to sending those signals out
01:31:58 over electrical lines.
01:32:00 But it is somewhat peculiar to my eye
01:32:05 that they have to be integrated at this package level.
01:32:07 And those people, I mean, they’re so smart.
01:32:09 Those are my colleagues that I respect a great deal.
01:32:12 So it’s very clear that it’s not just
01:32:16 they’re making a bad choice.
01:32:18 This is what physics is telling us.
01:32:20 It just wouldn’t make any sense
01:32:22 to try to stick them together.
01:32:24 Yeah, so even if it’s difficult,
01:32:28 it’s easier than the alternative, unfortunately.
01:32:30 I think so, yes.
01:32:31 And again, I need to go back
01:32:33 and make sure that I’m not taking the wrong way.
01:32:35 I’m not saying that the pursuit
01:32:36 of integrating compound semiconductors with silicon
01:32:39 is fruitless and shouldn’t be pursued.
01:32:41 It should, and people are doing great work.
01:32:43 Kai Mei Lau and John Bowers, others,
01:32:45 they’re doing it and they’re making progress.
01:32:48 But to my eye, it doesn’t look like that’s ever going to be
01:32:53 just the standard monolithic light source
01:32:57 on silicon process.
01:32:58 I just don’t see it.
01:33:00 Yeah, so nature kind of points the way usually.
01:33:02 And if you resist nature,
01:33:04 you’re gonna have to do a lot more work.
01:33:05 And it’s gonna be expensive and not scalable.
01:33:07 Got it.
01:33:08 But okay, so let’s go far into the future.
01:33:11 Let’s imagine this gigantic neuromorphic computing system
01:33:14 that simulates all of our realities.
01:33:17 It currently is Mantra Matrix 4.
01:33:19 So this thing, this powerful computer,
01:33:23 how does it operate?
01:33:24 So what are the neurons?
01:33:27 What is the communication?
01:33:29 What’s your sense?
01:33:30 All right, so let me now,
01:33:32 after spending 45 minutes trashing
01:33:34 light source integration with silicon,
01:33:36 let me now say why I’m basing my entire life,
01:33:40 professional life, on integrating light sources
01:33:43 with electronics.
01:33:44 I think the game is completely different
01:33:47 when you’re talking about superconducting electronics.
01:33:49 For several reasons, let me try to go through them.
01:33:54 One is that, as I mentioned,
01:33:56 it’s difficult to integrate
01:33:57 those compound semiconductor light sources with silicon.
01:34:01 With silicon is a requirement that is introduced
01:34:04 by the fact that you’re using semiconducting electronics.
01:34:07 In superconducting electronics,
01:34:08 you’re still gonna start with a silicon wafer,
01:34:10 but it’s just the bread for your sandwich in a lot of ways.
01:34:13 You’re not using that silicon
01:34:15 in precisely the same way for the electronics.
01:34:17 You’re now depositing superconducting materials
01:34:20 on top of that.
01:34:21 The prospects for integrating light sources
01:34:24 with that kind of an electronic process
01:34:27 are certainly less explored,
01:34:30 but I think much more promising
01:34:31 because you don’t need those light sources
01:34:34 to be intimately integrated with the transistors.
01:34:36 That’s where the problems come up.
01:34:37 They don’t need to be lattice matched to the silicon,
01:34:39 all that kind of stuff.
01:34:41 Instead, it seems possible
01:34:43 that you can take those compound semiconductor light sources,
01:34:47 stick them on the silicon wafer,
01:34:49 and then grow your superconducting electronics
01:34:51 on the top of that.
01:34:52 It’s at least not obviously going to fail.
01:34:55 So the computation would be done
01:34:57 on the superconductive material as well?
01:35:00 Yes, the computation is done
01:35:01 in the superconducting electronics,
01:35:03 and the light sources receive signals
01:35:06 that say, hey, a neuron reached threshold,
01:35:08 produce a pulse of light,
01:35:09 send it out to all your downstream synaptic connections.
01:35:12 Those are, again, superconducting electronics.
01:35:16 Perform your computation,
01:35:18 and you’re off to the races.
01:35:19 Your network works.
01:35:20 So then if we can rewind real quick,
01:35:22 so what are the limitations of the challenges
01:35:25 of superconducting electronics
01:35:28 when we think about constructing these kinds of systems?
01:35:31 So actually, let me say one other thing
01:35:35 about the light sources,
01:35:37 and then I’ll move on, I promise,
01:35:39 because this is probably tedious for some.
01:35:42 This is super exciting.
01:35:44 Okay, one other thing about the light sources.
01:35:45 I said that silicon is terrible at emitting photons.
01:35:48 It’s just not what it’s meant to do.
01:35:50 However, the game is different
01:35:52 when you’re at low temperature.
01:35:54 If you’re working with superconductors,
01:35:55 you have to be at low temperature
01:35:56 because they don’t work otherwise.
01:35:58 When you’re at four Kelvin,
01:36:00 silicon is not obviously a terrible light source.
01:36:03 It’s still not as efficient as compound semiconductors,
01:36:05 but it might be good enough for this application.
01:36:08 The final thing that I’ll mention about that is, again,
01:36:11 leveraging superconductors, as I said,
01:36:13 in a different context,
01:36:15 superconducting detectors can receive one single photon.
01:36:19 In that conversation, I failed to mention
01:36:21 that semiconductors can also receive photons.
01:36:23 That’s the primary mechanism by which it’s done.
01:36:26 A camera in your phone that’s receptive to visible light
01:36:29 is receiving photons.
01:36:31 It’s based on silicon,
01:36:32 or you can make it in different semiconductors
01:36:34 for different wavelengths,
01:36:36 but it requires on the order of a thousand,
01:36:39 a few thousand photons to receive a pulse.
01:36:43 Now, when you’re using a superconducting detector,
01:36:46 you need one photon, exactly one.
01:36:48 I mean, one or more.
01:36:50 So the fact that your synapses can now be based
01:36:54 on superconducting detectors
01:36:56 instead of semiconducting detectors
01:36:58 brings the light levels that are required
01:37:00 down by some three orders of magnitude.
01:37:03 So now you don’t need good light sources.
01:37:06 You can have the world’s worst light sources.
01:37:08 As long as they spit out maybe a few thousand photons
01:37:11 every time a neuron fires,
01:37:13 you have the hardware principles in place
01:37:17 that you might be able to perform
01:37:19 this optoelectronic integration.
01:37:21 To me optoelectronic integration is, it’s just so enticing.
01:37:25 We want to be able to leverage electronics for computation,
01:37:28 light for communication,
01:37:30 working with silicon microelectronics at room temperature
01:37:32 that has been exceedingly difficult.
01:37:35 And I hope that when we move to the superconducting domain,
01:37:40 target a different application space
01:37:41 that is neuromorphic instead of digital
01:37:44 and use superconducting detectors,
01:37:47 maybe optoelectronic integration comes to us.
01:37:50 Okay, so there’s a bunch of questions.
01:37:51 So one is temperature.
01:37:53 So in these kinds of hybrid heterogeneous systems,
01:37:58 what’s the temperature?
01:37:59 What are some of the constraints to the operation here?
01:38:01 Does it all have to be a four Kelvin as well?
01:38:03 Four Kelvin.
01:38:04 Everything has to be at four Kelvin.
01:38:06 Okay, so what are the other engineering challenges
01:38:09 of making this kind of optoelectronic systems?
01:38:14 Let me just dwell on that four Kelvin for a second
01:38:16 because some people hear four Kelvin
01:38:18 and they just get up and leave.
01:38:19 They just say, I’m not doing it, you know?
01:38:21 And to me, that’s very earth centric, species centric.
01:38:25 We live in 300 Kelvin.
01:38:27 So we want our technologies to operate there too.
01:38:29 I totally get it.
01:38:30 Yeah, what’s zero Celsius?
01:38:31 Zero Celsius is 273 Kelvin.
01:38:34 So we’re talking very, very cold here.
01:38:37 This is…
01:38:38 Not even Boston cold.
01:38:39 No.
01:38:40 This is real cold.
01:38:42 Yeah.
01:38:43 Siberia cold, no.
01:38:44 Okay, so just for reference,
01:38:45 the temperature of the cosmic microwave background
01:38:47 is about 2.7 Kelvin.
01:38:49 So we’re still warmer than deep space.
01:38:51 Yeah, good.
01:38:52 So that when the universe dies out,
01:38:56 it’ll be colder than four K.
01:38:57 It’s already colder than four K.
01:38:59 In the expanses, you know,
01:39:01 you don’t have to get that far away from the earth
01:39:05 in order to drop down to not far from four Kelvin.
01:39:08 So what you’re saying is the aliens that live at the edge
01:39:11 of the observable universe
01:39:13 are using superconductive material for their computation.
01:39:16 They don’t have to live at the edge of the universe.
01:39:17 The aliens that are more advanced than us
01:39:21 in their solar system are doing this
01:39:24 in their asteroid belt.
01:39:26 We can get to that.
01:39:27 Oh, because they can get that
01:39:30 to that temperature easier there?
01:39:31 Sure, yeah.
01:39:32 All you have to do is reflect the sunlight away
01:39:34 and you have a huge headstart.
01:39:36 Oh, so the sun is the problem here.
01:39:37 Like it’s warm here on earth.
01:39:39 Got it. Yeah.
01:39:39 Okay, so can you…
01:39:41 So how do we get to four K?
01:39:42 What’s…
01:39:43 Well, okay, so what I want to say about temperature…
01:39:47 Yeah.
01:39:48 What I want to say about temperature is that
01:39:50 if you can swallow that,
01:39:52 if you can say, all right, I give up applications
01:39:56 that have to do with my cell phone
01:39:58 and the convenience of a laptop on a train
01:40:02 and you instead…
01:40:03 For me, I’m very much in the scientific head space.
01:40:06 I’m not looking at products.
01:40:07 I’m not looking at what this will be useful
01:40:09 to sell to consumers.
01:40:11 Instead, I’m thinking about scientific questions.
01:40:13 Well, it’s just not that bad to have to work at four Kelvin.
01:40:16 We do it all the time in our labs at NIST.
01:40:19 And so does…
01:40:19 I mean, for reference,
01:40:21 the entire quantum computing sector
01:40:25 usually has to work at something like 100 millikelvin,
01:40:28 50 millikelvin.
01:40:29 So now you’re talking of another factor of 100
01:40:32 even colder than that, a fraction of a degree.
01:40:35 And everybody seems to think quantum computing
01:40:37 is going to take over the world.
01:40:39 It’s so much more expensive
01:40:40 to have to get that extra factor of 10 or whatever colder.
01:40:46 And yet it’s not stopping people from investing in that area.
01:40:50 And by investing, I mean putting their research into it
01:40:53 as well as venture capital or whatever.
01:40:55 So…
01:40:56 Oh, so based on the energy of what you’re commenting on,
01:40:59 I’m getting a sense that’s one of the criticism
01:41:01 of this approach is 4K, 4 Kelvin is a big negative.
01:41:06 It is the showstopper for a lot of people.
01:41:10 They just, I mean, and understandably,
01:41:12 I’m not saying that that’s not a consideration.
01:41:16 Of course it is.
01:41:17 For some…
01:41:18 Okay, so different motivations for different people.
01:41:21 In the academic world,
01:41:23 suppose you spent your whole life
01:41:24 learning about silicon microelectronic circuits.
01:41:26 You send a design to a foundry,
01:41:28 they send you back a chip
01:41:30 and you go test it at your tabletop.
01:41:33 And now I’m saying,
01:41:34 here now learn how to use all these cryogenics
01:41:36 so you can do that at 4 Kelvin.
01:41:38 No, come on, man.
01:41:39 I don’t wanna do that.
01:41:41 That sounds bad.
01:41:42 It’s the old momentum, the Titanic of the turning.
01:41:44 Yeah, kind of.
01:41:45 But you’re saying that’s not too much of a…
01:41:48 When we’re looking at large systems
01:41:50 and the gain you can potentially get from them,
01:41:52 that’s not that much of a cost.
01:41:53 And when you wanna answer the scientific question
01:41:55 about what are the physical limits of cognition?
01:41:58 Well, the physical limits,
01:41:59 they don’t care if you’re at 4 Kelvin.
01:42:01 If you can perform cognition at a scale
01:42:04 orders of magnitude beyond any room temperature technology,
01:42:07 but you gotta get cold to do it,
01:42:09 you’re gonna do it.
01:42:10 And to me, that’s the interesting application space.
01:42:14 It’s not even an application space,
01:42:16 that’s the interesting scientific paradigm.
01:42:18 So I personally am not going to let low temperature
01:42:22 stop me from realizing a technological domain or realm
01:42:29 that is achieving in most ways everything else
01:42:33 that I’m looking for in my hardware.
01:42:36 So that, okay, that’s a big one.
01:42:37 Is there other kind of engineering challenges
01:42:40 that you envision?
01:42:40 Yeah, yeah, yeah.
01:42:41 So let me take a moment here
01:42:43 because I haven’t really described what I mean
01:42:45 by a neuron or a network in this particular hardware.
01:42:49 Yeah, do you wanna talk about loop neurons
01:42:51 and there’s so many fascinating…
01:42:53 But you just have so many amazing papers
01:42:55 that people should definitely check out
01:42:57 and the titles alone are just killer.
01:42:59 So anyway, go ahead.
01:43:01 Right, so let me say big picture,
01:43:03 based on optics, photonics for communication,
01:43:07 superconducting electronics for computation,
01:43:10 how does this all work?
01:43:11 So a neuron in this hardware platform
01:43:17 can be thought of as circuits
01:43:19 that are based on Josephson junctions,
01:43:21 like we talked about before,
01:43:22 where every time a photon comes in…
01:43:25 So let’s start by talking about a synapse.
01:43:27 A synapse receives a photon, one or more,
01:43:29 from a different neuron
01:43:31 and it converts that optical signal
01:43:33 to an electrical signal.
01:43:35 The amount of current that that adds to a loop
01:43:38 is controlled by the synaptic weight.
01:43:40 So as I said before,
01:43:42 you’re popping fluxons into a loop, right?
01:43:44 So a photon comes in,
01:43:46 it hits a superconducting single photon detector,
01:43:49 one photon, the absolute physical minimum
01:43:52 that you can communicate
01:43:53 from one place to another with light.
01:43:54 And that detector then converts that
01:43:56 into an electrical signal
01:43:57 and the amount of signal
01:43:58 is correlated with some kind of weight.
01:44:01 Yeah, so the synaptic weight will tell you
01:44:02 how many fluxons you pop into the loop.
01:44:05 It’s an analog number.
01:44:06 We’re doing analog computation now.
01:44:08 Well, can you just linger on that?
01:44:09 What the heck is a fluxon?
01:44:10 Are we supposed to know this?
01:44:11 Or is this a funny,
01:44:14 is this like the big bang?
01:44:15 Is this a funny word for something deeply technical?
01:44:18 No, let’s try to avoid using the word fluxon
01:44:21 because it’s not actually necessary.
01:44:22 When a photon…
01:44:24 It’s fun to say though.
01:44:25 So it’s very necessary, I would say.
01:44:29 When a photon hits
01:44:30 that superconducting single photon detector,
01:44:32 current is added to a superconducting loop.
01:44:36 And the amount of current that you add
01:44:39 is an analog value,
01:44:40 can have eight bit equivalent resolution,
01:44:42 something like that.
01:44:44 10 bits, maybe.
01:44:45 That’s amazing, by the way.
01:44:46 This is starting to make a lot more sense.
01:44:48 When you’re using superconductors for this,
01:44:50 the energy of that circulating current
01:44:54 is less than the energy of that photon.
01:44:58 So your energy budget is not destroyed
01:45:01 by doing this analog computation.
01:45:04 So now in the language of a neuroscientist,
01:45:07 you would say that’s your postsynaptic signal.
01:45:09 You have this current being stored in a loop.
01:45:11 You can decide what you wanna do with it.
01:45:13 Most likely you’re gonna have it decay exponentially.
01:45:16 So every single synapse
01:45:18 is gonna have some given time constant.
01:45:20 And that’s determined by putting some resistor
01:45:25 in that superconducting loop.
01:45:27 So a synapse event occurs when a photon strikes a detector,
01:45:31 adds current to that loop, it decays over time.
01:45:33 That’s the postsynaptic signal.
01:45:35 Then you can process that in a dendritic tree.
01:45:38 Bryce Primavera and I have a paper
01:45:41 that we’ve submitted about that.
01:45:43 For the more neuroscience oriented people,
01:45:45 there’s a lot of dendritic processing,
01:45:47 a lot of plasticity mechanisms you can implement
01:45:49 with essentially exactly the same circuits.
01:45:51 You have this one simple building block circuit
01:45:54 that you can use for a synapse, for a dendrite,
01:45:57 for the neuron cell body, for all the plasticity functions.
01:46:00 It’s all based on the same building block,
01:46:02 just tweaking a couple parameters.
01:46:03 So this basic building block
01:46:05 has both an optical and an electrical component,
01:46:07 and then you just build arbitrary large systems with that?
01:46:11 Close, you’re not at fault
01:46:13 for thinking that that’s what I meant.
01:46:15 What I should say is that if you want it to be a synapse,
01:46:18 you tack a superconducting detector onto the front of it.
01:46:22 And if you want it to be anything else,
01:46:23 there’s no optical component.
01:46:25 Got it, so at the front,
01:46:28 optics in the front, electrical stuff in the back.
01:46:32 Electrical, yeah, in the processing
01:46:34 and in the output signal that it sends
01:46:36 to the next stage of processing further.
01:46:39 So the dendritic trees is electrical.
01:46:41 It’s all electrical.
01:46:42 It’s all electrical in the superconducting domain.
01:46:44 For anybody who’s up on their superconducting circuits,
01:46:48 it’s just based on a DC squid, the most ubiquitous,
01:46:52 which is a circuit composed of two Joseph’s injunctions.
01:46:55 So it’s a very bread and butter kind of thing.
01:46:58 And then the only place where you go beyond that
01:47:00 is the neuron cell body itself.
01:47:03 It’s receiving all these electrical inputs
01:47:05 from the synapses or dendrites
01:47:06 or however you’ve structured that particular unique neuron.
01:47:09 And when it reaches its threshold,
01:47:12 which occurs by driving a Joseph’s injunction
01:47:14 above its critical current,
01:47:15 it produces a pulse of current,
01:47:17 which starts an amplification sequence,
01:47:19 voltage amplification,
01:47:21 that produces light out of a transmitter.
01:47:24 So one of our colleagues, Adam McCann,
01:47:26 and Sonia Buckley as well,
01:47:27 did a lot of work on the light sources
01:47:30 and the amplifiers that drive the current
01:47:34 and produce sufficient voltage to drive current
01:47:37 through that now semiconducting part.
01:47:39 So that light source is the semiconducting part of a neuron.
01:47:43 And that, so the neuron has reached threshold.
01:47:45 It produces a pulse of light.
01:47:47 That light then fans out across a network of wave guides
01:47:51 to reach all the downstream synaptic terminals
01:47:54 that perform this process themselves.
01:47:57 So it’s probably worth explaining
01:47:59 what a network of wave guides is,
01:48:02 because a lot of listeners aren’t gonna know that.
01:48:04 Look up the papers by Jeff Chiles on this one.
01:48:07 But basically, light can be guided in a simple,
01:48:11 basically wire of usually an insulating material.
01:48:14 So silicon, silicon nitride,
01:48:18 different kinds of glass,
01:48:20 just like in a fiber optic, it’s glass, silicon dioxide.
01:48:23 That makes it a little bit big.
01:48:24 We wanna bring these down.
01:48:26 So we use different materials like silicon nitride,
01:48:28 but basically just imagine a rectangle of some material
01:48:32 that just goes and branches,
01:48:37 forms different branch points
01:48:39 that target different subregions of the network.
01:48:43 You can transition between layers of these.
01:48:45 So now we’re talking about building in the third dimension,
01:48:47 which is absolutely crucial.
01:48:48 So that’s what wave guides are.
01:48:50 Yeah, that’s great.
01:48:52 Why the third dimension is crucial?
01:48:54 Okay, so yes, you were talking about
01:48:56 what are some of the technical limitations.
01:48:59 One of the things that I believe we have to grapple with
01:49:04 is that our brains are miraculously compact.
01:49:08 For the number of neurons that are in our brain,
01:49:11 it sure does fit in a small volume,
01:49:13 as it would have to if we’re gonna be biological organisms
01:49:16 that are resource limited and things like that.
01:49:19 Any kind of hardware neuron
01:49:20 is almost certainly gonna be much bigger than that
01:49:23 if it is of comparable complexity,
01:49:26 whether it’s based on silicon transistors.
01:49:28 Okay, a transistor, seven nanometers,
01:49:30 that doesn’t mean a semiconductor based neuron
01:49:33 is seven nanometers.
01:49:34 They’re big.
01:49:35 They require many transistors,
01:49:38 different other things like capacitors and things
01:49:40 that store charge.
01:49:41 They end up being on the order of 100 microns
01:49:44 by 100 microns,
01:49:45 and it’s difficult to get them down any smaller than that.
01:49:48 The same is true for superconducting neurons,
01:49:50 and the same is true
01:49:52 if we’re trying to use light for communication.
01:49:54 Even if you’re using electrons for communication,
01:49:56 you have these wires where, okay,
01:50:00 the size of an electron might be angstroms,
01:50:03 but the size of a wire is not angstroms,
01:50:05 and if you try and make it narrower,
01:50:07 the resistance just goes up,
01:50:08 so you don’t actually win.
01:50:10 To communicate over long distances,
01:50:12 you need your wires to be microns wide,
01:50:15 and it’s the same thing for wave guides.
01:50:17 Wave guides are essentially limited
01:50:18 by the wavelength of light,
01:50:20 and that’s gonna be about a micron,
01:50:21 so whereas compare that to an axon,
01:50:24 the analogous component in the brain,
01:50:26 which is 10 nanometers in diameter, something like that,
01:50:32 they’re bigger when they need to communicate
01:50:33 over long distances,
01:50:34 but grappling with the size of these structures
01:50:37 is inevitable and crucial,
01:50:39 and so in order to make systems of comparable scale
01:50:45 to the human brain, by scale here,
01:50:46 I mean number of interconnected neurons,
01:50:49 you absolutely have to be using
01:50:51 the third spatial dimension,
01:50:53 and that means on the wafer,
01:50:55 you need multiple layers
01:50:57 of both active and passive components.
01:50:59 Active, I mean superconducting electronic circuits
01:51:03 that are performing computations,
01:51:05 and passive, I mean these wave guides
01:51:07 that are routing the optical signals to different places,
01:51:10 you have to be able to stack those.
01:51:11 If you can get to something like 10 planes
01:51:14 of each of those, or maybe not even 10,
01:51:16 maybe five, six, something like that,
01:51:18 then you’re in business.
01:51:19 Now you can get millions of neurons on a wafer,
01:51:22 but that’s not anywhere close to the brain scale.
01:51:26 In order to get to the scale of the human brain,
01:51:27 you’re gonna have to also use the third dimension
01:51:30 in the sense that entire wafers
01:51:32 need to be stacked on top of each other
01:51:34 with fiber optic communication between them,
01:51:36 and we need to be able to fill a space
01:51:38 the size of this table with stacked wafers,
01:51:42 and that’s when you can get to some 10 billion neurons
01:51:44 like your human brain,
01:51:45 and I don’t think that’s specific
01:51:46 to the optoelectronic approach that we’re taking.
01:51:48 I think that applies to any hardware
01:51:51 where you’re trying to reach commensurate scale
01:51:53 and complexity as the human brain.
01:51:55 So you need that fractal stacking,
01:51:57 so stacking on the wafer,
01:51:59 and stacking of the wafers,
01:52:01 and then whatever the system that combines,
01:52:03 this stacking of the tables with the wafers.
01:52:06 And it has to be fractal all the way,
01:52:07 you’re exactly right,
01:52:08 because that’s the only way
01:52:10 that you can efficiently get information
01:52:12 from a small point to across that whole network.
01:52:15 It has to have the power law connected.
01:52:17 And photons are like optics throughout.
01:52:20 Yeah, absolutely.
01:52:21 Once you’re at this scale, to me it’s just obvious.
01:52:23 Of course you’re using light for communication.
01:52:25 You have fiber optics given to us from nature, so simple.
01:52:30 The thought of even trying to do
01:52:32 any kind of electrical communication
01:52:34 just doesn’t make sense to me.
01:52:37 I’m not saying it’s wrong, I don’t know,
01:52:39 but that’s where I’m coming from.
01:52:40 So let’s return to loop neurons.
01:52:43 Why are they called loop neurons?
01:52:46 Yeah, the term loop neurons comes from the fact,
01:52:48 like we’ve been talking about,
01:52:49 that they rely heavily on these superconducting loops.
01:52:53 So even in a lot of forms of digital computing
01:52:57 with superconductors,
01:52:58 storing a signal in a superconducting loop
01:53:02 is a primary technique.
01:53:05 In this particular case,
01:53:06 it’s just loops everywhere you look.
01:53:08 So the strength of a synaptic weight
01:53:11 is gonna be set by the amount of current circulating
01:53:15 in a loop that is coupled to the synapse.
01:53:17 So memory is implemented as current circulating
01:53:22 in a superconducting loop.
01:53:24 The coupling between, say, a synapse and a dendrite
01:53:27 or a synapse in the neuron cell body
01:53:29 occurs through loop coupling through transformers.
01:53:33 So current circulating in a synapse
01:53:34 is gonna induce current in a different loop,
01:53:37 a receiving loop in the neuron cell body.
01:53:40 So since all of the computation is happening
01:53:44 in these flux storage loops
01:53:46 and they play such a central role
01:53:48 in how the information is processed,
01:53:50 how memories are formed, all that stuff,
01:53:52 I didn’t think too much about it,
01:53:53 I just called them loop neurons
01:53:55 because it rolls off the tongue a little bit better
01:53:58 than superconducting optoelectronic neurons.
01:54:02 Okay, so how do you design circuits for these loop neurons?
01:54:08 That’s a great question.
01:54:09 There’s a lot of different scales of design.
01:54:12 So at the level of just one synapse,
01:54:16 you can use conventional methods.
01:54:18 They’re not that complicated
01:54:21 as far as superconducting electronics goes.
01:54:23 It’s just four Joseph’s injunctions or something like that
01:54:27 depending on how much complexity you wanna add.
01:54:29 So you can just directly simulate each component in SPICE.
01:54:34 What’s SPICE?
01:54:35 It’s Standard Electrical Simulation Software, basically.
01:54:39 So you’re just explicitly solving the differential equations
01:54:42 that describe the circuit elements.
01:54:44 And then you can stack these things together
01:54:46 in that simulation software to then build circuits.
01:54:48 You can, but that becomes computationally expensive.
01:54:51 So one of the things when COVID hit,
01:54:54 we knew we had to turn some attention
01:54:55 to more things you can do at home in your basement
01:54:59 or whatever, and one of them was computational modeling.
01:55:02 So we started working on adapting,
01:55:07 abstracting out the circuit performance
01:55:10 so that you don’t have to explicitly solve
01:55:12 the circuit equations, which for Joseph’s injunctions
01:55:15 usually needs to be done on like a picosecond timescale
01:55:18 and you have a lot of nodes in your circuit.
01:55:21 So it results in a lot of differential equations
01:55:24 that need to be solved simultaneously.
01:55:26 We were looking for a way to simulate these circuits
01:55:29 that is scalable up to networks of millions or so neurons
01:55:33 is sort of where we’re targeting right now.
01:55:36 So we were able to analyze the behavior of these circuits.
01:55:40 And as I said, it’s based on these simple building blocks.
01:55:43 So you really only need to understand
01:55:45 this one building block.
01:55:46 And if you get a good model of that, boom, it tiles.
01:55:48 And you can change the parameters in there
01:55:51 to get different behaviors and stuff,
01:55:52 but it’s all based on now it’s one differential equation
01:55:56 that you need to solve.
01:55:57 So one differential equation for every synapse,
01:56:00 dendrite or neuron in your system.
01:56:03 And for the neuroscientists out there,
01:56:05 it’s just a simple leaky integrate and fire model,
01:56:08 leaky integrator, basically.
01:56:10 A synapse is a leaky integrator,
01:56:11 a dendrite is a leaky integrator.
01:56:13 So I’m really fascinated by how this one simple component
01:56:18 can be used to achieve lots of different types
01:56:22 of dynamical activity.
01:56:24 And to me, that’s where scalability comes from.
01:56:27 And also complexity as well.
01:56:29 Complexity is often characterized
01:56:30 by relatively simple building blocks
01:56:35 connected in potentially simple
01:56:37 or sometimes complicated ways,
01:56:39 and then emergent new behavior that was hard to predict
01:56:41 from those simple elements.
01:56:44 And that’s exactly what we’re working with here.
01:56:46 So it’s a very exciting platform,
01:56:49 both from a modeling perspective
01:56:50 and from a hardware manifestation perspective
01:56:52 where we can hopefully start to have this test bed
01:56:57 where we can explore things,
01:56:58 not just related to neuroscience,
01:57:00 but also related to other things
01:57:03 that connected to other physics like critical phenomenon,
01:57:07 Ising models, things like that.
01:57:08 So you were asking how we simulate these circuits.
01:57:13 It’s at different levels
01:57:14 and we’ve got the simple spice circuit stuff.
01:57:18 That’s no problem.
01:57:19 And now we’re building these network models
01:57:21 based on this more efficient leaky integrator.
01:57:23 So we can actually reduce every element
01:57:26 to one differential equation.
01:57:27 And then we can also step through it
01:57:28 on a much coarser time grid.
01:57:30 So it ends up being something like a factor
01:57:32 of a thousand to 10,000 speed improvement,
01:57:35 which allows us to simulate,
01:57:37 but hopefully up to millions of neurons.
01:57:40 Whereas before we would have been limited to tens,
01:57:44 a hundred, something like that.
01:57:45 And just like simulating quantum mechanical systems
01:57:48 with a quantum computer.
01:57:49 So the goal here is to understand such systems.
01:57:53 For me, the goal is to study this
01:57:55 as a scientific physical system.
01:57:58 I’m not drawn towards turning this
01:58:01 into an enterprise at this point.
01:58:03 I feel short term applications
01:58:05 that obviously make a lot of money
01:58:07 is not necessarily a curiosity driver for you at the moment.
01:58:11 Absolutely not.
01:58:12 If you’re interested in short term making money,
01:58:14 go with deep learning, use silicon microelectronics.
01:58:16 If you wanna understand things like the physics
01:58:20 of a fascinating system,
01:58:23 or if you wanna understand something more
01:58:25 along the lines of the physical limits
01:58:27 of what can be achieved,
01:58:29 then I think single photon communication,
01:58:32 superconducting electronics is extremely exciting.
01:58:35 What if I wanna use superconducting hardware
01:58:39 at four Kelvin to mine Bitcoin?
01:58:42 That’s my main interest.
01:58:44 The reason I wanted to talk to you today,
01:58:45 I wanna say, no, I don’t know.
01:58:47 What’s Bitcoin?
01:58:51 Look it up on the internet.
01:58:53 Somebody told me about it.
01:58:54 I’m not sure exactly what it is.
01:58:57 But let me ask nevertheless
01:58:59 about applications to machine learning.
01:59:01 Okay, so if you look at the scale of five, 10, 20 years,
01:59:07 is it possible to, before we understand the nature
01:59:11 of human intelligence and general intelligence,
01:59:14 do you think we’ll start falling out of this exploration
01:59:19 of neuromorphic systems ability to solve some
01:59:23 of the problems that the machine learning systems
01:59:25 of today can’t solve?
01:59:26 Well, I’m really hesitant to over promise.
01:59:31 So I really don’t know.
01:59:34 Also, I don’t really understand machine learning
01:59:36 in a lot of senses.
01:59:37 I mean, machine learning from my perspective appears
01:59:42 to require that you know precisely what your input is
01:59:49 and also what your goal is.
01:59:51 You usually have some objective function
01:59:53 or something like that.
01:59:54 And that’s very limiting.
01:59:57 I mean, of course, a lot of times that’s the case.
02:00:00 There’s a picture and there’s a horse in it, so you’re done.
02:00:03 But that’s not a very interesting problem.
02:00:06 I think when I think about intelligence,
02:00:09 it’s almost defined by the ability to handle problems
02:00:13 where you don’t know what your inputs are going to be
02:00:15 and you don’t even necessarily know
02:00:17 what you’re trying to accomplish.
02:00:18 I mean, I’m not sure what I’m trying to accomplish
02:00:21 in this world.
02:00:22 Yeah, at all scales.
02:00:24 Yeah, at all scales, right.
02:00:25 I mean, so I’m more drawn to the underlying phenomena,
02:00:33 the critical dynamics of this system,
02:00:37 trying to understand how elements that you build
02:00:41 into your hardware result in emergent fascinating activity
02:00:48 that was very difficult to predict, things like that.
02:00:51 So, but I gotta be really careful
02:00:53 because I think a lot of other people who,
02:00:55 if they found themselves working on this project
02:00:57 in my shoes, they would say, all right,
02:00:59 what are all the different ways we can use this
02:01:01 for machine learning?
02:01:02 Actually, let me just definitely mention colleague
02:01:05 at NIST, Mike Schneider.
02:01:06 He’s also very much interested,
02:01:09 particularly in the superconducting side of things,
02:01:11 using the incredible speed, power efficiency,
02:01:14 also Ken Seagal at Colgate,
02:01:16 other people working on specifically
02:01:18 the superconducting side of this for machine learning
02:01:22 and deep feed forward neural networks.
02:01:25 There, the advantages are obvious.
02:01:27 It’s extremely fast.
02:01:28 Yeah, so that’s less on the nature of intelligences
02:01:31 and more on various characteristics of this hardware
02:01:35 that you can use for the basic computation
02:01:38 as we know it today and communication.
02:01:40 One of the things that Mike Schneider’s working on right now
02:01:42 is an image classifier at a relatively small scale.
02:01:46 I think he’s targeting that nine pixel problem
02:01:48 where you can have three different characters
02:01:50 and you put in a nine pixel image
02:01:54 and you classify it as one of these three categories.
02:01:58 And that’s gonna be really interesting
02:02:00 to see what happens there,
02:02:01 because if you can show that even at that scale,
02:02:05 you just put these images in and you get it out
02:02:08 and he thinks he can do it,
02:02:09 I forgot if it’s a nanosecond
02:02:11 or some extremely fast classification time,
02:02:14 it’s probably less,
02:02:14 it’s probably a hundred picoseconds or something.
02:02:17 There you have challenges though,
02:02:18 because the Joseph’s injunctions themselves,
02:02:21 the electronic circuit is extremely power efficient.
02:02:24 Some orders of magnitude for something more
02:02:26 than a transistor doing the same thing,
02:02:29 but when you have to cool it down to four Kelvin,
02:02:31 you pay a huge overhead just for keeping it cold,
02:02:33 even if it’s not doing anything.
02:02:35 So it has to work at large scale
02:02:40 in order to overcome that power penalty,
02:02:43 but that’s possible.
02:02:45 It’s just, it’s gonna have to get that performance.
02:02:48 And this is sort of what you were asking about before
02:02:49 is like how much better than silicon would it need to be?
02:02:52 And the answer is, I don’t know.
02:02:54 I think if it’s just overall better than silicon
02:02:57 at a problem that a lot of people care about,
02:03:00 maybe it’s image classification,
02:03:02 maybe it’s facial recognition,
02:03:03 maybe it’s monitoring credit transactions, I don’t know,
02:03:07 then I think it will have a place.
02:03:09 It’s not gonna be in your cell phone,
02:03:10 but it could be in your data center.
02:03:12 So what about in terms of the data center,
02:03:16 I don’t know if you’re paying attention
02:03:17 to the various systems,
02:03:19 like Tesla recently announced DOJO,
02:03:23 which is a large scale machine learning training system,
02:03:27 that again, the bottleneck there
02:03:28 is probably going to be communication
02:03:30 between those systems.
02:03:32 Is there something from your work
02:03:34 on everything we’ve been talking about
02:03:38 in terms of superconductive hardware
02:03:41 that could be useful there?
02:03:43 Oh, I mean, okay, tomorrow, no.
02:03:46 In the long term, it could be the whole thing.
02:03:49 It could be nothing.
02:03:49 I don’t know, but definitely, definitely.
02:03:54 When you look at the,
02:03:55 so I don’t know that much about DOJO.
02:03:56 My understanding is that that’s new, right?
02:03:58 That’s just coming online.
02:04:01 Well, I don’t even know where it hasn’t come online.
02:04:06 And when you announce big, sexy,
02:04:09 so let me explain to you the way things work
02:04:11 in the world of business and marketing.
02:04:15 It’s not always clear where you are
02:04:18 on the coming online part of that.
02:04:20 So I don’t know where they are exactly,
02:04:22 but the vision is from a ground up
02:04:25 to build a very, very large scale,
02:04:28 modular machine learning, ASIC,
02:04:31 basically hardware that’s optimized
02:04:32 for training neural networks.
02:04:34 And of course, there’s a lot of companies
02:04:36 that are small and big working on this kind of problem.
02:04:39 The question is how to do it in a modular way
02:04:42 that has very fast communication.
02:04:45 The interesting aspect of Tesla is you have a company
02:04:49 that at least at this time is so singularly focused
02:04:54 on solving a particular machine learning problem
02:04:57 and is making obviously a lot of money doing so
02:05:00 because the machine learning problem
02:05:02 happens to be involved with autonomous driving.
02:05:05 So you have a system that’s driven by an application.
02:05:09 And that’s really interesting because you have maybe Google
02:05:15 working on TPUs and so on.
02:05:17 You have all these other companies with ASICs.
02:05:21 They’re usually more kind of always thinking general.
02:05:25 So I like it when it’s driven by a particular application
02:05:29 because then you can really get to the,
02:05:32 it’s somehow if you just talk broadly about intelligence,
02:05:37 you may not always get to the right solutions.
02:05:40 It’s nice to couple that sometimes
02:05:41 with specific clear illustration
02:05:45 of something that requires general intelligence,
02:05:47 which for me driving is one such case.
02:05:49 I think you’re exactly right.
02:05:51 Sometimes just having that focus on that application
02:05:54 brings a lot of people focuses their energy and attention.
02:05:57 I think that, so one of the things that’s appealing
02:06:00 about what you’re saying is not just
02:06:02 that the application is specific,
02:06:03 but also that the scale is big
02:06:06 and that the benefit is also huge.
02:06:10 Financial and to humanity.
02:06:12 Right, right, right.
02:06:13 Yeah, so I guess let me just try to understand
02:06:15 is the point of this dojo system
02:06:17 to figure out the parameters
02:06:21 that then plug into neural networks
02:06:23 and then you don’t need to retrain,
02:06:26 you just make copies of a certain chip
02:06:29 that has all the other parameters established or?
02:06:32 No, it’s straight up retraining a large neural network
02:06:36 over and over and over.
02:06:38 So you have to do it once for every new car?
02:06:41 No, no, you have to, so they do this interesting process,
02:06:44 which I think is a process for machine learning,
02:06:47 supervised machine learning systems
02:06:49 you’re going to have to do, which is you have a system,
02:06:53 you train your network once, it takes a long time.
02:06:56 I don’t know how long, but maybe a week.
02:06:58 Okay. To train.
02:07:00 And then you deploy it on, let’s say about a million cars.
02:07:05 I don’t know what the number is.
02:07:05 But that part, you just write software
02:07:07 that updates some weights in a table and yeah, okay.
02:07:10 But there’s a loop back.
02:07:12 Yeah, yeah, okay.
02:07:13 Each of those cars run into trouble, rarely,
02:07:18 but they catch the edge cases
02:07:23 of the performance of that particular system
02:07:26 and then send that data back
02:07:28 and either automatically or by humans,
02:07:31 that weird edge case data is annotated
02:07:34 and then the network has to become smart enough
02:07:37 to now be able to perform in those edge cases,
02:07:40 so it has to get retrained.
02:07:41 There’s clever ways of retraining different parts
02:07:43 of that network, but for the most part,
02:07:46 I think they prefer to retrain the entire thing.
02:07:49 So you have this giant monster
02:07:51 that kind of has to be retrained regularly.
02:07:54 I think the vision with Dojo is to have
02:07:58 a very large machine learning focused,
02:08:02 driving focused supercomputer
02:08:05 that then is sufficiently modular
02:08:07 that can be scaled to other machine learning applications.
02:08:11 So they’re not limiting themselves completely
02:08:12 to this particular application,
02:08:14 but this application is the way they kind of test
02:08:17 this iterative process of machine learning
02:08:19 is you make a system that’s very dumb,
02:08:23 deploy it, get the edge cases where it fails,
02:08:27 make it a little smarter, it becomes a little less dumb
02:08:30 and that iterative process achieves something
02:08:33 that you can call intelligent or is smart enough
02:08:36 to be able to solve this particular application.
02:08:37 So it has to do with training neural networks fast
02:08:43 and training neural networks that are large.
02:08:45 But also based on an extraordinary amount of diverse input.
02:08:49 Data, yeah.
02:08:50 And that’s one of the things,
02:08:51 so this does seem like one of those spaces
02:08:54 where the scale of superconducting optoelectronics,
02:08:58 the way that, so when you talk about the weaknesses,
02:09:02 like I said, okay, well, you have to cool it down.
02:09:04 At this scale, that’s fine.
02:09:05 Because that’s not too much of an added cost.
02:09:09 Most of your power is being dissipated
02:09:10 by the circuits themselves, not the cooling.
02:09:12 And also you have one centralized kind of cognitive hub,
02:09:19 if you will.
02:09:20 And so if we’re talking about putting
02:09:24 a superconducting system in a car, that’s questionable.
02:09:28 Do you really wanna cryostat
02:09:30 in the trunk of everyone in your car?
02:09:31 It’ll fit, it’s not that big of a deal,
02:09:32 but hopefully there’s a better way, right?
02:09:35 But since this is sort of a central supreme intelligence
02:09:39 or something like that,
02:09:40 and it needs to really have this massive data acquisition,
02:09:45 massive data integration,
02:09:47 I would think that that’s where large scale
02:09:49 spiking neural networks with vast communication
02:09:51 and all these things would have something
02:09:53 pretty tremendous to offer.
02:09:54 It’s not gonna happen tomorrow.
02:09:55 There’s a lot of development that needs to be done.
02:09:58 But we have to be patient with self driving cars
02:10:01 for a lot of reasons.
02:10:02 We were all optimistic that they would be here by now.
02:10:04 And okay, they are to some extent,
02:10:06 but if we’re thinking five or 10 years down the line,
02:10:09 it’s not unreasonable.
02:10:12 One other thing, let me just mention,
02:10:15 getting into self driving cars and technologies
02:10:17 that are using AI out in the world,
02:10:19 this is something NIST cares a lot about.
02:10:21 Elham Tabassi is leading up a much larger effort in AI
02:10:25 at NIST than my little project.
02:10:29 And really central to that mission
02:10:32 is this concept of trustworthiness.
02:10:35 So when you’re going to deploy this neural network
02:10:39 in every single automobile with so much on the line,
02:10:43 you have to be able to trust that.
02:10:45 So now how do we know that we can trust that?
02:10:48 How do we know that we can trust the self driving car
02:10:50 or the supercomputer that trained it?
02:10:53 There’s a lot of work there
02:10:54 and there’s a lot of that going on at NIST.
02:10:56 And it’s still early days.
02:10:58 I mean, you’re familiar with the problem and all that.
02:11:01 But there’s a fascinating dance in engineering
02:11:04 with safety critical systems.
02:11:06 There’s a desire in computer science,
02:11:08 just recently talked to Don Knuth,
02:11:13 for algorithms and for systems,
02:11:14 for them to be provably correct or provably safe.
02:11:17 And this is one other difference
02:11:20 between humans and biological systems
02:11:22 is we’re not provably anything.
02:11:24 And so there’s some aspect of imperfection
02:11:29 that we need to have built in,
02:11:32 like robustness to imperfection be part of our systems,
02:11:37 which is a difficult thing for engineers to contend with.
02:11:40 They’re very uncomfortable with the idea
02:11:42 that you have to be okay with failure
02:11:46 and almost engineer failure into the system.
02:11:49 Mathematicians hate it too.
02:11:50 But I think it was Turing who said something
02:11:53 along the lines of,
02:11:55 I can give you an intelligent system
02:11:57 or I can give you a flawless system,
02:11:59 but I can’t give you both.
02:12:00 And it’s in sort of creativity and abstract thinking
02:12:04 seem to rely somewhat on stochasticity
02:12:07 and not having components
02:12:11 that perform exactly the same way every time.
02:12:13 This is where like the disagreement I have with,
02:12:16 not disagreement, but a different view on the world.
02:12:18 I’m with Turing,
02:12:19 but when I talk to robotic, robot colleagues,
02:12:24 that sounds like I’m talking to robots,
02:12:26 colleagues that are roboticists,
02:12:29 the goal is perfection.
02:12:31 And to me is like, no,
02:12:33 I think the goal should be imperfection
02:12:38 that’s communicated.
02:12:40 And through the interaction between humans and robots,
02:12:44 that imperfection becomes a feature, not a bug.
02:12:49 Like together, seen as a system,
02:12:52 the human and the robot together
02:12:53 are better than either of them individually,
02:12:56 but the robot itself is not perfect in any way.
02:13:00 Of course, there’s a bunch of disagreements,
02:13:02 including with Mr. Elon about,
02:13:06 to me, autonomous driving is fundamentally
02:13:08 a human robot interaction problem,
02:13:10 not a robotics problem.
02:13:12 To Elon, it’s a robotics problem.
02:13:14 That’s actually an open and fascinating question,
02:13:18 whether humans can be removed from the loop completely.
02:13:24 We’ve talked about a lot of fascinating chemistry
02:13:27 and physics and engineering,
02:13:31 and we’re always running up against this issue
02:13:33 that nature seems to dictate what’s easy and what’s hard.
02:13:37 So you have this cool little paper
02:13:40 that I’d love to just ask you about.
02:13:43 It’s titled,
02:13:44 Does Cosmological Evolution Select for Technology?
02:13:48 So in physics, there’s parameters
02:13:53 that seem to define the way our universe works,
02:13:56 that physics works, that if it worked any differently,
02:13:59 we would get a very different world.
02:14:01 So it seems like the parameters are very fine tuned
02:14:04 to the kind of physics that we see.
02:14:06 All the beautiful E equals MC squared,
02:14:08 they would get these nice, beautiful laws.
02:14:10 It seems like very fine tuned for that.
02:14:13 So what you argue in this article
02:14:15 is it may be that the universe has also fine tuned
02:14:20 its parameters that enable the kind of technological
02:14:25 innovation that we see, the technology that we see.
02:14:29 Can you explain this idea?
02:14:31 Yeah, I think you’ve introduced it nicely.
02:14:33 Let me just try to say a few things in my language layout.
02:14:39 What is this fine tuning problem?
02:14:41 So physicists have spent centuries trying to understand
02:14:46 the system of equations that govern the way nature behaves,
02:14:51 the way particles move and interact with each other.
02:14:55 And as that understanding has become more clear over time,
02:15:00 it became sort of evident that it’s all well adjusted
02:15:07 to allow a universe like we see, very complex,
02:15:13 this large, long lived universe.
02:15:16 And so one answer to that is, well, of course it is
02:15:19 because we wouldn’t be here otherwise.
02:15:21 But I don’t know, that’s not very satisfying.
02:15:24 That’s sort of, that’s what’s known
02:15:25 as the weak anthropic principle.
02:15:27 It’s a statement of selection bias.
02:15:29 We can only observe a universe that is fit for us to live in.
02:15:33 So what does it mean for a universe
02:15:34 to be fit for us to live in?
02:15:36 Well, the pursuit of physics,
02:15:38 it is based partially on coming up with equations
02:15:42 that describe how things behave
02:15:44 and interact with each other.
02:15:46 But in all those equations you have,
02:15:48 so there’s the form of the equation,
02:15:49 sort of how different fields or particles
02:15:54 move in space and time.
02:15:56 But then there are also the parameters
02:15:58 that just tell you sort of the strength
02:16:01 of different couplings.
02:16:02 How strongly does a charged particle
02:16:05 couple to the electromagnetic field or masses?
02:16:07 How strongly does a particle couple
02:16:10 to the Higgs field or something like that?
02:16:12 And those parameters that define,
02:16:16 not the general structure of the equations,
02:16:19 but the relative importance of different terms,
02:16:23 they seem to be every bit as important
02:16:25 as the structure of the equations themselves.
02:16:27 And so I forget who it was.
02:16:29 Somebody, when they were working through this
02:16:31 and trying to see, okay, if I adjust the parameter,
02:16:34 this parameter over here,
02:16:34 call it the, say the fine structure constant,
02:16:36 which tells us the strength
02:16:37 of the electromagnetic interaction.
02:16:40 Oh boy, I can’t change it very much.
02:16:42 Otherwise nothing works.
02:16:43 The universe sort of doesn’t,
02:16:45 it just pops into existence and goes away
02:16:47 in a nanosecond or something like that.
02:16:48 And somebody had the phrase,
02:16:51 this looks like a put up job,
02:16:52 meaning every one of these parameters was dialed in.
02:16:57 It’s arguable how precisely they have to be dialed in,
02:17:00 but dialed in to some extent,
02:17:03 not just in order to enable our existence,
02:17:05 that’s a very anthropocentric view,
02:17:07 but to enable a universe like this one.
02:17:10 So, okay, maybe I think the majority position
02:17:14 of working physicists in the field is,
02:17:17 it has to be that way in order for us to exist.
02:17:18 We’re here, we shouldn’t be surprised
02:17:20 that that’s the way the universe is.
02:17:22 And I don’t know, for a while,
02:17:24 that never sat well with me,
02:17:26 but I just kind of moved on
02:17:28 because there are things to do
02:17:29 and a lot of exciting work.
02:17:31 It doesn’t depend on resolving this puzzle,
02:17:33 but as I started working more with technology,
02:17:39 getting into the more recent years of my career,
02:17:41 particularly when I started,
02:17:43 after having worked with silicon for a long time,
02:17:46 which was kind of eerie on its own,
02:17:49 but then when I switched over to superconductors,
02:17:51 I was just like, this is crazy.
02:17:53 It’s just absolutely astonishing
02:17:57 that our universe gives us superconductivity.
02:18:00 It’s one of the most beautiful physical phenomena
02:18:02 and it’s also extraordinarily useful for technology.
02:18:06 So you can argue that the universe
02:18:07 has to have the parameters it does for us to exist
02:18:11 because we couldn’t be here otherwise,
02:18:13 but why does it give us technology?
02:18:14 Why does it give us silicon that has this ideal oxide
02:18:18 that allows us to make a transistor
02:18:20 without trying that hard?
02:18:23 That can’t be explained by the same anthropic reasoning.
02:18:27 Yeah, so it’s asking the why question.
02:18:30 I mean, a slight natural extension of that question is,
02:18:34 I wonder if the parameters were different
02:18:39 if we would simply have just another set of paint brushes
02:18:44 to create totally other things
02:18:46 that wouldn’t look like anything
02:18:49 like the technology of today,
02:18:50 but would nevertheless have incredible complexity,
02:18:54 which is if you sort of zoom out and start defining things,
02:18:57 not by like how many batteries it needs
02:19:01 and whether it can make toast,
02:19:03 but more like how much complexity is within the system
02:19:06 or something like that.
02:19:07 Well, yeah, you can start to quantify things.
02:19:10 You’re exactly right.
02:19:10 So nowhere am I arguing that
02:19:13 in all of the vast parameter space
02:19:15 of everything that could conceivably exist
02:19:18 in the multiverse of nature,
02:19:20 there’s this one point in parameter space
02:19:23 where complexity arises.
02:19:25 I doubt it.
02:19:26 That would be a shameful waste of resources, it seems.
02:19:31 But it might be that we reside
02:19:33 at one place in parameter space
02:19:35 that has been adapted through an evolutionary process
02:19:40 to allow us to make certain technologies
02:19:43 that allow our particular kind of universe to arise
02:19:47 and sort of achieve the things it does.
02:19:49 See, I wonder if nature in this kind of discussion,
02:19:52 if nature is a catalyst for innovation
02:19:55 or if it’s a ceiling for innovation.
02:19:57 So like, is it going to always limit us?
02:20:00 Like you’re talking about silicon.
02:20:04 Is it just make it super easy to do awesome stuff
02:20:06 in a certain dimension,
02:20:08 but we could still do awesome stuff in other ways,
02:20:10 it’ll just be harder?
02:20:11 Or does it really set like the maximum we can do?
02:20:15 That’s a good thing to,
02:20:17 that’s a good subject to discuss.
02:20:19 I guess I feel like we need to lay
02:20:20 a little bit more groundwork.
02:20:23 So I want to make sure that
02:20:27 I introduce this in the context
02:20:29 of Lee Smolin’s previous idea.
02:20:31 So who’s Lee Smolin and what kind of ideas does he have?
02:20:35 Okay, Lee Smolin is a theoretical physicist
02:20:39 who back in the late 1980s published a paper
02:20:42 in the early 1990s introduced this idea
02:20:45 of cosmological natural selection,
02:20:47 which argues that the universe did evolve.
02:20:51 So his paper was called, did the universe evolve?
02:20:54 And I gave myself the liberty of titling my paper
02:20:59 does cosmological selection
02:21:01 or does cosmological evolution select for technology
02:21:03 in reference to that.
02:21:05 So he introduced that idea decades ago.
02:21:08 Now he primarily works on quantum gravity,
02:21:12 loop quantum gravity, other approaches to
02:21:14 unifying quantum mechanics with general relativity,
02:21:19 as you can read about in his most recent book, I believe,
02:21:22 and he’s been on your show as well.
02:21:24 So, but I want to introduce this idea
02:21:27 of cosmological natural selection,
02:21:29 because I think that is one of the core ideas
02:21:32 that could change our understanding
02:21:35 of how the universe got here, our role in it,
02:21:37 what technology is doing here.
02:21:39 But there’s a couple more pieces
02:21:41 that need to be set up first.
02:21:42 So the beginning of our universe is largely accepted
02:21:46 to be the big bang.
02:21:47 And what that means is if you look back in time
02:21:49 by looking far away in space,
02:21:52 you see that everything used to be at one point
02:21:56 and it expanded away from there.
02:21:58 There was an era in the evolutionary process of our universe
02:22:04 that was called inflation.
02:22:05 And this idea was developed primarily by Alan Guth
02:22:08 and others, Andre Linde and others in the 80s.
02:22:13 And this idea of inflation is basically that
02:22:16 when a singularity begins this process of growth,
02:22:25 there can be a temporary stage
02:22:27 where it just accelerates incredibly rapidly.
02:22:30 And based on quantum field theory,
02:22:33 this tells us that this should produce matter
02:22:35 in precisely the proportions that we find
02:22:37 of hydrogen and helium in the big bang,
02:22:39 lithium too, lithium also, and other things too.
02:22:44 So the predictions that come out of big bang
02:22:47 inflationary cosmology have stood up extremely well
02:22:50 to empirical verification,
02:22:52 the cosmic microwave background, things like this.
02:22:55 So most scientists working in the field
02:22:59 think that the origin of our universe is the big bang.
02:23:03 And I base all my thinking on that as well.
02:23:08 I’m just laying this out there so that people understand
02:23:11 that where I’m coming from is an extension,
02:23:14 not a replacement of existing well founded ideas.
02:23:19 In a paper, I believe it was 1986 with Alan Guth
02:23:23 and another author Farhi,
02:23:26 they wrote that a big bang,
02:23:30 I don’t remember the exact quote,
02:23:31 a big bang is inextricably linked with a black hole.
02:23:35 The singularity that we call our origin
02:23:39 is mathematically indistinguishable from a black hole.
02:23:42 They’re the same thing.
02:23:44 And Lee Smolin based his thinking on that idea,
02:23:48 I believe, I don’t mean to speak for him,
02:23:50 but this is my reading of it.
02:23:52 So what Lee Smolin will say is that
02:23:56 a black hole in one universe is a big bang
02:23:58 in another universe.
02:24:00 And this allows us to have progeny, offspring.
02:24:04 So a universe can be said to have come
02:24:08 before another universe.
02:24:10 And very crucially, Smolin argues,
02:24:14 I think this is potentially one of the great ideas
02:24:16 of all time, that’s my opinion,
02:24:18 that when a black hole forms, it’s not a classical entity,
02:24:22 it’s a quantum gravitational entity.
02:24:24 So it is subject to the fluctuations
02:24:27 that are inherent in quantum mechanics, the properties,
02:24:34 what we’re calling the parameters
02:24:35 that describe the physics of that system
02:24:38 are subject to slight mutations
02:24:40 so that the offspring universe
02:24:42 does not have the exact same parameters
02:24:45 defining its physics as its parent universe.
02:24:48 They’re close, but they’re a little bit different.
02:24:50 And so now you have a mechanism for evolution,
02:24:55 for natural selection.
02:24:57 So there’s mutation, so there’s,
02:24:59 and then if you think about the DNA of the universe
02:25:03 are the basic parameters that govern its laws.
02:25:05 Exactly, so what Smolin said is our universe results
02:25:11 from an evolutionary process that can be traced back
02:25:14 some, he estimated, 200 million generations.
02:25:17 Initially, there was something like a vacuum fluctuation
02:25:20 that produced through random chance a universe
02:25:25 that was able to reproduce just one.
02:25:27 So now it had one offspring.
02:25:28 And then over time, it was able to make more and more
02:25:30 until it evolved into a highly structured universe
02:25:35 with a very long lifetime, with a great deal of complexity
02:25:40 and importantly, especially importantly for Lee Smolin,
02:25:44 stars, stars make black holes.
02:25:47 Therefore, we should expect our universe
02:25:49 to be optimized, have its physical parameters optimized
02:25:53 to make very large numbers of stars
02:25:55 because that’s how you make black holes
02:25:57 and black holes make offspring.
02:25:59 So we expect the physics of our universe to have evolved
02:26:03 to maximize fecundity, the number of offspring.
02:26:06 And the way Lee Smolin argues you do that
02:26:09 is through stars that the biggest ones die
02:26:12 in these core collapse supernova
02:26:13 that make a black hole and a child.
02:26:15 Okay, first of all, I agree with you
02:26:19 that this is back to our fractal view of everything
02:26:24 from intelligence to our universe.
02:26:27 That is very compelling and a very powerful idea
02:26:31 that unites the origin of life
02:26:36 and perhaps the origin of ideas and intelligence.
02:26:39 So from a Dawkins perspective here on earth,
02:26:42 the evolution of those and then the evolution
02:26:45 of the laws of physics that led to us.
02:26:51 I mean, it’s beautiful.
02:26:52 And then you stacking on top of that,
02:26:54 that maybe we are one of the offspring.
02:26:57 Right, okay, so before getting into where I’d like
02:27:02 to take that idea, let me just a little bit more groundwork.
02:27:05 There is this concept of the multiverse
02:27:07 and it can be confusing.
02:27:08 Different people use the word multiverse in different ways.
02:27:11 In the multiverse that I think is relevant to picture
02:27:16 when trying to grasp Lee Smolin’s idea,
02:27:20 essentially every vacuum fluctuation
02:27:24 can be referred to as a universe.
02:27:25 It occurs, it borrows energy from the vacuum
02:27:28 for some finite amount of time
02:27:30 and it evanesces back into the quantum vacuum.
02:27:33 And ideas of Guth before that and Andrei Linde
02:27:38 with eternal inflation aren’t that different
02:27:42 that you would expect nature
02:27:44 due to the quantum properties of the vacuum,
02:27:46 which we know exist, they’re measurable
02:27:49 through things like the Casimir effect and others.
02:27:52 You know that there are these fluctuations
02:27:54 that are occurring.
02:27:55 What Smolin is arguing is that there is
02:27:58 this extensive multiverse, that this universe,
02:28:01 what we can measure and interact with
02:28:04 is not unique in nature.
02:28:07 It’s just our residents, it’s where we reside.
02:28:10 And there are countless, potentially infinity
02:28:13 other universes, other entire evolutionary trajectories
02:28:17 that have evolved into things like
02:28:19 what you were mentioning a second ago
02:28:21 with different parameters and different ways
02:28:24 of achieving complexity and reproduction
02:28:26 and all that stuff.
02:28:27 So it’s not that the evolutionary process
02:28:30 is a funnel towards this end point, not at all.
02:28:34 Just like the biological evolutionary process
02:28:37 that has occurred within our universe
02:28:39 is not a unique route toward achieving
02:28:42 one specific chosen kind of species.
02:28:45 No, we have extraordinary diversity around us.
02:28:49 That’s what evolution does.
02:28:50 And for any one species like us,
02:28:52 you might feel like we’re at the center of this process.
02:28:54 We’re the destination of this process,
02:28:57 but we’re just one of the many
02:28:59 nearly infinite branches of this process.
02:29:02 And I suspect it is exactly infinite.
02:29:04 I mean, I just can’t understand how with this idea,
02:29:09 you can ever draw a boundary around it and say,
02:29:11 no, the universe, I mean, the multiverse
02:29:13 has 10 to the one quadrillion components,
02:29:17 but not infinity.
02:29:18 I don’t know that.
02:29:20 Well, yeah, I have cognitively incapable
02:29:24 as I think all of us are
02:29:25 and truly understanding the concept of infinity.
02:29:29 And the concept of nothing as well.
02:29:31 And nothing, but also the concept of a lot
02:29:34 is pretty difficult.
02:29:35 I can just, I can count.
02:29:37 I run out of fingers at a certain point
02:29:39 and then you’re screwed.
02:29:40 And when you’re wearing shoes
02:29:41 and you can’t even get down to your toes, it’s like.
02:29:44 It’s like, all right, a thousand fine, a million.
02:29:47 Is that what?
02:29:48 And then it gets crazier and crazier.
02:29:50 Right, right.
02:29:51 So this particular, so when we say technology, by the way,
02:29:55 I mean, there’s some, not to over romanticize the thing,
02:30:00 but there is some aspect about this branch of ours
02:30:04 that allows us to, for the universe to know itself.
02:30:08 Yes, yes.
02:30:08 So to be, to have like little conscious cognitive fingers
02:30:15 that are able to feel like to scratch the head.
02:30:18 Right, right, right.
02:30:19 To be able to construct E equals MC squared
02:30:22 and to introspect, to start to gain some understanding
02:30:25 of the laws that govern it.
02:30:27 Isn’t that, isn’t that kind of amazing?
02:30:32 Okay, I’m just human, but it feels like that,
02:30:35 if I were to build a system that does this kind of thing,
02:30:39 that evolves laws of physics, that evolves life,
02:30:42 that evolves intelligence, that my goal would be
02:30:45 to come up with things that are able to think about itself.
02:30:48 Right, aren’t we kind of close to the design specs,
02:30:53 the destination?
02:30:54 We’re pretty close, I don’t know.
02:30:56 I mean, I’m spending my career designing things
02:30:58 that I hope will think about themselves,
02:30:59 so you and I aren’t too far apart on that one.
02:31:02 But then maybe that problem is a lot harder
02:31:05 than we imagine.
02:31:06 Maybe we need to.
02:31:07 Let’s not get, let’s not get too far
02:31:09 because I want to emphasize something that,
02:31:12 what you’re saying is, isn’t it fascinating
02:31:14 that the universe evolved something
02:31:16 that can be conscious, reflect on itself?
02:31:19 But Lee Smolin’s idea didn’t take us there, remember?
02:31:23 It took us to stars.
02:31:25 Lee Smolin has argued, I think,
02:31:29 right on almost every single way
02:31:32 that cosmological natural selection
02:31:35 could lead to a universe with rich structure.
02:31:38 And he argued that the structure,
02:31:41 the physics of our universe is designed
02:31:43 to make a lot of stars so that they can make black holes.
02:31:46 But that doesn’t explain what we’re doing here.
02:31:48 In order for that to be an explanation of us,
02:31:51 what you have to assume is that once you made that universe
02:31:55 that was capable of producing stars,
02:31:58 life, planets, all these other things,
02:32:00 we’re along for the ride.
02:32:01 They got lucky.
02:32:02 We’re kind of arising, growing up in the cracks,
02:32:05 but the universe isn’t here for us.
02:32:06 We’re still kind of a fluke in that picture.
02:32:09 And I can’t, I don’t necessarily have
02:32:12 like a philosophical opposition to that stance.
02:32:14 It’s just not, okay, so I don’t think it’s complete.
02:32:20 So it seems like whatever we got going on here to you,
02:32:22 it seems like whatever we have here on earth
02:32:25 seems like a thing you might want to select for
02:32:28 in this whole big process.
02:32:29 Exactly.
02:32:30 So if what you are truly,
02:32:32 if your entire evolutionary process
02:32:34 only cares about fecundity,
02:32:36 it only cares about making offspring universes
02:32:39 because then there’s gonna be the most of them
02:32:41 in that local region of hyperspace,
02:32:45 which is the set of all possible universes, let’s say.
02:32:50 You don’t care how those universes are made.
02:32:52 You know they have to be made by black holes.
02:32:54 This is what inflationary theory tells us.
02:32:57 The big bang tells us that black holes make universes.
02:33:02 But what if there was a technological means
02:33:04 to make universes?
02:33:05 Stars require a ton of matter
02:33:09 because they’re not thinking very carefully
02:33:11 about how you make a black hole.
02:33:12 They’re just using gravity, you know?
02:33:16 But if we devise technologies
02:33:19 that can efficiently compress matter into a singularity,
02:33:23 it turns out that if you can compress about 10 kilograms
02:33:26 into a very small volume,
02:33:28 that will make a black hole
02:33:29 that is likely highly probable to inflate
02:33:32 into its own offspring universe.
02:33:34 This is according to calculations done by other people
02:33:37 who are professional quantum theorists,
02:33:38 quantum field theorists,
02:33:40 and I hope I am grasping what they’re telling me correctly.
02:33:44 I am somewhat of a translator here.
02:33:47 But so that’s the position
02:33:50 that is particularly intriguing to me,
02:33:52 which is that what might have happened is that,
02:33:56 okay, this particular branch on the vast tree of evolution,
02:34:01 cosmological evolution that we’re talking about,
02:34:03 not biological evolution within our universe,
02:34:05 but cosmological evolution,
02:34:07 went through exactly the process
02:34:09 that Elise Mullen described,
02:34:10 got to the stage where stars were making lots of black holes
02:34:15 but then continued to evolve and somehow bridged that gap
02:34:19 and made intelligence and intelligence
02:34:22 capable of devising technologies
02:34:24 because technologies, intelligent species
02:34:27 working in conjunction with technologies
02:34:29 could then produce even more.
02:34:32 Yeah, more efficiently, more faster and better
02:34:35 and more different.
02:34:36 Then you start to have different kind of mechanisms
02:34:38 and mutation perhaps, all that kind of stuff.
02:34:40 And so if you do a simple calculation that says,
02:34:43 all right, if I want to,
02:34:44 we know roughly how many core collapse supernovae
02:34:50 have resulted in black holes in our galaxy
02:34:54 since the beginning of the universe
02:34:55 and it’s something like a billion.
02:34:57 So then you would have to estimate
02:35:00 that it would be possible for a technological civilization
02:35:04 to produce more than a billion black holes
02:35:07 with the energy and matter at their disposal.
02:35:09 And so one of the calculations in that paper,
02:35:12 back of the envelope,
02:35:14 but I think revealing nonetheless is that
02:35:16 if you take a relatively common asteroid,
02:35:20 something that’s about a kilometer in diameter,
02:35:23 what I’m thinking of is just scrap material
02:35:26 laying around in our solar system
02:35:28 and break it up into 10 kilogram chunks
02:35:31 and turn each of those into a universe,
02:35:33 then you would have made at least a trillion black holes
02:35:38 outpacing the star production rate
02:35:41 by some three orders of magnitude.
02:35:43 That’s one asteroid.
02:35:44 So now if you envision an intelligent species
02:35:46 that would potentially have been devised initially
02:35:50 by humans, but then based on superconducting
02:35:53 optoelectronic networks, no doubt,
02:35:55 and they go out and populate,
02:35:57 they don’t have to fill the galaxy.
02:35:58 They just have to get out to the asteroid belt.
02:36:01 They could potentially dramatically outpace
02:36:05 the rate at which stars are producing offspring universes.
02:36:07 And then wouldn’t you expect that
02:36:10 that’s where we came from instead of a star?
02:36:13 Yeah, so you have to somehow become masters of gravity,
02:36:16 so like, or generate.
02:36:17 John, this is really gravity.
02:36:18 So stars make black holes with gravity,
02:36:20 but any force that can make the energy density
02:36:26 can compactify matter to produce
02:36:28 a great enough energy density can form a singularity.
02:36:31 It doesn’t, it would not likely be gravity.
02:36:33 It’s the weakest force.
02:36:34 You’re more likely to use something like the technologies
02:36:38 that we’re developing for fusion, for example.
02:36:40 So I don’t know, the Large Ignition Facility
02:36:44 recently blasted a pellet with 100 really bright lasers
02:36:50 and caused that to get dense enough
02:36:53 to engage in nuclear fusion.
02:36:55 So something more like that,
02:36:56 or a tokamak with a really hot plasma, I’m not sure.
02:36:59 Something, I don’t know exactly how it would be done.
02:37:02 I do like the idea of that,
02:37:04 especially just been reading a lot about gravitational waves
02:37:07 and the fact that us humans with our technological
02:37:10 capabilities, one of the most impressive
02:37:14 technological accomplishments of human history is LIGO,
02:37:17 being able to precisely detect gravitational waves.
02:37:20 I’m particularly find appealing the idea
02:37:25 that other alien civilizations from very far distances
02:37:29 communicate with gravity, with gravitational waves,
02:37:34 because as you become greater and greater master of gravity,
02:37:37 which seems way out of reach for us right now,
02:37:40 maybe that seems like a effective way of sending signals,
02:37:44 especially if your job is to manufacture black holes.
02:37:48 Right.
02:37:49 So that, so let me ask there,
02:37:53 whatever, I mean, broadly thinking,
02:37:56 because we tend to think other alien civilizations
02:37:58 would be very human like,
02:38:00 but if we think of alien civilizations out there
02:38:04 as basically generators of black holes,
02:38:07 however they do it, because they got stars,
02:38:10 do you think there’s a lot of them
02:38:12 in our particular universe out there?
02:38:17 In our universe?
02:38:20 Well, okay, let me ask, okay, this is great.
02:38:23 Let me ask a very generic question
02:38:26 and then let’s see how you answer it,
02:38:29 which is how many alien civilizations are out there?
02:38:35 If the hypothesis that I just described
02:38:38 is on the right track,
02:38:40 it would mean that the parameters of our universe
02:38:43 have been selected so that intelligent civilizations
02:38:48 will occur in sufficient numbers
02:38:51 so that if they reach something
02:38:54 like supreme technological maturity,
02:38:56 let’s define that as the ability to produce black holes,
02:39:00 then that’s not a highly improbable event.
02:39:02 It doesn’t need to happen often
02:39:05 because as I just described,
02:39:06 if you get one of them in a galaxy,
02:39:09 you’re gonna make more black holes
02:39:10 than the stars in that galaxy.
02:39:12 But there’s also not a super strong motivation,
02:39:16 well, it’s not obvious that you need them
02:39:21 to be ubiquitous throughout the galaxy.
02:39:23 Right.
02:39:24 One of the things that I try to emphasize in that paper
02:39:27 is that given this idea
02:39:30 of how our parameters might’ve been selected,
02:39:35 it’s clear that it’s a series of trade offs, right?
02:39:39 If you make, I mean, in order for intelligent life
02:39:42 of our variety or anything resembling us to occur,
02:39:45 you need a bunch of stuff, you need stars.
02:39:47 So that’s right back to Smolin’s roots of this idea,
02:39:51 but you also need water to have certain properties.
02:39:54 You need things like the rocky planets,
02:39:58 like the Earth to be within the habitable zone,
02:40:00 all these things that you start talking about
02:40:02 in the field of astrobiology,
02:40:06 trying to understand life in the universe,
02:40:08 but you can’t over emphasize,
02:40:10 you can’t tune the parameters so precisely
02:40:13 to maximize the number of stars
02:40:15 or to give water exactly the properties
02:40:18 or to make rocky planets like Earth the most numerous.
02:40:22 You have to compromise on all these things.
02:40:24 And so I think the way to test this idea
02:40:27 is to look at what parameters are necessary
02:40:30 for each of these different subsystems,
02:40:32 and I’ve laid out a few that I think are promising,
02:40:35 there could be countless others,
02:40:36 and see how changing the parameters
02:40:40 makes it more or less likely that stars would form
02:40:43 and have long lifetimes or that rocky planets
02:40:46 in the habitable zone are likely to form,
02:40:48 all these different things.
02:40:49 So we can test how much these things are in a tug of war
02:40:53 with each other, and the prediction would be
02:40:56 that we kind of sit at this central point
02:40:58 where if you move the parameters too much,
02:41:02 stars aren’t stable, or life doesn’t form,
02:41:05 or technology’s infeasible,
02:41:07 because life alone, at least the kind of life
02:41:10 that we know of, cannot make black holes.
02:41:14 We don’t have this, well, I’m speaking for myself,
02:41:16 you’re a very fit and strong person,
02:41:18 but it might be possible for you,
02:41:20 but not for me to compress matter.
02:41:22 So we need these technologies, but we don’t know,
02:41:25 we have not been able to quantify yet
02:41:28 how finely adjusted the parameters would need to be
02:41:33 in order for silicon to have the properties it does.
02:41:35 Okay, this is not directly speaking to what you’re saying,
02:41:37 you’re getting to the Fermi paradox,
02:41:39 which is where are they, where are the life forms out there,
02:41:42 how numerous are they, that sort of thing.
02:41:44 What I’m trying to argue is that
02:41:46 if this framework is on the right track,
02:41:50 a potentially correct explanation for our existence,
02:41:53 we, it doesn’t necessarily predict
02:41:56 that intelligent civilizations are just everywhere,
02:41:59 because even if you just get one of them in a galaxy,
02:42:02 which is quite rare, it could be enough
02:42:05 to dramatically increase the fecundity
02:42:08 of the universe as a whole.
02:42:10 Yeah, and I wonder, once you start generating
02:42:12 the offspring for universes, black holes,
02:42:15 how that has effect on the,
02:42:18 what kind of effect does it have
02:42:19 on the other candidate’s civilizations
02:42:24 within that universe?
02:42:26 Maybe it has a destructive aspect,
02:42:28 or there could be some arguments
02:42:29 about once you have a lot of offspring,
02:42:32 that that just quickly accelerates
02:42:34 to where the other ones can’t even catch up.
02:42:35 It could, but I guess if you want me
02:42:39 to put my chips on the table or whatever,
02:42:42 I think I come down more on the side
02:42:46 that intelligent life civilizations are rare.
02:42:52 And I guess I follow Max Tegmark here.
02:42:57 And also there’s a lot of papers coming out recently
02:43:01 in the field of astrobiology that are seeming to say,
02:43:04 all right, you just work through the numbers
02:43:06 on some modified Drake equation or something like that.
02:43:09 And it looks like it’s not improbable.
02:43:13 You shouldn’t be surprised that an intelligent species
02:43:16 has arisen in our galaxy,
02:43:18 but if you think there’s one the next solar system over,
02:43:20 it’s highly improbable.
02:43:21 So I can see that the number,
02:43:23 the probability of finding a civilization in a galaxy,
02:43:28 maybe it’s most likely that you’re gonna find
02:43:31 one to a hundred or something.
02:43:32 But okay, now it’s really important
02:43:34 to put a time window on that, I think,
02:43:36 because does that mean in the entire lifetime of the galaxy
02:43:40 before it, so for in our case, before we run into Andromeda,
02:43:49 I think it’s highly probable, I shouldn’t say I think,
02:43:53 it’s tempting to believe that it’s highly probable
02:43:56 that in that entire lifetime of your galaxy,
02:44:00 you’re gonna get at least one intelligent species,
02:44:02 maybe thousands or something like that.
02:44:05 But it’s also, I think, a little bit naive to think
02:44:10 that they’re going to coincide in time
02:44:13 and we’ll be able to observe them.
02:44:14 And also, if you look at the span of life on Earth,
02:44:20 the Earth history, it was surprising to me
02:44:24 to kind of look at the amount of time,
02:44:27 first of all, the short amount of time,
02:44:29 there’s no life, it’s surprising.
02:44:31 Life sprang up pretty quickly.
02:44:33 It’s single cell.
02:44:35 But that’s the point I’m trying to make
02:44:36 is like so much of life on Earth
02:44:42 was just like single cell organisms, like most of it.
02:44:45 Most of it was like boring bacteria type of stuff.
02:44:48 Well, bacteria are fascinating, but I take your point.
02:44:50 No, I get it.
02:44:51 I mean, no offense to them.
02:44:52 But this kind of speaking from the perspective
02:44:56 of your paper of something that’s able
02:44:58 to generate technology as we kind of understand it,
02:45:01 that’s a very short moment in time
02:45:03 relative to that full history of life on Earth.
02:45:08 And maybe our universe is just saturated
02:45:12 with bacteria like humans.
02:45:15 Right.
02:45:17 But not the special extra AGI super humans,
02:45:24 that those are very rare.
02:45:25 And once those spring up, everything just goes to like,
02:45:30 it accelerates very quickly.
02:45:33 Yeah, we just don’t have enough data to really say,
02:45:36 but I find this whole subject extremely engaging.
02:45:40 I mean, there’s this concept,
02:45:41 I think it’s called the Rare Earth Hypothesis,
02:45:43 which is that basically stating that,
02:45:46 okay, microbes were here right away
02:45:49 after the Hadian era where we were being bombarded.
02:45:52 Well, after, yeah, bombarded by comets, asteroids,
02:45:54 things like that, and also after the moon formed.
02:45:57 So once things settled down a little bit,
02:45:59 in a few hundred million years,
02:46:02 you have microbes everywhere.
02:46:03 And it could have been, we don’t know exactly
02:46:05 when it could have been remarkably brief that that took.
02:46:08 So it does indicate that, okay,
02:46:10 life forms relatively easily.
02:46:12 I think that alone is sort of a checker on the scale
02:46:15 for the argument that the parameters that allow
02:46:21 even microbial life to form are not just a fluke.
02:46:24 But anyway, that aside, yes,
02:46:27 then there was this long dormant period,
02:46:29 not dormant, things were happening,
02:46:31 but important things were happening
02:46:34 for some two and a half billion years or something
02:46:37 after the metabolic process
02:46:40 that releases oxygen was developed.
02:46:42 Then basically the planet’s just sitting there,
02:46:46 getting more and more oxygenated,
02:46:47 more and more oxygenated until it’s enough
02:46:50 that you can build these large, complex organisms.
02:46:54 And so the Rare Earth Hypothesis would argue
02:46:56 that the microbes are common everywhere
02:47:01 in any planet that’s roughly in the habitable zone
02:47:04 and has some water on it, it’s probably gonna have those.
02:47:06 But then getting to this Cambrian explosion
02:47:09 that happened some between 500 and 600 million years ago,
02:47:13 that’s rare, you know?
02:47:16 And I buy that, I think that is rare.
02:47:19 So if you say how much life is in our galaxy,
02:47:21 I think that’s probably the right answer
02:47:24 is that microbes are everywhere.
02:47:26 Cambrian explosion is extremely rare.
02:47:29 And then, but the Cambrian explosion kind of went like that
02:47:32 where within a couple of tens or a hundred million years,
02:47:38 all of these body plans came into existence.
02:47:40 And basically all of the body plans
02:47:43 that are now in existence on the planet
02:47:46 were formed in that brief window
02:47:48 and we’ve just been shuffling around since then.
02:47:51 So then what caused humans to pop out of that?
02:47:54 I mean, that could be another extremely rare threshold
02:48:01 that a planet roughly in the habitable zone with water
02:48:06 is not guaranteed to cross, you know?
02:48:08 To me, it’s fascinating for being humble,
02:48:10 like the humans cannot possibly be the most amazing thing
02:48:13 that such, if you look at the entirety of the system
02:48:15 that Lee Smolin and you paint,
02:48:17 that cannot possibly be the most amazing thing
02:48:20 that process generates.
02:48:21 So like, if you look at the evolution,
02:48:23 what’s the equivalent in the cosmological evolution
02:48:27 and its selection for technology,
02:48:29 the equivalent of the human eye or the human brain?
02:48:32 Universes that are able to do some like,
02:48:35 they don’t need the damn stars.
02:48:37 They’re able to just do some incredible generation
02:48:42 of complexity fast, like much more than,
02:48:46 if you think about it,
02:48:47 it’s like most of our universe is pretty freaking boring.
02:48:50 There’s not much going on, there’s a few rocks flying around
02:48:53 and there’s some like apes
02:48:54 that are just like doing podcasts on some weird planet.
02:49:00 It just seems very inefficient.
02:49:02 If you think about like the amazing thing in the human eye,
02:49:05 the visual cortex can do, the brain, the nervous,
02:49:09 everything that makes us more powerful
02:49:12 than single cell organisms.
02:49:15 Like if there’s an equivalent of that for universes,
02:49:19 like the richness of physics
02:49:21 that could be expressed
02:49:24 through a particular set of parameters.
02:49:26 Like, I mean, like for me,
02:49:31 I’m a sort of from a computer science perspective,
02:49:33 huge fan of cellular automata,
02:49:35 which is a nice sort of pretty visual way
02:49:39 to illustrate how different laws
02:49:42 can result in drastically different levels of complexity.
02:49:46 So like, it’s like, yeah, okay.
02:49:49 So we’re all like celebrating,
02:49:50 look, our little cellular automata
02:49:52 is able to generate pretty triangles and squares
02:49:54 and therefore we achieve general intelligence.
02:49:57 And then there’ll be like some badass Chuck Norris type,
02:50:01 like universal Turing machine type of cellular automata.
02:50:06 They’re able to generate other cellular automata
02:50:09 that does any arbitrary level of computation off the bat.
02:50:14 Like those have to then exist.
02:50:16 And then we’re just like, we’ll be forgotten.
02:50:19 This story, this podcast just entertains
02:50:23 a few other apes for a few months.
02:50:26 Well, I’m kind of surprised to hear your cynicism.
02:50:30 No, I’m very up.
02:50:32 I usually think of you as like one who celebrates humanity
02:50:36 and all its forms and things like that.
02:50:37 And I guess I just, I don’t,
02:50:39 I see it the way you just described.
02:50:41 I mean, okay, we’ve been here for 13.7 billion years
02:50:44 and you’re saying, gosh, that’s a long time.
02:50:47 Let’s get on with the show already.
02:50:48 Some other universe could have kicked our butt by now,
02:50:51 but that’s putting a characteristic time.
02:50:55 I mean, why is 13.7 billion a long time?
02:50:58 I mean, compared to what?
02:51:00 I guess, so when I look at our universe,
02:51:02 I see this extraordinary hierarchy
02:51:05 that has developed over that time.
02:51:08 So at the beginning, it was a chaotic mess of some plasma
02:51:13 and nothing interesting going on there.
02:51:16 And even for the first stars to form,
02:51:18 that a lot of really interesting evolutionary processes
02:51:23 had to occur, by evolutionary in that sense,
02:51:26 I just mean taking place over extended periods of time
02:51:30 and structures are forming then.
02:51:32 And then it took that first generation of stars
02:51:34 in order to produce the metals
02:51:38 that then can more efficiently produce
02:51:41 another generation of stars.
02:51:42 We’re only the third generation of stars.
02:51:44 So we might still be pretty quick to the game here.
02:51:47 So, but I don’t think, I don’t, okay.
02:51:51 So then you have these stars
02:51:52 and then you have solar systems on those solar systems.
02:51:54 You have rocky worlds, you have gas giants,
02:51:58 like all this complexity.
02:51:59 And then you start getting life
02:52:01 and the complexity that’s evolved
02:52:03 through the evolutionary process in life forms
02:52:06 is just, it’s not a let down to me.
02:52:09 Just seeing that.
02:52:10 Some of it is like some of the planets is like icy,
02:52:14 it’s like different flavors of ice cream.
02:52:16 They’re icy, but there might be water underneath.
02:52:18 All kinds of life forms with some volcanoes,
02:52:21 all kinds of weird stuff.
02:52:22 No, no, I don’t, I think it’s beautiful.
02:52:24 I think our life is beautiful.
02:52:25 And I think it was designed that by design,
02:52:29 the scarcity of the whole thing.
02:52:31 I think mortality, as terrifying as it is,
02:52:33 is fundamental to the whole reason we enjoy everything.
02:52:37 No, I think it’s beautiful.
02:52:38 I just think that all of us conscious beings
02:52:42 in the grand scheme of basically every scale
02:52:45 will be completely forgotten.
02:52:46 Well, that’s true.
02:52:47 I think everything is transient
02:52:49 and that would go back to maybe something more like Lao Tzu,
02:52:52 the Tao Te Ching or something where it’s like,
02:52:55 yes, there is nothing but change.
02:52:57 There is nothing but emergence and dissolve and that’s it.
02:53:00 But I just, in this picture,
02:53:03 this hierarchy that’s developed,
02:53:04 I don’t mean to say that now it gets to us
02:53:06 and that’s the pinnacle.
02:53:07 In fact, I think at a high level,
02:53:10 the story I’m trying to tease out in my research is about,
02:53:14 okay, well, so then what’s the next level of hierarchy?
02:53:17 And if it’s, okay, we’re kind of pretty smart.
02:53:21 I mean, talking about people like Lee Small
02:53:23 and Alan Guth, Max Tegmark, okay, we’re really smart.
02:53:26 Talking about me, okay, we’re kind of,
02:53:28 we can find our way to the grocery store or whatever,
02:53:30 but what’s next?
02:53:33 I mean, what if there’s another level of hierarchy
02:53:36 that grows on top of us
02:53:37 that is even more profoundly capable?
02:53:40 And I mean, we’ve talked a lot
02:53:42 about superconducting sensors.
02:53:43 Imagine these cognitive systems far more capable than us
02:53:48 residing somewhere else in the solar system
02:53:52 off of the surface of the earth,
02:53:53 where it’s much darker, much colder,
02:53:55 much more naturally suited to them.
02:53:57 And they have these sensors that can detect single photons
02:54:00 of light from radio waves out to all across the spectrum
02:54:04 of the gamma rays and just see the whole universe.
02:54:07 And they just live in space
02:54:08 with these massive collection optics so that they,
02:54:12 what do they do?
02:54:13 They just look out and experience that vast array
02:54:18 of what’s being developed.
02:54:22 And if you’re such a system,
02:54:25 presumably you would do some things for fun.
02:54:28 And the kind of fun thing I would do
02:54:31 as somebody who likes video games
02:54:33 is I would create and maintain
02:54:37 and observe something like earth.
02:54:42 So in some sense, we’re like all what players on a stage
02:54:47 for this superconducting cold computing system out there.
02:54:54 I mean, all of this is fascinating to think.
02:54:56 The fact that you’re actually designing systems
02:54:59 here on earth that are trying to push this technological
02:55:01 at the very cutting edge and also thinking about
02:55:04 how does the like the evolution of physical laws
02:55:09 lead us to the way we are is fascinating.
02:55:14 That coupling is fascinating.
02:55:15 It’s like the ultimate rigorous application of philosophy
02:55:20 to the rigorous application of engineering.
02:55:23 So Jeff, you’re one of the most fascinating.
02:55:26 I’m so glad I did not know much about you
02:55:29 except through your work.
02:55:30 And I’m so glad we got this chance to talk.
02:55:34 You’re one of the best explainers
02:55:37 of exceptionally difficult concepts.
02:55:40 And you’re also, speaking of like fractal,
02:55:44 you’re able to function intellectually
02:55:46 at all levels of the stack, which I deeply appreciate.
02:55:50 This was really fun.
02:55:51 You’re a great educator, a great scientist.
02:55:53 It’s an honor that you would spend
02:55:56 your valuable time with me.
02:55:57 It’s an honor that you would spend your time with me as well.
02:56:00 Thanks, Jeff.
02:56:01 Thanks for listening to this conversation
02:56:03 with Jeff Schoenlein.
02:56:05 To support this podcast,
02:56:06 please check out our sponsors in the description.
02:56:09 And now let me leave you with some words
02:56:12 from the great John Carmack,
02:56:14 who surely will be a guest on this podcast soon.
02:56:18 Because of the nature of Moore’s Law,
02:56:20 anything that an extremely clever graphics programmer
02:56:22 can do at one point can be replicated
02:56:26 by a merely competent programmer
02:56:27 some number of years later.
02:56:30 Thank you for listening and hope to see you next time.