Jeffrey Shainline: Neuromorphic Computing and Optoelectronic Intelligence #225

Transcript

00:00:00 The following is a conversation with Jeff Schoenlein,

00:00:03 a scientist at NIST

00:00:04 interested in optoelectronic intelligence.

00:00:08 We have a deep technical dive into computing hardware

00:00:12 that will make Jim Keller proud.

00:00:14 I urge you to hop onto this rollercoaster ride

00:00:17 through neuromorphic computing

00:00:19 and superconducting electronics

00:00:21 and hold on for dear life.

00:00:24 Jeff is a great communicator of technical information

00:00:27 and so it was truly a pleasure to talk to him

00:00:30 about some physics and engineering.

00:00:33 To support this podcast,

00:00:34 please check out our sponsors in the description.

00:00:37 This is the Lex Friedman Podcast

00:00:39 and here is my conversation with Jeff Schoenlein.

00:00:44 I got a chance to read a fascinating paper you authored

00:00:48 called Optoelectronic Intelligence.

00:00:52 So maybe we can start by talking about this paper

00:00:55 and start with the basic questions.

00:00:57 What is optoelectronic intelligence?

00:01:00 Yeah, so in that paper,

00:01:02 the concept I was trying to describe

00:01:04 is sort of an architecture

00:01:06 for building brain inspired computing

00:01:10 that leverages light for communication

00:01:13 in conjunction with electronic circuits for computation.

00:01:17 In that particular paper,

00:01:18 a lot of the work we’re doing right now

00:01:20 in our project at NIST

00:01:22 is focused on superconducting electronics for computation.

00:01:25 I won’t go into why that is,

00:01:27 but that might make a little more sense in context

00:01:31 if we first describe what that is in contrast to,

00:01:35 which is semiconducting electronics.

00:01:37 So is it worth taking a couple minutes

00:01:39 to describe semiconducting electronics?

00:01:42 It might even be worthwhile to step back

00:01:45 and talk about electricity and circuits

00:01:49 and how circuits work

00:01:52 before we talk about superconductivity.

00:01:54 Right, okay.

00:01:56 How does a computer work, Jeff?

00:01:58 Well, I won’t go into everything

00:01:59 that makes a computer work,

00:02:01 but let’s talk about the basic building blocks,

00:02:05 a transistor, and even more basic than that,

00:02:08 a semiconductor material, silicon, say.

00:02:11 So in silicon, silicon is a semiconductor,

00:02:15 and what that means is at low temperature,

00:02:18 there are no free charges,

00:02:20 no free electrons that can move around.

00:02:22 So when you talk about electricity,

00:02:24 you’re talking about predominantly electrons

00:02:28 moving to establish electrical currents,

00:02:30 and they move under the influence of voltages.

00:02:33 So you apply voltages, electrons move around,

00:02:36 those can be measured as currents,

00:02:38 and you can represent information in that way.

00:02:40 So semiconductors are special

00:02:43 in the sense that they are really malleable.

00:02:46 So if you have a semiconductor material,

00:02:50 you can change the number of free electrons

00:02:52 that can move around by putting different elements,

00:02:56 different atoms in lattice sites.

00:02:58 So what is a lattice site?

00:03:00 Well, a semiconductor is a crystal,

00:03:02 which means all the atoms that comprise the material

00:03:06 are at exact locations

00:03:09 that are perfectly periodic in space.

00:03:10 So if you start at any one atom

00:03:12 and you go along what are called the lattice vectors,

00:03:14 you get to another atom and another atom and another atom,

00:03:17 and for high quality devices,

00:03:19 it’s important that it’s a perfect crystal

00:03:21 with very few defects,

00:03:23 but you can intentionally replace a silicon atom

00:03:27 with say a phosphorus atom,

00:03:29 and then you can change the number of free electrons

00:03:32 that are in a region of space

00:03:33 that has that excess of what are called dopants.

00:03:37 So picture a device that has a left terminal

00:03:40 and a right terminal,

00:03:42 and if you apply a voltage between those two,

00:03:44 you can cause electrical current to flow between them.

00:03:47 Now we add a third terminal up on top there,

00:03:52 and depending on the voltage

00:03:53 between the left and right terminal and that third voltage,

00:03:57 you can change that current.

00:03:58 So what’s commonly done in digital electronic circuits

00:04:02 is to leave a fixed voltage from left to right,

00:04:06 and then change that voltage

00:04:08 that’s applied at what’s called the gate,

00:04:10 the gate of the transistor.

00:04:11 So what you do is you make it to where

00:04:13 there’s an excess of electrons on the left,

00:04:15 excess of electrons on the right,

00:04:17 and very few electrons in the middle,

00:04:18 and you do this by changing the concentration

00:04:21 of different dopants in the lattice spatially.

00:04:24 And then when you apply a voltage to that gate,

00:04:27 you can either cause current to flow or turn it off,

00:04:30 and so that’s sort of your zero and one.

00:04:33 If you apply voltage, current can flow,

00:04:35 that current is representing a digital one,

00:04:38 and from that, from that basic element,

00:04:41 you can build up all the complexity

00:04:44 of digital electronic circuits

00:04:45 that have really had a profound influence on our society.

00:04:50 Now you’re talking about electrons.

00:04:51 Can you give a sense of what scale we’re talking about

00:04:54 when we’re talking about in silicon

00:04:57 being able to mass manufacture these kinds of gates?

00:05:01 Yeah, so scale in a number of different senses.

00:05:04 Well, at the scale of the silicon lattice,

00:05:07 the distance between two atoms there is half a nanometer.

00:05:10 So people often like to compare these things

00:05:14 to the width of a human hair.

00:05:16 I think it’s some six orders of magnitude smaller

00:05:20 than the width of a human hair, something on that order.

00:05:24 So remarkably small,

00:05:25 we’re talking about individual atoms here,

00:05:27 and electrons are of that length scale

00:05:29 when they’re in that environment.

00:05:31 But there’s another sense

00:05:32 that scale matters in digital electronics.

00:05:34 This is perhaps the more important sense,

00:05:36 although they’re related.

00:05:37 Scale refers to a number of things.

00:05:41 It refers to the size of that transistor.

00:05:44 So for example, I said you have a left contact,

00:05:47 a right contact, and some space between them

00:05:49 where the gate electrode sits.

00:05:52 That’s called the channel width or the channel length.

00:05:56 And what has enabled what we think of as Moore’s law

00:06:00 or the continued increased performance

00:06:03 in silicon microelectronic circuits

00:06:05 is the ability to make that size, that feature size,

00:06:08 ever smaller, ever smaller at a really remarkable pace.

00:06:14 I mean, that feature size has decreased consistently

00:06:20 every couple of years since the 1960s.

00:06:23 And that was what Moore predicted in the 1960s.

00:06:27 He thought it would continue for at least two more decades,

00:06:29 and it’s been much longer than that.

00:06:30 And so that is why we’ve been able to fit ever more devices,

00:06:35 ever more transistors, ever more computational power

00:06:39 on essentially the same size of chip.

00:06:41 So a user sits back and does essentially nothing.

00:06:44 You’re running the same computer program,

00:06:45 but those devices are getting smaller, so they get faster,

00:06:48 they get more energy efficient,

00:06:50 and all of our computing performance

00:06:51 just continues to improve.

00:06:53 And we don’t have to think too hard

00:06:56 about what we’re doing as, say,

00:06:59 a software designer or something like that.

00:07:00 I absolutely don’t mean to say

00:07:02 that there’s no innovation in software or the user side

00:07:05 of things, of course there is.

00:07:07 But from the hardware perspective,

00:07:09 we just have been given this gift

00:07:12 of continued performance improvement

00:07:14 through this scaling that is ever smaller feature sizes

00:07:19 with very similar, say, power consumption.

00:07:22 That power consumption has not continued to scale

00:07:25 in the most recent decades, but nevertheless,

00:07:29 we had a really good run there for a while.

00:07:31 And now we’re down to gates that are seven nanometers,

00:07:34 which is state of the art right now.

00:07:36 Maybe GlobalFoundries is trying to push it

00:07:38 even lower than that.

00:07:39 I can’t keep up with where the predictions are

00:07:42 that it’s gonna end.

00:07:43 But seven nanometer transistor has just a few tens of atoms

00:07:49 along the length of the conduction pathway.

00:07:51 So a naive semiconductor device physicist

00:07:56 would think you can’t go much further than that

00:07:58 without some kind of revolution in the way we think

00:08:02 about the physics of our devices.

00:08:03 Is there something to be said

00:08:04 about the mass manufacture of these devices?

00:08:08 Right, right, so that’s another thing.

00:08:09 So how have we been able

00:08:10 to make those transistors smaller and smaller?

00:08:13 Well, companies like Intel, GlobalFoundries,

00:08:17 they invest a lot of money in the lithography.

00:08:20 So how are these chips actually made?

00:08:22 Well, one of the most important steps

00:08:24 is this what’s called ion implantation.

00:08:27 So you start with sort of a pristine silicon crystal

00:08:31 and then using photolithography,

00:08:34 which is a technique where you can pattern

00:08:36 different shapes using light,

00:08:38 you can define which regions of space

00:08:41 you’re going to implant with different species of ions

00:08:45 that are going to change

00:08:46 the local electrical properties right there.

00:08:49 So by using ever shorter wavelengths of light

00:08:52 and different kinds of optical techniques

00:08:54 and different kinds of lithographic techniques,

00:08:56 things that go far beyond my knowledge base,

00:09:00 you can just simply shrink that feature size down.

00:09:03 And you say you’re at seven nanometers.

00:09:04 Well, the wavelength of light that’s being used

00:09:07 is over a hundred nanometers.

00:09:08 That’s already deep in the UV.

00:09:10 So how are those minute features patterned?

00:09:14 Well, there’s an extraordinary amount of innovation

00:09:16 that has gone into that,

00:09:18 but nevertheless, it stayed very consistent

00:09:20 in this ever shrinking feature size.

00:09:21 And now the question is, can you make it smaller?

00:09:24 And even if you do, do you still continue

00:09:26 to get performance improvements?

00:09:28 But that’s another kind of scaling

00:09:30 where these companies have been able to…

00:09:34 So, okay, you picture a chip that has a processor on it.

00:09:36 Well, that chip is not made as a chip.

00:09:38 It’s made on a wafer.

00:09:40 And using photolithography,

00:09:43 you basically print the same pattern on different dyes

00:09:47 all across the wafer, multiple layers,

00:09:49 tens, probably a hundred some layers

00:09:53 in a mature foundry process.

00:09:55 And you do this on ever bigger wafers too.

00:09:57 That’s another aspect of scaling

00:09:58 that’s occurred in the last several decades.

00:10:00 So now you have this 300 millimeter wafer.

00:10:02 It’s like as big as a pizza

00:10:04 and it has maybe a thousand processors on it.

00:10:06 And then you dice that up using a saw.

00:10:08 And now you can sell these things so cheap

00:10:11 because the manufacturing process was so streamlined.

00:10:14 I think a technology as revolutionary

00:10:17 as silicon microelectronics has to have

00:10:19 that kind of manufacturing scalability,

00:10:23 which I will just emphasize,

00:10:25 I believe is enabled by physics.

00:10:28 It’s not, I mean, of course there’s human ingenuity

00:10:31 that goes into it, but at least from my side where I sit,

00:10:35 it sure looks like the physics of our universe

00:10:38 allows us to produce that.

00:10:40 And we’ve discovered how more so than we’ve invented it,

00:10:45 although of course we have invented it,

00:10:47 humans have invented it,

00:10:48 but it’s almost as if it was there

00:10:50 waiting for us to discover it.

00:10:52 You mean the entirety of it

00:10:54 or are you specifically talking about

00:10:55 the techniques of photolithography,

00:10:58 like the optics involved?

00:10:59 I mean, the entirety of the scaling down

00:11:02 to the seven nanometers,

00:11:04 you’re able to have electrons not interfere with each other

00:11:08 in such a way that you could still have gates.

00:11:11 Like that’s enabled.

00:11:13 To achieve that scale, spatial and temporal,

00:11:16 it seems to be very special

00:11:18 and is enabled by the physics of our world.

00:11:21 All of the things you just said.

00:11:22 So starting with the silicon material itself,

00:11:25 silicon is a unique semiconductor.

00:11:28 It has essentially ideal properties

00:11:31 for making a specific kind of transistor

00:11:33 that’s extraordinarily useful.

00:11:35 So I mentioned that silicon has,

00:11:39 well, when you make a transistor,

00:11:40 you have this gate contact

00:11:42 that sits on top of the conduction channel.

00:11:44 And depending on the voltage you apply there,

00:11:47 you pull more carriers into the conduction channel

00:11:50 or push them away so it becomes more or less conductive.

00:11:53 In order to have that work

00:11:54 without just sucking those carriers right into that contact,

00:11:57 you need a very thin insulator.

00:11:59 And part of scaling has been to gradually decrease

00:12:03 the thickness of that gate insulator

00:12:06 so that you can use a roughly similar voltage

00:12:09 and still have the same current voltage characteristics.

00:12:12 So the material that’s used to do that,

00:12:14 or I should say was initially used to do that

00:12:16 was a silicon dioxide,

00:12:18 which just naturally grows on the silicon surface.

00:12:21 So you expose silicon to the atmosphere that we breathe

00:12:25 and well, if you’re manufacturing,

00:12:27 you’re gonna purify these gases,

00:12:29 but nevertheless,

00:12:30 that what’s called a native oxide will grow there.

00:12:33 There are essentially no other materials

00:12:36 on the entire periodic table

00:12:37 that have as good of a gate insulator

00:12:42 as that silicon dioxide.

00:12:43 And that has to do with nothing but the physics

00:12:46 of the interaction between silicon and oxygen.

00:12:49 And if it wasn’t that way,

00:12:51 transistors could not perform

00:12:54 in nearly the degree of capability that they have.

00:12:58 And that has to do with the way that the oxide grows,

00:13:02 the reduced density of defects there,

00:13:05 it’s insulation, meaning essentially it’s energy gaps.

00:13:08 You can apply a very large voltage there

00:13:10 without having current leak through it.

00:13:12 So that’s physics right there.

00:13:15 There are other things too.

00:13:16 Silicon is a semiconductor in an elemental sense.

00:13:19 You only need silicon atoms.

00:13:21 A lot of other semiconductors,

00:13:22 you need two different kinds of atoms,

00:13:24 like a compound from group three

00:13:26 and a compound from group five.

00:13:28 That opens you up to lots of defects that can occur

00:13:31 where one atom’s not sitting quite at the lattice site,

00:13:34 it is and it’s switched with another one

00:13:35 that degrades performance.

00:13:38 But then also on the side that you mentioned

00:13:40 with the manufacturing,

00:13:43 we have access to light sources

00:13:45 that can produce these very short wavelengths of light.

00:13:49 How does photolithography occur?

00:13:50 Well, you actually put this polymer on top of your wafer

00:13:54 and you expose it to light,

00:13:56 and then you use a aqueous chemical processing

00:14:00 to dissolve away the regions that were exposed to light

00:14:03 and leave the regions that were not.

00:14:05 And we are blessed with these polymers

00:14:08 that have the right property

00:14:09 where they can cause scission events

00:14:13 where the polymer splits where a photon hits.

00:14:16 I mean, maybe that’s not too surprising,

00:14:19 but I don’t know, it all comes together

00:14:21 to have this really complex,

00:14:24 manufacturable ecosystem

00:14:26 where very sophisticated technologies can be devised

00:14:30 and it works quite well.

00:14:33 And amazingly, like you said,

00:14:34 with a wavelength at like 100 nanometers

00:14:36 or something like that,

00:14:37 you’re still able to achieve on this polymer

00:14:39 precision of whatever we said, seven nanometers.

00:14:43 I think I’ve heard like four nanometers

00:14:45 being talked about, something like that.

00:14:48 If we could just pause on this

00:14:49 and we’ll return to superconductivity,

00:14:52 but in this whole journey from a history perspective,

00:14:56 what do you think is the most beautiful

00:14:59 at the intersection of engineering and physics

00:15:01 to you in this whole process

00:15:03 that we talked about with silicon and photolithography,

00:15:06 things that people were able to achieve

00:15:08 in order to push Moore’s law forward?

00:15:12 Is it the early days,

00:15:13 the invention of the transistor itself?

00:15:16 Is it some particular cool little thing

00:15:19 that maybe not many people know about?

00:15:21 Like, what do you think is most beautiful

00:15:24 in this whole process, journey?

00:15:26 The most beautiful is a little difficult to answer.

00:15:29 Let me try and sidestep it a little bit

00:15:32 and just say what strikes me about looking

00:15:35 at the history of silicon microelectronics is that,

00:15:42 so when quantum mechanics was developed,

00:15:44 people quickly began applying it to semiconductors

00:15:47 and it was broadly understood

00:15:49 that these are fascinating systems

00:15:50 and people cared about them for their basic physics,

00:15:52 but also their utility as devices.

00:15:55 And then the transistor was invented in the late forties

00:15:59 in a relatively crude experimental setup

00:16:02 where you just crammed a metal electrode

00:16:04 into the semiconductor and that was ingenious.

00:16:08 These people were able to make it work.

00:16:13 But so what I wanna get to that really strikes me

00:16:16 is that in those early days,

00:16:19 there were a number of different semiconductors

00:16:21 that were being considered.

00:16:22 They had different properties, different strengths,

00:16:23 different weaknesses.

00:16:24 Most people thought germanium was the way to go.

00:16:28 It had some nice properties related to things

00:16:33 about how the electrons move inside the lattice.

00:16:37 But other people thought that compound semiconductors

00:16:39 with group three and group five also had

00:16:42 really, really extraordinary properties

00:16:46 that might be conducive to making the best devices.

00:16:50 So there were different groups exploring each of these

00:16:52 and that’s great, that’s how science works.

00:16:54 You have to cast a broad net.

00:16:56 But then what I find striking is why is it that silicon won?

00:17:02 Because it’s not that germanium is a useless material

00:17:05 and it’s not present in technology

00:17:06 or compound semiconductors.

00:17:08 They’re both doing exciting and important things,

00:17:12 slightly more niche applications

00:17:14 whereas silicon is the semiconductor material

00:17:18 for microelectronics which is the platform

00:17:20 for digital computing which has transformed our world.

00:17:22 Why did silicon win?

00:17:24 It’s because of a remarkable assemblage of qualities

00:17:28 that no one of them was the clear winner

00:17:32 but it made these sort of compromises

00:17:34 between a number of different influences.

00:17:36 It had that really excellent gate oxide

00:17:40 that allowed us to make MOSFETs,

00:17:43 these high performance transistors,

00:17:45 so quickly and cheaply and easily

00:17:47 without having to do a lot of materials development.

00:17:49 The band gap of silicon is actually,

00:17:53 so in a semiconductor there’s an important parameter

00:17:56 which is called the band gap

00:17:57 which tells you there are sort of electrons

00:18:00 that fill up to one level in the energy diagram

00:18:04 and then there’s a gap where electrons aren’t allowed

00:18:06 to have an energy in a certain range

00:18:08 and then there’s another energy level above that.

00:18:11 And that difference between the lower sort of filled level

00:18:14 and the unoccupied level,

00:18:16 that tells you how much voltage you have to apply

00:18:19 in order to induce a current to flow.

00:18:22 So with germanium, that’s about 0.75 electron volts.

00:18:27 That means you have to apply 0.75 volts

00:18:29 to get a current moving.

00:18:32 And it turns out that if you compare that

00:18:34 to the thermal excitations that are induced

00:18:38 just by the temperature of our environment,

00:18:40 that gap’s not quite big enough.

00:18:42 You start to use it to perform computations,

00:18:45 it gets a little hot and you get all these accidental

00:18:47 carriers that are excited into the conduction band

00:18:50 and it causes errors in your computation.

00:18:53 Silicon’s band gap is just a little higher,

00:18:56 1.1 electron volts,

00:18:58 but you have an exponential dependence

00:19:01 on the number of carriers that are present

00:19:04 that can induce those errors.

00:19:06 It decays exponentially with that voltage.

00:19:08 So just that slight extra energy in that band gap

00:19:12 really puts it in an ideal position to be operated

00:19:17 in the conditions of our ambient environment.

00:19:20 It’s kind of fascinating that, like you mentioned,

00:19:22 errors decrease exponentially with the voltage.

00:19:27 So it’s funny because this error thing comes up

00:19:32 when you start talking about quantum computing.

00:19:34 And it’s kind of amazing that everything

00:19:36 we’ve been talking about, the errors,

00:19:37 as we scale down, seems to be extremely low.

00:19:41 Yes.

00:19:42 And like all of our computation is based

00:19:45 on the assumption that it’s extremely low.

00:19:47 Yes, well it’s digital computation.

00:19:49 Digital, sorry, digital computation.

00:19:51 So as opposed to our biological computation in our brain,

00:19:55 is like the assumption is stuff is gonna fail

00:19:58 all over the place and we somehow

00:19:59 have to still be robust to that.

00:20:01 That’s exactly right.

00:20:03 So this also, this is gonna be the most controversial part

00:20:05 of our conversation where you’re gonna make some enemies.

00:20:07 So let me ask,

00:20:09 because we’ve been talking about physics and engineering.

00:20:14 Which group of people is smarter

00:20:15 and more important for this one?

00:20:17 Let me ask the question in a better way.

00:20:20 Some of the big innovations,

00:20:22 some of the beautiful things that we’ve been talking about,

00:20:25 how much of it is physics?

00:20:26 How much of it is engineering?

00:20:28 My dad is a physicist and he talks down

00:20:31 to all the amazing engineering that we’re doing

00:20:34 in the artificial intelligence and the computer science

00:20:37 and the robotics and all that space.

00:20:39 So we argue about this all the time.

00:20:41 So what do you think?

00:20:42 Who gets more credit?

00:20:43 I’m genuinely not trying to just be politically correct here.

00:20:46 I don’t see how you would have any of the,

00:20:50 what we consider sort of the great accomplishments

00:20:52 of society without both.

00:20:54 You absolutely need both of those things.

00:20:55 Physics tends to play a key role earlier in the development

00:20:59 and then engineering optimization, these things take over.

00:21:04 And I mean, the invention of the transistor

00:21:09 or actually even before that,

00:21:10 the understanding of semiconductor physics

00:21:12 that allowed the invention of the transistor,

00:21:14 that’s all physics.

00:21:15 So if you didn’t have that physics,

00:21:17 you don’t even get to get on the field.

00:21:20 But once you have understood and demonstrated

00:21:24 that this is in principle possible,

00:21:26 more so as engineering.

00:21:28 Why we have computers more powerful

00:21:32 than old supercomputers in each of our phones,

00:21:36 that’s all engineering.

00:21:37 And I think I would be quite foolish to say

00:21:41 that that’s not valuable, that’s not a great contribution.

00:21:46 It’s a beautiful dance.

00:21:47 Would you put like Silicon,

00:21:49 the understanding of the material properties

00:21:52 in the space of engineering?

00:21:54 Like how does that whole process work?

00:21:55 To understand that it has all these nice properties

00:21:58 or even the development of photolithography,

00:22:02 is that basically,

00:22:03 would you put that in a category of engineering?

00:22:06 No, I would say that it is basic physics,

00:22:09 it is applied physics, it’s material science,

00:22:12 it’s X ray crystallography, it’s polymer chemistry,

00:22:17 it’s everything.

00:22:18 Chemistry even is thrown in there?

00:22:20 Absolutely, yes, absolutely.

00:22:22 Just no biology.

00:22:25 We can get to biology.

00:22:26 Or the biologies and the humans

00:22:28 that are engineering the system,

00:22:29 so it’s all integrated deeply.

00:22:31 Okay, so let’s return,

00:22:32 you mentioned this word superconductivity.

00:22:35 So what does that have to do with what we’re talking about?

00:22:38 Right, okay, so in a semiconductor,

00:22:40 as I tried to describe a second ago,

00:22:44 you can sort of induce currents by applying voltages

00:22:50 and those have sort of typical properties

00:22:52 that you would expect from some kind of a conductor.

00:22:55 Those electrons, they don’t just flow

00:22:59 perfectly without dissipation.

00:23:00 If an electron collides with an imperfection in the lattice

00:23:03 or another electron, it’s gonna slow down,

00:23:05 it’s gonna lose its momentum.

00:23:06 So you have to keep applying that voltage

00:23:09 in order to keep the current flowing.

00:23:11 In a superconductor, something different happens.

00:23:13 If you get a current to start flowing,

00:23:16 it will continue to flow indefinitely.

00:23:18 There’s no dissipation.

00:23:19 So that’s crazy.

00:23:21 How does that happen?

00:23:22 Well, it happens at low temperature and this is crucial.

00:23:26 It has to be a quite low temperature

00:23:30 and what I’m talking about there,

00:23:32 for essentially all of our conversation,

00:23:35 I’m gonna be talking about conventional superconductors,

00:23:39 sometimes called low TC superconductors,

00:23:42 low critical temperature superconductors.

00:23:45 And so those materials have to be at a temperature around,

00:23:50 say around four Kelvin.

00:23:51 I mean, their critical temperature might be 10 Kelvin,

00:23:54 something like that,

00:23:55 but you wanna operate them at around four Kelvin,

00:23:57 four degrees above absolute zero.

00:23:59 And what happens at that temperature,

00:24:01 at very low temperatures in certain materials

00:24:03 is that the noise of atoms moving around,

00:24:10 the lattice vibrating, electrons colliding with each other,

00:24:13 that becomes sufficiently low

00:24:15 that the electrons can settle into this very special state.

00:24:18 It’s sometimes referred to as a macroscopic quantum state

00:24:22 because if I had a piece of superconducting material here,

00:24:26 let’s say niobium is a very typical superconductor.

00:24:30 If I had a block of niobium here

00:24:33 and we cooled it below its critical temperature,

00:24:36 all of the electrons in that superconducting state

00:24:40 would be in one coherent quantum state.

00:24:42 The wave function of that state is described

00:24:47 in terms of all of the particles simultaneously,

00:24:49 but it extends across macroscopic dimensions,

00:24:52 the size of whatever block of that material

00:24:56 I have sitting here.

00:24:57 And the way this occurs is that,

00:25:01 let’s try to be a little bit light on the technical details,

00:25:03 but essentially the electrons coordinate with each other.

00:25:06 They are able to, in this macroscopic quantum state,

00:25:10 they’re able to sort of,

00:25:12 one can quickly take the place of the other.

00:25:14 You can’t tell electrons apart.

00:25:15 They’re what’s known as identical particles.

00:25:17 So if this electron runs into a defect

00:25:22 that would otherwise cause it to scatter,

00:25:25 it can just sort of almost miraculously avoid that defect

00:25:30 because it’s not really in that location.

00:25:32 It’s part of a macroscopic quantum state

00:25:34 and the entire quantum state

00:25:35 was not scattered by that defect.

00:25:37 So you can get a current that flows without dissipation

00:25:40 and that’s called a supercurrent.

00:25:42 That’s sort of just very much scratching the surface

00:25:47 of superconductivity.

00:25:49 There’s very deep and rich physics there,

00:25:52 just probably not the main subject

00:25:54 we need to go into right now.

00:25:55 But it turns out that when you have this material,

00:26:00 you can do usual things like make wires out of it

00:26:03 so you can get current to flow in a straight line on a chip,

00:26:06 but you can also make other devices

00:26:08 that perform different kinds of operations.

00:26:11 Some of them are kind of logic operations

00:26:14 like you’d get in a transistor.

00:26:16 The most common or the most,

00:26:21 I would say, diverse in its utility component

00:26:25 is a Josephson junction.

00:26:26 It’s not analogous to a transistor

00:26:28 in the sense that if you apply a voltage here,

00:26:31 it changes how much current flows from left to right,

00:26:33 but it is analogous in sort of a sense

00:26:36 of it’s the go to component

00:26:39 that a circuit engineer is going to use

00:26:42 to start to build up more complexity.

00:26:44 So these junctions serve as gates.

00:26:48 They can serve as gates.

00:26:50 So I’m not sure how concerned to be with semantics,

00:26:55 but let me just briefly say what a Josephson junction is

00:26:58 and we can talk about different ways that they can be used.

00:27:02 Basically, if you have a superconducting wire

00:27:05 and then a small gap of a different material

00:27:09 that’s not superconducting, an insulator or normal metal,

00:27:13 and then another superconducting wire on the other side,

00:27:15 that’s a Josephson junction.

00:27:17 So it’s sometimes referred to

00:27:18 as a superconducting weak link.

00:27:20 So you have this superconducting state on one side

00:27:24 and on the other side, and the superconducting wave function

00:27:27 actually tunnels across that gap.

00:27:30 And when you create such a physical entity,

00:27:35 it has very unusual current voltage characteristics.

00:27:41 In that gap, like weird stuff happens.

00:27:44 Through the entire circuit.

00:27:45 So you can imagine, suppose you had a loop set up

00:27:47 that had one of those weak links in the loop.

00:27:51 Current would flow in that loop independent,

00:27:53 even if you hadn’t applied a voltage to it,

00:27:55 and that’s called the Josephson effect.

00:27:57 So the fact that there’s this phase difference

00:28:00 in the quantum wave function from one side

00:28:02 of the tunneling barrier to the other

00:28:04 induces current to flow.

00:28:05 So how does you change state?

00:28:07 Right, exactly.

00:28:08 So how do you change state?

00:28:09 Now picture if I have a current bias coming down

00:28:13 this line of my circuit and there’s a Josephson junction

00:28:16 right in the middle of it.

00:28:18 And now I make another wire

00:28:19 that goes around the Josephson junction.

00:28:21 So I have a loop here, a superconducting loop.

00:28:24 I can add current to that loop by exceeding

00:28:28 the critical current of that Josephson junction.

00:28:30 So like any superconducting material,

00:28:34 it can carry this supercurrent that I’ve described,

00:28:37 this current that can propagate without dissipation

00:28:40 up to a certain level.

00:28:41 And if you try and pass more current than that

00:28:44 through the material, it’s going to become

00:28:47 a resistive material, normal material.

00:28:51 So in the Josephson junction, the same thing happens.

00:28:54 I can bias it above its critical current.

00:28:57 And then what it’s going to do,

00:28:58 it’s going to add a quantized amount of current

00:29:03 into that loop.

00:29:04 And what I mean by quantized is it’s going to come

00:29:07 in discrete packets with a well defined value of current.

00:29:11 So in the vernacular of some people working

00:29:15 in this community, you would say you pop a flux on

00:29:19 into the loop.

00:29:20 So a flux on.

00:29:21 You pop a flux on into the loop.

00:29:23 Yeah, so a flux on.

00:29:24 Sounds like skateboarder talk, I love it.

00:29:26 Okay, sorry, go ahead.

00:29:28 A flux on is one of these quantized sort of amounts

00:29:33 of current that you can add to a loop.

00:29:35 And this is a cartoon picture,

00:29:36 but I think it’s sufficient for our purposes.

00:29:38 So which, maybe it’s useful to say,

00:29:41 what is the speed at which these discrete packets

00:29:45 of current travel?

00:29:47 Because we’ll be talking about light a little bit.

00:29:49 It seems like the speed is important.

00:29:51 The speed is important, that’s an excellent question.

00:29:53 Sometimes I wonder where you, how you became so astute.

00:29:57 But so this.

00:30:00 Matrix 4 is coming out, so maybe that’s related.

00:30:04 I’m not sure.

00:30:05 I’m dressed for the job.

00:30:06 I was trying to get to become an extra on Matrix 4,

00:30:09 didn’t work out.

00:30:10 Anyway, so what’s the speed of these packets?

00:30:13 You’ll have to find another gig.

00:30:15 I know, I’m sorry.

00:30:16 So the speed of the pack is actually these flux ons,

00:30:19 these sort of pulses of current

00:30:24 that are generated by Joseph’s injunctions,

00:30:26 they can actually propagate very close

00:30:28 to the speed of light,

00:30:29 maybe something like a third of the speed of light.

00:30:31 That’s quite fast.

00:30:32 So one of the reasons why Joseph’s injunctions are appealing

00:30:37 is because their signals can propagate quite fast

00:30:40 and they can also switch very fast.

00:30:43 What I mean by switch is perform that operation

00:30:46 that I described where you add current to the loop.

00:30:49 That can happen within a few tens of picoseconds.

00:30:53 So you can get devices that operate

00:30:56 in the hundreds of gigahertz range.

00:30:58 And by comparison, most processors

00:31:02 in our conventional computers operate closer

00:31:04 to the one gigahertz range, maybe three gigahertz

00:31:08 seems to be kind of where those speeds have leveled out.

00:31:12 The gamers listening to this are getting really excited

00:31:15 to overclock their system to like, what is it?

00:31:18 Like four gigahertz or something,

00:31:19 a hundred sounds incredible.

00:31:21 Can I just as a tiny tangent,

00:31:24 is the physics of this understood well

00:31:26 how to do this stably?

00:31:28 Oh yes, the physics is understood well.

00:31:30 The physics of Joseph’s injunctions is understood well.

00:31:32 The technology is understood quite well too.

00:31:34 The reasons why it hasn’t displaced

00:31:37 silicon microelectronics in conventional digital computing

00:31:41 I think are more related to what I was alluding to before

00:31:45 about the myriad practical, almost mundane aspects

00:31:49 of silicon that make it so useful.

00:31:52 You can make a transistor ever smaller and smaller

00:31:55 and it will still perform its digital function quite well.

00:31:58 The same is not true of a Joseph’s injunction.

00:32:00 You really, they don’t, they just,

00:32:02 it’s not the same thing that there’s this feature

00:32:04 that you can keep making smaller and smaller

00:32:06 and it’ll keep performing the same operations.

00:32:08 This loop I described, any Joseph’s in circuit,

00:32:11 well, I wanna be careful, I shouldn’t say

00:32:13 any Joseph’s in circuit, but many Joseph’s in circuits,

00:32:17 the way they process information

00:32:19 or the way they perform whatever function it is

00:32:21 they’re trying to do,

00:32:22 maybe it’s sensing a weak magnetic field,

00:32:24 it depends on an interplay between the junction

00:32:27 and that loop.

00:32:28 And you can’t make that loop much smaller.

00:32:30 And it’s not for practical reasons

00:32:32 that have to do with lithography.

00:32:33 It’s for fundamental physical reasons

00:32:35 about the way the magnetic field interacts

00:32:38 with that superconducting material.

00:32:41 There are physical limits that no matter how good

00:32:44 our technology got, those circuits would,

00:32:47 I think would never be able to be scaled down

00:32:50 to the densities that silicon microelectronics can.

00:32:54 I don’t know if we mentioned,

00:32:55 is there something interesting

00:32:56 about the various superconducting materials involved

00:33:00 or is it all?

00:33:01 There’s a lot of stuff that’s interesting.

00:33:02 And it’s not silicon.

00:33:04 It’s not silicon, no.

00:33:05 So like it’s some materials that also required

00:33:09 to be super cold, four Kelvin and so on.

00:33:12 So let’s dissect a couple of those different things.

00:33:15 The super cold part,

00:33:16 let me just mention for your gamers out there

00:33:19 that are trying to clock it at four gigahertz

00:33:21 and would love to go to 400.

00:33:22 What kind of cooling system can achieve four Kelvin?

00:33:24 Four Kelvin, you need liquid helium.

00:33:26 And so liquid helium is expensive.

00:33:29 It’s inconvenient.

00:33:29 You need a cryostat that sits there

00:33:32 and the energy consumption of that cryostat

00:33:36 is impracticable for, it’s not going in your cell phone.

00:33:40 So you can picture holding your cell phone like this

00:33:42 and then something the size of a keg of beer or something

00:33:46 on your back to cool it.

00:33:47 Like that makes no sense.

00:33:49 So if you’re trying to make this in consumer devices,

00:33:54 electronics that are ubiquitous across society,

00:33:57 superconductors are not in the race for that.

00:33:59 For now, but you’re saying,

00:34:01 so just to frame the conversation,

00:34:03 maybe the thing we’re focused on

00:34:05 is computing systems that serve as servers, like large.

00:34:10 Yes, large systems.

00:34:11 So then you can contrast what’s going on in your cell phone

00:34:14 with what’s going on at one of the supercomputers.

00:34:19 Colleague Katie Schuman invited us out to Oak Ridge

00:34:22 a few years ago, so we got to see Titan

00:34:24 and that was when they were building Summit.

00:34:26 So these are some high performance supercomputers

00:34:29 out in Tennessee and those are filling entire rooms

00:34:32 the size of warehouses.

00:34:33 So once you’re at that level, okay,

00:34:36 there you’re already putting a lot of power into cooling.

00:34:39 Cooling is part of your engineering task

00:34:42 that you have to deal with.

00:34:43 So there it’s not entirely obvious

00:34:45 that cooling to four Kelvin is out of the question.

00:34:49 It has not happened yet and I can speak to why that is

00:34:53 in the digital domain if you’re interested.

00:34:55 I think it’s not going to happen.

00:34:57 I don’t think superconductors are gonna replace

00:35:01 semiconductors for digital computation.

00:35:05 There are a lot of reasons for that,

00:35:07 but I think ultimately what it comes down to

00:35:09 is all things considered cooling errors,

00:35:13 scaling down to feature sizes, all that stuff,

00:35:16 semiconductors work better at the system level.

00:35:19 Is there some aspect of just curious

00:35:22 about the historical momentum of this?

00:35:25 Is there some power to the momentum of an industry

00:35:28 that’s mass manufacturing using a certain material?

00:35:31 Is this like a Titanic shifting?

00:35:33 Like what’s your sense when a good idea comes along,

00:35:37 how good does that idea need to be

00:35:39 for the Titanic to start shifting?

00:35:42 That’s an excellent question.

00:35:44 That’s an excellent way to frame it.

00:35:46 And you know, I don’t know the answer to that,

00:35:51 but what I think is, okay,

00:35:53 so the history of the superconducting logic

00:35:56 goes back to the 70s.

00:35:58 IBM made a big push to do

00:35:59 superconducting digital computing in the 70s.

00:36:02 And they made some choices about their devices

00:36:06 and their architectures and things that in hindsight,

00:36:09 were kind of doomed to fail.

00:36:11 And I don’t mean any disrespect for the people that did it,

00:36:13 it was hard to see at the time.

00:36:14 But then another generation of superconducting logic

00:36:17 was introduced, I wanna say the 90s,

00:36:22 someone named Lykarev and Seminov,

00:36:25 they proposed an entire family of circuits

00:36:28 based on Joseph’s injunctions

00:36:29 that are doing digital computing based on logic gates

00:36:33 and or not these kinds of things.

00:36:37 And they showed how it could go hundreds of times faster

00:36:41 than silicon microelectronics.

00:36:43 And it’s extremely exciting.

00:36:45 I wasn’t working in the field at that time,

00:36:47 but later when I went back and read the literature,

00:36:49 I was just like, wow, this is so awesome.

00:36:53 And so you might think, well,

00:36:56 the reason why it didn’t display silicon

00:36:58 is because silicon already had so much momentum

00:37:00 at that time.

00:37:01 But that was the 90s.

00:37:02 Silicon kept that momentum

00:37:04 because it had the simple way to keep getting better.

00:37:06 You just make features smaller and smaller.

00:37:08 So it would have to be,

00:37:11 I don’t think it would have to be that much better

00:37:13 than silicon to displace it.

00:37:15 But the problem is it’s just not better than silicon.

00:37:17 It might be better than silicon in one metric,

00:37:19 speed of a switching operation

00:37:21 or power consumption of a switching operation.

00:37:24 But building a digital computer is a lot more

00:37:26 than just that elemental operation.

00:37:28 It’s everything that goes into it,

00:37:31 including the manufacturing, including the packaging,

00:37:33 including the various materials aspects of things.

00:37:38 So the reason why,

00:37:40 and even in some of those early papers,

00:37:42 I can’t remember which one it was,

00:37:44 Lykarev said something along the lines of,

00:37:47 you can see how we could build an entire family

00:37:49 of digital electronic circuits based on these components.

00:37:52 They could go a hundred or more times faster

00:37:55 than semiconductor logic gates.

00:37:59 But I don’t think that’s the right way

00:38:00 to use superconducting electronic circuits.

00:38:02 He didn’t say what the right way was,

00:38:04 but he basically said digital logic,

00:38:07 trying to steal the show from silicon

00:38:11 is probably not what these circuits

00:38:13 are most suited to accomplish.

00:38:16 So if we can just linger and use the word computation.

00:38:20 When you talk about computation, how do you think about it?

00:38:24 Do you think purely on just the switching,

00:38:28 or do you think something a little bit larger scale,

00:38:31 a circuit taken together,

00:38:32 performing the basic arithmetic operations

00:38:36 that are then required to do the kind of computation

00:38:40 that makes up a computer?

00:38:42 Because when we talk about the speed of computation,

00:38:44 is it boiled down to the basic switching,

00:38:46 or is there some bigger picture

00:38:48 that you’re thinking about?

00:38:49 Well, all right, so maybe we should disambiguate.

00:38:52 There are a variety of different kinds of computation.

00:38:55 I don’t pretend to be an expert

00:38:57 in the theory of computation or anything like that.

00:39:00 I guess it’s important to differentiate though

00:39:02 between digital logic,

00:39:05 which represents information as a series of bits,

00:39:09 binary digits, which you can think of them

00:39:13 as zeros and ones or whatever.

00:39:14 Usually they correspond to a physical system

00:39:17 that has two very well separated states.

00:39:21 And then other kinds of computation,

00:39:22 like we’ll get into more the way your brain works,

00:39:25 which it is, I think,

00:39:27 indisputably processing information,

00:39:30 but where the computation begins and ends

00:39:34 is not anywhere near as well defined.

00:39:36 It doesn’t depend on these two levels.

00:39:39 Here’s a zero, here’s a one.

00:39:41 There’s a lot of gray area

00:39:42 that’s usually referred to as analog computing.

00:39:45 Also in conventional digital computers

00:39:49 or digital computers in general,

00:39:54 you have a concept of what’s called arithmetic depth,

00:39:57 which is jargon that basically means

00:39:59 how many sequential operations are performed

00:40:03 to turn an input into an output.

00:40:07 And those kinds of computations in digital systems

00:40:10 are highly serial, meaning that data streams,

00:40:14 they don’t branch off too far to the side.

00:40:16 You do, you have to pull some information over there

00:40:18 and access memory from here and stuff like that.

00:40:20 But by and large, the computation proceeds

00:40:24 in a serial manner.

00:40:26 It’s not that way in the brain.

00:40:27 In the brain, you’re always drawing information

00:40:30 from different places.

00:40:31 It’s much more network based computing.

00:40:33 Neurons don’t wait for their turn.

00:40:35 They fire when they’re ready to fire.

00:40:37 And so it’s asynchronous.

00:40:39 So one of the other things about a digital system

00:40:41 is you’re performing these operations on a clock.

00:40:44 And that’s a crucial aspect of it.

00:40:46 Get rid of a clock in a digital system,

00:40:48 nothing makes sense anymore.

00:40:50 The brain has no clock.

00:40:51 It builds its own timescales based on its internal activity.

00:40:56 So you can think of the brain as kind of like this,

00:40:59 like network computation,

00:41:00 where it’s actually really trivial, simple computers,

00:41:05 just a huge number of them and they’re networked.

00:41:08 I would say it is complex, sophisticated little processors

00:41:12 and there’s a huge number of them.

00:41:14 Neurons are not, are not simple.

00:41:16 I don’t mean to offend neurons.

00:41:17 They’re very complicated and beautiful and yeah,

00:41:19 but we often oversimplify them.

00:41:21 Yes, they’re actually like there’s computation happening

00:41:24 within a neuron.

00:41:25 Right, so I would say to think of a transistor

00:41:29 as the building block of a digital computer is accurate.

00:41:32 You use a few transistors to make your logic gates.

00:41:34 You build up more, you build up processors

00:41:37 from logic gates and things like that.

00:41:39 So you can think of a transistor

00:41:40 as a fundamental building block,

00:41:42 or you can think of,

00:41:43 as we get into more highly parallelized architectures,

00:41:46 you can think of a processor

00:41:47 as a fundamental building block.

00:41:49 To make the analogy to the neuro side of things,

00:41:53 a neuron is not a transistor.

00:41:55 A neuron is a processor.

00:41:57 It has synapses, even synapses are not transistors,

00:42:00 but they are more,

00:42:02 they’re lower on the information processing hierarchy

00:42:04 in a sense.

00:42:05 They do a bulk of the computation,

00:42:08 but neurons are entire processors in and of themselves

00:42:13 that can take in many different kinds of inputs

00:42:16 on many different spatial and temporal scales

00:42:18 and produce many different kinds of outputs

00:42:20 so that they can perform different computations

00:42:23 in different contexts.

00:42:24 So this is where enters this distinction

00:42:27 between computation and communication.

00:42:30 So you can think of neurons performing computation

00:42:34 and the inter, the networking,

00:42:36 the interconnectivity of neurons

00:42:39 is communication between neurons.

00:42:40 And you see this with very large server systems.

00:42:43 I’ve been, I mentioned offline,

00:42:45 we’ve been talking to Jim Keller,

00:42:46 whose dream is to build giant computers

00:42:48 that, you know, the bottom like there

00:42:51 is often the communication

00:42:52 between the different pieces of computing.

00:42:54 So in this paper that we mentioned,

00:42:57 Optoelectronic Intelligence,

00:42:59 you say electrons excel at computation

00:43:03 while light is excellent for communication.

00:43:08 Maybe you can linger and say in this context,

00:43:11 what do you mean by computation and communication?

00:43:13 What are electrons, what is light

00:43:17 and why do they excel at those two tasks?

00:43:20 Yeah, just to first speak to computation

00:43:23 versus communication,

00:43:25 I would say computation is essentially taking in

00:43:30 some information, performing operations

00:43:33 on that information and producing new,

00:43:37 hopefully more useful information.

00:43:39 So for example, imagine you have a picture in front of you

00:43:45 and there is a key in it

00:43:48 and that’s what you’re looking for,

00:43:48 for whatever reason, you wanna find the key,

00:43:50 we all wanna find the key.

00:43:51 So the input is that entire picture

00:43:56 and the output might be the coordinates where the key is.

00:43:59 So you’ve reduced the total amount of information you have

00:44:01 but you found the useful information

00:44:03 for you in that present moment,

00:44:04 that’s the useful information.

00:44:05 And you think about this computation

00:44:07 as the controlled synchronous sequential?

00:44:10 Not necessarily, it could be,

00:44:12 that could be how your system is performing the computation

00:44:16 or it could be asynchronous,

00:44:19 there are lots of ways to find the key.

00:44:21 It depends on the nature of the data,

00:44:23 it depends on, that’s a very simplified example,

00:44:27 a picture with a key in it,

00:44:28 what about if you’re in the world

00:44:30 and you’re trying to decide the best way

00:44:32 to live your life?

00:44:35 It might be interactive,

00:44:37 it might be there might be some recurrence

00:44:38 or some weird asynchrony, I got it.

00:44:41 But there’s an input and there’s an output

00:44:43 and you do some stuff in the middle

00:44:44 that actually goes from the input to the output.

00:44:46 You’ve taken in information

00:44:47 and output different information,

00:44:49 hopefully reducing the total amount of information

00:44:51 and extracting what’s useful.

00:44:53 Communication is then getting that information

00:44:57 from the location at which it’s stored

00:44:59 because information is physical as Landauer emphasized

00:45:02 and so it is in one place

00:45:04 and you need to get that information to another place

00:45:07 so that something else can use it

00:45:10 for whatever computation it’s working on.

00:45:12 Maybe it’s part of the same network

00:45:13 and you’re all trying to solve the same problem

00:45:15 but neuron A over here just deduced something

00:45:20 based on its inputs

00:45:21 and it’s now sending that information across the network

00:45:25 to another location

00:45:26 so that would be the act of communication.

00:45:28 Can you linger on Landauer

00:45:29 and saying information is physical?

00:45:31 Rolf Landauer, not to be confused with Lev Landauer.

00:45:35 Yeah, and he made huge contributions

00:45:38 to our understanding of the reversibility of information

00:45:42 and this concept that energy has to be dissipated

00:45:46 in computing when the computation is irreversible

00:45:50 but if you can manage to make it reversible

00:45:52 then you don’t need to expend energy

00:45:55 but if you do expend energy to perform a computation

00:45:59 there’s sort of a minimal amount that you have to do

00:46:02 and it’s KT log two.

00:46:04 And it’s all somehow related

00:46:05 to the second law of thermodynamics

00:46:07 and that the universe is an information process

00:46:09 and then we’re living in a simulation.

00:46:11 So okay, sorry, sorry for that tangent.

00:46:13 So that’s the defining the distinction

00:46:17 between computation and communication.

00:46:19 Let me say one more thing just to clarify.

00:46:21 Communication ideally does not change the information.

00:46:27 It moves it from one place to another

00:46:28 but it is preserved.

00:46:30 Got it, okay.

00:46:32 All right, that’s beautiful.

00:46:33 So then the electron versus light distinction

00:46:38 and why are electrons good at computation

00:46:42 and light good at communication?

00:46:44 Yes, there’s a lot that goes into it I guess

00:46:48 but just try to speak to the simplest part of it.

00:46:54 Electrons interact strongly with one another.

00:46:56 They’re charged particles.

00:46:58 So if I pile a bunch of them over here

00:47:02 they’re feeling a certain amount of force

00:47:03 and they wanna move somewhere else.

00:47:05 They’re strongly interactive.

00:47:06 You can also get them to sit still.

00:47:08 You can, an electron has a mass

00:47:10 so you can cause it to be spatially localized.

00:47:15 So for computation that’s useful

00:47:18 because now I can make these little devices

00:47:20 that put a bunch of electrons over here

00:47:21 and then I change the state of a gate

00:47:25 like I’ve been describing,

00:47:26 put a different voltage on this gate

00:47:28 and now I move the electrons over here.

00:47:29 Now they’re sitting somewhere else.

00:47:31 I have a physical mechanism

00:47:33 with which I can represent information.

00:47:36 It’s spatially localized and I have knobs

00:47:38 that I can adjust to change where those electrons are

00:47:41 or what they’re doing.

00:47:42 Light by contrast, photons of light

00:47:45 which are the discrete packets of energy

00:47:48 that were identified by Einstein,

00:47:50 they do not interact with each other

00:47:54 especially at low light levels.

00:47:56 If you’re in a medium and you have a bright high light level

00:48:00 you can get them to interact with each other

00:48:02 through the interaction with that medium that they’re in

00:48:05 but that’s a little bit more exotic.

00:48:07 And for the purposes of this conversation

00:48:10 we can assume that photons don’t interact with each other.

00:48:13 So if you have a bunch of them

00:48:16 all propagating in the same direction

00:48:17 they don’t interfere with each other.

00:48:19 If I wanna send, if I have a communication channel

00:48:22 and I put one more photon on it,

00:48:24 it doesn’t screw up with those other ones.

00:48:26 It doesn’t change what those other ones were doing at all.

00:48:29 So that’s really useful for communication

00:48:31 because that means you can sort of allow

00:48:33 a lot of these photons to flow

00:48:37 without disruption of each other

00:48:38 and they can branch really easily and things like that.

00:48:41 But it’s not good for computation

00:48:42 because it’s very hard for this packet of light

00:48:46 to change what this packet of light is doing.

00:48:48 They pass right through each other.

00:48:50 So in computation you want to change information

00:48:53 and if photons don’t interact with each other

00:48:55 it’s difficult to get them to change the information

00:48:58 represented by the others.

00:48:59 So that’s the fundamental difference.

00:49:01 Is there also something about the way they travel

00:49:04 through different materials

00:49:07 or is that just a particular engineering?

00:49:10 No, it’s not, that’s deep physics I think.

00:49:12 So this gets back to electrons interact with each other

00:49:17 and photons don’t.

00:49:18 So say I’m trying to get a packet of information

00:49:22 from me to you and we have a wire going between us.

00:49:25 In order for me to send electrons across that wire

00:49:29 I first have to raise the voltage on my end of the wire

00:49:32 and that means putting a bunch of charges on it

00:49:34 and then that charge packet has to propagate along the wire

00:49:39 and it has to get all the way over to you.

00:49:41 That wire is gonna have something that’s called capacitance

00:49:44 which basically tells you how much charge

00:49:46 you need to put on the wire

00:49:48 in order to raise the voltage on it

00:49:49 and the capacitance is gonna be proportional

00:49:52 to the length of the wire.

00:49:54 So the longer the length of the wire is

00:49:56 the more charge I have to put on it

00:49:59 and the energy required to charge up that line

00:50:03 and move those electrons to you

00:50:04 is also proportional to the capacitance

00:50:06 and goes as the voltage squared.

00:50:08 So you get this huge penalty if you wanna send electrons

00:50:13 across a wire over appreciable distances.

00:50:16 So distance is an important thing here

00:50:19 when you’re doing communication.

00:50:20 Distance is an important thing.

00:50:22 So is the number of connections I’m trying to make.

00:50:25 Me to you, okay one, that’s not so bad.

00:50:27 If I want to now send it to 10,000 other friends

00:50:31 then all of those wires are adding tons

00:50:34 of extra capacitance.

00:50:35 Now not only does it take forever

00:50:37 to put the charge on that wire

00:50:39 and raise the voltage on all those lines

00:50:41 but it takes a ton of power

00:50:43 and the number 10,000 is not randomly chosen.

00:50:46 That’s roughly how many connections

00:50:49 each neuron in your brain makes.

00:50:50 So a neuron in your brain needs to send 10,000 messages

00:50:55 every time it has something to say.

00:50:56 You can’t do that if you’re trying to drive electrons

00:51:00 from here to 10,000 different places.

00:51:02 The brain does it in a slightly different way

00:51:03 which we can discuss.

00:51:04 How can light achieve the 10,000 connections

00:51:07 and why is it better?

00:51:09 In terms of like the energy use required

00:51:12 to use light for the communication of the 10,000 connections.

00:51:15 Right, right.

00:51:16 So now instead of trying to send electrons

00:51:17 from me to you, I’m trying to send photons.

00:51:19 So I can make what’s called a wave guide

00:51:21 which is just a simple piece of a material.

00:51:25 It could be glass like an optical fiber

00:51:27 or silicon on a chip.

00:51:29 And I just have to inject photons into that wave guide

00:51:34 and independent of how long it is,

00:51:35 independent of how many different connections I’m making,

00:51:39 it doesn’t change the voltage or anything like that

00:51:43 that I have to raise up on the wire.

00:51:45 So if I have one more connection,

00:51:47 if I add additional connections,

00:51:49 I need to add more light to the wave guide

00:51:51 because those photons need to split

00:51:53 and go to different paths.

00:51:55 That makes sense but I don’t have a capacitive penalty.

00:51:58 Sometimes these are called wiring parasitics.

00:52:01 There are no parasitics associated with light

00:52:03 in that same sense.

00:52:04 So this might be a dumb question

00:52:07 but how do I catch a photon on the other end?

00:52:11 Is it material?

00:52:12 Is it the polymer stuff you were talking about

00:52:15 for a different application for photolithography?

00:52:19 Like how do you catch a photon?

00:52:20 There’s a lot of ways to catch a photon.

00:52:22 It’s not a dumb question.

00:52:23 It’s a deep and important question

00:52:25 that basically defines a lot of the work

00:52:29 that goes on in our group at NIST.

00:52:31 One of my group leaders, Seywoon Nam,

00:52:34 has built his career around

00:52:35 these superconducting single photon detectors.

00:52:38 So if you’re going to try to sort of reach a lower limit

00:52:42 and detect just one particle of light,

00:52:45 superconductors come back into our conversation

00:52:47 and just picture a simple device

00:52:50 where you have current flowing

00:52:51 through a superconducting wire and…

00:52:54 A loop again or no?

00:52:56 Let’s say yes, you have a loop.

00:52:57 So you have a superconducting wire

00:52:59 that goes straight down like this

00:53:01 and on your loop branch, you have a little ammeter,

00:53:04 something that measures current.

00:53:05 There’s a resistor up there too.

00:53:07 Go with me here.

00:53:09 So your current biasing this,

00:53:12 so there’s current flowing

00:53:13 through that superconducting branch.

00:53:14 Since there’s a resistor over here,

00:53:16 all the current goes through the superconducting branch.

00:53:18 Now a photon comes in, strikes that superconductor.

00:53:22 We talked about this superconducting

00:53:24 macroscopic quantum state.

00:53:25 That’s going to be destroyed by the energy of that photon.

00:53:28 So now that branch of the circuit is resistive too.

00:53:32 And you’ve properly designed your circuit

00:53:33 so that the resistance on that superconducting branch

00:53:36 is much greater than the other resistance.

00:53:38 Now all of your current’s going to go that way.

00:53:40 Your ammeter says, oh, I just got a pulse of current.

00:53:43 That must mean I detected a photon.

00:53:45 Then where you broke that superconductivity

00:53:47 in a matter of a few nanoseconds,

00:53:49 it cools back off, dissipates that energy

00:53:51 and the current flows back

00:53:52 through that superconducting branch.

00:53:54 This is a very powerful superconducting device

00:53:59 that allows us to understand quantum states of light.

00:54:02 I didn’t realize a loop like that

00:54:04 could be sensitive to a single photon.

00:54:07 I mean, that seems strange to me because,

00:54:13 I mean, so what happens when you just barrage it

00:54:15 with photons?

00:54:16 If you put a bunch of photons in there,

00:54:18 essentially the same thing happens.

00:54:19 You just drive it into the normal state,

00:54:21 it becomes resistive and it’s not particularly interesting.

00:54:25 So you have to be careful how many photons you send.

00:54:27 Like you have to be very precise with your communication.

00:54:30 Well, it depends.

00:54:31 So I would say that that’s actually in the application

00:54:34 that we’re trying to use these detectors for.

00:54:37 That’s a feature because what we want is for,

00:54:41 if a neuron sends one photon to a synaptic connection

00:54:46 and one of these superconducting detectors is sitting there,

00:54:49 you get this pulse of current.

00:54:51 And that synapse says event,

00:54:54 then I’m gonna do what I do when there’s a synapse event,

00:54:56 I’m gonna perform computations, that kind of thing.

00:54:58 But if accidentally you send two there or three or five,

00:55:02 it does the exact same.

00:55:03 Got it.

00:55:04 And so this is how in the system that we’re devising here,

00:55:10 communication is entirely binary.

00:55:12 And that’s what I tried to emphasize a second ago.

00:55:15 Communication should not change the information.

00:55:17 You’re not saying, oh, I got this kind of communication

00:55:21 event for photons.

00:55:22 No, we’re not keeping track of that.

00:55:23 This neuron fired, this synapse says that neuron fired,

00:55:26 that’s it.

00:55:27 So that’s a noise filtering property of those detectors.

00:55:31 However, there are other applications

00:55:33 where you’d rather know the exact number of photons

00:55:36 that can be very useful in quantum computing with light.

00:55:39 And our group does a lot of work

00:55:41 around another kind of superconducting sensor

00:55:44 called a transition edge sensor that Adrian Alita

00:55:48 in our group does a lot of work on that.

00:55:49 And that can tell you based on the amplitude

00:55:53 of the current pulse you divert exactly how many photons

00:55:58 were in that pulse.

00:56:00 What’s that useful for?

00:56:02 One way that you can encode information

00:56:04 in quantum states of light is in the number of photons.

00:56:07 You can have what are called number states

00:56:09 and a number state will have a well defined number

00:56:11 of photons and maybe the output of your quantum computation

00:56:16 encodes its information in the number of photons

00:56:19 that are generated.

00:56:20 So if you have a detector that is sensitive to that,

00:56:23 it’s extremely useful.

00:56:24 Can you achieve like a clock with photons

00:56:29 or is that not important?

00:56:30 Is there a synchronicity here?

00:56:33 In general, it can be important.

00:56:36 Clock distribution is a big challenge

00:56:39 in especially large computational systems.

00:56:43 And so yes, optical clocks, optical clock distribution

00:56:47 is a very powerful technology.

00:56:51 I don’t know the state of that field right now,

00:56:53 but I imagine that if you’re trying to distribute a clock

00:56:55 across any appreciable size computational system,

00:56:58 you wanna use light.

00:57:00 Yeah, I wonder how these giant systems work,

00:57:04 especially like supercomputers.

00:57:07 Do they need to do clock distribution

00:57:09 or are they doing more ad hoc parallel

00:57:14 like concurrent programming?

00:57:15 Like there’s some kind of locking mechanisms or something.

00:57:18 That’s a fascinating question,

00:57:19 but let’s zoom in at this very particular question

00:57:23 of computation on a processor

00:57:28 and communication between processors.

00:57:31 So what does this system look like

00:57:36 that you’re envisioning?

00:57:38 One of the places you’re envisioning it

00:57:40 is in the paper on optoelectronic intelligence.

00:57:43 So what are we talking about?

00:57:44 Are we talking about something

00:57:46 that starts to look a lot like the human brain

00:57:48 or does it still look a lot like a computer?

00:57:51 What are the size of this thing?

00:57:52 Is it going inside a smartphone or as you said,

00:57:55 does it go inside something that’s more like a house?

00:57:58 Like what should we be imagining?

00:58:01 What are you thinking about

00:58:02 when you’re thinking about these fundamental systems?

00:58:05 Let me introduce the word neuromorphic.

00:58:07 There’s this concept of neuromorphic computing

00:58:09 where what that broadly refers to

00:58:12 is computing based on the information processing principles

00:58:17 of the brain.

00:58:19 And as digital computing seems to be pushing

00:58:23 towards some fundamental performance limits,

00:58:26 people are considering architectural advances,

00:58:29 drawing inspiration from the brain,

00:58:30 more distributed parallel network kind of architectures

00:58:33 and stuff.

00:58:34 And so there’s this continuum of neuromorphic

00:58:37 from things that are pretty similar to digital computers,

00:58:42 but maybe there are more cores

00:58:45 and the way they send messages is a little bit more

00:58:49 like the way brain neurons send spikes.

00:58:52 But for the most part, it’s still digital electronics.

00:58:56 And then you have some things in between

00:58:58 where maybe you’re using transistors,

00:59:02 but now you’re starting to use them

00:59:03 instead of in a digital way, in an analog way.

00:59:06 And so you’re trying to get those circuits

00:59:08 to behave more like neurons.

00:59:10 And then that’s a little bit,

00:59:12 quite a bit more on the neuromorphic side of things.

00:59:14 You’re trying to get your circuits,

00:59:17 although they’re still based on silicon,

00:59:19 you’re trying to get them to perform operations

00:59:22 that are highly analogous to the operations in the brain.

00:59:24 And that’s where a great deal of work is

00:59:26 in neuromorphic computing,

00:59:27 people like Giacomo Indoveri and Gert Kauenberg,

00:59:30 Jennifer Hasler, countless others.

00:59:32 It’s a rich and exciting field going back to Carver Mead

00:59:36 in the late 1980s.

00:59:39 And then all the way on the other extreme of the continuum

00:59:44 is where you say, I’ll give up anything related

00:59:48 to transistors or semiconductors or anything like that.

00:59:51 I’m not starting with the assumption

00:59:53 that I’m gonna use any kind

00:59:55 of conventional computing hardware.

00:59:57 And instead, what I wanna do is try and understand

00:59:59 what makes the brain powerful

01:00:00 at the kind of information processing it does.

01:00:03 And I wanna think from first principles

01:00:05 about what hardware is best going to enable us

01:00:10 to capture those information processing principles

01:00:14 in an artificial system.

01:00:16 And that’s where I live.

01:00:17 That’s where I’m doing my exploration these days.

01:00:21 So what are the first principles

01:00:25 of brain like computation communication?

01:00:29 Right, yeah, this is so important

01:00:32 and I’m glad we booked 14 hours for this because.

01:00:35 I only have 13, I’m sorry.

01:00:38 Okay, so the brain is notoriously complicated.

01:00:41 And I think that’s an important part

01:00:44 of why it can do what it does.

01:00:46 But okay, let me try to break it down.

01:00:49 Starting with the devices, neurons, as I said before,

01:00:54 they’re sophisticated devices in and of themselves

01:00:57 and synapses are too.

01:00:58 They can change their state based on the activity.

01:01:03 So they adapt over time.

01:01:04 That’s crucial to the way the brain works.

01:01:06 They don’t just adapt on one timescale,

01:01:09 they can adapt on myriad timescales

01:01:12 from the spacing between pulses,

01:01:16 the spacing between spikes that come from neurons

01:01:18 all the way to the age of the organism.

01:01:23 Also relevant, perhaps I think the most important thing

01:01:28 that’s guided my thinking is the network structure

01:01:32 of the brain, so.

01:01:33 Which can also be adjusted on different scales.

01:01:36 Absolutely, yes, so you’re making new,

01:01:39 you’re changing the strength of contacts,

01:01:41 you’re changing the spatial distribution of them,

01:01:44 although spatial distribution doesn’t change that much

01:01:46 once you’re a mature organism.

01:01:49 But that network structure is really crucial.

01:01:52 So let me dwell on that for a second.

01:01:55 You can’t talk about the brain without emphasizing

01:01:58 that most of the neurons in the neocortex

01:02:02 or the prefrontal cortex, the part of the brain

01:02:04 that we think is most responsible for high level reasoning

01:02:08 and things like that,

01:02:09 those neurons make thousands of connections.

01:02:11 So you have this network that is highly interconnected.

01:02:15 And I think it’s safe to say that one of the primary reasons

01:02:19 that they make so many different connections

01:02:23 is that allows information to be communicated very rapidly

01:02:26 from any spot in the network

01:02:28 to any other spot in the network.

01:02:30 So that’s a sort of spatial aspect of it.

01:02:33 You can quantify this in terms of concepts

01:02:38 that are related to fractals and scale invariants,

01:02:41 which I think is a very beautiful concept.

01:02:43 So what I mean by that is kind of,

01:02:46 no matter what spatial scale you’re looking at in the brain

01:02:50 within certain bounds, you see the same

01:02:53 general statistical pattern.

01:02:54 So if I draw a box around some region of my cortex,

01:02:59 most of the connections that those neurons

01:03:02 within that box make are gonna be within the box

01:03:04 to each other in their local neighborhood.

01:03:06 And that’s sort of called clustering, loosely speaking.

01:03:09 But a non negligible fraction

01:03:10 is gonna go outside of that box.

01:03:12 And then if I draw a bigger box,

01:03:14 the pattern is gonna be exactly the same.

01:03:16 So you have this scale invariants,

01:03:18 and you also have a non vanishing probability

01:03:22 of a neuron making connection very far away.

01:03:25 So suppose you wanna plot the probability

01:03:28 of a neuron making a connection as a function of distance.

01:03:32 If that were an exponential function,

01:03:34 it would go e to the minus radius

01:03:36 over some characteristic radius,

01:03:38 and it would drop off up to some certain radius,

01:03:41 the probability would be reasonably close to one,

01:03:44 and then beyond that characteristic length R zero,

01:03:49 it would drop off sharply.

01:03:51 And so that would mean that the neurons in your brain

01:03:53 are really localized, and that’s not what we observe.

01:03:58 Instead, what you see is that the probability

01:04:00 of making a longer distance connection, it does drop off,

01:04:03 but it drops off as a power law.

01:04:05 So the probability that you’re gonna have a connection

01:04:08 at some radius R goes as R to the minus some power.

01:04:13 And that’s more, that’s what we see with forces in nature,

01:04:16 like the electromagnetic force

01:04:18 between two particles or gravity

01:04:20 goes as one over the radius squared.

01:04:23 So you can see this in fractals.

01:04:24 I love that there’s like a fractal dynamics of the brain

01:04:28 that if you zoom out, you draw the box

01:04:31 and you increase that box by certain step sizes,

01:04:35 you’re gonna see the same statistics.

01:04:36 I think that’s probably very important

01:04:40 to the way the brain processes information.

01:04:41 It’s not just in the spatial domain,

01:04:43 it’s also in the temporal domain.

01:04:45 And what I mean by that is…

01:04:48 That’s incredible that this emerged

01:04:50 through the evolutionary process

01:04:52 that potentially somehow connected

01:04:54 to the way the physics of the universe works.

01:04:57 Yeah, I couldn’t agree more that it’s a deep

01:05:00 and fascinating subject that I hope to be able

01:05:02 to spend the rest of my life studying.

01:05:04 You think you need to solve, understand this,

01:05:07 this fractal nature in order to understand intelligence

01:05:10 and communication. I do think so.

01:05:11 I think they’re deeply intertwined.

01:05:13 Yes, I think power laws are right at the heart of it.

01:05:16 So just to push that one through,

01:05:19 the same thing happens in the temporal domain.

01:05:21 So suppose your neurons in your brain

01:05:26 were always oscillating at the same frequency,

01:05:28 then the probability of finding a neuron oscillating

01:05:31 as a function of frequency

01:05:32 would be this narrowly peaked function

01:05:34 around that certain characteristic frequency.

01:05:36 That’s not at all what we see.

01:05:37 The probability of finding neurons oscillating

01:05:40 or producing spikes at a certain frequency

01:05:43 is again a power law,

01:05:45 which means there’s no defined scale

01:05:49 of the temporal activity in the brain.

01:05:53 At what speed do your thoughts occur?

01:05:56 Well, there’s a fastest speed they can occur

01:05:58 and that is limited by communication and other things,

01:06:01 but there’s not a characteristic scale.

01:06:03 We have thoughts on all temporal scales

01:06:06 from a few tens of milliseconds,

01:06:10 which is physiologically limited by our devices,

01:06:13 compare that to tens of picoseconds

01:06:15 that I talked about in superconductors,

01:06:17 all the way up to the lifetime of the organism.

01:06:19 You can still think about things

01:06:20 that happened to you when you were a kid.

01:06:22 Or if you wanna be really trippy

01:06:24 then across multiple organisms

01:06:25 in the entirety of human civilization,

01:06:27 you have thoughts that span organisms, right?

01:06:29 Yes, taking it to that level, yes.

01:06:31 If you’re willing to see the entirety of the human species

01:06:34 as a single organism with a collective intelligence

01:06:37 and that too on a spatial and temporal scale,

01:06:39 there’s thoughts occurring.

01:06:41 And then if you look at not just the human species,

01:06:44 but the entirety of life on earth

01:06:46 as an organism with thoughts that are occurring,

01:06:49 that are greater and greater sophisticated thoughts,

01:06:51 there’s a different spatial and temporal scale there.

01:06:54 This is getting very suspicious.

01:06:57 Well, hold on though, before we’re done,

01:06:58 I just wanna just tie the bow

01:07:00 and say that the spatial and temporal aspects

01:07:04 are intimately interrelated with each other.

01:07:06 So activity between neurons that are very close to each other

01:07:10 is more likely to happen on this faster timescale

01:07:13 and information is gonna propagate

01:07:15 and encompass more of the brain,

01:07:17 more of your cortices, different modules in the brain

01:07:20 are gonna be engaged in information processing

01:07:23 on longer timescales.

01:07:25 So there’s this concept of information integration

01:07:27 where neurons are specialized.

01:07:31 Any given neuron or any cluster of neuron

01:07:33 has its specific purpose,

01:07:35 but they’re also very much integrated.

01:07:39 So you have neurons that specialize,

01:07:41 but share their information.

01:07:43 And so that happens through these fractal nested oscillations

01:07:47 that occur across spatial and temporal scales.

01:07:49 I think capturing those dynamics in hardware,

01:07:53 to me, that’s the goal of neuromorphic computing.

01:07:57 So does it need to look,

01:07:58 so first of all, that’s fascinating.

01:08:00 We stated some clear principles here.

01:08:03 Now, does it have to look like the brain

01:08:08 outside of those principles as well?

01:08:09 Like what other characteristics

01:08:11 have to look like the human brain?

01:08:13 Or can it be something very different?

01:08:15 Well, it depends on what you’re trying to use it for.

01:08:18 And so I think a lot of the community

01:08:21 asks that question a lot.

01:08:23 What are you gonna do with it?

01:08:24 And I completely get it.

01:08:26 I think that’s a very important question.

01:08:28 And it’s also sometimes not the most helpful question.

01:08:31 What if what you wanna do with it is study it?

01:08:33 What if you just wanna see,

01:08:37 what do you have to build into your hardware

01:08:39 in order to observe these dynamical principles?

01:08:43 And also, I ask myself that question every day

01:08:47 and I’m not sure I’m able to answer that.

01:08:49 So like, what are you gonna do

01:08:51 with this particular neuromorphic machine?

01:08:53 So suppose what we’re trying to do with it

01:08:55 is build something that thinks.

01:08:56 We’re not trying to get it to make us any money

01:08:59 or drive a car.

01:09:00 Maybe we’ll be able to do that, but that’s not our goal.

01:09:02 Our goal is to see if we can get the same types of behaviors

01:09:07 that we observe in our own brain.

01:09:08 And by behaviors in this sense,

01:09:10 what I mean the behaviors of the components,

01:09:14 the neurons, the network, that kind of stuff.

01:09:16 I think there’s another element that I didn’t really hit on

01:09:19 that you also have to build into this.

01:09:21 And those are architectural principles.

01:09:22 They have to do with the hierarchical modular construction

01:09:26 of the network.

01:09:27 And without getting too lost in jargon,

01:09:30 the main point that I think is relevant there,

01:09:33 let me try and illustrate it with a cartoon picture

01:09:35 of the architecture of the brain.

01:09:38 So in the brain, you have the cortex,

01:09:41 which is sort of this outer sheet.

01:09:44 It’s actually, it’s a layered structure.

01:09:46 You can, if you could take it out of your brain,

01:09:48 you could unroll it on the table

01:09:50 and it would be about the size of a pizza sitting there.

01:09:53 And that’s a module.

01:09:56 It does certain things.

01:09:57 It processes as Yogi Buzaki would say,

01:10:00 it processes the what of what’s going on around you.

01:10:03 But you have another really crucial module

01:10:06 that’s called the hippocampus.

01:10:08 And that network is structured entirely differently.

01:10:10 First of all, this cortex that had described

01:10:12 10 billion neurons in there.

01:10:14 So numbers matter here.

01:10:16 And they’re organized in that sort of power law distribution

01:10:20 where the probability of making a connection drops off

01:10:22 as a power law in space.

01:10:24 The hippocampus is another module that’s important

01:10:26 for understanding how, where you are,

01:10:30 when you are keeping track of your position

01:10:36 in space and time.

01:10:37 And that network is very much random.

01:10:39 So the probability of making a connection,

01:10:41 it almost doesn’t even drop off as a function of distance.

01:10:44 It’s the same probability that you’ll make it here

01:10:46 to over there, but there are only about 100 million neurons

01:10:50 there, so you can have that huge densely connected module

01:10:54 because it’s not so big.

01:10:57 And the neocortex or the cortex and the hippocampus,

01:11:02 they talk to each other constantly.

01:11:04 And that communication is largely facilitated

01:11:07 by what’s called the thalamus.

01:11:09 I’m not a neuroscientist here.

01:11:10 I’m trying to do my best to recite things.

01:11:12 Cartoon picture of the brain, I gotcha.

01:11:14 Yeah, something like that.

01:11:15 So this thalamus is coordinating the activity

01:11:18 between the neocortex and the hippocampus

01:11:20 and making sure that they talk to each other

01:11:23 at the right time and send messages

01:11:25 that will be useful to one another.

01:11:26 So this all taken together is called

01:11:29 the thalamocortical complex.

01:11:31 And it seems like building something like that

01:11:34 is going to be crucial to capturing the types of activity

01:11:39 we’re looking for because those responsibilities,

01:11:43 those separate modules, they do different things,

01:11:45 that’s gotta be central to achieving these states

01:11:51 of efficient information integration across space and time.

01:11:55 By the way, I am able to achieve this state

01:11:58 by watching simulations, visualizations

01:12:01 of the thalamocortical complex.

01:12:03 There’s a few people I forget from where.

01:12:06 They’ve created these incredible visual illustrations

01:12:09 of visual stimulation from the eye or something like that.

01:12:14 And this image flowing through the brain.

01:12:18 Wow, I haven’t seen that, I gotta check that out.

01:12:20 So it’s one of those things,

01:12:22 you find this stuff in the world,

01:12:24 and you see on YouTube, it has 1,000 views,

01:12:26 these visualizations of the human brain

01:12:30 processing information.

01:12:32 And because there’s chemistry there,

01:12:36 because this is from actual human brains,

01:12:38 I don’t know how they’re doing the coloring,

01:12:40 but they’re able to actually trace

01:12:42 the different, the chemical and the electrical signals

01:12:46 throughout the brain, and the visual thing,

01:12:48 it’s like, whoa, because it looks kinda like the universe,

01:12:51 I mean, the whole thing is just incredible.

01:12:53 I recommend it highly, I’ll probably post a link to it.

01:12:56 But you can just look for, one of the things they simulate

01:13:00 is the thalamocortical complex and just visualization.

01:13:05 You can find that yourself on YouTube, but it’s beautiful.

01:13:09 The other question I have for you is,

01:13:11 how does memory play into all of this?

01:13:14 Because all the signals sending back and forth,

01:13:17 that’s computation and communication,

01:13:20 but that’s kinda like processing of inputs and outputs,

01:13:26 to produce outputs in the system,

01:13:27 that’s kinda like maybe reasoning,

01:13:29 maybe there’s some kind of recurrence.

01:13:30 But is there a storage mechanism that you think about

01:13:33 in the context of neuromorphic computing?

01:13:35 Yeah, absolutely, so that’s gotta be central.

01:13:37 You have to have a way that you can store memories.

01:13:41 And there are a lot of different kinds

01:13:43 of memory in the brain.

01:13:45 That’s yet another example of how it’s not a simple system.

01:13:49 So there’s one kind of memory,

01:13:53 one way of talking about memory,

01:13:56 usually starts in the context of Hopfield networks.

01:13:59 You were lucky to talk to John Hopfield on this program.

01:14:02 But the basic idea there is working memory

01:14:05 is stored in the dynamical patterns

01:14:07 of activity between neurons.

01:14:10 And you can think of a certain pattern of activity

01:14:14 as an attractor, meaning if you put in some signal

01:14:19 that’s similar enough to other

01:14:22 previously experienced signals like that,

01:14:26 then you’re going to converge to the same network dynamics

01:14:29 and you will see these neurons

01:14:31 participate in the same network patterns of activity

01:14:36 that they have in the past.

01:14:37 So you can talk about the probability

01:14:39 that different inputs will allow you to converge

01:14:42 to different basins of attraction

01:14:44 and you might think of that as,

01:14:46 oh, I saw this face and then I excited

01:14:49 this network pattern of activity

01:14:50 because last time I saw that face,

01:14:53 I was at some movie and that’s a famous person

01:14:56 that’s on the screen or something like that.

01:14:58 So that’s one memory storage mechanism.

01:15:00 But crucial to the ability to imprint those memories

01:15:04 in your brain is the ability to change

01:15:07 the strength of connection between one neuron and another,

01:15:11 that synaptic connection between them.

01:15:13 So synaptic weight update is a massive field of neuroscience

01:15:18 and neuromorphic computing as well.

01:15:19 So there are two poles on that spectrum.

01:15:26 Okay, so more in the language of machine learning,

01:15:28 we would talk about supervised and unsupervised learning.

01:15:32 And when I’m trying to tie that down

01:15:33 to neuromorphic computing,

01:15:35 I will use a definition of supervised learning,

01:15:38 which basically means the external user,

01:15:42 the person who’s controlling this hardware

01:15:45 has some knob that they can tune

01:15:48 to change each of the synaptic weights,

01:15:50 depending on whether or not the network

01:15:52 is doing what you want it to do.

01:15:53 Whereas what I mean in this conversation

01:15:56 when I say unsupervised learning

01:15:57 is that those synaptic weights

01:15:59 are dynamically changing in your network

01:16:03 based on nothing that the user is doing,

01:16:05 nothing that there’s no wire from the outside

01:16:07 going into any of those synapses.

01:16:09 The network itself is reconfiguring those synaptic weights

01:16:12 based on physical properties

01:16:15 that you’ve built into the devices.

01:16:17 So if the synapse receives a pulse from here

01:16:21 and that causes the neuron to spike,

01:16:23 some circuit built in there with no help from me

01:16:27 or anybody else adjust the weight

01:16:29 in a way that makes it more likely

01:16:31 to store the useful information

01:16:34 and excite the useful network patterns

01:16:36 and makes it less likely that random noise,

01:16:39 useless communication events

01:16:41 will have an important effect on the network activity.

01:16:45 So there’s memory encoded in the weights,

01:16:48 the synaptic weights.

01:16:49 What about the formation of something

01:16:51 that’s not often done in machine learning,

01:16:53 the formation of new synaptic connections?

01:16:56 Right, well, that seems to,

01:16:57 so again, not a neuroscientist here,

01:17:00 but my reading of the literature

01:17:01 is that that’s particularly crucial

01:17:04 in early stages of brain development

01:17:06 where a newborn is born

01:17:09 with tons of extra synaptic connections

01:17:11 and it’s actually pruned over time.

01:17:13 So the number of synapses decreases

01:17:16 as opposed to growing new long distance connections.

01:17:19 It is possible in the brain to grow new neurons

01:17:22 and assign new synaptic connections

01:17:26 but it doesn’t seem to be the primary mechanism

01:17:29 by which the brain is learning.

01:17:31 So for example, like right now,

01:17:34 sitting here talking to you,

01:17:35 you say lots of interesting things

01:17:37 and I learn from you

01:17:38 and I can remember things that you just said

01:17:41 and I didn’t grow new axonal connections

01:17:44 down to new synapses to enable those.

01:17:47 It’s plasticity mechanisms

01:17:50 in the synaptic connections between neurons

01:17:52 that enable me to learn on that timescale.

01:17:55 So at the very least,

01:17:57 you can sufficiently approximate that

01:17:59 with just weight updates.

01:18:01 You don’t need to form new connections.

01:18:02 I would say weight updates are a big part of it.

01:18:05 I also think there’s more

01:18:06 because broadly speaking,

01:18:08 when we’re doing machine learning,

01:18:10 our networks, say we’re talking about feed forward,

01:18:12 deep neural networks,

01:18:14 the temporal domain is not really part of it.

01:18:16 Okay, you’re gonna put in an image

01:18:18 and you’re gonna get out a classification

01:18:20 and you’re gonna do that as fast as possible.

01:18:22 So you care about time

01:18:23 but time is not part of the essence of this thing really.

01:18:27 Whereas in spiking neural networks,

01:18:30 what we see in the brain,

01:18:31 time is as crucial as space

01:18:33 and they’re intimately intertwined

01:18:34 as I’ve tried to say.

01:18:36 And so adaptation on different timescales

01:18:40 is important not just in memory formation,

01:18:44 although it plays a key role there,

01:18:45 but also in just keeping the activity

01:18:48 in a useful dynamic range.

01:18:50 So you have other plasticity mechanisms,

01:18:52 not just weight update,

01:18:54 or at least not on the timescale

01:18:56 of many action potentials,

01:18:58 but even on the shorter timescale.

01:19:00 So a synapse can become much less efficacious.

01:19:04 It can transmit a weaker signal

01:19:07 after the second, third, fourth,

01:19:08 that can second, third, fourth action potential

01:19:11 to occur in a sequence.

01:19:13 So that’s what’s called short term synaptic plasticity,

01:19:15 which is a form of learning.

01:19:17 You’re learning that I’m getting too much stimulus

01:19:19 from looking at something bright right now.

01:19:21 So I need to tone that down.

01:19:24 There’s also another really important mechanism

01:19:28 in learning that’s called metoplasticity.

01:19:30 What that seems to be is a way

01:19:33 that you change not the weights themselves,

01:19:37 but the rate at which the weights change.

01:19:40 So when I am in say a lecture hall and my,

01:19:45 this is a potentially terrible cartoon example,

01:19:48 but let’s say I’m in a lecture hall

01:19:49 and it’s time to learn, right?

01:19:51 So my brain will release more,

01:19:54 perhaps dopamine or some neuromodulator

01:19:57 that’s gonna change the rate

01:20:00 at which synaptic plasticity occurs.

01:20:02 So that can make me more sensitive

01:20:03 to learning at certain times,

01:20:05 more sensitive to overriding previous information

01:20:08 and less sensitive at other times.

01:20:10 And finally, as long as I’m rattling off the list,

01:20:13 I think another concept that falls in the category

01:20:16 of learning or memory adaptation is homeostasis

01:20:20 or homeostatic adaptation,

01:20:22 where neurons have the ability

01:20:24 to control their firing rate.

01:20:27 So if one neuron is just like blasting way too much,

01:20:31 it will naturally tone itself down.

01:20:33 Its threshold will adjust

01:20:35 so that it stays in a useful dynamical range.

01:20:38 And we see that that’s captured in deep neural networks

01:20:41 where you don’t just change the synaptic weights,

01:20:43 but you can also move the thresholds of simple neurons

01:20:46 in those models.

01:20:47 And so to achieve the spiking neural networks,

01:20:53 you want to use,

01:20:58 you want to implement the first principles

01:21:01 that you mentioned of the temporal

01:21:03 and the spatial fractal dynamics here.

01:21:07 So you can communicate locally,

01:21:09 you can communicate across much greater distances

01:21:13 and do the same thing in space

01:21:16 and do the same thing in time.

01:21:18 Now, you have like a chapter called

01:21:21 Superconducting Hardware for Neuromorphic Computing.

01:21:24 So what are some ideas that integrate

01:21:27 some of the things we’ve been talking about

01:21:29 in terms of the first principles of neuromorphic computing

01:21:32 and the ideas that you outline

01:21:34 in optoelectronic intelligence?

01:21:38 Yeah, so let me start, I guess,

01:21:40 on the communication side of things,

01:21:42 because that’s what led us down this track

01:21:46 in the first place.

01:21:47 By us, I’m talking about my team of colleagues at NIST,

01:21:51 Saeed Han, Bryce Brimavera, Sonia Buckley,

01:21:54 Jeff Chiles, Adam McCallum to name,

01:21:57 Alex Tate to name a few,

01:21:58 our group leaders, Saewoo Nam and Rich Mirren.

01:22:01 We’ve all contributed to this.

01:22:02 So this is not me saying necessarily

01:22:05 just the things that I’ve proposed,

01:22:07 but sort of where our team’s thinking

01:22:09 has evolved over the years.

01:22:11 Can I quickly ask, what is NIST

01:22:14 and where is this amazing group of people located?

01:22:18 NIST is the National Institute of Standards and Technology.

01:22:23 The larger facility is out in Gaithersburg, Maryland.

01:22:26 Our team is located in Boulder, Colorado.

01:22:31 NIST is a federal agency under the Department of Commerce.

01:22:36 We do a lot with, by we, I mean other people at NIST,

01:22:40 do a lot with standards,

01:22:43 making sure that we understand the system of units,

01:22:46 international system of units, precision measurements.

01:22:49 There’s a lot going on in electrical engineering,

01:22:53 material science.

01:22:54 And it’s historic.

01:22:56 I mean, it’s one of those, it’s like MIT

01:22:58 or something like that.

01:22:59 It has a reputation over many decades

01:23:00 of just being this really a place

01:23:04 where there’s a lot of brilliant people have done

01:23:06 a lot of amazing things.

01:23:07 But in terms of the people in your team,

01:23:10 in this team of people involved

01:23:12 in the concept we’re talking about now,

01:23:14 I’m just curious,

01:23:15 what kind of disciplines are we talking about?

01:23:17 What is it?

01:23:18 Mostly physicists and electrical engineers,

01:23:20 some material scientists,

01:23:23 but I would say,

01:23:24 yeah, I think physicists and electrical engineers,

01:23:27 my background is in photonics,

01:23:29 the use of light for technology.

01:23:31 So coming from there, I tend to have found colleagues

01:23:36 that are more from that background.

01:23:38 Although Adam McConn,

01:23:40 more of a superconducting electronics background,

01:23:42 we need a diversity of folks.

01:23:44 This project is sort of cross disciplinary.

01:23:46 I would love to be working more

01:23:48 with neuroscientists and things,

01:23:50 but we haven’t reached that scale yet.

01:23:53 But yeah.

01:23:54 You’re focused on the hardware side,

01:23:56 which requires all the disciplines that you mentioned.

01:23:59 And then of course,

01:24:00 neuroscientists may be a source of inspiration

01:24:02 for some of the longterm vision.

01:24:04 I would actually call it more than inspiration.

01:24:06 I would call it sort of a roadmap.

01:24:11 We’re not trying to build exactly the brain,

01:24:15 but I don’t think it’s enough to just say,

01:24:17 oh, neurons kind of work like that.

01:24:19 Let’s kind of do that thing.

01:24:20 I mean, we’re very much following the concepts

01:24:25 that the cognitive sciences have laid out for us,

01:24:27 which I believe is a really robust roadmap.

01:24:30 I mean, just on a little bit of a tangent,

01:24:33 it’s often stated that we just don’t understand the brain.

01:24:36 And so it’s really hard to replicate it

01:24:37 because we just don’t know what’s going on there.

01:24:40 And maybe five or seven years ago,

01:24:43 I would have said that,

01:24:44 but as I got more interested in the subject,

01:24:47 I read more of the neuroscience literature

01:24:50 and I was just taken by the exact opposite sense.

01:24:53 I can’t believe how much they know about this.

01:24:55 I can’t believe how mathematically rigorous

01:24:59 and sort of theoretically complete

01:25:02 a lot of the concepts are.

01:25:04 That’s not to say we understand consciousness

01:25:06 or we understand the self or anything like that,

01:25:08 but what is the brain doing

01:25:11 and why is it doing those things?

01:25:13 Neuroscientists have a lot of answers to those questions.

01:25:16 So if you’re a hardware designer

01:25:17 that just wants to get going,

01:25:19 whoa, it’s pretty clear which direction to go in, I think.

01:25:23 Okay, so I love the optimism behind that,

01:25:28 but in the implementation of these systems

01:25:32 that uses superconductivity, how do you make it happen?

01:25:39 So to me, it starts with thinking

01:25:41 about the communication network.

01:25:43 You know for sure that the ability of each neuron

01:25:47 to communicate to many thousands of colleagues

01:25:50 across the network is indispensable.

01:25:52 I take that as a core principle of my architecture,

01:25:56 my thinking on the subject.

01:25:58 So coming from a background in photonics,

01:26:02 it was very natural to say,

01:26:03 okay, we’re gonna use light for communication.

01:26:05 Just in case listeners may not know,

01:26:08 light is often used in communication.

01:26:10 I mean, if you think about radio, that’s light,

01:26:12 it’s long wavelengths, but it’s electromagnetic radiation.

01:26:15 It’s the same physical phenomenon

01:26:17 obeying exactly the same Maxwell’s equations.

01:26:20 And then all the way down to fiber, fiber optics.

01:26:24 Now you’re using visible

01:26:26 or near infrared wavelengths of light,

01:26:27 but the way you send messages across the ocean

01:26:30 is now contemporary over optical fibers.

01:26:33 So using light for communication is not a stretch.

01:26:37 It makes perfect sense.

01:26:38 So you might ask, well, why don’t you use light

01:26:41 for communication in a conventional microchip?

01:26:45 And the answer to that is, I believe, physical.

01:26:49 If we had a light source on a silicon chip

01:26:53 that was as simple as a transistor,

01:26:55 there would not be a processor in the world

01:26:58 that didn’t use light for communication,

01:26:59 at least above some distance.

01:27:01 How many light sources are needed?

01:27:04 Oh, you need a light source at every single point.

01:27:06 A light source per neuron.

01:27:08 Per neuron, per little,

01:27:09 but then if you could have a really small

01:27:13 and nice light source,

01:27:15 your definition of neuron could be flexible.

01:27:17 Could be, yes, yes.

01:27:19 Sometimes it’s helpful to me to say,

01:27:21 in this hardware, a neuron is that entity

01:27:24 which has a light source.

01:27:25 That, and I can explain.

01:27:27 And then there was light.

01:27:29 I mean, I can explain more about that, but.

01:27:32 Somehow this like rhymes with consciousness

01:27:34 because people will often say the light of consciousness.

01:27:38 So that consciousness is that which is conscious.

01:27:41 I got it.

01:27:43 That’s not my quote.

01:27:44 That’s me, that’s my quote.

01:27:47 You see, that quote comes from my background.

01:27:49 Yours is in optics, mine in light, mine’s in darkness.

01:27:55 So go ahead.

01:27:56 So the point I was making there is that

01:27:59 if it was easy to manufacture light sources

01:28:02 along with transistors on a silicon chip,

01:28:05 they would be everywhere.

01:28:07 And it’s not easy.

01:28:08 People have been trying for decades

01:28:10 and it’s actually extremely difficult.

01:28:11 I think an important part of our research

01:28:14 is dwelling right at that spot there.

01:28:16 So.

01:28:17 Is it physics or engineering?

01:28:18 It’s physics.

01:28:19 So, okay, so it’s physics, I think.

01:28:22 So what I mean by that is, as we discussed,

01:28:26 silicon is the material of choice for transistors

01:28:29 and it’s very difficult to imagine

01:28:33 that that’s gonna change anytime soon.

01:28:35 Silicon is notoriously bad at emitting light.

01:28:39 And that has to do with the immutable properties

01:28:43 of silicon itself.

01:28:44 The way that the energy bands are structured in silicon,

01:28:47 you’re never going to make silicon efficient

01:28:50 as a light source at room temperature

01:28:53 without doing very exotic things

01:28:55 that degrade its ability to interface nicely

01:28:58 with those transistors in the first place.

01:28:59 So that’s like one of these things where it’s,

01:29:02 why is nature dealing us that blow?

01:29:05 You give us these beautiful transistors

01:29:07 and you give us all the motivation

01:29:08 to use light for communication,

01:29:10 but then you don’t give us a light source.

01:29:11 So, well, okay, you do give us a light source.

01:29:14 Compound semiconductors,

01:29:15 like we talked about back at the beginning,

01:29:16 an element from group three and an element from group five

01:29:19 form an alloy where every other lattice site

01:29:21 switches which element it is.

01:29:23 Those have much better properties for generating light.

01:29:27 You put electrons in, light comes out.

01:29:30 Almost 100% of the electron hold,

01:29:33 it can be made efficient.

01:29:36 I’ll take your word for it, okay.

01:29:37 However, I say it’s physics, not engineering,

01:29:39 because it’s very difficult

01:29:41 to get those compound semiconductor light sources

01:29:45 situated with your silicon.

01:29:47 In order to do that ion implantation

01:29:49 that I talked about at the beginning,

01:29:50 high temperatures are required.

01:29:52 So you gotta make all of your transistors first

01:29:55 and then put the compound semiconductors on top of there.

01:29:58 You can’t grow them afterwards

01:30:00 because that requires high temperature.

01:30:02 It screws up all your transistors.

01:30:04 You try and stick them on there.

01:30:05 They don’t have the same lattice constant.

01:30:07 The spacing between atoms is different enough

01:30:10 that it just doesn’t work.

01:30:11 So nature does not seem to be telling us that,

01:30:15 hey, go ahead and combine light sources

01:30:17 with your digital switches

01:30:19 for conventional digital computing.

01:30:22 And conventional digital computing

01:30:24 will often require smaller scale, I guess,

01:30:27 in terms of like smartphone.

01:30:30 So in which kind of systems does nature hint

01:30:35 that we can use light and photons for communication?

01:30:40 Well, so let me just try and be clear.

01:30:42 You can use light for communication in digital systems,

01:30:46 just the light sources are not intimately integrated

01:30:49 with the silicon.

01:30:50 You manufacture all the silicon,

01:30:52 you have your microchip, plunk it down.

01:30:54 And then you manufacture your light sources,

01:30:56 separate chip, completely different process

01:30:58 made in a different foundry.

01:31:00 And then you put those together at the package level.

01:31:03 So now you have some,

01:31:06 I would say a great deal of architectural limitations

01:31:09 that are introduced by that sort of

01:31:13 package level integration

01:31:15 as opposed to monolithic on the same chip integration,

01:31:18 but it’s still a very useful thing to do.

01:31:19 And that’s where I had done some work previously

01:31:23 before I came to NIST.

01:31:24 There’s a project led by Vladimir Stoyanovich

01:31:27 that now spun out into a company called IR Labs

01:31:30 led by Mark Wade and Chen Sun

01:31:33 where they’re doing exactly that.

01:31:34 So you have your light source chip,

01:31:36 your silicon chip, whatever it may be doing,

01:31:39 maybe it’s digital electronics,

01:31:40 maybe it’s some other control purpose, something.

01:31:43 And the silicon chip drives the light source chip

01:31:47 and modulates the intensity of the lights.

01:31:49 You can get data out of the package on an optical fiber.

01:31:52 And that still gives you tremendous advantages in bandwidth

01:31:56 as opposed to sending those signals out

01:31:58 over electrical lines.

01:32:00 But it is somewhat peculiar to my eye

01:32:05 that they have to be integrated at this package level.

01:32:07 And those people, I mean, they’re so smart.

01:32:09 Those are my colleagues that I respect a great deal.

01:32:12 So it’s very clear that it’s not just

01:32:16 they’re making a bad choice.

01:32:18 This is what physics is telling us.

01:32:20 It just wouldn’t make any sense

01:32:22 to try to stick them together.

01:32:24 Yeah, so even if it’s difficult,

01:32:28 it’s easier than the alternative, unfortunately.

01:32:30 I think so, yes.

01:32:31 And again, I need to go back

01:32:33 and make sure that I’m not taking the wrong way.

01:32:35 I’m not saying that the pursuit

01:32:36 of integrating compound semiconductors with silicon

01:32:39 is fruitless and shouldn’t be pursued.

01:32:41 It should, and people are doing great work.

01:32:43 Kai Mei Lau and John Bowers, others,

01:32:45 they’re doing it and they’re making progress.

01:32:48 But to my eye, it doesn’t look like that’s ever going to be

01:32:53 just the standard monolithic light source

01:32:57 on silicon process.

01:32:58 I just don’t see it.

01:33:00 Yeah, so nature kind of points the way usually.

01:33:02 And if you resist nature,

01:33:04 you’re gonna have to do a lot more work.

01:33:05 And it’s gonna be expensive and not scalable.

01:33:07 Got it.

01:33:08 But okay, so let’s go far into the future.

01:33:11 Let’s imagine this gigantic neuromorphic computing system

01:33:14 that simulates all of our realities.

01:33:17 It currently is Mantra Matrix 4.

01:33:19 So this thing, this powerful computer,

01:33:23 how does it operate?

01:33:24 So what are the neurons?

01:33:27 What is the communication?

01:33:29 What’s your sense?

01:33:30 All right, so let me now,

01:33:32 after spending 45 minutes trashing

01:33:34 light source integration with silicon,

01:33:36 let me now say why I’m basing my entire life,

01:33:40 professional life, on integrating light sources

01:33:43 with electronics.

01:33:44 I think the game is completely different

01:33:47 when you’re talking about superconducting electronics.

01:33:49 For several reasons, let me try to go through them.

01:33:54 One is that, as I mentioned,

01:33:56 it’s difficult to integrate

01:33:57 those compound semiconductor light sources with silicon.

01:34:01 With silicon is a requirement that is introduced

01:34:04 by the fact that you’re using semiconducting electronics.

01:34:07 In superconducting electronics,

01:34:08 you’re still gonna start with a silicon wafer,

01:34:10 but it’s just the bread for your sandwich in a lot of ways.

01:34:13 You’re not using that silicon

01:34:15 in precisely the same way for the electronics.

01:34:17 You’re now depositing superconducting materials

01:34:20 on top of that.

01:34:21 The prospects for integrating light sources

01:34:24 with that kind of an electronic process

01:34:27 are certainly less explored,

01:34:30 but I think much more promising

01:34:31 because you don’t need those light sources

01:34:34 to be intimately integrated with the transistors.

01:34:36 That’s where the problems come up.

01:34:37 They don’t need to be lattice matched to the silicon,

01:34:39 all that kind of stuff.

01:34:41 Instead, it seems possible

01:34:43 that you can take those compound semiconductor light sources,

01:34:47 stick them on the silicon wafer,

01:34:49 and then grow your superconducting electronics

01:34:51 on the top of that.

01:34:52 It’s at least not obviously going to fail.

01:34:55 So the computation would be done

01:34:57 on the superconductive material as well?

01:35:00 Yes, the computation is done

01:35:01 in the superconducting electronics,

01:35:03 and the light sources receive signals

01:35:06 that say, hey, a neuron reached threshold,

01:35:08 produce a pulse of light,

01:35:09 send it out to all your downstream synaptic connections.

01:35:12 Those are, again, superconducting electronics.

01:35:16 Perform your computation,

01:35:18 and you’re off to the races.

01:35:19 Your network works.

01:35:20 So then if we can rewind real quick,

01:35:22 so what are the limitations of the challenges

01:35:25 of superconducting electronics

01:35:28 when we think about constructing these kinds of systems?

01:35:31 So actually, let me say one other thing

01:35:35 about the light sources,

01:35:37 and then I’ll move on, I promise,

01:35:39 because this is probably tedious for some.

01:35:42 This is super exciting.

01:35:44 Okay, one other thing about the light sources.

01:35:45 I said that silicon is terrible at emitting photons.

01:35:48 It’s just not what it’s meant to do.

01:35:50 However, the game is different

01:35:52 when you’re at low temperature.

01:35:54 If you’re working with superconductors,

01:35:55 you have to be at low temperature

01:35:56 because they don’t work otherwise.

01:35:58 When you’re at four Kelvin,

01:36:00 silicon is not obviously a terrible light source.

01:36:03 It’s still not as efficient as compound semiconductors,

01:36:05 but it might be good enough for this application.

01:36:08 The final thing that I’ll mention about that is, again,

01:36:11 leveraging superconductors, as I said,

01:36:13 in a different context,

01:36:15 superconducting detectors can receive one single photon.

01:36:19 In that conversation, I failed to mention

01:36:21 that semiconductors can also receive photons.

01:36:23 That’s the primary mechanism by which it’s done.

01:36:26 A camera in your phone that’s receptive to visible light

01:36:29 is receiving photons.

01:36:31 It’s based on silicon,

01:36:32 or you can make it in different semiconductors

01:36:34 for different wavelengths,

01:36:36 but it requires on the order of a thousand,

01:36:39 a few thousand photons to receive a pulse.

01:36:43 Now, when you’re using a superconducting detector,

01:36:46 you need one photon, exactly one.

01:36:48 I mean, one or more.

01:36:50 So the fact that your synapses can now be based

01:36:54 on superconducting detectors

01:36:56 instead of semiconducting detectors

01:36:58 brings the light levels that are required

01:37:00 down by some three orders of magnitude.

01:37:03 So now you don’t need good light sources.

01:37:06 You can have the world’s worst light sources.

01:37:08 As long as they spit out maybe a few thousand photons

01:37:11 every time a neuron fires,

01:37:13 you have the hardware principles in place

01:37:17 that you might be able to perform

01:37:19 this optoelectronic integration.

01:37:21 To me optoelectronic integration is, it’s just so enticing.

01:37:25 We want to be able to leverage electronics for computation,

01:37:28 light for communication,

01:37:30 working with silicon microelectronics at room temperature

01:37:32 that has been exceedingly difficult.

01:37:35 And I hope that when we move to the superconducting domain,

01:37:40 target a different application space

01:37:41 that is neuromorphic instead of digital

01:37:44 and use superconducting detectors,

01:37:47 maybe optoelectronic integration comes to us.

01:37:50 Okay, so there’s a bunch of questions.

01:37:51 So one is temperature.

01:37:53 So in these kinds of hybrid heterogeneous systems,

01:37:58 what’s the temperature?

01:37:59 What are some of the constraints to the operation here?

01:38:01 Does it all have to be a four Kelvin as well?

01:38:03 Four Kelvin.

01:38:04 Everything has to be at four Kelvin.

01:38:06 Okay, so what are the other engineering challenges

01:38:09 of making this kind of optoelectronic systems?

01:38:14 Let me just dwell on that four Kelvin for a second

01:38:16 because some people hear four Kelvin

01:38:18 and they just get up and leave.

01:38:19 They just say, I’m not doing it, you know?

01:38:21 And to me, that’s very earth centric, species centric.

01:38:25 We live in 300 Kelvin.

01:38:27 So we want our technologies to operate there too.

01:38:29 I totally get it.

01:38:30 Yeah, what’s zero Celsius?

01:38:31 Zero Celsius is 273 Kelvin.

01:38:34 So we’re talking very, very cold here.

01:38:37 This is…

01:38:38 Not even Boston cold.

01:38:39 No.

01:38:40 This is real cold.

01:38:42 Yeah.

01:38:43 Siberia cold, no.

01:38:44 Okay, so just for reference,

01:38:45 the temperature of the cosmic microwave background

01:38:47 is about 2.7 Kelvin.

01:38:49 So we’re still warmer than deep space.

01:38:51 Yeah, good.

01:38:52 So that when the universe dies out,

01:38:56 it’ll be colder than four K.

01:38:57 It’s already colder than four K.

01:38:59 In the expanses, you know,

01:39:01 you don’t have to get that far away from the earth

01:39:05 in order to drop down to not far from four Kelvin.

01:39:08 So what you’re saying is the aliens that live at the edge

01:39:11 of the observable universe

01:39:13 are using superconductive material for their computation.

01:39:16 They don’t have to live at the edge of the universe.

01:39:17 The aliens that are more advanced than us

01:39:21 in their solar system are doing this

01:39:24 in their asteroid belt.

01:39:26 We can get to that.

01:39:27 Oh, because they can get that

01:39:30 to that temperature easier there?

01:39:31 Sure, yeah.

01:39:32 All you have to do is reflect the sunlight away

01:39:34 and you have a huge headstart.

01:39:36 Oh, so the sun is the problem here.

01:39:37 Like it’s warm here on earth.

01:39:39 Got it. Yeah.

01:39:39 Okay, so can you…

01:39:41 So how do we get to four K?

01:39:42 What’s…

01:39:43 Well, okay, so what I want to say about temperature…

01:39:47 Yeah.

01:39:48 What I want to say about temperature is that

01:39:50 if you can swallow that,

01:39:52 if you can say, all right, I give up applications

01:39:56 that have to do with my cell phone

01:39:58 and the convenience of a laptop on a train

01:40:02 and you instead…

01:40:03 For me, I’m very much in the scientific head space.

01:40:06 I’m not looking at products.

01:40:07 I’m not looking at what this will be useful

01:40:09 to sell to consumers.

01:40:11 Instead, I’m thinking about scientific questions.

01:40:13 Well, it’s just not that bad to have to work at four Kelvin.

01:40:16 We do it all the time in our labs at NIST.

01:40:19 And so does…

01:40:19 I mean, for reference,

01:40:21 the entire quantum computing sector

01:40:25 usually has to work at something like 100 millikelvin,

01:40:28 50 millikelvin.

01:40:29 So now you’re talking of another factor of 100

01:40:32 even colder than that, a fraction of a degree.

01:40:35 And everybody seems to think quantum computing

01:40:37 is going to take over the world.

01:40:39 It’s so much more expensive

01:40:40 to have to get that extra factor of 10 or whatever colder.

01:40:46 And yet it’s not stopping people from investing in that area.

01:40:50 And by investing, I mean putting their research into it

01:40:53 as well as venture capital or whatever.

01:40:55 So…

01:40:56 Oh, so based on the energy of what you’re commenting on,

01:40:59 I’m getting a sense that’s one of the criticism

01:41:01 of this approach is 4K, 4 Kelvin is a big negative.

01:41:06 It is the showstopper for a lot of people.

01:41:10 They just, I mean, and understandably,

01:41:12 I’m not saying that that’s not a consideration.

01:41:16 Of course it is.

01:41:17 For some…

01:41:18 Okay, so different motivations for different people.

01:41:21 In the academic world,

01:41:23 suppose you spent your whole life

01:41:24 learning about silicon microelectronic circuits.

01:41:26 You send a design to a foundry,

01:41:28 they send you back a chip

01:41:30 and you go test it at your tabletop.

01:41:33 And now I’m saying,

01:41:34 here now learn how to use all these cryogenics

01:41:36 so you can do that at 4 Kelvin.

01:41:38 No, come on, man.

01:41:39 I don’t wanna do that.

01:41:41 That sounds bad.

01:41:42 It’s the old momentum, the Titanic of the turning.

01:41:44 Yeah, kind of.

01:41:45 But you’re saying that’s not too much of a…

01:41:48 When we’re looking at large systems

01:41:50 and the gain you can potentially get from them,

01:41:52 that’s not that much of a cost.

01:41:53 And when you wanna answer the scientific question

01:41:55 about what are the physical limits of cognition?

01:41:58 Well, the physical limits,

01:41:59 they don’t care if you’re at 4 Kelvin.

01:42:01 If you can perform cognition at a scale

01:42:04 orders of magnitude beyond any room temperature technology,

01:42:07 but you gotta get cold to do it,

01:42:09 you’re gonna do it.

01:42:10 And to me, that’s the interesting application space.

01:42:14 It’s not even an application space,

01:42:16 that’s the interesting scientific paradigm.

01:42:18 So I personally am not going to let low temperature

01:42:22 stop me from realizing a technological domain or realm

01:42:29 that is achieving in most ways everything else

01:42:33 that I’m looking for in my hardware.

01:42:36 So that, okay, that’s a big one.

01:42:37 Is there other kind of engineering challenges

01:42:40 that you envision?

01:42:40 Yeah, yeah, yeah.

01:42:41 So let me take a moment here

01:42:43 because I haven’t really described what I mean

01:42:45 by a neuron or a network in this particular hardware.

01:42:49 Yeah, do you wanna talk about loop neurons

01:42:51 and there’s so many fascinating…

01:42:53 But you just have so many amazing papers

01:42:55 that people should definitely check out

01:42:57 and the titles alone are just killer.

01:42:59 So anyway, go ahead.

01:43:01 Right, so let me say big picture,

01:43:03 based on optics, photonics for communication,

01:43:07 superconducting electronics for computation,

01:43:10 how does this all work?

01:43:11 So a neuron in this hardware platform

01:43:17 can be thought of as circuits

01:43:19 that are based on Josephson junctions,

01:43:21 like we talked about before,

01:43:22 where every time a photon comes in…

01:43:25 So let’s start by talking about a synapse.

01:43:27 A synapse receives a photon, one or more,

01:43:29 from a different neuron

01:43:31 and it converts that optical signal

01:43:33 to an electrical signal.

01:43:35 The amount of current that that adds to a loop

01:43:38 is controlled by the synaptic weight.

01:43:40 So as I said before,

01:43:42 you’re popping fluxons into a loop, right?

01:43:44 So a photon comes in,

01:43:46 it hits a superconducting single photon detector,

01:43:49 one photon, the absolute physical minimum

01:43:52 that you can communicate

01:43:53 from one place to another with light.

01:43:54 And that detector then converts that

01:43:56 into an electrical signal

01:43:57 and the amount of signal

01:43:58 is correlated with some kind of weight.

01:44:01 Yeah, so the synaptic weight will tell you

01:44:02 how many fluxons you pop into the loop.

01:44:05 It’s an analog number.

01:44:06 We’re doing analog computation now.

01:44:08 Well, can you just linger on that?

01:44:09 What the heck is a fluxon?

01:44:10 Are we supposed to know this?

01:44:11 Or is this a funny,

01:44:14 is this like the big bang?

01:44:15 Is this a funny word for something deeply technical?

01:44:18 No, let’s try to avoid using the word fluxon

01:44:21 because it’s not actually necessary.

01:44:22 When a photon…

01:44:24 It’s fun to say though.

01:44:25 So it’s very necessary, I would say.

01:44:29 When a photon hits

01:44:30 that superconducting single photon detector,

01:44:32 current is added to a superconducting loop.

01:44:36 And the amount of current that you add

01:44:39 is an analog value,

01:44:40 can have eight bit equivalent resolution,

01:44:42 something like that.

01:44:44 10 bits, maybe.

01:44:45 That’s amazing, by the way.

01:44:46 This is starting to make a lot more sense.

01:44:48 When you’re using superconductors for this,

01:44:50 the energy of that circulating current

01:44:54 is less than the energy of that photon.

01:44:58 So your energy budget is not destroyed

01:45:01 by doing this analog computation.

01:45:04 So now in the language of a neuroscientist,

01:45:07 you would say that’s your postsynaptic signal.

01:45:09 You have this current being stored in a loop.

01:45:11 You can decide what you wanna do with it.

01:45:13 Most likely you’re gonna have it decay exponentially.

01:45:16 So every single synapse

01:45:18 is gonna have some given time constant.

01:45:20 And that’s determined by putting some resistor

01:45:25 in that superconducting loop.

01:45:27 So a synapse event occurs when a photon strikes a detector,

01:45:31 adds current to that loop, it decays over time.

01:45:33 That’s the postsynaptic signal.

01:45:35 Then you can process that in a dendritic tree.

01:45:38 Bryce Primavera and I have a paper

01:45:41 that we’ve submitted about that.

01:45:43 For the more neuroscience oriented people,

01:45:45 there’s a lot of dendritic processing,

01:45:47 a lot of plasticity mechanisms you can implement

01:45:49 with essentially exactly the same circuits.

01:45:51 You have this one simple building block circuit

01:45:54 that you can use for a synapse, for a dendrite,

01:45:57 for the neuron cell body, for all the plasticity functions.

01:46:00 It’s all based on the same building block,

01:46:02 just tweaking a couple parameters.

01:46:03 So this basic building block

01:46:05 has both an optical and an electrical component,

01:46:07 and then you just build arbitrary large systems with that?

01:46:11 Close, you’re not at fault

01:46:13 for thinking that that’s what I meant.

01:46:15 What I should say is that if you want it to be a synapse,

01:46:18 you tack a superconducting detector onto the front of it.

01:46:22 And if you want it to be anything else,

01:46:23 there’s no optical component.

01:46:25 Got it, so at the front,

01:46:28 optics in the front, electrical stuff in the back.

01:46:32 Electrical, yeah, in the processing

01:46:34 and in the output signal that it sends

01:46:36 to the next stage of processing further.

01:46:39 So the dendritic trees is electrical.

01:46:41 It’s all electrical.

01:46:42 It’s all electrical in the superconducting domain.

01:46:44 For anybody who’s up on their superconducting circuits,

01:46:48 it’s just based on a DC squid, the most ubiquitous,

01:46:52 which is a circuit composed of two Joseph’s injunctions.

01:46:55 So it’s a very bread and butter kind of thing.

01:46:58 And then the only place where you go beyond that

01:47:00 is the neuron cell body itself.

01:47:03 It’s receiving all these electrical inputs

01:47:05 from the synapses or dendrites

01:47:06 or however you’ve structured that particular unique neuron.

01:47:09 And when it reaches its threshold,

01:47:12 which occurs by driving a Joseph’s injunction

01:47:14 above its critical current,

01:47:15 it produces a pulse of current,

01:47:17 which starts an amplification sequence,

01:47:19 voltage amplification,

01:47:21 that produces light out of a transmitter.

01:47:24 So one of our colleagues, Adam McCann,

01:47:26 and Sonia Buckley as well,

01:47:27 did a lot of work on the light sources

01:47:30 and the amplifiers that drive the current

01:47:34 and produce sufficient voltage to drive current

01:47:37 through that now semiconducting part.

01:47:39 So that light source is the semiconducting part of a neuron.

01:47:43 And that, so the neuron has reached threshold.

01:47:45 It produces a pulse of light.

01:47:47 That light then fans out across a network of wave guides

01:47:51 to reach all the downstream synaptic terminals

01:47:54 that perform this process themselves.

01:47:57 So it’s probably worth explaining

01:47:59 what a network of wave guides is,

01:48:02 because a lot of listeners aren’t gonna know that.

01:48:04 Look up the papers by Jeff Chiles on this one.

01:48:07 But basically, light can be guided in a simple,

01:48:11 basically wire of usually an insulating material.

01:48:14 So silicon, silicon nitride,

01:48:18 different kinds of glass,

01:48:20 just like in a fiber optic, it’s glass, silicon dioxide.

01:48:23 That makes it a little bit big.

01:48:24 We wanna bring these down.

01:48:26 So we use different materials like silicon nitride,

01:48:28 but basically just imagine a rectangle of some material

01:48:32 that just goes and branches,

01:48:37 forms different branch points

01:48:39 that target different subregions of the network.

01:48:43 You can transition between layers of these.

01:48:45 So now we’re talking about building in the third dimension,

01:48:47 which is absolutely crucial.

01:48:48 So that’s what wave guides are.

01:48:50 Yeah, that’s great.

01:48:52 Why the third dimension is crucial?

01:48:54 Okay, so yes, you were talking about

01:48:56 what are some of the technical limitations.

01:48:59 One of the things that I believe we have to grapple with

01:49:04 is that our brains are miraculously compact.

01:49:08 For the number of neurons that are in our brain,

01:49:11 it sure does fit in a small volume,

01:49:13 as it would have to if we’re gonna be biological organisms

01:49:16 that are resource limited and things like that.

01:49:19 Any kind of hardware neuron

01:49:20 is almost certainly gonna be much bigger than that

01:49:23 if it is of comparable complexity,

01:49:26 whether it’s based on silicon transistors.

01:49:28 Okay, a transistor, seven nanometers,

01:49:30 that doesn’t mean a semiconductor based neuron

01:49:33 is seven nanometers.

01:49:34 They’re big.

01:49:35 They require many transistors,

01:49:38 different other things like capacitors and things

01:49:40 that store charge.

01:49:41 They end up being on the order of 100 microns

01:49:44 by 100 microns,

01:49:45 and it’s difficult to get them down any smaller than that.

01:49:48 The same is true for superconducting neurons,

01:49:50 and the same is true

01:49:52 if we’re trying to use light for communication.

01:49:54 Even if you’re using electrons for communication,

01:49:56 you have these wires where, okay,

01:50:00 the size of an electron might be angstroms,

01:50:03 but the size of a wire is not angstroms,

01:50:05 and if you try and make it narrower,

01:50:07 the resistance just goes up,

01:50:08 so you don’t actually win.

01:50:10 To communicate over long distances,

01:50:12 you need your wires to be microns wide,

01:50:15 and it’s the same thing for wave guides.

01:50:17 Wave guides are essentially limited

01:50:18 by the wavelength of light,

01:50:20 and that’s gonna be about a micron,

01:50:21 so whereas compare that to an axon,

01:50:24 the analogous component in the brain,

01:50:26 which is 10 nanometers in diameter, something like that,

01:50:32 they’re bigger when they need to communicate

01:50:33 over long distances,

01:50:34 but grappling with the size of these structures

01:50:37 is inevitable and crucial,

01:50:39 and so in order to make systems of comparable scale

01:50:45 to the human brain, by scale here,

01:50:46 I mean number of interconnected neurons,

01:50:49 you absolutely have to be using

01:50:51 the third spatial dimension,

01:50:53 and that means on the wafer,

01:50:55 you need multiple layers

01:50:57 of both active and passive components.

01:50:59 Active, I mean superconducting electronic circuits

01:51:03 that are performing computations,

01:51:05 and passive, I mean these wave guides

01:51:07 that are routing the optical signals to different places,

01:51:10 you have to be able to stack those.

01:51:11 If you can get to something like 10 planes

01:51:14 of each of those, or maybe not even 10,

01:51:16 maybe five, six, something like that,

01:51:18 then you’re in business.

01:51:19 Now you can get millions of neurons on a wafer,

01:51:22 but that’s not anywhere close to the brain scale.

01:51:26 In order to get to the scale of the human brain,

01:51:27 you’re gonna have to also use the third dimension

01:51:30 in the sense that entire wafers

01:51:32 need to be stacked on top of each other

01:51:34 with fiber optic communication between them,

01:51:36 and we need to be able to fill a space

01:51:38 the size of this table with stacked wafers,

01:51:42 and that’s when you can get to some 10 billion neurons

01:51:44 like your human brain,

01:51:45 and I don’t think that’s specific

01:51:46 to the optoelectronic approach that we’re taking.

01:51:48 I think that applies to any hardware

01:51:51 where you’re trying to reach commensurate scale

01:51:53 and complexity as the human brain.

01:51:55 So you need that fractal stacking,

01:51:57 so stacking on the wafer,

01:51:59 and stacking of the wafers,

01:52:01 and then whatever the system that combines,

01:52:03 this stacking of the tables with the wafers.

01:52:06 And it has to be fractal all the way,

01:52:07 you’re exactly right,

01:52:08 because that’s the only way

01:52:10 that you can efficiently get information

01:52:12 from a small point to across that whole network.

01:52:15 It has to have the power law connected.

01:52:17 And photons are like optics throughout.

01:52:20 Yeah, absolutely.

01:52:21 Once you’re at this scale, to me it’s just obvious.

01:52:23 Of course you’re using light for communication.

01:52:25 You have fiber optics given to us from nature, so simple.

01:52:30 The thought of even trying to do

01:52:32 any kind of electrical communication

01:52:34 just doesn’t make sense to me.

01:52:37 I’m not saying it’s wrong, I don’t know,

01:52:39 but that’s where I’m coming from.

01:52:40 So let’s return to loop neurons.

01:52:43 Why are they called loop neurons?

01:52:46 Yeah, the term loop neurons comes from the fact,

01:52:48 like we’ve been talking about,

01:52:49 that they rely heavily on these superconducting loops.

01:52:53 So even in a lot of forms of digital computing

01:52:57 with superconductors,

01:52:58 storing a signal in a superconducting loop

01:53:02 is a primary technique.

01:53:05 In this particular case,

01:53:06 it’s just loops everywhere you look.

01:53:08 So the strength of a synaptic weight

01:53:11 is gonna be set by the amount of current circulating

01:53:15 in a loop that is coupled to the synapse.

01:53:17 So memory is implemented as current circulating

01:53:22 in a superconducting loop.

01:53:24 The coupling between, say, a synapse and a dendrite

01:53:27 or a synapse in the neuron cell body

01:53:29 occurs through loop coupling through transformers.

01:53:33 So current circulating in a synapse

01:53:34 is gonna induce current in a different loop,

01:53:37 a receiving loop in the neuron cell body.

01:53:40 So since all of the computation is happening

01:53:44 in these flux storage loops

01:53:46 and they play such a central role

01:53:48 in how the information is processed,

01:53:50 how memories are formed, all that stuff,

01:53:52 I didn’t think too much about it,

01:53:53 I just called them loop neurons

01:53:55 because it rolls off the tongue a little bit better

01:53:58 than superconducting optoelectronic neurons.

01:54:02 Okay, so how do you design circuits for these loop neurons?

01:54:08 That’s a great question.

01:54:09 There’s a lot of different scales of design.

01:54:12 So at the level of just one synapse,

01:54:16 you can use conventional methods.

01:54:18 They’re not that complicated

01:54:21 as far as superconducting electronics goes.

01:54:23 It’s just four Joseph’s injunctions or something like that

01:54:27 depending on how much complexity you wanna add.

01:54:29 So you can just directly simulate each component in SPICE.

01:54:34 What’s SPICE?

01:54:35 It’s Standard Electrical Simulation Software, basically.

01:54:39 So you’re just explicitly solving the differential equations

01:54:42 that describe the circuit elements.

01:54:44 And then you can stack these things together

01:54:46 in that simulation software to then build circuits.

01:54:48 You can, but that becomes computationally expensive.

01:54:51 So one of the things when COVID hit,

01:54:54 we knew we had to turn some attention

01:54:55 to more things you can do at home in your basement

01:54:59 or whatever, and one of them was computational modeling.

01:55:02 So we started working on adapting,

01:55:07 abstracting out the circuit performance

01:55:10 so that you don’t have to explicitly solve

01:55:12 the circuit equations, which for Joseph’s injunctions

01:55:15 usually needs to be done on like a picosecond timescale

01:55:18 and you have a lot of nodes in your circuit.

01:55:21 So it results in a lot of differential equations

01:55:24 that need to be solved simultaneously.

01:55:26 We were looking for a way to simulate these circuits

01:55:29 that is scalable up to networks of millions or so neurons

01:55:33 is sort of where we’re targeting right now.

01:55:36 So we were able to analyze the behavior of these circuits.

01:55:40 And as I said, it’s based on these simple building blocks.

01:55:43 So you really only need to understand

01:55:45 this one building block.

01:55:46 And if you get a good model of that, boom, it tiles.

01:55:48 And you can change the parameters in there

01:55:51 to get different behaviors and stuff,

01:55:52 but it’s all based on now it’s one differential equation

01:55:56 that you need to solve.

01:55:57 So one differential equation for every synapse,

01:56:00 dendrite or neuron in your system.

01:56:03 And for the neuroscientists out there,

01:56:05 it’s just a simple leaky integrate and fire model,

01:56:08 leaky integrator, basically.

01:56:10 A synapse is a leaky integrator,

01:56:11 a dendrite is a leaky integrator.

01:56:13 So I’m really fascinated by how this one simple component

01:56:18 can be used to achieve lots of different types

01:56:22 of dynamical activity.

01:56:24 And to me, that’s where scalability comes from.

01:56:27 And also complexity as well.

01:56:29 Complexity is often characterized

01:56:30 by relatively simple building blocks

01:56:35 connected in potentially simple

01:56:37 or sometimes complicated ways,

01:56:39 and then emergent new behavior that was hard to predict

01:56:41 from those simple elements.

01:56:44 And that’s exactly what we’re working with here.

01:56:46 So it’s a very exciting platform,

01:56:49 both from a modeling perspective

01:56:50 and from a hardware manifestation perspective

01:56:52 where we can hopefully start to have this test bed

01:56:57 where we can explore things,

01:56:58 not just related to neuroscience,

01:57:00 but also related to other things

01:57:03 that connected to other physics like critical phenomenon,

01:57:07 Ising models, things like that.

01:57:08 So you were asking how we simulate these circuits.

01:57:13 It’s at different levels

01:57:14 and we’ve got the simple spice circuit stuff.

01:57:18 That’s no problem.

01:57:19 And now we’re building these network models

01:57:21 based on this more efficient leaky integrator.

01:57:23 So we can actually reduce every element

01:57:26 to one differential equation.

01:57:27 And then we can also step through it

01:57:28 on a much coarser time grid.

01:57:30 So it ends up being something like a factor

01:57:32 of a thousand to 10,000 speed improvement,

01:57:35 which allows us to simulate,

01:57:37 but hopefully up to millions of neurons.

01:57:40 Whereas before we would have been limited to tens,

01:57:44 a hundred, something like that.

01:57:45 And just like simulating quantum mechanical systems

01:57:48 with a quantum computer.

01:57:49 So the goal here is to understand such systems.

01:57:53 For me, the goal is to study this

01:57:55 as a scientific physical system.

01:57:58 I’m not drawn towards turning this

01:58:01 into an enterprise at this point.

01:58:03 I feel short term applications

01:58:05 that obviously make a lot of money

01:58:07 is not necessarily a curiosity driver for you at the moment.

01:58:11 Absolutely not.

01:58:12 If you’re interested in short term making money,

01:58:14 go with deep learning, use silicon microelectronics.

01:58:16 If you wanna understand things like the physics

01:58:20 of a fascinating system,

01:58:23 or if you wanna understand something more

01:58:25 along the lines of the physical limits

01:58:27 of what can be achieved,

01:58:29 then I think single photon communication,

01:58:32 superconducting electronics is extremely exciting.

01:58:35 What if I wanna use superconducting hardware

01:58:39 at four Kelvin to mine Bitcoin?

01:58:42 That’s my main interest.

01:58:44 The reason I wanted to talk to you today,

01:58:45 I wanna say, no, I don’t know.

01:58:47 What’s Bitcoin?

01:58:51 Look it up on the internet.

01:58:53 Somebody told me about it.

01:58:54 I’m not sure exactly what it is.

01:58:57 But let me ask nevertheless

01:58:59 about applications to machine learning.

01:59:01 Okay, so if you look at the scale of five, 10, 20 years,

01:59:07 is it possible to, before we understand the nature

01:59:11 of human intelligence and general intelligence,

01:59:14 do you think we’ll start falling out of this exploration

01:59:19 of neuromorphic systems ability to solve some

01:59:23 of the problems that the machine learning systems

01:59:25 of today can’t solve?

01:59:26 Well, I’m really hesitant to over promise.

01:59:31 So I really don’t know.

01:59:34 Also, I don’t really understand machine learning

01:59:36 in a lot of senses.

01:59:37 I mean, machine learning from my perspective appears

01:59:42 to require that you know precisely what your input is

01:59:49 and also what your goal is.

01:59:51 You usually have some objective function

01:59:53 or something like that.

01:59:54 And that’s very limiting.

01:59:57 I mean, of course, a lot of times that’s the case.

02:00:00 There’s a picture and there’s a horse in it, so you’re done.

02:00:03 But that’s not a very interesting problem.

02:00:06 I think when I think about intelligence,

02:00:09 it’s almost defined by the ability to handle problems

02:00:13 where you don’t know what your inputs are going to be

02:00:15 and you don’t even necessarily know

02:00:17 what you’re trying to accomplish.

02:00:18 I mean, I’m not sure what I’m trying to accomplish

02:00:21 in this world.

02:00:22 Yeah, at all scales.

02:00:24 Yeah, at all scales, right.

02:00:25 I mean, so I’m more drawn to the underlying phenomena,

02:00:33 the critical dynamics of this system,

02:00:37 trying to understand how elements that you build

02:00:41 into your hardware result in emergent fascinating activity

02:00:48 that was very difficult to predict, things like that.

02:00:51 So, but I gotta be really careful

02:00:53 because I think a lot of other people who,

02:00:55 if they found themselves working on this project

02:00:57 in my shoes, they would say, all right,

02:00:59 what are all the different ways we can use this

02:01:01 for machine learning?

02:01:02 Actually, let me just definitely mention colleague

02:01:05 at NIST, Mike Schneider.

02:01:06 He’s also very much interested,

02:01:09 particularly in the superconducting side of things,

02:01:11 using the incredible speed, power efficiency,

02:01:14 also Ken Seagal at Colgate,

02:01:16 other people working on specifically

02:01:18 the superconducting side of this for machine learning

02:01:22 and deep feed forward neural networks.

02:01:25 There, the advantages are obvious.

02:01:27 It’s extremely fast.

02:01:28 Yeah, so that’s less on the nature of intelligences

02:01:31 and more on various characteristics of this hardware

02:01:35 that you can use for the basic computation

02:01:38 as we know it today and communication.

02:01:40 One of the things that Mike Schneider’s working on right now

02:01:42 is an image classifier at a relatively small scale.

02:01:46 I think he’s targeting that nine pixel problem

02:01:48 where you can have three different characters

02:01:50 and you put in a nine pixel image

02:01:54 and you classify it as one of these three categories.

02:01:58 And that’s gonna be really interesting

02:02:00 to see what happens there,

02:02:01 because if you can show that even at that scale,

02:02:05 you just put these images in and you get it out

02:02:08 and he thinks he can do it,

02:02:09 I forgot if it’s a nanosecond

02:02:11 or some extremely fast classification time,

02:02:14 it’s probably less,

02:02:14 it’s probably a hundred picoseconds or something.

02:02:17 There you have challenges though,

02:02:18 because the Joseph’s injunctions themselves,

02:02:21 the electronic circuit is extremely power efficient.

02:02:24 Some orders of magnitude for something more

02:02:26 than a transistor doing the same thing,

02:02:29 but when you have to cool it down to four Kelvin,

02:02:31 you pay a huge overhead just for keeping it cold,

02:02:33 even if it’s not doing anything.

02:02:35 So it has to work at large scale

02:02:40 in order to overcome that power penalty,

02:02:43 but that’s possible.

02:02:45 It’s just, it’s gonna have to get that performance.

02:02:48 And this is sort of what you were asking about before

02:02:49 is like how much better than silicon would it need to be?

02:02:52 And the answer is, I don’t know.

02:02:54 I think if it’s just overall better than silicon

02:02:57 at a problem that a lot of people care about,

02:03:00 maybe it’s image classification,

02:03:02 maybe it’s facial recognition,

02:03:03 maybe it’s monitoring credit transactions, I don’t know,

02:03:07 then I think it will have a place.

02:03:09 It’s not gonna be in your cell phone,

02:03:10 but it could be in your data center.

02:03:12 So what about in terms of the data center,

02:03:16 I don’t know if you’re paying attention

02:03:17 to the various systems,

02:03:19 like Tesla recently announced DOJO,

02:03:23 which is a large scale machine learning training system,

02:03:27 that again, the bottleneck there

02:03:28 is probably going to be communication

02:03:30 between those systems.

02:03:32 Is there something from your work

02:03:34 on everything we’ve been talking about

02:03:38 in terms of superconductive hardware

02:03:41 that could be useful there?

02:03:43 Oh, I mean, okay, tomorrow, no.

02:03:46 In the long term, it could be the whole thing.

02:03:49 It could be nothing.

02:03:49 I don’t know, but definitely, definitely.

02:03:54 When you look at the,

02:03:55 so I don’t know that much about DOJO.

02:03:56 My understanding is that that’s new, right?

02:03:58 That’s just coming online.

02:04:01 Well, I don’t even know where it hasn’t come online.

02:04:06 And when you announce big, sexy,

02:04:09 so let me explain to you the way things work

02:04:11 in the world of business and marketing.

02:04:15 It’s not always clear where you are

02:04:18 on the coming online part of that.

02:04:20 So I don’t know where they are exactly,

02:04:22 but the vision is from a ground up

02:04:25 to build a very, very large scale,

02:04:28 modular machine learning, ASIC,

02:04:31 basically hardware that’s optimized

02:04:32 for training neural networks.

02:04:34 And of course, there’s a lot of companies

02:04:36 that are small and big working on this kind of problem.

02:04:39 The question is how to do it in a modular way

02:04:42 that has very fast communication.

02:04:45 The interesting aspect of Tesla is you have a company

02:04:49 that at least at this time is so singularly focused

02:04:54 on solving a particular machine learning problem

02:04:57 and is making obviously a lot of money doing so

02:05:00 because the machine learning problem

02:05:02 happens to be involved with autonomous driving.

02:05:05 So you have a system that’s driven by an application.

02:05:09 And that’s really interesting because you have maybe Google

02:05:15 working on TPUs and so on.

02:05:17 You have all these other companies with ASICs.

02:05:21 They’re usually more kind of always thinking general.

02:05:25 So I like it when it’s driven by a particular application

02:05:29 because then you can really get to the,

02:05:32 it’s somehow if you just talk broadly about intelligence,

02:05:37 you may not always get to the right solutions.

02:05:40 It’s nice to couple that sometimes

02:05:41 with specific clear illustration

02:05:45 of something that requires general intelligence,

02:05:47 which for me driving is one such case.

02:05:49 I think you’re exactly right.

02:05:51 Sometimes just having that focus on that application

02:05:54 brings a lot of people focuses their energy and attention.

02:05:57 I think that, so one of the things that’s appealing

02:06:00 about what you’re saying is not just

02:06:02 that the application is specific,

02:06:03 but also that the scale is big

02:06:06 and that the benefit is also huge.

02:06:10 Financial and to humanity.

02:06:12 Right, right, right.

02:06:13 Yeah, so I guess let me just try to understand

02:06:15 is the point of this dojo system

02:06:17 to figure out the parameters

02:06:21 that then plug into neural networks

02:06:23 and then you don’t need to retrain,

02:06:26 you just make copies of a certain chip

02:06:29 that has all the other parameters established or?

02:06:32 No, it’s straight up retraining a large neural network

02:06:36 over and over and over.

02:06:38 So you have to do it once for every new car?

02:06:41 No, no, you have to, so they do this interesting process,

02:06:44 which I think is a process for machine learning,

02:06:47 supervised machine learning systems

02:06:49 you’re going to have to do, which is you have a system,

02:06:53 you train your network once, it takes a long time.

02:06:56 I don’t know how long, but maybe a week.

02:06:58 Okay. To train.

02:07:00 And then you deploy it on, let’s say about a million cars.

02:07:05 I don’t know what the number is.

02:07:05 But that part, you just write software

02:07:07 that updates some weights in a table and yeah, okay.

02:07:10 But there’s a loop back.

02:07:12 Yeah, yeah, okay.

02:07:13 Each of those cars run into trouble, rarely,

02:07:18 but they catch the edge cases

02:07:23 of the performance of that particular system

02:07:26 and then send that data back

02:07:28 and either automatically or by humans,

02:07:31 that weird edge case data is annotated

02:07:34 and then the network has to become smart enough

02:07:37 to now be able to perform in those edge cases,

02:07:40 so it has to get retrained.

02:07:41 There’s clever ways of retraining different parts

02:07:43 of that network, but for the most part,

02:07:46 I think they prefer to retrain the entire thing.

02:07:49 So you have this giant monster

02:07:51 that kind of has to be retrained regularly.

02:07:54 I think the vision with Dojo is to have

02:07:58 a very large machine learning focused,

02:08:02 driving focused supercomputer

02:08:05 that then is sufficiently modular

02:08:07 that can be scaled to other machine learning applications.

02:08:11 So they’re not limiting themselves completely

02:08:12 to this particular application,

02:08:14 but this application is the way they kind of test

02:08:17 this iterative process of machine learning

02:08:19 is you make a system that’s very dumb,

02:08:23 deploy it, get the edge cases where it fails,

02:08:27 make it a little smarter, it becomes a little less dumb

02:08:30 and that iterative process achieves something

02:08:33 that you can call intelligent or is smart enough

02:08:36 to be able to solve this particular application.

02:08:37 So it has to do with training neural networks fast

02:08:43 and training neural networks that are large.

02:08:45 But also based on an extraordinary amount of diverse input.

02:08:49 Data, yeah.

02:08:50 And that’s one of the things,

02:08:51 so this does seem like one of those spaces

02:08:54 where the scale of superconducting optoelectronics,

02:08:58 the way that, so when you talk about the weaknesses,

02:09:02 like I said, okay, well, you have to cool it down.

02:09:04 At this scale, that’s fine.

02:09:05 Because that’s not too much of an added cost.

02:09:09 Most of your power is being dissipated

02:09:10 by the circuits themselves, not the cooling.

02:09:12 And also you have one centralized kind of cognitive hub,

02:09:19 if you will.

02:09:20 And so if we’re talking about putting

02:09:24 a superconducting system in a car, that’s questionable.

02:09:28 Do you really wanna cryostat

02:09:30 in the trunk of everyone in your car?

02:09:31 It’ll fit, it’s not that big of a deal,

02:09:32 but hopefully there’s a better way, right?

02:09:35 But since this is sort of a central supreme intelligence

02:09:39 or something like that,

02:09:40 and it needs to really have this massive data acquisition,

02:09:45 massive data integration,

02:09:47 I would think that that’s where large scale

02:09:49 spiking neural networks with vast communication

02:09:51 and all these things would have something

02:09:53 pretty tremendous to offer.

02:09:54 It’s not gonna happen tomorrow.

02:09:55 There’s a lot of development that needs to be done.

02:09:58 But we have to be patient with self driving cars

02:10:01 for a lot of reasons.

02:10:02 We were all optimistic that they would be here by now.

02:10:04 And okay, they are to some extent,

02:10:06 but if we’re thinking five or 10 years down the line,

02:10:09 it’s not unreasonable.

02:10:12 One other thing, let me just mention,

02:10:15 getting into self driving cars and technologies

02:10:17 that are using AI out in the world,

02:10:19 this is something NIST cares a lot about.

02:10:21 Elham Tabassi is leading up a much larger effort in AI

02:10:25 at NIST than my little project.

02:10:29 And really central to that mission

02:10:32 is this concept of trustworthiness.

02:10:35 So when you’re going to deploy this neural network

02:10:39 in every single automobile with so much on the line,

02:10:43 you have to be able to trust that.

02:10:45 So now how do we know that we can trust that?

02:10:48 How do we know that we can trust the self driving car

02:10:50 or the supercomputer that trained it?

02:10:53 There’s a lot of work there

02:10:54 and there’s a lot of that going on at NIST.

02:10:56 And it’s still early days.

02:10:58 I mean, you’re familiar with the problem and all that.

02:11:01 But there’s a fascinating dance in engineering

02:11:04 with safety critical systems.

02:11:06 There’s a desire in computer science,

02:11:08 just recently talked to Don Knuth,

02:11:13 for algorithms and for systems,

02:11:14 for them to be provably correct or provably safe.

02:11:17 And this is one other difference

02:11:20 between humans and biological systems

02:11:22 is we’re not provably anything.

02:11:24 And so there’s some aspect of imperfection

02:11:29 that we need to have built in,

02:11:32 like robustness to imperfection be part of our systems,

02:11:37 which is a difficult thing for engineers to contend with.

02:11:40 They’re very uncomfortable with the idea

02:11:42 that you have to be okay with failure

02:11:46 and almost engineer failure into the system.

02:11:49 Mathematicians hate it too.

02:11:50 But I think it was Turing who said something

02:11:53 along the lines of,

02:11:55 I can give you an intelligent system

02:11:57 or I can give you a flawless system,

02:11:59 but I can’t give you both.

02:12:00 And it’s in sort of creativity and abstract thinking

02:12:04 seem to rely somewhat on stochasticity

02:12:07 and not having components

02:12:11 that perform exactly the same way every time.

02:12:13 This is where like the disagreement I have with,

02:12:16 not disagreement, but a different view on the world.

02:12:18 I’m with Turing,

02:12:19 but when I talk to robotic, robot colleagues,

02:12:24 that sounds like I’m talking to robots,

02:12:26 colleagues that are roboticists,

02:12:29 the goal is perfection.

02:12:31 And to me is like, no,

02:12:33 I think the goal should be imperfection

02:12:38 that’s communicated.

02:12:40 And through the interaction between humans and robots,

02:12:44 that imperfection becomes a feature, not a bug.

02:12:49 Like together, seen as a system,

02:12:52 the human and the robot together

02:12:53 are better than either of them individually,

02:12:56 but the robot itself is not perfect in any way.

02:13:00 Of course, there’s a bunch of disagreements,

02:13:02 including with Mr. Elon about,

02:13:06 to me, autonomous driving is fundamentally

02:13:08 a human robot interaction problem,

02:13:10 not a robotics problem.

02:13:12 To Elon, it’s a robotics problem.

02:13:14 That’s actually an open and fascinating question,

02:13:18 whether humans can be removed from the loop completely.

02:13:24 We’ve talked about a lot of fascinating chemistry

02:13:27 and physics and engineering,

02:13:31 and we’re always running up against this issue

02:13:33 that nature seems to dictate what’s easy and what’s hard.

02:13:37 So you have this cool little paper

02:13:40 that I’d love to just ask you about.

02:13:43 It’s titled,

02:13:44 Does Cosmological Evolution Select for Technology?

02:13:48 So in physics, there’s parameters

02:13:53 that seem to define the way our universe works,

02:13:56 that physics works, that if it worked any differently,

02:13:59 we would get a very different world.

02:14:01 So it seems like the parameters are very fine tuned

02:14:04 to the kind of physics that we see.

02:14:06 All the beautiful E equals MC squared,

02:14:08 they would get these nice, beautiful laws.

02:14:10 It seems like very fine tuned for that.

02:14:13 So what you argue in this article

02:14:15 is it may be that the universe has also fine tuned

02:14:20 its parameters that enable the kind of technological

02:14:25 innovation that we see, the technology that we see.

02:14:29 Can you explain this idea?

02:14:31 Yeah, I think you’ve introduced it nicely.

02:14:33 Let me just try to say a few things in my language layout.

02:14:39 What is this fine tuning problem?

02:14:41 So physicists have spent centuries trying to understand

02:14:46 the system of equations that govern the way nature behaves,

02:14:51 the way particles move and interact with each other.

02:14:55 And as that understanding has become more clear over time,

02:15:00 it became sort of evident that it’s all well adjusted

02:15:07 to allow a universe like we see, very complex,

02:15:13 this large, long lived universe.

02:15:16 And so one answer to that is, well, of course it is

02:15:19 because we wouldn’t be here otherwise.

02:15:21 But I don’t know, that’s not very satisfying.

02:15:24 That’s sort of, that’s what’s known

02:15:25 as the weak anthropic principle.

02:15:27 It’s a statement of selection bias.

02:15:29 We can only observe a universe that is fit for us to live in.

02:15:33 So what does it mean for a universe

02:15:34 to be fit for us to live in?

02:15:36 Well, the pursuit of physics,

02:15:38 it is based partially on coming up with equations

02:15:42 that describe how things behave

02:15:44 and interact with each other.

02:15:46 But in all those equations you have,

02:15:48 so there’s the form of the equation,

02:15:49 sort of how different fields or particles

02:15:54 move in space and time.

02:15:56 But then there are also the parameters

02:15:58 that just tell you sort of the strength

02:16:01 of different couplings.

02:16:02 How strongly does a charged particle

02:16:05 couple to the electromagnetic field or masses?

02:16:07 How strongly does a particle couple

02:16:10 to the Higgs field or something like that?

02:16:12 And those parameters that define,

02:16:16 not the general structure of the equations,

02:16:19 but the relative importance of different terms,

02:16:23 they seem to be every bit as important

02:16:25 as the structure of the equations themselves.

02:16:27 And so I forget who it was.

02:16:29 Somebody, when they were working through this

02:16:31 and trying to see, okay, if I adjust the parameter,

02:16:34 this parameter over here,

02:16:34 call it the, say the fine structure constant,

02:16:36 which tells us the strength

02:16:37 of the electromagnetic interaction.

02:16:40 Oh boy, I can’t change it very much.

02:16:42 Otherwise nothing works.

02:16:43 The universe sort of doesn’t,

02:16:45 it just pops into existence and goes away

02:16:47 in a nanosecond or something like that.

02:16:48 And somebody had the phrase,

02:16:51 this looks like a put up job,

02:16:52 meaning every one of these parameters was dialed in.

02:16:57 It’s arguable how precisely they have to be dialed in,

02:17:00 but dialed in to some extent,

02:17:03 not just in order to enable our existence,

02:17:05 that’s a very anthropocentric view,

02:17:07 but to enable a universe like this one.

02:17:10 So, okay, maybe I think the majority position

02:17:14 of working physicists in the field is,

02:17:17 it has to be that way in order for us to exist.

02:17:18 We’re here, we shouldn’t be surprised

02:17:20 that that’s the way the universe is.

02:17:22 And I don’t know, for a while,

02:17:24 that never sat well with me,

02:17:26 but I just kind of moved on

02:17:28 because there are things to do

02:17:29 and a lot of exciting work.

02:17:31 It doesn’t depend on resolving this puzzle,

02:17:33 but as I started working more with technology,

02:17:39 getting into the more recent years of my career,

02:17:41 particularly when I started,

02:17:43 after having worked with silicon for a long time,

02:17:46 which was kind of eerie on its own,

02:17:49 but then when I switched over to superconductors,

02:17:51 I was just like, this is crazy.

02:17:53 It’s just absolutely astonishing

02:17:57 that our universe gives us superconductivity.

02:18:00 It’s one of the most beautiful physical phenomena

02:18:02 and it’s also extraordinarily useful for technology.

02:18:06 So you can argue that the universe

02:18:07 has to have the parameters it does for us to exist

02:18:11 because we couldn’t be here otherwise,

02:18:13 but why does it give us technology?

02:18:14 Why does it give us silicon that has this ideal oxide

02:18:18 that allows us to make a transistor

02:18:20 without trying that hard?

02:18:23 That can’t be explained by the same anthropic reasoning.

02:18:27 Yeah, so it’s asking the why question.

02:18:30 I mean, a slight natural extension of that question is,

02:18:34 I wonder if the parameters were different

02:18:39 if we would simply have just another set of paint brushes

02:18:44 to create totally other things

02:18:46 that wouldn’t look like anything

02:18:49 like the technology of today,

02:18:50 but would nevertheless have incredible complexity,

02:18:54 which is if you sort of zoom out and start defining things,

02:18:57 not by like how many batteries it needs

02:19:01 and whether it can make toast,

02:19:03 but more like how much complexity is within the system

02:19:06 or something like that.

02:19:07 Well, yeah, you can start to quantify things.

02:19:10 You’re exactly right.

02:19:10 So nowhere am I arguing that

02:19:13 in all of the vast parameter space

02:19:15 of everything that could conceivably exist

02:19:18 in the multiverse of nature,

02:19:20 there’s this one point in parameter space

02:19:23 where complexity arises.

02:19:25 I doubt it.

02:19:26 That would be a shameful waste of resources, it seems.

02:19:31 But it might be that we reside

02:19:33 at one place in parameter space

02:19:35 that has been adapted through an evolutionary process

02:19:40 to allow us to make certain technologies

02:19:43 that allow our particular kind of universe to arise

02:19:47 and sort of achieve the things it does.

02:19:49 See, I wonder if nature in this kind of discussion,

02:19:52 if nature is a catalyst for innovation

02:19:55 or if it’s a ceiling for innovation.

02:19:57 So like, is it going to always limit us?

02:20:00 Like you’re talking about silicon.

02:20:04 Is it just make it super easy to do awesome stuff

02:20:06 in a certain dimension,

02:20:08 but we could still do awesome stuff in other ways,

02:20:10 it’ll just be harder?

02:20:11 Or does it really set like the maximum we can do?

02:20:15 That’s a good thing to,

02:20:17 that’s a good subject to discuss.

02:20:19 I guess I feel like we need to lay

02:20:20 a little bit more groundwork.

02:20:23 So I want to make sure that

02:20:27 I introduce this in the context

02:20:29 of Lee Smolin’s previous idea.

02:20:31 So who’s Lee Smolin and what kind of ideas does he have?

02:20:35 Okay, Lee Smolin is a theoretical physicist

02:20:39 who back in the late 1980s published a paper

02:20:42 in the early 1990s introduced this idea

02:20:45 of cosmological natural selection,

02:20:47 which argues that the universe did evolve.

02:20:51 So his paper was called, did the universe evolve?

02:20:54 And I gave myself the liberty of titling my paper

02:20:59 does cosmological selection

02:21:01 or does cosmological evolution select for technology

02:21:03 in reference to that.

02:21:05 So he introduced that idea decades ago.

02:21:08 Now he primarily works on quantum gravity,

02:21:12 loop quantum gravity, other approaches to

02:21:14 unifying quantum mechanics with general relativity,

02:21:19 as you can read about in his most recent book, I believe,

02:21:22 and he’s been on your show as well.

02:21:24 So, but I want to introduce this idea

02:21:27 of cosmological natural selection,

02:21:29 because I think that is one of the core ideas

02:21:32 that could change our understanding

02:21:35 of how the universe got here, our role in it,

02:21:37 what technology is doing here.

02:21:39 But there’s a couple more pieces

02:21:41 that need to be set up first.

02:21:42 So the beginning of our universe is largely accepted

02:21:46 to be the big bang.

02:21:47 And what that means is if you look back in time

02:21:49 by looking far away in space,

02:21:52 you see that everything used to be at one point

02:21:56 and it expanded away from there.

02:21:58 There was an era in the evolutionary process of our universe

02:22:04 that was called inflation.

02:22:05 And this idea was developed primarily by Alan Guth

02:22:08 and others, Andre Linde and others in the 80s.

02:22:13 And this idea of inflation is basically that

02:22:16 when a singularity begins this process of growth,

02:22:25 there can be a temporary stage

02:22:27 where it just accelerates incredibly rapidly.

02:22:30 And based on quantum field theory,

02:22:33 this tells us that this should produce matter

02:22:35 in precisely the proportions that we find

02:22:37 of hydrogen and helium in the big bang,

02:22:39 lithium too, lithium also, and other things too.

02:22:44 So the predictions that come out of big bang

02:22:47 inflationary cosmology have stood up extremely well

02:22:50 to empirical verification,

02:22:52 the cosmic microwave background, things like this.

02:22:55 So most scientists working in the field

02:22:59 think that the origin of our universe is the big bang.

02:23:03 And I base all my thinking on that as well.

02:23:08 I’m just laying this out there so that people understand

02:23:11 that where I’m coming from is an extension,

02:23:14 not a replacement of existing well founded ideas.

02:23:19 In a paper, I believe it was 1986 with Alan Guth

02:23:23 and another author Farhi,

02:23:26 they wrote that a big bang,

02:23:30 I don’t remember the exact quote,

02:23:31 a big bang is inextricably linked with a black hole.

02:23:35 The singularity that we call our origin

02:23:39 is mathematically indistinguishable from a black hole.

02:23:42 They’re the same thing.

02:23:44 And Lee Smolin based his thinking on that idea,

02:23:48 I believe, I don’t mean to speak for him,

02:23:50 but this is my reading of it.

02:23:52 So what Lee Smolin will say is that

02:23:56 a black hole in one universe is a big bang

02:23:58 in another universe.

02:24:00 And this allows us to have progeny, offspring.

02:24:04 So a universe can be said to have come

02:24:08 before another universe.

02:24:10 And very crucially, Smolin argues,

02:24:14 I think this is potentially one of the great ideas

02:24:16 of all time, that’s my opinion,

02:24:18 that when a black hole forms, it’s not a classical entity,

02:24:22 it’s a quantum gravitational entity.

02:24:24 So it is subject to the fluctuations

02:24:27 that are inherent in quantum mechanics, the properties,

02:24:34 what we’re calling the parameters

02:24:35 that describe the physics of that system

02:24:38 are subject to slight mutations

02:24:40 so that the offspring universe

02:24:42 does not have the exact same parameters

02:24:45 defining its physics as its parent universe.

02:24:48 They’re close, but they’re a little bit different.

02:24:50 And so now you have a mechanism for evolution,

02:24:55 for natural selection.

02:24:57 So there’s mutation, so there’s,

02:24:59 and then if you think about the DNA of the universe

02:25:03 are the basic parameters that govern its laws.

02:25:05 Exactly, so what Smolin said is our universe results

02:25:11 from an evolutionary process that can be traced back

02:25:14 some, he estimated, 200 million generations.

02:25:17 Initially, there was something like a vacuum fluctuation

02:25:20 that produced through random chance a universe

02:25:25 that was able to reproduce just one.

02:25:27 So now it had one offspring.

02:25:28 And then over time, it was able to make more and more

02:25:30 until it evolved into a highly structured universe

02:25:35 with a very long lifetime, with a great deal of complexity

02:25:40 and importantly, especially importantly for Lee Smolin,

02:25:44 stars, stars make black holes.

02:25:47 Therefore, we should expect our universe

02:25:49 to be optimized, have its physical parameters optimized

02:25:53 to make very large numbers of stars

02:25:55 because that’s how you make black holes

02:25:57 and black holes make offspring.

02:25:59 So we expect the physics of our universe to have evolved

02:26:03 to maximize fecundity, the number of offspring.

02:26:06 And the way Lee Smolin argues you do that

02:26:09 is through stars that the biggest ones die

02:26:12 in these core collapse supernova

02:26:13 that make a black hole and a child.

02:26:15 Okay, first of all, I agree with you

02:26:19 that this is back to our fractal view of everything

02:26:24 from intelligence to our universe.

02:26:27 That is very compelling and a very powerful idea

02:26:31 that unites the origin of life

02:26:36 and perhaps the origin of ideas and intelligence.

02:26:39 So from a Dawkins perspective here on earth,

02:26:42 the evolution of those and then the evolution

02:26:45 of the laws of physics that led to us.

02:26:51 I mean, it’s beautiful.

02:26:52 And then you stacking on top of that,

02:26:54 that maybe we are one of the offspring.

02:26:57 Right, okay, so before getting into where I’d like

02:27:02 to take that idea, let me just a little bit more groundwork.

02:27:05 There is this concept of the multiverse

02:27:07 and it can be confusing.

02:27:08 Different people use the word multiverse in different ways.

02:27:11 In the multiverse that I think is relevant to picture

02:27:16 when trying to grasp Lee Smolin’s idea,

02:27:20 essentially every vacuum fluctuation

02:27:24 can be referred to as a universe.

02:27:25 It occurs, it borrows energy from the vacuum

02:27:28 for some finite amount of time

02:27:30 and it evanesces back into the quantum vacuum.

02:27:33 And ideas of Guth before that and Andrei Linde

02:27:38 with eternal inflation aren’t that different

02:27:42 that you would expect nature

02:27:44 due to the quantum properties of the vacuum,

02:27:46 which we know exist, they’re measurable

02:27:49 through things like the Casimir effect and others.

02:27:52 You know that there are these fluctuations

02:27:54 that are occurring.

02:27:55 What Smolin is arguing is that there is

02:27:58 this extensive multiverse, that this universe,

02:28:01 what we can measure and interact with

02:28:04 is not unique in nature.

02:28:07 It’s just our residents, it’s where we reside.

02:28:10 And there are countless, potentially infinity

02:28:13 other universes, other entire evolutionary trajectories

02:28:17 that have evolved into things like

02:28:19 what you were mentioning a second ago

02:28:21 with different parameters and different ways

02:28:24 of achieving complexity and reproduction

02:28:26 and all that stuff.

02:28:27 So it’s not that the evolutionary process

02:28:30 is a funnel towards this end point, not at all.

02:28:34 Just like the biological evolutionary process

02:28:37 that has occurred within our universe

02:28:39 is not a unique route toward achieving

02:28:42 one specific chosen kind of species.

02:28:45 No, we have extraordinary diversity around us.

02:28:49 That’s what evolution does.

02:28:50 And for any one species like us,

02:28:52 you might feel like we’re at the center of this process.

02:28:54 We’re the destination of this process,

02:28:57 but we’re just one of the many

02:28:59 nearly infinite branches of this process.

02:29:02 And I suspect it is exactly infinite.

02:29:04 I mean, I just can’t understand how with this idea,

02:29:09 you can ever draw a boundary around it and say,

02:29:11 no, the universe, I mean, the multiverse

02:29:13 has 10 to the one quadrillion components,

02:29:17 but not infinity.

02:29:18 I don’t know that.

02:29:20 Well, yeah, I have cognitively incapable

02:29:24 as I think all of us are

02:29:25 and truly understanding the concept of infinity.

02:29:29 And the concept of nothing as well.

02:29:31 And nothing, but also the concept of a lot

02:29:34 is pretty difficult.

02:29:35 I can just, I can count.

02:29:37 I run out of fingers at a certain point

02:29:39 and then you’re screwed.

02:29:40 And when you’re wearing shoes

02:29:41 and you can’t even get down to your toes, it’s like.

02:29:44 It’s like, all right, a thousand fine, a million.

02:29:47 Is that what?

02:29:48 And then it gets crazier and crazier.

02:29:50 Right, right.

02:29:51 So this particular, so when we say technology, by the way,

02:29:55 I mean, there’s some, not to over romanticize the thing,

02:30:00 but there is some aspect about this branch of ours

02:30:04 that allows us to, for the universe to know itself.

02:30:08 Yes, yes.

02:30:08 So to be, to have like little conscious cognitive fingers

02:30:15 that are able to feel like to scratch the head.

02:30:18 Right, right, right.

02:30:19 To be able to construct E equals MC squared

02:30:22 and to introspect, to start to gain some understanding

02:30:25 of the laws that govern it.

02:30:27 Isn’t that, isn’t that kind of amazing?

02:30:32 Okay, I’m just human, but it feels like that,

02:30:35 if I were to build a system that does this kind of thing,

02:30:39 that evolves laws of physics, that evolves life,

02:30:42 that evolves intelligence, that my goal would be

02:30:45 to come up with things that are able to think about itself.

02:30:48 Right, aren’t we kind of close to the design specs,

02:30:53 the destination?

02:30:54 We’re pretty close, I don’t know.

02:30:56 I mean, I’m spending my career designing things

02:30:58 that I hope will think about themselves,

02:30:59 so you and I aren’t too far apart on that one.

02:31:02 But then maybe that problem is a lot harder

02:31:05 than we imagine.

02:31:06 Maybe we need to.

02:31:07 Let’s not get, let’s not get too far

02:31:09 because I want to emphasize something that,

02:31:12 what you’re saying is, isn’t it fascinating

02:31:14 that the universe evolved something

02:31:16 that can be conscious, reflect on itself?

02:31:19 But Lee Smolin’s idea didn’t take us there, remember?

02:31:23 It took us to stars.

02:31:25 Lee Smolin has argued, I think,

02:31:29 right on almost every single way

02:31:32 that cosmological natural selection

02:31:35 could lead to a universe with rich structure.

02:31:38 And he argued that the structure,

02:31:41 the physics of our universe is designed

02:31:43 to make a lot of stars so that they can make black holes.

02:31:46 But that doesn’t explain what we’re doing here.

02:31:48 In order for that to be an explanation of us,

02:31:51 what you have to assume is that once you made that universe

02:31:55 that was capable of producing stars,

02:31:58 life, planets, all these other things,

02:32:00 we’re along for the ride.

02:32:01 They got lucky.

02:32:02 We’re kind of arising, growing up in the cracks,

02:32:05 but the universe isn’t here for us.

02:32:06 We’re still kind of a fluke in that picture.

02:32:09 And I can’t, I don’t necessarily have

02:32:12 like a philosophical opposition to that stance.

02:32:14 It’s just not, okay, so I don’t think it’s complete.

02:32:20 So it seems like whatever we got going on here to you,

02:32:22 it seems like whatever we have here on earth

02:32:25 seems like a thing you might want to select for

02:32:28 in this whole big process.

02:32:29 Exactly.

02:32:30 So if what you are truly,

02:32:32 if your entire evolutionary process

02:32:34 only cares about fecundity,

02:32:36 it only cares about making offspring universes

02:32:39 because then there’s gonna be the most of them

02:32:41 in that local region of hyperspace,

02:32:45 which is the set of all possible universes, let’s say.

02:32:50 You don’t care how those universes are made.

02:32:52 You know they have to be made by black holes.

02:32:54 This is what inflationary theory tells us.

02:32:57 The big bang tells us that black holes make universes.

02:33:02 But what if there was a technological means

02:33:04 to make universes?

02:33:05 Stars require a ton of matter

02:33:09 because they’re not thinking very carefully

02:33:11 about how you make a black hole.

02:33:12 They’re just using gravity, you know?

02:33:16 But if we devise technologies

02:33:19 that can efficiently compress matter into a singularity,

02:33:23 it turns out that if you can compress about 10 kilograms

02:33:26 into a very small volume,

02:33:28 that will make a black hole

02:33:29 that is likely highly probable to inflate

02:33:32 into its own offspring universe.

02:33:34 This is according to calculations done by other people

02:33:37 who are professional quantum theorists,

02:33:38 quantum field theorists,

02:33:40 and I hope I am grasping what they’re telling me correctly.

02:33:44 I am somewhat of a translator here.

02:33:47 But so that’s the position

02:33:50 that is particularly intriguing to me,

02:33:52 which is that what might have happened is that,

02:33:56 okay, this particular branch on the vast tree of evolution,

02:34:01 cosmological evolution that we’re talking about,

02:34:03 not biological evolution within our universe,

02:34:05 but cosmological evolution,

02:34:07 went through exactly the process

02:34:09 that Elise Mullen described,

02:34:10 got to the stage where stars were making lots of black holes

02:34:15 but then continued to evolve and somehow bridged that gap

02:34:19 and made intelligence and intelligence

02:34:22 capable of devising technologies

02:34:24 because technologies, intelligent species

02:34:27 working in conjunction with technologies

02:34:29 could then produce even more.

02:34:32 Yeah, more efficiently, more faster and better

02:34:35 and more different.

02:34:36 Then you start to have different kind of mechanisms

02:34:38 and mutation perhaps, all that kind of stuff.

02:34:40 And so if you do a simple calculation that says,

02:34:43 all right, if I want to,

02:34:44 we know roughly how many core collapse supernovae

02:34:50 have resulted in black holes in our galaxy

02:34:54 since the beginning of the universe

02:34:55 and it’s something like a billion.

02:34:57 So then you would have to estimate

02:35:00 that it would be possible for a technological civilization

02:35:04 to produce more than a billion black holes

02:35:07 with the energy and matter at their disposal.

02:35:09 And so one of the calculations in that paper,

02:35:12 back of the envelope,

02:35:14 but I think revealing nonetheless is that

02:35:16 if you take a relatively common asteroid,

02:35:20 something that’s about a kilometer in diameter,

02:35:23 what I’m thinking of is just scrap material

02:35:26 laying around in our solar system

02:35:28 and break it up into 10 kilogram chunks

02:35:31 and turn each of those into a universe,

02:35:33 then you would have made at least a trillion black holes

02:35:38 outpacing the star production rate

02:35:41 by some three orders of magnitude.

02:35:43 That’s one asteroid.

02:35:44 So now if you envision an intelligent species

02:35:46 that would potentially have been devised initially

02:35:50 by humans, but then based on superconducting

02:35:53 optoelectronic networks, no doubt,

02:35:55 and they go out and populate,

02:35:57 they don’t have to fill the galaxy.

02:35:58 They just have to get out to the asteroid belt.

02:36:01 They could potentially dramatically outpace

02:36:05 the rate at which stars are producing offspring universes.

02:36:07 And then wouldn’t you expect that

02:36:10 that’s where we came from instead of a star?

02:36:13 Yeah, so you have to somehow become masters of gravity,

02:36:16 so like, or generate.

02:36:17 John, this is really gravity.

02:36:18 So stars make black holes with gravity,

02:36:20 but any force that can make the energy density

02:36:26 can compactify matter to produce

02:36:28 a great enough energy density can form a singularity.

02:36:31 It doesn’t, it would not likely be gravity.

02:36:33 It’s the weakest force.

02:36:34 You’re more likely to use something like the technologies

02:36:38 that we’re developing for fusion, for example.

02:36:40 So I don’t know, the Large Ignition Facility

02:36:44 recently blasted a pellet with 100 really bright lasers

02:36:50 and caused that to get dense enough

02:36:53 to engage in nuclear fusion.

02:36:55 So something more like that,

02:36:56 or a tokamak with a really hot plasma, I’m not sure.

02:36:59 Something, I don’t know exactly how it would be done.

02:37:02 I do like the idea of that,

02:37:04 especially just been reading a lot about gravitational waves

02:37:07 and the fact that us humans with our technological

02:37:10 capabilities, one of the most impressive

02:37:14 technological accomplishments of human history is LIGO,

02:37:17 being able to precisely detect gravitational waves.

02:37:20 I’m particularly find appealing the idea

02:37:25 that other alien civilizations from very far distances

02:37:29 communicate with gravity, with gravitational waves,

02:37:34 because as you become greater and greater master of gravity,

02:37:37 which seems way out of reach for us right now,

02:37:40 maybe that seems like a effective way of sending signals,

02:37:44 especially if your job is to manufacture black holes.

02:37:48 Right.

02:37:49 So that, so let me ask there,

02:37:53 whatever, I mean, broadly thinking,

02:37:56 because we tend to think other alien civilizations

02:37:58 would be very human like,

02:38:00 but if we think of alien civilizations out there

02:38:04 as basically generators of black holes,

02:38:07 however they do it, because they got stars,

02:38:10 do you think there’s a lot of them

02:38:12 in our particular universe out there?

02:38:17 In our universe?

02:38:20 Well, okay, let me ask, okay, this is great.

02:38:23 Let me ask a very generic question

02:38:26 and then let’s see how you answer it,

02:38:29 which is how many alien civilizations are out there?

02:38:35 If the hypothesis that I just described

02:38:38 is on the right track,

02:38:40 it would mean that the parameters of our universe

02:38:43 have been selected so that intelligent civilizations

02:38:48 will occur in sufficient numbers

02:38:51 so that if they reach something

02:38:54 like supreme technological maturity,

02:38:56 let’s define that as the ability to produce black holes,

02:39:00 then that’s not a highly improbable event.

02:39:02 It doesn’t need to happen often

02:39:05 because as I just described,

02:39:06 if you get one of them in a galaxy,

02:39:09 you’re gonna make more black holes

02:39:10 than the stars in that galaxy.

02:39:12 But there’s also not a super strong motivation,

02:39:16 well, it’s not obvious that you need them

02:39:21 to be ubiquitous throughout the galaxy.

02:39:23 Right.

02:39:24 One of the things that I try to emphasize in that paper

02:39:27 is that given this idea

02:39:30 of how our parameters might’ve been selected,

02:39:35 it’s clear that it’s a series of trade offs, right?

02:39:39 If you make, I mean, in order for intelligent life

02:39:42 of our variety or anything resembling us to occur,

02:39:45 you need a bunch of stuff, you need stars.

02:39:47 So that’s right back to Smolin’s roots of this idea,

02:39:51 but you also need water to have certain properties.

02:39:54 You need things like the rocky planets,

02:39:58 like the Earth to be within the habitable zone,

02:40:00 all these things that you start talking about

02:40:02 in the field of astrobiology,

02:40:06 trying to understand life in the universe,

02:40:08 but you can’t over emphasize,

02:40:10 you can’t tune the parameters so precisely

02:40:13 to maximize the number of stars

02:40:15 or to give water exactly the properties

02:40:18 or to make rocky planets like Earth the most numerous.

02:40:22 You have to compromise on all these things.

02:40:24 And so I think the way to test this idea

02:40:27 is to look at what parameters are necessary

02:40:30 for each of these different subsystems,

02:40:32 and I’ve laid out a few that I think are promising,

02:40:35 there could be countless others,

02:40:36 and see how changing the parameters

02:40:40 makes it more or less likely that stars would form

02:40:43 and have long lifetimes or that rocky planets

02:40:46 in the habitable zone are likely to form,

02:40:48 all these different things.

02:40:49 So we can test how much these things are in a tug of war

02:40:53 with each other, and the prediction would be

02:40:56 that we kind of sit at this central point

02:40:58 where if you move the parameters too much,

02:41:02 stars aren’t stable, or life doesn’t form,

02:41:05 or technology’s infeasible,

02:41:07 because life alone, at least the kind of life

02:41:10 that we know of, cannot make black holes.

02:41:14 We don’t have this, well, I’m speaking for myself,

02:41:16 you’re a very fit and strong person,

02:41:18 but it might be possible for you,

02:41:20 but not for me to compress matter.

02:41:22 So we need these technologies, but we don’t know,

02:41:25 we have not been able to quantify yet

02:41:28 how finely adjusted the parameters would need to be

02:41:33 in order for silicon to have the properties it does.

02:41:35 Okay, this is not directly speaking to what you’re saying,

02:41:37 you’re getting to the Fermi paradox,

02:41:39 which is where are they, where are the life forms out there,

02:41:42 how numerous are they, that sort of thing.

02:41:44 What I’m trying to argue is that

02:41:46 if this framework is on the right track,

02:41:50 a potentially correct explanation for our existence,

02:41:53 we, it doesn’t necessarily predict

02:41:56 that intelligent civilizations are just everywhere,

02:41:59 because even if you just get one of them in a galaxy,

02:42:02 which is quite rare, it could be enough

02:42:05 to dramatically increase the fecundity

02:42:08 of the universe as a whole.

02:42:10 Yeah, and I wonder, once you start generating

02:42:12 the offspring for universes, black holes,

02:42:15 how that has effect on the,

02:42:18 what kind of effect does it have

02:42:19 on the other candidate’s civilizations

02:42:24 within that universe?

02:42:26 Maybe it has a destructive aspect,

02:42:28 or there could be some arguments

02:42:29 about once you have a lot of offspring,

02:42:32 that that just quickly accelerates

02:42:34 to where the other ones can’t even catch up.

02:42:35 It could, but I guess if you want me

02:42:39 to put my chips on the table or whatever,

02:42:42 I think I come down more on the side

02:42:46 that intelligent life civilizations are rare.

02:42:52 And I guess I follow Max Tegmark here.

02:42:57 And also there’s a lot of papers coming out recently

02:43:01 in the field of astrobiology that are seeming to say,

02:43:04 all right, you just work through the numbers

02:43:06 on some modified Drake equation or something like that.

02:43:09 And it looks like it’s not improbable.

02:43:13 You shouldn’t be surprised that an intelligent species

02:43:16 has arisen in our galaxy,

02:43:18 but if you think there’s one the next solar system over,

02:43:20 it’s highly improbable.

02:43:21 So I can see that the number,

02:43:23 the probability of finding a civilization in a galaxy,

02:43:28 maybe it’s most likely that you’re gonna find

02:43:31 one to a hundred or something.

02:43:32 But okay, now it’s really important

02:43:34 to put a time window on that, I think,

02:43:36 because does that mean in the entire lifetime of the galaxy

02:43:40 before it, so for in our case, before we run into Andromeda,

02:43:49 I think it’s highly probable, I shouldn’t say I think,

02:43:53 it’s tempting to believe that it’s highly probable

02:43:56 that in that entire lifetime of your galaxy,

02:44:00 you’re gonna get at least one intelligent species,

02:44:02 maybe thousands or something like that.

02:44:05 But it’s also, I think, a little bit naive to think

02:44:10 that they’re going to coincide in time

02:44:13 and we’ll be able to observe them.

02:44:14 And also, if you look at the span of life on Earth,

02:44:20 the Earth history, it was surprising to me

02:44:24 to kind of look at the amount of time,

02:44:27 first of all, the short amount of time,

02:44:29 there’s no life, it’s surprising.

02:44:31 Life sprang up pretty quickly.

02:44:33 It’s single cell.

02:44:35 But that’s the point I’m trying to make

02:44:36 is like so much of life on Earth

02:44:42 was just like single cell organisms, like most of it.

02:44:45 Most of it was like boring bacteria type of stuff.

02:44:48 Well, bacteria are fascinating, but I take your point.

02:44:50 No, I get it.

02:44:51 I mean, no offense to them.

02:44:52 But this kind of speaking from the perspective

02:44:56 of your paper of something that’s able

02:44:58 to generate technology as we kind of understand it,

02:45:01 that’s a very short moment in time

02:45:03 relative to that full history of life on Earth.

02:45:08 And maybe our universe is just saturated

02:45:12 with bacteria like humans.

02:45:15 Right.

02:45:17 But not the special extra AGI super humans,

02:45:24 that those are very rare.

02:45:25 And once those spring up, everything just goes to like,

02:45:30 it accelerates very quickly.

02:45:33 Yeah, we just don’t have enough data to really say,

02:45:36 but I find this whole subject extremely engaging.

02:45:40 I mean, there’s this concept,

02:45:41 I think it’s called the Rare Earth Hypothesis,

02:45:43 which is that basically stating that,

02:45:46 okay, microbes were here right away

02:45:49 after the Hadian era where we were being bombarded.

02:45:52 Well, after, yeah, bombarded by comets, asteroids,

02:45:54 things like that, and also after the moon formed.

02:45:57 So once things settled down a little bit,

02:45:59 in a few hundred million years,

02:46:02 you have microbes everywhere.

02:46:03 And it could have been, we don’t know exactly

02:46:05 when it could have been remarkably brief that that took.

02:46:08 So it does indicate that, okay,

02:46:10 life forms relatively easily.

02:46:12 I think that alone is sort of a checker on the scale

02:46:15 for the argument that the parameters that allow

02:46:21 even microbial life to form are not just a fluke.

02:46:24 But anyway, that aside, yes,

02:46:27 then there was this long dormant period,

02:46:29 not dormant, things were happening,

02:46:31 but important things were happening

02:46:34 for some two and a half billion years or something

02:46:37 after the metabolic process

02:46:40 that releases oxygen was developed.

02:46:42 Then basically the planet’s just sitting there,

02:46:46 getting more and more oxygenated,

02:46:47 more and more oxygenated until it’s enough

02:46:50 that you can build these large, complex organisms.

02:46:54 And so the Rare Earth Hypothesis would argue

02:46:56 that the microbes are common everywhere

02:47:01 in any planet that’s roughly in the habitable zone

02:47:04 and has some water on it, it’s probably gonna have those.

02:47:06 But then getting to this Cambrian explosion

02:47:09 that happened some between 500 and 600 million years ago,

02:47:13 that’s rare, you know?

02:47:16 And I buy that, I think that is rare.

02:47:19 So if you say how much life is in our galaxy,

02:47:21 I think that’s probably the right answer

02:47:24 is that microbes are everywhere.

02:47:26 Cambrian explosion is extremely rare.

02:47:29 And then, but the Cambrian explosion kind of went like that

02:47:32 where within a couple of tens or a hundred million years,

02:47:38 all of these body plans came into existence.

02:47:40 And basically all of the body plans

02:47:43 that are now in existence on the planet

02:47:46 were formed in that brief window

02:47:48 and we’ve just been shuffling around since then.

02:47:51 So then what caused humans to pop out of that?

02:47:54 I mean, that could be another extremely rare threshold

02:48:01 that a planet roughly in the habitable zone with water

02:48:06 is not guaranteed to cross, you know?

02:48:08 To me, it’s fascinating for being humble,

02:48:10 like the humans cannot possibly be the most amazing thing

02:48:13 that such, if you look at the entirety of the system

02:48:15 that Lee Smolin and you paint,

02:48:17 that cannot possibly be the most amazing thing

02:48:20 that process generates.

02:48:21 So like, if you look at the evolution,

02:48:23 what’s the equivalent in the cosmological evolution

02:48:27 and its selection for technology,

02:48:29 the equivalent of the human eye or the human brain?

02:48:32 Universes that are able to do some like,

02:48:35 they don’t need the damn stars.

02:48:37 They’re able to just do some incredible generation

02:48:42 of complexity fast, like much more than,

02:48:46 if you think about it,

02:48:47 it’s like most of our universe is pretty freaking boring.

02:48:50 There’s not much going on, there’s a few rocks flying around

02:48:53 and there’s some like apes

02:48:54 that are just like doing podcasts on some weird planet.

02:49:00 It just seems very inefficient.

02:49:02 If you think about like the amazing thing in the human eye,

02:49:05 the visual cortex can do, the brain, the nervous,

02:49:09 everything that makes us more powerful

02:49:12 than single cell organisms.

02:49:15 Like if there’s an equivalent of that for universes,

02:49:19 like the richness of physics

02:49:21 that could be expressed

02:49:24 through a particular set of parameters.

02:49:26 Like, I mean, like for me,

02:49:31 I’m a sort of from a computer science perspective,

02:49:33 huge fan of cellular automata,

02:49:35 which is a nice sort of pretty visual way

02:49:39 to illustrate how different laws

02:49:42 can result in drastically different levels of complexity.

02:49:46 So like, it’s like, yeah, okay.

02:49:49 So we’re all like celebrating,

02:49:50 look, our little cellular automata

02:49:52 is able to generate pretty triangles and squares

02:49:54 and therefore we achieve general intelligence.

02:49:57 And then there’ll be like some badass Chuck Norris type,

02:50:01 like universal Turing machine type of cellular automata.

02:50:06 They’re able to generate other cellular automata

02:50:09 that does any arbitrary level of computation off the bat.

02:50:14 Like those have to then exist.

02:50:16 And then we’re just like, we’ll be forgotten.

02:50:19 This story, this podcast just entertains

02:50:23 a few other apes for a few months.

02:50:26 Well, I’m kind of surprised to hear your cynicism.

02:50:30 No, I’m very up.

02:50:32 I usually think of you as like one who celebrates humanity

02:50:36 and all its forms and things like that.

02:50:37 And I guess I just, I don’t,

02:50:39 I see it the way you just described.

02:50:41 I mean, okay, we’ve been here for 13.7 billion years

02:50:44 and you’re saying, gosh, that’s a long time.

02:50:47 Let’s get on with the show already.

02:50:48 Some other universe could have kicked our butt by now,

02:50:51 but that’s putting a characteristic time.

02:50:55 I mean, why is 13.7 billion a long time?

02:50:58 I mean, compared to what?

02:51:00 I guess, so when I look at our universe,

02:51:02 I see this extraordinary hierarchy

02:51:05 that has developed over that time.

02:51:08 So at the beginning, it was a chaotic mess of some plasma

02:51:13 and nothing interesting going on there.

02:51:16 And even for the first stars to form,

02:51:18 that a lot of really interesting evolutionary processes

02:51:23 had to occur, by evolutionary in that sense,

02:51:26 I just mean taking place over extended periods of time

02:51:30 and structures are forming then.

02:51:32 And then it took that first generation of stars

02:51:34 in order to produce the metals

02:51:38 that then can more efficiently produce

02:51:41 another generation of stars.

02:51:42 We’re only the third generation of stars.

02:51:44 So we might still be pretty quick to the game here.

02:51:47 So, but I don’t think, I don’t, okay.

02:51:51 So then you have these stars

02:51:52 and then you have solar systems on those solar systems.

02:51:54 You have rocky worlds, you have gas giants,

02:51:58 like all this complexity.

02:51:59 And then you start getting life

02:52:01 and the complexity that’s evolved

02:52:03 through the evolutionary process in life forms

02:52:06 is just, it’s not a let down to me.

02:52:09 Just seeing that.

02:52:10 Some of it is like some of the planets is like icy,

02:52:14 it’s like different flavors of ice cream.

02:52:16 They’re icy, but there might be water underneath.

02:52:18 All kinds of life forms with some volcanoes,

02:52:21 all kinds of weird stuff.

02:52:22 No, no, I don’t, I think it’s beautiful.

02:52:24 I think our life is beautiful.

02:52:25 And I think it was designed that by design,

02:52:29 the scarcity of the whole thing.

02:52:31 I think mortality, as terrifying as it is,

02:52:33 is fundamental to the whole reason we enjoy everything.

02:52:37 No, I think it’s beautiful.

02:52:38 I just think that all of us conscious beings

02:52:42 in the grand scheme of basically every scale

02:52:45 will be completely forgotten.

02:52:46 Well, that’s true.

02:52:47 I think everything is transient

02:52:49 and that would go back to maybe something more like Lao Tzu,

02:52:52 the Tao Te Ching or something where it’s like,

02:52:55 yes, there is nothing but change.

02:52:57 There is nothing but emergence and dissolve and that’s it.

02:53:00 But I just, in this picture,

02:53:03 this hierarchy that’s developed,

02:53:04 I don’t mean to say that now it gets to us

02:53:06 and that’s the pinnacle.

02:53:07 In fact, I think at a high level,

02:53:10 the story I’m trying to tease out in my research is about,

02:53:14 okay, well, so then what’s the next level of hierarchy?

02:53:17 And if it’s, okay, we’re kind of pretty smart.

02:53:21 I mean, talking about people like Lee Small

02:53:23 and Alan Guth, Max Tegmark, okay, we’re really smart.

02:53:26 Talking about me, okay, we’re kind of,

02:53:28 we can find our way to the grocery store or whatever,

02:53:30 but what’s next?

02:53:33 I mean, what if there’s another level of hierarchy

02:53:36 that grows on top of us

02:53:37 that is even more profoundly capable?

02:53:40 And I mean, we’ve talked a lot

02:53:42 about superconducting sensors.

02:53:43 Imagine these cognitive systems far more capable than us

02:53:48 residing somewhere else in the solar system

02:53:52 off of the surface of the earth,

02:53:53 where it’s much darker, much colder,

02:53:55 much more naturally suited to them.

02:53:57 And they have these sensors that can detect single photons

02:54:00 of light from radio waves out to all across the spectrum

02:54:04 of the gamma rays and just see the whole universe.

02:54:07 And they just live in space

02:54:08 with these massive collection optics so that they,

02:54:12 what do they do?

02:54:13 They just look out and experience that vast array

02:54:18 of what’s being developed.

02:54:22 And if you’re such a system,

02:54:25 presumably you would do some things for fun.

02:54:28 And the kind of fun thing I would do

02:54:31 as somebody who likes video games

02:54:33 is I would create and maintain

02:54:37 and observe something like earth.

02:54:42 So in some sense, we’re like all what players on a stage

02:54:47 for this superconducting cold computing system out there.

02:54:54 I mean, all of this is fascinating to think.

02:54:56 The fact that you’re actually designing systems

02:54:59 here on earth that are trying to push this technological

02:55:01 at the very cutting edge and also thinking about

02:55:04 how does the like the evolution of physical laws

02:55:09 lead us to the way we are is fascinating.

02:55:14 That coupling is fascinating.

02:55:15 It’s like the ultimate rigorous application of philosophy

02:55:20 to the rigorous application of engineering.

02:55:23 So Jeff, you’re one of the most fascinating.

02:55:26 I’m so glad I did not know much about you

02:55:29 except through your work.

02:55:30 And I’m so glad we got this chance to talk.

02:55:34 You’re one of the best explainers

02:55:37 of exceptionally difficult concepts.

02:55:40 And you’re also, speaking of like fractal,

02:55:44 you’re able to function intellectually

02:55:46 at all levels of the stack, which I deeply appreciate.

02:55:50 This was really fun.

02:55:51 You’re a great educator, a great scientist.

02:55:53 It’s an honor that you would spend

02:55:56 your valuable time with me.

02:55:57 It’s an honor that you would spend your time with me as well.

02:56:00 Thanks, Jeff.

02:56:01 Thanks for listening to this conversation

02:56:03 with Jeff Schoenlein.

02:56:05 To support this podcast,

02:56:06 please check out our sponsors in the description.

02:56:09 And now let me leave you with some words

02:56:12 from the great John Carmack,

02:56:14 who surely will be a guest on this podcast soon.

02:56:18 Because of the nature of Moore’s Law,

02:56:20 anything that an extremely clever graphics programmer

02:56:22 can do at one point can be replicated

02:56:26 by a merely competent programmer

02:56:27 some number of years later.

02:56:30 Thank you for listening and hope to see you next time.