Transcript
00:00:00 The following is a conversation with Gilbert Strang.
00:00:03 He’s a professor of mathematics at MIT
00:00:05 and perhaps one of the most famous
00:00:07 and impactful teachers of math in the world.
00:00:10 His MIT OpenCourseWare lectures on linear algebra
00:00:13 have been viewed millions of times.
00:00:15 As an undergraduate student,
00:00:17 I was one of those millions of students.
00:00:19 There’s something inspiring about the way he teaches.
00:00:22 There’s at once calm, simple, and yet full of passion
00:00:26 for the elegance inherent to mathematics.
00:00:29 I remember doing the exercise in his book,
00:00:31 Introduction to Linear Algebra,
00:00:33 and slowly realizing that the world of matrices,
00:00:35 of vector spaces, of determinants and eigenvalues,
00:00:39 of geometric transformations and matrix decompositions
00:00:43 reveal a set of powerful tools
00:00:45 in the toolbox of artificial intelligence.
00:00:47 From signals to images,
00:00:49 from numerical optimization to robotics,
00:00:51 computer vision, deep learning, computer graphics,
00:00:54 and everywhere outside AI,
00:00:56 including, of course, a quantum mechanical study
00:01:00 of our universe.
00:01:01 This is the Artificial Intelligence Podcast.
00:01:04 If you enjoy it, subscribe on YouTube,
00:01:06 give it five stars on Apple Podcast,
00:01:08 support on Patreon,
00:01:10 or simply connect with me on Twitter
00:01:12 at Lex Friedman, spelled F R I D M A N.
00:01:16 This podcast is supported by ZipRecruiter.
00:01:19 Hiring great people is hard,
00:01:21 and to me is the most important element
00:01:23 of a successful mission driven team.
00:01:26 I’ve been fortunate to be a part of
00:01:28 and to lead several great engineering teams.
00:01:30 The hiring I’ve done in the past
00:01:32 was mostly through tools that we built ourselves,
00:01:35 but reinventing the wheel was painful.
00:01:38 ZipRecruiter is a tool that’s already available for you.
00:01:41 It seeks to make hiring simple, fast, and smart.
00:01:45 For example, Codable cofounder Gretchen Huebner
00:01:47 used ZipRecruiter to find a new game artist
00:01:50 to join her education tech company.
00:01:52 By using ZipRecruiter’s screening questions
00:01:54 to filter candidates,
00:01:56 Gretchen found it easier to focus on the best candidates
00:01:59 and finally hiring the perfect person for the role
00:02:02 in less than two weeks from start to finish.
00:02:05 ZipRecruiter, the smartest way to hire.
00:02:08 See why ZipRecruiter is effective
00:02:10 for businesses of all sizes by signing up,
00:02:12 as I did, for free at ziprecruiter.com slash lexpod.
00:02:17 That’s ziprecruiter.com slash lexpod.
00:02:22 This show is presented by Cash App,
00:02:24 the number one finance app in the App Store.
00:02:26 I personally use Cash App to send money to friends,
00:02:29 but you can also use it to buy, sell, and deposit Bitcoin.
00:02:33 Most Bitcoin exchanges take days
00:02:34 for a bank transfer to become investable.
00:02:37 Through Cash App, it takes seconds.
00:02:39 Cash App also has a new investing feature.
00:02:42 You can buy fractions of a stock,
00:02:44 which to me is a really interesting concept.
00:02:47 So you can buy $1 worth no matter what the stock price is.
00:02:51 Brokerage services are provided by Cash App Investing,
00:02:54 a subsidiary of Square, and member SIPC.
00:02:58 I’m excited to be working with Cash App
00:03:00 to support one of my favorite organizations
00:03:02 that many of you may know and have benefited from,
00:03:05 called First, best known for their first robotics
00:03:09 and Lego competitions.
00:03:10 They educate and inspire hundreds of thousands of students
00:03:14 in over 110 countries,
00:03:16 and have a perfect rating on Charity Navigator,
00:03:19 which means the donated money is used
00:03:21 to maximum effectiveness.
00:03:23 When you get Cash App from the App Store or Google Play,
00:03:26 and use code LexPodcast,
00:03:29 you get $10, and Cash App will also donate $10 to First,
00:03:33 which again, is an organization
00:03:35 that I’ve personally seen inspire girls and boys
00:03:38 to dream of engineering a better world.
00:03:41 And now, here’s my conversation with Gilbert Strang.
00:03:44 How does it feel to be one of the modern day rock stars
00:03:50 of mathematics?
00:03:51 I don’t feel like a rock star.
00:03:53 That’s kind of crazy for an old math person.
00:03:57 But it’s true that the videos in linear algebra
00:04:03 that I made way back in 2000, I think,
00:04:08 have been watched a lot.
00:04:09 And well, partly the importance of linear algebra,
00:04:14 which I’m sure you’ll ask me,
00:04:15 and give me a chance to say that linear algebra
00:04:19 as a subject has just surged in importance.
00:04:22 But also, it was a class that I taught a bunch of times,
00:04:26 so I kind of got it organized and enjoyed doing it.
00:04:32 The videos were just the class.
00:04:34 So they’re on OpenCourseWare and on YouTube
00:04:36 and translated, and it’s fun.
00:04:38 But there’s something about that chalkboard
00:04:41 and the simplicity of the way you explain
00:04:44 the basic concepts in the beginning.
00:04:46 To be honest, when I went to undergrad.
00:04:50 You didn’t do linear algebra, probably.
00:04:52 Of course I didn’t do linear algebra.
00:04:53 You did.
00:04:54 Yeah, yeah, yeah, of course.
00:04:55 But before going through the course at my university,
00:05:00 there was going through OpenCourseWare.
00:05:02 You were my instructor for linear algebra.
00:05:04 Right, yeah.
00:05:05 And that, I mean, we’re using your book.
00:05:07 And I mean, the fact that there is thousands,
00:05:13 hundreds of thousands, millions of people
00:05:14 that watch that video, I think that’s really powerful.
00:05:18 So how do you think the idea of putting lectures online,
00:05:23 what really MIT OpenCourseWare has innovated?
00:05:27 That was a wonderful idea.
00:05:29 I think the story that I’ve heard is the committee
00:05:34 was appointed by the president, President Vest,
00:05:37 at that time, a wonderful guy.
00:05:40 And the idea of the committee was to figure out
00:05:43 how MIT could be like other universities,
00:05:48 market the work we were doing.
00:05:52 And then they didn’t see a way.
00:05:54 And after a weekend, and they had an inspiration,
00:05:57 came back to President Vest and said,
00:06:00 what if we just gave it away?
00:06:02 And he decided that was okay, good idea.
00:06:07 So.
00:06:08 You know, that’s a crazy idea.
00:06:10 If we think of a university as a thing
00:06:12 that creates a product, isn’t knowledge,
00:06:17 the kind of educational knowledge,
00:06:19 isn’t the product and giving that away,
00:06:22 are you surprised that it went through?
00:06:26 The result that he did it,
00:06:28 well, knowing a little bit President Vest, it was like him,
00:06:32 I think, and it was really the right idea.
00:06:38 MIT is a kind of, it’s known for being high level,
00:06:43 technical things, and this is the best way we can say,
00:06:48 tell, we can show what MIT really is like,
00:06:52 because in my case, those 1806 videos
00:06:57 are just teaching the class.
00:06:59 They were there in 26, 100.
00:07:03 They’re kind of fun to look at.
00:07:04 People write to me and say, oh, you’ve got a sense of humor,
00:07:08 but I don’t know where that comes through.
00:07:10 Somehow I’m friendly with the class, I like students.
00:07:15 And then your algebra, the subject,
00:07:19 we gotta give the subject most of the credit.
00:07:21 It really has come forward in importance in these years.
00:07:29 So let’s talk about linear algebra a little bit,
00:07:32 because it is such a, it’s both a powerful
00:07:34 and a beautiful subfield of mathematics.
00:07:39 So what’s your favorite specific topic in linear algebra,
00:07:44 or even math in general to give a lecture on,
00:07:46 to convey, to tell a story, to teach students?
00:07:50 Okay, well, on the teaching side,
00:07:54 so it’s not deep mathematics at all,
00:07:56 but I’m kind of proud of the idea of the four subspaces,
00:08:02 the four fundamental subspaces,
00:08:06 which are of course known before,
00:08:09 long before my name for them, but.
00:08:13 Can you go through them?
00:08:14 Can you go through the four subspaces?
00:08:15 Sure I can, yeah.
00:08:17 So the first one to understand is,
00:08:19 so the matrix is, maybe I should say the matrix is.
00:08:22 What is a matrix?
00:08:23 What’s a matrix?
00:08:24 Well, so we have like a rectangle of numbers.
00:08:28 So it’s got n columns, got a bunch of columns,
00:08:32 and also got an m rows, let’s say,
00:08:36 and the relation between,
00:08:37 so of course the columns and the rows,
00:08:39 it’s the same numbers.
00:08:41 So there’s gotta be connections there,
00:08:44 but they’re not simple.
00:08:45 The columns might be longer than the rows,
00:08:50 and they’re all different, the numbers are mixed up.
00:08:53 First space to think about is take the columns,
00:08:57 so those are vectors, those are points in n dimensions.
00:09:01 What’s a vector?
00:09:02 So a physicist would imagine a vector
00:09:05 or might imagine a vector as a arrow in space
00:09:10 or the point it ends at in space.
00:09:14 For me, it’s a column of numbers.
00:09:18 You often think of, this is very interesting
00:09:20 in terms of linear algebra, in terms of a vector,
00:09:23 you think a little bit more abstract
00:09:26 than how it’s very commonly used, perhaps.
00:09:30 You think this arbitrary multidimensional space.
00:09:34 Right away, I’m in high dimensions.
00:09:38 Dreamland.
00:09:39 Yeah, that’s right.
00:09:40 In the lecture, I try to,
00:09:42 so if you think of two vectors in 10 dimensions,
00:09:46 I’ll do this in class, and I’ll readily admit
00:09:50 that I have no good image in my mind
00:09:54 of a vector of an arrow in 10 dimensional space,
00:09:58 but whatever.
00:10:00 You can add one bunch of 10 numbers
00:10:03 to another bunch of 10 numbers,
00:10:05 so you can add a vector to a vector,
00:10:08 and you can multiply a vector by three,
00:10:10 and that’s, if you know how to do those,
00:10:12 you’ve got linear algebra.
00:10:14 10 dimensions, there’s this beautiful thing about math,
00:10:18 if we look at string theory and all these theories,
00:10:21 which are really fundamentally derived through math,
00:10:24 but are very difficult to visualize.
00:10:26 How do you think about the things,
00:10:28 like a 10 dimensional vector,
00:10:31 that we can’t really visualize?
00:10:34 And yet, math reveals some beauty underlying our world
00:10:39 in that weird thing we can’t visualize.
00:10:43 How do you think about that difference?
00:10:46 Well, probably, I’m not a very geometric person,
00:10:48 so I’m probably thinking in three dimensions,
00:10:51 and the beauty of linear algebra is that
00:10:55 it goes on to 10 dimensions with no problem.
00:10:58 I mean, that if you’re just seeing what happens
00:11:01 if you add two vectors in 3D,
00:11:04 yeah, then you can add them in 10D.
00:11:06 You’re just adding the 10 components.
00:11:10 So, I can’t say that I have a picture,
00:11:14 but yet I try to push the class
00:11:16 to think of a flat surface in 10 dimensions.
00:11:21 So a plane in 10 dimensions,
00:11:23 and so that’s one of the spaces.
00:11:27 Take all the columns of the matrix,
00:11:29 take all their combinations,
00:11:31 so much of this column, so much of this one,
00:11:35 then if you put all those together,
00:11:36 you get some kind of a flat surface
00:11:39 that I call a vector space, space of vectors.
00:11:44 And my imagination is just seeing
00:11:47 like a piece of paper in 3D, but anyway,
00:11:52 so that’s one of the spaces, that’s space number one,
00:11:55 the column space of the matrix.
00:11:58 And then there’s the row space, which is, as I said,
00:12:01 different, but came from the same numbers.
00:12:04 So we got the column space,
00:12:07 all combinations of the columns,
00:12:10 and then we’ve got the row space,
00:12:11 all combinations of the rows.
00:12:14 So those words are easy for me to say,
00:12:17 and I can’t really draw them on a blackboard,
00:12:20 but I try with my thick chalk.
00:12:22 Everybody likes that railroad chalk, and me too.
00:12:27 I wouldn’t use anything else now.
00:12:30 And then the other two spaces are perpendicular to those.
00:12:35 So like if you have a plane in 3D,
00:12:39 just a plane is just a flat surface in 3D,
00:12:43 then perpendicular to that plane would be a line.
00:12:47 So that would be the null space.
00:12:50 So we’ve got two, we’ve got a column space, a row space,
00:12:54 and there are two perpendicular spaces.
00:12:56 So those four fit together in a beautiful picture
00:13:01 of a matrix, yeah, yeah.
00:13:03 It’s sort of a fundamental, it’s not a difficult idea.
00:13:06 It comes pretty early in 1806, and it’s basic.
00:13:12 Planes in these multidimensional spaces,
00:13:16 how difficult of an idea is that to come to, do you think?
00:13:20 If you look back in time,
00:13:23 I think mathematically it makes sense,
00:13:26 but I don’t know if it’s intuitive for us to imagine,
00:13:29 just as we were talking about.
00:13:31 It feels like calculus is easier to intuit.
00:13:34 Well, I have to admit, calculus came earlier,
00:13:38 earlier than linear algebra.
00:13:39 So Newton and Leibniz were the great men
00:13:42 to understand the key ideas of calculus.
00:13:47 But linear algebra to me is like, okay,
00:13:50 it’s the starting point,
00:13:51 because it’s all about flat things.
00:13:54 Calculus has got, all the complications of calculus
00:13:57 come from the curves, the bending, the curved surfaces.
00:14:03 Linear algebra, the surfaces are all flat.
00:14:05 Nothing bends in linear algebra.
00:14:08 So it should have come first, but it didn’t.
00:14:11 And calculus also comes first in high school classes,
00:14:17 in college class, it’ll be freshman math,
00:14:20 it’ll be calculus, and then I say, enough of it.
00:14:24 Like, okay, get to the good stuff.
00:14:27 And that’s…
00:14:28 Do you think linear algebra should come first?
00:14:30 Well, it really, I’m okay with it not coming first,
00:14:34 but it should, yeah, it should.
00:14:37 It’s simpler.
00:14:39 Because everything is flat.
00:14:40 Yeah, everything’s flat.
00:14:41 Well, of course, for that reason,
00:14:43 calculus sort of sticks to one dimension,
00:14:46 or eventually you do multivariate,
00:14:49 but that basically means two dimensions.
00:14:52 Linear algebra, you take off into 10 dimensions, no problem.
00:14:55 It just feels scary and dangerous
00:14:57 to go beyond two dimensions, that’s all.
00:15:01 If everything’s flat, you can’t go wrong.
00:15:03 So what concept or theorem in linear algebra or in math
00:15:09 you find most beautiful,
00:15:12 that gives you pause that leaves you in awe?
00:15:15 Well, I’ll stick with linear algebra here.
00:15:18 I hope the viewer knows that really,
00:15:20 mathematics is amazing, amazing subject
00:15:23 and deep, deep connections between ideas
00:15:28 that didn’t look connected, they turned out they were.
00:15:32 But if we stick with linear algebra…
00:15:35 So we have a matrix.
00:15:37 That’s like the basic thing, a rectangle of numbers.
00:15:40 And it might be a rectangle of data.
00:15:42 You’re probably gonna ask me later about data science,
00:15:46 where often data comes in a matrix.
00:15:50 You have maybe every column corresponds to a drug
00:15:57 and every row corresponds to a patient.
00:16:00 And if the patient reacted favorably to the drug,
00:16:06 then you put up some positive number in there.
00:16:09 Anyway, rectangle of numbers, a matrix is basic.
00:16:14 So the big problem is to understand all those numbers.
00:16:18 You got a big, big set of numbers.
00:16:20 And what are the patterns, what’s going on?
00:16:23 And so one of the ways to break down that matrix
00:16:29 into simple pieces is uses something called singular values.
00:16:36 And that’s come on as fundamental in the last,
00:16:41 certainly in my lifetime.
00:16:44 Eigenvalues, if you have viewers who’ve done engineering,
00:16:48 math, or basic linear algebra, eigenvalues were in there.
00:16:55 But those are restricted to square matrices.
00:16:58 And data comes in rectangular matrices.
00:17:01 So you gotta take that next step.
00:17:04 I’m always pushing math faculty, get on, do it, do it.
00:17:09 Singular values.
00:17:11 So those are a way to break, to find the important pieces
00:17:18 of the matrix, which add up to the whole matrix.
00:17:22 So you’re breaking a matrix into simple pieces.
00:17:26 And the first piece is the most important part of the data.
00:17:30 The second piece is the second most important part.
00:17:33 And then often, so a data set is a matrix.
00:17:38 And often, so a data scientist will like,
00:17:41 if a data scientist can find those first and second pieces,
00:17:46 stop there, the rest of the data is probably round off,
00:17:55 experimental error maybe.
00:17:57 So you’re looking for the important part.
00:18:00 So what do you find beautiful about singular values?
00:18:03 Well, yeah, I didn’t give the theorem.
00:18:06 So here’s the idea of singular values.
00:18:09 Every matrix, every matrix, rectangular, square, whatever,
00:18:15 can be written as a product
00:18:16 of three very simple special matrices.
00:18:20 So that’s the theorem.
00:18:21 Every matrix can be written as a rotation times a stretch,
00:18:26 which is just a diagonal matrix,
00:18:30 otherwise all zeros except on the one diagonal.
00:18:34 And then the third factor is another rotation.
00:18:37 So rotation, stretch, rotation
00:18:41 is the breakup of any matrix.
00:18:45 The structure of that, the ability that you can do that,
00:18:48 what do you find appealing?
00:18:49 What do you find beautiful about it?
00:18:51 Well, geometrically, as I freely admit,
00:18:54 the action of a matrix is not so easy to visualize,
00:18:59 but everybody can visualize a rotation.
00:19:02 Take two dimensional space and just turn it
00:19:07 around the center.
00:19:09 Take three dimensional space.
00:19:10 So a pilot has to know about,
00:19:13 well, what are the three, the yaw is one of them.
00:19:16 I’ve forgotten all the three turns that a pilot makes.
00:19:22 Up to 10 dimensions, you’ve got 10 ways to turn,
00:19:25 but you can visualize a rotation.
00:19:28 Take the space and turn it.
00:19:30 And you can visualize a stretch.
00:19:32 So to break a matrix with all those numbers in it
00:19:38 into something you can visualize,
00:19:41 rotate, stretch, rotate is pretty neat.
00:19:44 It’s pretty neat.
00:19:45 That’s pretty powerful.
00:19:47 On YouTube, just consuming a bunch of videos
00:19:51 and just watching what people connect with
00:19:53 and what they really enjoy and are inspired by,
00:19:57 math seems to come up again and again.
00:19:59 I’m trying to understand why that is.
00:20:03 Perhaps you can help give me clues.
00:20:06 So it’s not just the kinds of lectures that you give,
00:20:10 but it’s also just other folks like with Numberphile,
00:20:14 there’s a channel where they just chat about things
00:20:16 that are extremely complicated, actually.
00:20:19 People nevertheless connect with them.
00:20:22 What do you think that is?
00:20:24 It’s wonderful, isn’t it?
00:20:25 I mean, I wasn’t really aware of it.
00:20:28 We’re conditioned to think math is hard,
00:20:32 math is abstract, math is just for a few people,
00:20:35 but it isn’t that way.
00:20:36 A lot of people quite like math and they liked it.
00:20:41 I get messages from people saying,
00:20:44 now I’m retired, I’m gonna learn some more math.
00:20:46 I get a lot of those.
00:20:47 It’s really encouraging.
00:20:49 And I think what people like is that there’s some order,
00:20:53 a lot of order and things are not obvious, but they’re true.
00:21:00 So it’s really cheering to think that so many people
00:21:06 really wanna learn more about math.
00:21:08 Yeah.
00:21:08 And in terms of truth, again,
00:21:11 sorry to slide into philosophy at times,
00:21:15 but math does reveal pretty strongly what things are true.
00:21:20 I mean, that’s the whole point of proving things.
00:21:23 It is, yeah.
00:21:24 And yet, sort of our real world is messy and complicated.
00:21:29 It is.
00:21:30 What do you think about the nature of truth
00:21:33 that math reveals?
00:21:34 Oh, wow.
00:21:35 Because it is a source of comfort like you’ve mentioned.
00:21:37 Yeah, that’s right.
00:21:39 Well, I have to say, I’m not much of a philosopher.
00:21:43 I just like numbers.
00:21:44 As a kid, this was before you had to go in,
00:21:52 when you had a filly in your teeth,
00:21:54 you had to kind of just take it.
00:21:56 So what I did was think about math,
00:21:59 like take powers of two, two, four, eight, 16,
00:22:03 up until the time the tooth stopped hurting
00:22:05 and the dentist said you’re through.
00:22:08 Or counting.
00:22:09 Yeah.
00:22:10 So that was a source of just, source of peace almost.
00:22:14 Yeah.
00:22:15 What is it about math do you think that brings that?
00:22:19 Yeah.
00:22:20 What is that?
00:22:21 Well, you know where you are.
00:22:22 Yeah, it’s symmetry, it’s certainty.
00:22:25 The fact that, you know, if you multiply two by itself
00:22:29 10 times, you get 1,024 period.
00:22:33 Everybody’s gonna get that.
00:22:34 Do you see math as a powerful tool or as an art form?
00:22:39 So it’s both.
00:22:40 That’s really one of the neat things.
00:22:42 You can be an artist and like math,
00:22:46 you can be an engineer and use math.
00:22:50 Which are you?
00:22:51 Which am I?
00:22:53 What did you connect with most?
00:22:54 Yeah, I’m somewhere between.
00:22:57 I’m certainly not a artist type, philosopher type person.
00:23:01 Might sound that way this morning, but I’m not.
00:23:04 Yeah, I really enjoy teaching engineers
00:23:09 because they go for an answer.
00:23:13 And yeah, so probably within the MIT math department,
00:23:20 most people enjoy teaching people,
00:23:23 teaching students who get the abstract idea.
00:23:26 I’m okay with, I’m good with engineers
00:23:32 who are looking for a way to find answers.
00:23:34 Yeah.
00:23:35 Actually, that’s an interesting question.
00:23:37 Do you think for teaching and in general,
00:23:41 thinking about new concepts,
00:23:42 do you think it’s better to plug in the numbers
00:23:46 or to think more abstractly?
00:23:49 So looking at theorems and proving the theorems
00:23:53 or actually building up a basic intuition of the theorem
00:23:58 or the method, the approach,
00:23:59 and then just plugging in numbers and seeing it work.
00:24:02 Yeah, well, certainly many of us like to see examples.
00:24:09 First, we understand,
00:24:11 it might be a pretty abstract sounding example,
00:24:13 like a three dimensional rotation.
00:24:16 How are you gonna understand a rotation in 3D?
00:24:22 Or in 10D?
00:24:28 And then some of us like to keep going with it
00:24:30 to the point where you got numbers,
00:24:32 where you got 10 angles, 10 axes, 10 angles.
00:24:38 But the best, the great mathematicians probably,
00:24:43 I don’t know if they do that,
00:24:44 because for them, an example would be a highly abstract thing
00:24:53 to the rest of it.
00:24:54 Right, but nevertheless, working in the space of examples.
00:24:57 Yeah, examples.
00:24:58 It seems to.
00:24:59 Examples of structure.
00:25:01 Our brains seem to connect with that.
00:25:03 Yeah, yeah.
00:25:04 So I’m not sure if you’re familiar with him,
00:25:07 but Andrew Yang is a presidential candidate
00:25:11 currently running with math in all capital letters
00:25:17 and his hats as a slogan.
00:25:18 I see.
00:25:19 Stands for Make America Think Hard.
00:25:21 Okay, I’ll vote for him.
00:25:25 So, and his name rhymes with yours, Yang, Strang.
00:25:28 But he also loves math and he comes from that world
00:25:31 of, but he also, looking at it,
00:25:35 makes me realize that math, science, and engineering
00:25:38 are not really part of our politics, political discourse,
00:25:43 about political government in general.
00:25:46 Why do you think that is?
00:25:48 Well.
00:25:49 What are your thoughts on that in general?
00:25:51 Well, certainly somewhere in the system,
00:25:52 we need people who are comfortable with numbers,
00:25:56 comfortable with quantities.
00:25:58 You know, if you say this leads to that,
00:26:02 they see it and it’s undeniable.
00:26:05 But isn’t that strange to you that we have almost no,
00:26:10 I mean, I’m pretty sure we have no elected officials
00:26:14 in Congress or obviously the president
00:26:18 that either has an engineering degree or a math degree.
00:26:22 Yeah, well, that’s too bad.
00:26:25 A few could, a few who could make the connection.
00:26:30 Yeah, it would have to be people who understand
00:26:35 engineering or science and at the same time
00:26:38 can make speeches and lead, yeah.
00:26:44 Yeah, inspire people.
00:26:45 Yeah, inspire, yeah.
00:26:46 You were, speaking of inspiration,
00:26:49 the president of the Society
00:26:50 for Industrial and Applied Mathematics.
00:26:52 Oh, yes.
00:26:53 It’s a major organization in math, applied math.
00:26:57 What do you see as a role of that society,
00:27:01 you know, in our public discourse?
00:27:02 Right.
00:27:03 In public.
00:27:04 Yeah, so, well, it was fun to be president at the time.
00:27:08 A couple years, a few years.
00:27:09 Two years, around 2000.
00:27:13 I just hope that’s president of a pretty small society.
00:27:16 But nevertheless, it was a time when math
00:27:19 was getting some more attention in Washington.
00:27:24 But yeah, I got to give a little 10 minutes
00:27:29 to a committee of the House of Representatives
00:27:33 talking about who I met.
00:27:35 And then, actually, it was fun
00:27:36 because one of the members of the House
00:27:42 had been a student, had been in my class.
00:27:44 What do you think of that?
00:27:46 Yeah, as you say, pretty rare, most members of the House
00:27:49 have had a different training, different background.
00:27:52 But there was one from New Hampshire
00:27:56 who was my friend, really, by being in the class.
00:28:02 Yeah, so those years were good.
00:28:05 Then, of course, other things take over in importance
00:28:10 in Washington, and math just, at this point,
00:28:16 is not so visible.
00:28:18 But for a little moment, it was.
00:28:20 There’s some excitement, some concern
00:28:23 about artificial intelligence in Washington now.
00:28:26 Yes, sure. About the future.
00:28:27 Yeah. And I think at the core
00:28:28 of that is math.
00:28:30 Well, it is, yeah.
00:28:32 Maybe it’s hidden.
00:28:32 Maybe it’s wearing a different hat.
00:28:34 Well, artificial intelligence, and particularly,
00:28:39 can I use the words deep learning?
00:28:41 Deep learning is a particular approach
00:28:44 to understanding data.
00:28:47 Again, you’ve got a big, whole lot of data
00:28:51 where data is just swamping the computers of the world.
00:28:56 And to understand it, out of all those numbers,
00:29:00 to find what’s important in climate, in everything.
00:29:05 And artificial intelligence is two words
00:29:08 for one approach to data.
00:29:11 Deep learning is a specific approach there,
00:29:15 which uses a lot of linear algebra.
00:29:17 So I got into it.
00:29:19 I thought, okay, I’ve gotta learn about this.
00:29:21 So maybe from your perspective,
00:29:24 let me ask the most basic question.
00:29:27 How do you think of a neural network?
00:29:30 What is a neural network?
00:29:31 Yeah, okay.
00:29:32 So can I start with the idea about deep learning?
00:29:37 What does that mean?
00:29:38 What is deep learning?
00:29:39 What is deep learning, yeah.
00:29:41 So we’re trying to learn, from all this data,
00:29:46 we’re trying to learn what’s important.
00:29:47 What’s it telling us?
00:29:50 So you’ve got data, you’ve got some inputs
00:29:55 for which you know the right outputs.
00:29:57 The question is, can you see the pattern there?
00:30:02 Can you figure out a way for a new input,
00:30:04 which we haven’t seen, to understand
00:30:09 what the output will be from that new input?
00:30:12 So we’ve got a million inputs with their outputs.
00:30:15 So we’re trying to create some pattern,
00:30:19 some rule that’ll take those inputs,
00:30:22 those million training inputs, which we know about,
00:30:25 to the correct million outputs.
00:30:28 And this idea of a neural net
00:30:32 is part of the structure of our new way to create a rule.
00:30:40 We’re looking for a rule that will take
00:30:43 these training inputs to the known outputs.
00:30:48 And then we’re gonna use that rule on new inputs
00:30:51 that we don’t know the output and see what comes.
00:30:56 Linear algebra is a big part of finding that rule.
00:30:59 That’s right, linear algebra is a big part.
00:31:01 Not all the part.
00:31:03 People were leaning on matrices, that’s good, still do.
00:31:08 Linear is something special.
00:31:10 It’s all about straight lines and flat planes.
00:31:13 And data isn’t quite like that.
00:31:18 It’s more complicated.
00:31:21 So you gotta introduce some complication.
00:31:23 So you have to have some function
00:31:25 that’s not a straight line.
00:31:27 And it turned out, nonlinear, nonlinear, not linear.
00:31:31 And it turned out that it was enough to use the function
00:31:35 that’s one straight line and then a different one.
00:31:38 Halfway, so piecewise linear.
00:31:40 One piece has one slope,
00:31:44 one piece, the other piece has the second slope.
00:31:47 And so that, getting that nonlinear,
00:31:52 simple nonlinearity in blew the problem open.
00:31:56 That little piece makes it sufficiently complicated
00:31:58 to make things interesting.
00:32:00 Because you’re gonna use that piece
00:32:02 over and over a million times.
00:32:03 So it has a fold in the graph, the graph, two pieces.
00:32:10 But when you fold something a million times,
00:32:13 you’ve got a pretty complicated function
00:32:17 that’s pretty realistic.
00:32:19 So that’s the thing about neural networks
00:32:21 is they have a lot of these.
00:32:23 A lot of these, that’s right.
00:32:25 So why do you think neural networks,
00:32:29 by using sort of formulating an objective function,
00:32:34 very not a plain function of the folds,
00:32:39 lots of folds of the inputs, the outputs,
00:32:42 why do you think they work to be able to find a rule
00:32:47 that we don’t know is optimal,
00:32:48 but it just seems to be pretty good in a lot of cases?
00:32:53 What’s your intuition?
00:32:54 Is it surprising to you as it is to many people?
00:32:58 Do you have an intuition of why this works at all?
00:33:01 Well, I’m beginning to have a better intuition.
00:33:04 This idea of things that are piecewise linear,
00:33:08 flat pieces but with folds between them.
00:33:12 Like think of a roof of a complicated,
00:33:14 infinitely complicated house or something.
00:33:17 That curve, it almost curved, but every piece is flat.
00:33:24 That’s been used by engineers,
00:33:26 that idea has been used by engineers,
00:33:29 is used by engineers, big time.
00:33:32 Something called the finite element method.
00:33:34 If you want to design a bridge,
00:33:36 design a building, design an airplane,
00:33:40 you’re using this idea of piecewise flat
00:33:47 as a good, simple, computable approximation.
00:33:52 But you have a sense that there’s a lot of expressive power
00:33:57 in this kind of piecewise linear.
00:33:58 Yeah, you used the right word.
00:34:01 If you measure the expressivity,
00:34:04 how complicated a thing can this piecewise flat guys express?
00:34:12 The answer is very complicated, yeah.
00:34:15 What do you think are the limits of such piecewise linear
00:34:20 or just of neural networks?
00:34:22 The expressivity of neural networks.
00:34:24 Well, you would have said a while ago
00:34:26 that they’re just computational limits.
00:34:28 It’s a problem beyond a certain size.
00:34:33 A supercomputer isn’t gonna do it.
00:34:36 But those keep getting more powerful.
00:34:39 So that limit has been moved
00:34:44 to allow more and more complicated surfaces.
00:34:47 So in terms of just mapping from inputs to outputs,
00:34:52 looking at data, what do you think of,
00:34:58 in the context of neural networks in general,
00:35:00 data is just tensor, vectors, matrices, tensors.
00:35:04 Right.
00:35:05 How do you think about learning from data?
00:35:09 How much of our world can be expressed in this way?
00:35:12 How useful is this process?
00:35:16 I guess that’s another way to ask you,
00:35:17 what are the limits of this approach?
00:35:19 Well, that’s a good question, yeah.
00:35:21 So I guess the whole idea of deep learning
00:35:24 is that there’s something there to learn.
00:35:26 If the data is totally random,
00:35:28 just produced by random number generators,
00:35:31 then we’re not gonna find a useful rule
00:35:36 because there isn’t one.
00:35:38 So the extreme of having a rule is like knowing Newton’s law.
00:35:43 If you hit a ball, it moves.
00:35:46 So that’s where you had laws of physics.
00:35:48 Newton and Einstein and other great, great people
00:35:54 have found those laws and laws of the distribution
00:36:02 of oil in an underground thing.
00:36:05 I mean, so engineers, petroleum engineers understand
00:36:10 how oil will sit in an underground basin.
00:36:18 So there were rules.
00:36:20 Now, the new idea of artificial intelligence is
00:36:25 learn the rules instead of figuring out the rules
00:36:29 with help from Newton or Einstein.
00:36:32 The computer is looking for the rules.
00:36:35 So that’s another step.
00:36:36 But if there are no rules at all
00:36:39 that the computer could find,
00:36:41 if it’s totally random data, well, you’ve got nothing.
00:36:45 You’ve got no science to discover.
00:36:48 It’s an automated search for the underlying rules.
00:36:51 Yeah, search for the rules.
00:36:53 Yeah, exactly.
00:36:54 And there will be a lot of random parts.
00:36:57 A lot of, I mean, I’m not knocking random
00:36:59 because that’s there.
00:37:05 There’s a lot of randomness built in,
00:37:07 but there’s gotta be some basic.
00:37:09 It’s almost always signal, right?
00:37:10 In most things.
00:37:11 There’s gotta be some signal, yeah.
00:37:12 If it’s all noise, then you’re not gonna get anywhere.
00:37:17 Well, this world around us does seem to be,
00:37:19 does seem to always have a signal of some kind.
00:37:22 Yeah, yeah, that’s right.
00:37:23 To be discovered.
00:37:24 Right, that’s it.
00:37:25 So what excites you more?
00:37:30 We just talked about a little bit of application.
00:37:32 What excites you more, theory
00:37:35 or the application of mathematics?
00:37:38 Well, for myself, I’m probably a theory person.
00:37:43 I’m not, I’m speaking here pretty freely about applications,
00:37:49 but I’m not the person who really,
00:37:53 I’m not a physicist or a chemist or a neuroscientist.
00:37:58 So for myself, I like the structure
00:38:03 and the flat subspaces
00:38:06 and the relation of matrices, columns to rows.
00:38:12 That’s my part in the spectrum.
00:38:17 So really, science is a big spectrum of people
00:38:22 from asking practical questions
00:38:25 and answering them using some math,
00:38:28 then some math guys like myself who are in the middle of it
00:38:33 and then the geniuses of math and physics and chemistry
00:38:40 who are finding fundamental rules
00:38:43 and then doing the really understanding nature.
00:38:50 That’s incredible.
00:38:51 At its lowest, simplest level,
00:38:54 maybe just a quick in broad strokes from your perspective,
00:38:58 where does linear algebra sit as a subfield of mathematics?
00:39:04 What are the various subfields that you think about
00:39:10 in relation to linear algebra?
00:39:12 So the big fields of math are algebra as a whole
00:39:18 and problems like calculus and differential equations.
00:39:21 So that’s a second, quite different field.
00:39:24 Then maybe geometry deserves to be thought of
00:39:28 as a different field to understand the geometry
00:39:31 of high dimensional surfaces.
00:39:35 So I think, am I allowed to say this here?
00:39:39 I think this is where personal view comes in.
00:39:46 I think math, we’re thinking about undergraduate math,
00:39:51 what millions of students study.
00:39:54 I think we overdo the calculus at the cost of the algebra,
00:40:00 at the cost of linear.
00:40:02 So you have this talk titled Calculus Versus Linear Algebra.
00:40:05 That’s right, that’s right.
00:40:07 And you say that linear algebra wins.
00:40:09 So can you dig into that a little bit?
00:40:13 Why does linear algebra win?
00:40:17 Right, well, okay, the viewer is gonna think
00:40:21 this guy is biased.
00:40:22 Not true, I’m just telling the truth as it is.
00:40:27 Yeah, so I feel linear algebra is just a nice part of math
00:40:31 that people can get the idea of.
00:40:34 They can understand something that’s a little bit abstract
00:40:37 because once you get to 10 or 100 dimensions
00:40:42 and very, very, very useful,
00:40:44 that’s what’s happened in my lifetime
00:40:47 is the importance of data,
00:40:52 which does come in matrix form.
00:40:54 So it’s really set up for algebra.
00:40:56 It’s not set up for differential equation.
00:40:59 And let me fairly add probability,
00:41:03 the ideas of probability and statistics
00:41:06 have become very, very important, have also jumped forward.
00:41:11 So, and that’s different from linear algebra,
00:41:14 quite different.
00:41:15 So now we really have three major areas to me,
00:41:20 calculus, linear algebra, matrices,
00:41:26 and probability statistics.
00:41:28 And they all deserve an important place.
00:41:33 And calculus has traditionally had a lion’s share
00:41:40 of the time.
00:41:40 A disproportionate share.
00:41:41 It is, thank you, disproportionate, that’s a good word.
00:41:45 Of the love and attention from the excited young minds.
00:41:50 Yeah.
00:41:52 I know it’s hard to pick favorites,
00:41:55 but what is your favorite matrix?
00:41:57 What’s my favorite matrix?
00:41:59 Okay, so my favorite matrix is square, I admit it.
00:42:03 It’s a square bunch of numbers
00:42:05 and it has twos running down the main diagonal.
00:42:10 And on the next diagonal,
00:42:13 so think of top left to bottom right,
00:42:15 twos down the middle of the matrix
00:42:18 and minus ones just above those twos
00:42:22 and minus ones just below those twos
00:42:25 and otherwise all zeros.
00:42:26 So mostly zeros, just three nonzero diagonals coming down.
00:42:32 What is interesting about it?
00:42:34 Well, all the different ways it comes up.
00:42:37 You see it in engineering,
00:42:39 you see it as analogous in calculus to second derivative.
00:42:44 So calculus learns about taking the derivative,
00:42:47 the figuring out how much, how fast something’s changing.
00:42:51 But second derivative, now that’s also important.
00:42:55 That’s how fast the change is changing,
00:42:58 how fast the graph is bending, how fast it’s curving.
00:43:06 And Einstein showed that that’s fundamental
00:43:10 to understand space.
00:43:11 So second derivatives should have a bigger place in calculus.
00:43:17 Second, my matrices,
00:43:21 which are like the linear algebra version
00:43:24 of second derivatives are neat in linear algebra.
00:43:30 Yeah, just everything comes out right with those guys.
00:43:34 Beautiful.
00:43:35 What did you learn about the process of learning
00:43:38 by having taught so many students math over the years?
00:43:42 Ooh, that is hard.
00:43:45 I’ll have to admit here that I’m not really a good teacher
00:43:51 because I don’t get into the exam part.
00:43:55 The exam is the part of my life that I don’t like
00:43:59 and grading them and giving the students A or B or whatever.
00:44:04 I do it because I’m supposed to do it,
00:44:08 but I tell the class at the beginning,
00:44:11 I don’t know if they believe me.
00:44:13 Probably they don’t.
00:44:14 I tell the class, I’m here to teach you.
00:44:18 I’m here to teach you math and not to grade you.
00:44:22 But they’re thinking, okay, this guy is gonna,
00:44:26 when is he gonna give me an A minus?
00:44:28 Is he gonna give me a B plus?
00:44:30 What?
00:44:31 What have you learned about the process of learning?
00:44:34 Of learning.
00:44:34 Yeah, well, maybe to give you a legitimate answer
00:44:40 about learning, I should have paid more attention
00:44:43 to the assessment, the evaluation part at the end.
00:44:47 But I like the teaching part at the start.
00:44:49 That’s the sexy part.
00:44:52 To tell somebody for the first time about a matrix, wow.
00:44:56 Is there, are there moments,
00:44:58 so you are teaching a concept,
00:45:01 are there moments of learning that you just see
00:45:05 in the student’s eyes?
00:45:06 You don’t need to look at the grades.
00:45:08 But you see in their eyes that you hook them,
00:45:11 that you connect with them in a way where,
00:45:16 you know what, they fall in love
00:45:18 with this beautiful world of math.
00:45:21 They see that it’s got some beauty there.
00:45:24 Or conversely, that they give up at that point
00:45:28 is the opposite.
00:45:29 The dark could say that math, I’m just not good at math.
00:45:32 I don’t wanna walk away.
00:45:33 Yeah, yeah, yeah.
00:45:34 Maybe because of the approach in the past,
00:45:37 they were discouraged, but don’t be discouraged.
00:45:40 It’s too good to miss.
00:45:44 Yeah, well, if I’m teaching a big class,
00:45:48 do I know when, I think maybe I do.
00:45:51 Sort of, I mentioned at the very start,
00:45:55 the four fundamental subspaces
00:45:59 and the structure of the fundamental theorem
00:46:03 of linear algebra.
00:46:04 The fundamental theorem of linear algebra.
00:46:06 That is the relation of those four subspaces,
00:46:11 those four spaces.
00:46:13 Yeah, so I think that, I feel that the class gets it.
00:46:17 At length.
00:46:18 Yeah.
00:46:19 What advice do you have to a student
00:46:22 just starting their journey in mathematics today?
00:46:25 How do they get started?
00:46:27 Oh, yeah, that’s hard.
00:46:30 Well, I hope you have a teacher, professor,
00:46:34 who is still enjoying what he’s doing,
00:46:39 what he’s teaching.
00:46:41 They’re still looking for new ways to teach
00:46:44 and to understand math.
00:46:47 Cause that’s the pleasure,
00:46:51 the moment when you see, oh yeah, that works.
00:46:54 So it’s less about the material you study,
00:46:58 it’s more about the source of the teacher
00:47:02 being full of passion.
00:47:03 Yeah, more about the fun.
00:47:05 Yeah, the moment of getting it.
00:47:10 But in terms of topics, linear algebra?
00:47:14 Well, that’s my topic,
00:47:16 but oh, there’s beautiful things in geometry to understand.
00:47:21 What’s wonderful is that in the end,
00:47:24 there’s a pattern, there are rules
00:47:28 that are followed in biology as there are in every field.
00:47:37 You describe the life of a mathematician
00:47:41 as 100% wonderful.
00:47:44 No.
00:47:45 Except for the grade stuff.
00:47:47 Yeah.
00:47:47 And the grades.
00:47:48 Except for grades.
00:47:49 Yeah, when you look back at your life,
00:47:52 what memories bring you the most joy and pride?
00:47:55 Well, that’s a good question.
00:47:59 I certainly feel good when I,
00:48:01 maybe I’m giving a class in 1806,
00:48:06 that’s MIT’s linear algebra course that I started.
00:48:09 So sort of, there’s a good feeling that,
00:48:11 okay, I started this course,
00:48:13 a lot of students take it, quite a few like it.
00:48:17 Yeah, so I’m sort of happy
00:48:21 when I feel I’m helping make a connection
00:48:25 between ideas and students,
00:48:27 between theory and the reader.
00:48:32 Yeah, it’s, I get a lot of very nice messages
00:48:38 from people who’ve watched the videos and it’s inspiring.
00:48:43 I just, I’ll maybe take this chance to say thank you.
00:48:48 Well, there’s millions of students
00:48:50 who you’ve taught and I am grateful to be one of them.
00:48:54 So Gilbert, thank you so much, it’s been an honor.
00:48:56 Thank you for talking today.
00:48:58 It was a pleasure, thanks.
00:49:00 Thank you for listening to this conversation
00:49:02 with Gilbert Strang.
00:49:04 And thank you to our presenting sponsor, Cash App.
00:49:07 Download it, use code LexPodcast,
00:49:09 you’ll get $10 and $10 will go to FIRST,
00:49:12 a STEM education nonprofit
00:49:14 that inspires hundreds of thousands of young minds
00:49:17 to learn and to dream of engineering our future.
00:49:20 If you enjoy this podcast, subscribe on YouTube.
00:49:23 We have five stars in Apple Podcast,
00:49:25 support on Patreon or connect with me on Twitter.
00:49:29 Finally, some closing words of advice
00:49:31 from the great Richard Feynman.
00:49:33 Study hard what interests you the most
00:49:36 in the most undisciplined, irreverent
00:49:38 and original manner possible.
00:49:41 Thank you for listening and hope to see you next time.