Transcript
00:00:00 The following is a conversation with Michael Kearns.
00:00:03 He’s a professor at the University of Pennsylvania
00:00:06 and a coauthor of the new book, Ethical Algorithm,
00:00:09 that is the focus of much of this conversation.
00:00:12 It includes algorithmic fairness, bias, privacy,
00:00:16 and ethics in general.
00:00:18 But that is just one of many fields
00:00:20 that Michael is a world class researcher in,
00:00:22 some of which we touch on quickly,
00:00:24 including learning theory
00:00:26 or the theoretical foundation of machine learning,
00:00:29 game theory, quantitative finance,
00:00:31 computational social science, and much more.
00:00:34 But on a personal note,
00:00:35 when I was an undergrad, early on,
00:00:38 I worked with Michael
00:00:39 on an algorithmic trading project
00:00:41 and competition that he led.
00:00:43 That’s when I first fell in love
00:00:44 with algorithmic game theory.
00:00:46 While most of my research life
00:00:48 has been in machine learning
00:00:49 and human robot interaction,
00:00:51 the systematic way that game theory
00:00:53 reveals the beautiful structure
00:00:55 in our competitive and cooperating world of humans
00:00:58 has been a continued inspiration to me.
00:01:01 So for that and other things,
00:01:03 I’m deeply thankful to Michael
00:01:05 and really enjoyed having this conversation
00:01:07 again in person after so many years.
00:01:11 This is the Artificial Intelligence Podcast.
00:01:13 If you enjoy it, subscribe on YouTube,
00:01:16 give it five stars on Apple Podcast,
00:01:18 support on Patreon,
00:01:19 or simply connect with me on Twitter
00:01:21 at Lex Friedman, spelled F R I D M A N.
00:01:25 This episode is supported
00:01:27 by an amazing podcast called Pessimists Archive.
00:01:31 Jason, the host of the show,
00:01:32 reached out to me looking to support this podcast,
00:01:35 and so I listened to it, to check it out.
00:01:38 And by listened, I mean I went through it,
00:01:40 Netflix binge style, at least five episodes in a row.
00:01:44 It’s not one of my favorite podcasts,
00:01:46 and I think it should be one of the top podcasts
00:01:48 in the world, frankly.
00:01:50 It’s a history show
00:01:51 about why people resist new things.
00:01:54 Each episode looks at a moment in history
00:01:56 when something new was introduced,
00:01:59 something that today we think of as commonplace,
00:02:01 like recorded music, umbrellas, bicycles, cars,
00:02:04 chess, coffee, the elevator,
00:02:06 and the show explores why it freaked everyone out.
00:02:10 The latest episode on mirrors and vanity
00:02:12 still stays with me as I think about vanity
00:02:15 in the modern day of the Twitter world.
00:02:18 That’s the fascinating thing about the show,
00:02:20 is that stuff that happened long ago,
00:02:22 especially in terms of our fear of new things,
00:02:25 repeats itself in the modern day,
00:02:26 and so has many lessons for us to think about
00:02:29 in terms of human psychology
00:02:31 and the role of technology in our society.
00:02:34 Anyway, you should subscribe
00:02:35 and listen to Pessimist Archive.
00:02:37 I highly recommend it.
00:02:39 And now, here’s my conversation with Michael Kearns.
00:02:44 You mentioned reading Fear and Loathing in Las Vegas
00:02:47 in high school, and having a more,
00:02:50 or a bit more of a literary mind.
00:02:52 So, what books, non technical, non computer science,
00:02:56 would you say had the biggest impact on your life,
00:02:59 either intellectually or emotionally?
00:03:02 You’ve dug deep into my history, I see.
00:03:04 Went deep.
00:03:05 Yeah, I think, well, my favorite novel is
00:03:08 Infinite Jest by David Foster Wallace,
00:03:10 which actually, coincidentally,
00:03:13 much of it takes place in the halls of buildings
00:03:15 right around us here at MIT.
00:03:18 So that certainly had a big influence on me.
00:03:20 And as you noticed, like, when I was in high school,
00:03:22 I actually even started college as an English major.
00:03:25 So, I was very influenced by sort of that genre of journalism
00:03:29 at the time, and thought I wanted to be a writer,
00:03:30 and then realized that an English major teaches you to read,
00:03:33 but it doesn’t teach you how to write,
00:03:35 and then I became interested in math
00:03:36 and computer science instead.
00:03:38 Well, in your new book, Ethical Algorithm,
00:03:41 you kind of sneak up from an algorithmic perspective
00:03:45 on these deep, profound philosophical questions
00:03:48 of fairness, of privacy.
00:03:55 In thinking about these topics,
00:03:56 how often do you return to that literary mind that you had?
00:04:01 Yeah, I’d like to claim there was a deeper connection,
00:04:05 but, you know, I think both Aaron and I
00:04:07 kind of came at these topics first and foremost
00:04:10 from a technical angle.
00:04:11 I mean, you know, I kind of consider myself primarily
00:04:14 and originally a machine learning researcher,
00:04:17 and I think as we just watched, like the rest of the society,
00:04:20 the field technically advance, and then quickly on the heels
00:04:23 of that kind of the buzzkill of all of the antisocial behavior
00:04:27 by algorithms, just kind of realized
00:04:29 there was an opportunity for us to do something about it
00:04:31 from a research perspective.
00:04:34 You know, more to the point in your question,
00:04:36 I mean, I do have an uncle who is literally a moral philosopher,
00:04:41 and so in the early days of my life,
00:04:43 he was a philosopher, and so in the early days
00:04:46 of our technical work on fairness topics,
00:04:48 I would occasionally, you know, run ideas behind him.
00:04:51 So, I mean, I remember an early email I sent to him
00:04:53 in which I said, like, oh, you know,
00:04:55 here’s a specific definition of algorithmic fairness
00:04:57 that we think is some sort of variant of Rawlsian fairness.
00:05:02 What do you think?
00:05:03 And I thought I was asking a yes or no question,
00:05:06 and I got back your kind of classical philosopher’s
00:05:09 response saying, well, it depends.
00:05:10 Hey, then you might conclude this, and that’s when I realized
00:05:14 that there was a real kind of rift between the ways
00:05:19 philosophers and others had thought about things
00:05:21 like fairness, you know, from sort of a humanitarian perspective
00:05:25 and the way that you needed to think about it
00:05:27 as a computer scientist if you were going to kind of
00:05:30 implement actual algorithmic solutions.
00:05:34 But I would say the algorithmic solutions take care
00:05:39 of some of the low hanging fruit.
00:05:41 Sort of the problem is a lot of algorithms,
00:05:44 when they don’t consider fairness,
00:05:47 they are just terribly unfair.
00:05:50 And when they don’t consider privacy,
00:05:51 they’re terribly, they violate privacy.
00:05:55 Sort of the algorithmic approach fixes big problems.
00:05:59 But there’s still, when you start pushing into the gray area,
00:06:04 that’s when you start getting into this philosophy
00:06:06 of what it means to be fair, starting from Plato,
00:06:09 what is justice kind of questions?
00:06:12 Yeah, I think that’s right.
00:06:13 And I mean, I would even not go as far as you want to say
00:06:16 that sort of the algorithmic work in these areas
00:06:19 is solving like the biggest problems.
00:06:22 And, you know, we discuss in the book,
00:06:24 the fact that really we are, there’s a sense in which
00:06:27 we’re kind of looking where the light is in that,
00:06:30 you know, for example, if police are racist
00:06:34 in who they decide to stop and frisk,
00:06:37 and that goes into the data,
00:06:39 there’s sort of no undoing that downstream
00:06:41 by kind of clever algorithmic methods.
00:06:45 And I think, especially in fairness,
00:06:47 I mean, I think less so in privacy,
00:06:49 where we feel like the community kind of really has settled
00:06:52 on the right definition, which is differential privacy.
00:06:56 If you just look at the algorithmic fairness literature
00:06:58 already, you can see it’s going to be much more of a mess.
00:07:01 And, you know, you’ve got these theorems saying,
00:07:03 here are three entirely reasonable,
00:07:06 desirable notions of fairness.
00:07:09 And, you know, here’s a proof that you cannot simultaneously
00:07:12 have all three of them.
00:07:14 So I think we know that algorithmic fairness
00:07:17 compared to algorithmic privacy
00:07:19 is going to be kind of a harder problem.
00:07:21 And it will have to revisit, I think,
00:07:23 things that have been thought about by,
00:07:26 you know, many generations of scholars before us.
00:07:29 So it’s very early days for fairness, I think.
00:07:32 TK So before we get into the details
00:07:34 of differential privacy, and on the fairness side,
00:07:37 let me linger on the philosophy a bit.
00:07:39 Do you think most people are fundamentally good?
00:07:43 Or do most of us have both the capacity
00:07:46 for good and evil within us?
00:07:48 SB I mean, I’m an optimist.
00:07:50 I tend to think that most people are good
00:07:52 and want to do right.
00:07:55 And that deviations from that are, you know,
00:07:58 kind of usually due to circumstance,
00:08:00 not due to people being bad at heart.
00:08:02 TK With people with power,
00:08:05 are people at the heads of governments,
00:08:08 people at the heads of companies,
00:08:10 people at the heads of, maybe, so financial power markets,
00:08:15 do you think the distribution there is also,
00:08:19 most people are good and have good intent?
00:08:21 SB Yeah, I do.
00:08:22 I mean, my statement wasn’t qualified to people
00:08:26 not in positions of power.
00:08:28 I mean, I think what happens in a lot of the, you know,
00:08:30 the cliche about absolute power corrupts absolutely.
00:08:34 I mean, you know, I think even short of that,
00:08:37 you know, having spent a lot of time on Wall Street,
00:08:40 and also in arenas very, very different from Wall Street,
00:08:44 like academia, you know, one of the things
00:08:47 I think I’ve benefited from by moving between
00:08:50 two very different worlds is you become aware
00:08:53 that, you know, these worlds kind of develop
00:08:57 their own social norms, and they develop
00:08:59 their own rationales for, you know,
00:09:02 behavior, for instance, that might look
00:09:03 unusual to outsiders.
00:09:05 But when you’re in that world,
00:09:07 it doesn’t feel unusual at all.
00:09:09 And I think this is true of a lot of,
00:09:11 you know, professional cultures, for instance.
00:09:15 And, you know, so then your maybe slippery slope
00:09:18 is too strong of a word.
00:09:19 But, you know, you’re in some world
00:09:21 where you’re mainly around other people
00:09:23 with the same kind of viewpoints and training
00:09:25 and worldview as you.
00:09:27 And I think that’s more of a source of,
00:09:30 of, you know, kind of abuses of power
00:09:34 than sort of, you know, there being good people
00:09:36 and evil people, and that somehow the evil people
00:09:40 are the ones that somehow rise to power.
00:09:43 Oh, that’s really interesting.
00:09:44 So it’s the, within the social norms
00:09:46 constructed by that particular group of people,
00:09:50 you’re all trying to do good.
00:09:52 But because as a group, you might be,
00:09:54 you might drift into something
00:09:56 that for the broader population,
00:09:58 it does not align with the values of society.
00:10:00 That kind of, that’s the word.
00:10:01 Yeah, I mean, or not that you drift,
00:10:03 but even the things that don’t make sense
00:10:07 to the outside world don’t seem unusual to you.
00:10:11 So it’s not sort of like a good or a bad thing,
00:10:13 but, you know, like, so for instance,
00:10:14 you know, on, in the world of finance, right?
00:10:18 There’s a lot of complicated types of activity
00:10:21 that if you are not immersed in that world,
00:10:22 you cannot see why the purpose of that,
00:10:25 you know, that activity exists at all.
00:10:27 It just seems like, you know, completely useless
00:10:30 and people just like, you know, pushing money around.
00:10:33 And when you’re in that world, right,
00:10:34 you’re, and you learn more,
00:10:36 your view does become more nuanced, right?
00:10:39 You realize, okay, there is actually a function
00:10:41 to this activity.
00:10:43 And in some cases, you would conclude that actually,
00:10:46 if magically we could eradicate this activity tomorrow,
00:10:50 it would come back because it actually is like
00:10:52 serving some useful purpose.
00:10:54 It’s just a useful purpose that’s very difficult
00:10:56 for outsiders to see.
00:10:59 And so I think, you know, lots of professional work
00:11:02 environments or cultures, as I might put it,
00:11:06 kind of have these social norms that, you know,
00:11:08 don’t make sense to the outside world.
00:11:10 Academia is the same, right?
00:11:11 I mean, lots of people look at academia and say,
00:11:13 you know, what the hell are all of you people doing?
00:11:16 Why are you paid so much in some cases
00:11:18 at taxpayer expenses to do, you know,
00:11:21 to publish papers that nobody reads?
00:11:24 You know, but when you’re in that world,
00:11:25 you come to see the value for it.
00:11:27 And, but even though you might not be able to explain it
00:11:30 to, you know, the person in the street.
00:11:33 Right.
00:11:33 And in the case of the financial sector,
00:11:36 tools like credit might not make sense to people.
00:11:39 Like, it’s a good example of something that does seem
00:11:41 to pop up and be useful or just the power of markets
00:11:45 and just in general capitalism.
00:11:47 Yeah.
00:11:47 In finance, I think the primary example
00:11:49 I would give is leverage, right?
00:11:51 So being allowed to borrow, to sort of use ten times
00:11:56 as much money as you’ve actually borrowed, right?
00:11:58 So that’s an example of something that before I had
00:12:00 any experience in financial markets,
00:12:02 I might have looked at and said,
00:12:03 well, what is the purpose of that?
00:12:05 That just seems very dangerous and it is dangerous
00:12:08 and it has proven dangerous.
00:12:10 But, you know, if the fact of the matter is that,
00:12:13 you know, sort of on some particular time scale,
00:12:16 you are holding positions that are,
00:12:19 you know, very unlikely to, you know,
00:12:23 lose, you know, your value at risk or variance
00:12:26 is like one or five percent, then it kind of makes sense
00:12:30 that you would be allowed to use a little bit more
00:12:32 than you have because you have, you know,
00:12:35 some confidence that you’re not going to lose
00:12:37 it all in a single day.
00:12:39 Now, of course, when that happens,
00:12:42 we’ve seen what happens, you know, not too long ago.
00:12:45 But, you know, but the idea that it serves
00:12:48 no useful economic purpose under any circumstances
00:12:52 is definitely not true.
00:12:54 We’ll return to the other side of the coast,
00:12:57 Silicon Valley, and the problems there as we talk about privacy,
00:13:02 as we talk about fairness.
00:13:05 At the high level, and I’ll ask some sort of basic questions
00:13:09 with the hope to get at the fundamental nature of reality.
00:13:12 But from a very high level, what is an ethical algorithm?
00:13:18 So I can say that an algorithm has a running time
00:13:20 of using big O notation n log n.
00:13:24 I can say that a machine learning algorithm
00:13:27 classified cat versus dog with 97 percent accuracy.
00:13:31 Do you think there will one day be a way to measure
00:13:36 sort of in the same compelling way as the big O notation
00:13:39 of this algorithm is 97 percent ethical?
00:13:44 First of all, let me riff for a second on your specific n log n example.
00:13:48 So because early in the book when we’re just kind of trying to describe
00:13:51 algorithms period, we say like, okay, you know,
00:13:54 what’s an example of an algorithm or an algorithmic problem?
00:13:58 First of all, like it’s sorting, right?
00:14:00 You have a bunch of index cards with numbers on them
00:14:02 and you want to sort them.
00:14:03 And we describe, you know, an algorithm that sweeps all the way through,
00:14:07 finds the smallest number, puts it at the front,
00:14:09 then sweeps through again, finds the second smallest number.
00:14:12 So we make the point that this is an algorithm
00:14:14 and it’s also a bad algorithm in the sense that, you know,
00:14:17 it’s quadratic rather than n log n,
00:14:20 which we know is kind of optimal for sorting.
00:14:23 And we make the point that sort of like, you know,
00:14:26 so even within the confines of a very precisely specified problem,
00:14:31 there, you know, there might be many, many different algorithms
00:14:35 for the same problem with different properties.
00:14:37 Like some might be faster in terms of running time,
00:14:40 some might use less memory, some might have, you know,
00:14:43 better distributed implementations.
00:14:46 And so the point is that already we’re used to, you know,
00:14:50 in computer science thinking about trade offs
00:14:53 between different types of quantities and resources
00:14:56 and there being, you know, better and worse algorithms.
00:15:00 And our book is about that part of algorithmic ethics
00:15:08 that we know how to kind of put on that same kind of quantitative footing right now.
00:15:13 So, you know, just to say something that our book is not about,
00:15:17 our book is not about kind of broad, fuzzy notions of fairness.
00:15:22 It’s about very specific notions of fairness.
00:15:25 There’s more than one of them.
00:15:28 There are tensions between them, right?
00:15:30 But if you pick one of them, you can do something akin to saying
00:15:35 that this algorithm is 97% ethical.
00:15:39 You can say, for instance, the, you know, for this lending model,
00:15:44 the false rejection rate on black people and white people is within 3%, right?
00:15:51 So we might call that a 97% ethical algorithm and a 100% ethical algorithm
00:15:57 would mean that that difference is 0%.
00:15:59 In that case, fairness is specified when two groups, however,
00:16:04 they’re defined are given to you.
00:16:06 That’s right.
00:16:07 So the, and then you can sort of mathematically start describing the algorithm.
00:16:11 But nevertheless, the part where the two groups are given to you,
00:16:20 I mean, unlike running time, you know, we don’t in computer science
00:16:24 talk about how fast an algorithm feels like when it runs.
00:16:29 True.
00:16:29 We measure it and ethical starts getting into feelings.
00:16:33 So, for example, an algorithm runs, you know, if it runs in the background,
00:16:38 it doesn’t disturb the performance of my system.
00:16:40 It’ll feel nice.
00:16:41 I’ll be okay with it.
00:16:42 But if it overloads the system, it’ll feel unpleasant.
00:16:45 So in that same way, ethics, there’s a feeling of how socially acceptable it is.
00:16:50 How does it represent the moral standards of our society today?
00:16:55 So in that sense, and sorry to linger on that first of high,
00:16:59 low philosophical questions.
00:17:00 Do you have a sense we’ll be able to measure how ethical an algorithm is?
00:17:05 First of all, I didn’t, certainly didn’t mean to give the impression that you can kind of
00:17:09 measure, you know, memory speed trade offs, you know, and that there’s a complete mapping from
00:17:16 that onto kind of fairness, for instance, or ethics and accuracy, for example.
00:17:22 In the type of fairness definitions that are largely the objects of study today and starting
00:17:28 to be deployed, you as the user of the definitions, you need to make some hard decisions before you
00:17:35 even get to the point of designing fair algorithms.
00:17:40 One of them, for instance, is deciding who it is that you’re worried about protecting,
00:17:45 who you’re worried about being harmed by, for instance, some notion of discrimination or
00:17:50 unfairness.
00:17:52 And then you need to also decide what constitutes harm.
00:17:55 So, for instance, in a lending application, maybe you decide that, you know, falsely rejecting
00:18:02 a creditworthy individual, you know, sort of a false negative, is the real harm and that false
00:18:08 positives, i.e. people that are not creditworthy or are not gonna repay your loan, that get a loan,
00:18:14 you might think of them as lucky.
00:18:17 And so that’s not a harm, although it’s not clear that if you don’t have the means to repay a loan,
00:18:22 that being given a loan is not also a harm.
00:18:26 So, you know, the literature is sort of so far quite limited in that you sort of need to say,
00:18:33 who do you want to protect and what would constitute harm to that group?
00:18:37 And when you ask questions like, will algorithms feel ethical?
00:18:42 One way in which they won’t, under the definitions that I’m describing, is if, you know, if you are
00:18:47 an individual who is falsely denied a loan, incorrectly denied a loan, all of these definitions
00:18:54 basically say like, well, you know, your compensation is the knowledge that we are also
00:19:00 falsely denying loans to other people, you know, in other groups at the same rate that we’re doing
00:19:05 it to you.
00:19:05 And, you know, and so there is actually this interesting even technical tension in the field
00:19:12 right now between these sort of group notions of fairness and notions of fairness that might
00:19:18 actually feel like real fairness to individuals, right?
00:19:22 They might really feel like their particular interests are being protected or thought about
00:19:27 by the algorithm rather than just, you know, the groups that they happen to be members of.
00:19:33 Is there parallels to the big O notation of worst case analysis?
00:19:37 So, is it important to looking at the worst violation of fairness for an individual?
00:19:45 Is it important to minimize that one individual?
00:19:48 So like worst case analysis, is that something you think about or?
00:19:52 I mean, I think we’re not even at the point where we can sensibly think about that.
00:19:56 So first of all, you know, we’re talking here both about fairness applied at the group level,
00:20:03 which is a relatively weak thing, but it’s better than nothing.
00:20:08 And also the more ambitious thing of trying to give some individual promises, but even
00:20:14 that doesn’t incorporate, I think something that you’re hinting at here is what I might
00:20:18 call subjective fairness, right?
00:20:20 So a lot of the definitions, I mean, all of the definitions in the algorithmic fairness
00:20:25 literature are what I would kind of call received wisdom definitions.
00:20:28 It’s sort of, you know, somebody like me sits around and things like, okay, you know, I
00:20:33 think here’s a technical definition of fairness that I think people should want or that they
00:20:37 should, you know, think of as some notion of fairness, maybe not the only one, maybe
00:20:41 not the best one, maybe not the last one.
00:20:44 But we really actually don’t know from a subjective standpoint, like what people really
00:20:52 think is fair.
00:20:53 You know, we just started doing a little bit of work in our group at actually doing kind
00:21:01 of human subject experiments in which we, you know, ask people about, you know, we ask
00:21:09 them questions about fairness, we survey them, we, you know, we show them pairs of individuals
00:21:15 in, let’s say, a criminal recidivism prediction setting, and we ask them, do you think these
00:21:20 two individuals should be treated the same as a matter of fairness?
00:21:24 And to my knowledge, there’s not a large literature in which ordinary people are asked
00:21:31 about, you know, they have sort of notions of their subjective fairness elicited from
00:21:37 them.
00:21:38 It’s mainly, you know, kind of scholars who think about fairness kind of making up their
00:21:43 own definitions.
00:21:44 And I think this needs to change actually for many social norms, not just for fairness,
00:21:50 right?
00:21:50 So there’s a lot of discussion these days in the AI community about interpretable AI
00:21:56 or understandable AI.
00:21:58 And as far as I can tell, everybody agrees that deep learning or at least the outputs
00:22:04 of deep learning are not very understandable, and people might agree that sparse linear
00:22:11 models with integer coefficients are more understandable.
00:22:15 But nobody’s really asked people.
00:22:17 You know, there’s very little literature on, you know, sort of showing people models
00:22:21 and asking them, do they understand what the model is doing?
00:22:25 And I think that in all these topics, as these fields mature, we need to start doing more
00:22:32 behavioral work.
00:22:34 Yeah, which is one of my deep passions is psychology.
00:22:38 And I always thought computer scientists will be the best future psychologists in a sense
00:22:44 that data is, especially in this modern world, the data is a really powerful way to understand
00:22:51 and study human behavior.
00:22:53 And you’ve explored that with your game theory side of work as well.
00:22:56 Yeah, I’d like to think that what you say is true about computer scientists and psychology
00:23:02 from my own limited wandering into human subject experiments.
00:23:07 We have a great deal to learn, not just computer science, but AI and machine learning more
00:23:11 specifically, I kind of think of as imperialist research communities in that, you know, kind
00:23:17 of like physicists in an earlier generation, computer scientists kind of don’t think of
00:23:22 any scientific topic that’s off limits to them.
00:23:25 They will like freely wander into areas that others have been thinking about for decades
00:23:30 or longer.
00:23:31 And, you know, we usually tend to embarrass ourselves in those efforts for some amount
00:23:37 of time.
00:23:38 Like, you know, I think reinforcement learning is a good example, right?
00:23:41 So a lot of the early work in reinforcement learning, I have complete sympathy for the
00:23:48 control theorists that looked at this and said like, okay, you are reinventing stuff
00:23:53 that we’ve known since like the forties, right?
00:23:55 But, you know, in my view, eventually this sort of, you know, computer scientists have
00:24:01 made significant contributions to that field, even though we kind of embarrassed ourselves
00:24:06 for the first decade.
00:24:07 So I think if computer scientists are gonna start engaging in kind of psychology, human
00:24:12 subjects type of research, we should expect to be embarrassing ourselves for a good 10
00:24:18 years or so, and then hope that it turns out as well as, you know, some other areas that
00:24:23 we’ve waded into.
00:24:25 So you kind of mentioned this, just to linger on the idea of an ethical algorithm, of idea
00:24:30 of groups, sort of group thinking and individual thinking.
00:24:33 And we’re struggling that.
00:24:35 One of the amazing things about algorithms and your book and just this field of study
00:24:39 is it gets us to ask, like forcing machines, converting these ideas into algorithms is
00:24:46 forcing us to ask questions of ourselves as a human civilization.
00:24:50 So there’s a lot of people now in public discourse doing sort of group thinking, thinking like
00:24:58 there’s particular sets of groups that we don’t wanna discriminate against and so on.
00:25:02 And then there is individuals, sort of in the individual life stories, the struggles
00:25:08 they went through and so on.
00:25:10 Now, like in philosophy, it’s easier to do group thinking because you don’t, it’s very
00:25:16 hard to think about individuals.
00:25:17 There’s so much variability, but with data, you can start to actually say, you know what
00:25:23 group thinking is too crude.
00:25:26 You’re actually doing more discrimination by thinking in terms of groups and individuals.
00:25:30 Can you linger on that kind of idea of group versus individual and ethics?
00:25:36 And is it good to continue thinking in terms of groups in algorithms?
00:25:41 So let me start by answering a very good high level question with a slightly narrow technical
00:25:49 response, which is these group definitions of fairness, like here’s a few groups, like
00:25:54 different racial groups, maybe gender groups, maybe age, what have you.
00:25:59 And let’s make sure that for none of these groups, do we have a false negative rate,
00:26:06 which is much higher than any other one of these groups.
00:26:09 Okay, so these are kind of classic group aggregate notions of fairness.
00:26:13 And you know, but at the end of the day, an individual you can think of as a combination
00:26:18 of all of their attributes, right?
00:26:19 They’re a member of a racial group, they have a gender, they have an age, and many other
00:26:26 demographic properties that are not biological, but that are still very strong determinants
00:26:33 of outcome and personality and the like.
00:26:36 So one, I think, useful spectrum is to sort of think about that array between the group
00:26:43 and the specific individual, and to realize that in some ways, asking for fairness at
00:26:49 the individual level is to sort of ask for group fairness simultaneously for all possible
00:26:56 combinations of groups.
00:26:57 So in particular, you know, if I build a predictive model that meets some definition of fairness,
00:27:06 definition of fairness by race, by gender, by age, by what have you, marginally, to get
00:27:14 it slightly technical, sort of independently, I shouldn’t expect that model to not discriminate
00:27:20 against disabled Hispanic women over age 55, making less than $50,000 a year annually,
00:27:27 even though I might have protected each one of those attributes marginally.
00:27:32 So the optimization, actually, that’s a fascinating way to put it.
00:27:35 So you’re just optimizing, the one way to achieve the optimizing fairness for individuals
00:27:42 is just to add more and more definitions of groups that each individual belongs to.
00:27:46 That’s right.
00:27:47 So, you know, at the end of the day, we could think of all of ourselves as groups of size
00:27:50 one because eventually there’s some attribute that separates you from me and everybody else
00:27:55 in the world, okay?
00:27:57 And so it is possible to put, you know, these incredibly coarse ways of thinking about fairness
00:28:03 and these very, very individualistic specific ways on a common scale.
00:28:09 And you know, one of the things we’ve worked on from a research perspective is, you know,
00:28:14 so we sort of know how to, you know, in relative terms, we know how to provide fairness guarantees
00:28:20 at the core system of the scale.
00:28:22 We don’t know how to provide kind of sensible, tractable, realistic fairness guarantees at
00:28:28 the individual level, but maybe we could start creeping towards that by dealing with more
00:28:33 refined subgroups.
00:28:35 I mean, we gave a name to this phenomenon where, you know, you protect, you enforce
00:28:41 some definition of fairness for a bunch of marginal attributes or features, but then
00:28:46 you find yourself discriminating against a combination of them.
00:28:49 We call that fairness gerrymandering because like political gerrymandering, you know, you’re
00:28:55 giving some guarantee at the aggregate level, but when you kind of look in a more granular
00:29:01 way at what’s going on, you realize that you’re achieving that aggregate guarantee by sort
00:29:06 of favoring some groups and discriminating against other ones.
00:29:10 And so there are, you know, it’s early days, but there are algorithmic approaches that
00:29:15 let you start creeping towards that, you know, individual end of the spectrum.
00:29:22 Does there need to be human input in the form of weighing the value of the importance of
00:29:30 each kind of group?
00:29:33 So for example, is it like, so gender, say crudely speaking, male and female, and then
00:29:42 different races, are we as humans supposed to put value on saying gender is 0.6 and race
00:29:51 is 0.4 in terms of in the big optimization of achieving fairness?
00:29:59 Is that kind of what humans are supposed to do here?
00:30:01 I mean, of course, you know, I don’t need to tell you that, of course, technically one
00:30:05 could incorporate such weights if you wanted to into a definition of fairness.
00:30:10 You know, fairness is an interesting topic in that having worked in the book being about
00:30:19 both fairness, privacy, and many other social norms, fairness, of course, is a much, much
00:30:24 more loaded topic.
00:30:27 So privacy, I mean, people want privacy, people don’t like violations of privacy, violations
00:30:32 of privacy cause damage, angst, and bad publicity for the companies that are victims of them.
00:30:40 But sort of everybody agrees more data privacy would be better than less data privacy.
00:30:48 And you don’t have these, somehow the discussions of fairness don’t become politicized along
00:30:53 other dimensions like race and about gender and, you know, whether we, and, you know,
00:31:01 you quickly find yourselves kind of revisiting topics that have been kind of unresolved forever,
00:31:10 like affirmative action, right?
00:31:12 Sort of, you know, like, why are you protecting, and some people will say, why are you protecting
00:31:16 this particular racial group?
00:31:20 And others will say, well, we need to do that as a matter of retribution.
00:31:26 Other people will say, it’s a matter of economic opportunity.
00:31:30 And I don’t know which of, you know, whether any of these are the right answers, but you
00:31:34 sort of, fairness is sort of special in that as soon as you start talking about it, you
00:31:39 inevitably have to participate in debates about fair to whom, at what expense to who
00:31:46 else.
00:31:47 I mean, even in criminal justice, right, you know, where people talk about fairness in
00:31:56 criminal sentencing or, you know, predicting failures to appear or making parole decisions
00:32:02 or the like, they will, you know, they’ll point out that, well, these definitions of
00:32:08 fairness are all about fairness for the criminals.
00:32:13 And what about fairness for the victims, right?
00:32:16 So when I basically say something like, well, the false incarceration rate for black people
00:32:22 and white people needs to be roughly the same, you know, there’s no mention of potential
00:32:28 victims of criminals in such a fairness definition.
00:32:33 And that’s the realm of public discourse.
00:32:34 I should actually recommend, I just listened to people listening, Intelligence Squares
00:32:41 debates, US edition just had a debate.
00:32:45 They have this structure where you have old Oxford style or whatever they’re called, debates,
00:32:50 you know, it’s two versus two and they talked about affirmative action and it was incredibly
00:32:55 interesting that there’s really good points on every side of this issue, which is fascinating
00:33:03 to listen to.
00:33:04 Yeah, yeah, I agree.
00:33:05 And so it’s interesting to be a researcher trying to do, for the most part, technical
00:33:12 algorithmic work, but Aaron and I both quickly learned you cannot do that and then go out
00:33:17 and talk about it and expect people to take it seriously if you’re unwilling to engage
00:33:22 in these broader debates that are entirely extra algorithmic, right?
00:33:28 They’re not about, you know, algorithms and making algorithms better.
00:33:31 They’re sort of, you know, as you said, sort of like, what should society be protecting
00:33:35 in the first place?
00:33:36 When you discuss the fairness, an algorithm that achieves fairness, whether in the constraints
00:33:42 and the objective function, there’s an immediate kind of analysis you can perform, which is
00:33:48 saying, if you care about fairness in gender, this is the amount that you have to pay for
00:33:56 it in terms of the performance of the system.
00:33:59 Like do you, is there a role for statements like that in a table, in a paper, or do you
00:34:03 want to really not touch that?
00:34:06 No, no, we want to touch that and we do touch it.
00:34:09 So I mean, just again, to make sure I’m not promising your viewers more than we know how
00:34:16 to provide, but if you pick a definition of fairness, like I’m worried about gender discrimination
00:34:21 and you pick a notion of harm, like false rejection for a loan, for example, and you
00:34:27 give me a model, I can definitely, first of all, go audit that model.
00:34:30 It’s easy for me to go, you know, from data to kind of say like, okay, your false rejection
00:34:36 rate on women is this much higher than it is on men, okay?
00:34:41 But once you also put the fairness into your objective function, I mean, I think the table
00:34:47 that you’re talking about is what we would call the Pareto curve, right?
00:34:51 You can literally trace out, and we give examples of such plots on real data sets in the book,
00:34:58 you have two axes.
00:34:59 On the X axis is your error, on the Y axis is unfairness by whatever, you know, if it’s
00:35:06 like the disparity between false rejection rates between two groups.
00:35:12 And you know, your algorithm now has a knob that basically says, how strongly do I want
00:35:17 to enforce fairness?
00:35:19 And the less unfair, you know, if the two axes are error and unfairness, we’d like to
00:35:24 be at zero, zero.
00:35:26 We’d like zero error and zero unfairness simultaneously.
00:35:31 Anybody who works in machine learning knows that you’re generally not going to get to
00:35:34 zero error period without any fairness constraint whatsoever.
00:35:38 So that’s not going to happen.
00:35:41 But in general, you know, you’ll get this, you’ll get some kind of convex curve that
00:35:46 specifies the numerical trade off you face.
00:35:49 You know, if I want to go from 17% error down to 16% error, what will be the increase in
00:35:57 unfairness that I experienced as a result of that?
00:36:02 And so this curve kind of specifies the, you know, kind of undominated models.
00:36:09 Models that are off that curve are, you know, can be strictly improved in one or both dimensions.
00:36:14 You can, you know, either make the error better or the unfairness better or both.
00:36:18 And I think our view is that not only are these objects, these Pareto curves, you know,
00:36:26 with efficient frontiers as you might call them, not only are they valuable scientific
00:36:34 objects, I actually think that they in the near term might need to be the interface between
00:36:41 researchers working in the field and stakeholders in given problems.
00:36:46 So you know, you could really imagine telling a criminal jurisdiction, look, if you’re concerned
00:36:55 about racial fairness, but you’re also concerned about accuracy.
00:36:58 You want to, you know, you want to release on parole people that are not going to recommit
00:37:05 a violent crime and you don’t want to release the ones who are.
00:37:08 So you know, that’s accuracy.
00:37:10 But if you also care about those, you know, the mistakes you make not being disproportionately
00:37:15 on one racial group or another, you can show this curve.
00:37:19 I’m hoping that in the near future, it’ll be possible to explain these curves to non
00:37:23 technical people that are the ones that have to make the decision, where do we want to
00:37:29 be on this curve?
00:37:30 Like, what are the relative merits or value of having lower error versus lower unfairness?
00:37:38 You know, that’s not something computer scientists should be deciding for society, right?
00:37:43 That, you know, the people in the field, so to speak, the policymakers, the regulators,
00:37:49 that’s who should be making these decisions.
00:37:51 But I think and hope that they can be made to understand that these trade offs generally
00:37:56 exist and that you need to pick a point and like, and ignoring the trade off, you know,
00:38:03 you’re implicitly picking a point anyway, right?
00:38:06 You just don’t know it and you’re not admitting it.
00:38:09 Just to linger on the point of trade offs, I think that’s a really important thing to
00:38:12 sort of think about.
00:38:15 So you think when we start to optimize for fairness, there’s almost always in most system
00:38:22 going to be trade offs.
00:38:25 Can you like, what’s the trade off between just to clarify, there have been some sort
00:38:30 of technical terms thrown around, but sort of a perfectly fair world.
00:38:39 Why is that?
00:38:40 Why will somebody be upset about that?
00:38:43 The specific trade off I talked about just in order to make things very concrete was
00:38:47 between numerical error and some numerical measure of unfairness.
00:38:53 What is numerical error in the case of…
00:38:56 Just like say predictive error, like, you know, the probability or frequency with which
00:39:01 you release somebody on parole who then goes on to recommit a violent crime or keep incarcerated
00:39:08 somebody who would not have recommitted a violent crime.
00:39:10 So in the case of awarding somebody parole or giving somebody parole or letting them
00:39:17 out on parole, you don’t want them to recommit a crime.
00:39:21 So it’s your system failed in prediction if they happen to do a crime.
00:39:26 Okay, so that’s one axis.
00:39:30 And what’s the fairness axis?
00:39:31 So then the fairness axis might be the difference between racial groups in the kind of false
00:39:39 positive predictions, namely people that I kept incarcerated predicting that they would
00:39:47 recommit a violent crime when in fact they wouldn’t have.
00:39:51 Right.
00:39:52 And the unfairness of that, just to linger it and allow me to in eloquently to try to
00:40:00 sort of describe why that’s unfair, why unfairness is there.
00:40:06 The unfairness you want to get rid of is that in the judge’s mind, the bias of having being
00:40:13 brought up to society, the slight racial bias, the racism that exists in the society, you
00:40:18 want to remove that from the system.
00:40:21 Another way that’s been debated is sort of equality of opportunity versus equality of
00:40:28 outcome.
00:40:30 And there’s a weird dance there that’s really difficult to get right.
00:40:35 And we don’t, affirmative action is exploring that space.
00:40:40 Right.
00:40:41 And then this also quickly bleeds into questions like, well, maybe if one group really does
00:40:48 recommit crimes at a higher rate, the reason for that is that at some earlier point in
00:40:55 the pipeline or earlier in their lives, they didn’t receive the same resources that the
00:41:00 other group did.
00:41:02 And so there’s always in kind of fairness discussions, the possibility that the real
00:41:08 injustice came earlier, right?
00:41:11 Earlier in this individual’s life, earlier in this group’s history, et cetera, et cetera.
00:41:16 And so a lot of the fairness discussion is almost, the goal is for it to be a corrective
00:41:20 mechanism to account for the injustice earlier in life.
00:41:25 By some definitions of fairness or some theories of fairness, yeah.
00:41:29 Others would say like, look, it’s not to correct that injustice, it’s just to kind of level
00:41:35 the playing field right now and not falsely incarcerate more people of one group than
00:41:40 another group.
00:41:41 But I mean, I think just it might be helpful just to demystify a little bit about the many
00:41:46 ways in which bias or unfairness can come into algorithms, especially in the machine
00:41:54 learning era, right?
00:41:55 I think many of your viewers have probably heard these examples before, but let’s say
00:42:00 I’m building a face recognition system, right?
00:42:04 And so I’m kind of gathering lots of images of faces and trying to train the system to
00:42:12 recognize new faces of those individuals from training on a training set of those faces
00:42:17 of individuals.
00:42:19 And it shouldn’t surprise anybody or certainly not anybody in the field of machine learning
00:42:24 if my training data set was primarily white males and I’m training the model to maximize
00:42:34 the overall accuracy on my training data set, that the model can reduce its error most by
00:42:44 getting things right on the white males that constitute the majority of the data set, even
00:42:48 if that means that on other groups, they will be less accurate, okay?
00:42:53 Now, there’s a bunch of ways you could think about addressing this.
00:42:57 One is to deliberately put into the objective of the algorithm not to optimize the error
00:43:05 at the expense of this discrimination, and then you’re kind of back in the land of these
00:43:09 kind of two dimensional numerical trade offs.
00:43:13 A valid counter argument is to say like, well, no, you don’t have to, there’s no, you know,
00:43:18 the notion of the tension between error and accuracy here is a false one.
00:43:22 You could instead just go out and get much more data on these other groups that are in
00:43:27 the minority and, you know, equalize your data set, or you could train a separate model
00:43:34 on those subgroups and, you know, have multiple models.
00:43:38 The point I think we would, you know, we tried to make in the book is that those things have
00:43:43 cost too, right?
00:43:45 Going out and gathering more data on groups that are relatively rare compared to your
00:43:51 plurality or more majority group that, you know, it may not cost you in the accuracy
00:43:55 of the model, but it’s going to cost, you know, it’s going to cost the company developing
00:43:59 this model more money to develop that, and it also costs more money to build separate
00:44:04 predictive models and to implement and deploy them.
00:44:07 So even if you can find a way to avoid the tension between error and accuracy in training
00:44:14 a model, you might push the cost somewhere else, like money, like development time, research
00:44:20 time and the like.
00:44:22 There are fundamentally difficult philosophical questions, in fairness, and we live in a very
00:44:30 divisive political climate, outraged culture.
00:44:34 There is alt right folks on 4chan, trolls.
00:44:38 There is social justice warriors on Twitter.
00:44:43 There’s very divisive, outraged folks on all sides of every kind of system.
00:44:49 How do you, how do we as engineers build ethical algorithms in such divisive culture?
00:44:57 Do you think they could be disjoint?
00:44:59 The human has to inject your values, and then you can optimize over those values.
00:45:04 But in our times, when you start actually applying these systems, things get a little
00:45:09 bit challenging for the public discourse.
00:45:13 How do you think we can proceed?
00:45:14 Yeah, I mean, for the most part in the book, a point that we try to take some pains to
00:45:21 make is that we don’t view ourselves or people like us as being in the position of deciding
00:45:29 for society what the right social norms are, what the right definitions of fairness are.
00:45:34 Our main point is to just show that if society or the relevant stakeholders in a particular
00:45:41 domain can come to agreement on those sorts of things, there’s a way of encoding that
00:45:47 into algorithms in many cases, not in all cases.
00:45:50 One other misconception that hopefully we definitely dispel is sometimes people read
00:45:55 the title of the book and I think not unnaturally fear that what we’re suggesting is that the
00:46:00 algorithms themselves should decide what those social norms are and develop their own notions
00:46:05 of fairness and privacy or ethics, and we’re definitely not suggesting that.
00:46:10 The title of the book is Ethical Algorithm, by the way, and I didn’t think of that interpretation
00:46:13 of the title.
00:46:14 That’s interesting.
00:46:15 Yeah, yeah.
00:46:16 I mean, especially these days where people are concerned about the robots becoming our
00:46:21 overlords, the idea that the robots would also sort of develop their own social norms
00:46:25 is just one step away from that.
00:46:29 But I do think, obviously, despite disclaimer that people like us shouldn’t be making those
00:46:35 decisions for society, we are kind of living in a world where in many ways computer scientists
00:46:40 have made some decisions that have fundamentally changed the nature of our society and democracy
00:46:46 and sort of civil discourse and deliberation in ways that I think most people generally
00:46:53 feel are bad these days, right?
00:46:55 But they had to make, so if we look at people at the heads of companies and so on, they
00:47:01 had to make those decisions, right?
00:47:02 There has to be decisions, so there’s two options, either you kind of put your head
00:47:08 in the sand and don’t think about these things and just let the algorithm do what it does,
00:47:14 or you make decisions about what you value, you know, of injecting moral values into the
00:47:19 algorithm.
00:47:20 Look, I never mean to be an apologist for the tech industry, but I think it’s a little
00:47:26 bit too far to sort of say that explicit decisions were made about these things.
00:47:31 So let’s, for instance, take social media platforms, right?
00:47:34 So like many inventions in technology and computer science, a lot of these platforms
00:47:40 that we now use regularly kind of started as curiosities, right?
00:47:45 I remember when things like Facebook came out and its predecessors like Friendster,
00:47:49 which nobody even remembers now, people really wonder, like, why would anybody want to spend
00:47:55 time doing that?
00:47:56 I mean, even the web when it first came out, when it wasn’t populated with much content
00:48:01 and it was largely kind of hobbyists building their own kind of ramshackle websites, a lot
00:48:07 of people looked at this and said, well, what is the purpose of this thing?
00:48:09 Why is this interesting?
00:48:11 Who would want to do this?
00:48:12 And so even things like Facebook and Twitter, yes, technical decisions were made by engineers,
00:48:18 by scientists, by executives in the design of those platforms, but, you know, I don’t
00:48:23 think 10 years ago anyone anticipated that those platforms, for instance, might kind
00:48:32 of acquire undue, you know, influence on political discourse or on the outcomes of elections.
00:48:42 And I think the scrutiny that these companies are getting now is entirely appropriate, but
00:48:47 I think it’s a little too harsh to kind of look at history and sort of say like, oh,
00:48:53 you should have been able to anticipate that this would happen with your platform.
00:48:56 And in this sort of gaming chapter of the book, one of the points we’re making is that,
00:49:00 you know, these platforms, right, they don’t operate in isolation.
00:49:05 So unlike the other topics we’re discussing, like fairness and privacy, like those are
00:49:09 really cases where algorithms can operate on your data and make decisions about you
00:49:13 and you’re not even aware of it, okay?
00:49:16 Things like Facebook and Twitter, these are, you know, these are systems, right?
00:49:20 These are social systems and their evolution, even their technical evolution because machine
00:49:25 learning is involved, is driven in no small part by the behavior of the users themselves
00:49:31 and how the users decide to adopt them and how to use them.
00:49:35 And so, you know, I’m kind of like who really knew that, you know, until we saw it happen,
00:49:44 who knew that these things might be able to influence the outcome of elections?
00:49:48 Who knew that, you know, they might polarize political discourse because of the ability
00:49:55 to, you know, decide who you interact with on the platform and also with the platform
00:50:00 naturally using machine learning to optimize for your own interest that they would further
00:50:05 isolate us from each other and, you know, like feed us all basically just the stuff
00:50:10 that we already agreed with.
00:50:12 So I think, you know, we’ve come to that outcome, I think, largely, but I think it’s
00:50:18 something that we all learned together, including the companies as these things happen.
00:50:24 You asked like, well, are there algorithmic remedies to these kinds of things?
00:50:29 And again, these are big problems that are not going to be solved with, you know, somebody
00:50:35 going in and changing a few lines of code somewhere in a social media platform.
00:50:40 But I do think in many ways, there are definitely ways of making things better.
00:50:44 I mean, like an obvious recommendation that we make at some point in the book is like,
00:50:49 look, you know, to the extent that we think that machine learning applied for personalization
00:50:55 purposes in things like newsfeed, you know, or other platforms has led to polarization
00:51:03 and intolerance of opposing viewpoints.
00:51:07 As you know, right, these algorithms have models, right, and they kind of place people
00:51:11 in some kind of metric space, and they place content in that space, and they sort of know
00:51:17 the extent to which I have an affinity for a particular type of content.
00:51:22 And by the same token, they also probably have that same model probably gives you a
00:51:26 good idea of the stuff I’m likely to violently disagree with or be offended by, okay?
00:51:32 So you know, in this case, there really is some knob you could tune that says like, instead
00:51:37 of showing people only what they like and what they want, let’s show them some stuff
00:51:43 that we think that they don’t like, or that’s a little bit further away.
00:51:46 And you could even imagine users being able to control this, you know, just like everybody
00:51:51 gets a slider, and that slider says like, you know, how much stuff do you want to see
00:51:58 that’s kind of, you know, you might disagree with, or is at least further from your interest.
00:52:02 It’s almost like an exploration button.
00:52:05 So just get your intuition.
00:52:08 Do you think engagement, so like you staying on the platform, you’re staying engaged.
00:52:15 Do you think fairness, ideas of fairness won’t emerge?
00:52:19 Like how bad is it to just optimize for engagement?
00:52:23 Do you think we’ll run into big trouble if we’re just optimizing for how much you love
00:52:28 the platform?
00:52:29 Well, I mean, optimizing for engagement kind of got us where we are.
00:52:34 So do you, one, have faith that it’s possible to do better?
00:52:39 And two, if it is, how do we do better?
00:52:44 I mean, it’s definitely possible to do different, right?
00:52:47 And again, you know, it’s not as if I think that doing something different than optimizing
00:52:51 for engagement won’t cost these companies in real ways, including revenue and profitability
00:52:57 potentially.
00:52:58 In the short term at least.
00:53:00 Yeah.
00:53:01 In the short term.
00:53:02 Right.
00:53:03 And again, you know, if I worked at these companies, I’m sure that it would have seemed
00:53:08 like the most natural thing in the world also to want to optimize engagement, right?
00:53:12 And that’s good for users in some sense.
00:53:14 You want them to be, you know, vested in the platform and enjoying it and finding it useful,
00:53:19 interesting, and or productive.
00:53:21 But you know, my point is, is that the idea that there is, that it’s sort of out of their
00:53:27 hands as you said, or that there’s nothing to do about it, never say never, but that
00:53:31 strikes me as implausible as a machine learning person, right?
00:53:34 I mean, these companies are driven by machine learning and this optimization of engagement
00:53:39 is essentially driven by machine learning, right?
00:53:42 It’s driven by not just machine learning, but you know, very, very large scale A, B
00:53:47 experimentation where you kind of tweak some element of the user interface or tweak some
00:53:53 component of an algorithm or tweak some component or feature of your click through prediction
00:53:59 model.
00:54:01 And my point is, is that anytime you know how to optimize for something, you, you know,
00:54:06 by def, almost by definition, that solution tells you how not to optimize for it or to
00:54:10 do something different.
00:54:13 Engagement can be measured.
00:54:16 So sort of optimizing for sort of minimizing divisiveness or maximizing intellectual growth
00:54:25 over the lifetime of a human being are very difficult to measure.
00:54:30 That’s right.
00:54:31 And I’m not claiming that doing something different will immediately make it apparent
00:54:38 that this is a good thing for society and in particular, I mean, I think one way of
00:54:42 thinking about where we are on some of these social media platforms is that, you know,
00:54:47 it kind of feels a bit like we’re in a bad equilibrium, right?
00:54:50 That these systems are helping us all kind of optimize something myopically and selfishly
00:54:55 for ourselves and of course, from an individual standpoint at any given moment, like why would
00:55:02 I want to see things in my newsfeed that I found irrelevant, offensive or, you know,
00:55:07 or the like, okay?
00:55:09 But you know, maybe by all of us, you know, having these platforms myopically optimized
00:55:15 in our interests, we have reached a collective outcome as a society that we’re unhappy with
00:55:20 in different ways.
00:55:21 Let’s say with respect to things like, you know, political discourse and tolerance of
00:55:26 opposing viewpoints.
00:55:28 And if Mark Zuckerberg gave you a call and said, I’m thinking of taking a sabbatical,
00:55:34 could you run Facebook for me for six months?
00:55:37 What would you, how?
00:55:39 I think no thanks would be my first response, but there are many aspects of being the head
00:55:45 of the entire company that are kind of entirely exogenous to many of the things that we’re
00:55:51 discussing here.
00:55:52 Yes.
00:55:53 And so I don’t really think I would need to be CEO of Facebook to kind of implement the,
00:55:58 you know, more limited set of solutions that I might imagine.
00:56:02 But I think one concrete thing they could do is they could experiment with letting people
00:56:08 who chose to, to see more stuff in their newsfeed that is not entirely kind of chosen to optimize
00:56:17 for their particular interests, beliefs, et cetera.
00:56:22 So the, the kind of thing, so I could speak to YouTube, but I think Facebook probably
00:56:27 does something similar is they’re quite effective at automatically finding what sorts of groups
00:56:34 you belong to, not based on race or gender or so on, but based on the kind of stuff you
00:56:40 enjoy watching in the case of YouTube.
00:56:43 Sort of, it’s a, it’s a difficult thing for Facebook or YouTube to then say, well, you
00:56:50 know what?
00:56:51 We’re going to show you something from a very different cluster.
00:56:54 Even though we believe algorithmically, you’re unlikely to enjoy that thing sort of that’s
00:57:00 a weird jump to make.
00:57:02 There has to be a human, like at the very top of that system that says, well, that will
00:57:07 be longterm healthy for you.
00:57:09 That’s more than an algorithmic decision.
00:57:11 Or that same person could say that’ll be longterm healthy for the platform or for the platform’s
00:57:18 influence on society outside of the platform, right?
00:57:22 And it, you know, it’s easy for me to sit here and say these things, but conceptually
00:57:27 I do not think that these are kind of totally or should, they shouldn’t be kind of completely
00:57:32 alien ideas, right?
00:57:34 That, you know, you could try things like this and it wouldn’t be, you know, we wouldn’t
00:57:40 have to invent entirely new science to do it because if we’re all already embedded in
00:57:45 some metric space and there’s a notion of distance between you and me and every other,
00:57:50 every piece of content, then, you know, we know exactly, you know, the same model that
00:57:56 tells, you know, dictates how to make me really happy also tells how to make me as unhappy
00:58:03 as possible as well.
00:58:04 Right.
00:58:05 The focus in your book and algorithmic fairness research today in general is on machine learning,
00:58:11 like we said, is data, but, and just even the entire AI field right now is captivated
00:58:16 with machine learning, with deep learning.
00:58:19 Do you think ideas in symbolic AI or totally other kinds of approaches are interesting,
00:58:25 useful in the space, have some promising ideas in terms of fairness?
00:58:31 I haven’t thought about that question specifically in the context of fairness.
00:58:35 I definitely would agree with that statement in the large, right?
00:58:39 I mean, I am, you know, one of many machine learning researchers who do believe that the
00:58:46 great successes that have been shown in machine learning recently are great successes, but
00:58:51 they’re on a pretty narrow set of tasks.
00:58:53 I mean, I don’t, I don’t think we’re kind of notably closer to general artificial intelligence
00:59:00 now than we were when I started my career.
00:59:03 I mean, there’s been progress and I do think that we are kind of as a community, maybe
00:59:08 looking a bit where the light is, but the light is shining pretty bright there right
00:59:12 now and we’re finding a lot of stuff.
00:59:13 So I don’t want to like argue with the progress that’s been made in areas like deep learning,
00:59:18 for example.
00:59:19 This touches another sort of related thing that you mentioned and that people might misinterpret
00:59:25 from the title of your book, ethical algorithm.
00:59:27 Is it possible for the algorithm to automate some of those decisions?
00:59:31 Sort of a higher level decisions of what kind of, like what, what should be fair, what should
00:59:37 be fair.
00:59:38 The more you know about a field, the more aware you are of its limitations.
00:59:43 And so I’m a, I’m pretty leery of sort of trying, you know, there’s, there’s so much
00:59:47 we don’t all, we already don’t know in fairness, even when we’re the ones picking the fairness
00:59:53 definitions and, you know, comparing alternatives and thinking about the tensions between different
00:59:58 definitions that the idea of kind of letting the algorithm start exploring as well.
01:00:05 I definitely think, you know, this is a much narrower statement.
01:00:08 I definitely think that kind of algorithmic auditing for different types of unfairness,
01:00:12 right?
01:00:13 So like in this gerrymandering example where I might want to prevent not just discrimination
01:00:18 against very broad categories, but against combinations of broad categories.
01:00:23 You know, you quickly get to a point where there’s a lot of, a lot of categories.
01:00:27 There’s a lot of combinations of end features and, you know, you can use algorithmic techniques
01:00:33 to sort of try to find the subgroups on which you’re discriminating the most and try to
01:00:38 fix that.
01:00:39 That’s actually kind of the form of one of the algorithms we developed for this fairness
01:00:42 gerrymandering problem.
01:00:44 But I’m, I’m, you know, partly because of our technological, you know, our sort of our
01:00:49 scientific ignorance on these topics right now.
01:00:53 And also partly just because these topics are so loaded emotionally for people that
01:00:58 I just don’t see the value.
01:01:00 I mean, again, never say never, but I just don’t think we’re at a moment where it’s
01:01:03 a great time for computer scientists to be rolling out the idea like, hey, you know,
01:01:08 you know, not only have we kind of figured fairness out, but, you know, we think the
01:01:12 algorithm should start deciding what’s fair or giving input on that decision.
01:01:16 I just don’t, it’s like the cost benefit analysis to the field of kind of going there
01:01:22 right now just doesn’t seem worth it to me.
01:01:24 That said, I should say that I think computer scientists should be more philosophically,
01:01:29 like should enrich their thinking about these kinds of things.
01:01:32 I think it’s been too often used as an excuse for roboticists working on autonomous vehicles,
01:01:38 for example, to not think about the human factor or psychology or safety in the same
01:01:43 way like computer science design algorithms that have been sort of using it as an excuse.
01:01:47 And I think it’s time for basically everybody to become a computer scientist.
01:01:51 I was about to agree with everything you said except that last point.
01:01:54 I think that the other way of looking at it is that I think computer scientists, you know,
01:01:59 and many of us are, but we need to weigh it out into the world more, right?
01:02:06 I mean, just the influence that computer science and therefore computer scientists have had
01:02:12 on society at large just like has exponentially magnified in the last 10 or 20 years or so.
01:02:21 And you know, before when we were just tinkering around amongst ourselves and it didn’t matter
01:02:26 that much, there was no need for sort of computer scientists to be citizens of the world more
01:02:32 broadly.
01:02:33 And I think those days need to be over very, very fast.
01:02:36 And I’m not saying everybody needs to do it, but to me, like the right way of doing it
01:02:40 is to not to sort of think that everybody else is going to become a computer scientist.
01:02:44 But you know, I think people are becoming more sophisticated about computer science,
01:02:49 even lay people.
01:02:50 You know, I think one of the reasons we decided to write this book is we thought 10 years
01:02:55 ago I wouldn’t have tried this just because I just didn’t think that sort of people’s
01:03:00 awareness of algorithms and machine learning, you know, the general population would have
01:03:06 been high.
01:03:07 I mean, you would have had to first, you know, write one of the many books kind of just explicating
01:03:12 that topic to a lay audience first.
01:03:14 Now I think we’re at the point where like lots of people without any technical training
01:03:18 at all know enough about algorithms and machine learning that you can start getting to these
01:03:22 nuances of things like ethical algorithms.
01:03:26 I think we agree that there needs to be much more mixing, but I think a lot of the onus
01:03:31 of that mixing needs to be on the computer science community.
01:03:35 Yeah.
01:03:36 So just to linger on the disagreement, because I do disagree with you on the point that I
01:03:41 think if you’re a biologist, if you’re a chemist, if you’re an MBA business person, all of those
01:03:50 things you can, like if you learned a program, and not only program, if you learned to do
01:03:57 machine learning, if you learned to do data science, you immediately become much more
01:04:02 powerful in the kinds of things you can do.
01:04:04 And therefore literature, like library sciences, like, so you were speaking, I think, I think
01:04:11 it holds true what you’re saying for the next few years.
01:04:14 But long term, if you’re interested to me, if you’re interested in philosophy, you should
01:04:21 learn a program, because then you can scrape data and study what people are thinking about
01:04:27 on Twitter, and then start making philosophical conclusions about the meaning of life.
01:04:33 I just feel like the access to data, the digitization of whatever problem you’re trying to solve,
01:04:41 will fundamentally change what it means to be a computer scientist.
01:04:44 I mean, a computer scientist in 20, 30 years will go back to being Donald Knuth style theoretical
01:04:51 computer science, and everybody would be doing basically, exploring the kinds of ideas that
01:04:56 you explore in your book.
01:04:57 It won’t be a computer science major.
01:04:58 Yeah, I mean, I don’t think I disagree enough, but I think that that trend of more and more
01:05:05 people in more and more disciplines adopting ideas from computer science, learning how
01:05:11 to code, I think that that trend seems firmly underway.
01:05:14 I mean, you know, like an interesting digressive question along these lines is maybe in 50
01:05:21 years, there won’t be computer science departments anymore, because the field will just sort
01:05:27 of be ambient in all of the different disciplines.
01:05:30 And people will look back and having a computer science department will look like having an
01:05:35 electricity department or something that’s like, you know, everybody uses this, it’s
01:05:39 just out there.
01:05:40 I mean, I do think there will always be that kind of Knuth style core to it, but it’s not
01:05:45 an implausible path that we kind of get to the point where the academic discipline of
01:05:50 computer science becomes somewhat marginalized because of its very success in kind of infiltrating
01:05:56 all of science and society and the humanities, etcetera.
01:06:00 What is differential privacy, or more broadly, algorithmic privacy?
01:06:07 Algorithmic privacy more broadly is just the study or the notion of privacy definitions
01:06:15 or norms being encoded inside of algorithms.
01:06:19 And so, you know, I think we count among this body of work just, you know, the literature
01:06:27 and practice of things like data anonymization, which we kind of at the beginning of our discussion
01:06:33 of privacy say like, okay, this is sort of a notion of algorithmic privacy.
01:06:38 It kind of tells you, you know, something to go do with data, but, you know, our view
01:06:44 is that it’s, and I think this is now, you know, quite widespread, that it’s, you know,
01:06:50 despite the fact that those notions of anonymization kind of redacting and coarsening are the most
01:06:57 widely adopted technical solutions for data privacy, they are like deeply fundamentally
01:07:03 flawed.
01:07:05 And so, you know, to your first question, what is differential privacy?
01:07:11 Differential privacy seems to be a much, much better notion of privacy that kind of avoids
01:07:16 a lot of the weaknesses of anonymization notions while still letting us do useful stuff with
01:07:24 data.
01:07:25 What is anonymization of data?
01:07:27 So by anonymization, I’m, you know, kind of referring to techniques like I have a database.
01:07:34 The rows of that database are, let’s say, individual people’s medical records, okay?
01:07:40 And I want to let people use that data.
01:07:43 Maybe I want to let researchers access that data to build predictive models for some disease,
01:07:49 but I’m worried that that will leak, you know, sensitive information about specific people’s
01:07:56 medical records.
01:07:57 So anonymization broadly refers to the set of techniques where I say like, okay, I’m
01:08:01 first going to like, I’m going to delete the column with people’s names.
01:08:06 I’m going to not put, you know, so that would be like a redaction, right?
01:08:09 I’m just redacting that information.
01:08:12 I am going to take ages and I’m not going to like say your exact age.
01:08:17 I’m going to say whether you’re, you know, zero to 10, 10 to 20, 20 to 30, I might put
01:08:23 the first three digits of your zip code, but not the last two, et cetera, et cetera.
01:08:27 And so the idea is that through some series of operations like this on the data, I anonymize
01:08:31 it.
01:08:32 You know, another term of art that’s used is removing personally identifiable information.
01:08:38 And you know, this is basically the most common way of providing data privacy, but that it’s
01:08:45 in a way that still lets people access the, some variant form of the data.
01:08:50 So at a slightly broader picture, as you talk about what does anonymization mean when you
01:08:56 have multiple database, like with a Netflix prize, when you can start combining stuff
01:09:01 together.
01:09:02 So this is exactly the problem with these notions, right?
01:09:05 Is that notions of a anonymization, removing personally identifiable information, the kind
01:09:10 of fundamental conceptual flaw is that, you know, these definitions kind of pretend as
01:09:16 if the data set in question is the only data set that exists in the world or that ever
01:09:21 will exist in the future.
01:09:23 And of course, things like the Netflix prize and many, many other examples since the Netflix
01:09:28 prize, I think that was one of the earliest ones though, you know, you can reidentify
01:09:33 people that were, you know, that were anonymized in the data set by taking that anonymized
01:09:38 data set and combining it with other allegedly anonymized data sets and maybe publicly available
01:09:43 information about you.
01:09:44 You know,
01:09:45 for people who don’t know the Netflix prize was, was being publicly released this data.
01:09:50 So the names from those rows were removed, but what was released is the preference or
01:09:55 the ratings of what movies you like and you don’t like.
01:09:58 And from that combined with other things, I think forum posts and so on, you can start
01:10:03 to figure out
01:10:04 I guess it was specifically the internet movie database where, where lots of Netflix users
01:10:10 publicly rate their movie, you know, their movie preferences.
01:10:15 And so the anonymized data and Netflix, when it’s just this phenomenon, I think that we’ve
01:10:21 all come to realize in the last decade or so is that just knowing a few apparently irrelevant
01:10:29 innocuous things about you can often act as a fingerprint.
01:10:33 Like if I know, you know, what, what rating you gave to these 10 movies and the date on
01:10:39 which you entered these movies, this is almost like a fingerprint for you in the sea of all
01:10:43 Netflix users.
01:10:44 There were just another paper on this in science or nature of about a month ago that, you know,
01:10:49 kind of 18 attributes.
01:10:51 I mean, my favorite example of this is, was actually a paper from several years ago now
01:10:57 where it was shown that just from your likes on Facebook, just from the time, you know,
01:11:03 the things on which you clicked on the thumbs up button on the platform, not using any information,
01:11:09 demographic information, nothing about who your friends are, just knowing the content
01:11:14 that you had liked was enough to, you know, in the aggregate accurately predict things
01:11:20 like sexual orientation, drug and alcohol use, whether you were the child of divorced parents.
01:11:27 So we live in this era where, you know, even the apparently irrelevant data that we offer
01:11:32 about ourselves on public platforms and forums often unbeknownst to us, more or less acts
01:11:38 as signature or, you know, fingerprint.
01:11:42 And that if you can kind of, you know, do a join between that kind of data and allegedly
01:11:46 anonymized data, you have real trouble.
01:11:50 So is there hope for any kind of privacy in a world where a few likes can identify you?
01:11:58 So there is differential privacy, right?
01:12:00 What is differential privacy?
01:12:01 Yeah, so differential privacy basically is a kind of alternate, much stronger notion
01:12:06 of privacy than these anonymization ideas.
01:12:10 And, you know, it’s a technical definition, but like the spirit of it is we compare two
01:12:18 alternate worlds, okay?
01:12:20 So let’s suppose I’m a researcher and I want to do, you know, there’s a database of medical
01:12:26 records and one of them is yours, and I want to use that database of medical records to
01:12:31 build a predictive model for some disease.
01:12:33 So based on people’s symptoms and test results and the like, I want to, you know, build a
01:12:39 probably model predicting the probability that people have disease.
01:12:42 So, you know, this is the type of scientific research that we would like to be allowed
01:12:46 to continue.
01:12:48 And in differential privacy, you ask a very particular counterfactual question.
01:12:53 We basically compare two alternatives.
01:12:57 One is when I do this, I build this model on the database of medical records, including
01:13:04 your medical record.
01:13:07 And the other one is where I do the same exercise with the same database with just your medical
01:13:15 record removed.
01:13:16 So basically, you know, it’s two databases, one with N records in it and one with N minus
01:13:22 one records in it.
01:13:23 The N minus one records are the same, and the only one that’s missing in the second
01:13:27 case is your medical record.
01:13:30 So differential privacy basically says that any harms that might come to you from the
01:13:40 analysis in which your data was included are essentially nearly identical to the harms
01:13:47 that would have come to you if the same analysis had been done without your medical record
01:13:52 included.
01:13:53 So in other words, this doesn’t say that bad things cannot happen to you as a result of
01:13:58 data analysis.
01:13:59 It just says that these bad things were going to happen to you already, even if your data
01:14:05 wasn’t included.
01:14:06 And to give a very concrete example, right, you know, like we discussed at some length,
01:14:12 the study that, you know, in the 50s that was done that established the link between
01:14:17 smoking and lung cancer.
01:14:19 And we make the point that, like, well, if your data was used in that analysis and, you
01:14:25 know, the world kind of knew that you were a smoker because, you know, there was no stigma
01:14:28 associated with smoking before those findings, real harm might have come to you as a result
01:14:35 of that study that your data was included in.
01:14:37 In particular, your insurer now might have a higher posterior belief that you might have
01:14:42 lung cancer and raise your premium.
01:14:44 So you’ve suffered economic damage.
01:14:47 But the point is, is that if the same analysis has been done with all the other N minus one
01:14:54 medical records and just yours missing, the outcome would have been the same.
01:14:58 Or your data wasn’t idiosyncratically crucial to establishing the link between smoking and
01:15:05 lung cancer because the link between smoking and lung cancer is like a fact about the world
01:15:10 that can be discovered with any sufficiently large database of medical records.
01:15:14 But that’s a very low value of harm.
01:15:17 Yeah.
01:15:18 So that’s showing that very little harm is done.
01:15:20 Great.
01:15:21 But how what is the mechanism of differential privacy?
01:15:24 So that’s the kind of beautiful statement of it.
01:15:27 It’s the mechanism by which privacy is preserved.
01:15:30 Yeah.
01:15:31 So it’s basically by adding noise to computations, right?
01:15:34 So the basic idea is that every differentially private algorithm, first of all, or every
01:15:40 good differentially private algorithm, every useful one, is a probabilistic algorithm.
01:15:45 So it doesn’t, on a given input, if you gave the algorithm the same input multiple times,
01:15:51 it would give different outputs each time from some distribution.
01:15:55 And the way you achieve differential privacy algorithmically is by kind of carefully and
01:15:59 tastefully adding noise to a computation in the right places.
01:16:05 And to give a very concrete example, if I wanna compute the average of a set of numbers,
01:16:11 the non private way of doing that is to take those numbers and average them and release
01:16:17 like a numerically precise value for the average.
01:16:21 In differential privacy, you wouldn’t do that.
01:16:24 You would first compute that average to numerical precisions, and then you’d add some noise
01:16:29 to it, right?
01:16:30 You’d add some kind of zero mean, Gaussian or exponential noise to it so that the actual
01:16:37 value you output is not the exact mean, but it’ll be close to the mean, but it’ll be close…
01:16:44 The noise that you add will sort of prove that nobody can kind of reverse engineer any
01:16:50 particular value that went into the average.
01:16:53 So noise is a savior.
01:16:56 How many algorithms can be aided by adding noise?
01:17:01 Yeah, so I’m a relatively recent member of the differential privacy community.
01:17:07 My co author, Aaron Roth is really one of the founders of the field and has done a great
01:17:12 deal of work and I’ve learned a tremendous amount working with him on it.
01:17:15 It’s a pretty grown up field already.
01:17:17 Yeah, but now it’s pretty mature.
01:17:18 But I must admit, the first time I saw the definition of differential privacy, my reaction
01:17:22 was like, wow, that is a clever definition and it’s really making very strong promises.
01:17:28 And I first saw the definition in much earlier days and my first reaction was like, well,
01:17:34 my worry about this definition would be that it’s a great definition of privacy, but that
01:17:38 it’ll be so restrictive that we won’t really be able to use it.
01:17:43 We won’t be able to compute many things in a differentially private way.
01:17:47 So that’s one of the great successes of the field, I think, is in showing that the opposite
01:17:51 is true and that most things that we know how to compute, absent any privacy considerations,
01:18:00 can be computed in a differentially private way.
01:18:02 So for example, pretty much all of statistics and machine learning can be done differentially
01:18:08 privately.
01:18:09 So pick your favorite machine learning algorithm, back propagation and neural networks, cart
01:18:15 for decision trees, support vector machines, boosting, you name it, as well as classic
01:18:21 hypothesis testing and the like in statistics.
01:18:24 None of those algorithms are differentially private in their original form.
01:18:29 All of them have modifications that add noise to the computation in different places in
01:18:35 different ways that achieve differential privacy.
01:18:39 So this really means that to the extent that we’ve become a scientific community very dependent
01:18:47 on the use of machine learning and statistical modeling and data analysis, we really do have
01:18:53 a path to provide privacy guarantees to those methods and so we can still enjoy the benefits
01:19:02 of the data science era while providing rather robust privacy guarantees to individuals.
01:19:10 So perhaps a slightly crazy question, but if we take the ideas of differential privacy
01:19:16 and take it to the nature of truth that’s being explored currently.
01:19:20 So what’s your most favorite and least favorite food?
01:19:24 Hmm.
01:19:25 I’m not a real foodie, so I’m a big fan of spaghetti.
01:19:29 Spaghetti?
01:19:30 Yeah.
01:19:31 What do you really don’t like?
01:19:35 I really don’t like cauliflower.
01:19:37 Wow, I love cauliflower.
01:19:39 Okay.
01:19:40 Is there one way to protect your preference for spaghetti by having an information campaign
01:19:46 bloggers and so on of bots saying that you like cauliflower?
01:19:51 So like this kind of the same kind of noise ideas, I mean if you think of in our politics
01:19:56 today there’s this idea of Russia hacking our elections.
01:20:01 What’s meant there I believe is bots spreading different kinds of information.
01:20:07 Is that a kind of privacy or is that too much of a stretch?
01:20:10 No it’s not a stretch.
01:20:12 I’ve not seen those ideas, you know, that is not a technique that to my knowledge will
01:20:19 provide differential privacy, but to give an example like one very specific example
01:20:24 about what you’re discussing is there was a very interesting project at NYU I think
01:20:30 led by Helen Nissenbaum there in which they basically built a browser plugin that tried
01:20:38 to essentially obfuscate your Google searches.
01:20:41 So to the extent that you’re worried that Google is using your searches to build, you
01:20:46 know, predictive models about you to decide what ads to show you which they might very
01:20:51 reasonably want to do, but if you object to that they built this widget you could plug
01:20:56 in and basically whenever you put in a query into Google it would send that query to Google,
01:21:01 but in the background all of the time from your browser it would just be sending this
01:21:06 torrent of irrelevant queries to the search engine.
01:21:11 So you know it’s like a weed and chaff thing so you know out of every thousand queries
01:21:16 let’s say that Google was receiving from your browser one of them was one that you put in
01:21:21 but the other 999 were not okay so it’s the same kind of idea kind of you know privacy
01:21:27 by obfuscation.
01:21:29 So I think that’s an interesting idea, doesn’t give you differential privacy.
01:21:34 It’s also I was actually talking to somebody at one of the large tech companies recently
01:21:39 about the fact that you know just this kind of thing that there are some times when the
01:21:45 response to my data needs to be very specific to my data right like I type mountain biking
01:21:53 into Google, I want results on mountain biking and I really want Google to know that I typed
01:21:58 in mountain biking, I don’t want noise added to that.
01:22:01 And so I think there’s sort of maybe even interesting technical questions around notions
01:22:06 of privacy that are appropriate where you know it’s not that my data is part of some
01:22:10 aggregate like medical records and that we’re trying to discover important correlations
01:22:15 and facts about the world at large but rather you know there’s a service that I really want
01:22:20 to you know pay attention to my specific data yet I still want some kind of privacy guarantee
01:22:26 and I think these kind of obfuscation ideas are sort of one way of getting at that but
01:22:30 maybe there are others as well.
01:22:32 So where do you think we’ll land in this algorithm driven society in terms of privacy?
01:22:36 So sort of China like Kai Fuli describes you know it’s collecting a lot of data on its
01:22:44 citizens but in the best form it’s actually able to provide a lot of sort of protect human
01:22:52 rights and provide a lot of amazing services and it’s worst forms that can violate those
01:22:57 human rights and limit services.
01:23:01 So where do you think we’ll land because algorithms are powerful when they use data.
01:23:08 So as a society do you think we’ll give over more data?
01:23:12 Is it possible to protect the privacy of that data?
01:23:16 So I’m optimistic about the possibility of you know balancing the desire for individual
01:23:24 privacy and individual control of privacy with kind of societally and commercially beneficial
01:23:32 uses of data not unrelated to differential privacy or suggestions that say like well
01:23:37 individuals should have control of their data.
01:23:40 They should be able to limit the uses of that data.
01:23:43 They should even you know there’s you know fledgling discussions going on in research
01:23:48 circles about allowing people selective use of their data and being compensated for it.
01:23:54 And then you get to sort of very interesting economic questions like pricing right.
01:23:59 And one interesting idea is that maybe differential privacy would also you know be a conceptual
01:24:05 framework in which you could talk about the relative value of different people’s data
01:24:09 like you know to demystify this a little bit.
01:24:12 If I’m trying to build a predictive model for some rare disease and I’m trying to use
01:24:17 machine learning to do it, it’s easy to get negative examples because the disease is rare
01:24:22 right.
01:24:23 But I really want to have lots of people with the disease in my data set okay.
01:24:30 And so somehow those people’s data with respect to this application is much more valuable
01:24:35 to me than just like the background population.
01:24:37 And so maybe they should be compensated more for it.
01:24:43 And so you know I think these are kind of very, very fledgling conceptual questions
01:24:48 that maybe we’ll have kind of technical thought on them sometime in the coming years.
01:24:54 But I do think we’ll you know to kind of get more directly answer your question.
01:24:56 I think I’m optimistic at this point from what I’ve seen that we will land at some you
01:25:02 know better compromise than we’re at right now where again you know privacy guarantees
01:25:08 are few far between and weak and users have very, very little control.
01:25:15 And I’m optimistic that we’ll land in something that you know provides better privacy overall
01:25:20 and more individual control of data and privacy.
01:25:22 But you know I think to get there it’s again just like fairness it’s not going to be enough
01:25:27 to propose algorithmic solutions.
01:25:29 There’s going to have to be a whole kind of regulatory legal process that prods companies
01:25:34 and other parties to kind of adopt solutions.
01:25:38 And I think you’ve mentioned the word control a lot and I think giving people control that’s
01:25:43 something that people don’t quite have in a lot of these algorithms and that’s a really
01:25:48 interesting idea of giving them control.
01:25:50 Some of that is actually literally an interface design question sort of just enabling because
01:25:57 I think it’s good for everybody to give users control.
01:26:00 It’s almost not a trade off except that you have to hire people that are good at interface
01:26:06 design.
01:26:07 Yeah.
01:26:08 I mean the other thing that has to be said right is that you know it’s a cliche but you
01:26:13 know we as the users of many systems platforms and apps you know we are the product.
01:26:21 We are not the customer.
01:26:23 The customer are advertisers and our data is the product.
01:26:26 Okay.
01:26:27 So it’s one thing to kind of suggest more individual control of data and privacy and
01:26:32 uses but this you know if this happens in sufficient degree it will upend the entire
01:26:40 economic model that has supported the internet to date.
01:26:44 And so some other economic model will have to be you know we’ll have to replace it.
01:26:50 So the idea of markets you mentioned by exposing the economic model to the people they will
01:26:56 then become a market.
01:26:57 They could be participants in it.
01:27:00 And you know this isn’t you know this is not a weird idea right because there are markets
01:27:04 for data already.
01:27:05 It’s just that consumers are not participants and there’s like you know there’s sort of
01:27:10 you know publishers and content providers on one side that have inventory and then their
01:27:14 advertisers on the others and you know you know Google and Facebook are running you know
01:27:19 they’re pretty much their entire revenue stream is by running two sided markets between those
01:27:25 parties right.
01:27:27 And so it’s not a crazy idea that there would be like a three sided market or that you know
01:27:32 that on one side of the market or the other we would have proxies representing our interest.
01:27:37 It’s not you know it’s not a crazy idea but it would it’s not a crazy technical idea but
01:27:43 it would have pretty extreme economic consequences.
01:27:49 Speaking of markets a lot of fascinating aspects of this world arise not from individual human
01:27:55 beings but from the interaction of human beings.
01:27:59 You’ve done a lot of work in game theory.
01:28:02 First can you say what is game theory and how does it help us model and study?
01:28:07 Yeah game theory of course let us give credit where it’s due.
01:28:11 You know it comes from the economist first and foremost but as I’ve mentioned before
01:28:16 like you know computer scientists never hesitate to wander into other people’s turf and so
01:28:22 there is now this 20 year old field called algorithmic game theory.
01:28:26 But you know game theory first and foremost is a mathematical framework for reasoning
01:28:33 about collective outcomes in systems of interacting individuals.
01:28:40 You know so you need at least two people to get started in game theory and many people
01:28:46 are probably familiar with Prisoner’s Dilemma as kind of a classic example of game theory
01:28:50 and a classic example where everybody looking out for their own individual interests leads
01:28:57 to a collective outcome that’s kind of worse for everybody than what might be possible
01:29:02 if they cooperated for example.
01:29:05 But cooperation is not an equilibrium in Prisoner’s Dilemma.
01:29:09 And so my work in the field of algorithmic game theory more generally in these areas
01:29:16 kind of looks at settings in which the number of actors is potentially extraordinarily large
01:29:24 and their incentives might be quite complicated and kind of hard to model directly but you
01:29:31 still want kind of algorithmic ways of kind of predicting what will happen or influencing
01:29:36 what will happen in the design of platforms.
01:29:39 So what to you is the most beautiful idea that you’ve encountered in game theory?
01:29:47 There’s a lot of them.
01:29:48 I’m a big fan of the field.
01:29:50 I mean you know I mean technical answers to that of course would include Nash’s work just
01:29:56 establishing that you know there is a competitive equilibrium under very very general circumstances
01:30:02 which in many ways kind of put the field on a firm conceptual footing because if you don’t
01:30:09 have equilibrium it’s kind of hard to ever reason about what might happen since you know
01:30:14 there’s just no stability.
01:30:16 So just the idea that stability can emerge when there’s multiple.
01:30:20 Not that it will necessarily emerge just that it’s possible right.
01:30:23 Like the existence of equilibrium doesn’t mean that sort of natural iterative behavior
01:30:28 will necessarily lead to it.
01:30:30 In the real world.
01:30:31 Yeah.
01:30:32 Maybe answering a slightly less personally than you asked the question I think within
01:30:35 the field of algorithmic game theory perhaps the single most important kind of technical
01:30:43 contribution that’s been made is the realization between close connections between machine
01:30:49 learning and game theory and in particular between game theory and the branch of machine
01:30:53 learning that’s known as no regret learning and this sort of provides a very general framework
01:31:00 in which a bunch of players interacting in a game or a system each one kind of doing
01:31:07 something that’s in their self interest will actually kind of reach an equilibrium and
01:31:12 actually reach an equilibrium in a you know a pretty you know a rather you know short
01:31:18 amount of steps.
01:31:21 So you kind of mentioned acting greedily can somehow end up pretty good for everybody.
01:31:30 Or pretty bad.
01:31:31 Or pretty bad.
01:31:32 Yeah.
01:31:33 It will end up stable.
01:31:34 Yeah.
01:31:35 Right.
01:31:36 And and you know stability or equilibrium by itself is neither is not necessarily either
01:31:41 a good thing or a bad thing.
01:31:43 So what’s the connection between machine learning and the ideas.
01:31:45 Well I think we kind of talked about these ideas already in kind of a non technical way
01:31:50 which is maybe the more interesting way of understanding them first which is you know
01:31:57 we have many systems platforms and apps these days that work really hard to use our data
01:32:04 and the data of everybody else on the platform to selfishly optimize on behalf of each user.
01:32:12 OK.
01:32:13 So you know let me let me give I think the cleanest example which is just driving apps
01:32:17 navigation apps like you know Google Maps and Waze where you know miraculously compared
01:32:24 to when I was growing up at least you know the objective would be the same when you wanted
01:32:28 to drive from point A to point B spend the least time driving not necessarily minimize
01:32:33 the distance but minimize the time.
01:32:35 Right.
01:32:36 And when I was growing up like the only resources you had to do that were like maps in the car
01:32:41 which literally just told you what roads were available and then you might have like half
01:32:46 hourly traffic reports just about the major freeways but not about side roads.
01:32:51 So you were pretty much on your own.
01:32:54 And now we’ve got these apps you pull it out and you say I want to go from point A to point
01:32:57 B and in response kind of to what everybody else is doing if you like what all the other
01:33:03 players in this game are doing right now here’s the you know the route that minimizes your
01:33:09 driving time.
01:33:10 So it is really kind of computing a selfish best response for each of us in response to
01:33:16 what all of the rest of us are doing at any given moment.
01:33:20 And so you know I think it’s quite fair to think of these apps as driving or nudging
01:33:26 us all towards the competitive or Nash equilibrium of that game.
01:33:32 Now you might ask like well that sounds great why is that a bad thing.
01:33:36 Well you know it’s known both in theory and with some limited studies from actual like
01:33:45 traffic data that all of us being in this competitive equilibrium might cause our collective
01:33:52 driving time to be higher maybe significantly higher than it would be under other solutions.
01:33:59 And then you have to talk about what those other solutions might be and what the algorithms
01:34:04 to implement them are which we do discuss in the kind of game theory chapter of the
01:34:07 book.
01:34:09 But similarly you know on social media platforms or on Amazon you know all these algorithms
01:34:17 that are essentially trying to optimize our behalf they’re driving us in a colloquial
01:34:22 sense towards some kind of competitive equilibrium and you know one of the most important lessons
01:34:26 of game theory is that just because we’re at equilibrium doesn’t mean that there’s not
01:34:30 a solution in which some or maybe even all of us might be better off.
01:34:35 And then the connection to machine learning of course is that in all these platforms I’ve
01:34:39 mentioned the optimization that they’re doing on our behalf is driven by machine learning
01:34:44 you know like predicting where the traffic will be predicting what products I’m going
01:34:48 to like predicting what would make me happy in my newsfeed.
01:34:52 Now in terms of the stability and the promise of that I have to ask just out of curiosity
01:34:56 how stable are these mechanisms that you game theory is just the economist came up with
01:35:02 and we all know that economists don’t live in the real world just kidding sort of what’s
01:35:08 do you think when we look at the fact that we haven’t blown ourselves up from the from
01:35:15 a game theoretic concept of mutually shared destruction what are the odds that we destroy
01:35:21 ourselves with nuclear weapons as one example of a stable game theoretic system?
01:35:28 Just to prime your viewers a little bit I mean I think you’re referring to the fact
01:35:32 that game theory was taken quite seriously back in the 60s as a tool for reasoning about
01:35:38 kind of Soviet US nuclear armament disarmament detente things like that.
01:35:45 I’ll be honest as huge of a fan as I am of game theory and its kind of rich history it
01:35:52 still surprises me that you know you had people at the RAND Corporation back in those days
01:35:57 kind of drawing up you know two by two tables and one the row player is you know the US
01:36:02 and the column player is Russia and that they were taking seriously you know I’m sure if
01:36:08 I was there maybe it wouldn’t have seemed as naive as it does at the time you know.
01:36:12 Seems to have worked which is why it seems naive.
01:36:15 Well we’re still here.
01:36:16 We’re still here in that sense.
01:36:17 Yeah even though I kind of laugh at those efforts they were more sensible then than
01:36:22 they would be now right because there were sort of only two nuclear powers at the time
01:36:26 and you didn’t have to worry about deterring new entrants and who was developing the capacity
01:36:32 and so we have many you know it’s definitely a game with more players now and more potential
01:36:39 entrants.
01:36:40 I’m not in general somebody who advocates using kind of simple mathematical models when
01:36:46 the stakes are as high as things like that and the complexities are very political and
01:36:51 social but we are still here.
01:36:55 So you’ve worn many hats one of which the one that first caused me to become a big fan
01:37:00 of your work many years ago is algorithmic trading.
01:37:04 So I have to just ask a question about this because you have so much fascinating work
01:37:08 there in the 21st century what role do you think algorithms have in space of trading
01:37:15 investment in the financial sector?
01:37:19 Yeah it’s a good question I mean in the time I’ve spent on Wall Street and in finance you
01:37:27 know I’ve seen a clear progression and I think it’s a progression that kind of models the
01:37:31 use of algorithms and automation more generally in society which is you know the things that
01:37:38 kind of get taken over by the algos first are sort of the things that computers are
01:37:44 obviously better at than people right so you know so first of all there needed to be this
01:37:50 era of automation right where just you know financial exchanges became largely electronic
01:37:56 which then enabled the possibility of you know trading becoming more algorithmic because
01:38:01 once you know that exchanges are electronic an algorithm can submit an order through an
01:38:06 API just as well as a human can do at a monitor quickly can read all the data so yeah and
01:38:11 so you know I think the places where algorithmic trading have had the greatest inroads and
01:38:18 had the first inroads were in kind of execution problems kind of optimized execution problems
01:38:24 so what I mean by that is at a large brokerage firm for example one of the lines of business
01:38:30 might be on behalf of large institutional clients taking you know what we might consider
01:38:36 difficult trade so it’s not like a mom and pop investor saying I want to buy a hundred
01:38:40 shares of Microsoft it’s a large hedge fund saying you know I want to buy a very very
01:38:45 large stake in Apple and I want to do it over the span of a day and it’s such a large volume
01:38:52 that if you’re not clever about how you break that trade up not just over time but over
01:38:57 perhaps multiple different electronic exchanges that all let you trade Apple on their platform
01:39:02 you know you will you will move you’ll push prices around in a way that hurts your your
01:39:07 execution so you know this is the kind of you know this is an optimization problem this
01:39:11 is a control problem right and so machines are better we we know how to design algorithms
01:39:19 you know that are better at that kind of thing than a person is going to be able to do because
01:39:23 we can take volumes of historical and real time data to kind of optimize the schedule
01:39:29 with which we trade and you know similarly high frequency trading you know which is closely
01:39:35 related but not the same as optimized execution where you’re just trying to spot very very
01:39:41 temporary you know mispricings between exchanges or within an asset itself or just predict
01:39:48 directional movement of a stock because of the kind of very very low level granular buying
01:39:54 and selling data in the in the exchange machines are good at this kind of stuff it’s kind of
01:40:00 like the mechanics of trading what about the can machines do long terms of prediction yeah
01:40:08 so I think we are in an era where you know clearly there have been some very successful
01:40:13 you know quant hedge funds that are you know in what we would traditionally call you know
01:40:19 still in this the stat arb regime like so you know what’s that stat arb referring to
01:40:24 statistical arbitrage but but for the purposes of this conversation what it really means
01:40:28 is making directional predictions in asset price movement or returns your prediction
01:40:35 about that directional movement is good for you know you you have a view that it’s valid
01:40:42 for some period of time between a few seconds and a few days and that’s the amount of time
01:40:48 that you’re going to kind of get into the position hold it and then hopefully be right
01:40:51 about the directional movement and you know buy low and sell high as the cliche goes.
01:40:57 So that is a you know kind of a sweet spot I think for quant trading and investing right
01:41:04 now and has been for some time when you really get to kind of more Warren Buffett style timescales
01:41:11 right like you know my cartoon of Warren Buffett is that you know Warren Buffett sits and thinks
01:41:16 what the long term value of Apple really should be and he doesn’t even look at what Apple
01:41:22 is doing today he just decides you know you know I think that this is what its long term
01:41:27 value is and it’s far from that right now and so I’m going to buy some Apple or you
01:41:31 know short some Apple and I’m going to I’m going to sit on that for 10 or 20 years okay.
01:41:37 So when you’re at that kind of timescale or even more than just a few days all kinds of
01:41:45 other sources of risk and information you know so now you’re talking about holding things
01:41:51 through recessions and economic cycles, wars can break out.
01:41:56 So there you have to understand human nature at a level that.
01:41:59 Yeah and you need to just be able to ingest many many more sources of data that are on
01:42:03 wildly different timescales right.
01:42:06 So if I’m an HFT I’m a high frequency trader like I don’t I don’t I really my main source
01:42:13 of data is just the data from the exchanges themselves about the activity in the exchanges
01:42:18 right and maybe I need to pay you know I need to keep an eye on the news right because you
01:42:22 know that can cause sudden you know the CEO gets caught in a scandal or you know gets
01:42:29 run over by a bus or something that can cause very sudden changes but you know I don’t need
01:42:33 to understand economic cycles I don’t need to understand recessions I don’t need to worry
01:42:38 about the political situation or war breaking out in this part of the world because you
01:42:43 know all I need to know is as long as that’s not going to happen in the next 500 milliseconds
01:42:49 then you know my model is good.
01:42:52 When you get to these longer timescales you really have to worry about that kind of stuff
01:42:55 and people in the machine learning community are starting to think about this.
01:42:59 We held a we jointly sponsored a workshop at Penn with the Federal Reserve Bank of Philadelphia
01:43:06 a little more than a year ago on you know I think the title is something like machine
01:43:10 learning for macroeconomic prediction.
01:43:14 You know macroeconomic referring specifically to these longer timescales and you know it
01:43:19 was an interesting conference but it you know my it left me with greater confidence that
01:43:26 we have a long way to go to you know and so I think that people that you know in the grand
01:43:32 scheme of things you know if somebody asked me like well whose job on Wall Street is safe
01:43:37 from the bots I think people that are at that longer you know timescale and have that appetite
01:43:42 for all the risks involved in long term investing and that really need kind of not just algorithms
01:43:49 that can optimize from data but they need views on stuff they need views on the political
01:43:54 landscape economic cycles and the like and I think you know they’re they’re they’re pretty
01:44:01 safe for a while as far as I can tell.
01:44:02 So Warren Buffett’s job is not seeing you know a robo Warren Buffett anytime soon.
01:44:08 Give him comfort.
01:44:10 Last question.
01:44:11 If you could go back to if there’s a day in your life you could relive because it made
01:44:18 you truly happy.
01:44:21 Maybe you outside family what otherwise you know what what what day would it be.
01:44:29 But can you look back you remember just being profoundly transformed in some way or blissful.
01:44:40 I’ll answer a slightly different question which is like what’s a day in my my life or
01:44:44 my career that was kind of a watershed moment.
01:44:49 I went straight from undergrad to doctoral studies and you know that’s not at all atypical
01:44:55 and I’m also from an academic family like my my dad was a professor my uncle on his
01:45:00 side as a professor both my grandfathers were professors.
01:45:03 All kinds of majors to philosophy.
01:45:05 Yeah they’re kind of all over the map yeah and I was a grad student here just up the
01:45:10 river at Harvard and came to study with Les Valiant which was a wonderful experience.
01:45:15 But you know I remember my first year of graduate school I was generally pretty unhappy and
01:45:21 I was unhappy because you know at Berkeley as an undergraduate you know yeah I studied
01:45:25 a lot of math and computer science but it was a huge school first of all and I took
01:45:29 a lot of other courses as we’ve discussed I started as an English major and took history
01:45:34 courses and art history classes and had friends you know that did all kinds of different things.
01:45:40 And you know Harvard’s a much smaller institution than Berkeley and its computer science department
01:45:44 especially at that time was was a much smaller place than it is now.
01:45:48 And I suddenly just felt very you know like I’d gone from this very big world to this
01:45:54 highly specialized world and now all of the classes I was taking were computer science
01:45:59 classes and I was only in classes with math and computer science people.
01:46:04 And so I was you know I thought often in that first year of grad school about whether I
01:46:09 really wanted to stick with it or not and you know I thought like oh I could you know
01:46:14 stop with a master’s I could go back to the Bay Area and to California and you know this
01:46:19 was in one of the early periods where there was you know like you could definitely get
01:46:23 a relatively good job paying job at one of the one of the tech companies back you know
01:46:28 that were the big tech companies back then.
01:46:31 And so I distinctly remember like kind of a late spring day when I was kind of you know
01:46:36 sitting in Boston Common and kind of really just kind of chewing over what I wanted to
01:46:40 do with my life and I realized like okay and I think this is where my academic background
01:46:45 helped me a great deal.
01:46:46 I sort of realized you know yeah you’re not having a great time right now this feels really
01:46:50 narrowing but you know that you’re here for research eventually and to do something original
01:46:56 and to try to you know carve out a career where you kind of you know choose what you
01:47:02 want to think about you know and have a great deal of independence.
01:47:06 And so you know at that point I really didn’t have any real research experience yet I mean
01:47:10 it was trying to think about some problems with very little success but I knew that like
01:47:15 I hadn’t really tried to do the thing that I knew I’d come to do and so I thought you
01:47:23 know I’m going to stick through it for the summer and you know and that was very formative
01:47:30 because I went from kind of contemplating quitting to you know a year later it being
01:47:37 very clear to me I was going to finish because I still had a ways to go but I kind of started
01:47:42 doing research it was going well it was really interesting and it was sort of a complete
01:47:46 transformation you know it’s just that transition that I think every doctoral student makes
01:47:52 at some point which is to sort of go from being like a student of what’s been done before
01:48:00 to doing you know your own thing and figure out what makes you interested in what your
01:48:04 strengths and weaknesses are as a researcher and once you know I kind of made that decision
01:48:09 on that particular day at that particular moment in Boston Common you know I’m glad
01:48:15 I made that decision.
01:48:16 And also just accepting the painful nature of that journey.
01:48:19 Yeah exactly exactly.
01:48:21 In that moment said I’m gonna I’m gonna stick it out yeah I’m gonna stick around for a while.
01:48:26 Well Michael I’ve looked off do you work for a long time it’s really nice to talk to you
01:48:30 thank you so much.
01:48:31 It’s great to get back in touch with you too and see how great you’re doing as well.
01:48:34 Thanks a lot.
01:48:35 Thank you.