Luís and João Batalha: Fermat's Library and the Art of Studying Papers #209

Transcript

00:00:00 The following is a conversation with Luis and João Botala, brothers and cofounders of Fermat’s

00:00:06 Library, which is an incredible platform for annotating papers. As they write on the Fermat’s

00:00:12 Library website, just as Pierre de Fermat scribbled his famous last theorem in the margins, professional

00:00:19 scientists, academics, and citizen scientists can annotate equations, figures, ideas, and write

00:00:25 in the margins. Fermat’s Library is also a really good Twitter account to follow. I highly recommend

00:00:31 it. They post little visual factoids and explorations that reveal the beauty of mathematics.

00:00:38 I love it. Quick mention of our sponsors. Skiff, SimplySafe, Indeed, NetSuite, and Four Sigmatic.

00:00:47 Check them out in the description to support this podcast. As a side note, let me say a few words

00:00:52 about the dissemination of scientific ideas. I believe that all scientific articles should be

00:00:59 freely accessible to the public. They currently are not. In one analysis I saw, more than 70%

00:01:05 of published research articles are behind a paywall. In case you don’t know, the funders of

00:01:11 the research, whether that’s government or industry, aren’t the ones putting up the paywall. The journals

00:01:18 are the ones putting up the paywall, while using unpaid labor from researchers for the peer review

00:01:24 process. Where is all that money from the paywall going? In this digital age, the costs here should

00:01:31 be minimal. This cost can easily be covered through donation, advertisement, or public funding of

00:01:37 science. The benefit versus the cost of all papers being free to read is obvious, and the fact that

00:01:43 they’re not free goes against everything science should stand for, which is the free dissemination

00:01:48 of ideas that educate and inspire. Science cannot be a gated institution. The more people can freely

00:01:56 learn and collaborate on ideas, the more problems we can solve in the world together, and the faster

00:02:02 we can drive old ideas out and bring new, better ideas in. Science is beautiful and powerful, and

00:02:10 its dissemination in this digital age should be free. This is the Lex Friedman podcast, and here’s

00:02:17 my conversation with Luis and Joao Batala. Luis, you suggested an interesting idea. Imagine if most

00:02:26 papers had a backstory section, the same way that they have an abstract. So knowing more about how

00:02:34 the authors ended up working on a paper can be extremely insightful. And then you went on to give

00:02:40 a backstory for the Feynman QED paper. This is all in a tweet, by the way. We’re doing tweet analysis

00:02:45 today. How much of the human backstory do you think is important in understanding the idea itself

00:02:51 that’s presented in the paper or in general? I think this gives way more context to the work of

00:02:57 scientists. I think people, a lot of people have this almost kind of romantic misconception that

00:03:03 the way a lot of scientists work is almost as the sum of eureka moments where all of a sudden they

00:03:09 sit down and start writing two papers in a row, and the papers are usually isolated. And when you

00:03:15 actually look at it, the papers are chapters of a way more complex story. And the Feynman QED paper

00:03:23 is a good example. So Feynman was actually going through a pretty dark phase before writing that

00:03:29 paper. He lost enthusiasm with physics and doing physics problems. And there was one time when he

00:03:35 was in the cafeteria of Cornell, and he saw a guy that was throwing plates in the air. And he noticed

00:03:41 that when the plate was in the air, there were two movements there. The plate was wobbling, but he

00:03:47 also noticed that the Cornell symbol was rotating. And he was able to figure out the equations of

00:03:52 motions of those plates. And that led him to kind of think a little bit about electron orbits in

00:04:03 relativity, which led to the paper about quantum electrodynamics. So that kind of reignited his

00:04:11 interest in physics and ended up publishing the paper that led to his Nobel Prize, basically.

00:04:18 And I think there are a lot of really interesting backstories about papers that readers never get to

00:04:24 know. For instance, we did a couple of months ago an AMA around a paper, a pretty famous paper,

00:04:33 the GAMS paper with Ian Goodfellow. And so we did an AMA where everyone could ask questions about

00:04:38 the paper, and Ian was responding to those questions. And he was also telling the story of

00:04:44 how he got the idea for that paper in a bar. So that was also an interesting backstory. I also

00:04:50 read a book by Cedric Villani. Cedric Villani is this mathematician, a Fields Medalist. And in his

00:04:59 book, he tries to explain how he got from a PhD student to the Fields Medal. And he tries to be

00:05:05 as descriptive as possible, every single step, how he got to the Fields Medal. And it’s interesting

00:05:11 also to see just the amount of random interactions and discussions with other researchers, sometimes

00:05:16 over coffee, and how it led to like fundamental breakthroughs and some of his most important

00:05:22 papers. So I think it’s super interesting to have that context of the backstory.

00:05:25 Well, the Ian Goodfellow story is kind of interesting, and perhaps that’s true for

00:05:29 Feynman as well. I don’t know if it’s romanticizing the thing, but it seems like just a few little

00:05:35 insights and a little bit of work does most of the leap required. Do you have a sense that for a lot

00:05:41 of the stuff you’ve looked at, just looking back through history, it wasn’t necessarily the grind

00:05:49 of like Andrew Wiles or the Fermat’s Last Theorem, for example. It was more like a brilliant moment

00:05:55 of insight. In fact, Ian Goodfellow has a kind of sadness to him almost, in that at that time in

00:06:02 machine learning, like at that time, especially for GANs, you could code something up really

00:06:10 quickly in a single machine and almost do the invention, go from idea to experimental validation

00:06:18 in like a single night, a single person could do it. And now there’s kind of a sadness that a lot

00:06:22 of the breakthroughs you might have in machine learning kind of require large scale experiments.

00:06:27 So it was almost like the early days. So I wonder how many low hanging fruit there are

00:06:35 in science and mathematics and even engineering where it’s like you could do that little experiment

00:06:41 quickly, like you have an insight in a bar. Why is it always a bar? But you have an insight at a bar

00:06:46 and then just implement and the world changes. It’s a good point. I think it also depends a lot

00:06:53 on the maturity of the field. When you look at a field like mathematics, like it’s a pretty mature

00:06:59 field, a field like machine learning, it’s growing pretty fast. And it’s actually pretty

00:07:07 interesting. I looked up like the number of new papers on archive with the keyword machine learning

00:07:14 and like 50% of those papers have been published in the last 12 months. So you can see just the

00:07:20 5.0, 50%. So you can see the magnitude of growth in that field. And so I think like as fields

00:07:29 mature, like those types of moments, I think naturally are less frequent. It’s just a consequence

00:07:37 of that. The other point that is interesting about the backstory is that it can really make it

00:07:42 more memorable in a way. And by making it more memorable, it kind of sediments the knowledge

00:07:47 more in your mind. I remember also reading the sort of the backstory to Dijkstra’s shortest path

00:07:54 algorithm. He came up with it essentially while he was sitting down at a coffee shop in Amsterdam.

00:08:03 And he came up with that algorithm over 20 minutes. And one interesting aspect is he didn’t have any

00:08:09 pen or paper at the time. And so he had to do it all in his mind. And so there’s only so much

00:08:13 complexity that he can handle if you’re just thinking about it in your mind. And that like

00:08:18 when you think about the simplicity of Dijkstra’s shortest path finding algorithm, it’s knowing that

00:08:24 backstory helps sediment that algorithm in your mind so that you don’t forget about it as easily.

00:08:30 It might be from you that I saw a meme about Dijkstra. It’s like he’s trying to solve it and

00:08:38 he comes up with some kind of random path. And then it’s like my parents aren’t home. And then

00:08:45 he does. He figures out the algorithm for the shortest path. He tried through words to convey

00:08:53 memes, but it’s hilarious. I don’t know if it’s in post that we construct stories that romanticize

00:08:59 it. Apparently with Newton, there was no apple. Especially when you’re working on problems that

00:09:05 have a physical manifestation or a visual manifestation, it feels like the world

00:09:12 could be an inspiration to you. So it doesn’t have to be completely on paper. Like you could

00:09:19 be sitting at a bar and all of a sudden see something and a pattern will spark another

00:09:25 pattern and you can visualize it and rethink a problem in a particular way. Of course, you can

00:09:31 also load the math that you have on paper and always carry that with you. So when you show up to

00:09:35 the bar, some little inspiration could be the thing that changes it. Is there any other people

00:09:41 almost on the human side, whether it’s physics with Feynman,

00:09:45 Dirac, Einstein, or computer science, Turing, anybody else? Any backstories that you remember

00:09:51 that jump out? Because I’m also referring to not necessarily these stories where something magical

00:09:58 happens, but these are personalities. They have big egos. Some of them are super friendly. Some of

00:10:04 them are like self obsessed. Some of them have anger issues. Some of them, how do I describe

00:10:11 Feynman? But he appears to have a appreciation of the beautiful in all its forms. He has a wit

00:10:19 and a cleverness and a humor about him. So does that come into play in terms of the construction

00:10:24 of the science? Well, I think you brought up Newton. Newton is a good example also to think

00:10:29 about his backstory because there’s a certain backstory of Newton that people always talk about,

00:10:35 but then there’s a whole another aspect of him that is also a big part of the person that he was,

00:10:41 but he was really into alchemy and that he spent a lot of time thinking about that and writing

00:10:48 about it and he took it very seriously. He was really into Bible interpretation, trying to

00:10:53 predict things based on the Bible. And so there’s also a whole backstory then and of course you need

00:10:58 to look at it in the context and the time that when Newton lived, but it adds to his personality

00:11:06 and it’s important to also understand those aspects that maybe people are not as proud to

00:11:12 teach to little kids, but it’s important. It was part of who he was and maybe without those,

00:11:18 who knows what he would have done otherwise. Well, the cool thing about alchemy,

00:11:24 I don’t know how it was viewed at the time, but it almost like to me symbolizes

00:11:30 dreaming of the impossible. Like most of the breakthrough ideas kind of seem impossible

00:11:36 until they’re actually done. It’s like achieving human flight. It’s not completely obvious to me

00:11:40 that alchemy is impossible or putting myself in the mindset of the time. And perhaps even still,

00:11:49 everything that some of the most incredible breakthroughs would seem impossible.

00:11:56 And I wonder the value of believing, almost like focusing and dreaming of the impossible,

00:12:04 such that it actually is possible in your mind and that in itself manifests

00:12:09 whether the accomplishing that goal or making progress in some unexpected direction.

00:12:14 So alchemy almost symbolizes that for me. I distinctly remember having the same thought

00:12:19 of thinking, when I learned about atoms and that they have protons and electrons,

00:12:24 I was like, okay, to make gold, you just take whatever has an atomic weight below it and then

00:12:28 shove another proton in there and then you have a bunch of gold. So like, why don’t people do that?

00:12:33 It seemed like conceptually is like, this sounds feasible. You might be able to do it,

00:12:37 you might be able to do it. And you can actually, it’s just very, very expensive.

00:12:41 Yeah, exactly. So in a sense we do have alchemy and maybe even back then it wasn’t as crazy that

00:12:48 he was so into it, but people just don’t like to talk about that as much. But Newton in general

00:12:54 was a very interesting fellow. Anybody else come to mind? In terms of people that inspire you,

00:13:01 in terms of people that you just are happy that they have once or still exist on this earth?

00:13:08 I think, I mean, Freeman Dyson for me.

00:13:12 Yeah, Freeman Dyson was, I’ve had a chance to actually exchange a couple of emails with him.

00:13:17 It was probably one of the most humble scientists that I’ve ever met. And that had a big impact

00:13:23 on me. We were trying, we’re actually trying to convince him to annotate a paper on Fermat’s

00:13:29 library. And I sent him an email asking him if he could annotate a paper. And his response was

00:13:37 something like, I have very limited knowledge. I just know a couple of things about certain fields.

00:13:42 I’m not sure if I’m qualified to do that. That was his first response. And this was someone that

00:13:48 should have won an Obel prize and worked on a bunch of different fields, did some really,

00:13:53 really great work. And then just the interactions that I had with him, every time I asked him a

00:13:59 couple of questions about his papers, and he always responded saying, I’m not here to answer

00:14:05 your questions. I just want to open more questions. And so that had a big impact on me. It was like

00:14:13 just an example of an extremely humble, yet accomplished scientist. And Feynman was also

00:14:21 a big, big inspiration in the sense that he was able to be, again, extremely talented and scientist,

00:14:29 but at the same time, socially, he was able to, he was also really smart from a social perspective.

00:14:35 And he was able to interact with people. He was also a really good teacher and was also to did

00:14:43 awesome work in terms of explaining physics to the masses and motivating and getting people

00:14:49 interested in physics. And that for me was, was also a big inspiration.

00:14:54 Yeah. I liked the childlike curiosity of some of those folks that you mentioned,

00:14:57 Feynman. I’ve, Daniel Kahneman, I got a chance to meet and interact with some of these truly

00:15:03 special scientists. What makes them special is that even in older age, there’s still like,

00:15:10 there’s still that fire of childlike curiosity that burns. And some of that is like not taking

00:15:17 yourself so seriously that you think you’ve figured it all out, but almost like thinking that

00:15:21 you don’t know much of it. And that’s like step one in having a great conversation or collaboration

00:15:31 or exploring a scientific question. And it’s cool how the very thing that probably earned people

00:15:37 the Nobel prize or, or work that’s seminal in some way is the very thing that still burns even after

00:15:45 they’ve won the prize. It’s cool to see. And they’re rare humans, it seems.

00:15:50 And to that point, I remember like the last email that I sent to Freeman Dyson was like in his last

00:15:54 birthday, he was really into number theory and primes. So what I did is I took like a photo of

00:16:00 him picture, and then I turned that into like a giant prime number. So I converted the picture

00:16:07 into a bunch of one and eights. And then I moved some numbers around until it was a prime. And

00:16:13 then I sent him that. Oh, so the, the visual, like it’s still look like the picture, it’s made up of

00:16:18 a prime. That’s tricky to do. It’s hard to do. It looks harder than it actually is. So the way you

00:16:24 do it is like you convert the darker regions into eights and the lighter regions in ones.

00:16:30 And then there’s… And then just keep flipping numbers until… But there’s like some primality

00:16:35 tests that are cheaper from a computational standpoint. But what it gives you is like

00:16:41 it excludes numbers that are not prime. Then you end up with a set of numbers that you don’t know

00:16:46 if they are prime or not. And then you run the full primality test on that. So you just have to

00:16:50 keep iterating on that. And it was, it’s funny because when you got the picture, it was like,

00:16:55 how did you do that? It was super curious too. And then we got into the details. And again,

00:17:00 this was, it was already 90, I think 92 or something. And that curiosity was still there.

00:17:06 So you can really see that in some of these scientists.

00:17:10 So could we talk about Fermat’s library?

00:17:12 Yeah, absolutely.

00:17:13 What is it? What’s the main goal? What’s the dream?

00:17:18 It is a platform for annotating papers in its essence, right? And so academic papers can be

00:17:24 one of the densest forms of content out there and generally pretty hard to understand at times. And

00:17:31 the idea is that you can make them more accessible and easier to understand by adding these rich

00:17:36 annotations to the site, right? And so we can just imagine a PDF view on your browser, and then you

00:17:41 have annotations on each side. And then when you click on them, a sidebar expands and then you have

00:17:46 annotations that support LaTeX and Markdown. And so the idea is that you can, say, explain a

00:17:51 tougher part of a paper where there’s a step that is not completely obvious. And then you can

00:17:56 add more context to it. And then over time, papers can become easier and easier to understand and can

00:18:04 evolve in a way. But it really came from myself, Luis, and two other friends. We’ve had this

00:18:11 long running habit of kind of running a journal club amongst us. We come from different backgrounds,

00:18:17 right? I studied CS, we studied physics. And so we’d read papers and present them to the

00:18:21 each other. And then we tried to bring some of that online. And that’s when we decided to

00:18:30 build Fermat’s library. And then over time, it kind of grew into something with a broader goal.

00:18:40 And really, what we’re trying to do is trying to help move science in the right direction.

00:18:46 That’s really the ultimate goal and where we want to take it now.

00:18:52 Well, so there’s a lot to be said. So first of all, for people who haven’t seen it,

00:18:58 the interface is exceptionally well done. Execution is really important here.

00:19:02 Absolutely.

00:19:03 The other thing is just to mention for a large number of people, apparently, which is new to me,

00:19:10 don’t know what LaTeX is. So it’s spelled like LaTeX. So be careful googling it if you haven’t

00:19:16 before. It’s a, sorry, I don’t even know the correct terminology.

00:19:21 Type setting language?

00:19:22 It’s a type setting language where you’re basically program, writing a program that

00:19:28 then generates something that looks, from a typography perspective, beautiful.

00:19:34 Absolutely.

00:19:34 And so a lot of academics use it to write papers. I think there’s like a bunch of communities that

00:19:41 use it to write papers. I would say it’s mathematics, physics, computer science.

00:19:47 Yeah.

00:19:47 That’s, yeah, that’s the main.

00:19:49 Because I’m collaborating currently on a paper with two neuroscientists from Stanford.

00:19:54 And they don’t know LaTeX.

00:19:55 So I’m using Microsoft Word and Mendeley, and like all of those kinds of things. And I’m being

00:20:04 very zen like about the whole process, but it’s fascinating. It’s a little heartbreaking actually,

00:20:11 because it actually, it’s funny to say, but, and we’ll talk about open science, actually,

00:20:18 the bigger mission behind Fermat’s library is like,

00:20:20 really opening up the world of science to everybody. Is these silly two facts of like

00:20:28 one community uses LaTeX and another uses Word, is actually a barrier between them.

00:20:35 That’s like, it’s like boring and practical in a sense, but it makes it very difficult to collaborate.

00:20:41 Just on that, like, I think there are some people that should have received like a Nobel

00:20:46 Prize, but will never get it. And I think one of those is like Donald Knuth, because of tech

00:20:51 and LaTeX. And then, because it had a huge impact in terms of like just making it easier for

00:20:57 researchers to put their content out there, like making it uniform as much as possible.

00:21:02 Oh, you mean like a Nobel Peace Prize?

00:21:05 Maybe a Nobel Peace Prize. Maybe a Nobel Peace Prize. Yeah, yeah, yeah. I don’t know.

00:21:12 Maybe a Nobel Peace Prize. Yeah, yeah, yeah. I think so.

00:21:17 I mean, he at a very young age, got the Turing Award for his work in algorithms and so on. So

00:21:21 like an incredibly, I think it’s in, it might be even the sixties, but I think it’s the seventies.

00:21:27 So when he was really young and then he went on to do like incredible work with his book

00:21:33 and yeah, with tech that people don’t know.

00:21:36 And going back just on the reason why we ended up, because I think this is interesting. The reason

00:21:41 why we ended up using the name Fermat’s Library, this was because of Fermat’s Last Theorem. And

00:21:47 Fermat’s Last Theorem is actually a funny story. So Pierre de Fermat, he was like a lawyer and he

00:21:54 wrote like on a book that he had a solution to Fermat’s Last Theorem, which, but that didn’t

00:22:01 fit the margin of that book. And so Fermat’s Last Theorem basically states that there’s no solution.

00:22:08 If you have integers a, b and c, there’s no solution to a to the power of n plus b to the

00:22:14 power of n equals to c to the power of n, if n is bigger than two. So there’s no solutions.

00:22:21 And he said that, and that problem remained open for almost 300 years, I believe. And a lot of the

00:22:30 most famous mathematicians tried to tackle that problem. No one was able to figure that out until

00:22:35 Andrew Wiles, I think was in the 90s, was able to publish the solution, which was, I believe, almost

00:22:42 300 pages long. And so it’s kind of an anecdote that, you know, there’s a lot of knowledge and

00:22:50 insights that can be trapped in the margins. And there’s a lot of potential energy that you can

00:22:55 release if you actually spend some time trying to digest that. And that was the origin story for

00:23:03 for the name. Yes, you can share the contents of the margins with the world that could inspire

00:23:09 a solution or a communication that then leads to a solution. And if you think about papers,

00:23:15 like papers are, as Joan was saying, probably one of the densest pieces of text that any human can

00:23:21 read. And you have these researchers, like some of the brightest minds in these fields, working on

00:23:27 like new discoveries and publishing these work on journals that are imposing them restrictions in

00:23:32 terms of the number of pages that they can have to explain a new scientific breakthrough. So at the

00:23:38 end of the day, papers are not optimized for clarity and for a proper explanation of that

00:23:45 content because there are so many restrictions. So there’s, as I mentioned, there’s a lot of

00:23:50 potential energy that can be freed if you actually try to digest a lot of the contents of papers.

00:23:57 Trey Lockerbie Can you explain some of the other things? So margins, librarian, journal club?

00:24:02 Ofer Yusuf So journal club is what a lot of people know us for, where we every week,

00:24:06 we release an annotated paper and in all sorts of different fields, but physics, CS, math,

00:24:12 margins is kind of the same software that we use to run the journal club and to

00:24:16 host the annotations. But we’ve made that available for free to anybody that wants to use it. And so

00:24:22 folks use it at universities and for running journal clubs. And so we’ve just made that freely

00:24:29 available. And then librarian is a browser extension that we developed that is sort of an

00:24:34 overlay on top of archive. So it’s about bringing some of the same functionality around comments,

00:24:40 plus adding some extra niceties to archive, like being able to very easily extract the references

00:24:48 of a paper that you’re looking at or being able to extract the bib deck in order to cite that

00:24:52 paper yourself. So it’s an overlay on top of archive. The idea is that you can have that

00:24:57 commenting interface without having to leave archive. Trey Lockerbie It’s kind of incredible.

00:25:01 I didn’t know about it. And once I learned of it, it’s like, holy shit. Why isn’t it more popular,

00:25:10 given how popular archive is? Like everybody should be using it. Archive sucks in terms of

00:25:14 its interface. Let me rephrase that. It’s limited in terms of its interface.

00:25:19 Archive is a pretty incredible project. And it is, in a way, the growth has been completely

00:25:29 linear over time. If you look at the number of papers published on archive, it’s pretty much

00:25:34 a straight line for the past 20 years. Especially if you’re coming from a startup background and

00:25:40 then you were trying to do archive, you’d probably try all sorts of growth acts and try to then maybe

00:25:47 have paid features and things like that. And that would kind of maybe ruin it. And so there’s a

00:25:53 subtle balance there. And I don’t know what aspects you can change about it.

00:25:57 For some tools in science, it just takes time for them to grow. Archive has just turned 30,

00:26:04 I believe. And for people that don’t know, archive is these kind of online repositories

00:26:09 where people put preprints, which are versions of the papers, before they actually make it to journals.

00:26:15 A R X I V, for people who don’t know. And it’s actually a really vibrant place to publish your

00:26:23 papers in the aforementioned communities of mathematics, physics, and computer science.

00:26:29 It started with mathematics and physics, and then over the last 30 years, it evolved. And now,

00:26:34 actually computer science now, it’s a more popular category than physics and math on archive.

00:26:40 And there’s also, which I don’t know very much about, like a biology, medical version of that.

00:26:46 Bioarchive. Yeah, bioarchive. It’s interesting because if you look at these platforms for

00:26:54 preprints, they actually play a super important role. Because if you look at a category like math,

00:27:01 for some papers in math, it might take close to three years after you click upload paper on the

00:27:08 journal website, and the paper gets published on the website of the journal. So this is literally

00:27:14 the longest upload period on the internet. And during those three years, their content is just

00:27:24 locked. And so that’s why it’s so important for people to have websites that are open to

00:27:29 people to have websites like archive so that you can share that before it goes to the journal

00:27:35 with the rest of the world. There was actually on archive that Perelman published the three

00:27:40 papers that led to the proof of the Poincare conjecture. And then you have other fields like

00:27:46 machine learning, for instance, where the field is evolving at such a high rate that people don’t

00:27:52 even wait before the papers go to journals before they start working on top of those papers. So they

00:27:57 publish them on archive, then other people see them, they start working on that. And archive did

00:28:02 a really good job at building that core platform to host papers. But I think there’s a really,

00:28:07 really big opportunity in building more features on top of that platform, apart from just hosting

00:28:13 papers. So collaboration, annotations, and having other things apart from papers like code

00:28:20 and other things. Because, for instance, in the field like machine learning,

00:28:24 there’s a really big, as I mentioned, people start working on top of preprints, and they are

00:28:29 assuming that preprint is correct. But you really need a way, for instance, to maybe, it’s not peer

00:28:38 review, but distinguish what is good work from bad work on archive. How do you do that? So like

00:28:44 a commenting interface like librarian, it’s useful for that so that you can distinguish that in a

00:28:50 field that is growing so fast as machine learning. And then you have platforms that focus, for

00:28:56 instance, on just biology. BioArchive is a good example. BioArchive is also super interesting

00:29:02 because there’s actually an interesting experiment that was run in the 60s. So in the 60s, the NIH

00:29:12 supported this experiment called the Information Exchange Group, which at the time was a way for

00:29:19 researchers to share biology preprints via mail or using libraries. And that project in the 1960s

00:29:27 got canceled six years after it started. And it was due to intense pressure from the journals

00:29:32 to kill that project because they were fearing competition from the preprints for the journal

00:29:41 industry. Crick was one of the famous scientists that opposed to the Information Exchange Group.

00:29:50 And it’s interesting because right now, if you analyze the number of biology papers that appear

00:29:55 first as preprints, it’s only 2% of the papers. And this was almost 50 years after that first

00:30:03 experiment. So you can see that pressure from the journals to cancel that initial version of a

00:30:08 preprint repo had a tremendous impact on the number of papers that are showing up in biology

00:30:15 as preprints. So it delayed a lot that revolution. But now platforms like BioArchive are doing that

00:30:24 work. But there’s still a lot of room for growth there. And I think it’s super important because

00:30:28 those are the papers that are open that everyone can read. Okay. But if we just look at the entire

00:30:33 process of science as a big system, can we just talk about how it can be revolutionized? So you

00:30:40 have an idea, depending on the field, you want to make that idea concrete, you want to run a few

00:30:46 experiments in computer science, there might be some code, there’d be a data set for, you know,

00:30:53 some of the more sort of biology, psychology, you might be collecting the data set that’s called,

00:31:02 you know, a study, right? So that’s part of that, that’s part of the methodology.

00:31:08 And so you are putting all that into a paper form. And then you have some results. And then you

00:31:16 submit that to a place for review through the peer review process. And there’s a process where,

00:31:23 how would you summarize the peer review process? But it’s really just like a handful of people look

00:31:28 over your paper and comment. And based on that, decide whether your paper is good or not. So

00:31:34 there’s a whole broken nature to it. At the same time, I love the peer review process when I buy

00:31:40 stuff on Amazon, like for like the commenting system, whatever that is. So, okay. So there’s

00:31:48 a bunch of possibilities for revolutions there. And then there’s the other side, which is the

00:31:53 collaborative aspect of the science, which is people annotating, people commenting, sort of the

00:31:58 low effort collaboration, which is a comment. Sometimes, as you’ve talked about, a comment can

00:32:04 change everything. But, you know, or a higher effort collaboration, like more like maybe

00:32:10 annotations, or even like contributing to the paper, you can think of like, collaborative

00:32:17 updating of the paper over time. So there’s all these possibilities for doing things better than

00:32:24 they’ve been done. Can we talk about some ideas in this space? Some ideas that you’re working on,

00:32:31 some ideas that you’re not yet working on, but should be revolutionized? Because it does seem

00:32:37 that archive and like open review, for example, are like the craigslist of science. Like,

00:32:45 like, yeah, okay. I’m very grateful that we have it. But it just feels like, it’s like 10 to 20

00:32:53 years. Like, it doesn’t feel like that’s a feature. The simplicity of it is a feature. It feels like

00:32:58 it’s a, it’s a bug. But then again, the pushback there is Wikipedia has the same kind of simplicity

00:33:07 to it. And it seems to work exceptionally well in the crowdsourcing aspect of it. So I’m sorry,

00:33:14 there’s a bunch of stuff going on on the table. Let’s just pick random things that we can talk

00:33:19 about. Wikipedia, you know, for me, it’s the cosmological constant of the internet. It’s,

00:33:24 I think we are lucky to live in the parallel universe where Wikipedia exists. Yes. Because

00:33:29 if someone had pitched me Wikipedia, like a publicly edited encyclopedia, like a couple of

00:33:35 years ago, like it would be, I don’t know how many people would have said that that would have

00:33:40 survived. I mean, it makes almost no sense. It’s like having a Google doc that everybody on the

00:33:45 internet can edit. And like, that will be like the most reliable source for knowledge. And I don’t

00:33:51 know how many, but hundreds of thousands of topics. Yeah. It’s insane. It’s insane. And like you have,

00:33:58 and then you have users, like there’s one single user that edited one third of the articles on

00:34:03 Wikipedia. So we have these really, really big power users. There are a substantial part of like

00:34:10 what makes Wikipedia successful. And so like, no one would have ever imagined that that could

00:34:18 happen. And so that’s one thing. I completely agree with what you just said. I also…

00:34:24 Sorry to interrupt briefly. Maybe let’s inject that into the discussion of everything else.

00:34:29 I also believe, I’ve seen that with Stack Overflow, that one individual or a small collection of

00:34:35 individuals contribute or revolutionized most of the community. Like if you create a really

00:34:42 powerful system for archive or like open review and made it really easy and compelling and exciting

00:34:50 for one person who is like a 10x contributor to do their thing, that’s going to change everything.

00:34:57 It seems like that was the mechanism that changed everything for Wikipedia. And that’s the mechanism

00:35:02 that changed everything for Stack Overflow. Is gamifying or making it exciting or just making

00:35:07 it fun or pleasant or fulfilling in some way for those people who are insane enough to like answer

00:35:15 thousands of questions or write thousands of factoids and like research them and check them,

00:35:21 all those kinds of things, or read thousands of papers.

00:35:25 Yeah. No, Stack Overflow is another great example of that. And it’s just, and those are both to

00:35:31 do incredibly productive communities that generate a ton of value and capture almost none of it.

00:35:38 Right. And it’s in a way, it’s almost like, it’s very counterintuitive that these communities would

00:35:49 exist and thrive. And it’s really hard to, there aren’t that many communities like that.

00:35:57 Right. So how do we do that for science? Do you have ideas there?

00:36:01 Like what are the biggest problems that you see? You’re working on some of them.

00:36:05 Like just on that, there are a couple of really interesting experiments that people are running.

00:36:09 An example would be like the Polymath projects. So this is a kind of a social experiment that was

00:36:16 created by Tim Gowers, Fields Medalist. And his idea was to try to prove that,

00:36:22 is it possible to do mathematics in a massively collaborative way on the internet? So he decided

00:36:29 to pick a couple of problems and test that. And they found out that it actually is possible for

00:36:36 specific types of problems, namely problems that you’re able to break down in little pieces and

00:36:42 go step by step. You might need, as with open source, you might need people that are just kind

00:36:48 of reorganizing the house every once in a while. And then people throw a bunch of ideas and then

00:36:55 you make some progress, then you reorganize, you reframe the problem and you go step by step.

00:36:59 But they were actually able to prove that it is possible to collaborate online and do progress

00:37:07 in terms of mathematics. And so I’m confident that there are other avenues that could be

00:37:13 explored here. Can we talk about peer review, for example?

00:37:16 Absolutely. I think in terms of the peer review, I think it’s important to look at the bigger

00:37:23 picture here of what the scientific publishing ecosystem looks like. Because for me, there are

00:37:32 a lot of things that are wrong about that entire process. So if you look at what publishing means

00:37:39 in a traditional journal, you have journals that pay authors for their articles, and then they might

00:37:48 pay reviewers to review those articles. And finally, they pay people or distributors to

00:37:56 distribute the content. In the scientific publishing world, you have scientists that are

00:38:02 usually backed by government grants. They are giving away their work for free in the form of

00:38:07 papers. And then you have other scientists that are reviewing their work. This process is known

00:38:13 as the peer review process, again for free. And then finally, we have government backed

00:38:21 universities and libraries that are buying back all that work so that other scientists can read.

00:38:29 So this is, for me, it’s bizarre. You have the government that is funding the research,

00:38:33 it’s paying the salaries of the scientists, it’s paying the salaries of the reviewers,

00:38:37 and it’s buying back all that product of their work again. And I think the problem with this

00:38:43 system and it’s why it’s so difficult to break this suboptimal equilibrium is because of the way

00:38:52 academia works right now and the way you can progress in your academic life. And so in a lot

00:38:59 of fields, the competition in academia is really insane. So you have hundreds of PhD students,

00:39:06 they are trying to get to a professor position and it’s hyper competitive. And the only way for

00:39:15 you to get there is if you publish papers, ideally in journals with a high impact factor.

00:39:23 In computer science, it’s often conferences are also very prestigious or actually more

00:39:28 prestigious than journals now. So that’s the one discipline where, I mean, that has to do

00:39:33 with the thing we’ve discussed in terms of how quickly the field turns around. But like NeurIPS,

00:39:40 CVPR, those conferences are more prestigious, or at the very least as prestigious as the journals.

00:39:47 But it doesn’t matter. The process is what it is.

00:39:51 So for people that don’t know, the impact factor of a journal is basically the average number of

00:39:56 citations that a paper would get if it gets published on that journal. But so you can really

00:40:02 think that the problem with the impact factor is that it’s a way to turn papers into accounting

00:40:11 units. And let me unpack this because the impact factor is almost like a nobility title. Because

00:40:19 papers are born with impact even before anyone reads them. So the researchers, they don’t have

00:40:25 the incentive to care about if this paper is going to have a long term impact on the world.

00:40:32 What they care, their goal, their end goal is the paper to get published so that they get that

00:40:37 value upfront. So for me, that is one of the problems of that. And that really creates a

00:40:43 tyranny of metrics. Because at the end of the day, if you are a dean, what you want to hire is

00:40:49 people, researchers that publish papers on journals with high impact factors because that will

00:40:55 increase the ranking of your university and will allow you to charge more for tuition,

00:40:59 so on and so forth. And that, especially when you are in super competitive areas,

00:41:07 that people will try to gamify that system and misconduct starts showing up. There’s a really

00:41:14 interesting book on this topic called Gaming the Metrics. It’s a book by a researcher called Mario

00:41:21 Biagioli. It goes a lot into how the impact factor and metrics affect science negatively.

00:41:29 And it’s interesting to think, especially in terms of citations, if you look at the early work of

00:41:34 looking at citations, there was a lot of work that was done by a guy called Eugene Garfield.

00:41:39 And this guy, the early work in terms of citation, they wanted to use citations from a descriptive

00:41:46 point of view. So what they wanted to create was a map. And that map would create a visual

00:41:52 representation of influence. So citations would be links between papers. And ideally, what they

00:42:00 would show, they would represent is that you read someone else’s paper and it had an impact on your

00:42:05 research. They weren’t supposed to be counted. I think this inspired Larry and Sergey’s work for

00:42:12 Google. Exactly. I think they even mentioned that. But what happens is, as you start counting

00:42:16 citations, you create a market. And the work of Eugene Garfield was a big inspiration for Larry

00:42:25 and Sergey and for the PageRank algorithm that led to the creation of Google. And they even

00:42:31 recognized that. And if you think about it, it’s like the same way there’s a gigantic market for

00:42:37 search engine optimization, SEO, where people try to optimize the PageRank and how a web page will

00:42:46 rank on Google. The same will happen for papers. People will try to optimize the impact factors

00:42:52 and the citations that they get. And that creates a really big problem. And it’s super interesting

00:42:58 to actually analyze them. If you look at the distribution of the impact factors of journals,

00:43:05 you have like Nature. Nature, I believe, is in the low 40s. And then you have, I believe, Science

00:43:11 is high 30s. And then you have a really good set of good journals that will fall between 10 and 30.

00:43:20 And then you have a gigantic tale of journals that have impact factor below 2. And you can really

00:43:26 see two economies here. You see the universities that are maybe less prestigious, less known,

00:43:35 where the faculty are pressured to just publish papers regardless of the journal. What I want to

00:43:41 do is increase the ranking of my university. And so they end up publishing as many papers as they

00:43:47 can in journals with low impact factor. And unfortunately, this represents a lot of the

00:43:53 global south. And then you have the luxury good economy. And there are also problems here in

00:44:02 the luxury good economy. So if you look at the journal like Nature, so with impact factor in

00:44:07 the low 40s, there’s no way that you’re going to be able to sustain that level of impact factor

00:44:14 by just grabbing the attention of scientists. What I mean by that is for the journals,

00:44:22 the articles that get published in Nature, they need to be New York Times grade. So they need to

00:44:29 make it to the big media. They need to be captured by the big media. And because that’s the only way

00:44:34 for you to capture enough attention to sustain that level of citations. And that, of course,

00:44:40 creates problems because people then will try to, again, gamify the system and have like titles or

00:44:47 abstracts or that are bigger, make claims that are bigger than what is actually can be, you know,

00:44:55 sustained by the data or the content of the paper. And you’ll have clickbait titles or clickbait

00:45:01 abstracts. And again, this is all a consequence of science or metrics. And this is a very dangerous

00:45:10 cycle that I think it’s very hard to break, but it’s happening in academia in a lot of fields

00:45:16 right now. Is it fundamentally the existence of metrics or the metrics just need to be

00:45:21 significantly improved? Because like I said, the metrics used for Amazon for purchasing,

00:45:29 I don’t know, computer parts is pretty damn good in terms of selecting which are the good ones,

00:45:34 which are not. In that same way, if we had Amazon type of review system in the space of ideas,

00:45:42 in the space of science, it feels like that those metrics would be a little bit better.

00:45:47 Sort of when it’s significantly more open to the crowdsource nature of the internet,

00:45:55 of the scientific internet, meaning as opposed to, like my biggest problem with peer review

00:46:02 has always been that it’s like five, six, seven people, usually even less. And it’s often,

00:46:09 nobody’s incentivized to do a good job in the whole process. Meaning it’s anonymous in a way

00:46:17 that doesn’t incentivize, like doesn’t gamify or incentivize great work. And also it doesn’t

00:46:26 necessarily have to be anonymous. Like there has to be, the entire system is, doesn’t encourage

00:46:36 actual sort of rigorous review. For example, like open review does kind of incentivize that kind of

00:46:45 process of collaborative review, but it’s also imperfect. But it just feels like the thing that

00:46:50 Amazon has, which is like thousands of people contributing their reviews to a product,

00:46:58 it feels like that could be applied to science where the same kind of thing you’re doing with

00:47:04 Fermat’s library, but doing at a scale that’s much larger. It feels like that should be possible

00:47:11 given the number of grad students, given the number of general public that’s getting,

00:47:17 for example, I personally, as a person who got an education in mathematics and computer science,

00:47:24 like I can be a quote unquote, like reviewer on a lot bigger set of things than is my exact

00:47:36 expertise. If I’m one of thousands of reviewers, if I’m the only reviewer, one of five,

00:47:43 then I better be like an expert in the thing. But if I, and I’ve learned this with COVID,

00:47:48 which is like, you can just use your basic skills as a data analyst to contribute to

00:47:55 the review process and a particular little aspect of a paper and be able to comment, be able to

00:48:00 sort of draw in some references that challenge the ideas presented or to enrich the ideas that

00:48:07 are presented. It just feels like crowdsourcing the review process would be able to allow you to have

00:48:16 metrics in terms of how good a paper is that are much better representative of its actual impact

00:48:22 in the world, of its actual value to the world, as opposed to some kind of arbitrary, gamified

00:48:30 version of its impact. I agree with that. I think there’s definitely the possibility,

00:48:36 at least for a more resilient system than what we have today. And I think that’s kind of what

00:48:41 you’re describing, Alex. I mean, to an extent, we kind of have like a little bit of a Heisenberg

00:48:48 uncertainty principle. When you pick a metric, as soon as you do it, then maybe it works as a good

00:48:52 heuristic for a short amount of time, but soon enough, people would start gamifying. But then

00:48:58 you can definitely have metrics that are more resilient to gamification and they’ll work as a

00:49:03 better heuristic to try to push you in the best direction. But I guess down the line,

00:49:10 the underlying problem you’re saying is there’s a shortage of positions in academia. That’s a big

00:49:15 problem for me. Yeah. And that, and so they’re going to be constantly gamifying the metrics.

00:49:21 It’s a bit of a zero sum game. It’s a very competitive field. And that’s what usually

00:49:26 happens in very competitive fields. Yeah. Yeah. But I think some of the peer review problems,

00:49:33 like scale helps, I think. And it’s interesting to look at what you’re mentioning, breaking it

00:49:37 down, maybe in like smaller parts and having more people jumping in. But this is definitely

00:49:45 a problem. And the peer review problem, as I mentioned, is correlated with the problem of

00:49:50 academic career progression. And it’s all intertwined. And that’s why I think it’s so hard

00:49:55 to break it. There are a couple of really interesting things that are being done right

00:50:01 now. There are a couple of, for instance, journals that are overlay journals on top of

00:50:06 platforms like archive and bio archive that want to remove like the more traditional journals from

00:50:12 the equation. So essentially a journal is just a collection of links to papers. And what they’re

00:50:19 trying to do is like removing that middleman and trying to make the review process a little bit

00:50:24 more transparent and not charging universities. There are a couple of more famous ones. There’s

00:50:33 one discrete analysis in mathematics. There’s one called the quantum journal, which we’re actually

00:50:39 working with them. We have a partnership with them for the papers that get published in quantum

00:50:43 journal. They also get the annotations on formats and they’re doing pretty well. They’ve been able

00:50:48 to grow substantially. The problem there is getting to critical mass. So it’s again, convincing the

00:50:53 researchers and especially the young researchers that need that impact factor, need those

00:50:59 publications to have citations to not publish on the traditional journal and go on an open journal

00:51:05 and publish their work there. There, I think there are a couple of really high profile scientists of

00:51:11 people like Tim Gowers. They are trying to incentivize like famous scientists that already

00:51:16 have tenure and that don’t need that to publish that to increase the reputation of those journals.

00:51:22 So that other maybe younger scientists can start publishing on those as well. And so that you can

00:51:27 try to break that vicious cycle of the more traditional journals.

00:51:32 I mean, another possible way to break this cycle is to like raise public awareness and just by

00:51:39 force, like ban paid journals. Like what exactly are they contributing to the world? Like basically

00:51:47 making it illegal to forget the fact that it’s mostly federally funded. So that’s that’s

00:51:55 a super ugly picture too. But like, why should knowledge be so expensive? Like where everyone

00:52:03 is working for the public good. And then there’s these gatekeepers that, you know, most people

00:52:10 can’t read most papers without having to pay money. And that’s, that doesn’t make any sense.

00:52:17 That’s like that, that should be illegal.

00:52:20 I mean, that’s what you’re saying is exactly right. I mean, for instance,

00:52:23 right, I went to school here in the US, we studied in Europe, and you would sit like you’d

00:52:28 ask me all the time to download papers and send it to him because he just couldn’t get it. And like

00:52:32 papers that he needed for his research. And so,

00:52:35 But he’s a student, like he’s a grad student.

00:52:37 He was a grad student. But that, you know, I’m even referring to just regular people.

00:52:42 Oh, yeah. Okay. That too. Yeah.

00:52:44 And I think during 2020, because of COVID, a lot of journals put down the

00:52:50 walls for certain kind of coronavirus related papers. But like, that just gave me an indication

00:52:56 that like, this should be done for everything. It’s absurd. Like people should be outraged

00:53:03 that there’s these gates. Because, so the moment you dissolve the journals,

00:53:09 then there will be an opportunity for startups to build stuff on top of archive. It’d be an

00:53:16 opportunity for like, for my library to step up to scale up to something much even larger.

00:53:22 I mean, that was the original dream of Google, which I’ve always admired, which is make the

00:53:28 world’s information accessible. Actually, it’s interesting that Google hasn’t, maybe you guys

00:53:33 can correct me, but they put together Google Scholar, which is incredible. But they, and they’ve

00:53:39 did the scanning of books, but they haven’t really tried to make science accessible in the following

00:53:47 way. Like, besides doing Google Scholar, they haven’t like delved into the papers, right?

00:53:54 Which is especially curious given what Luis was saying, right? That it’s kind of in their genesis.

00:53:59 There’s this, you know, research that was very connected with how papers reference each other

00:54:04 and like building a network out of that. Interestingly enough, like Google, I think there was a,

00:54:09 there was not intended, Google Plus was like the Google social network that got cancelled, was used

00:54:14 by a lot of researchers. Yes, it was. Which I think was just a, you know, side, kind of a side effect.

00:54:19 But then a lot of people ended up migrating to Twitter, but it was not on purpose. But yeah,

00:54:23 I agree with you. Like they haven’t gone past Google Scholar and I don’t know why. Well, that said,

00:54:29 Google Scholar is incredible. For people who are not familiar, it’s one of the best aggregation of

00:54:35 all the scientific work that’s out there and especially the network that connects to all of

00:54:39 them. What sites, what. And also trying to aggregate all of the versions of the papers that are

00:54:45 available there and trying to merge them in a way that one particular work, even though it’s available

00:54:50 in a bunch of places, counts as, you know, like a central hub of what that work is, according to

00:54:56 multiple versions. But that almost seems like a fun pet project of a couple of engineers within

00:55:03 within Google, as opposed to a serious effort to make the world science accessible. But going back

00:55:09 to just the journals, when you’re talking about that, Lex, I believe that in that front, I think

00:55:16 we might be past the event horizon. So I think the model, the business model for the journals,

00:55:23 you know, doesn’t make sense. They’re a middle layer that is not adding a lot of value. And you

00:55:27 see a lot of motions, whereas like in Europe, a lot of the papers that are funded by the European

00:55:35 Union, they will have to be open to the public. And I think there’s a lot of… Bill Gates too,

00:55:41 like what the Gates Foundation funds, like the demand that it’s accessible to everybody.

00:55:49 So I think it’s a question of time before that wall kind of falls. And that is going to open a

00:55:55 lot of possibilities. Because, you know, imagine if you had like the layer of that gigantic layer

00:56:02 of papers all available online, you know, that unlocks a lot of potential as a platform for

00:56:08 people to build things on top of that. But to what you’re saying, it is weird, like you can literally

00:56:13 go and listen to any song that was ever made on your phone, right? You open Spotify and you might

00:56:20 not even pay for it. You might be on the free version and you can listen to any song that was

00:56:24 ever made, pretty much. But there’s like, you don’t have access to a huge percentage of academic

00:56:33 papers, which is just like this fundamental knowledge that we’re all funding. But you as an

00:56:37 individual don’t have access to it. And some of it, you know, you don’t have access to, you know,

00:56:42 don’t have access to it. And somehow, you know, like the problem for music got solved. But for

00:56:48 papers, it’s still like… It’s just not yet. It could be ad supported, all those kinds of things.

00:56:53 And then hopefully, that would change the way we do science. This is the most exciting thing for me

00:56:58 is, especially once I started like making videos and this silly podcast thing, I started to realize

00:57:04 like that if you want to do science, one of the most effective ways is to do like couple the

00:57:13 paper with a set of YouTube videos, like explaining it. That also seems like there’s a lot of room for

00:57:21 disruption there. What is the paper 2.0 going to look like? I think like LaTeX and the PDF,

00:57:28 seems like if you… It’s interesting. If you look at the first paper that got published in Nature,

00:57:33 and if you look at the paper that got published in Nature today, if you look at the two side by

00:57:37 side, they are fundamentally the same. And even though like the paper that gets published today,

00:57:42 you know, you get… Even code, like right now, people put like code, like on a PDF.

00:57:51 And there are so many things that are related to papers today, you know, you have data,

00:57:56 you have code, you might need videos to better explain the concepts. So, I think for me,

00:58:03 it’s natural that there’s going to be also an evolution there, that papers are not going to be

00:58:07 just the static PDFs or LaTeX, there’s going to be a next interface. So, in academia, a lot of

00:58:15 things that are judged, you’re judged by is often quantity, not quality. I wonder if there’s an

00:58:22 opportunity to have like… I tend to judge people by the best work they’ve ever done as opposed to…

00:58:28 I wonder if there’s a possibility for that to encourage sort of focusing on the quality,

00:58:34 and not necessarily in paper form, but maybe a subset of a paper, subset of idea, almost even a

00:58:39 blog post or an experiment. Like, why does it have to be published in a journal to be legitimate?

00:58:46 And it’s interesting that you mentioned that I also think like, yeah, it’s why is that the only

00:58:53 format? Why can’t a blog post or… We were even experimenting with these a few months ago,

00:59:01 or can you actually like publish something or like a new scientific breakthrough or

00:59:10 something that you’ve discovered in the form of like a set of tweets,

00:59:13 a Twitter thread. Why can’t that be possible? And we were experimenting with that idea.

00:59:20 We even, yeah, we ran a couple of… Like some people submitted a couple of those,

00:59:26 like I think the limit was three or four tweets. Maybe it’s a new way to look at a proof or

00:59:31 something, but I think it just serves to show that there should be other ways to publish

00:59:37 like scientific discoveries that don’t fit the paper format. Well, but so even with the Twitter

00:59:43 thread, it would be nice to have some mechanism of formalizing it and making it into an NFT.

00:59:52 Like a concrete thing that you can reference as a link that’s unique. Because, I mean, everything

01:00:00 we’ve been saying, all of that, while being true, it’s also true that the constraints and the

01:00:11 formalism of a paper works well. It like forces you, constraints forces you to narrow down your

01:00:18 thing and literally put it on paper. But, you know, make concrete. And that’s why, I mean,

01:00:29 it’s not broken. It just could be better. And that’s the main idea. I think there’s something

01:00:34 about writing, whether it’s a blog post or Twitter thread or a paper, that’s really nice to

01:00:42 concretize a particular little idea that can then be referenced by other ideas, then it can be built

01:00:51 on top of with other ideas. So let me ask, you’ve read quite a few papers. You’ve annotated quite a

01:01:01 few papers. Let’s talk about the process itself. How do you advise people read papers? Or maybe

01:01:08 you want to broaden it beyond just papers, but just read concrete pieces of information to

01:01:14 understand the insights that labeled in. I would say for paper specifically, I would bring back

01:01:20 kind of what Louise was talking about, is that it’s important to keep in mind that papers are

01:01:24 not optimized for ease of understanding. And so, right, there’s all sorts of restrictions and size

01:01:32 and format and language that they can use. And so it’s important to keep that in mind. And so that

01:01:38 if you’re struggling to read a paper, that might not mean that the underlying material is actually

01:01:45 that hard. And so that’s definitely something that, especially for us, that we read papers and

01:01:53 most of the time the papers are completely outside of our comfort zone, I guess. And so it would be

01:01:59 completely new areas to us. So I always try to keep that in mind. So there’s usually a certain

01:02:05 kind of structure, like abstract introductions, methodology, depending on the community and so on.

01:02:12 Is there something about the process of how to read it, whether you want to skim it to try to

01:02:19 find the parts that are easy to understand or not, reading it multiple times? Is there any kind of

01:02:26 hacks that you can comment on? I remember like Feynman had this kind of hack when he was reading

01:02:32 papers, where you would basically, I think I believe you would read the conclusion of the

01:02:39 paper. And we would try to just see if he would be able to figure out how to get to the conclusion

01:02:47 in like a couple of minutes by himself. And he would read a lot of papers that way. And I think

01:02:54 Fermi also did that almost. And Fermi was known for doing a lot of back of the envelope calculation.

01:02:59 So he was a master at doing that. And in terms of like, especially when reading a paper, I think

01:03:08 a lot of times people might feel discouraged about the first time you read it. You know,

01:03:14 it’s very hard to grasp or you don’t understand a huge fraction of the paper. And I think it’s

01:03:20 having read a lot of papers in my life, I think I’m in peace with like the fact that you might

01:03:25 spend hours where you’re just reading a paper and jumping from paper to paper, reading citations.

01:03:31 And like your level of understanding sometimes of the paper is very close to 0%. And all of a

01:03:38 sudden, you know, everything kind of makes sense in your mind. And then, you know, you have this

01:03:45 quantum jump where all of a sudden you understand the big picture of the paper. And this is an

01:03:54 exercise that I have to do when reading papers and especially like more complex papers, like,

01:03:58 okay, you don’t understand because you’re just going through the process and just keep going.

01:04:02 And like, it might feel super chaotic, especially if you’re jumping from reference to reference.

01:04:07 You know, you might end up with like 20 tabs open and you’re reading a ton of other papers,

01:04:12 but it’s just trusting that process that at the end, like you’ll find light. And I think for me,

01:04:18 that’s a good framework when reading a paper. It’s hard because, you know, you might end up spending

01:04:24 a lot of time and it looks like you’re lost, but that’s the process to actually, you know,

01:04:32 understand what they’re talking about in the paper.

01:04:34 Yeah, I think that process, I enjoy, I’ve found a lot of value in the process, especially for

01:04:40 things outside my field of reading a lot of related work sections and kind of going down

01:04:47 that path of getting a big context of the field. Because what’s, especially when they’re well

01:04:52 written, there’s opinions injected into the really related work. Like what work is important,

01:04:57 what is not. And if you read multiple related work sections that cite or don’t cite each other,

01:05:03 like the papers, you get a sense of where the field, where the tensions of the field are,

01:05:09 where the field is striving. And that helps you put into context, like whether the work is radical,

01:05:16 whether it’s overselling itself, whether it’s underselling itself, all those things. And added

01:05:23 on top of that, I find that often the related work section is the most kind of accessible and

01:05:31 readable part of a paper because it’s kind of, it’s brief to the point, it’s like summarizing,

01:05:38 it’s almost like a Wikipedia style article. The introduction is supposed to be a compelling story

01:05:43 or whatever, but it’s often like overselling, there’s like an agenda in the introduction.

01:05:49 The related work usually has the least amount of agenda except for the few like elements where

01:05:54 you’re trying to talk shit about previous work where you’re trying to sell that you’re doing

01:05:58 much better. But other than that, when you’re just painting where the field came from or where the

01:06:05 field stands, that’s really valuable. And also again, just to agree with Fionn in the conclusion,

01:06:11 I get a lot of value from the breadth first search, kind of read the conclusion, then read

01:06:17 the related work, and then go through the references in the related work, read the conclusion,

01:06:24 read the related work, and just go down the tree until you like hit dead ends or run out of coffee.

01:06:30 And then through that process, you go back up the tree and now you can see the results in their

01:06:35 proper context, unless of course the paper is truly revolutionary, which even that process will help

01:06:42 you understand that is in fact truly revolutionary. You’ve also, you talked about just following your

01:06:50 Twitter thread in depth first search. You talked about that you read the book on Grisha Perlman,

01:06:59 Grigori Perlman, and then you had a really nice Twitter thread on it and you were taking notes

01:07:07 throughout. So at a high level, is there suggestions you can give on how to take good notes?

01:07:13 Whether it’s we’re talking about annotations or just for yourself to try to put on paper ideas

01:07:20 as you progress through the work in order to then like understand the work better.

01:07:24 For me, I always try not to underestimate how much you can forget within six months after

01:07:31 you’ve read something. I thought you were going to say five minutes, but yeah, six months is good.

01:07:34 Yeah, or even shorter. And so that’s something that I always try to keep in mind. And it’s often,

01:07:42 I mean, every once in a while I’ll read back a paper that I annotated on Fermat and I’ll read

01:07:49 through my own annotations and I’ve completely forgotten what I had written. But it’s interesting

01:07:59 because in a way, after you just understood something, you’re kind of the best possible

01:08:03 teacher that can teach your future self. After you’ve forgotten it, you’re kind of your own best

01:08:12 possible teacher at that moment. And so it can be great to try to capture that.

01:08:19 It’s brilliant. It just made me kind of realize it’s really nice to put yourself in the position

01:08:27 of teaching an older version of yourself that returns to this paper, almost like thinking literally.

01:08:33 That’s under explored, but it’s super powerful because you were the person that you can,

01:08:38 if you look at the scale from one not knowing anything about the topic and 10,

01:08:44 you are the one that progressed from one to 10 and you know which steps you struggled with.

01:08:48 So you’re really the best person to help yourself make that transition from one to 10.

01:08:53 And a lot of the times, I really believe that the framework that we have to expose ourselves to

01:09:02 be talking to us when we were an expert, when we were taking that class and we knew everything

01:09:07 about quantum mechanics. And then six months later, you don’t remember half of the things.

01:09:12 How could we make it easier to have those conversations between you and your past

01:09:19 expert self? I think there might be, it’s an under explored idea. I think notes on paper

01:09:26 are probably not the best way. I’m not sure if it’s a combination of video,

01:09:31 audio, where it’s like you have a guided framework that you follow to extract information from

01:09:36 yourself so that you can later kind of revisit to make it easier to remember. But that’s,

01:09:44 I think it’s an interesting idea worth exploring that I haven’t seen a lot of people kind of

01:09:50 trying to distill that problem.

01:09:53 Yeah, I’m creating the kind of tools. I find if I record, it sounds weird, but I’ll take notes,

01:09:59 but if I record audio, like little clips of thoughts, like rants, that’s really effective

01:10:09 at capturing something that notes can’t. Because when I replay them, for some reason,

01:10:15 it loads my brain back into where I was when I was reading that in a way that notes don’t. Like

01:10:22 when I read notes, I’ll often be like, what? What was I thinking there? But when I listened

01:10:29 to the audio, it brings you right back to that place. And maybe with video, with visual, that

01:10:35 might be even more powerful.

01:10:36 I think so.

01:10:38 And I think just the process of verbalizing it, that alone kind of makes you have to structure

01:10:45 your thought and put it in a way that somebody else could come and understand it. And just

01:10:51 the process of that is useful to organize your thoughts and yeah, just that alone.

01:10:58 Does the Fermat’s Library Journal Club have a video component or no?

01:11:03 Not natively. We sometimes will include videos, but it’s always embedded.

01:11:07 Do people build videos on top of it to explain the paper? Because you’re doing all the hard work

01:11:12 of understanding deeply the paper.

01:11:16 We haven’t seen that happening too much. But we were actually playing around with the idea of

01:11:22 creating some sort of podcast version where we try to distill the paper on an audio format that

01:11:29 not maybe you can have access to.

01:11:30 Might be tricky.

01:11:31 Might be trickier, but there are definitely people that could be interested in the paper

01:11:35 and that topic, but are not willing to read it. But they might listen to a 30 minute episode

01:11:40 on that paper. You could reach more people and you might even bring the authors to the

01:11:44 conversation, but it’s tricky. And especially for like more technical papers. We’ve thought

01:11:49 about doing that, but we haven’t converged. I’m not sure if you have any tips.

01:11:55 Well, I’m going to take that as a small project to take one of the Fermat’s almost like

01:12:01 half advertisement and half as a challenge for myself to take one of the annotated papers and

01:12:05 use it as a basis for creating a quick video. I’ve seen like, hopefully I’m saying the name

01:12:14 correctly, but machine learning street talk. I think that’s the name of the show that I

01:12:20 recommend highly. That’s the right name. But they do exactly that, which is multiple hour

01:12:26 breakdown of a paper with video component. Sometimes with authors, people love it.

01:12:32 It’s very effective.

01:12:33 There’s also, I’ve seen, I haven’t seen the entire, in its entirety, but I’ve seen like the

01:12:39 founder of comma.ai, George. I’ve seen him like just taking a paper and then, you know,

01:12:46 distilling the paper and coding it, coding it sometimes during 10 hours. And he was able to,

01:12:53 you know, get a lot of people interested in that and viewing him.

01:12:57 I’m a huge fan of that. Like, George is a personality. I think a lot of people like

01:13:03 listen to this podcast for the same reason. It’s not necessarily the contents. They like to listen

01:13:08 to like a silly Russian who has a childlike brain and mumbles and all those like struggle with

01:13:16 ideas, right? And George is a madman who people just enjoy. Like, how is he going to struggle

01:13:22 in implementing this particular paper? How is he going to struggle with this idea? It’s fun to

01:13:26 watch and that actually pulls you in. The personality is important there.

01:13:29 True. But there’s, you know, I agree with you, but there also, it’s visible, like it’s,

01:13:34 there’s an extraordinary ability that is there. Like, he’s talented and you need to have,

01:13:40 there’s a craft and this guy definitely has talent and he’s doing something that is not easy.

01:13:45 And I think that also draws the attention of people.

01:13:47 Oh yeah.

01:13:48 And like the other day we were actually, we ran into this YouTube channel of this guy that was

01:13:53 restoring art, right? And it was basically just a video of him, like the production is really like

01:14:03 really well done. And it’s just him taking really old pieces of art, like, and then paintings and

01:14:09 then restoring them. But he’s really good at that. And he describes that process. And that draws

01:14:14 attention, draws the attention of people, regardless of your craft, be it like annotating a

01:14:20 paper or like restoring it.

01:14:22 Procurement, excellence. Yeah. Like George is incredibly good at programming.

01:14:26 Like quick, like, you know, those competitive programmers, like Topcoder and all those kinds of

01:14:34 stuff, he has the same kind of element where the brain just jumps around really quickly.

01:14:38 And that’s, yeah, just like, it’s motivating, but you’re right in watching people who are good at

01:14:48 what they do. It’s motivating. Even if the thing you’re trying to do is not what they’re doing,

01:14:52 it’s contagious when they’re really good at it. And the same kind of analysis with the paper,

01:14:57 I think, not just like the final result, but the process of struggling with it. That’s really

01:15:03 interesting. Yeah. I think, I mean, I think Twitch proved that, like, you know, that there’s

01:15:07 really a market for that, for watching people do things that they’re really good at. And you’ll

01:15:14 just watch it. You will enjoy that. That might even spike your interest in that specific topic.

01:15:20 And yeah, and people will enjoy watching sometimes hours on end of great craftsmans.

01:15:27 Do you mind if we talk about some of the papers? Do any papers come to mind

01:15:31 that have been annotated on Fermat’s library?

01:15:34 The papers that we annotated can be about completely random topics, but that’s part of

01:15:39 what we enjoy as well. It forces you to explore these topics that otherwise maybe you’d never

01:15:43 run into. And so the ones that come to mind are, to me, are fairly random. But one that I really

01:15:51 enjoyed learning more about is a paper written by a mathematician, actually, Thomas Postol,

01:16:00 and about a tunnel in a Greek island off the coast of Turkey. So it’s very random.

01:16:10 Yeah. Okay, so what’s interesting about this tunnel? So this tunnel was built in the sixth

01:16:18 century BC. And it was built in the island of Samos, which is, as I said, off the coast of Turkey.

01:16:27 And they had the city on one side, and they had a mountain, and then they had a bunch of

01:16:33 springs on the other side, and they wanted to bring water into the city. Building an aqueduct

01:16:40 would be pretty hard because of the way the mountain was shaped. And it would also, if they

01:16:44 were under a siege, they could just easily destroy that aqueduct, and then the water wouldn’t have

01:16:51 any water supply. The city wouldn’t have any water supply. And so they decided to build a tunnel,

01:16:57 and they decided to try to do it quickly. And so they started digging from both ends

01:17:06 at the same time through the mountain. And so when you start thinking about this,

01:17:12 it’s a fairly difficult problem. And this is like sixth century BC, so you had very limited access

01:17:20 to the mathematical tools that you had at the time, were very limited. And so what this paper

01:17:25 is about is about the story of how they built it and about the fact that for about 2000 years,

01:17:32 kind of the accepted explanation of how they built it was actually wrong. And so this tunnel

01:17:38 has been famous for a while. There are a number of historians that talked about it since ancient

01:17:43 Egypt. And the method that they described for building it was just wrong. And so these researchers

01:17:54 went there and were able to figure that out. And so basically, kind of the way that they thought

01:18:00 they had built it was basically, if you can imagine looking at the mountain from the top,

01:18:06 and you have the mountain and then you have both entrances. And so what they thought,

01:18:11 and this is what the ancient historians described, is that they effectively tried to draw a right

01:18:19 angle triangle with the two entrances at each end of the hypotenuse. And the way they did it is like

01:18:27 they would go around the mountain and kind of walking in a grid fashion. And then you can figure

01:18:32 out the two sides of the triangle. And then after you have that triangle, you can effectively draw

01:18:40 two smaller triangles at each entrance that are proportional to that big triangle. And then you

01:18:47 kind of have arrows pointing in each way. And then you know at least that you have a line going

01:18:54 through the mountain that connects both entrances. The issue with that is like once you go to this

01:19:01 mountain and you start thinking of doing this, you realize that especially given that the tools that

01:19:06 they had at the time, that your error margin would be too small. You wouldn’t be able to do it.

01:19:14 Just the fact of trying to build this triangle in that fashion, the error would accumulate and you

01:19:20 would end up missing. You’d start building these tunnels and they would miss each other. So the task

01:19:24 ultimately is to figure out like really perfectly as close as possible the direction you should be

01:19:30 digging. First of all that it’s possible to have a straight line through and then what that the

01:19:35 direction would be. And then you are trying to infer that by constructing a right triangle

01:19:42 by doing. I’m not exactly sure about how to do that rigorously like by tracing the mountain,

01:19:49 by walking along the mountain. You said grids? Yeah you kind of walk as if you were in a grid

01:19:55 and so you just walk in right angles. But then you have to walk really precisely then. Exactly.

01:20:00 You have to use tools to measure this and the terrain is probably a mess. So this makes more

01:20:07 sense in 2D and 3D gets even weirder. So okay gotcha. But so this method was described by like

01:20:14 an ancient Egyptian historian. I think hero of Alexandria. And then for about like yeah for

01:20:20 about 2000 years that’s how we thought that they had built this tunnel. And then these

01:20:28 researchers went there and found out that actually they must have had to use other methods. And then

01:20:35 in this paper they describe these other methods and of course they can’t know for sure. But

01:20:41 there’s they present a bunch of plausible alternatives. The one that for me was is the

01:20:46 most plausible is that what they probably must have done is to use something that is similar to

01:20:52 an iron sight on a rifle. The way you can line up your rifle with a target off in the distance

01:21:00 by having an iron sight. And they must have done something similar to that

01:21:08 effectively with three sticks. And that way they were able to line up sticks along the side of the

01:21:16 mountain that were all on the same height. And so that then you could get to the other side

01:21:22 and you could and then you could draw that line. So this for me is the most plausible

01:21:27 way that they might have done that. But then they described this in detail and other possible

01:21:34 approaches in this paper. So this is a mathematician doing this? Yeah this is a mathematician that did

01:21:40 this. Which I suppose is the right mindset instead of skills required to solve an ancient problem

01:21:46 right? Yeah. Yeah. There’s mathematicians and engineers a lot of things. Because they didn’t

01:21:52 have computers or drones or lidar back then or whatever technology you would use modern day for

01:21:58 the civil engineering. Yeah. Another fascinating thing is that like you know after effectively

01:22:04 after the the downfall of the Roman civilization people didn’t build tunnels for about a thousand

01:22:09 years. We go a thousand years without tunnels and then like only in like in late middle ages that

01:22:15 we start doing them again. But here is the tunnel like sixth century BC like incredibly limited

01:22:20 mathematics and they and they build it in this way. And it was a mystery for a long time exactly

01:22:28 how they did it. And then these mathematicians went there and basically with no archaeology

01:22:35 kind of background were able to figure it out. How do annotations for this paper look like? What is

01:22:40 it what’s a successful annotation for paper like this? Yeah so sometimes you’re for this paper

01:22:46 sometimes adding some more context on on a specific part like sometimes they they mentioned

01:22:54 for instance these instruments that were common in ancient Greece and ancient Rome for for building

01:23:02 things. And and and so in some of those annotations I describe these instruments in more detail and

01:23:08 how they worked because sometimes it can be hard to to visualize these.

01:23:12 Then this paper I forget exactly when when this was published I believe maybe maybe the 70s.

01:23:21 And then there was further research into this tunnel and more interesting other interesting

01:23:26 aspects about it. I add those to that paper as well. There’s historical context that I also go

01:23:32 into there for instance the fact that as a as I said that effectively after the downfall of the

01:23:39 Roman Empire no tunnels were built like this something that I that I go that I that I added

01:23:43 to the paper as well. Yeah so so when other people look at the paper how do they usually consume the

01:23:49 annotations? So they it’s like is there a commenting feature is the I mean like this is a really

01:23:55 enriching experience the way you read a paper. What what aspects do you do people usually talk

01:24:02 about that they they value from this? So yeah so anybody can just go on there and and either add

01:24:09 a new annotation or add a comment to an existing annotation and so you can start a kind of a thread

01:24:15 within an existing annotation. And that’s something that happens relative frequency and then

01:24:21 because I was the original author of the initial annotation I get pinged and so oftentimes I’ll go

01:24:27 back and and and add on to to that thread. How’d you pick the paper? I mean first of all this whole

01:24:33 process is really exciting. I’m gonna especially after this conversation I’m gonna make sure I

01:24:38 participate much more actively on papers that I know a lot about and on paper I know nothing about.

01:24:44 I should both of us annotate the paper. I would love to. I also I mean I realized that uh there’s

01:24:52 like it’s an opportunity for people like me to publicly annotate a paper. Or do an AMA around

01:25:02 the paper. Yeah exactly but yeah but like be um be in the conversation about a paper. It’s like a

01:25:09 place to have a conversation about an idea. You get the other way to do it that’s much more ad hoc

01:25:15 is on Twitter right? But this is more like formal and you don’t have to be in the conversation

01:25:19 on Twitter right? But this is more like formal and you could actually probably integrate the two.

01:25:24 You have a conversation about the conversation. So the Twitter is the conversation about a

01:25:29 conversation and the main conversation is in this space of annotations. There’s an interesting effect

01:25:34 that we we see sometimes with the annotations on our papers is that a lot of people especially if

01:25:39 we the annotations are really well done people sometimes are afraid of adding more annotations

01:25:46 because they see that as a kind of a finished work. Yes. And so they they don’t want to pollute

01:25:51 that or uh and especially if it’s like a silly question. This is I don’t think that’s good. I

01:25:57 think you know we should as much as possible try to lower the barrier for someone to jump in and

01:26:02 ask questions. I think it only like most of the times it adds value but it’s some feedback that

01:26:07 we got from users and and readers. I’m not exactly sure how to to kind of fight that but um. Well I

01:26:17 think I think if I serve as an inspiration in any way is by asking a lot of dumb questions and saying

01:26:26 a bunch of dumb shit all the time and hopefully that inspires the rest of the other folks to do

01:26:31 the same because that’s the only way to knowledge I think is to be willing to ask the dumb questions.

01:26:37 And there are papers that are like um you know we have a lot of papers on Fermat’s where it’s just

01:26:42 one page or really short papers and you we have like the shortest paper ever published in a math

01:26:48 journal like with like just a couple of words. Yeah. One of my favorite papers on the platform

01:26:53 is actually a paper um written by Enrico Fermi. Yeah. And the title of the paper is my obs I think

01:26:59 is my observations at Trinity. So basically Fermi was part of the Manhattan Project. So he was in

01:27:06 New Mexico when they exploded the first atomic bomb and so he was a couple of miles away from

01:27:13 the explosion and he was probably one of the first persons to calculate the energy of the explosion.

01:27:18 And so the way he did that was he took a piece of paper and he tore down a piece of paper in in

01:27:25 little pieces and when the bomb exploded the Trinity bomb was the name of the bomb like he

01:27:30 waited for the blast to arrive at where he was and then he threw those pieces of paper in the air

01:27:37 and he calculated the energy based on the displacement of the paper the pieces of paper

01:27:42 and then he wrote a report which was classified until like a couple of years ago one page report

01:27:47 like calculating the energy of the explosion. Oh that’s so badass. And I we actually went there

01:27:53 and kind of unpacked and like I think he just mentions basically the energy and we we actually

01:27:58 went and one of the annotations is like explaining how he did that. I wonder how accurate he was.

01:28:05 It was maybe I think like 20 or 25 percent off. Then there was another person that actually

01:28:10 calculated the energy based on images after the explosion at the rate and the rate at which the

01:28:18 like the mushroom of the explosion expanded and it’s more accurate to calculate the energy based

01:28:23 on that and I think it was like 20 20 percent off but it’s it’s really interesting because you know

01:28:29 Fermi was known for all these being a master at these back of the envelope calculations always

01:28:34 like the the Fermi problems are well known for for that and it’s super interesting to see like

01:28:40 that just one page report that was also actually classified and it’s interesting because a couple

01:28:45 months ago when the Beirut explosion happened there was a video circulating of these a bride

01:28:52 that was doing a photo shoot when the explosion in Beirut happened and so you can see a video of

01:28:57 her with the wedding dress and then the explosion happens in the blast arrives at where she was she

01:29:02 was a couple of miles away from the last and you can see like the displacement of the dress as well

01:29:09 and I actually looked and that video went viral on Twitter and I actually looked at that video

01:29:13 and based I used the same techniques that Fermi used to calculate the energy of the explosion

01:29:18 based on the displacement of the dress and you could actually see where where she was at the

01:29:23 the distance from the explosion because there was a store behind her and you could look the name of

01:29:27 the store and and so I calculated that it was the distance and then you can the based on the

01:29:32 distance where she was from the the explosion and also on the the displacement of the dress

01:29:37 like because you can when the blast happens like you can see the dress going back and then going

01:29:43 back to the original position and like by just looking at like how much the the dress moved you

01:29:49 can estimate the explode the energy of the explosion I assume you published this on Twitter

01:29:54 was just a Twitter thread but it actually like a lot of people share that and it was picked up by

01:30:00 a couple of news outlets but I was hoping it would be like a formal title and it would be an archive

01:30:07 no no no it may be submitted just the Twitter thread but it was interesting because it was

01:30:11 exactly the same method that Fermi used is there something else that jumps to mind like what is

01:30:17 there something um I know like in terms of papers like I know the Bitcoin paper is super popular

01:30:24 is there something interesting to be said about any of the white papers in the cryptocurrency space

01:30:29 yeah the the the Bitcoin paper was the first paper that we put on for months and uh why that

01:30:36 why that choice as the first paper it was a while ago and it was one of the papers that I read

01:30:42 and then uh and then kind of explained it to to louise and or to other friends that do this

01:30:48 journal club with us and um I did some research in cryptography uh as an undergrad and so it was a

01:30:56 topic that I was interested in um but even for me that I I had that background but reading the

01:31:02 Bitcoin paper it took me a few reads to really kind of wrap my head around it it’s it’s right

01:31:08 it’s it uses very spartan precise language in a way it’s like you feel like you can’t take any

01:31:14 word out of it without something falling apart and uh and it’s all there I think it’s a beautiful

01:31:20 paper and it’s it’s it’s very well written of course but um you know we wanted to try to make

01:31:28 it accessible so that anybody that maybe is an undergrad in computer science could go on there

01:31:33 and then and and know that you have all the information in in that page that you’re going

01:31:39 to need to understand the mechanics of Bitcoin and so like I explain you know the basic

01:31:47 public key cryptography that you need to to know in order to understand it like explain

01:31:52 okay what are the properties of a hash function and how they are useful in this context

01:31:57 um explain what a merkle tree is so a bunch of those basic concepts that maybe if you’re reading

01:32:02 it for a first time and you’re an undergrad and you know you don’t know those terms you’re going

01:32:06 to be you know discouraged because maybe okay now I have to go and google around until I understand

01:32:10 these before I can make progress in the paper um and and this way it’s all there you know so so

01:32:16 there’s a magic to also to the fact that over time more people went on there and and added further

01:32:23 annotations so the the idea that the paper gets easier and more accessible over time but that’s

01:32:29 still you’re still looking at the original content the way the the author intended it to be uh but

01:32:36 there’s just more context and the toughest bits have have more in depth explanations okay I think

01:32:43 like there’s a just so many interesting papers uh there like I remember reading the paper that was

01:32:50 written by Freeman Dyson on the like the the first time that he explained he came up with

01:32:56 the concept of the Dyson sphere and he he put that out like it’s again it’s a one page paper

01:33:02 um and he what he explained was that eventually if a civilization develops and and grows there’s

01:33:10 going to be a point where when the resources on the planet are not are not enough for the energy

01:33:16 requirements of that civilization so if you want to go the next step is you need to go to the next

01:33:22 star and extract energy from that star and the way to do it is you need to build some sort of cap

01:33:28 around the star that extracts the energy so he theorized this idea of the the Dyson sphere

01:33:35 and he went on to kind of analyze how you would build that the stability of that sphere like

01:33:40 if something happens if there’s like a small oscillation with that fear collapse into the

01:33:45 star or know what what would happen and even went on to kind of say that a good way for us to look

01:33:53 for signs of intelligent life out there is to look for signals of these Dyson spheres and because you

01:34:00 know according to the law of second law of thermodynamics like there’s going to be some a

01:34:04 lot of infrared radiation that is going to be emitted as a consequence of extracting energy

01:34:08 from the star and we should be able to see those signals of like infrared if we look at the sky

01:34:14 but all these like from the introduction of the concept like the how to build the Dyson sphere

01:34:20 the problems of like having a Dyson sphere how to detect how that could be used as a signal for

01:34:24 intelligence life really that’s all in the paper all in one like one page paper and it’s like it’s

01:34:29 it’s for me it’s beautiful it’s like where was this published i don’t remember it’s fascinating

01:34:35 that papers like that could be yeah i mean the guts it takes to put that all together in a paper

01:34:41 before you know that that kind of challenges our previous discussion that of paper i mean papers

01:34:46 can be beautiful you can play with the format right it but there’s a lot to unpack there that’s

01:34:52 like the the that’s the the starting point but it’s it’s beautiful that you’re able to put that

01:34:57 in one page and then people can build on top of that and but the key ideas are there yeah exactly

01:35:04 what about have you looked at any of the the big seminal papers throughout the history of

01:35:09 science like you look at simple like einstein papers have any of those been annotated yeah

01:35:16 yeah no we we have some more seminal papers that that people have heard about um you know we have

01:35:22 the the DNA double elix paper on there we have the Higgs boson uh paper um yeah there’s papers

01:35:33 that they’ll that we know that it’s they’re not going to be finding out about them because of us

01:35:39 but it’s papers that we think um should be more widely read and that folks would benefit from

01:35:45 having some annotations there and so we also have a number of those a lot of like discovery papers

01:35:50 for fundamental like particles and all that there’s we have a lot of those on from us library

01:35:56 um yeah we i would like to end we haven’t annotated that one but i’d like to on the

01:36:00 Riemann hypothesis that’s a really interesting paper as well um and and but we haven’t annotated

01:36:06 that one but there’s a lot of like more historical landmark papers um on the platform have you done

01:36:13 uh Poincare conjecture with uh with Perlman that’s too much that’s too much that’s too much

01:36:19 too much for me but it’s uh it’s it’s interesting that you know and going back to our discussion

01:36:25 like the the Poincare paper was like published on archive and and it was not on a journal like

01:36:30 the three papers and yeah what do you make of that i mean he’s such a fascinating human being

01:36:34 exactly i mentioned to you offline that i’m going to russia he’s somebody i’m really uh to interview

01:36:40 yeah i well i so i definitely will interview him i um and i believe i will i believe i can i just

01:36:48 don’t know how to i know where he lives so here okay my my uh my hope is my conjecture is that

01:36:57 if i just show up to the house and look desperate enough uh that uh or threatening enough for some

01:37:03 combination of both that like the only way to get rid of me is to just get the thing done that’s the

01:37:08 hope it’s actually interesting that you mentioned that because i after i um so a couple of weeks ago

01:37:14 i was searching for like stuff about Perlman Perlman online ended up on this twitter account

01:37:19 of like this guy that claims to be Perlman Perlman’s assistant and he’s like he has been

01:37:26 posting a bunch of pictures like next to Perlman you can see like Perlman in in a library and he’s

01:37:30 like next to him like taking a selfie or like Perlman walking on the street and like maybe you

01:37:36 could reach out to his assistant then i’ll send you i’ll send you this twitter account so maybe

01:37:42 you’re onto something no but but going going back to like Perlman is super interesting because the

01:37:47 fact that he published the the the the proofs on archive is what was also like a way for him to

01:37:53 because he really didn’t like the scientific publishing industry and the fact that you had to

01:37:58 pay to get uh access to to articles and that was a form of like protest and that’s why he published

01:38:06 um those papers there i mean i i think Perlman is just a fascinating like character and for me

01:38:11 it’s this kind of ideal of a platonic ideal of what a mathematician should be you know it’s it’s

01:38:18 someone that is you know it’s just cares about deeply cares about mathematics you know it cares

01:38:23 about fair attribution of of um disregards money and um and and like the fact that he published

01:38:31 like on archive was is a good example of what about the Fields Medal that he turned down the

01:38:35 Fields Medal what’s what’s yeah what do you make of that yeah i mean if you look at like the reasons

01:38:42 why he rejected the Fields Medal so after so Perlman did a postdoc in the US and when he came

01:38:47 back to Russia um do you know how good his English is i think it’s very fairly good it’s pretty good

01:38:53 i think it’s really good especially given lectures yeah but i haven’t been able to listen to anything

01:38:59 well certainly not listen but i haven’t been able to get anybody because i know a lot of people have

01:39:03 been to those lectures i’m not able to get a sense of like yeah but how strong is the accent what are

01:39:10 we talking about here is this going to have to be in Russian it’s going to have to be in English

01:39:13 is fascinating but he writes the papers in English so true like there’s there’s but there’s so many

01:39:18 like it’s such a fascinating character and um there are a couple of examples like him like at

01:39:23 i think 28 or 29 he proved like a really famous uh conjecture called the soul conjecture i believe

01:39:29 it was like in a very short four page proof of that was a really big breakthrough then he went

01:39:34 to Princeton to give a lecture on that and after the lecture uh the the chair of the math department

01:39:40 at Princeton a guy called Peter Sarnak went up to to Perelman was trying to recruit him

01:39:46 trying to offer him a position at Princeton and he was and at some point he asked for Perelman’s

01:39:52 resume and Perelman responded saying just gave a lecture on like this really tough problem why do

01:39:59 you need my resume like i’m not gonna send you like i just proved like my value uh but uh but

01:40:06 going back to the Fields Medal like when when Perelman went to back to Russia he arrived at a

01:40:12 time where the the salary of postdocs were so much off in regards to inflation that they were not

01:40:20 making any money like the the people didn’t even bother to pick up the checks at the end of the

01:40:25 month because they were just like ridiculous but thankfully he had some money that he had

01:40:30 uh gained while he was doing this postdoc so he just concentrated on like the Poincare the the

01:40:36 Poincare conjecture problem which he when he when he took that um he took it after uh it was

01:40:42 reframed by this uh mathematician called Richard Hamilton which posed the problem in a way that

01:40:48 it turned into this super like math Olympiad problem with like perfect boundaries well defined

01:40:54 and that was perfect for Perelman to attack and so he spent like seven years working on that and

01:40:59 then in 2002 he started publishing those papers on archive and people started jumping on that

01:41:06 reading those papers and there was like a lot of excitement around that a couple of years later

01:41:12 there were two researchers i believe it was they were from Harvard that took Perelman’s Perelman’s

01:41:18 work they sanded some of the edges and they republished that saying that you know based on

01:41:25 Perelman’s work they were able to figure out the the Poincare conjecture and then there was um

01:41:31 at the time at the the international um conference of mathematics in 2000 2006 i believe that’s when

01:41:39 they were going to give out the Fields Medal there was a lot of debate like oh who’s who’s

01:41:44 like we should get the credit for solving this big problem and for Perelman it like it it felt

01:41:52 really sad that people were even considering that he was not the person that solved that

01:41:57 and and the claims that those like researchers when they published after Perelman they were

01:42:03 false claims that they were the ones they just sanded a couple of edges like Perelman did all

01:42:07 the really hard work and so just just the fact that they doubted that Perelman had done that

01:42:14 like was enough for him to say i’m not i’m not interested in this prize and that was one of the

01:42:19 reasons why he rejected the Fields Medal then he also rejected the clay prize so the Poincare

01:42:25 conjecture was one of the millennium prizes there was a million dollar prize associated with that

01:42:30 problem and that has to add to do with the fact that for them to attribute that prize i think it

01:42:35 had to be published on a journal yes the proof and again Perelman’s principles of like interfered

01:42:43 here and and he also just didn’t care about the money he was like um clay i think was a businessman

01:42:48 and he’s like doesn’t have to do anything with with mathematics i don’t care about these like

01:42:54 that’s one of the reasons why he rejected it yeah there’s it’s hard to convert into words but

01:43:00 at MIT i’m distinctly aware of the distinction between when i enter a room there’s a certain

01:43:07 kind of music to the way people talk when we’re talking about ideas versus what that music sounds

01:43:15 like when we’re talking when it’s like bickering in the space of like whether it’s politics or

01:43:24 funding or egos it’s a different sound to it and i’m distinctly aware of the two and i kind of

01:43:35 sort of to me personally happiness was just like swimming around the one that like is the political

01:43:42 stuff or the money stuff and all that or egos and i think that’s probably what problem is as well

01:43:51 like the moment he senses there’s any as what it feels metal like the moment you start to have any

01:43:57 kind of drama around credit assignment all those kinds of things it’s almost not that it’s important

01:44:03 who gets the credit it’s like the drama in itself gets in the way of the exploration of the ideas

01:44:08 or the fundamental thing that makes science so damn beautiful and and you can really see that

01:44:13 is also a product of that russian school of like doing science and you can see that that um that

01:44:21 people were you know during the cold war a lot of mathematicians they were not making any money they

01:44:26 were doing math for the sake of math like for the intellectual pleasure of like solving a difficult

01:44:33 problem yeah and you know even even if it was a flawed system and there were a lot of problems with

01:44:38 with that there’s these they were able to to actually achieve these and there were a lot of

01:44:43 and perelman for me is the perfect product of that he just cared about like working on tough problems

01:44:49 he didn’t care about anything else it was just math you know pure math yeah there’s a like for

01:44:56 the broader audience i think another example of that is like professional sports versus olympics

01:45:02 i’ve especially in russia i’ve seen that clear distinction where because the state manages

01:45:10 so much of the olympic process in russia as people know with the steroids yes yes yes but outside

01:45:16 of steroids thing uh is like the athlete can focus on the pure artistry of the sport like

01:45:26 like not worry about the money not just in the way they talk about it the way they think about it the

01:45:31 way they define excellence versus like in the perhaps a bit of a capitalist system in united

01:45:38 states with american football with baseball basketball so much of the discussion is about

01:45:46 money now of course at the end of the day it’s about excellence and artistry and all that but

01:45:53 when the culture is so richly grounded in discussions of money and

01:45:58 uh sort of this capitalistic like uh merch and uh businesses and all those kinds of things

01:46:04 it changes the nature of the activity and it’s in a way that’s hard again to describe in words but

01:46:11 when it’s purely about the activity itself it’s almost like you quiet down all the noise

01:46:18 enough to hear the signal enough to hear the beauty like whenever you’re talking about the

01:46:23 like whenever you’re talking about the money that’s when the marketing people come and the

01:46:28 business people the non creatives come and they fill the room and there’s and they create drama

01:46:32 and they know how to create the drama and the noise as opposed to the people who are truly

01:46:36 excellent at what they do the the person in their arena right like when you remove all the money

01:46:44 and you just let that thing shine that’s when true excellence can and can come out and that was

01:46:51 one of the few things that worked with the communist system in the Soviet Union to me at

01:46:56 least as somebody who loves sport and loves mathematics and uh science that worked well

01:47:03 removing the money from the picture uh you know not that I’m um not that I’m saying poverty is

01:47:11 good for science there’s some level in which not worrying about money is good for science it’s a

01:47:18 weird I’m not exactly sure what to make of that because capitalism works really damn well yeah but

01:47:25 it’s um it’s tricky how to find that balance one field’s medalist that is interesting to look at

01:47:32 and I think you mentioned it earlier but it’s Cédric Villani which is might be the only uh

01:47:38 field’s medalist that is also a politician now but so it’s this it’s this brilliant French

01:47:43 mathematician that won the field’s medal and and after that he decided that one of the ways that

01:47:51 he could have could have uh you know the biggest leverage kind of is in pushing science in the

01:47:58 direction that he thinks science should go would be to to try to go into politics and so that’s

01:48:04 what he did and and uh and he has ran I’m not sure if he has won any election but I think he’s running

01:48:11 I think he’s running for mayor for mayor of Paris or something like that but it’s this brilliant

01:48:15 mathematician that uh that uh before winning the field’s medal had only been just a brilliant

01:48:22 mathematician but but after that he decided to go into politics to to try to to have an impact and

01:48:27 try to change some of the things that he he would complain about um before so so there’s that

01:48:33 component as well yeah and I’ve always thought mathematics and science should be like like James

01:48:40 Bond would in my eyes I think be sexier if he did math like we should as a society put

01:48:49 excellence in mathematics at the same level as being able to kill a man with your bare hands

01:48:53 like those are both useful features like that’s admirable it’s like oh like that makes you like

01:48:59 that makes the person interesting like being extremely well read about history or philosophy

01:49:05 being good at mathematics being able to kill a man with bare hands these are all the same in my book

01:49:10 so I think all are useful for action stars uh and I think the society will benefit for uh for giving

01:49:16 more value to that like one of the things that bothers me about American culture is the I don’t

01:49:24 know the right words to use but like the nerdiness associated with science like like in I I don’t

01:49:32 think nerd is a good word in in American culture because uh it’s seen as like weakness there’s like

01:49:40 images that come with that and it’s fine you could you could be all kinds of shapes and colors and

01:49:47 personalities but like to me uh having sophisticated knowledge in science being good at math

01:49:56 doesn’t mean you’re weak in fact it could be the very opposite and so it’s it’s an

01:50:01 interesting thing because it was very much differently viewed in the uh in the Soviet Union

01:50:07 so I know for sure as an existence proof that uh it doesn’t have to be that way but it um

01:50:15 I also feel like we lack a lot of role models in terms if you ask people are mentioned to mention

01:50:22 one mathematician that they know that is alive today I think a lot of people would struggle

01:50:26 to answer that question um and I also think I love Neil deGrasse Tyson okay but there is uh

01:50:38 having more role models is good like different kinds of personalities he he has kind of fun and

01:50:44 and it’s very it’s uh like Bill Nye the science guy I don’t know if you guys know him so like that

01:50:49 spectrum that yeah but there there’s not like Feynman is no longer there uh those kinds of

01:50:56 personality Carl Sagan even Carl Sagan yeah like a seriousness that’s like not playful like not

01:51:05 apologetical yeah exactly not apologetic about being knowledgeable like like in fact like the

01:51:12 kind of energy where you feel uh self conscious about not having thought about some of these

01:51:21 questions right just like when I see James Bond I feel bad about that I don’t have never killed a

01:51:28 man like I need to make sure I fix that that’s the way I feel the same way I want to feel like

01:51:33 that way well Carl Sagan talks I I feel like I need to have that same kind of seriousness about

01:51:38 science like if I don’t know something I want to I want to know well what about Terrence Tao

01:51:44 he’s kind of a superstar what are your thoughts about him true it’s probably one of the most

01:51:48 famous mathematicians alive today and probably one of I mean regardless of like is of course

01:51:54 he won a field the Fields Medal is really smart and talented mathematician um it’s also like a

01:52:02 big inspiration for us um at least for some of the work that we do with Fermat’s library

01:52:10 so Terrence Tao is is known for having you know a big blog and he’s pretty open about

01:52:15 um like his research and he also he tries to make his work as public as possible um through his blog

01:52:23 posts um in fact there’s a really interesting um problem that got solved a couple of years ago

01:52:30 so Tao was working with uh on a problem on an Erdos problem actually so Paul Erdos was this

01:52:37 mathematician from Hungary and he was known for like um the Erdos for a lot of things but one of

01:52:44 the things that he was also known was for the Erdos problem so he was always like um creating

01:52:49 these problems and usually associating prizes with those problems and a lot of those problems

01:52:54 are still open like and and there will be some of them will be open for like maybe

01:52:58 a couple hundred years and I think that’s actually an interesting hack for him to collaborate with

01:53:02 future mathematicians you know his his name will keep coming up and you know for future generations

01:53:08 but so Tao was working on one of these problems called the Erdos discrepancy and he published a

01:53:14 blog post on like uh about that problem about that problem and he reached like a dead end and then um

01:53:21 all of a sudden there was this guy from from Germany that wrote like a comment on his blog post

01:53:26 saying okay like some of the that so this problem is like a Sudoku like flavor and some of the

01:53:32 machinery that we’re using to solve Sudoku could be used here and that was actually the key to

01:53:38 solve the Erdos discrepancy problem so the there was a comment on his blog and I think that that

01:53:43 that for me is an example of like how to do again going back to collaborative science online um and

01:53:50 the power that it has but Tao is also like pretty public about like some of the struggles and of

01:53:57 of being a a mathematician like and and even he wrote about some of the unintended consequences

01:54:04 of having extraordinary ability in a field and he used himself as an example when he was growing up

01:54:10 he was extremely talented in in mathematics from a young age like Tao was a person he won an

01:54:17 uh medal in like one of the IMOs at the age I think was a gold medal at the age of 10 or something

01:54:22 like that and so he mentioned that when he was growing up like and especially in college when he

01:54:28 was in a class that he enjoyed it didn’t it just came very natural for him and he didn’t have to

01:54:34 work hard to just ace the class and when he found that the class was boring like it didn’t work and

01:54:40 he barely passed barely passed even some I think in college he almost failed two classes and and

01:54:47 he was talking about that and how he brought those studying habits or like uh in existence of studying

01:54:53 habits when he went to Princeton Princeton for his PhD and in Princeton when he you know started kind

01:54:58 of um delving into more complex problems and classes he struggled a lot because he didn’t have

01:55:05 that uh those those habits like he wasn’t taking notes and he was he wasn’t studying hard when he

01:55:11 when he faced problems and he almost failed out of his his PhD uh he almost failed his PhD exam

01:55:18 and um it it talks about like having this conversation with with his advisor and the

01:55:23 advisor pointing out like you’re not this is not working you you might have to get out of the

01:55:28 program and like how that was a kind of a turning point for him and um he was like you know he was

01:55:35 um and like it was super important in his career so I think Tao is also like this figure that apart

01:55:41 from being just an exceptional mathematician he’s also pretty open about you know what what it takes

01:55:46 to to to be a mathematician and some of the struggles of this type of careers and and I think

01:55:50 it’s that’s super important in many ways he’s a contributor to open science and open humanity

01:55:56 so he’s being an open human true by communicating uh Scott Aaronson is another in computer science

01:56:03 world who’s a very different style very different style but there’s something about a blog that

01:56:09 is authentic and real and just gives us a window into the into the mind and soul of of of these

01:56:15 brilliant folks so it’s it’s definitely a gift let me ask you about Fermat’s library on twitter

01:56:22 which uh I mean I don’t know how to describe it people should definitely just follow Fermat’s

01:56:27 library on twitter I keep following and unfollowing from his library because because uh it’s so

01:56:35 it it gives when I follow it um leads me on down rabbit holes often that um that um that are very

01:56:46 fruitful but but anyway so the the posts you do with the on twitter are just these beautiful

01:56:53 are things that reveal some beautiful aspect of mathematics um is there um is there something you

01:57:00 could say about the approach there yeah and um maybe maybe broadly what you find beautiful about

01:57:10 mathematics and then more specifically how you convert that into a rigorous process of revealing

01:57:17 that in tweet form that’s a good point I think there’s something about math that you know a lot

01:57:22 you know a lot of the mathematical content and you know papers or like little proofs um you know

01:57:29 has in a way sort of an infinite half life what I mean by that is that if you look at like Euclid’s

01:57:36 elements it’s as valid today as it was when it was created like 2000 years ago and that’s not true

01:57:42 for a lot of other scientific fields um and so in regards to twitter I think there’s also a very

01:57:51 it’s a very under under explored platform from a learning perspective I think if you look at

01:57:58 content on twitter it’s very easy to consume it’s very easy to read um and especially when you’re

01:58:07 trying to explain something you know we humans get a dopamine hit if we learn something new

01:58:14 and that’s a very very powerful feeling and that’s why you know people go to classes when

01:58:20 you have a really good professor you know it’s looking for those dopamine hits and

01:58:25 and and that’s something that we try to explore when we’re producing content on twitter imagine

01:58:31 if we could if you would on a line to a restaurant you could go go to your phone to learn something

01:58:36 new instead of social going to a you know social network and to just and so and I think it’s very

01:58:44 hard to to sometimes to kind of provide that feeling because you need to sometimes digest

01:58:51 content and and put it um in a way you know that it feeds 280 characters um and and it requires a

01:58:58 lot of sometimes time to do that uh even though it’s easy to consume it’s hard to make but once

01:59:04 you are able to to provide that eureka moment to people like that’s very powerful they get that

01:59:10 dopamine hit and like you create this feedback cycle and people come back for for more and in

01:59:16 twitter compared to like you know an online course for a book you have a zero percent dropout so

01:59:21 people will read the content the content so that it’s it’s like it’s part of the creators like the

01:59:27 person that is creating the content if you’re able to actually get that feedback cycle it’s

01:59:32 super super powerful yeah but some of the stuff is like like how the heck do you find that and

01:59:38 and i don’t know why it’s so appealing it uh like uh this is from uh what is it

01:59:45 a couple days ago i’ll just read out the number 23456789 is the largest prime number with

01:59:53 consecutive increasing digits i mean that is so cool that’s like some weird like glimpse

02:00:00 into some deep universal truth even though it’s just a number i mean that’s like so arbitrary

02:00:07 like why why is it so pleasant that that’s a thing but it is in some way it’s almost like

02:00:12 it is a little glimpse at some much bigger like um and and i think like especially if we’re talking

02:00:19 about science there’s something unique about you go and with a lot of the tweets you go sometimes

02:00:25 from a state of not knowing something to knowing something and that is very particular to science

02:00:30 math physics and that again is extra extremely addictive and that’s that’s how i i i i feel about

02:00:37 that and um that’s why i think people engage so much with with our tweets and go into rabbit holes

02:00:43 and then they you know we start with prime numbers and all of a sudden you are spending hours reading

02:00:49 number theory things and you go into wikipedia and you lose a lot of time there but uh well the

02:00:56 variety is really interesting too there’s human things there’s uh there’s physics things there’s

02:01:01 like numeric things like i just mentioned but there’s also more rigorous mathematical things

02:01:08 there’s stuff that’s tied to the history of math and the proofs and there’s visual there’s

02:01:12 animations uh there are looping animations that are incredible that reveal something

02:01:17 there’s uh andrew wiles on being smart this is just me now like ignoring you guys is just going

02:01:24 through yeah we’re a bit like math drug dealers we’re just trying to get you hooked we’re trying

02:01:29 to give you that hit and trying to get you hooked yes some people are brighter than others but i

02:01:35 really believe that most people can really get to to quite a good level of mathematics if they’re

02:01:40 prepared to deal with these psychological issues of how to handle the situation of being stuck

02:01:45 yeah there’s some truth to that that’s truth i feel that’s like really it’s some truth in terms

02:01:51 of research and also about startups you’re stuck a lot of the time before you you get to a

02:01:57 breakthrough and and it’s difficult to endure that process of like being stuck and because you’re not

02:02:02 trained to to be in that position um i feel uh yeah that’s yeah most people are broken by the

02:02:09 stuckness or like their district like uh i i’ve i’ve been very cognizant of the fact that

02:02:17 more and more social media becomes a thing like distractions become a thing that that moment of

02:02:24 being stuck is uh your mind wants to to go do stuff that’s unrelated to being stuck and you

02:02:32 should be stuck i’m referring to small stucknesses like you’re like trying to design something and

02:02:39 it’s a dead end basically little dead ends dead ends of programming dead ends and trying to think

02:02:44 through something and then your mind wants to like like like uh this is the problem this like

02:02:51 work life balance culture is like take a break like as if taking a break will solve everything

02:02:58 sometimes it solves quite a bit but like sometimes you need to sit in the stuckness and suffer a

02:03:02 little bit and then take a break but you you definitely need to be this and like most people

02:03:08 quit from that psychological battle being stuck so success is people who who who uh persevere

02:03:17 through that yeah yeah and and in the creative process that’s also true i was the other day i

02:03:22 was i think was reading about is this um what is his name ed sheeran like the musician yeah was

02:03:28 talking a little bit about the creative process and using was using this analogy of a faucet like

02:03:33 where you when you turn on a faucet is as like the dirty water coming out in the beginning and you

02:03:38 just have to you know keep trusting that at some point your clean clean clear water will come out

02:03:45 but you have to endure that process like in the beginning it’s going to be dirty water and and and

02:03:50 just you know embrace that yeah actually this uh the entirety of my youtube channel and this

02:03:56 podcast have been following that philosophy of dirty water like i’ve been you know i do believe

02:04:02 that like you have to get all the crap out of your system first and uh sometimes it it’s it’s all

02:04:08 sometimes it’s all crappy work i tend to be very self critical but i do think that quantity leads

02:04:14 to quality for some people it does for my the way my mind works is like just keep putting stuff out

02:04:20 there keep creating and uh the quality will come as opposed to sitting there waiting not doing

02:04:28 anything until the thing seems perfect because the perfect may never come but just just on like on

02:04:35 on on our twitter like profile i really and sometimes when you look on some of those tweets

02:04:40 they might seem like pretty kind of um you know why is this interesting it’s like so raw uh like

02:04:47 it’s just a number but i really believe that especially with math or physics it is possible

02:04:53 to get everyone to love math or physics even if you think you hate it it’s it’s not a function of

02:04:59 the student or the person that is on the other side i think is just purely a function of like

02:05:04 how you explain uh hidden beauty that they hadn’t realized before it’s not easy but i think it’s

02:05:11 like a lot of the times it’s on like on the creator’s side to to be able to like show that

02:05:16 beauty to the other person i think some of that is native to to humans we just have that curiosity

02:05:22 and you look at small toddlers and babies and like them trying to figure things out and there’s

02:05:28 just something that is born with us that we we we want for that understanding of the world

02:05:32 we want for that understanding we want to figure out the world around us and and so yeah it shouldn’t

02:05:38 be like uh whether or not people are going are going to to enjoy it like i i i also really

02:05:45 believe that everybody has that capacity to fall in love with with math and physics you mentioned

02:05:51 startup what do you think it takes to build a successful startup yeah that it’s what what

02:05:58 luis was saying that um you need to in to be able to endure being stuck and and i think the best way

02:06:06 to put it is that startups don’t have a linear reward function right you oftentimes don’t get

02:06:14 rewarded for effort and and and most of our lives we go through these processes that do

02:06:22 give you those small rewards for effort right in school you study hard generally you’ll get a good

02:06:27 grade and then you good you get like good grades ever or you get grades every semester and so you’re

02:06:33 slowly getting rewarded and pushed in the right direction for for startups and startups are not

02:06:40 the only thing that is like this but for startups it’s you know you can put in a ton of effort into

02:06:45 something that and then get no reward for it right it’s it’s like like sisyphus boulder where you’re

02:06:51 pushing that boulder up the mountain and and and you get to the top and then it just rolls all the

02:06:57 way back down and and so that’s something that i think a lot of people are not equipped to deal

02:07:02 with and can be incredibly demoralizing especially if that happens more than than a few times and so

02:07:10 but i think it’s absolutely essential to to power through it because um by the nature of startups

02:07:16 it’s oftentimes you know you’re dealing with with with non obvious ideas and things that

02:07:23 there might be contrarian and so you’re gonna you’re gonna run into into that a lot you’re

02:07:28 gonna do things that are not gonna work out and you need to be prepared to deal with that but

02:07:34 but if we’re not coming out of college you’re you’re just not equipped i’m not sure if there’s

02:07:38 a way to train people to deal with those nonlinear reward functions but it’s definitely i think one

02:07:45 of the most difficult things to you know about doing a startup and also happens in research

02:07:51 sometimes you know we’re talking about the default state is being stuck you just you know you don’t

02:07:56 like you try things you get zero results you close doors you constantly closing doors until you you

02:08:02 know find something and um yeah that is a big thing what about sort of this point when you’re

02:08:08 stuck there’s a kind of decision whether if you have a vision to persist through with this direction

02:08:15 that you’ve been going along or what a lot of startups do or businesses is pivot how do you

02:08:21 decide whether like to give up on a particular flavor of the way you’ve imagined the design

02:08:29 and to like adjust it or completely like alter it i think that’s a core question for startups that

02:08:38 i’ve asked myself exactly and like i i’ve never been able to come up with a great framework to

02:08:44 make those decisions um i think that’s really at the core of uh yeah out of a lot of the the

02:08:51 toughest questions that that people that’s that started a company you have to deal with um i think

02:08:57 maybe the best framework that i i have was able to figure out like when you run out of ideas

02:09:04 you just you know you’re exploring something is not working you try it in a different angle uh

02:09:09 you know we try a different business model when you run out of ideas like you don’t have any more

02:09:14 cards just switch and yeah it’s not perfect because you also it’s you have a lot of stories

02:09:22 of startups is like people kept pushing and then you know that paid off and then you have uh

02:09:29 philosophies is like fail fast and pivot fast um so it’s you know it’s hard to you know balance

02:09:36 these two worlds and understand what is the best framework and i mean if you look at for miles

02:09:42 library you’re maybe you can correct me but it feels like you’re an operating in a space

02:09:48 where there’s a lot of things that are broken and or could be significantly improved so it

02:09:53 feels like there’s a lot of possibilities for pivoting or like how do you revolutionize science

02:10:00 how do you revolutionize the aggregation the the annotation the commenting the community around

02:10:08 information of knowledge structured knowledge i mean that’s kind of what like stack overflow

02:10:13 and stack exchange has struggled with to come up with a solution and they’ve come up i think with

02:10:19 an interesting set of solutions that are also i think flawed in some ways but they’re much much

02:10:23 better than the alternatives but there’s a lot of other possibilities if we just look at papers as

02:10:29 we talked about there’s so many possible revolutions and they’re a lot of money to be

02:10:33 potentially made those revolutions plus coupled with that the benefit to humanity and so like

02:10:39 you’re sitting there like i don’t know how many people are legitimately from a business perspective

02:10:45 playing with these ideas it feels like there’s a lot of ideas here true it varies are you right

02:10:50 now grinding in a particular direction like is there a video like a five year vision that you’re

02:10:55 thinking in your mind for us it’s more like a 20 year vision in the sense that uh we we’ve

02:11:03 consciously tried to make the decision of so we so we run formats as it’s a side project and it’s

02:11:10 separate in the sense like it’s not what we’re working on full time and uh but our thesis there

02:11:18 is that we actually think that it’s that’s a good thing at least for for this stage of formats

02:11:24 library um and also because some of these projects you just if you’re coming from a start from a

02:11:32 startup framework you probably try to try to fit every single idea into something that can change

02:11:40 the world within three to five years and there’s just some problems that take longer than that

02:11:45 right and so you know we’re talking about archive and i’m very doubtful that you could grow like

02:11:51 archive into what it is today like within two or three years no matter how much money you throw at

02:11:56 it there’s just some things that can take longer but you need to be able to power through the the

02:12:02 that the time that it takes um but if you look at it as okay this is a company this is a startup

02:12:08 we have to grow fast we have to raise money then uh then sometimes you might forego those ideas

02:12:14 because of that um because they don’t very well fit into the the typical startup framework and

02:12:22 so for us formats it’s something that we’re okay with growing with having it grow slowly and and

02:12:27 maybe taking many years and and and that’s why we think it’s it’s not a bad thing that it is a side

02:12:34 project because it makes it much more um acceptable in a way and that to to be able to be okay with

02:12:41 that that said i think what happens is if you keep pushing new little features new little ideas

02:12:48 i feel like there’s like certain ideas will just become viral like and then you just won’t be able

02:12:54 to help yourself but it’ll revolutionize things it feels like there needs to be not needs to be

02:13:00 but there’s um opportunity for viral ideas to change science absolutely and maybe we don’t know

02:13:08 what those are yet it might be a very small kind of thing maybe you don’t even know if should this

02:13:13 be a for profit company doing these it’s the wikipedia question yeah um is that there are a lot

02:13:19 of questions like really fundamental questions about this space um that we’ve we’ve talked about

02:13:24 i mean you take wikipedia and you try to run it as a startup and by now we’d have a paywall you’d be

02:13:29 paying 9.99 a month to to read more than 20 articles i mean that’s that’s one view yeah the

02:13:34 other the ad driven model so they rejected the ad driven model i don’t know if we could i mean this

02:13:41 is a difficult question you know if archive was supported by ads i don’t know if that’s bad for

02:13:48 archive if for mass library was supported by ads i don’t know i don’t i’m not it’s not trivial to

02:13:55 me i’m unlike i think a lot of people uh i’m not against advertisements i think ads when done

02:14:02 well are really good i think the problem with facebook and all the social networks are the way

02:14:07 the lack of transparency around the way they use data and uh the lack of control the users have

02:14:12 over their data not the fact that data is being collected and used to sell advertisements it’s a

02:14:18 lack of transparency lack of control if you if you do a good job with that i feel like it’s really

02:14:23 nice way to make stuff free yeah it’s like stack overflow right i mean i think they’ve done a okay

02:14:29 a good job with that uh even though as we said like they’re capturing very little of the value

02:14:33 that they’re putting out there right but but it makes it a sustainable company and and they’re

02:14:39 providing a lot of it’s a fantastic and very productive community let me ask a a ridiculous

02:14:46 tangent of a question please you wrote a paper on uh on game of thrones battle of winterfell just

02:14:53 as a side little i i’m sorry i noticed i’m sure you’ve done a lot of ridiculous stuff like this i

02:15:02 just noticed that particular one by ridiculous i mean ridiculously awesome can you describe the uh

02:15:08 the approach in this work which i believe is a legitimate publication so going back to the

02:15:14 original like uh when we were talking about the backstory of of papers and the importance of that

02:15:18 you know there was a when the last season of the the show was airing uh this was a during a company

02:15:25 lunch we there was in in the last season there’s the there’s a really big battle against the the

02:15:32 forces of evil and the you know the forces of good and it’s called the battle of winterfell

02:15:39 and um in this battle there are like these two armies and there’s a very particular thing that

02:15:46 they have to take into account is that in the army of dead like if someone dies in the army of

02:15:51 the living uh like that person is gonna you know be a reborn as a soldier in the army of the dead

02:15:59 yes and so that was a an important thing to take into account and the initial conditions as you

02:16:04 specify it’s about a hundred thousand on each side exactly so i was able i was able to like based on

02:16:08 some images or like on previous episodes to figure out what was the size of the armies and so what i

02:16:13 want what we wanted to what we were theorizing was like how many soldiers does like a soldier

02:16:19 on the army of the living has to kill in order for them to be able to destroy the army of the dead

02:16:26 without like losing because every time one of the good soldiers dies gonna turn into like the other

02:16:33 side and so it’s so i we we were theorizing that and and i wrote wrote a couple of differential

02:16:39 equations and i was able to figure out that based on the size of the armies i think i think was the

02:16:43 ratio had to be like 1.7 so it had to kill like 1.7 soldiers like the army of the dead in order

02:16:50 for them to win the battle well yeah that’s that’s science it is it’s it’s the most powerful

02:16:58 and this is also somehow a pitch for uh like a hiring pitch in a sense like this is the kind of

02:17:05 yeah important science you do exactly well turned out to be you know as as for people that have

02:17:11 watched these shows is like they know that every time you try to predict something that is going

02:17:16 to happen it’s going to you’re going to fail miserably and that’s what happened so it was not

02:17:20 not at all important for the show but yeah we ended up like putting that out and there was a

02:17:26 lot of people that share that i think was some like elements of the of the show the cast of the

02:17:30 show that actually retweeted that and shared that that first one was fun i would love if this kind

02:17:35 of calculation happened uh like during the making of the show or you know i love it like in um

02:17:44 for example i i now know uh alex garland the director of ex machina and i love it he doesn’t

02:17:51 seem to be some not many people seem to do this but i love it when directors and people who wrote

02:17:58 the story really think through the technical details like whether it’s knowing like how things

02:18:04 even if it’s science fiction if you were to try to do this how would you do this uh like

02:18:10 stephen wolfram and his son were um were collaborating with the movie arrival in

02:18:16 designing the alien language how you communicate with aliens like how would you really have

02:18:21 uh a math based language that uh that could span the alien and uh being and the human being so i

02:18:31 i love it when they have that kind of rigor the martian was also big on that like the book and

02:18:35 the movie was all about like can we actually you know is this plausible can this happen it was all

02:18:41 about that and that can really bring you in like the sometimes those small details uh i mean the

02:18:46 i mean the the guy that wrote the martian book is another book uh that is also filled with those

02:18:52 like things that when you realize that okay these are grounded in in science can just really bring

02:18:58 you in yeah like the like the he has a book about a colony on the colony on the moon and he goes

02:19:04 about like all the details that would you know be required about setting up a colony in the moon

02:19:09 and like things that he wouldn’t think about like the the fact that um they would you know it’s

02:19:14 hard to bring like uh air to the moon say so they wouldn’t like how do you make that breathable that

02:19:21 environment breathable you need to bring oxygen but like you you you probably wouldn’t be bring

02:19:26 nitrogen so what you do is like instead of having a an atmosphere that is 100 oxygen you like

02:19:34 decrease the pressure so that you have the same ratio of oxygen on earth but like lowering the

02:19:39 pressure here and so like things like water boils at the lower temperature so people would would

02:19:45 have coffee and the coffee would be colder like there was a problem in this uh environment in

02:19:50 the moon so like and these are like small things in the book but i studied physics so like when i

02:19:57 read these like that throws me into like uh tangents and i start researching that and it’s

02:20:04 like i really like to read books and watch movies when they go to that level of detail uh about

02:20:11 science yeah i think interstellar was one where they also consulted heavily with with a number of

02:20:15 yeah i think even resulted in a couple of papers a couple of papers about like the black hole

02:20:20 um visualizations and um yeah but there isn’t and there’s even more examples of interesting science

02:20:27 around like these fantasy we were reading at some point like these guys that were trying to figure

02:20:33 out if if the tolkien’s middle earth if it was uh round if it was like a sphere yeah it’s like a

02:20:42 flat based on the map based on the map and some of the references in the in the books and so uh

02:20:50 yeah we actually i think we tweeted about that you can yeah we did based on the distance between the

02:20:54 cities you can actually prove that that could be like a map of a sphere or like a spheroid

02:21:00 and and you can actually calculate the radius of that planet uh that’s fascinating i mean yeah

02:21:09 that’s fascinating but there’s something about like calculating the number like exactly the

02:21:16 calculation you did for the battle winterfell is um something fascinating about that because

02:21:23 that’s not like being that’s very mathematical versus like grounded in physics and that’s really

02:21:30 interesting i mean that’s like injecting mathematics into fantasy there’s there’s something um i see

02:21:38 what you’re saying about that and and that for me that’s why i think it’s also when you look at

02:21:42 things like like Fermat’s last theorem like problems that are very kind of self contained

02:21:49 and simple to study i think like that’s the same with that paper it’s very easy to understand the

02:21:54 boundaries of the problem you know um and and that for me that’s why those and that’s why math is so

02:22:01 appealing and those like problems are also so appealing to the general public it’s not that

02:22:06 they look simple or that people think that they are easy to like solve but i feel that a lot of

02:22:12 the times they are almost intellectually democratic because everyone understands the starting point

02:22:18 you know you look at Fermat’s last theorem everyone understands like is this is the the

02:22:23 universe of the problem and the same maybe with that paper everyone understands okay these are

02:22:27 the starting conditions and um and and yeah that the fact that it becomes intellectually democratic

02:22:34 and i think that’s a huge motivation for people and that’s why so so many people gravitate towards

02:22:39 these like Riemann hypotheses or Fermat’s last theorem or that simple paper which is like just

02:22:44 one page it was very simple and i just talked to somebody i don’t know if you know who he is

02:22:48 Jocko Willink who was uh this person who among many things loves military tactics so he would

02:22:58 probably either publish a follow on paper maybe you guys should collaborate but he would see the

02:23:04 fundamental the basic assumptions that you started that paper with is flawed because you know there’s

02:23:09 like dragons too right there’s like like you have to integrate tactics because not it’s not it’s not

02:23:15 a homogeneous system it’s not i don’t take into account the dragons and like and he would say

02:23:20 tactics fundamentally change the dynamics of the system and so like that’s what happened

02:23:27 so uh yeah so at least from a scientific perspective he was right but he never published

02:23:32 so there you go uh let me ask the most important question you guys are from

02:23:36 um portugal both yeah portugal uh so who is the greatest soccer player footballer of all time

02:23:45 yeah i think we’re a little bit biased on this topic but i i mean i don’t know i i have i have

02:23:52 a huge i have a you know tremendous respect for for what um here we go this is the political

02:23:59 you can convince you i i i mean i have tremendous respect for what ronaldo has achieved in his

02:24:05 career and and i think soccer is one of those sports where i think you can get to maybe be one

02:24:10 of the best players in the world we if you just have like natural talent and even if you don’t put

02:24:16 a lot of hard work and discipline into soccer you can be one of the best players in the world

02:24:22 and i think ronaldo is kind of like of course he’s naturally talented but he also ronaldo should say

02:24:27 the the football from exactly from portugal um and and not uh not the brazilian in this case

02:24:33 and so um and ronaldo put like came from nothing he is known from being probably one of the hardest

02:24:39 working athletes in the game and and i see that sometimes a lot of these discussions about the

02:24:44 best player a lot of people tend to gravitate towards like um you know this person is naturally

02:24:51 talented and the other person has to work hard and so and so as if it was bad if he had to work

02:24:58 hard to to be good at something and i think that you know the the i think so many people fall into

02:25:04 that trap and the reason why so many people fall into that trap is because if you’re saying that

02:25:10 someone is good and achieved a lot of success by working hard as opposed to achieving success

02:25:17 because he has some sort of god given natural talent that you can’t explain why the person was

02:25:22 born with that what does it tell you about you it tells you that maybe if you work hard on a lot of

02:25:28 fields you could have could accomplish a lot of great things and i think that’s hard to digest

02:25:33 for a lot of people and and in that way ronaldo’s inspiring that i think so you find hard work

02:25:39 inspiring but he’s he’s way too good looking that’s that’s the yeah i don’t like him no i

02:25:45 like the part of the hard work and like of him being like one of the hardest working athletes

02:25:50 in in soccer so he is to you the greatest of all time is he up there is he would be number

02:25:56 okay do you agree with this thing well i definitely disagree i mean i i like him very much he works

02:26:03 hard i admire i admire you know um what like he’s incredible uh goal scorer right um but i

02:26:15 i so first of all leo messi and there was some confusion because i’ve kept saying maradona is

02:26:22 my favorite player but i i think i think leo has surpassed them so uh um it’s messy then maradona

02:26:30 then pelé for me but the the reason is is um there’s certain aesthetic definitions of beauty

02:26:39 that i admire whether it came by hard work or through god given talent or through anything and

02:26:45 it doesn’t it doesn’t really matter to me there’s certain aesthetic like genius when i when i see it

02:26:51 to me and uh especially it doesn’t have to be consistent it is in the case of messi in case

02:26:57 in the ronaldo but just even moments of genius which is where maradona really shines it i even

02:27:04 if that doesn’t translate into like results and goals being scored right right and that’s the

02:27:09 challenge like they did that uh because that’s where people that tell me that leo messi’s never

02:27:18 even on strong teams have led his the national team people as far as the world cup right as

02:27:24 really important and to me no it’s the moment like winning to me was never important what’s

02:27:30 more important is the moments of genius and but you’re you’re talking to the human story and

02:27:40 yeah christiano ronaldo definitely has a beautiful human story yeah and i think you can’t

02:27:44 i for me it’s hard to decouple those two um i don’t i don’t just look at you know the the

02:27:49 list of achievements but i like how he got there and how he keeps pushing the boundaries at like

02:27:54 almost 40 yeah and how that sets up an example like maybe 10 years ago i wouldn’t have ever imagined

02:28:00 that like one of the top players in the world could be a top player at like 37 or but so and

02:28:06 there’s an interesting tent the human story is really important but like if you look at ronaldo

02:28:11 he’s like he’s somebody like kids could aspire to be but at the same time i also like maradona who

02:28:18 like is a is a tragic figure in many ways is like the you know the drugs the the temper

02:28:26 all of those things that’s beautiful too like i don’t necessarily think to me first the flaws

02:28:31 are beautiful too in in athletes i don’t think you need to be perfect i agree uh from a personality

02:28:39 perspective those flaws are also beautiful so but yeah there is something about hard work and

02:28:47 uh there’s also something about the being an underdog and being able to carry a team

02:28:53 uh that’s that’s an argument for maradona i don’t know if you can make that argument for

02:28:57 messi and ronaldo either because they’ve all played on superstar teams for most of their lives

02:29:04 um so i don’t know how it you know it’s it’s difficult to know how they would do um when they

02:29:12 had to work like did what maradona had to do to carry a team on his shoulders true and pelle did

02:29:19 as well and depending on the the context yeah maybe you could argue that with the portuguese

02:29:24 national team but then we have a good team uh yeah but maybe what maradona did with you know

02:29:30 lap naples and and a couple other teams it’s it’s incredible it speaks to the beauty of the game that

02:29:35 you know we’re talking about all these different players that have or especially you know if you’re

02:29:39 if you’re comparing messi and ronaldo that have such different you know styles of play and also

02:29:45 even their bodies are so different and and and but these two very different players can be at the top

02:29:52 of the game and that’s not that’s the there are not a lot of other sports where you where you have

02:29:58 that you know like you have kind of a mental image of a basketball player and like the the top

02:30:04 basketball players kind of fit that mental image and they look a certain way and um but for soccer

02:30:11 there’s some there’s the it’s it’s not so much like that and and that’s i think that’s that’s

02:30:17 beautiful uh but that really adds something to the sport well do you play soccer yourself

02:30:23 have you played that in your your life what do you find beautiful about the game yeah i mean it’s one

02:30:27 of the i’d say it’s the biggest sport in portugal and so growing up we played a lot did you see the

02:30:33 paper from deep mind i didn’t look at it where they’re like uh doing some uh analysis on soccer

02:30:39 strategy interesting i i saved that paper uh i haven’t read it yet um it’s actually i i when i

02:30:46 was in college i actually did some research on on applying um machine learning and statistics in

02:30:54 sports and in our case in our case we’re doing it for basketball um but uh what they’re effectively

02:31:02 trying to do was have you ever watched moneyball like so they’re trying to do something similar

02:31:08 right taking that in this case basketball taking a statistical approach to to to basketball um

02:31:15 the interesting thing there is that baseball is much more about having these discrete events that

02:31:20 happen kind of in similar conditions and so it’s easier to take a statistical approach to it whereas

02:31:25 basketball it’s a much more dynamic game uh it’s harder to measure um it’s hard to to replicate

02:31:33 these conditions and so you you have to think about it in a slightly different way and so we

02:31:39 were doing work on that and working like with the celtics to analyze the the data that they had like

02:31:44 they had these cameras in the in the arena they were tracking the players and so you so they had

02:31:49 a ton of data but they didn’t really know what to do with it and so we we were doing work on that

02:31:54 and and and soccer is maybe even a step further it’s it’s right it’s a game where you don’t have

02:31:59 as many in basketball you have a lot of field goals and so you can measure success uh soccer

02:32:05 it’s it’s right it’s more of a process almost where it’s like you have a goal like or two in

02:32:11 in a game in terms of metrics i wonder if there’s a way and i’ve actually have thought about this

02:32:15 in the past never coming up with any good solution if there’s a way to definitively say whether it’s

02:32:20 messy or not they’re the greatest of all time like like honestly sort of measure interesting

02:32:25 like convert the game of soccer into metrics like you said baseball but like those moments

02:32:30 of genius like past like um you know if it’s just about goals or passes that led to goals

02:32:36 yeah that feels like it doesn’t capture the genius of the play yeah they’ll be like you know like

02:32:43 like you kind of do you have more metrics for instance in chess right and you can try to

02:32:48 understand how hard of a move that was you know there’s like bobby fisher has this move that like

02:32:54 that it’s i think it’s called the move of the century where uh you have to go so deep into the

02:33:00 tree to understand that that was the right move and you can quantify how hard it was uh so it’d

02:33:05 be interesting to try to think of those type of metrics but say yeah for soccer and computer

02:33:09 vision unlocks some of that for us that’s that’s one possibility i have a cool idea a computer

02:33:14 vision product legs that you could build for soccer let’s go i’m taking notes if you could

02:33:20 detect the ball and like imagine that um it seems like totally doable right now but like if you could

02:33:27 detect when the ball enters one of the goals and like just had like um you know a crowd cheering

02:33:33 for you when you’re playing soccer with your friends every time you score a goal or you had

02:33:37 like the the champions league song going on yeah and like having that like you go play soccer with

02:33:42 your friends just turn that on and there’s like a computer vision like program analyzing the ball

02:33:46 detects the ball every time there’s a goal like if you miss like there’s a you know the fans are

02:33:50 reacting to that and then it should be pretty simple by now it’s like i think there’s an

02:33:55 opportunity yeah just throwing that i’m gonna go all out but by the way i did uh i’ve never

02:34:01 released i was thinking of just putting on github but i did write exactly that which is the trackers

02:34:05 for the players uh for the for the bodies of the player is this is the hard part actually

02:34:10 uh the detection of player bodies and the ball is not hard what’s hard is very like robust tracking

02:34:18 through time of each of those so like so i wrote a track of this pretty damn good this is this is

02:34:25 that is that open source you open so i know i’ve never released it because interesting because i

02:34:29 felt like i need to i would this is the perfection thing because i knew it was going to be like

02:34:36 it’s gonna pull me in and and it wasn’t really that done and so i’ve never actually been part

02:34:42 of a github project where it’s like really active development and i didn’t want to make it i knew

02:34:47 there’s a nonzero probability that it will become my life for like a half a year that uh could just

02:34:52 how much i love soccer and all those kinds of things and and ultimately it will be all for just

02:34:58 the the joy of analyzing the game which i’m all for i remember you also like one of in one of the

02:35:03 episodes you mentioned that you did also a lot of high tracking analysis on like joe rogan’s that

02:35:08 was the that was the research side of my life interesting yeah and you have that library right

02:35:12 you you kind of downloaded all the episodes yep allegedly i and of course i didn’t if you’re a

02:35:18 lawyer i’m listening to this no it is yeah i i was listening to the episode where you mentioned

02:35:21 that and i was actually there was something that i and i might ask you for for access to that to

02:35:27 like allegedly that library uh but i was doing some not not regarding like eye tracking but i was

02:35:33 playing around with um analyzing the distribution of silences on uh one of the joe rogan episodes

02:35:40 so like i did that for the elon uh conversation where it’s like you just take all the silences

02:35:47 like after joe asked the question and elon responded and you plot that distribution and

02:35:52 like and see how how how that looks like yeah i think there’s a huge opportunity especially

02:35:57 long form podcasts to do that kind of analysis bigger than joe exactly but it has to be a fairly

02:36:04 unedited podcast so that you don’t get the silence so one of the benefits i have like doing this

02:36:10 podcast is like the what we’re recording today is there’s individual audio being recorded makes it

02:36:17 so like i have the raw information it’s when it’s published it’s all combined together and individual

02:36:22 video feeds so even when you’re listening which i usually don’t i only show one video stream

02:36:27 i i’ll know i can track your blinks and so on um but yeah but ultimately the hope is you don’t

02:36:35 need that raw data because if you don’t need the raw data for whatever analysis you’re doing

02:36:39 you can then do a huge number of parts because there’s so it’s quickly growing now the number

02:36:44 especially comedians there’s uh quite a few comedians with with long form podcasts and

02:36:51 they have a lot of facial expressions they have a lot of fun and all those kinds of things and

02:36:54 it’s it’s prone for analysis and it’s there’s so many interesting things that that that idea

02:37:01 actually sparked because i was watching a um a q a by by steve jobs and i think it was at mit and

02:37:08 then like people’s like he did a talk there and then the q a started and people starting asking

02:37:13 questions that i was i was working while listening to it and like someone asked the question and he

02:37:18 goes like on a 20 second silence before answering the question i like i had to check if the if the

02:37:24 video hadn’t paused or or something and and i was thinking about like like if that is a feature of

02:37:30 a person like how long on average you take to respond to a question and if it’s like oh that’s

02:37:36 that has to do with the like how thoughtful you are and if that changes over time oh but it also

02:37:41 could be this really fascinating metric because it also could be it’s certainly a feature of a

02:37:46 person but it’s also a function of the question like if you normalize to the person you can

02:37:52 probably infer a bunch of stuff about the question so it’s a nice flag like it’s a really strong

02:37:56 signal the length of that silence but relative to the usual silence they have so one the silence is

02:38:03 a measure of how thoughtful they are and two the particular silence is a measure how thoughtful

02:38:08 the question was thoughtful the question was it’s really interesting i mean yeah yeah i just

02:38:13 analyzed elon’s uh episode but i think there’s like room for exploration there i feel like the

02:38:19 average they could do for comedians would be like i mean the time would be so small because you’re

02:38:24 trained to like i would i would think you’re reacting to hecklers you’re reacting to all

02:38:28 sorts of things you have to be like so quick maybe right yeah but some of the greatest comedians are

02:38:33 very good at sitting in the silence i mean there there’s louis ck they play with that because you

02:38:40 have a rhythm and like um dave chappelle a comedian who did a joe show recently he has uh

02:38:50 especially when he’s just having a conversation he does long pauses it’s kind of cool because it

02:38:55 uh it it’s one of the ways to have people hang in your word is to play with the pauses to play

02:39:03 with the silences and the emphasis and like mid sentence there’s a bunch of different things that

02:39:09 uh it’d be interesting to really really analyze but still soccer to me is uh that that one’s

02:39:15 fascinating just i just want a conclusive definitive statement about because like there’s so

02:39:20 many soccer highlights of both messi and ronaldo i just feel like the raw data is there

02:39:29 um because you don’t have that with pelé and mardona just yeah true but here’s a huge amount

02:39:38 of high dev data then the the annoying the difficult thing and this is really hard for

02:39:43 tracking and this is actually where i kind of gave up because i didn’t really give much effort

02:39:48 but i gave up to the the way that highlights or usually football match filmed is they switch

02:39:56 to camera so they’ll they’ll do a different switch perspective so you have to it’s a really

02:40:02 interesting computer vision problem when the perspective is switched you still have a lot of

02:40:06 overlap about the players but the perspective is sufficiently different that you have to like

02:40:10 recompute everything so i there there’s two ways to solve this so one is doing it the full way where

02:40:21 you’re constantly doing the slam problem you you’re doing a 3d reconstruction the whole time

02:40:25 and projecting into that 3d world but you could also there could be some hacks that i wonder like

02:40:32 some trick where you can hop like when the perspective shifts do a high probability

02:40:39 from one object to another but i i thought especially in exciting moments when when when

02:40:47 you’re passing players like you’re doing a single ball dribble across players and you switch

02:40:53 perspective which is when they often do when you’re making a run on goal if you switch your

02:40:57 perspective it’s it feels like that’s going to be really tricky to get right uh that’s

02:41:02 automatically but in that case for instance i feel like if somebody released that data set

02:41:06 or it’s like you just have all like these this data set a massive data set of all these games

02:41:12 from from say ronaldo and messy like and just you just add that in like whatever csv format and some

02:41:19 some publicly available data set like that i feel like people would just there there would be so

02:41:24 many cool things that you could do with it and you just set it free and then like the world would

02:41:28 like do its thing and then like interesting things would come out of it by the way i have this data

02:41:34 set so the two the two things i’ve did of this scale uh is soccer so his body pose and ball

02:41:40 tracking for soccer and then um i try it’s pupil tracking and blink tracking for it was joe rogan

02:41:48 and a few other podcasts that i did so those are the two data sets i have did you analyze any of

02:41:53 your podcasts no i i think i really started doing this podcast after after doing that work and it’s

02:42:03 difficult to maybe i’d be afraid of what i find i’m already annoyed with my own voice and video

02:42:12 like editing it uh but perhaps that’s the honest thing to do because uh one useful thing i about

02:42:18 doing computer vision about myself is like i know what i was thinking at the time so you can start

02:42:24 to like connect the particular the behavioral peculiarities of like the way you blink the way

02:42:32 you squint the way you close your eyes like talking about details there’s it’s like for example i just

02:42:39 closed my eyes is that a blink or no like figuring that out in terms of timing in terms of the link

02:42:46 dynamics it’s tricky it’s very doable i i think there’s universal laws about what is a blink and

02:42:53 what is a closed eye and all those things plus makeup and eyelashes i actually um have annoyingly

02:43:00 long eyelashes so i remember when i was doing a lot of this work i i would cut off my eyelashes

02:43:06 which when like especially it was funny like female colleagues were like what the fuck are

02:43:11 you doing like those no keep the eyelashes but because it got in the way made the computer

02:43:16 vision a lot more difficult but super interesting topics yeah but speaking about the one uh still

02:43:23 on the topic of the data sets for sports there’s one um one paper that and i actually annotated

02:43:28 on format and and uh it’s it was published in 90s 90s i believe 90s or 80s i forget but it would

02:43:37 they you the researcher was effectively looking at the hot end phenomena in basketball right so

02:43:45 whether like the fact that you just made a field goal um if you know if on your next attempt if

02:43:52 you’re more likely to make it or not um and it was super interesting because they i mean he polled

02:43:59 like i think 100 undergrads and i think from stanford and cornell and asking people like do

02:44:04 you you think that’s that you have a higher likelihood of making your free throw if you just

02:44:09 just made one and i think it’s like 68 68 percent said yes they believe that and then he looked

02:44:16 at the data and this was back in as i said like a few decades ago and so i think he had the data

02:44:22 set of about uh he looked at it specifically for free throws and he had a data set of about 5,000

02:44:28 free throws and um and effectively what he found was that specifically in the case of free throws

02:44:37 he didn’t for the aggregate data he didn’t find um that he couldn’t really spot that correlation

02:44:44 that hot end correlation so if you made the first one you weren’t more likely to to make the second

02:44:50 one what he did find was that they were just better at the second one because you just got

02:44:55 like maybe a tiny practice and you just attempted once and then and then you’re going to be better

02:45:00 at the next one and then i i then i went and there’s a data set on kaggle that has like 600,000

02:45:06 free throws and i reran the the same computations and and and confirmed like you can see a very

02:45:13 clear pattern that they’re just better at their second free throw um that’s interesting because

02:45:18 i think there’s similarly that kind of analysis is so awesome because i think with tennis they

02:45:23 have like uh like a fault like when you serve they have analysis of like are you most likely

02:45:29 to miss the second serve if you missed the first obviously um yeah i think that’s the case so that

02:45:34 integrates that’s so cool when psychology is converted into metrics in that way and in sports

02:45:40 sports it’s especially cool because it’s such a constrained system that you can really study

02:45:46 human psychology because it’s repeated it’s constrained so many things are controlled which

02:45:51 is something you rarely have in in the wild psychological experiments so it’s cool uh plus

02:45:59 everyone loves it like sports is really cool to analyze people actually care about the results

02:46:04 yeah um i still think well like i uh i and i will definitely publish uh this work on messy

02:46:11 versus ronaldo and i’d love to read it objective fully objective to peer review

02:46:18 um yeah this is very true this is not past period

02:46:22 um let me ask sort of um an advice question to uh to young folks you’ve explored a lot of

02:46:30 fascinating ideas in your life you built a startup worked on physics worked on computer science

02:46:37 what advice would you give to young people today in high school maybe early college about life

02:46:43 about career about science and mathematics i remember like uh i read like i remember reading

02:46:52 that um punk area was once asked by a um a french journal about his advice for young people and what

02:47:00 was his teaching philosophy and he said that like one of the most important things that parents

02:47:05 should teach their kids is how to be enthusiastic um in regards to like the mysteries of the world

02:47:12 and that he said like striking that balance was actually one of the most important things between

02:47:17 like in education you know you want to have your kids be enthusiastic about the mysteries of the

02:47:21 world but you also don’t want to traumatize them like if you really force them into something

02:47:25 and i think like especially if you’re young i think you should be curious and i think you should

02:47:35 explore that curiosity to the fullest to the point where you even become almost as an expert

02:47:41 on that topic and now and you might start with something that it’s small like you might start

02:47:47 with you know you’re interested in numbers and how to factor numbers into primes and then all of a

02:47:51 sudden you go and and you’re like lost in number theory and you discover cryptography and then all

02:47:57 of a sudden you’re buying bitcoin and i and i think you should do this um you should really try

02:48:03 to fulfill this curiosity and you should live in a society that allows you to fulfill this curiosity

02:48:08 which is also important and i think you should do this not to get to some sort of status or fame or

02:48:14 money but i think this is the way this iterative process i think this is the way to find happiness

02:48:20 and and i think this is also allows you to find the meaning for your life i think it’s all about

02:48:26 like being curious and being able to fulfill that curiosity and that path to fulfilling that your

02:48:33 curiosity yeah the the start small let the fire build this kind of interesting way to think about

02:48:39 it and you never know where you’re going to end up it’s it’s like for us is just a really good

02:48:44 example we started like by doing this as an internal like thing that we did in the company

02:48:50 and then we started putting out there and now a lot of people follow it and know about it and so

02:48:56 and you still don’t know where from our library is going to end up actually true exactly so um

02:49:01 yeah i think that would be my piece of advice with very limited experience of course but yeah yeah i

02:49:07 agree i agree uh i mean is there something in from particular joao from the computer science

02:49:14 versus physics perspective uh do do you regret not doing physics do you regret not doing computer

02:49:20 science which one is the the wiser the better human being this is messy versus ronaldo um

02:49:27 those are very i i don’t know if you would agree but they’re kind of different disciplines true

02:49:33 yeah very much so um i actually i actually uh i was i had that question in my mind i i took

02:49:42 physics classes as an undergrad or like besides what i had to take and um it’s definitely something

02:49:50 that i considered at some point um and and that i i i do feel like later in life that might be

02:50:01 something that i’m not sure if regret is is the right word but it’s it’s kind of something that

02:50:07 i can imagine in an alternative universe what would have happened if i if i got into physics

02:50:13 um i try to think that like well depends on what your path ends up being but that it’s it’s not

02:50:22 super important right like exactly what you decide to major on like i think there’s there’s um

02:50:29 um i think tim urban like the blogger had a good visualization of this where it’s like

02:50:34 you know like he he he has a picture where you have all sorts of paths that you could pursue in

02:50:39 your life and then maybe you’re in the middle of it and so there’s maybe some paths that are not

02:50:43 accessible to you but like the the tree that is still in front of you gives you a lot of optionality

02:50:48 and so um there’s two lessons to learn from that like we have a huge number of options now

02:50:53 and probably you’re just one to reflect like to try to uh derive wisdom from the one little path

02:51:02 you’ve taken so far may be flawed because there’s all these other paths you could have taken yeah so

02:51:07 it’s like uh so one it’s inspiring that you can take any path now and two it’s like you you the

02:51:13 path you’ve taken so far is just one of many possible ones but it does seem that like physics

02:51:20 and computer science both open a lot of doors and a lot of different doors it’s very interesting it

02:51:26 is i i like in this case like and especially in in our case because i could see the difference i

02:51:31 studied i i went to college in europe and joel went to college here in the u.s so i could see

02:51:37 the difference and like in the european system is um more rigid in the sense that when you decide

02:51:43 to study physics you don’t have a lot especially in the early years you don’t have a lot of um

02:51:47 you can’t choose to take like a class from like computer science course or something like that

02:51:52 don’t have a lot of freedom to explore in that sense in university as opposed to here in the

02:51:56 u.s where you have more freedom and i think um i think that’s important i think that’s what

02:52:02 constitutes you know a good kind of educational system is one that gravitates towards the interests

02:52:07 of a student as as you progress but i think in order for you to do that you need to explore

02:52:13 different areas and i i felt like if i had a chance to take say more computer science class

02:52:18 when i was in college i would have probably have taken those classes but um yeah but i ended up

02:52:23 like focusing maybe too like too much in physics and uh i think here at least my perception is

02:52:28 that you can explore more more fields but there is a kind of it’s funny but physics can be difficult

02:52:36 so i don’t see too many computer science people than exploring into physics it’s only like the one

02:52:42 the not the one but one of the beneficial things of physics it feels like it uh what was it rutherford

02:52:51 that said like like basically that physics is the hard thing and everything is easy uh so like

02:52:58 there’s a certain sense once you’ve figured out some basic like physics that it’s not that you

02:53:02 need the tools of physics to understand the other disciplines it’s that you’re empowered by having

02:53:08 done difficult shit i mean the ultimate i think is probably mathematics there yeah true uh so maybe

02:53:15 just doing difficult things and proving to yourself that you can do difficult things whatever those

02:53:20 are that’s net positive i believe net positive yeah and i think like i i before i started company

02:53:25 i had like i i worked in the financial sector for a bit and like i think having a physics background

02:53:32 i was i felt i was not afraid of like learning like finance things and i think like when you

02:53:37 come from those backgrounds you are generally not afraid of stepping into other fields and learning

02:53:42 about those because um yeah i feel we’ve learned a lot of difficult things and um yeah that’s an

02:53:49 added benefit i believe this was uh an incredible conversation louise joao we started with uh who

02:53:58 do we start with fineman ended up with messi and ronaldo so this is like the perfect conversation

02:54:02 it’s really an honor that you guys would waste all this time with me today it was really fun

02:54:07 thanks for talking so much for having us yeah thank you so much thanks for listening to this

02:54:11 conversation with louise and joao batala and thank you to skiff simply safe and thank you

02:54:17 for watching this podcast simply safe indeed net sweet and for sigmatic check them out in the

02:54:24 description to support this podcast and now let me leave you with some words from richard fineman

02:54:30 nobody ever figures out what life is all about and it doesn’t matter explore the world nearly

02:54:37 everything is really interesting if you go into it deeply enough thank you for listening i hope

02:54:43 to see you next time