‘Art is a great area to study luck’

IN 2017, AMERICAN data scientist and economist Seth Stephens-Davidowitz published the book Everybody Lies: What the Internet Can Tell Us About Who We Really Are, which analysed large-scale data sets associated with Google searches. As Stephens-Davidowitz demonstrated through the course of that book—just about everybody is less than honest while filling out standard questionnaires. The unvarnished truth is to be found only in our Google searches (which is why the author wanted to name the book How Big Is My Penis?, a suggestion that his publishers begged to disagree with).

Stephens-Davidowitz is now out with his second book, a follow-up called Don’t Trust Your Gut: Using Data Instead of Instinct to Make Better Choices (Bloomsbury; 320 pages; ₹550). It retains many of his debut’s core qualities — clear-eyed analysis, sharp and humorous observations on hu­man behaviour patterns, and a wide-eyed wonder about the rev­elations that Big Data has to offer. In this book you’ll find entertain­ing insights about a wide range of human endeavours—why do people endure the inevitable heartache that comes with supporting a sports team, why are some artists demonstra­bly luckier than others, how can we optimise the consumption of alcohol and many other rabbit-holes of curiosity.

During a Zoom interview, the New York Times opinion writer spoke about the meaning of luck and the limits of ‘gut instinct’. Excerpts from the conversation:

Your previous book was called Everybody Lies and in it you called Google a “digital truth serum” because it represents pretty much everybody at a higher level of honesty than they usually are. Given how data-driven decision-making as an idea is gaining mainstream popularity today, can we say that everybody lies must necessarily imply everybody fudges data? Have we reached a point in human history where everybody needs to be educated about ‘good’ and ‘bad’ data?

Definitely, I think relying on only data can be dangerous; you can use data to mislead people in various ways. What’s a good study, what’s a good data-set, wheth­er causality has been proven, whether the sample size is large enough—these are all crucial questions when you think about data. Without figuring out these things first, data-driven decision-making can become a tricky exercise.

What are some of the most common data-related biases or fallacies that you see around you, in everyday life?

For one, I think people love shocking conclusions. And often, those shocking conclusions are just wrong. There’s a reason it’s shocking; it doesn’t pass the smell test. Another danger is if you rely too heavily on one study, and that study has a really small sample size. You may read it and become really influenced by it quickly—it transforms your thinking about the world, and that’s really danger­ous. Of course, on the flip side sometimes you might read one study that has such a large data set, with such a compelling methodology that it deserves to be treated with extraordinary respect and attention. Like in the book I talk about this study of American capitalism in the 21st century, which analyses huge data sets from all the taxpayers in the country — now that’s a great study with revolutionary insights about who’s rich and who’s not. But if a couple of professors interviewed 50 people in a lab and claim that they have totally transformed our understanding of how the human mind works…we should be wary about that sort of thing.

In the introduction here, you write about tracing which parts of Everybody Lies were the most highlighted on Kindle. “I noted that people frequently underlined passages about how they could improve their lives and rarely underlined pas­sages about how to improve the world. People are drawn to self-help, I conclud­ed, whether they admit it or not.” Why do you think people are resistant to the idea of liking self-help books as a genre?

I don’t really know, it’s a great mystery! There is certainly nothing un-intellec­tual about wanting to improve yourself. And if you look historically, there have been legendary writers and thinkers who have written about improving yourself, like [Henry David] Thoreau and [Friedrich] Nietzsche. But for some reason in this modern age people are embarrassed to say “self-help”. I think part of the reason is that a lot of the early movers in this genre, who wrote some of the first self-help books, were not serious intellectuals and they gave the genre a bad name. Only serious intellectuals write about, say, 17th-century British history. A lot of not-so-serious people, however, write about how to date better or how to be a better parent.

If you take a step back, though, and really think about it, it’s bizarre and mysterious that people are embarrassed to say that they tried to help themselves be better.

In Chapter 6, you write about “hacking” luck — the idea that people tradition­ally seen as “lucky” are working harder and smarter than most, that they’re just perceived as lucky because they’re giving themselves a greater chance at succeed­ing (“fortune favours the data-driven decision maker”, you write). Could you tell us a little bit more about this idea, and what it looks like in practice?

In that chapter, I talk a lot about art — I think art is a great area to study luck because there’s tremendous evidence that luck had a role to play behind the making and evaluation of some of the bestselling pieces of art in the world. If you talk to some of the most successful artists in the world today, they’ll readily admit to some lucky breaks they caught through their career.

I think one of the things people forget, when they think of luck driving success, is that over the course of a lifetime, you’re supposed to get ‘lucky’. If you’ve literally never met someone who helped you or never came across an idea that was just right for you at that moment in time, you’re possibly the unluckiest person of all time. If you live 4,000 weeks, your “luckiest” three weeks should be really, really lucky — that’s how the math works. Now, one of the ways you can in­crease that number to say 10 lucky weeks or 20 is simply by putting out more work in the world. In the chapter, one of the studies on artists I cite says that artists who present their work at a wider array of venues gave themselves that much more chance to become ‘lucky’ and have their work discovered.

In the final chapter of your book, you write about “Mappiness”, a large-scale happiness data set that uses iPhones to create 3 million data points about what makes people happy. As a reader what were some of the most interesting or surprising data points (and resultant studies) that you came across here?

The study which analyses alcohol consumption patterns was very interest­ing to me, and some of the conclusions were things that I feel ought to be studied more. Unsurprisingly, the study did find that a moderate amount of alcohol does make you happier. But what’s interesting to me was that alcohol seemed to ‘work’ better when people consumed it as they were doing chores; laundry, cleaning the dishes and so on. For most people, alcohol is on the table when you’re already having a good time at the club or with friends—you want to convert good times into great times. But this study would seem to sug­gest that we’re using alcohol all wrong! So if you’re preparing for a night out with friends, drink a little while you’re getting ready, just before heading out and then stay sober thereafter to enjoy the rest of the evening. This, according to the study, would be the optimum usage.

Of course, the usual caveats apply — alcohol is dangerous, especially for those who have addictive traits in their personality. For example, I have zero ad­dictive personality traits and so, even my parents keep pushing me to use more substances: ‘You should drink more, have you tried cocaine?’ (laughs) So if you’re like me and you don’t have that addictive personality, you should con­sider tweaking your alcohol habits. The mistake we generally make is we try and make great experiences into epic experi­ences whereas we should be trying to make boring or unpleasant experiences okay or tolerable or even pleasant.

The Mappiness findings also included some very interesting thoughts on the nature of sports fandoms. Apparently, when the team they’re supporting loses, people report an average of 8 ‘unhappi­ness points’ whereas a victory only results in 4 ‘happiness points’. Given that reality, I must ask: how/why do sports fans exist? (You are a big baseball fan yourself).

We’re all really bad at maximising our own happiness. We have false memo­ries, so maybe we black out some of the horrible experiences we’ve had as sports fans and tend to focus on the few truly great moments. For me that was 1995, ALDS (American League Division Se­ries) Game Two, (New York) Yankees vs (Seattle) Mariners, being in the bleachers hugging my dad when in the fifteenth innings, Jim Leyritz ended the game with a two-run home run. And that was such an amazing experience that I tend to forget all the horrors that sports fandom has delivered in my life. I have a section in the penultimate chapter where I talk about how poor we are at remembering what makes us happy. We remember the peaks of our experiences but not necessarily the averages. I do watch a little less sport now than I used to for the longest time. And one of the things that I took from the research was that people seem to be happier when they’re watching sports without the pressure of fandom, when they’re just watching in general.

I was intrigued by the Stanford study you cite in the final chapter; “The gain in happiness from not using Facebook was about 25–40 per cent as large as the gain in well-being from entering individual therapy.” Conventional wisdom, after all, would suggest that social media helps people make more connections than they would otherwise. Is this one of the ‘counter-counterintuitive’ ideas you describe in the book?

That was actually the first randomised controlled experiment, the one that I’ve cited in the book. Since then, there have been a few follow-up studies that have incorporated Twitter and TikTok into the mix, too, and the results have been very similar. Obviously, there are individual variations — there is clearly a certain type of person who will find a lot to gain through social media usage. But in general, what I took from that research was that heavy usage of these platforms was harmful to one’s well-being. It’s very performative and it can make you jealous very quickly. Apparently, everybody is staying at a Caribbean hotel or enjoying the perfect vacation or is in a perfect marriage. In­stinctively we all know that nobody can be super happy all of the time, and yet this sort of thing affects all of us when we see it on social media.

I found, for example, that in terms of word prevalence, most people described their husbands on social media as “the best”, “my best friend”, “sensitive” and so on. But on Google searches, the most searched phrases in this context were “my husband is a jerk”, “why is my husband so annoying” and so on. So you have to be careful about the many, many ways in which social media can make you feel terrible about your own life and your choices.

artbooksdataeconomistluckSeth Stephens-Davidowitztechnology