Don’t Trust Your Gut (2022) turns that tried-and-true wisdom about trusting your gut on its head. Not only does trusting your gut instinct often lead you to make the wrong decision, there’s a pretty foolproof method to ensure you make the right decision – analyzing the available data and acting on it.
Introduction: Make better decisions through data analysis.
Let’s say you have a particularly difficult decision to make. What do you do? Make a list of pros and cons? Canvas friends and family for their opinions? Go online and search for advice in various forums? Turn to self-help books?
What happens if you do all those things and you still can’t come to a firm decision? Well, according to conventional wisdom, there’s one decision-making strategy that you can still use. And the good news is, it’s often framed as one of the simplest yet most effective strategies around: you can trust your gut. When you follow your intuition, the thinking goes, you’ll almost always make the right choice.
But here’s the thing: your gut is probably wrong.
Instead of trusting your gut, you should be following the data. These days, there’s more data available than ever before, and data analysis techniques are more sophisticated than they’ve ever been. And, time after time, the data shows that counter-intuitive decisions, choices that go against the prevailing tides of wisdom, are more effective than the so-called intuitive choices we make when we follow our gut.
You might not be feeling inspired to say goodbye to instinct and intuition – yet. But after this summary to Don’t Trust Your Gut, you just might be.
Wall Street, Silicon Valley, and pro sports are all across data-driven decision-making.
When it comes to making important decisions, we’re often told to “go with our gut.” But that might not be the best advice. Contrary to popular opinion, you can’t always trust your gut. In fact, every decision you make based on gut feeling could be costing you – big time.
The best way to make effective decisions isn’t to follow your instincts; instead, you’re better off basing your decision-making process on data.
That’s right, data. Thanks to the internet, we have huge stores of data, from Wikipedia profiles to Facebook relationship status updates, at our fingertips. Advances in data analysis techniques mean it’s easier than ever to generate insights from these enormous data sets. Whatever your dilemma – whether you’re weighing up proposing to your partner or wondering about moving to a new city – odds are that some enthusiastic data scientist has crunched the numbers and generated findings that can help you make the right decision. And, when you look at the data for yourself, you might be surprised to find that your gut feeling was actually way off base.
Don’t believe me? From pro baseball to Wall Street, data analysis has underpinned a lot of winning decisions.
The Oakland A’s had one of the lowest payrolls in the league when they reached the playoffs in 2002 and 2003. Rather than trying to draft star players with high batting averages, manager Billy Beane looked at the data. The data showed that other metrics, such as time spent on-base or slugging average, were both better predictors of match success than batting average and undervalued by the market. With these insights, Beane was able to assemble a first-class team with a comparatively low budget.
In Silicon Valley, data is king. One Google designer quit the company over a dispute about which shade of blue to use in an ad link. The designer wanted to go with their intuition, choosing a shade that fit their design sensibilities. The data showed a different shade would lead to a higher click conversion rate. And guess what? Conversion metrics prove Google was right to trust their data over their designer.
Renaissance Technologies is one of the most prestigious, not to mention profitable, hedge funds on Wall Street. Founder James Simon started the company with something more valuable than seed money: he purchased a huge set of raw financial data. Simon, and a team of expert mathematicians, mined the dataset for patterns and trends. Now, every trade Renaissance makes is data-driven. And, in the years since its founding, Renaissance has delivered a 66 percent return since it was founded – pretty impressive, when you consider the S&P 500 delivered a 10 percent return in that same time period.
Ready to ignore your gut? The next four chapters will show you how to make better decisions with data.
Don’t hit the gym – to get more Tinder matches, try data!
Let me tell you about a friend of mine, who I’m going to call Eddie. Eddie’s single and, like a lot of single people who’d prefer to be coupled, he’s downloaded a few dating apps and uploaded his profile. Unfortunately, he hasn’t gotten a lot of matches. How can he boost his success rate?
Eddie’s gut feeling tells him that the more conventionally attractive someone appears, the better luck they’ll have on dating platforms. So, he could choose to hit the gym, whiten his teeth, and get a haircut. Would that be the right decision?
Well, yes and no says Christian Rutter, a data whiz who’s analyzed tens of millions of profiles on the dating platform OkCupid. Christian’s research confirms Eddie’s gut feeling – dazzlingly good-looking people do tend to outperform their more average-looking counterparts on these platforms. But, here’s the catch. Eddie is nice-looking. If he worked on his appearance, well, he’d be nic-er looking. He still wouldn’t be Brad-Pitt-level attractive. And according to Rutter’s research, unless you’re extremely attractive, your appearance won’t sway the number of matches you get.
But that doesn’t mean Eddie can’t hack the system to get more matches. See, Rutter also found that it’s not just extremely good-looking people who’ve found success on OkCupid – it’s also extremely tattooed people, and people with extremely unusual haircuts or styles of dress. Basically, if you look extreme in any way, you’re more likely to provoke a strong reaction from prospective matches. If Eddie got a face tattoo he’d provoke a much stronger reaction from the platform’s users. Lots of people would probably be turned off by his profile. But the people who were interested would be really interested. Interested enough to match with him.
Luckily for Eddie, the data shows there are some alternative options to up his online dating success rate.
He can earn more. Easier said than done, perhaps, but for heterosexual daters, men who earn in the $150,000–$200,000 bracket attract 8.9 percent more matches than men in the $35,000–$50,000 bracket. High-earning women, on the other hand, can only expect to attract 3.9 percent more matches than their lower-earning peers.
So, rich men attract more matches. That’s hardly counter-intuitive. But the data also shows us that for men, job title is just as important as salary, if not more so. An accountant might earn $150,000 but a lawyer who also earns $150,000 tends to attract more matches. The same goes for doctors, soldiers, policemen, and firefighters. Teachers and hospitality workers, on the other hand, are out of luck.
The data also shows that similarity, not difference, is the most attractive trait. A study of over one million eHarmony matches finds that profiles with shared descriptors – for example, profiles where both singles described themselves as “adventurous” or both described themselves as “introverted” – were far more likely to match.
Finally, Eddie can ditch the idea that opposites attract and search for someone similar to him.
The data on happily ever after.
These days, AI technology is pretty impressive. AI can defeat human masters at games like chess and Go. It can predict emerging health issues, like Parkinson’s disease, before the patient notices that same issue for themselves. It can reliably pinpoint when social unrest will break out simply by analyzing discussions on Twitter.
But it can’t explain why some romantic relationships succeed where others fail.
Scientists have tried. Data expert Samantha Joel pulled together a dataset of over 11,000 couples, including data on their physical appearances, ages, salaries, interests, and values. Among all that data, Joel didn’t find any reliable predictors of romantic success.
Does that tell us that there’s no science to building a happy, lasting relationship?
It might. Or, it might tell us that when it comes to looking for a long-term partner, we’re simply looking for the wrong things. All the factors that Joel and her team studied map pretty closely to the factors that we know are predictors of desirability. Just think back to the last chapter: extremely attractive, high-earning people get more matches. We’re more likely to match with people whose interests and values align with our own. And yet all these factors that we prioritize in dating turn out to have little to do with long-term romantic success.
What gives? Joel had the same question. And she did ultimately uncover a few key qualities to look for in a long-term partner. One is satisfaction – you’re more likely to be happy with someone long-term if they’re already happy in most areas of their life. Another is a growth mindset – basically, if your partner believes they can learn new skills, hone their talents, and improve themselves as a person, then they have a growth mindset.
Now, you can’t really tell from a 30-second profile perusal if someone has these important qualities. You need to spend time with them and get to know them to make that call.
Ready for the romance hack? In the dating market, certain people are more desirable than others: for example, a man between 6’3” and 6’4” is 65 percent more likely to match with a woman than a man between 5’7” and 5’8”. A 6’ man earning $62,500 is just as desirable as a 5’6” man earning $237, 500. Those extra six inches of height are worth a whopping $175,000. Here’s the thing. A short man is just as likely as a tall man to have a growth mindset or a good level of satisfaction. If a tall man isn’t more likely to make a woman happy, why would a single woman concentrate her efforts on dating tall men who are, on the dating market, overvalued? That single woman should concentrate on “undervalued assets” – like short men, who are considered less conventionally desirable.
No matter what dating pool you’re in, this is solid advice: if predictors of desirability don’t correlate with predictors of long-term happiness, pinpoint who the undervalued assets are, and target them. They might pay big dividends.
Forget your preconceived notions about professional success.
Quick! List some iconic tech entrepreneurs.
Let me guess. You thought of Zuckerberg. Jobs. Gates. Fadell.
Wait. You mean you haven’t heard of Tony Fadell, former CEO of Nest Labs, a company specializing in programmable thermostats?
Most people haven’t, actually. But that probably doesn’t bother Fadell, who sold Nest Labs to Google for the tidy sum of $3.2 billion.
What’s interesting about Fadell is that he doesn’t fit the popular image of a wildly successful tech entrepreneur. He’s not like Zuckerberg, Jobs, or Gates. Those three all shot to success when they were in their early 20s, after founding scrappy start-ups in their garages. None of them had much employment experience – instead, they earned reputations as renegades and rule-breakers, whose outsider status helped them succeed.
Not Fadell. He was in his early 40s when he founded Nest Labs. He wasn’t a rebel rule-breaker. He had an impressive CV, including stints at Phillips and Apple, which gave him the engineering know-how and managerial experience that helped make Nest Labs so successful. And he wasn’t an outsider, either. He recruited his team from a pre-existing network of peers and colleagues.
Fadell seems like the outlier here. Actually, he fits the profile of a successful founder better than any of the other three. See, the reason Zuckerberg and co capture our imagination is that their trajectories are so untypical. A study of 2.7 million entrepreneurs reveals that the median age for founders is 41.9. And, up until their 60s, older founders have the edge over their younger counterparts – the data shows they’re more likely to build sustainable, successful start-ups. So, to find entrepreneurial success, emulate Fadell: gain deep experience in a narrow field, and draw on your network when you strike out on your own.
Your gut might be telling you that because you’re not a 20-year-old tech whiz, you can’t be a successful founder. Well, remember Tony Fadell – and tell your gut to pipe down.
Nature or nurture? The answer’s in the data.
New parents are expected to make, on average, 1,750 difficult decisions in the first year of their baby’s life. What should they call their bundle of joy? Breastfeeding or bottles? Cot or co-sleeping?
But research shows that these decisions might have very little to do with how a baby turns out. In other words: nature trumps nurture. Take the case of twins, Jim Lewis and Jim Springer who were raised separately from birth. When they reunited at age 39, both Jims were six-foot-tall and weighed 180 pounds. Both bit their nails and worked in law enforcement. Both had a childhood dog called Toy and both smoked Salem Lights.
There was one big difference. Jim Lewis called his son James Alan, while Jim Springer called his son James Allan – with two l’s.
One story doesn’t make a trend – but the data bears out the idea that most parenting choices aren’t make or break. Studies find that breastfed children enjoy no significant long-term health benefits than bottle-fed children. Children who are encouraged to play cognitively stimulating games like chess don’t, on average, grow up smarter than their peers. And children who are exposed to television don’t score worse on tests than those who aren’t.
Interestingly enough, there’s one area where a parental decision can significantly affect a child’s outcomes. And it’s got nothing to do with enrolling them in bassoon lessons or afternoon Latin classes.
The most impactful choice a parent can make for their child is where they choose to raise them. In the USA, simply moving to Seattle can boost your child’s projected future earnings by 11 percent. Not bad, right? But more important than choosing a specific city, is choosing the neighborhood where your kid grows up. Should you choose a neighborhood with a great school? Take on a big mortgage and move to a neighborhood with a high median income?
Not necessarily. The neighborhoods that a large study has found to be most advantageous for the children that grow up there all share three key traits. A high percentage of two-parent households, which tend to be stable. A high percentage of college graduates, who tend to be accomplished. And a high percentage of people who return their census forms, who tend to be engaged citizens.
Now, it doesn’t matter if you’re a single parent who never finished high school and tossed their census form in the trash by accident as long as you’re surrounded by other adults who embody these three traits. Why? Well, the data suggests that it’s not just parents that shape a child’s trajectory, but all the adults they routinely come into contact with. In fact, they might be more important than you. After all, your kids will probably want to rebel against you. They’ll likely have a much less complicated relationship with Mr. and Mrs. Suarez down the road, and therefore more happily accept them as role models.
It seems the data backs up the old African proverb: it takes a village to raise a child.
Summary
Follow your intuition. Trust your gut. Listen to your heart. Turns out all that old advice is, well, questionable. Often, the right decision is the one that seems risky or counter-intuitive. Luckily, it’s easy to uncover the best course of action, by relying on data insights. Time and again, from Wall Street to Silicon Valley and from romance to parenting, data-driven decision-making has been shown to yield results.
About the author
Seth Stephens-Davidowitz is a contributing op-ed writer for the New York Times, a lecturer at The Wharton School, and a former Google data scientist. He received a BA from Stanford and a PhD from Harvard. His research has appeared in the Journal of Public Economics and other prestigious publications. He lives in New York City.
Genres
Science, Personal Development, Psychology, Self Help, Economics, Sociology, Business, Education, Management, Leadership, Popular Culture in Social Sciences, Decision-Making, Problem Solving, Personal Transformation Self-Help
Table of Contents
Dedication
Introduction: Self-Help for Data Geeks
Chapter 1: The AI Marriage
Chapter 2: Location. Location. Location. The Secret to Great Parenting.
Chapter 3: The Likeliest Path to Athletic Greatness If You Have No Talent
Chapter 4: Who Is Secretly Rich in America?
Chapter 5: The Long, Boring Slog of Success
Chapter 6: Hacking Luck to Your Advantage
Chapter 7: Makeover: Nerd Edition
Chapter 8: The Life-Changing Magic of Leaving Your Couch
Chapter 9: The Misery-Inducing Traps of Modern Life
Conclusion
Acknowledgments
Appendix
Notes
Index
About the Author
Overview
Big decisions are hard. We consult friends and family, make sense of confusing “expert” advice online, maybe we read a self-help book to guide us. In the end, we usually just do what feels right, pursuing high stakes self-improvement—such as who we marry, how to date, where to live, what makes us happy—based solely on what our gut instinct tells us. But what if our gut is wrong? Biased, unpredictable, and misinformed, our gut, it turns out, is not all that reliable. And data can prove this.
In Don’t Trust Your Gut, economist, former Google data scientist, and New York Times bestselling author Seth Stephens-Davidowitz reveals just how wrong we really are when it comes to improving our own lives. In the past decade, scholars have mined enormous datasets to find remarkable new approaches to life’s biggest self-help puzzles. Data from hundreds of thousands of dating profiles have revealed surprising successful strategies to get a date; data from hundreds of millions of tax records have uncovered the best places to raise children; data from millions of career trajectories have found previously unknown reasons why some rise to the top.
Telling fascinating, unexpected stories with these numbers and the latest big data research, Stephens-Davidowitz exposes that, while we often think we know how to better ourselves, the numbers disagree. Hard facts and figures consistently contradict our instincts and demonstrate self-help that actually works—whether it involves the best time in life to start a business or how happy it actually makes us to skip a friend’s birthday party for a night of Netflix on the couch. From the boring careers that produce the most wealth, to the old-school, data-backed relationship advice so well-worn it’s become a literal joke, he unearths the startling conclusions that the right data can teach us about who we are and what will make our lives better.
Lively, engrossing, and provocative, the end result opens up a new world of self-improvement made possible with massive troves of data. Packed with fresh, entertaining insights, Don’t Trust Your Gut redefines how to tackle our most consequential choices, one that hacks the market inefficiencies of life and leads us to make smarter decisions about how to improve our lives. Because in the end, the numbers don’t lie.
Review/Endorsements/Praise/Award
“Seth Stephens-Davidowitz is more than a data scientist. He is a prophet for how to use the data revolution to reimagine your life. Don’t Trust Your Gut is a tour de force—an intoxicating blend of analysis, humor, and humanity.” – Daniel H. Pink, #1 New York Times bestselling author of When, Drive, and To Sell Is Human
This must-read book is packed with helpful discoveries you can use to improve your life, and each is grounded in data. It’s also a page-turner—Seth Stephens-Davidowitz is a smart, witty writer with an extraordinary ability to make charts and statistics engrossing. – Katherine Milkman, author of How to Change
There are two ways to look at big data: as a threat to your intuition or as a resource to test your intuition. Seth Stephens-Davidowitz is an expert on data-driven thinking, and this engaging book is full of surprising, useful insights for using the information at your fingertips to make better decisions. – Adam Grant, #1 New York Times bestselling author of Think Again and host of the TED podcast WorkLife
How can you look your best? Who should you marry? What makes a good parent? Are you too old to start a business? How can you get rich? What would make you happy? Would you read a book that helps you answer even one of these questions? Seth Stephens-Davidowitz delivers: a cross between Freakonomics and How to Win Friends and Influence People, Don’t Trust Your Gut is your guide for reliable data-driven hacks to get an edge in life. – Ian Bremmer, president and founder of Eurasia Group
Seth Stephens-Davidowitz’s book is a brilliant and clever look into the critical importance of making data-informed decisions for a data-first organization. His truly game-changing approach provided a pivotal moment for me as a leader and his insightful yet humorous writing style is sure to do the same for many others. – Mindy Grossman, CEO of Weight Watchers
I love the way Seth Stephens-Davidowitz explains how we can better live our lives by exploiting the small advantages in life. On the basketball court, I made a career out of finding these types of minor advantages, and I’ve found that most successful individuals in life value the accumulation of small advantages. In the end, they add up to significant life benefits. – Shane Battier, two-time NBA Champion basketball player for the Miami Heat
Stephens-Davidowitz maintains a breezy, conversational style that lends a lighthearted touch to all the wonkery. Whether confirming or debunking conventional wisdom, the smooth presentation and quantitative detail bring a welcome analytical rigor to the self-help genre. – Publishers Weekly
Video and Podcast
Read an Excerpt/PDF Preview
Introduction: Self-Help for Data Geeks
You can make better life decisions. Big Data can help you.
We are living through a quiet revolution in our understanding of the most important areas of human life—thanks to the internet and all the data it has created. In the past few years, scholars have mined a variety of enormous datasets—everything from OkCupid messages to Wikipedia profiles to Facebook relationship statuses. In these thousands or millions of data points, they have found, for perhaps the first time, credible answers to fundamental questions. Questions such as:
What makes a good parent?
Who is secretly rich—and why?
What are the odds of becoming a celebrity?
Why are some people unusually lucky?
What predicts a happy marriage?
What, more generally, makes people happy?
Often, the answers revealed in the data are not what you might have guessed, and they suggest making different decisions than you might otherwise make. Quite simply, there are insights in these mounds of new data that can allow you, or someone you know, to make better decisions.
Here are three examples uncovered from researchers studying very different parts of life.
Example # 1: Suppose you are a single man or woman who isn’t getting as many dates as you would; like. You try to improve yourself in every way that others suggest. You dress better. You whiten your teeth. You get a pricey new haircut. But still. The dates, they’re not coming.
Insights from Big Data might help.
The mathematician and author Christian Rudder studied tens of millions of preferences on OkCupid to learn the qualities of the site’s most successful daters. He found—and this was not at all surprising—that the most prized daters are those blessed with conventional beauty: the Brad Pitts and Natalie Portmans of the world.
But he found, in the mounds of data, other daters who did surprisingly well: those with extreme looks. Think, for example, of people with blue hair, body art, wild glasses, or shaved heads.
Why? The key to these unconventional daters’ success is that, while many people aren’t especially attracted to them, or find them plainly unattractive, some people are really attracted to them. And in dating that is what is most important.
In dating, unless you are drop-dead gorgeous, the best strategy is, in Rudder’s words, to get “lots of Yes, lots of No, but very little Meh.” Such a strategy, Rudder discovered, can lead to about 70 percent more messages. Be an extreme version of yourself, the data says, and some people will find you extremely attractive.
And example # 2: Suppose you just had a baby.* You need to pick a neighborhood in which to raise this child. You know the drill. You consult a few friends, Google some basic facts, visit a couple of homes. And voila! You’ve got yourself a home for your family. You assume there isn’t much more of a science to this.
There is a science to neighborhood-hunting now.
Researchers recently took advantage of newly digitized tax records to study the life trajectories of hundreds of millions of Americans. The scientists discovered that being raised in certain cities—and even certain blocks within those cities—can dramatically improve a person’s life outcomes. And these great neighborhoods are not necessarily the ones people suspect. Nor are they the ones that cost the most. There are now maps that can inform parents, based on extensive data analysis, about the quality of every tiny neighborhood of the United States.
That’s not all. Researchers have also mined data to find traits that the best neighborhoods for raising kids tend to share; in the process, they have upended much conventional wisdom about child-rearing. Thanks to Big Data, we are finally able to tell parents what really matters for raising a successful kid (hint: adult role models) and what matters a lot less (hint: the fanciest schools).
And example # 3: Suppose you are an aspiring artist who can’t seem to catch your big break. You buy every book you can on your craft. You get feedback from your friends. You revise your pieces again and again and again. But nothing seems to work. You can’t figure out what you are doing wrong.
Big Data has uncovered a likely mistake.
A recent study of the career trajectories of hundreds of thousands of painters, led by Samuel P. Fraiberger, has uncovered a previously hidden pattern in why some succeed, and others don’t. So, what’s the secret that differentiates the big names from the anonymous strugglers?
It is often how they present their work. Artists who never break through, the data tells us, tend to present their work to the same few places over and over again. The artists who make it big, in contrast, present to a far wider set of places, allowing themselves to stumble upon a big break.
Many people have talked about the importance in your career of showing up. But data scientists have found it’s about showing up to a wide range of places.
This book isn’t meant to give advice only for single people, new parents, or aspiring artists—though there will be more lessons here for all of them. My goal is to offer some lessons in new, big datasets that are useful for you, no matter what stage of life you are in. There will be lessons recently uncovered by data scientists in how to be happier, look better, advance your career, and much more. And the idea for the book all came to me one evening while . . . I was watching a baseball game.
Moneyball for Your Life
I and other baseball fans can’t help but notice: baseball is a very different game than it was three decades ago. When I was a young boy and cheering on my beloved New York Mets, baseball teams made decisions using their gut and intuition. They chose whether to bunt or steal based on the feelings of the manager. They chose which players to draft based on the impressions of scouts.
However, in the latter part of the twentieth century, there were hints of a better way. Every year of my childhood, my father would bring home a new book by Bill James. James, who worked the night shift as a security guard at a pork and beans cannery in Kansas, was an obsessive fan of baseball. And he had a nonstandard approach to analyzing the game: newly available computers and digitized data. James and his peers—called sabermetricians—discovered, in their data analysis, that many of the decisions that teams typically made, when they relied on their gut, were dead wrong.
How much should teams bunt? Much less, the sabermetricians said. How much should they steal? Almost never. How much were players who drew a lot of walks worth? More than teams thought. Whom should teams draft? More college pitchers.
My father wasn’t the only one intrigued by James’s work. Billy Beane, a baseball player turned baseball executive, was a big Bill James fan. And, when he became general manager of the Oakland A’s, he chose to run his team using the principles of sabermetrics.
The results were remarkable. As famously told in the book and movie Moneyball, the Oakland A’s, despite having one of the lowest payrolls in baseball, reached the playoffs in 2002 and 2003. And the role of analytics in baseball has exploded since then. The Tampa Bay Rays, who have been called “a team more Moneyball than the Moneyball A’s themselves,” reached the 2020 World Series despite the third-lowest payroll in baseball.
Further, the principles of Moneyball and the powerful underlying idea—that data can be useful in correcting our biases—have transformed many other institutions. Other sports, for example. NBA teams increasingly rely on analytics that track the trajectory of every shot. In data from 300 million shots, large deviations from optimal shooting have been found. The average NBA jump shooter, it turns out, is twice as likely to miss a shot too short as opposed to too long. And, when he shoots from the corner, he is more likely to miss to the side away from the backboard, perhaps because he is too afraid of hitting it. Players have utilized such information to correct these biases—and make more shots.
Silicon Valley firms have been built largely on Moneyball principles. Google, where I formerly worked as a data scientist, certainly believes in the power of data to make major decisions. A designer famously quit the company because it frequently ignored the intuition of trained designers in favor of data. The final straw for the designer was an experiment that tested forty-one shades of blue in an ad link on Gmail to collect data on which one would lead to the most clicks. The designer may have been frustrated, but the data experiment netted Google an estimated $200 million per year in additional ad revenue and Google has never wavered on its belief in data as it built its $1.8 trillion company. As Eric Schmidt, its former CEO, put it, “In God we trust. All others have to bring data.”
James Simons, a world-class mathematician and founder of Renaissance Technologies, brought rigorous data analysis to Wall Street. He and a team of quants built an unprecedented dataset of stock prices and real-world events and mined it for patterns. What tends to happen to stocks after earnings announcements? Bread shortages? Company mentions in newspapers?
Since its founding, Renaissance’s flagship Medallion fund—trading entirely based on patterns in data—has returned 66 percent per year before fees. Over the same time period, the S&P 500 has returned 10 percent per year. Kenneth French, an economist associated with the efficient market hypothesis, which suggests it is virtually impossible to meaningfully outperform the S&P 500, explained Renaissance’s success as follows: “It appears that they’re just better than the rest of us.”
But how do we make big decisions in our personal lives? How do we pick whom to marry, how to date, how to spend our time, whether to take a job?
Are we more like the A’s in 2002 or the other baseball teams back then? More like Google or a mom-and-pop shop? More like Renaissance Technologies or a traditional money manager?
I would argue that the vast majority of us, the vast majority of the time, rely heavily on our gut to make our biggest decisions. We might consult some friends, family members, or self-proclaimed life gurus. We might read some advice that isn’t based on much. We might squint at some very basic stats. Then, we will do what feels right.
What might happen, I wondered, as I watched that baseball game, if we took a data-based approach to our biggest life decisions? What if we ran our personal lives the way that Billy Beane ran the Oakland A’s?
I knew that such an approach to life is increasingly possible these days. My previous book, Everybody Lies, explored how all the new data available thanks to the internet is transforming our understanding of society and the human mind. The stats revolution may have come to baseball first thanks to all the data that its stats-obsessed fans had demanded and collected. The Lifeball revolution is now possible thanks to all the data that our smartphones and computers have collected.
Consider this not-too-trivial question: what makes people happy?
Data to answer this question in a rigorous, systematic way was not available in the twentieth century. When the Moneyball revolution hit baseball, sabermetricians may have been able to analyze data from play-by-plays that had been dutifully recorded for every game. Back then, however, data scientists did not have the equivalent of play-by-plays for people’s life decisions and resulting mood. Back then, happiness, unlike baseball, was not open to rigorous quantitative research.
But it is now.
Brilliant researchers such as George MacKerron and Susana Mourato have utilized iPhones to build an unprecedented happiness dataset—a project they call Mappiness. They recruited tens of thousands of users and pinged them on their smartphones throughout the day. They asked simple questions such as what they were doing, who they were with, and how happy they were. From this they created a dataset of more than 3 million happiness points, a far cry from the dozens of data points that had previously been the stuff of happiness research.
Sometimes the results in these millions of data points are provocative, such as that sports fans get more pain from their teams’ losses than they gain pleasure from their teams’ wins. Sometimes the results are counterintuitive, such as that drinking alcohol tends to give a bigger happiness boost to someone doing chores than someone socializing with friends. Sometimes the results are profound, such as that work tends to make people miserable unless they work with their friends.
But, always, the results are useful. Ever wondered precisely how weather affects our mood? Which activities tend to systematically deceive us in how much pleasure they will bring? The real role that money plays in happiness? How much our surroundings determine how we feel? We now, thanks to MacKerron, Mourato, and others, have credible answers to all these questions—answers that will be the stuff of Chapters 8 and 9. In fact, I will even conclude this book with a reliable formula for happiness that has been uncovered in millions of smartphone pings. I call it the Data-Driven Answer to Life.
So, for the past four years, motivated by a baseball game, I have disappeared into intensive study. I have talked to researchers. I have read academic papers. I’ve pored over the appendices of papers in ways that I am pretty sure no researcher was expecting. And I’ve done some of my own research and interpretation. I viewed my job as finding the Bill Jameses of arenas such as marriage, parenting, athletic achievement, wealth, entrepreneurship, luck, style, and happiness—and allowing all of you to become the Billy Beane of your personal lives. I am now ready to report everything that I have learned.
Call it “Moneyball for your life.”
The Infield Shifts of Life
Before I explored the research, I asked myself some basic questions. What might a life built on Moneyball principles look like? How might our personal decision-making look if, like the A’s and the Rays, we followed the data instead of our instincts? One thing that is striking from watching baseball post-Moneyball is that some of the decisions made by analytics-driven baseball teams seem . . . well, a little odd. Consider this example: the location of infielders.
In the post-Moneyball era, baseball teams increasingly engage in infield shifts. They load up many of their defenders in the same part of the field, leaving wide swaths of the field completely unguarded, seemingly wide-open for a hitter to direct the ball. The infield shift looks positively insane to fans of traditional baseball. But insane it is not. Such shifts are justified in mounds of data that predict where particular players are most likely to hit the ball. The numbers tell baseball teams that, even though it looks wrong, it is right.
If we take a Moneyball approach to life, we might similarly expect to find that some seemingly odd decisions—call them the infield shifts of life—are justified.
We’ve already discussed a couple. Shaving your head or dyeing your hair blue to get more dates is an infield shift of life. Here’s another one, uncovered in the Big Data of sales.
Suppose you are trying to sell something. This is an increasingly common experience. As the author Daniel Pink put it in his book To Sell Is Human, whether we are “pitching colleagues, persuading funders, [or] cajoling kids . . . we’re all in sales now.”
Anyway, whatever your pitch, you give it your best shot.
You write up your pitch. (This is good!) You practice your pitch. (Good!) You get a good night’s sleep. (Good!) You eat a hearty breakfast. (Good!) You fight through your nerves and get up there. (Good!)
And, as you make your sales case, you remember to convey your excitement with a big, hearty, toothy smile. (This is . . . not good.)
A recent study analyzed the effects of a salesperson’s emotional expression on how much they sell.
The dataset: 99,451 sales pitches on a livestreaming retailing platform. (These days, people are increasingly buying products on services such as Amazon Live, which allows people to pitch their products by video to potential customers.) Researchers were given videos of each sales pitch along with data on how much product was sold afterward. (They also had data on the product being sold, the price of the product, and whether they offered free shipping.)
The methods: artificial intelligence and deep learning. The researchers converted their 62.32 million frames of video into data. In particular, the AI was able to code the emotional expression of the salesperson during the video. Did the salesperson appear angry? Disgusted? Scared? Surprised? Sad? Or happy?
The result: the researchers found that the emotional expression of a salesperson was a major predictor of how much product they sold. Not surprisingly, when a salesperson expressed negative emotions, such as anger or disgust, they sold less. Rage doesn’t sell. More surprisingly, when a salesperson expressed highly positive emotions, such as happiness or surprise, they sold less. Joy doesn’t sell. When it comes to increasing sales, a salesperson’s limiting their excitement—having a poker face instead of a smile—proves about twice as valuable as free shipping.
Sometimes, to sell your product, you should convey less enthusiasm for your product. It might feel wrong, but the data says that it’s right.
From Everybody Lies to Don’t Trust Your Gut
Brief pause while I justify this book to readers of my first book, Everybody Lies. Some of you may have been drawn to this book because you were fans of that book. And if that doesn’t explain how you came to this book at all, perhaps I can convince you in the next few paragraphs to buy that book as well. I try.
In Everybody Lies, I discussed my research on how we can use Google searches to uncover what people really think and do. I called Google searches “digital truth serum” because people are so honest to the search engine. And I called Google searches the most important dataset ever collected on the human psyche.
I showed that:
Racist Google searches predicted where Barack Obama underperformed in the 2008 and 2012 presidential elections.
People frequently type full sentences into Google, things like “I hate my boss,” “I’m drunk,” or “I love my girlfriend’s boobs.”
The top Google search that starts “my husband wants . . .” in India is “my husband wants me to breastfeed him.” In India, there are almost as many Google searches looking for advice on how to breastfeed a husband as there are on how to breastfeed a baby.
Google searches for do-it-yourself abortions are almost perfectly concentrated in parts of the United States where it is hard to get a legal abortion.
Men make more searches for information on how to make their penis bigger than how to tune a guitar, change a tire, or make an omelet. One of their top Googled questions about their penis is “How big is my penis?”
At the end of that book, I suggested my next book would be called Everybody (Still) Lies and would keep exploring what Google searches tell us. Sorry, I guess I lied about that. Figures, from the author of Everybody Lies.
This book is, on its surface, very different. And, if you were hoping to get further analysis of men’s searches about their genitals, you will be sorely disappointed. Eh, fine. I’ll give you one more. Did you know that men sometimes type into Google full sentences stating the size of their penis? They type into Google, for example, “My penis is 5 inches.” And, if you examine the data on all these searches, they reveal a close-to-normal distribution of reported-to-Google penis sizes centered around five inches.
But let’s move on from my research into the wacky world of Google search data, which, as mentioned, you can learn more about in Everybody Lies.
Most of the studies featured in this book, unlike in Everybody Lies, are from other people, not from me. This book is more practical, tightly focused around self-improvement rather than explorations of random parts of modern life. Further, this book has noticeably less emphasis on sex than my previous book. Any discussion of sex in this book will not focus on the secret sexual desires or insecurities of people, topics that are heavily featured in my previous book. The discussion of sex here, instead, is limited to the question of whether sex makes people happy (spoiler: yes).
But I do think this is a natural follow-up to my first book for two reasons.
First, the motivation of this book is partly based on following the data of what readers really want, not what they say they want. After I wrote Everybody Lies, like any good market researcher, I asked readers what resonated with them most. Most people told me they were particularly moved by some of the sections on the world’s biggest problems and how we might fix them—sections on child abuse or inequality, for example.
But, as the author of Everybody Lies, I was skeptical of what people said and wanted to see some other data—perhaps some digital truth serum. I looked at the most underlined sections on Amazon Kindle versions of the book. I noted that people frequently underlined passages about how they could improve their lives and rarely underlined passages about how to improve the world. People are drawn to self-help, I concluded, whether they admit it or not.
A more extensive study of Amazon Kindle data came to a similar conclusion. Researchers found, over a large sample of books, that the word “you” was twelve times more likely to appear in the most underlined sentences than other sentences. People, in other words, really like sentences that include the word “you.”
Hence, the first paragraph of Don’t Trust Your Gut:
“You can make better life decisions. Big Data can help you.”
That was a data-driven, not a gut-driven, first paragraph. It was delivered to you in a book written to help you get more of what you want in your life. Did you like it?
The popularity of books that can offer help to readers is also confirmed by a deep dive into the most popular books in history. I examined the best-selling books of all time. The biggest category of nonfiction best-sellers is self-help (making up about 42 percent of the most popular nonfiction books of all time). Next biggest is memoirs of celebrities (28 percent). And third is sex studies (8 percent).
What I’m trying to say is that, by following the data, I will write this self-help book first. Then I will write Sex: The Data. Then I hope that will make me famous enough to write Seth: Memoir of the Author Who Got Famous by Following the Data on What Books Sell.
The second connection between Everybody Lies and Don’t Trust Your Gut is that this book is also about using data to uncover the secrets of modern life. One of the reasons that data is so useful in making better decisions is that basic facts about the world are hidden from us. There are secrets about who gets what they want in life that are uncovered by Big Data.
Take this secret: who is rich? Clearly, knowing this would help any person who wants to earn more money. But knowing this is complicated by the fact that many rich people don’t want other people to know that they are rich.
A recent study utilized newly digitized tax records to perform, by far, the most comprehensive study of rich people. The researchers learned that the typical rich American is not a tech tycoon, corporate bigwig, or some of the other people you might naturally have expected. The typical rich American is, in the words of the authors, the owner of “a regional business,” such as “an auto dealer [or] beverage distributor.” Who knew?!? In Chapter 4, we will talk about why that is—and what it implies for how to pick a career.
The media also lies to us—or at least gives us a misleading impression of how the world works by only selecting certain stories to tell us. Using data to cut through those lies can lead to information that is helpful in making decisions.
An example: age and entrepreneurial success. Data has uncovered that the media gives us a distorted view of the age of typical entrepreneurs. A recent study found that the median age of entrepreneurs featured in business magazines is twenty-seven years old. The media loves telling us the sexy stories of the wunderkinds who created major companies.
But how old is the typical entrepreneur, really? A recent study of the entire universe of entrepreneurs found that the average successful entrepreneur is forty-two years old. And the odds of starting a successful business increase up until the age of sixty. Further, the advantage of age in entrepreneurship is true even in tech, a field that most people believe requires youth to master the new tools.
Surely, the advantage of age in all types of entrepreneurship is useful information for someone who has hit middle age and thinks the chance of starting a business has passed them by. In Chapter 5, we will bust a few myths about entrepreneurial success and discuss a reliable formula uncovered in data that is likely to maximize anyone’s chances of creating a successful business.
When you know the data on how the world really works—and avoid the lies of individuals and the media—you are prepared to make better life decisions.
From God to Feelings to Data
In the final chapter of Homo Deus, Yuval Noah Harari writes that we are going through a “tremendous religious revolution, the like of which has not been seen since the eighteenth century.” The new religion, Harari says, is Dataism, or faith in data.
How did we get here?
For much of human history, of course, the most learned people in the world placed the highest authority in God. Harari writes, “When people didn’t know whom to marry, what career to choose or whether to start a war, they read the Bible and followed its counsel.”
The humanist revolution, which Harari places in the eighteenth century, questioned the God-centered worldview. Scholars such as Voltaire, John Locke, and my favorite philosopher, David Hume, suggested that God was a figment of human imagination and the rules of the Bible were faulty. With no external authority to guide us, philosophers suggested that human beings guide themselves. The ways to make big decisions, in the age of humanism, Harari writes, were “listen[ing] to yourself,” “watching sunsets,” “keeping a private diary,” and “having heart-to-heart talks with a good friend.”
The Dataist revolution, which has just started and, Harari says, may take decades or more to be fully embraced, questioned the feelings-centered worldview of the humanists. The quasi-religious status of our feelings was called into question by life scientists and biologists. They discovered that, in Harari’s words, “organisms are algorithms” and feelings merely “processes of biochemical calculations.”
Further, legendary behavioral scientists, such as Amos Tversky and Daniel Kahneman, discovered that our feelings frequently lead us astray. The mind, Tversky and Kahneman told us, is riddled with biases.
Think your gut is a reliable guide? Not so, they said. We are frequently too optimistic; overestimate the prevalence of easily remembered stories; latch on to information that fits what we want to believe; wrongly conclude that we can explain events that, at the time, were unpredictable; and on and on and on.
“Listening to yourself” may have sounded liberating and romantic to the humanists. But “listening to yourself” sounds, frankly, dangerous after reading the latest issue of Psychological Review or Wikipedia’s wonderful article, “List of cognitive biases.”
Finally, the Big Data revolution offers us an alternative to listening to ourselves. While our intuitions—and the counsel of our fellow human beings—may have seemed to the humanists like the only sources of wisdom that we could lean on in a godless universe, data scientists are now building and analyzing enormous datasets that can free us from the biases of our own minds.
More Harari: “In the twenty-first century, feelings are no longer the best algorithms in the world. We are developing superior algorithms that utilize unprecedented computing power and giant databases.” Under Dataism, “When you contemplate whom to marry, which career to pursue and whether to start a war,” the answer is now “algorithms [that] know us better than we know ourselves.”
I’m not quite arrogant enough to claim that Don’t Trust Your Gut is the bible of Dataism or to try to write the Ten Commandments of Dataism, though I would love it if you thought of the other researchers whose work I discuss as the prophets of Dataism. (They really are that trailblazing.)
But I do hope that this book will show you what the new worldview of Dataism looks like, along with offering you some algorithms that might be useful to you or a friend facing a big decision. Don’t Trust Your Gut includes nine chapters; each one explores what data can tell us about a major area of life. And the first one will focus on perhaps life’s biggest decision and the decision that Harari lists first as one that might be transformed by Dataism.
So, Dataists and potential Dataism converts: can an algorithm help you pick “whom to marry”?
Chapter 1: The AI Marriage
Whom should you marry?
This may be the most consequential decision of a person’s life. The billionaire investor Warren Buffett certainly thinks so. He calls whom you marry “the most important decision that you make.”
And yet people have rarely turned to science for help with this all-important decision. Truth be told, science has had little help to offer.
Scholars of relationship science have been trying to find answers. But it has proven difficult and expensive to recruit large samples of couples. The studies in this field tended to rely on tiny samples, and different studies often showed conflicting results. In 2007, the distinguished scholar Harry Reis of the University of Rochester compared the field of relationship science to an adolescent: “sprawling, at times unruly, and perhaps more mysterious than we might wish.”
But a few years ago, a young, energetic, uber-curious, and brilliant Canadian scientist, Samantha Joel, aimed to change that. Joel, like so many in her field, was interested in what predicts successful relationships. But she had a noticeably different approach from others. Joel did not merely recruit a new, tiny sample of couples. Instead, she joined together data from other, already-existing studies. Joel reasoned that, if she could merge data from the existing small studies, she could have a large dataset—and have enough data to reliably find what predicts relationship success and what does not.
Joel’s plan worked. She recruited every professor she could find who had collected data on relationships—her team ended up including eighty-five other scientists—and was able to build a dataset of 11,196 couples.*
The size of the dataset was impressive. So was the information contained in it.
For each couple, Joel and her team of researchers had measures of how happy each partner reported being in their relationship. And they had data on just about anything you could think to measure about the two people in that relationship.
The researchers had data on:
demographics (e.g., age, education, income, and race)
physical appearance (e.g., How attractive did other people rate each partner?)
sexual tastes (e.g., How frequently did each partner want sex? How freaky did they want that sex to be?)
interests and hobbies
mental and physical health
values (e.g., their views on politics, relationships, and child-rearing)
and much, much more.
Further, Joel and her team didn’t just have more data than others in the field. They had better statistical methods. Joel and some of the other researchers had mastered machine learning, a subset of artificial intelligence that allows contemporary scholars to detect subtle patterns in large mounds of data. One might call Joel’s project the AI Marriage, as it was among the first studies to utilize these advanced techniques to try to predict relationship happiness.
If you like guessing games, you can try to predict the results. What do you think are the biggest predictors of relationship success? Are common interests more important than common values? How important is sexual compatibility in the long term? Does coming from a similar background as a mate make you happier?
After building her team and collecting and analyzing the data, Joel was ready to present the results—results of likely the most exciting project in the history of relationship science.
Joel scheduled a talk in October 2019 at the University of Waterloo in Canada with the straightforward title: “Can we help people pick better romantic partners?”
So, can Samantha Joel—teaming up with eighty-five of the world’s most renowned scientists, combining data from forty-three studies, mining hundreds of variables collected from more than ten thousand couples, and utilizing state-of-the-art machine learning models—help people pick better romantic partners?
No.
The number one—and most surprising—lesson in the data, Samantha Joel told me in a Zoom interview, is “how unpredictable relationships seem to be.” Joel and her coauthors found that the demographics, preferences, and values of two people had surprisingly little power in predicting whether those two people were happy in a romantic relationship.
And there you have it, folks. Artificial intelligence can now:
defeat the world’s most talented humans at chess and Go;
reliably predict social unrest five days before it happens merely based on chatter on the internet; and
inform people of an emerging health issue, such as Parkinson’s disease, based on the odors they emit.
But ask AI to figure out whether a set of two human beings can build a happy life together. And it is just as clueless as the rest of us.
WELL . . . THAT SURE SEEMS LIKE A LETDOWN—AS WELL AS A truly horrific start to a chapter in my book with the bold thesis that data science can revolutionize how we make life decisions. Does data science really have nothing to offer us in picking a romantic partner, perhaps the most important decision that we will face in life?
Not quite. In truth, there are important lessons in Joel and her coauthors’ machine learning project, even if computers’ ability to predict romantic success is worse than many of us might have guessed.
For one, while Joel and her team found that the power of all the variables that they had collected to predict a couple’s happiness was surprisingly small, they did find a few variables in a mate that at least slightly increase the odds you will be happy with them. More important, the surprising difficulty in predicting romantic success has counterintuitive implications for how we should pick romantic partners.
Think about it. Many people certainly believe that many of the variables that Joel and her team studied are important in picking a romantic partner. They compete ferociously for partners with certain traits, assuming that these traits will make them happy. If, on average, as Joel and her coauthors found, many of the traits that are most competed for in the dating market do not correlate with romantic happiness, this suggests that many people are dating wrong.
This brings us to another age-old question that has also recently been attacked with revolutionary new data: how do people pick a romantic partner?
In the past few years, other teams of researchers have mined online dating sites, combing through large, new datasets on the traits and swipes of tens of thousands of single people to determine what predicts romantic desirability. The findings from the research on romantic desirability, unlike the research on romantic happiness, has been definitive. While data scientists have found that it is surprisingly difficult to detect the qualities in romantic partners that lead to happiness, data scientists have found it strikingly easy to detect the qualities that are catnip in the dating scene.
A recent study, in fact, found that not only is it possible to predict with great accuracy whether someone will swipe left or right on a particular person on an online dating site. It is even possible to predict, with remarkable accuracy, the time it will take for someone to swipe. (People tend to take longer to swipe for someone close to their threshold of dating acceptability.)
Another way to say all this: Good romantic partners are difficult to predict with data. Desired romantic partners are easy to predict with data. And that suggests that many of us are dating all wrong.*
What People Look for in a Partner
The major development in the search for romance in the early part of the twenty-first century has been the rise of online dating. In 1990, there were six predominant ways that people met their spouses. The most frequent way was through friends, followed by: as coworkers, in bars, through family, in school, as neighbors, and in church.
In 1994, kiss.com was founded as the first modern online dating site. One year later, Match.com was started. And, in 2000, I excitedly set up my profile on JDate, an online Jewish dating site, confident that I had discovered the cool new thing . . . only to quickly realize that, once again, the cool new thing I thought I had found was actually predominantly used by weirdos like myself.
However, the use of online dating has since exploded. By 2017, nearly 40 percent of couples met online. And this number continues to rise every year.
Has online dating been good for people’s romantic lives? This is debatable. And many single people complain that the apps and websites lead to disappointing interactions, matches, and dates. Some recent comments on online dating on Quora, the question-and-answering website, include the following complaints: “it is exhausting”; “A significant number of the profiles of very attractive and/or flirtatious women are really Nigerian scammers”; and there are too many “unsolicited pictures of men’s anatomy.”
But one effect of online dating is undebatable: it has been an unambiguous gain for scientists who study romance. It is fair to say that nobody in the field of romantic science complains about the existence of dating apps and websites.
You see, in the previous century, when the courtship process happened offline, the decisions that single people made were known by a select few and forgotten shortly afterward. If scientists wanted to know what people looked for in a partner, they basically had one approach: to ask them. A groundbreaking 1947 study by Harold T. Christensen did just that. Christensen surveyed 1,157 students and asked them to rate the importance of twenty-one traits in a potential romantic partner. The number one, single most important trait reported by both men and women was “dependable character.” Right near the bottom for both men and women, the traits they said they cared about least, were “good looks” and “good financial prospects.”
But can we trust these self-reports? People have long been known to lie on sensitive topics. (This, in fact, was the theme of my previous book, Everybody Lies.) Perhaps people don’t want to admit just how much they prefer to date people with pleasant faces, skinny waists, and hefty wallets.
In this century, researchers have better ways to figure out what people desire in a partner than merely asking them. When such a large percentage of courtships happens on apps or websites, daters’ profiles, clicks, and messages can be retained as data. The “Yays” and “Nays” are easily coded to csv files. And researchers around the world have mined data from OkCupid, eHarmony, Match.com, Hinge, and other matching services to determine how much just about every factor contributes to one’s desirability in the dating market. They have, quite simply, gathered unprecedented insights into what makes a human being desirable to other human beings.
As mentioned in the Introduction, there is some variation in what people find attractive—and daters can sometimes take advantage of this variation by occupying a niche market. However, the traits that make people more attractive, on average, are predictable.
So, what traits make people desirable to others?
Well, the first truth about what people look for in romantic partners, like so many important truths about life, was expressed by a rock star before the scientists figured it out. As Adam Duritz of the Counting Crows told us in his 1993 masterpiece “Mr. Jones”: we are all looking for “something beautiful.”
Someone Beautiful
A team of researchers—Günter J. Hitsch, Ali Hortaçsu, and Dan Ariely—studied thousands of heterosexual users of an online dating site.
Each user of the site included photos, and the researchers recruited and paid a different group of people to rate the attractiveness of every user, based on these photos, on a scale of 1 to 10.
With the help of the 1-to-10 ratings, the researchers had a measure of conventional physical attractiveness for every dater. They could test how much looks influence how desirable someone is. They measured desirability based on how many unsolicited messages a person received and how frequently their messages were responded to.
The researchers found that looks matter. A lot.
Roughly 30 percent of how well a female heterosexual dater performed on the site could be explained by their looks. Heterosexual women are a little less shallow but still plenty shallow. About 18 percent of male heterosexual daters’ success could be explained by their looks. Beauty, it turns out, is, for both sexes, the most important predictor of how many potential partners message and respond to one’s messages in online dating.
Place that finding in the “no duh” department, as well as in the “See, I knew when people told me that looks don’t matter, they were secretly superficial and thus totally and completely full of crap” department.
Someone Tall (If a Man)
The same team of researchers that studied how looks affect daters’ desirability also studied how height affected daters’ desirability. (Each dater on the site reported how tall they were.)
Once again, the results were stark. A man’s height had an enormous impact on how desirable he was to women. The most popular men were between 6′3″ and 6′4″; such men received 65 percent more messages than men who were between 5′7″ and 5′8″.
The researchers also studied the effects of income on daters’ desirability, which I will discuss shortly. This allowed them to make an interesting comparison between income and height in the dating market. They could ask how much more money a shorter man would have to earn to overcome his height disadvantage.
They found that a 6-foot man earning $62,500 per year is, on average, as desirable as a similar 5′6″ man who earns $237,500. In other words, those six inches of height are worth about $175,000 in salary in the dating market.
Source: Hitsch, Hortaçsu, and Ariely (2010)
The effect of height on desirability was reversed and far less pronounced for women. Generally, taller women had less success on the dating site. A woman who is 6′3″ tall, the researchers found, could expect to receive 42 percent fewer messages than a 5′5″ woman.
Someone of a Desired Race (Even if They’d Never Admit It)
Continuing the theme of superficial factors about a person that play a disturbing role in their success on the dating market, scientists have found significant evidence of racial discrimination in dating. Christian Rudder, a mathematician who was one of the cofounders of OkCupid, analyzed data from the messages of more than one million OkCupid users. He describes the results in his fascinating book, Dataclysm.
The two extremely disturbing charts below show the reply rates on OkCupid when heterosexual males and females of different races send messages to each other. If race did not influence dating decisions, the numbers in the chart would be exactly the same. In other words, a Black woman and a white woman sending a message to a white man would have the same chance of getting a reply. Instead, the numbers are very different. A Black woman has a 32 percent chance of getting a reply from a white male; a white woman has a 41 percent chance of getting a reply.
Overall, perhaps the most striking finding in the data is the difficulties African-American women face in the dating market. Note the second row of the first chart. Men of just about every racial group are less likely to respond to messages from Black women.
The second column of the second chart shows how African-American women respond to this harsh treatment by men: they become far less picky. For just about any group of men sending messages, Black women are the most likely to respond.
The dating experience of Black women is notably different from that of white males. White males tend to be significantly more likely to have their messages responded to. This is seen in the final row of the second chart. And they, in turn, become more picky, becoming the least likely to respond to messages from women. This is seen in the final column of the first chart.
Among males, the racial groups that receive the lowest response rates are Blacks and Asians.
Rudder’s charts are blunt. They show the overall response rates of every racial pair, but they do not consider any other differences between the groups that might lead to differences in response rates. Perhaps some of the reason that some racial groups, on average, perform better or worse in the dating market is that some racial groups earn, on average, different incomes.
Hitsch, Hortaçsu, and Ariely try to correct for these factors. They found that, when you take into account these other factors, the bias against Asian men becomes even more severe. Since Asian men in the United States have above-average incomes, which tends to be attractive to women, the low response rates to their messages is even more striking. The researchers determined that an Asian man would have to earn a staggering $247,000 more in annual income to be as attractive to the average white woman as he would if he were white.
Someone Rich
Back to findings in the no-surprise department: income affects one’s desirability in dating, with the biggest effect on males.
Hitsch, Hortaçsu, and Ariely found that, all else being equal, if a man increases his earnings from $35K to $50K to $150K to $200K, he can expect the average woman to be 8.9 percent more likely to contact him. If a woman’s earnings increase by the same amount, she can expect the average man to be 3.9 percent more likely to contact her.
Of course, it is well-known that a man with a substantial income can be attractive to heterosexual women. Consider the first line of Jane Austen’s Pride and Prejudice: “It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife.” Or consider the thought from the band Barenaked Ladies—who are, of course, really men: that if they “had a million dollars,” they’d be able to buy someone’s love.
Because the effects of wealth on romantic desirability and the efforts men make to earn more money are such cliches, I was actually surprised that the effects of income were rather modest.
Next, I will discuss the significant effects that a man’s occupation has on his romantic desirability, independent of his income. For example, all else being equal, males can expect significantly more romantic attention if they are firefighters than if they are waiters.
It turns out, sometimes a switch to a different, more attractive occupation can make a male more desirable than a large salary increase. For example, the data from online dating sites suggests that a man who earned $60K in the hospitality industry would become more desirable, on average, if he earned the same amount as a firefighter than if he stayed in the same industry but upped his salary to $200K. In other words, a male firefighter who earns $60K tends to be more attractive to the average heterosexual woman than a hospitality worker who earns $200K.
While many men believe they need to earn a substantial salary to “buy” a woman’s love, the data suggests that having a cool job is frequently more attractive than having a boring, but lucrative, job.
An Enforcer of the Laws or a Helper of Other People in Trouble (If a Man)
One’s job matters in the mating market—if you’re a man.
Hitsch, Hortaçsu, and Ariely had data from their online dating site on the occupation of daters. It turns out that a woman’s occupation doesn’t impact how many messages she receives, when you take into account her physical attractiveness.
For men trying to attract women, the story is different. Men who work in certain occupations receive more messages. And this is true taking into account everything else researchers know about them, including their income.
Male lawyers, police officers, firefighters, soldiers, and doctors get more messages than men who earn similar incomes, have similarly prestigious educations, are equally attractive, and are of the same height. Lawyers would be, on average, less attractive to women if they were accountants.*
Here is the list of occupations, ranked from most to least desirable to heterosexual women on online dating sites.
Desirability of Occupations in Men (Holding Constant Income)
Occupation
Percent Increase in Approaches from Women, Relative to a Student
Legal/attorney
8.6 %
Law enforcement/firefighter
7.7 %
Military
6.7 %
Health professional
5 %
Administrative/clerical/secretarial
4.9 %
Entertainment/broadcasting/film
4.2 %
Executive/managerial
4.0 %
Manufacturing
3.7 %
Financial/accounting
2.4 %
Self-employed
2.2 %
Political/government/civil
1.7 %
Artistic/musical/writer
1.7 %
Sales/marketing
1.4 %
Technical/science/engineering/research/computers
1.2 %
Transportation
1.0 %
Teacher/educator/professor
1.0 %
Student
0 %
Laborer/construction
−0.3 %
Service/hospitality/food
−3 %
Source: Hitsch, Hortaçsu, and Ariely (2010)
Someone with a Sexy Name
Some years ago, researchers randomly sent messages to online daters with different first names; they didn’t include a photo or any other information. They found that some names were as much as twice as likely to get clicks as other names. The sexiest names (those most likely to get a response) included:
Alexander
Charlotte
Emma
Hannah
Jacob
Marie
Max
Peter
The least sexy names (those least likely to get a response) included:
Celina
Chantal
Dennis
Jacqueline
Justin
Kevin
Mandy
Marvin
Someone Just Like Themselves
Do we look for mates who are similar to us or different than us?
Emma Pierson, a computer scientist and data scientist, studied 1 million matches on eHarmony and wrote up her results on the data journalism site FiveThirtyEight. She examined 102 traits that eHarmony measures on partners and crunched the numbers to see whether people were more likely to pair with someone who shared the trait. Pierson found it was no contest: similarity, rather than difference, leads to attraction.
Heterosexual women are especially drawn to similarity. Pierson found that, for literally every one of the 102 traits, a man’s sharing the same trait was positively correlated with a woman’s contacting him. This included seemingly central traits such as age, education, and income, as well as quirkier traits, such as how many photos they included in their profile or if they used the same adjectives in their profiles. A woman who describes herself as “creative” is more likely to message a man who describes himself the same way. Heterosexual men also showed a preference for women like themselves, although the preference wasn’t quite as strong.*
As Pierson’s FiveThirtyEight article was titled, “In the End, People May Really Just Want to Date Themselves.”
Pierson’s findings that similarity leads to matches was confirmed in another study, this one using Hinge data. These researchers also had a clever title for their study, “Polar Similars.” The researchers also discovered a new, quirky dimension in which daters are drawn to similarity: initials. Hinge users are 11.3 percent more likely to match with someone who shares their initials. And this effect isn’t driven by people from the same religions both sharing initials and matching more frequently—say, Adam Cohen matching with Ariel Cohen. The elevated match propensity of people who share the same initials holds taking into account religious affiliation.*
Opposites attract, the data tells us, is a myth. Similarity attracts—and the effects are large.
What Predicts Romantic Happiness
The fascinating, if sometimes disturbing, data from online dating sites tells us that single people predictably are drawn to certain qualities. But should they be drawn to these qualities?
If you are like the average single dater—predictably clicking on people with the traits the scientists found are most desired—are you going about dating correctly? Or are you dating all wrong?*
Recall, at the beginning of this chapter, I discussed the research of Samantha Joel and coauthors. Recall that they collected history’s largest dataset on couples and the qualities of those couples. Recall that they found that it was surprisingly difficult to predict whether a person was happy with a romantic partner based on a large list of traits. There is not a set of traits that guarantee romantic happiness or preclude romantic happiness. And no algorithm in the world can predict, with enormous accuracy, whether two people will end up happy together.
That said, there was some predictive power in some traits, some factors that do increase the odds at least somewhat that a person will be happy in their romantic relationship. I will now discuss what does predict romantic happiness—and how little it has to do with the qualities that people look for in a romantic partner.
“It’s Not You, It’s Me”: The Data Science Says So
Say there is a person, John, and he is partnered with Sally. You want to predict whether John is happy in the relationship. You are allowed to ask John and/or Sally any three questions about themselves and use this information to predict John’s relationship happiness.
What questions would you want to ask? What would you want to know about the two members of this couple?
According to my read of the research of Joel and her coauthors, the best three questions to figure out whether John is happy with Sally would have nothing to do with Sally; in fact, all would be related to John. The best questions to predict John’s happiness with Sally would look something like these:
“John, were you satisfied with your life before you met Sally?”
“John, were you free from depression before you met Sally?”
“John, did you have a positive affect before you met Sally?”
Researchers found that people who answered “yes” to questions such as these are significantly more likely to report being happy in their romantic relationship. In other words, a person who is happy outside their relationship is far more likely to be happy inside their relationship, as well.
Further—and this was quite striking—how a person answered questions about themselves was roughly four times more predictive of their relationship happiness than all the traits of their romantic partner combined.*
Of course, the finding that one’s happiness outside of a relationship can have an enormous impact on one’s happiness inside that relationship is hardly a revolutionary idea. Consider this saying that was featured on Daily Inspirational Quotes: “Nobody can make you happy until you’re happy with yourself first.”
This is the type of quote that often makes cynical data geeks like myself roll our eyes. However, now, after reading the work of Joel and her coauthors, I have become convinced that this quote is largely true.
This relates to an important point about living a data-driven life. We data geeks may be most excited when we learn of a finding that goes against conventional wisdom or cliched advice. This plays to our natural need to know something that the rest of the world doesn’t. But we data geeks must also accept when the data confirms conventional wisdom or cliched advice. We must be willing to go wherever the data takes us, even if that is to findings like those featured on Daily Inspirational Quotes.
So, as discovered by both a team of eighty-six scientists and whoever writes Daily Inspirational Quotes, one’s own happiness outside a relationship is by far the biggest predictor of one’s happiness in a romantic relationship. But what else predicts romantic happiness beyond one’s own preexisting mental state? What qualities of a mate are predictive of romantic happiness? Let’s start with the qualities of one’s mate that are least predictive of romantic happiness.
Looks Are Overrated—and Other Advice That You Have Long Been Told and Consistently Ignored but Might Be Slightly More Likely to Follow, Knowing That Data Science Has Confirmed It
Among more than 11,000 long-term couples, machine learning models found that the traits listed below, in a mate, were among the least predictive of happiness with that mate. Let’s call these traits the Irrelevant Eight, as partners appear about as likely to end up happy in their relationship when they pair off with people with any combo of these traits:
Race/ethnicity
Religious affiliation
Height
Occupation
Physical attractiveness
Previous marital status
Sexual tastes
Similarity to oneself
What should we make of this list, the Irrelevant Eight? I was immediately struck by an overlap between the list of irrelevant traits and another data-driven list discussed in this chapter.
Recall that I had previously discussed the qualities that make people most desirable as romantic partners, according to Big Data from online dating sites. It turns out that that list—the qualities that are most valued in the dating market, according to Big Data from online dating sites—almost perfectly overlaps with the list of traits in a partner that don’t correlate with long-term relationship happiness, according to the large dataset Joel and her coauthors analyzed.
Consider, say, conventional attractiveness. Beauty, you will recall, is the single most valued trait in the dating market; Hitsch, Hortaçsu, and Ariely found in their study of tens of thousands of single people on an online dating site that who receives messages and who has their messages responded to can, to a large degree, be explained by how conventionally attractive they are. But Joel and her coauthors found, in their study of more than 11,000 long-term couples, that the conventional attractiveness of one’s partner does not predict romantic happiness. Similarly, tall men, men with sexy occupations, people of certain races, and people who remind others of themselves are valued tremendously in the dating market. (See: the evidence from earlier in this chapter.) But ask thousands of long-term couples and there is no evidence that people who succeeded in pairing off with mates with these desired traits are any happier in their relationship.
If I had to sum up, in one sentence, the most important finding in the field of relationship science, thanks to these Big Data studies, it would be something like as follows (call it the First Law of Love): In the dating market, people compete ferociously for mates with qualities that do not increase one’s chances of romantic happiness.
Moreover, if I had to define the qualities that are highly desired even though they don’t lead to long-term romantic happiness, I would call many of them shiny qualities. Such qualities immediately grab our attention. Just about all of us are quickly drawn to the conventionally beautiful, for example. But these attention-grabbing, shiny qualities, the data suggests, make no difference to our long-term romantic happiness. The data suggests that single people are predictably tricked by shininess.
Youkilis of Love: A Data-Driven Emphasis on Undervalued Assets
After poring through all this research in relationship science, it occurred to me that the dating market today has a striking similarity to the baseball market in the 1990s.
Recall the Moneyball revolution, which was the motivation for this book. The Oakland A’s and a few other teams realized, thanks to data analysis, that the market for baseball was all off. There was a disconnect between the cost of players in the open market (think: the salary you would have to pay them) and the value players brought to your team (think: how many wins they can add).
Players were frequently drafted and paid based not on the value they were likely to bring to the team but on other factors. The baseball market tended to overvalue shiny qualities in baseball players, such as being good-looking, and undervalued players who didn’t immediately look like they should be baseball stars.
One such undervalued player was Kevin Youkilis. Youkilis has been described as “a fat third baseman who couldn’t run, throw, or field.” His college coach explained, “He was kind of a square-shaped body, a guy [who] in a uniform didn’t look all that athletic. He wasn’t a tall, prospect-y looking guy. He looked chubby in a uniform.” Despite incredible college statistics, Youkilis’s odd-for-a-baseball-player appearance caused him to fall to the eighth round of the draft.
But data analysts knew that, despite not looking the part of a great professional baseball player, Youkilis had all the tools that really mattered. The Boston Red Sox, motivated by such data analysis, took him in the eighth round, to the great frustration of the A’s general manager, Billy Beane. Beane desperately wanted him but thought he would fall even further. The relatively short and chubby pick would eventually become a three-time All-Star and help lead his team to two World Series championships.
Data-driven teams in the 1990s, in other words, had success by focusing their attention on players, like Youkilis, who lacked the shiny traits that excited teams that didn’t know the data. As Michael Lewis put it, “The human mind played tricks on itself when it relied exclusively on what it saw, and every trick it played was a financial opportunity for someone who saw through the illusion to the reality.”
Similarly, the data has revealed striking inefficiencies in the dating market, with the mind playing tricks on single people. There is a disconnect between the cost of potential matches (think: how difficult it is to get dates with them) and the value of potential matches (think: the chances that they will make you happy in a long-term relationship).
So, could you approach dating with a similar mindset as that of Billy Beane? Could you focus more of your dating attention on targets whom the rest of the dating market ignores, even though they are just as likely to be great romantic partners?
The following groups of people, data has proven, all are dramatically less competed over in dating, even though the evidence suggests they are just as capable of making a partner happy.
Massively Undervalued Groups in the Dating Market
Short men
Extremely tall women
Asian men
African-American women
Men who are students or work in less desirable fields, such as education, hospitality, science, construction, or transportation
Conventionally less attractive men and women
By focusing more of your romantic attention on these groups of people, you will face less competition for an amazing mate. You are more likely to find a great partner whom others incorrectly ignore. You may find your Youkilis of Love!
OF COURSE, TELLING PEOPLE TO CARE LESS ABOUT SHINY QUALITIES, such as conventional attractiveness, that are overly valued in the dating market, while it may be sound, data-driven advice, seems like difficult advice to follow. There is a reason that shiny qualities are desired: shininess, almost by definition, grabs our attention. Return to Adam Duritz’s point that we are all looking for “something beautiful.” Is there any evidence-based way to allow yourself to stop being fooled by shininess in your search for romantic fulfillment?
One important, relevant, fascinating, and data-driven finding was uncovered by researchers at the University of Texas. In the beginning of a course, the professors asked all the heterosexual students in that course to rate the attractiveness of each of their opposite-sex classmates. Not surprisingly, there was a good deal of consensus. Most people picked the same classmates as the most attractive; these people were, by definition, conventionally attractive. Think Brad Pitt or Natalie Portman or the closest equivalents in the class.
At the end of the course, professors again asked the students to rate the attractiveness of each of their opposite-sex classmates. This is where the study got interesting. Now there was more disagreement in the attractiveness ratings. At the end of the class, people were far more likely to rate a person that other people didn’t find so attractive as the most attractive.
What happened between the beginning and the end of the course that led so many people to change the rankings of their classmates’ attractiveness? The students spent time with each other.
The man with the hunter eyes and chiseled chin may have seemed attractive at the beginning of the class. But he became less attractive to people who didn’t enjoy their time talking to him. The woman with the hooked nose and the low cheekbones may have seemed unattractive at the beginning of the class. But she became more attractive to people who enjoyed their time talking to her.
The results of the study have profound implications for how we approach dating. Recall that we tend to seek out mates with conventional physical attractiveness and some other shiny traits—the types of people who would have scored very high on attractiveness ratings on the first day of a course—even though these people are highly competed-after and don’t make for better mates. When we come across people lacking these shiny traits, we tend to not feel attraction toward them and not go on dates with them.
The research suggests that we might overrule our initial lack of attraction. Physical attraction, research shows, can grow over time if we like a person (or disappear over time if we don’t like a person). The data suggests we should go on more dates with undervalued assets (those who might not have the qualities that so many people find so alluring) even if we don’t initially find them attractive—and be patient, allowing a potential attraction to grow.
So much for the qualities that don’t predict relationship happiness, such as conventional attractiveness. What are the qualities that do?
The Most Likely Best Mate: Someone Satisfied with Life, Secure with Who They Are, Who Conscientiously Tries to Better Themselves
Joel and her coauthors found a few qualities of partners that did have some predictive power in how happy their romantic partners were. Their research suggests the following qualities are among the most predictive of a good mate:
Satisfaction with life
Secure attachment style (Be patient if you do not know what this—and a few other phrases—mean; I will explain them shortly.)
Conscientiousness
Growth mindset
What should we make of this list?
The first lesson may be that, to up your odds of romantic happiness, you should . . . read obscure psychology journals to learn what their terms mean. It turns out that the best predictors of how happy you are likely to be with a romantic partner are how they score on various quizzes that psychologists have come up with. This means, next time your partner suggests that you should turn off the ballgame and join her on the couch to take some new psychology quiz she discovered on the internet—instead of throwing a big fit about how you hate these stupid psychology quizzes and just want to be left alone to watch sports for one freakin’ night and maybe it is best being single in life anyway—you should join her. Then you can find out if she has the qualities that make for a good long-term mate. Better yet, you might suggest taking these quizzes yourself.
So what do these quizzes determine?
Satisfaction with life is self-explanatory. People who are satisfied with life tend to make for better long-term relationship partners. Warning: silly Mick Jagger semi-joke coming. When Jagger gets onstage and sings “I Can’t Get No Satisfaction,” his voice, rhythm, and charisma may be hot, but listen to the words and you will notice a red flag for his capability to make a woman happy in a long-term relationship.
Attachment styles are explained in the excellent book Attached by Amir Levine and Rachel Heller. People with secure attachment style—this is the ideal trait in a partner—can trust people and are trustworthy, are comfortable expressing interest and affection, and have an easier time being intimate with others. An attachment style test can be taken here: https://www.attachmentproject.com/attachment-style-quiz/.
Conscientiousness is one of the Big Five personality traits, first proposed by Ernest Tupes and Raymond Christal in 1961. Conscientious people are disciplined, efficient, organized, reliable, and, according to Joel and her coauthors, better long-term partners. A conscientiousness quiz can be taken here: https://www.truity.com/test/how-conscientious-are-you.
Growth mindset is a trait developed by the psychologist Carol Dweck. People who have a growth mindset tend to believe they can improve their talents and abilities through hard work and persistence. Such people may work to become better romantic partners, which may be why they end up being just that. A growth mindset test can be taken here: https://www.idrlabs.com/growth-mindset-fixed-mindset/test.php.
The traits most predictive of romantic happiness are quite striking—and have profound implications for how we think about the romantic market. Recall the frequently depressing data from online dating sites on the ruthless superficiality that daters show. People really, really want sexy mates, even though there aren’t that many sexy mates to go around.
It would certainly have been possible, in the data from actual relationships, that people who ended up with sexy mates ended up happier. Perhaps they really felt long-term joy from the wild sex or were satisfied that they could impress people at parties with the hotness of their partner. But the data from thousands of couples shows that this just isn’t so. If anybody is likely to end up happier, it is daters who picked a mate with nice character traits.
And you can learn from the romantic successes and failures of more than ten thousand other couples. In looking for a mate, don’t judge people based on the color of their skin, the symmetry of their faces, the height of their bodies, the sexiness of their profession, or whether they happen to share your initials. In the long term, data tells us, it really is the content of their character that matters most.
The Seemingly Random, Unpredictable Connection Between Two People
Why do some couples get happier over time? Why do some relationships that start great fall apart over time?
Samantha Joel and her coauthors also tried to answer these questions. The researchers took advantage of the fact that, for many relationships in the dataset, the romantic partners had been surveyed multiple times, sometimes years apart. Some romantic partners reported that, whereas they started unhappy in their relationship, they became increasingly happy. Others reported the reverse. What do couples that get better over time tend to have in common? What about those that get worse?
As part of their groundbreaking study, Joel and her coauthors used machine learning on the extensive data they had collected on thousands of couples to try to predict changes in romantic happiness. Note that this is a different question from the one in the last project we discussed, in which the researchers tried to predict whether members of romantic couples were happy at a particular point in time.
So, what could enormous datasets and machine learning tell us about the long-term trajectory of relationships? What do a couple’s demographics, values, psychological traits, and preferences tell us about whether their relationship will get better or worse over time?
Nothing.
Joel and her coauthors’ models had virtually no predictive power in predicting changes in romantic happiness. Happy couples are more likely to be happy in the future. Unhappy couples are more likely to be unhappy in the future. But there is nothing else about the two people that could improve predictions about future happiness.
The failure of the predictive models here has, I would argue, important implications for people’s romantic decisions.
Certainly, many people make romantic decisions based on projected changes in happiness. How many times has one of your friends stayed in a relationship in which they weren’t happy because they think, on paper, they should be happy—and eventually will be happy? “Sure, I’m miserable now,” the friend might say. “But this relationship should work. It has to get better.”
The results suggest that people are largely making a mistake when they expect their happiness in a relationship to change based on various qualities of them and their partner. The friend who stays in a relationship in which he is unhappy because he thinks he and his partner have so much in common and will eventually be happy is making a mistake.
The data suggests there is no better way to predict your future romantic happiness than your current romantic happiness. And if a partner does not make you happy now, you are wrong to assume that, because of the qualities of you and your partner, you will be happy in the future.
Or, to sum up all the data-driven advice on how to pick a mate: When you are single, focus more of your romantic search energy on people who lack the traits that are highly competed over. Focus more of your attention on people with strong psychological traits. Once you are in a relationship, pay attention exclusively to how happy you are with your partner—and do not worry about or gain false confidence from the similarities or differences between you and your partner. Don’t think you have some ability to recognize a currently good relationship that will go south or a currently bad relationship that will go north. If the world’s greatest contemporary scientists, using the most comprehensive dataset ever assembled, can’t predict these types of changes, you can’t, either.
Up Next
If you find a mate, you may have kids. And, if you have kids, you will undoubtedly wonder how you can be a better parent. There are new, important insights in what makes a great parent in enormous datasets, particularly the tax records of hundreds of millions of Americans.
Chapter 2: Location. Location. Location. The Secret to Great Parenting.
Parenting is, in a word, challenging. A recent study calculated that, in the first year of a baby’s life, parents face 1,750 difficult decisions. These include what to name the baby, whether to breastfeed the baby, how to sleep-train the baby, what pediatrician to get the baby, and whether to post pics of the baby on social media.
And that is only year one! Parenting doesn’t become easier after that. In fact, parents have ranked the age of eight as the most difficult year to parent.
How can parents make these thousands of difficult decisions? Of course, parents can always turn to Google, where they will find plenty of supposed answers to just about any parenting question they have. But conventional parenting advice tends to be split between the obvious and the conflicting.
Examples of the obvious: KidsHealth.org urges parents to “Be a Good Role Model” and “Show That Your Love Is Unconditional.” Examples of the conflicting: recently, the New York Times published an article that recommended parents “Try timeouts” to discipline their kids. In 2016, PBS NewsHour published a column online, “Why you should never use timeouts on your kids.”
A frustrated mother, Ava Neyer, ranted after reading large numbers of books on parenting, particularly on baby sleep and development:
Swaddle the baby tightly, but not too tightly. Put them on their backs to sleep, but don’t let them be on their backs too long or they will be developmentally delayed. Give them a pacifier to reduce SIDS. Be careful about pacifiers because they can cause nursing problems and stop your baby from sleeping soundly. If your baby sleeps too soundly, they’ll die of SIDS.
Ava Neyer, I would be lying if I said I could relate. (I’m not a parent; I’m merely an uncle. My decision-making process largely consists of asking my mom what gift I should get my nephew and her telling me “get him a truck” and me getting him a truck and then my nephew thanking me for the next four years for once having gotten him a truck.)
Regardless, I’ve scoured the parenting literature to understand what data can tell Ava and all the other parents out there. Is there anything parents can learn that is both not obvious and not conflicting? Can science offer advice for the thousands of difficult decisions that parents must make?
While there is not yet a convincing science-driven answer to every one of the 1,750 first-year parenting decisions or the thousands of decisions beyond that, there are two extremely important, scientifically proven, and non-obvious lessons from science about parenting.
Lesson number one: the overall effect of most of the decisions that parents make add up to less than most people expect; this suggests that parents fret too much about the vast majority of decisions that they must make.
Lesson number two: there is one decision that a parent makes that is the most important—and many parents make the wrong decision here. If a parent makes a great, data-driven decision on this choice, that by itself would make any parent a far above-average one.
We will explore these lessons—and the evidence for them—in turn.
The Overall Effects of Parents
Let’s start with the most basic question about parenting: How much do parents matter? How much can “great” parents improve a kid’s life, compared to average parents?
You could imagine three different worlds.
World 1 (Great Parents Can Turn a Potential Flight Attendant into a Dental Hygienist)
In this world, great parents can raise a kid who would otherwise have a middle-income job, making about $59,000 per year (say, a plumber or flight attendant), and turn them into a person with a job that pays slightly above average, making $75,000 per year (perhaps as a registered nurse or dental hygienist).
World 2 (Great Parents Can Turn a Potential Flight Attendant into an Engineer)
In this world, great parents can raise a kid who would otherwise have a middle-income job, making about $59,000 per year (say, a plumber or flight attendant), and bring them into the upper middle class, making $100,000 per year (perhaps as an engineer or a judge).
World 3 (Great Parents Can Turn a Potential Flight Attendant into a Brain Surgeon)
In this world, great parents can raise a kid who would otherwise have a middle-income job, making about $59,000 per year (say, a plumber or flight attendant), and make them rich, earning $200,000 per year (perhaps as a surgeon or psychiatrist).
Many people think that we live in World 2 or World 3—that skilled parents can help propel just about any kid a few rungs up on the socioeconomic ladder.
And it is not in doubt that certain parents have raised more than their fair share of noteworthy kids. Just consider Benjamin and Marsha Emanuel—and their three sons, Ari, Ezekiel, and Rahm.
Ari is a high-powered Hollywood agent who was the basis of Ari Gold in the HBO show Entourage.
Ezekiel is a vice provost at the University of Pennsylvania.
Rahm was the White House chief of staff to Barack Obama and the mayor of Chicago.
In other words, Benjamin and Marsha reared sons who reached the highest echelons of business, academia, and politics.
Now, I know what some of my Jewish readers are thinking after learning of the life outcomes of Benjamin and Marsha Emanuel’s offspring. Some of you Jews are thinking, “Yeah. That’s all well and good. But did the Emanuels rear a doctor?”
An old Jewish joke (1/n of old Jewish jokes in this book) goes as follows:
“The first Jew ever was elected president. At the inauguration, his mother is sitting amongst all the dignitaries, as he is being sworn in. She tells everyone around her, in a loud voice, ‘See that fellow up there being sworn in? His brother is a doctor.’ ”
No need to worry. Ezekiel, in addition to his academic career, is an oncologist.
There is even a supposed lesson in parenting from the success of the Emanuel brothers, as Ezekiel wrote about their upbringing in his book, Brothers Emanuel.*
In Brothers Emanuel, we learn that every Sunday, while most families would watch Chicago Bears games, the Emanuel family would go on a cultural excursion, perhaps to the Art Institute of Chicago or a musical. When the boys yearned for lessons in karate or jujitsu, their mother insisted they instead take lessons in ballet. All three were taunted by other kids for it but now think the experience helped build discipline, character, and fearlessness.
The seeming lesson from the Emanuels: Encourage your kids to be cultured and different. Make your boy wear tights even if other boys mock him.
But, in truth, a single family, no matter how accomplished, cannot prove the validity of any given parenting strategy. And it is easy to find counterexamples to the Emanuel lesson. Take, for example, Dale Fernsby.* Fernsby recently responded to a mother on Quora, the community question-and-answer site, who wanted advice on whether she should sign her son up for ballet. Fernsby noted that his mother had enrolled him in many artsy activities as a kid, even though he hated them and was bullied for them. He says he learned from this experience that he was not allowed to have his own opinions or identity. He believes it caused him to have low self-esteem, an inability to express himself, and resentment toward his mother.
One challenge with learning how parents influence their kids is that one anecdote can never tell us all that much. Should we learn from the Emanuel story or the Fernsby story?
Another challenge with learning about parental influence: correlation doesn’t imply causation.
For much of the twentieth century, scholars searched for correlations in reasonably sized datasets between parenting strategies and child outcomes. They found many. Some of these correlations were summarized in the excellent book The Nurture Assumption, by Judith Rich Harris. For example, kids whose parents read a lot to them tend to have great educational accomplishments.
But how much of these correlations is causal? There is a major confounding issue: genetics. You see, parents don’t just give their kids museum visits or ballet lessons or books. They also give them DNA. Return to the correlation between reading to kids and having well-educated kids. Are the kids drawn to education because of the books their parents read? Or are both parent and child drawn to books and knowledge because of their genetics? Is it nature or nurture?
Many stories have shown the power of genes in driving people’s outcomes. Consider the evidence from identical twins who were raised apart. These people share the exact same genes but none of the same upbringing. Take, for example, the identical twins Jim Lewis and Jim Springer, who were raised separately from the age of four weeks. They reunited at the age of thirty-nine and found that they were each six feet tall and weighed 180 pounds; bit their nails and had tension headaches; owned a dog when they were kids and named him Toy; went on family vacations at the same beach in Florida; had worked part-time in law enforcement; and liked Miller Lite beer and Salem cigarettes.
There was, however, one notable difference between the two Jims. They had given different middle names to their firstborn children. Jim Lewis named his firstborn James Alan, while Jim Springer named his James Allan.
Had Mr. Lewis and Mr. Springer never met each other, they might have assumed their parents played big roles in creating some of their tastes. But it appears those interests were, to a large degree, coded in their DNA.
Steve Jobs, who was adopted, had his own epiphany in the importance of genetics when he met, for the first time, his biological sister, Mona Simpson, at the age of twenty-seven. He was struck by how similar they were, including having both risen to the top of a creative field. (Simpson is an award-winning novelist.) As Jobs told the New York Times, “I used to be way over on the nurture side, but I’ve swung way over to the nature side.”
Even the Emanuel brothers’ story, which, on its surface, seems to show the power of great parenting, has a little-known kicker that suggests nurture may not have been the driver of the three boys’ success. After having their three biological kids, Benjamin and Marsha Emanuel adopted a fourth child, Shoshana. Despite sharing the same cultural exposure of her three brothers, she did not share their genes—and has not had the same success.*
Is there any scientific way to determine just how much parents can affect their kids? To test the causal effects of parents on kids, we would seemingly have to randomly assign different kids to different parents—and study how they turned out. It turns out, this has been done. The first convincing evidence on the overall effects of parents is all thanks to . . . a documentary on the Korean War.
One day, in 1954, an Oregon couple, Harry and Bertha Holt, parents to six children, saw a documentary on a theme they previously knew little about: Korean “G.I. Babies.” These kids lost their parents during the Korean War and were now being raised in orphanages—with a shortage of food and love.
Now, Harry and Bertha Holt did not respond to this documentary the way that I tend to respond to documentaries: by zoning out during the entire film and then desperately trying to fool my girlfriend into thinking I had any idea what it was about. No, the Holts responded to the documentary on Korean orphans by deciding that they wanted to go to Korea and adopt many of these orphans.
The Holts’ ambitious plan to adopt many orphans they had just seen in a documentary faced just one obstacle: the law. At the time, American laws allowed people to adopt at most two foreign children.
This obstacle proved fleeting. The Holts lobbied Congress to change the law. Congresspeople, impressed by the Holts’ desire to do good, were persuaded. The Holts went to Korea. And, in a short time, they were back in Oregon—accompanied by eight new kids. The Holts were now a family of sixteen!
Soon news organizations covered the Holts’ story. Radio stations told it. Newspapers wrote of it. Television stations broadcast it.
Further, just as the Holts were moved to action when they first learned the story of the “G.I. Babies,” thousands of Americans were moved to action when they learned the story of the Holts. One after another, Americans said they wanted to follow in the path of the Holts. They wanted to adopt orphans as well.
Enter Holt International Children’s Services, a foundation that makes it easier for Americans to adopt foreign babies. Over the years, more than thirty thousand Korean children have been adopted into the United States thanks to this organization. Parents merely sign up, get approved, and then get the next available child.
What does this story have to do with the science of parenting? Well, Bruce Sacerdote, a Dartmouth College economist, heard about the Holt program. Like so many other Americans, he was motivated to do something. In fact, he was motivated to run regressions!
You see, the process Holt uses to assign children to parents is essentially random, which means scientists have an easy way to test the effects of parents. They can simply compare adopted brothers and sisters who were randomly assigned to the same parents. The more the parents can influence the child, the more these brothers and sisters will end up alike. And, unlike with studies of genetically related children, we don’t have to worry about genetics driving any correlations.
One cool thing about Sacerdote’s study of the Holt program is that it allows us to see the effects of parenting, which we will get to in a bit. Another cool thing about the study is that we can see the difference between how nonprofit leaders and an economist describe the same foundation.
First, here is the description of Holt International Children’s Services, from the organization itself. They report that they are “bringing light to the darkest situations” and “help strengthen vulnerable families, care for orphans, and find adoptive families for kids.”
Next, here is the description of Holt International Children’s Services by the economist, Sacerdote:
The random assignment of adoptees to families ensures that birth mother’s education is uncorrelated with adoptive mother’s education . . . [T]herefore β1 is not biased by the omission of the first and third terms in (1).
Holt International Children’s Services believes they are “bringing light to the darkest situations.” Sacerdote thinks they are making sure “β1 is not biased by the omission of the first and third terms in (1).” I would argue that they are both right!
Anyway, what did that unbiased β1 tell us? In most cases, that the family a kid is raised in has surprisingly little impact on how that kid ends up. Adopted brothers and sisters who were essentially randomly assigned to be raised in the same home end up only a little more similar than adopted brothers and sisters who were raised separately.
Remember, earlier I said there were three possible worlds, each representing a different degree to which parents might influence their kids. Sacerdote’s study suggests that we live in World 1, the one in which parents don’t have an enormous impact. A one standard deviation increase in the environment in which a child is raised, Sacerdote found, might raise a child’s adult income by about 26 percent—not nothing but not too many rungs up the socioeconomic latter. Further, Sacerdote found the effects of nature on a child’s income were some 2.5 times larger than the effects of nurture.
Sacerdote’s study was just part of the evidence on the surprisingly limited effects of parenting. Other researchers have done further studies of adoptees. Researchers have also developed an ingenious method involving twins that allows them to disentangle the effects of nature and nurture, a method that I will explain in the next chapter.
Over and over, these studies converged on similar results, results that were summarized by Bryan Caplan in his provocative book, Selfish Reasons to Have More Kids: “Twin and adoption studies find that the long-run effects of parenting are shockingly small.”
Parents, as surprising as it seems, and as the best evidence on the topic suggests, have only small effects on:
Life expectancy
Overall health
Education
Religiosity
Adult income
They do have moderate effects on:
Religious affiliation
Drug and alcohol use and sexual behavior, particularly during the teens
How kids feel about their parents
There are, of course, extreme examples in which parents can have an enormous impact on things like education and income. Consider the billionaire Charles Kushner, who gave $2.5 million to Harvard University—which likely caused the school to accept his son Jared despite a fairly low high school GPA and SAT scores—and then gave Jared a stake in his lucrative real estate business. It is fair to say that Jared’s educational achievements and wealth are far greater than they would be if he had had a different father. At the risk of being presumptuous, I think it is clear that Kushner’s estimated $800 million net worth is many times higher than it would have been had he not inherited a real estate empire. But the data suggests that the average parent—the one deciding between, say, how much to read to their kids, rather than how many millions to give to Harvard—has limited effects on a kid’s education and income.
If the overall effects of parenting are smaller than we expect, this suggests that the effects of individual parenting decisions are likely to be smaller than we expect. Think about it this way: if parents face thousands of decisions and the parents who make far better decisions only have kids who turn out some 26 percent more accomplished, each of the thousands of decisions, by itself, can’t make a large difference.
Indeed, the best studies—many of them discussed in important books by Emily Oster—have generally failed to find much effect from even the most-debated parenting techniques. Some examples:
The only randomized controlled trial on breastfeeding found that it had no significant long-term effect on a variety of child outcomes.
A careful study of television use found that exposure to TV had no long-term effects on child test scores.
A careful randomized trial suggests that teaching kids cognitively demanding games, such as chess, doesn’t make them smarter in the long term.
A careful meta-analysis of bilingual education finds that it only has small effects on various measures of a child’s cognitive performance, and the effects may be entirely due to a bias in favor of publishing positive results.
Also, as it relates to the Emanuel/Fernsby debate about the merits of getting ballet lessons for boys, a meta-analysis found “limited evidence” that participation in dance programs may reduce anxiety; however, the authors suggest that this may be due to studies that are of “low methodological quality” and the results “should be treated with caution.”
Look at careful studies rather than the latest attention-grabbing study, and you find that many of the things that parents worry about most turn out to have surprisingly little effects on their kids. Quite simply, most of the decisions that parents make matter less than they expect—and less than the parenting-industrial complex would like us to believe.
As Caplan put it,
If your child had grown up in a very different family—or if you had been a very different parent—he probably would have turned out about the same. You don’t have to live up to the exhausting standards of the Supermom and Superdad next door. Instead, you can raise your kids in the way that feels comfortable for you, and stop worrying. They’ll still turn out fine.
Or as Caplan titled one of his sections of his book, offering his best advice to parents based on a couple of decades of social science research: “Lighten Up.”
That, I would say, would be the best read of scientific-based parenting advice circa 2011, when Caplan wrote his book. Since 2011, the evidence has continued to accumulate that the overall effects of everything a parent does are smaller than most expect and that most of the decisions parents worry about don’t have a measurable impact on how a kid turns out. However, there is an important update now, as there is some evidence that one decision that parents make may be the most important by far—and worth deeper consideration.
I would now advise parents: “Lighten Up . . . Except for One Choice You Make.”
The Effects of a Neighborhood
“Asiyefunzwa na mamaye hufunzwa na ulimwengu.”
Those are the words of my favorite African proverb. It translates, from Swahili, roughly as follows: “It takes a village to raise a child.”
In case anyone is curious, my other favorite African proverbs (in English) are:
“Rain does not fall on one roof alone.”
“Not everyone who chased the zebra caught it; but he who caught it, chased it.”
“No matter how hot your anger is, it cannot cook yams.”
But back to “It takes a village to raise a child.”
In January 1996, Hillary Clinton, then the first lady of the United States, expanded this proverb into a book, It Takes a Village: And Other Lessons Children Teach Us. Clinton’s book—and that African proverb—argue that a child’s life is shaped by many people in the child’s neighborhood: the firefighters and police officers, mail carriers and garbage collectors, teachers and coaches.
At first, Clinton’s book seemed like another in a long line of entirely uncontroversial books that politicians write before seeking higher office, joining the likes of John F. Kennedy’s 1956 profiles of courageous people, George H. W. Bush’s 1987 recommendation to “look forward,” and Jimmy Carter’s 1975 defense of doing one’s best.
However, a few months after Clinton’s book was published, Bob Dole, the 1996 Republican nominee for president, thought he might be able to capitalize on the negative feelings many held towards the then first lady. And he eyed a seeming weakness in Clinton’s seemingly incontestable thesis. Dole argued that Clinton’s book, by emphasizing the important role that community members can play in a child’s life, minimized the responsibility that parents have to raise these children. Clinton’s argument, Dole claimed, was actually a subtle attack on family values. At the Republican convention, Dole pounced. “With all due respect,” Dole said, “I am here to tell you: it does not take a village to raise a child. It takes a family to raise a child.” The crowd roared. And that, friends, is the story of how the largest ovation at the 1996 Republican convention was devoted to an attack on a beautiful, moving, and touching African proverb.
So, who was right, Bob Dole or Africa?
For twenty-two years, the honest answer by data-minded scholars would be . . . (shoulder shrug). There hasn’t been conclusive research one way or the other. The problem, once again: that difficulty establishing causality.
Sure, some neighborhoods produce more successful kids. Here’s a fun fact that I talked about in my last book: Among Baby Boomers, one in every 864 kids born in Washtenaw, Michigan, the county that includes the University of Michigan, achieved something notable enough to warrant an entry in Wikipedia. One in 31,167 kids born in Harlan, Kentucky, a largely rural county, made it to Wikipedia. But how much of this is due to the kids of professors and other upper-middle-class professionals being really smart and ambitious—intelligence and drive they also would have used had they been born in rural Kentucky? Quite simply, the populations born in different neighborhoods are different, making it seemingly impossible to know how much a given neighborhood is causing its kids to succeed.
The shoulder-shrug-best-response to the effects of a neighborhood was the case until roughly five years ago. That was when the economist Raj Chetty began looking at this question.
RAJ CHETTY IS A GENIUS. DON’T BELIEVE ME? BELIEVE THE MacArthur Foundation, who in 2012 awarded him their “Genius Grant.” Or believe the economics profession, who in 2013 gave him their John Bates Clark Medal for the best economist under the age of forty. Or believe the government of India, who in 2015 gave him the Padma Shri, one of their highest honors. Or believe the economist Tyler Cowen, who has called Chetty “the single most influential economist in the world today.”
So, yeah, basically everybody is united in the belief that Chetty, who got his BA from Harvard in three years and his PhD three years later and now ping-pongs between teaching at Stanford and Harvard, is extraordinary. (Chetty was a professor of mine in my PhD program at Harvard.)
A short while ago, Chetty and a team of researchers—including Nathaniel Hendren, Emmanuel Saez, and Patrick Kline—were given by the Internal Revenue Service de-identified and anonymized data on the complete universe of American taxpayers. Most important, by linking the tax records of children and their parents, Chetty and his team knew where people spent every year of their childhood—and how much they ended up earning as adults. If a kid spent the first five years of her life in Los Angeles and then the rest of her childhood in Denver, Chetty and his team knew that. They knew that not for a small sample of people; they knew it for the entire population of Americans. It was an extraordinary dataset in the hands of an extraordinary mind.
What can you do with this data on the entire universe of taxpayers to uncover neighborhood effects? Now, the naïve thing you could do is just bluntly compare the adult incomes of people who grew up in different places. But this would run into the problem already discussed: correlation does not imply causation.
This is where Chetty’s cleverness—or, in the judgment of the MacArthur Foundation, genius—came into play. The team’s trick was to focus on a particular, very interesting subset of Americans: siblings who moved as kids. Because the dataset was so large—remember, they were looking at every American taxpayer—they had a substantial number of such people to study.
How do siblings who moved as kids help establish the causal effects of neighborhoods? Let’s think through how this might work.
Take a hypothetical family of two children, Sarah and Emily Johnson,…
