Who Does Hard Things, and Why

People do all kinds of work, some of it is easy and some of it is absurd. What is really interesting is when people do really hard things, ones that have a very low probability of success. These are things like starting startups or doing things that have never been done before. I have noticed a pattern in when these things are done and why it is hard to understand logically.

While observing my friends, colleagues or some of the people who have done or are doing hard things, a common pattern comes out. They are either exceptionally good or exceptionally bad, nothing in between. I was curious to understand what makes people do hard things. Why can’t hard things come out of logical, rational people?

The above graph is of one of the famous cognitive biases, the Dunning-Kruger effect. To put it simply, the difference between perceived ability and actual ability is larger for people with lower ability. This means people who tend to show low ability are the ones overestimating their ability the most. Another interesting observation from the same bias is that people with higher ability tend to assess their actual ability lower than what it actually is.

Read more here: https://pubmed.ncbi.nlm.nih.gov/10626367/

The cognitive bias on its own does not explain why people who are either really bad or really good end up doing the hardest things. It only shows how the gap between perceived and actual ability changes as actual ability increases.

To understand why these two groups take on hard problems, we need to extend the idea. Instead of only asking what people think their ability is, we should ask what they think their ability is to increase that ability. In other words, how hard they believe it will be to reach the level needed for success.

Here is a simple example. Imagine you want to build a startup that requires solving a difficult problem, and you have some level of ability today. Consider people in the bottom quartile of the above graph. The numbers here are only examples to illustrate the idea (% and approximation is added to make a point).

Assume people believe that having an ability of 90 percent or above gives them a good chance of success in the startup.

Bottom cohort

Actual ability: 10%
Perceived ability: 50%
Perceived difference: 40%

Middle cohort

Actual ability: 45%
Perceived ability: 65%
Perceived difference: 25%

Top cohort

Actual ability: 90%
Perceived ability: 85%
Perceived difference: 5%

It is not hard to see why people with high ability tend to do hard things. They have a clear sense of what needs to be done and they do it. But shouldn’t people in the middle cohort be the next most likely to do hard things? After all perceived difference in ability is lesser than the bottom cohort?

People usually start hard work not when they think they can already do it, but when they believe it will be easy to acquire the ability once they begin. You attempt a very hard problem if you think learning how to solve it will be easy.

This is the part that explains the pattern. The bottom cohort, with very low actual ability, believes they need to increase their ability by 40 percent. The middle cohort believes they need to increase their ability by 25 percent. However, the ability to see the actual work required to make this improvement depends on their current ability at the task.

Bottom cohort: 0.1 × 40 = 4
Middle cohort: 0.45 × 25 = 11.25
Top Cohort: 0.9 × 5 = 4.5

People in the bottom cohort, because of their low ability, think the amount of work required to acquire the ability to do this job is much easier than it actually is. In fact, the lower your ability, the more likely you are to attempt the hard thing, because the perceived ability does not change much as actual ability keeps going down.

A more general model for the same would be to treat perceived effort as the product of two things: how capable you are today and how much skill you still need to acquire. If actual ability is a∈[0,1], natural model is

$$E(a)=Ka(1−a)$$

where E(a) is the perceived effort and K is just a scale factor. This captures the idea that effort feels small when you are too unskilled to understand the difficulty, and small again when you are highly skilled and close to the threshold. This function is zero at very low and very high ability and reaches its maximum somewhere in the middle: beginners and experts feel the climb is small, while people in the middle feel the mountain is highest.