Flashover

In his podcast Revisionist History, Malcolm Gladwell does two episodes inspired by William Henderson's paper The Lsat, Law School Exams and Meritocracy: The Surprising and Undertheorized Role of Test-Taking Speed. Those two episodes opened my eyes to how we constantly judge people on false merit and how our society often has a bias toward people with good test-taking speed. It's a great listen; you should check it out.

Henderson and Gladwell focus on the world of law. But in this post I'm going to discuss their idea from the perspective of programming.

Who's the better chess player?

Here is Gladwell's argument in a nutshell: We constantly measure people's reasoning ability with time pressured tests. We use those tests to decide for example who is accepted to high-ranking universities. However, science says that test-taking speed does not correlate with reasoning ability. Time pressured tests measure the speed of our reasoning ability, not the actual quality of it.

Gladwell divides people into two categories based on test-taking speed: tortoises and hares. Tortoises need their time to figure out solutions and answers while hares are fast thinkers. And just like in the fable of the tortoise and the hare, tortoises triumph over hares when the game is not about speed. For example, Henderson found out that law students who don't do so great in time constrained tests, are still able to get solid A's from take-home exams and papers.

You can see this same pattern in the world of chess. Chess games have a time limit. This is so that the games wouldn't go on forever and end up in ties. However, the time limits are not always the same: Classical chess games start from 90 minutes per side for the first 40 moves. In blitz chess, each player gets 10 minutes or less for all their moves.

Chess player Fabiano Caruana is ranked 2nd in the world. His peer Hikaru Nakamura is eleven ranks below him at 13th. However, these rankings are based on classical chess games. In blitz, Caruana wouldn't stand a chance against Nakamura. Nakamura dominates in the world of speed chess. Caruana's strength is his calculation skills while Nakamura's strength is his pattern recognition skills. When you decrease the play time, pattern recognition skills give you a greater advantage as a player.

So who is actually the better chess player? Nakamura or Caruana? We have the official ranking to answer this question. But if the organization in charge of the ranking would decide to decrease play time dramatically, the ranking would look something completely different. Great chess players are great in different ways. The official ranking is just one side of the story.

Why is someone a great programmer?

Think about your colleagues' and your own personal strengths and weaknesses. Think about Nakamura and Caruana. Think about the fact that test-taking speed doesn't correlate with reasoning ability. Are you able to agree with the following statements:

• Great developers have great reasoning ability
• However, your learning speed doesn't correlate with your reasoning ability
• ...your delivery speed doesn't correlate with your reasoning ability
• ...your ability to succeed in whiteboard interviews doesn't correlate with your reasoning ability

Even if you agree with those statements, you probably also make conclusions like these (consciously or unconsciously):

• Because person A learned C++ in a month, she's a better programmer than person B who spent 6 months learning the language
• Because person A delivers features faster than person B, she's the better programmer in our team
• Because person A nailed our whiteboard interview, she's the best candidate and therefore the best programmer we can hire

It's likely that inside your team and your organization, there is a bias toward tortoises or hares. It can be that because of your interview process, you end up hiring more hares or tortoises. Because of the way delivered work gets highlighted and successes celebrated inside your organization, you might be promoting either more hares or tortoises.

It's also possible that there isn't anything wrong with your biases. If you need hares, by all means hire hares. If you need tortoises, hire tortoises.

But what if you haven't thought about whether the work you have set out to do is meant for hares or tortoises. What is the version of the fable people tell themselves inside your organization? In the minds of people, is the winner the hare or the tortoise? And if you look at the facts objectively, does their version of the story match with reality?

My interview story

A little more than a year ago, I was interviewing for a job. I was asked to take a couple of online tests to measure my reasoning abilities. The tests had such strict time constraints that most people weren't able to finish all the questions in time. You had to mark your answers fast and not second-guess anything.

I took the tests and I scored fairly well (not amazing but also not bad). Based on the results, I thought that the tests suggested that I'm smart. But in fact, all I had managed to prove anyone is that I have pretty good test-taking speed. I, and the interviewers, falsely thought that the tests said anything truly meaningful about my reasoning ability.

By the way, just so you don't think I'm trying to do a humblebrag with this story, let me drill my point into you by quoting Henderson's paper: "it is widely acknowledged that test-taking speed and reasoning ability are separate abilities with little or no correlation to each other." Just because you're fast, doesn't mean you're smart, and vice versa.

I actually don't know if I would have been hired. I dropped out of the interview process. I figured that the job required much more contemplation and deep reflection that I had the patience for. It was work meant for a tortoise and I was a hare.

Sadly, the company had forgotten the lesson of the fable. They were betting on the hares of the world.

Engineering as marketing, or free tools as marketing, is a marketing channel where you utilize your engineering resources to build free and useful stuff (calculators, microsites, plugins, widgets, etc.) for your customers. Lately I've been cultivating an interest for this idea. Here's why:

• Engineering as marketing seems like a lot of fun. As a developer, you get to build something with a hackathon-mentality.
• Everyone keeps saying that engineering as marketing is an underutilized channel.
• As a marketing channel, it's very non-creepy and anti-hucksterish. Engineering as marketing is not about microtargeting. Engineering as marketing is about providing meaningful value to your customers before they give you a single cent.

Here are some reasons why free tools can create high growth for a company:

• Engineering as marketing is similar to content marketing in the sense that you are building assets. Free tools can grow virally. Free tools can stay relevant for your users even as the years go by.
• Even if your customers have the budget to buy software, free is still a lucrative option. Expensing anything (or asking permission for buying software) is a pain in the butt.
• You get to build trust. You are able to show your customers that not only do you understand their problems but also are able to deliver concrete value to them.
• You have a higher chance of standing out. It's not another blog or e-book.

I think the last point is one of the main reasons why why people say engineering as marketing is an underutilized channel. But why are you actually able to stand out? Why isn't every company showering their customers with free tools? There's at least three major challenges with engineering as marketing:

1. You need developer resources

You need developers to build free tools and developers are high in demand. It's hard to recruit great programmers and it's hard to take your current programmers away from their ongoing projects.

This is why you need to timebox aggressively. Find one developer and give her a week (not weeks or months) to deliver a working version of your free tool. If you can't fit the work into a week, you need to think smaller. You're not building a product but a tool with one main function.

2. Software projects are hard to manage

The reason why software projects are hard to manage is that they tend to be rather complex ventures with both technical and management problems.

However, you can simplify things. Focus on one core feature. Assemble a team of two: one customer expert and one developer. Empower your team with autonomy. Let them manage themselves however they want to with a budget of one week.

3. It may not work out

There is no guarantee of visitors once you release your free tool. Maybe you end up building the wrong thing. Maybe people won't care enough to tell their friends about your tool. Maybe you get the traffic but you don't end up getting actual customers.

Your free tool should solve a problem for your customer that leads to them wanting your actual products and services, or it should indicate that someone is a high quality lead. For this to happen, you need to build something that's actually relevant for your customers. Do your research. Ask your sales and support teams what are the problems your customers face during the customer journey.

Final notes

This is not a free resource, but Gabriel Weinberg and Justin Mares cover engineering as marketing with concrete examples in their book Traction. Here's the Amazon link for it.

This post is meant to be read left to right, line by line, top to bottom. In fact, all my posts are meant to be read that way. However, most of the time that is not what's happening.

Most of the time a visitor browses through my text and word-spots: They read the first sentence. After that they spot five interesting words, and read another one or two sentences.

If their random sample of words and sentences is interesting enough, and they have the time to focus on learning, they will read my post the way it's meant to be read (left to right, line by line).

This is how we read things online. We don't read the way the author wrote the article to be read. We are constantly bombarded with information and the only way we can navigate through this digital jungle is to be frugal with our attention.

Not everyone who asks for our focus gets it. We want a sample before we make a decision to invest our time on a web site or an article.

Assume that you get five words to communicate your message. Assume that a visitor only sees the top left and perhaps the bottom right corner of your site. Squint your eyes, let them navigate your site freely, and scan the content. What do you see?

Additional notes

Often people scan content using an F-shaped pattern. Here is Jakob Nielsen's original study about it: www.nngroup.com/articles/f-shaped-pattern-reading-web-content-discovered/. This is a longer article that revisits the F-shaped pattern of reading: www.nngroup.com/articles/f-shaped-pattern-reading-web-content/.

A year ago, I went from no blog and zero writing to blogging every week here on Flashover. I'm pleasantly surprised that I managed to post every week without fail. But I'm much more amazed at the effects writing has had on me. I've become more curious and a better thinker. Before, writing was a source of anxiety to me. Now, it brings me joy.

Here are the three things I've gained from writing every week for a year. As a bonus, I've included a list of my most popular posts so far.

I became more curious

Before I started writing my very first post, I had prepared a short list of post ideas for the first few weeks. After I had thrown out the half-baked ideas and turned the other ideas into actual posts, I found myself struggling to figure out what I should write about next.

This forced me to learn how to seek out interesting perspectives and ideas worth writing about. I had to pay more attention to the stories people were telling me or others. I started to think more about the why: Why did I react that way? Why do we keep struggling with this issue? Why do we do it like this and not like that?

I learned how to think

Persuasive writing requires organized thinking. To practice writing is to practice reasoning as well as communication. I've learned that many ideas sound better in my head than on paper. I have had to abandon thoughts that didn't actually go anywhere once I started writing them down. But I have also seen some thoughts take an even more sophisticated form once I opened my laptop and started typing away.

I feel that my one year younger self was much more disorganized with his thoughts than he ever realized.

I started to like writing

School taught me to avoid writing as much as possible. Compared to subjects like math and history, writing was especially strenuous work that didn't pay off in good grades no matter how much effort I put into it. If you can get an A from something else with much less effort, why should you bother with writing?

I was afraid of writing and I was terrified about what others might think of my writing.

But once I got to the rhythm of writing every week, those anxieties began to seem more and more insignificant: Does it really matter if this is not my best post? I'm still going to write another post next week and the week after that.

I finally consider it enjoyable to sit down with a cup of coffee on a weekend morning and start drafting a post.

Final notes

I'm deeply grateful for the people who read this blog and who have shared my posts during this past year. Here are the most read posts (in case you missed any of them):

1. Using TDD as a Mind Hack
2. Innovation Doesn't Sell
3. Doherty’s Threshold Is a Lie
4. Takeaways from Slush 2018
5. Surviving Conferences

Software developers are constantly being asked to estimate work. The work to be estimated can be a small task that requires input from an individual developer or a feature that involves a full team of people. Estimations might happen during a weekly meeting or while drafting a project proposal.

Software developers are also constantly wrong with their estimates. We keep thinking that things are easier than they truly are. We all go over budget and miss deadlines all the time. On Monday, you started working on a feature that you thought was going to be out by the end of the day, and on Friday, you were barely able to submit your pull request before heading home for the weekend.

This repeating pattern of underestimating the difficulty of future tasks is referred to as the planning fallacy and it's a result of multiple factors: We present overconfident estimations because overconfidence is often rewarded in our society starting from early childhood. We omit the time required for coordinating work from our estimates because our brains are not good at understanding how difficult it is to build things together with other people. We procrastinate because we have a tendency to go for the tasks that give us instant gratification.

Famous psychologists Daniel Kahneman and Amos Tversky argue that since humans are inherently bad at estimations we should rely more on data driven planning. The first step towards data driven planning is to start tracking performance. Unfortunately, we can't trust ourselves or others to provide us accurate historical data about past projects. Our brains are not always able to think mathematically or store information in a reliable manner. Therefore, we need to collect our plans and the actual durations and costs to some kind of database.

Researcher Yael Grushka-Cockayne's ideal database of completed projects includes the planned schedule, information about project tasks, and actual durations and costs. She's also suggesting that in the future we will be applying machine learning for our project plans. Here's a link to her 10 minute talk about the planning fallacy and data driven planning.

Even after you have collected a comprehensive database of past projects, your estimations aren't going to be accurate all the time. But this is not a good enough excuse to ignore tracking. Estimates based on historical data are still going to help you miss deadlines and spending limits less often. In turn, this will lead to better consistency, less stress, and more trust.

It's also true that our work as software developers is creative work characterized by many unknowns and novel domains; no two projects are alike. However, you can still identify reoccurring problems within your work. We end up building authentication systems, retrieving data from external APIs, and designing CRUD features over and over again. You can break your projects into separate chunks that you are able to estimate using historical actuals–but only after you have started to collect the performance data required for those estimates.

You are probably able to recall examples inside your company where projects ran over budget. When it comes to your own work, you know that you keep being overly optimistic about available time and required effort. Is this the time for your company to start tracking plans and actuals? How about yourself? Should you start tracking your personal estimates and actual durations?