In many ways, hiring software services is similar to buying a used car. In this article, I’ll show how you can use research on used car market economics to improve your ability to hire high quality developers.
If you’ve ever hired in software, whether a single developer or a vendor for web development consulting, you’re probably well aware of the extreme difference in quality between good and bad developers. Mythical man-month raised the idea of “good” programmers being not 20 or even 100% better than bad ones, but more like 5-10x more productive.
The survival of startups, big initiatives, and many careers depends on being able to figure out the relative quality of a programmer.
That being said, you’ve probably been wrong more than once. Why is that?
It’s because:
- Every programmer (or vendor) claims to be very good (anyone who goes around telling interviewers they’re a poor programmer won’t be getting many job offers)
- It’s not a simple task to decide who is faking (or inflating themselves) and who is not. Even if someone does well on your programming tests, there are many other characteristics that are hard to account for.
Only the programmers (or vendor team) themselves know if they are highly skilled or not, and that information asymmetry leads to expensive mistakes.
A famous economics paper (The Market for Lemons: Quality Uncertainty and the Market Mechanism by George Akerlof in 1970) described the market for used cars as having these exact same conditions and coined the term “lemon market” to describe it. Subsequent research (Job Market Signaling by Michael Spence in 1973) has revealed strategies that many industries use to help solve the problem of information asymmetry.
2. Software Hiring is a Lemon Market
3. Solutions for a Lemon Market
What are Lemon Markets?
Akerlof's paper (which later led to a nobel prize in economics) highlights conditions for a market in which buyers have poor information with which to decide the quality of goods.
Imagine you’re buying a used car. You narrow your choice down to two cars, each supplied by different vendors.
Vendor A - Low Quality
Vendor A’s car has been in 4 accidents, hasn’t received an oil change in years, and has been neglected to the point that it will require major maintenance costs very soon. If all of this information was disclosed, the car would be worth $100.
Vendor B - High Quality
Vendor B’s car has been meticulously maintained. Given it’s great condition, it is worth $300. It recently got a fresh paint job and a great car wash, making it look flawless from the outside.
Vendor A and B are both telling you their car is the highest quality. Who do you trust? Since you know the value of their cars range from $100 to $300, you might decide to meet in the middle and offer $200.
Vendor B is dejected that you’re offering to pay less than the car is worth and declines. Vendor A is delighted, gladly accepts your money, and walks away with $100 more than he deserves.
This, in a nutshell, is the theory of Lemon Markets. Information asymmetry between buyers and sellers leads to:
- Buyers often choosing the lower quality product and therefore disappointed in the result.
- Sellers of high quality products getting priced out of the market, which can lead to a situation where the majority of the available products are low quality.
The conditions for this type of market are:
- Information asymmetry
- An incentive for sellers to portray all products as high quality
- A lack of credible “disclosure technology”
- Average seller quality being low
- A lack of effective quality assurance guarantees (once you make your purchasing decision, you’re stuck with the results)
Does this sound familiar? That’s because these conditions are identical to hiring a developer, or a team of developers.
Software hiring is a lemon market
Let’s review the conditions listed above and apply them to software:
1. Information asymmetry
Hiring managers have no great way of distinguishing the quality from the fakers. To make matters worse, some developers genuinely think they're good, but they don't know what they don't know. Inexperienced developers tend to have an inflated view of their skill.
2. An incentive for sellers to portray all products as high quality
Software developers compete on quality of their skills in addition to speed and cost.
3. A lack of credible “disclosure technology”
Resumes and portfolios can be faked. Programming quizzes don’t tell the full story.
4. Average seller quality being low
Any hiring manager that’s been spammed with outsourcing firms (or worked with enough of them) knows how difficult it is to find a good programmer or team.
5. A lack of effective quality assurance guarantees
Even if it’s possible to realize your mistake and change course, it’s extremely expensive to change your team in the middle of a project.
There is a huge range of programmers for hire, in terms of cost. For example, take Upwork:
Compare that with Accenture, where you can hire a team of programmers for $250/hr. I can promise you that many of the Accenture developers are not 5x more productive than the Upwork programmers.
In between those spectrums, there are many other options: hiring your own developers, hiring a smaller, more specific vendor team, hiring an outsourcing company, etc. Yet for almost all of these, you can’t know the quality of the team.
A recent high profile example of this problem is the Obamacare website debacle several years ago. Their failure to distinguish a quality web development consulting team lead to millions of wasted taxpayer dollars and plenty of embarrassment.
Let’s talk next about ways to solve this problem - screening and signaling.
Solutions for a Lemon Market
Michael Spence examined solutions in a follow up paper called Job Market Signaling.
Sellers must use forms of “signaling” to convey information to prospective buyers. Buyers must use forms of “screening” to learn as much as possible about the seller.
Signaling
Signaling in this context means the seller sends a signal that indirectly suggests their goods or services are high quality.
A few examples of signaling in non-software industries:
- Buying a super bowl ad
- Paying dividends to shareholders
- Branding
- Voluntary information disclosure for used cars
In the software world, developer job applicants send a signal about their intelligence and ability levels by acquiring an impressive degree. Even if the degree doesn’t actually directly improve their relevant skills for the job in question (though it’s hard to prove this definitively), most employers would see value in an economics degree from Stanford for all the indirect attributes it implies about the candidate.
Presume you have two very equal candidates - similar experience, interview scores, and no obvious disqualifiers. If one candidate acquired their experience from being self-taught online and the other has a degree from Carnegie Mellon’s Computer Science program, you’d likely choose the latter, even if you had no reason to believe this degree enhanced the candidate’s skill level. You know that this degree took significantly more sacrifice and effort to acquire, which signals higher quality.
Screening
As a buyer (assuming no obvious signaling is present or that you can’t trust the signals to be valid indicators), how do you separate the good from the bad?
The first level of defense might be basic screening via due diligence on your product or service - calling references, a thorough interview process, test driving a used car, etc.
If you’re clever, you might devise a way to screen applicants to tease out behaviors correlated with high quality.
A 1976 paper by Stiglitz and Rothschild gave an example of how the insurance industry does this. Car insurance companies can’t truly know which of their prospective customers are risky drivers (a low quality investment from their perspective) and which are safe drivers (a high quality investment). The drivers themselves are the only party that knows how responsible and safe they are, which makes car insurance a lemon market.
The insurance company can gain partial information from your accident report or your criminal history, but this still doesn’t provide enough information. To solve this, they provide two options - a high deductible, low premium option, and a low deductible, high premium option. Truly safe drivers will choose the high deductible, knowing that they’re unlikely to need it. Risky drivers will reluctantly choose the high premium to avoid higher costs if they get in an accident.
Another example is offering two types of contracts to salesperson candidates - low base salary and higher commissions, or high base salary and lower commissions. The best salespeople would self-select for the higher commission option.
Applying it to software
Let’s take the lessons from economics and apply them to hiring a software developer or a team of developers.
1. Look for signals
Look for obvious signals of quality - attributes that are difficult for a low quality candidate to acquire.
Besides impressive education credentials, look for experience with open source. A candidate who is active in existing projects, actively creates their own projects, and clearly does this because of their own interest (not as a strategy to get hired) will likely be higher quality than a candidate who doesn’t work on open source. This is because low quality candidates would never be able to pull off working on open source projects (it requires greater skill) and candidates attempting to fake interest and proficiency would never invest this much time and energy into projects they don’t truly enjoy.
Not all signals end up mattering in the real world. Google has completely stopped using GPA or test scores as a screen for recruits after discovering that there was no correlation to success in the company.
2. Basic screening
Beyond interviewing, always spend the extra effort to screen through obvious channels. Ask for references. Ask for proof of shipped projects. Ask to look at sample code and let your best developers examine it for quality.
3. Advanced screening
Many employers check credit scores for job candidates. Credit history is considered a way of screening for responsibility, which happens to also be an attribute correlated with a good employee. This is now illegal in some states, but the idea of finding secondary methods to tease out relevant information remains attractive.
For example, consider the following "screens" you might try during a long in-person interview. To be clear, these are things I've never tried. None of these would be a primary indicator of developer quality, but might be a good sign, all other things being equal.
- Leaving a Brita filter with only one glass of water remaining, but a faucet nearby.
- Seeing if they wash their coffee mug or leave it dirty in the office sink.
- Leaving an empty toilet paper roll in the bathroom, with a new roll nearby. Just kidding :)
Some people will spend the extra 30 seconds to "do the right thing" in these situations, despite no likely recognition for this small act. Others will do the easy thing.
I’d hypothesize that the former group are probably better developers. The same instincts to leave a room more organized than how you found it seem to translate well to development tasks.
What other, more ethical experiments, could you use to tease out behaviors found in good developers?
4. Self selecting
Rather than hiring a developer or a team indefinitely, consider a short initial trial period, such as a 2 month contract. This provides both parties a chance to work together for enough time to evaluate their fit before committing to a long-term relationship.
A high quality team would gladly take this opportunity, confidently knowing this would be a chance for their skills to shine.
Or, even more cunning, offer them the choice between a lower priced, longer term contract, and a higher priced, shorter term contract, with an extension depending on performance. Lower quality teams and developers will opt for the job security, but the higher quality teams would take the latter option.
Wrapping up
Software is a lemon market. The lessons from other lemon market provide useful insights to software managers. Heed these lessons and you’ll have a much higher chance of avoiding a team full of lemons!
development
Previous Post
Next Post
