Scaling Karat

For the past few months I’ve taken on a contracting role at Karat, a company that specializes in first-round engineering interviews for other software companies. It’s really a lovely opportunity for me, providing a bit of income, the flexibility I need to pursue my startups, and a great community.

I’m at their annual conference hosted in Seattle and can’t help but find myself comparing their current stage of growth with Medallia.

Similarities:

  • Karat is at ~30 FTE and ~60 contractors, Medallia was around 80 FTE
  • both full of humble but passionate people with exceptional energy and a belief that they’re making a difference
  • respect and appreciation of culture, diversity
  • receptive to feedback
  • will soon be facing scaling issues in training/consistency/etc

Differences:

  • immediate sense of accomplishment when completing an interview, as opposed to rolling out features and waiting for feedback/adoption
  • Karat offers flexible hours; you’re welcome to take a month off to do whatever and come back
  • interview finishes and if you made a mistake, there’s no lasting pain/regret unlike bug fixes required in normal software development lifecycles
  • contractor with flat rate, no expectation of career advancement, stock options cycles, vesting/IPO concerns

Potential for Failure

I love the company, I love the mission and I want them to succeed which is why I fear they may fail. So where are the potential weaknesses?

Focus on Quality

Karat prides itself on high quality interviews which provide consistently fair and positive candidate experiences. The cost of maintaining that bar as you scale becomes increasing difficult:

  • layers of management are required once you reach ~100s of interviewers
  • if you split into teams/groups, easy to become silo’d
  • if you don’t, then people feel like they are just a number
  • training at scale becomes very difficult if you rely on heroes/heroines

As much as I love culture though, at the end of the day recruiting is a always a numbers game. I think it’s fair to say that “most” software companies are using the standardized one phone screen + on-site visit. From their perspective, Karat replaces the initial phone screen. From a culture perspective, the initial phone screen should be unbiased and provide a fantastic candidate experience. From a business perspective though, the initial phone screen needs to be “good enough of a filter” for final round and shouldn’t resources be allocated towards ensuring a great experience for final round candidates (who are more valuable) than initial round?

If it’s easy for competition to hire contractors to do screens, and the only differentation is quality; well we’ve seen that story at Medallia. Haven’t done the full calculations, but quality has a cost to it, and the majority of software hires are not made at big companies with a reputation to protect.

The other side of quality is the means by which you ensure high quality. Metrics and tracking are great, but the key is to balance the need to apply process to achieve standardization vs. allowing people the freedom to exercise creativity with guidelines/principles.

Incremental vs. 10x Opportunities

Where does Karat go next? The current focus seems to be UI enhancements to improve an existing process allowing Karat to scale to maybe x2-x5 number of interviewers (and hence clients). Small wins are indeed necessary and Karat’s access to flexible developer resources through the interviewer contractors makes it quite easy to achieve these, but …

Where’s the next 10x opportunity here?

Sure, they’ve got a great data set of annotated interview data from questions to performance to biases. How can that be monetized or leveraged?

There’s a current project around creating interviews for specialized engineering roles, e.g. frontend developers or mobile or machine learning data scientists. Future expansions could easily go into e.g. management, QA, analyst positions or other non-technical roles too. Is that outside the vision though? to focus on non-technical interview types?

There’s the potential for a Karat certification which could candidates could use as a badge to bypass first round screens.

AI/ML applications to automate assessments is an obvious “hard” problem to solve, but perhaps that’s a solution in search of an actual problem.

LinkedIn partnership/acquisition or an independent social network amongst candidates would be another possible jump.

Integrations with ATS’s can provide a sharper market curve, but you still have to deal with the supply problem.

Achieving Market Dominance

Let’s suppose that Karat succeeds in its vision and “all software engineering companies use Karat”. The pool of candidates is always mostly filled with poor-performing candidates. What happens when they’ve failed a dozen Karat interviews? Does the question pool expand to hundreds of types? Is there a blacklist or a waiting period before re-attempting a Karat interview?

If all companies are using the same assessment tool for first round screens, do they compete with each other on sourcing and selling alone?

Servicing Component

Coming out of Medallia, I fear anything that isn’t self-service. Karat has specifically positioned itself as a service not a tool and this means a professional/client services arm.

Upsell opportunities usually center around tailoring/customization for a specific customer. In this case though, that involves tailoring the interview experience itself; which itself can introduce bias and/or less consistent experiences.

Conclusion

Karat definitely has some unique advantages over a traditional software company both in terms of business opportunity and the contractor vs FTE staffing. They’re full of people that are passionate about what they do. Very curious how leadership intends to scale and how they will empower the company to face those challenges though.

Name:

Comment: