Effective Hiring for Technical Talent

Whether we like it or not, part of being a technical leader is mastering the time-consuming, and sometimes painful recruiting and hiring process. 

To be successful as a technical leader you need an exceptional technical team but the good news as a leader, is that you own the responsibility to build and grow that team. 

While some companies have developed good processes for hiring technical teams, startup and small/mid-size companies often lack well-defined hiring strategies or use processes that aren’t effective for assessing technical hires. 

We have all made hiring mistakes that proved costly and painful so I view it as business critical for technical leaders to establish and implement an effective hiring process for building high performance technical teams. 

Earlier in my career as a senior software engineer, I was hired by the Peter Norton Computing Group, recently acquired by Symantec. 

They had the largest concentration of technical talent that I had seen up to that point in my career so I found it exciting to work with a group of such experienced technical talent and strong motivation, something of a “dream team”. 

While one of my childhood nicknames in elementary school was “the brain,” I was far from the smartest person in the room at Peter Norton. It was sufficient enough for my ego to believe that, in a small meeting, I might be a close second.

As a result of that experience I learned a few things about the technical teams I wanted to be a part of:

• I preferred working with other senior software engineers over entry-level engineers.
• I enjoyed working with other highly self-motivated people.
• I loved working with very intelligent and talented people and teams.
• I felt great satisfaction with the success in the marketplace that exceptional teams produce.

Hiring for Talent

It also led me to a philosophy about hiring for technical teams. 

Fundamentally, when hiring software engineers and all other things being equal, I value talent (i.e., innate ability) over skill (i.e., learned ability). 

Borrowing from the Agile Manifesto, while there is value in skills, I value talent more.

As a technical leader, I’ve discovered and adopted 2 tools that I have found useful in hiring for technical talent:

1. “Firing Squad” Technical Interview
2. Aptitude Assessment Test

“Firing Squad” Technical Interview

As part of their hiring process the Peter Norton Group used a panel interview called the “firing squad”.

This was not your typical subjective panel interview, which I’ve never really liked. 

The firing squad technical interview is different in that it is purely technical in content and more objective than the typical panel interview.

As I grew into becoming a technical leader and eventually CTO, I incorporated the firing squad technical interview into my own hiring process and found it to be effective in helping select intelligent and talented individuals. 

Let me explain the process and objectives as it relates to interviewing technical candidates. 

Suggested Process

• Hiring manager and selected technical staff (usually 3-4 developers) meet together with the candidate.
• Participating technical staff should have previously reviewed the candidate’s resume and have a good idea of the type of person we’re looking to hire.
• Participating technical staff should have a broad representation of technical skill sets, so that they can effectively drill down on all relevant technical areas.
• The candidate is informed that we’re going to ask questions of a technical nature to get a better feel for his/her technical strengths and weaknesses. They will be told it’s ok to respond with “I don’t know,” venture a guess (e.g., “I’m not sure, but I would think…”), or think through the question and problem out loud.
• Participating technical staff will ask technical questions, starting at a high level and then drill down on the specific topic until the topic is exhausted. Here is an example of a drill down into one specific topic:

“Your Resume mentions that you have some experience with SQL. Would you tell us more about your work in this area?”

“Can you give us an example of an SQL query?”

“Ok, so it sounds like you’re familiar with SQL queries. Can you explain what a Join is?”

“How about a Left Outer Join?”

“Ok, so if a row from the first/left table doesn’t have a match, does it show up in the result?”

• Asking general computer science questions can be useful in assessing the candidate’s technical capability (e.g., “what is inheritance?”), however, it is generally not beneficial to do research to come up with questions and answers. The collective knowledge of the interviewing team should be sufficient to assess the relevant technical capability of the candidate.
• Move from topic to topic until running out of time or exhausting relevant topics.
• After the candidate has left, each team member shares their thoughts and opinions of the candidate (i.e., what they liked/disliked).

Target Objectives

• How much of the candidate’s resume is fluff vs. real (e.g., did they just work on a project that used JavaScript or did they write JavaScript code, and how proficient are they?). 
• Insight into how the candidate thinks (i.e., we’re not just hiring for skills, we’re hiring for talent).
• Insight into the personality of the candidate and how they interact with others.
• How the candidate responds under pressure.
• Opportunity for the candidate to get to know members of our team and give them a feel for our personalities and how we interact with each other. The candidate is evaluating fit from their perspective as well.

Special Notes

The firing squad technical interview is an unfair match-up by design. 

No candidate comes out with a perfect score. 

A key objective is to go beyond assessing the candidate’s learned skills and see how they think (i.e., innate ability). 

The idea is NOT to beat up the candidate. Once it is clear the candidate is unable to effectively drill down further on a certain topic, move on to another topic rather than continue to ask questions they are unable to address. 

In practice the firing squad takes more time with a more knowledgeable and more talented candidate.

IMPORTANT: Remember that while we have a need to assess the technical capability of the candidate, we want to do this in a professional and friendly manner. If we decide to hire the candidate, we don’t want them to decline because they think we are jerks.

Aptitude Assessment Test

Later in my career, I was introduced to an additional tool – an aptitude assessment test. 

There are numerous aptitude assessment tests out there, but the one I’ve used is oriented toward software engineers. 

The test describes a simple hypothetical computer language, and then asks multiple choice questions related to this hypothetical language. 

The 20 questions on the timed test gradually introduce additional syntax, semantics, operators, etc. for this hypothetical language, and subsequent questions build upon prior information provided about the language. 

While prior experience with computer languages would likely be advantageous, it is not required as the test itself provides everything needed to answer the questions. Which makes sense, since it is an aptitude test, not a skills or knowledge test.

Before using the aptitude assessment test with candidates, I wanted a better feel for what the scores might mean. I took the test myself and invited other members of our technical team to take it if they wanted to. The test scores from internal team members were very closely aligned with my own personal experience of their talent. 

The person I considered to be the most talented member of the technical team achieved a perfect score. We also had a contractor working for us who was a retired military helicopter pilot, and more recently a retired database administrator (DBA) from a U.S. National Laboratory. His background and work experience were non-traditional for a software engineer, but I knew him to be very intelligent, talented, and an excellent problem solver. 

He scored higher than many of the more traditional software engineers on our technical team, which I found consistent with his observed job performance. 

Interestingly, he later told me that the military had pilot candidates take a similar aptitude assessment test to screen their helicopter pilots. 

One aberration to the internal test results was a talented member of the team who scored lower than I expected, but I believe his higher anxiety personality likely made him a poor test taker. Whatever the reason, it’s important to note that a single metric can’t definitively determine future performance.

Summary

The firing squad technical interview and aptitude assessment test are just two ways to help gauge technical talent. 

The extremely intelligent and talented team member who got a perfect score on the aptitude assessment test also went through our firing squad technical interview about 8 years previously. He had just recently graduated from a college of computer science and had some intern experience. We were looking for a Java developer, but his experience was primarily with C#. 

On paper, he was an unremarkable candidate – a wild card candidate that I tend to bring in for interviews. Despite his minimal real-world software engineering experience, he worked through the technical questions and problems, often with educated guesses. 

We observed his thought process as he figured out answers on-the-fly and correlated his C# knowledge to Java-specific questions. He knocked that interview out of the park and impressed us. 

Despite some reservations from team members that he lacked Java experience, we hired for talent that day and found a “diamond in the rough.” He was one of the best hires I ever made.

Interviewing and hiring for technical teams is a challenge, made harder by the fact that resumes often lie. 

The firing squad technical interview and aptitude assessment test are not silver bullets, nor are they the end-all-be-all in assessing candidates. You also need to consider other traditional factors when hiring for our technical teams, such as cultural fit. 

Hiring for exceptional talent helps build exceptional teams.

The firing squad technical interview and aptitude assessment test are useful tools to have in the toolbox for building high performing technical teams.

CTO Academy provides leadership courses and coaching for technology leaders around the world.