At Visual BI, every technical talent that gets hired, irrespective of the number of years of experience, must undergo computer-based, hands-on technical assessments.

Why? The answer is simple. Interviewing competence does not equate to on-the-job competence.

In the last few years, I interviewed more than a thousand candidates (a conservative estimate), hired 200+ associates, and observed a considerable number of our employees on the job over an extended period of time. What I learned in the process is this: that the technically proficient candidates need not be always good at interviewing, and that it is possible for someone with average proficiency to perform in a technical interview reasonably well.

To ensure that we only hire the best talent, we started administering practical, hands-on assessments. While computer-based assessments are very common in the industry for generic programming skills such as Python or C, we extrapolated it to other areas such as SAP BW, SAP HANA, SAP BO Data Services, SAP Web Intelligence, and more — areas where technical assessments are not common in the industry due to the complexity in setting up the environment, data & test scenarios — with great success.

Our technical assessments have evolved over time, and today we are at a stage where we rarely come across a candidate in an interview who is lacking in technical proficiency. (When candidates fail to clear the technical assessment, they just don’t get to interview with us)

What does a typical assessment look like?

Our typical assessment lasts 2–4 hours, requiring a candidate to resolve one or a series of technical challenges in the computer. These challenges replicate practical scenarios that require the candidate to not just have the knowledge but also application, troubleshooting, data blending & manipulation skills. The best part is that ALL our assessments are open book. In other words, you could google, look up StackOverflow, visit SAP forums, read blogs, or any extensive online help. (You can do everything but phone-a-friend).

We have also designed the assessments in such a way that any advance preparation can’t be of much help (in case the question leaks). We also recycle our scenarios & data from time to time.

The Top 5 Benefits Of Assessments

1. You know that you have a Do’er, not just a Know’er: During the interview, you can be assured that the person being assessed can not only talk about it, but also actually do it.
2. Assess the extent of technical proficiency: This helps us in two ways: (a) we can compare candidates not only with each other, but also comfortably benchmark against our existing employees, and (b) compensation decisions are fairer, as we don’t have to pay a lot of attention to the number of years of experience, which many companies end up doing in the absence of this measure.
3. It balances out a good deal of recruitment biases: It also improves the hiring manager’s confidence during interviews, which could be adversely impacted by behavior such as poor eye contact, nervousness, or lack of communication skills.
4. It can teach us more about the candidate, behaviorally: We get to observe the candidate in action — whether they ask questions before jumping in, how persistent they are when faced with an obstacle, how they deal with subsequent interview questions when their shortcomings are highlighted, and many more. We’ll discuss more on this in a later blog.
5. Best of all, it makes the hiring process super-efficient: If the candidate fails to clear the technical assessment, they just don’t get to interview with us. This saves a lot of time for our hiring managers, recruiters, and other experts needed for the interview panel. In our experience, roughly about 10–20% of experienced candidates clear our hands-on assessments. Imagine the time it saves us. We also get to identify to reasonable extent candidates who have exaggerated or misrepresented their experience.

In Conclusion

More than anything, hands-on technical assessments help us ensure that the talent density in the organization is not diluted. After all, the best service you can do to your employees is to give them the pleasure of a highly proficient peer group.