The following describes some techniques that I use when interviewing candidates for Software Engineering positions in offshore locations. I have brought these techniques together into five stages:
- Logic and Problem Solving Ability
- Computing Knowledge
- Specific Skills
- Spoken and Written English Ability
- Communication Skills and Personality
1. Logic and Problem Solving Ability
When I first started out interviewing offshore software engineering candidates in Malaysia, I wasted a lot of time looking at their CVs and using those as the basis for the first stages of interviews. This resulted in the candidates doing a lot of talking about projects they (claimed) they had done and skills they (thought) they had before I even started measuring their technical ability. Some CVs looked very impressive indeed, their authors claiming almost endless lists of skills acquired, many to "advanced" standards. Now, back in the UK, for the most part when talking about highly skilled jobs there is an unspoken rule when it comes to CVs, candidates only listing skills that are really worth listing and certainly being prepared to back up any claims of "advanced" levels of proficiency in any of those claimed skills. It is no surprise that upon receiving such impressive CVs in Malaysia I assumed the candidates were very high quality indeed and decided that the first hour of the interview should be about them talking about their experience (to help them relax into the interview) and me doing a bit of a sell on the role and company. Only after that would we dive into the technical questions, which looked like they would a breeze for them. Unfortunately, the aforementioned CV "rule" that applies in the UK does not apply in Malaysia, nor does it at any other offshore location that I have interviewed candidates from thus far. I could therefore quite easily waste the first hour of an interview talking to a candidate about their CV, and perhaps spending some time talking about the role and the company, before even thinking about getting their hands dirty with some technical questions. When the technical phase began, many candidates were turned down because it quickly became apparent that the person I had talked to for the previous hour or so was not the person who was on the piece of paper (the CV) in front of me; they had exaggerated wildly and in some cases blatantly lied on their CV.
When only recruiting for one or two positions, wasting an hour here and there talking to a candidate who has deliberately fabricated their CV is not a big deal. Indeed, many candidates I talked to were truthful and I subsequently hired them. However, when recruiting on a larger scale offshore, the numbers go against you and such an approach can be hugely inefficient. Given that I was recruiting on a larger scale, I had to find a way to determine as quickly as possible if a candidate I was interviewing was worth talking to further. I therefore put aside their CVs and piles of certificates and jumped straight into a bunch of logic and problem solving activities (which involve writing code) on the whiteboard; I was quietly amazed with the results.
The questions were short and simple, often programmatic, such as:
- Using the language of your choice (or even pseudocode for junior candidates), write a function to reverse a string.
- Using the language of your choice (or even pseudocode for junior candidates), write a function that prints all the prime numbers from 1 to n.
At the very start of the interview, before asking these questions, I would I often ask a candidate to rate themselves, 1-10 (1 being beginner, 10 being advanced), in each of the programming languages they listed on their CV, quite a few responding confidently that they were 8,9, 10's in languages such as C and Java. I would record these ratings on the whiteboard, in view of the candidate, for later reference. I then asked the candidate to complete questions similar to (1) and (2) on the whiteboard in front of me. The key with the questions is that I emphasise to the candidates that they are to choose which language they want to use when writing the solution to the problem, thus removing any potential for them to claim they struggled with the question due to a particular language being imposed on them. Furthermore, I am happy for them to use pseudocode / English if they are unable to code the solution (though that in itself will tell me something about the ability of the candidate and will set alarm bells off if they are applying for a more senior position). Based on the candidate's solution to problems such as these, it doesn't take long to establish if they are worth interviewing further for the role in question. We are talking minutes. For example, I still vividly remember an already very senior candidate C developer who had worked in the USA as an embedded engineer and was now back in Malaysia working on C code related to aviation systems. He applied for one of my senior software engineer jobs in Malaysia. On paper, he looked fantastic - good degree, strong background and the right skills. To my surprise, he struggled to reverse a string in his language of choice, C, for which he had rated himself as a 9 when asked at the start of the interview (and which I wrote on the board). I don't mean he got one or two statements wrong due to not remembering syntax, I mean he completely could not reverse a string as per question (1) above. After far too much guidance from me, eventually we got there. Thinking he was nervous, I then gave him the prime numbers question (2) as above. After some initial explanation from me as to what a prime number was (he did know it in the end, perhaps he forgot) he had no idea where to go and just wrote drivel on the board, continually wiping it out, puzzling his forehead and writing yet more drivel. He looked embarrassed. I stopped it there and asked him what he now thought his ranking was in C. I could see the look of torment on his face, like he still wanted to stick with his original answer. "5 or 6, perhaps?", he reluctantly admitted. Based on his claimed level of experience and the level job he was applying for in Malaysia, I had no further questions. Although I did not set a timer off, I would be surprised if the whole thing lasted 15 minutes.
I now never start an interview without asking similar questions to the above in the opening 15-30 minutes, no matter what the level of software engineer I am interviewing for. Candidates do not proceed to other stages without first getting past this stage. The level of role will merely determine how much leeway I give for incorrect answers. For example, for a very junior position, what I will look for is not necessarily the right answer, but how the candidate thinks about the solution. At the very least, they should be able to describe to me how their algorithm could solve the problem. In my view, even for such a junior candidate, if somebody has been through university, done a Computer Science degree, and cannot even explain how to reverse a string or does not know what a prime number is, they probably shouldn't work for me. Likewise, if somebody has been working for 10 years and cannot reverse a string in the language of their choice, they definitely shouldn't be working for me. Importantly, very importantly, no matter what the level of the candidate is, I ensure that they never guess the solution to my problems and try to bluff their way to an answer, talking about it as if it's the right answer to impress me. Anybody that has worked for me will know that I hate guessing in software engineering. A candidate who is willing to guess and try to bluff their way through an interview is likely to do the same when they are working on a task for me or someone else. For example, they may, not understanding a problem thoroughly enough and hence guessing, go off and write reams of code that they are equally unsure of. I always tell my staff that if they are unsure of the work they are doing, to stop what they are doing and come and see the team leader or me to discuss; never guess. So, I always jump onto any evidence of guessing during this stage and find out why the candidate is doing it.
One other point worth mentioning about the questioning techniques I describe above is that that are easy to conduct with candidates that are remote, as long as they have a computer and Internet connection. For example, I have interviewed candidates in completely different countries by setting up a shared whiteboard session (many Internet communications tools offer such a facility) or a shared Google Doc and asking them to type the solution to the problem while we talk over the phone. Arguably, given that we are not in the same room they could cheat by looking up solutions on the Internet, but since I do not allow much time for the questions and I am also on the phone at the time, this is unlikely. Furthermore, I take steps to search for any solutions to the problems I ask online and ensure they did not merely type out one of those. That said, even if I am suspicious that they copied a certain solution, it is trivial for me to build upon their solution and ask them to modify it to solve a related problem. Use of this technique has allowed me to screen many remote candidates before inviting them to travel to my place of work for an interview.
To summarise, my advice when interviewing offshore candidates is to get a quick handle on their Logic and Problem Solving ability before deciding whether or not to move on to talk about their experience and the role. Spend up to 30 minutes doing this and give them a fair chance to answer a range of questions, not just a single question. Make sure the questions involve actually writing code, but ensure the questions allow flexibility in the languages used unless the role you are recruiting for is a senior role that uses primarily mandates use of a specific language. By all means ask further Logic and Problem Solving questions in later stages, but the key of this stage is to provide a quick "Go" or "No Go" on a given candidate.
2. Computing Knowledge
Although I know of a number examples of colleagues that neither studied Computer Science at degree level nor had any knowledge of computers who went on to become exceptional software engineers during their career, when I interview offshore candidates I do look for general Computing Knowledge; so many aspects of the work, at least in my experience, that software engineers do every day depends upon a having a solid foundation in the principles of computing. Perhaps more obviously to me, I believe it to be of great advantage if a candidate has a genuine interest in computers and understands how they are work. More often than not, such candidates will have interacted with computers regularly as they were growing up, perhaps taking them apart, making modifications, playing games, configuring networks and suchlike. I always keep a lookout for these candidates and they certainly exist in offshore locations such as Malaysia.
A simple way to determine how much a candidate knows about computers is ask them to draw a diagram of a computer on a whiteboard, asking them to label the various components of the system. Then ask them to describe the function of these components. It's a simple question and how well they perform at this question will give me an idea of how much they know about computing. If they do well at the question, perhaps I'll throw in some more challenging questions about the hardware or maybe we'll move onto software such as talking about how a compiler works, or perhaps we'll talk about fundamental algorithms. The level of questions I ask depends on the seniority of the role being applied for, but I nearly always begin with a question about a computer. This exercise, since it is mainly on the whiteboard, also gives me a further opportunity, following the Logic and Problem Solving stage, to assess the candidate's communication skills.
When I was at Nottingham University in the UK reading for my degree in Computer Science, I was surrounded by people like me, people who loved computers and who "messed around" with them on a regular basis, just for the fun of it. In my view, people like this need to be looked out for, so I nearly always ask offshore candidates why they are pursuing a career in software engineering and try to find out how interested they are in computers.
My advice, therefore, when looking for offshore candidates is to look for those that have a genuine interest in computers, who possess a good understanding of their inner workings and who can answer typical computer science type questions with ease. Try to establish how good they are in this area before you move on to specific skills, as that stage will most likely require significantly more time and involve people other than yourself if you are the hiring manager.
3. Specific Skills
By this stage, following the previous two stages, which just involved me and the candidate, I will now have a pretty good "gut feel" on the candidate's suitability for the role. After a little more talk about their experience and profile (including talk about software development processes etc), as well as some more talk from me about the role and company, now is the time to get other people involved and start assessing specific skills. I normally involve at least two of my software engineering subordinates in the skills assessment stage, as well as at least one other people manager. If the candiate will have any dealings with the core team (most likely), I will also include engineers and managers from the core team offices e.g. in the UK or US. All are free to ask any questions they like and their views hold considerable weight in my decision-making process. After all, software development is very much a team sport and it is important to me that my team buys into the idea of a given candidate joining their team; they are the ones that will be working with them day-to-day. I therefore allow to several hours of talks with these various stakeholders, either on the same day or on alternative days if time does not permit. Some of these talks, if with overseas colleagues, take place via telephone, Skype, or suchlike.
I then usually finish off the skills assessment stage by giving them one or more online tests on relevant topics. I use a reputable supplier of such tests. Although these tests do help me form a view of a given candidate's skills, I normally give them far less weight than the opinions of my subordinates and other colleagues. In most cases, their ability to establish if a candidate can do the job far outweighs the results of these online tests, but it's all about forming a total picture of a candidate.
To summarise this stage, my advice about specific skills is to get as many technical and managerial people involved in the interview process as you can, including those from core teams if applicable. Meet up /discuss after all interviews are finished and come to a conclusion as a team, each giving a "thumbs up" or "thumbs down". Also use online testing tools to further assess specific skills, but use their results with caution.
4. Spoken and Written English Ability
For pretty much any native English-speaking business that is to interact with an offshore software development team that, most likely, speaks English as a second language, proficiency in spoken and written English is paramount. A given offshore software engineer may be a good programmer, but if they cannot communicate with colleagues in the main country where the business operates it will cause a new set of problems focused around communication. I remember back to around 2003 when one of my friends in the UK, who at the time was dealing with a computer equipment supplier in Taiwan, wrote them a technical question about their firmware code. Although I do not remember the precise question he asked, which was in an email, it was very open-ended, something to the effect of "Could you please describe the function of this firmware module in more detail". The answer he received, much to the amusement of all of the colleagues that were within his proximity at the time, was "Yes.". In Malaysia, where I currently run my business, English is spoken and written rather well as a second language. However, not all candidates that I have interviewed have had a strong command of the English language, largely down to the area in which they grew up and the schools and colleges that they attended. Conference calls with such candidates, or email exchanges, or document write-ups, would be very difficult indeed. I always, therefore, assess spoken and written English skills during an interview. The spoken part is trivial as the candidate, based on the previous three stages, will have talked to a number of my colleagues in addition to myself, so we can form an opinion on their working knowledge of English. For the written part, I did not used to spend much time investigating this if they spoke English well. However, one of my subordinates at the time once suggested to me that we have candidates write a short document on a non-technical subject that pretty much any candidate would be able to write about. For example, the topic to write about in English could be "Describe the person you most admire in the world and why". This is the kind of topic anybody should be able to write about, no matter what their career experience and technical background is. Some people may write about a great leader or scientist that they admire. Some may write about one of their parents or relatives. That is the beauty of such an open-ended question. I therefore now include this type of exercise wherever possible when interviewing an offshore candidate to assess their written English skills.
In summary, my advice for this stage is to realise the importance that spoken and written English ability play in offshore development scenarios This may sound obvious but it is something that can be overlooked in all the drama of assessing specific programming skills etc. In particular, written English ability can be easily overlooked if the candidate sounds like they can speak reasonably good English. Ultimately, failing to properly assess the written, as well as spoken,English skills of offshore candidates may place unnecessary burdens on the core team, who will end up losing time and getting frustrated in the process. To assess English skills, first of all ensure that all interviewers involved in the process take note the candidate's spoken English ability, particularly those conducting in-depth assessments of skills (for example, how well does the candidate articulate about a certain technical topic?). To assess written English skills, one trivial technique is to give the candidates a simple written English exercise that is open-ended and non-technical. Any native English speaker will be able to read their answer and quickly determine how good their written English skills are.
5. Communication Skills and Personality
In software development, given that it can be thought of in the context of a team sport, communication skills and personality traits naturally come into play. Assessing communication skills and personality traits is not something I leave until the end. In fact, it is something that is done in almost all of the stages prior to this. By this stage, I certainly have a good handle on a candidate's communication skills; this stage merely completes the process and considers Communication and Personality separately from the other stages. One of the things I like do in this stage, which I feel is quite important, is invite the candidate out to lunch with my team. This provides a relaxed atmosphere in which to talk about both work and non-work related topics, and is an opportunity for the team to further gain confidence in and acceptance of the candidate. It certainly gives a good picture of how a potential candidate will fit into the team. Likewise, it allows the candidate to chat with many members of the team and ask questions about life in the company, the type of work being done, and suchlike, so it is a beneficial process for them too.
On our return to the office after lunch, I have a final session with the candidate to ask them more communication and personality related questions. I am not a fan of psychometric assessments or suchlike, so I keep it verbal and rather informal, but the types of questions I ask are all about ascertaining if the candidate could fit into the offshore team as well as work with the core team. In addition to further discussing the role, I would perhaps ask fairly open questions like "What would you do if somebody modified your code and broke an area of functionality that you had implemented?" Or, "How would you react if the team leader insisted that you used their approach instead of yours?" Or, "What's your view on coding standards?" The answers to these types of questions can indicate personality traits that may be disruptive in a team environment and may need further investigation before making an offer. In some cases, for more senior positions, I give them a piece of code and ask them to review it, observing how they go about the process and what kind of issues they find. This is not really about the skill in conducting a review (we've already assessed skills), but more about how they communicate with me. I also like to ask questions about testing. A good software engineer knows how to test code that they write, and explaining this is an exercise in communication.
One final exercise I give them to demonstrate their communication skills is another whiteboard exercise. For example, I may ask the candidate to map out their career plans onto the whiteboard. This not only allows me to see how driven the candidate is with respect to their own career, but also shows me how good they are at presenting information to an audience. Another similar question I could do on the whiteboard is to ask them to describe a software development process that they claim to know about.
With the notes I make in this stage, together with the notes from all of the stakeholder that interviewed the candidate, I am now able to conclude if the candidate has the necessary Communication Skills and Personality traits that would make them a likely fit for the role they are applying for.
After a final discussion with the team and those that interviewed the candidate, I am now ready to decide whether or not an offer is to be made.
To conclude, these are the stages I go through when interviewing offshore software engineers. I should also point out that I have adapted the above stages quite easily to cater for interviews with other types of offshore candidates, for example Software Test Engineers.
By Jared Checkley
No comments:
Post a Comment