I used to just share whatever I learned, no matter how small it was. However, at one point, I forgot about the practice due to my work commitments. So, this time I would like to talk about what it looks like on the other side of the interview session, which is the Interviewer.

Let’s put the disclaimer out first. I haven’t done it for “years”, ok, maybe just around two years. So, I’m not what you would call a seasoned/expert interviewer, but I believe there will always be someone out there who can learn something from this post.

I have also only been an interviewer in my latest company and I never did it before in my previous companies. To this date, I have been involved in 80+ interviews. However, when I mentioned “interviews”, that doesn’t mean all of them are live interviews where I talked face to face with the candidates. It could be just me screening or reviewing their assignments. So, this post is limited to the experience I have from a single company.

Alright, let’s get right into it. In CoinGecko, we use lever.co as our tool to manage the candidates. I still remember we used Notion last time with their Kanban interface and this tool is so much better (and I believe it’s not cheap either).

Generally, we will use similar processes among the candidates. But, we change it from time to time depending on the needs. We are quite flexible. The common practice is to split them into these stages:

  1. Personal Statement
  2. Take-home Assignment
  3. Virtual Interview
    1. With the Engineers
    2. With the Founder

There are more steps involved that I didn’t mention as they don’t involve us much as Engineers. For a start, our colleagues in HR will help with the “Discovery Call” first before we Engineers start to get involved in the first stage. Our HR and Engineering Manager also screen most of them first before a candidate gets to us for the first stage. After the third stage, there are also reference and background checks by HR.

Before we start further, let’s talk about the HR roles in the process.

HR Roles#

I’m not going to dive too much into this as it’s not my place and I also don’t know the full details other than the updates posted by our HR. However, I want to make it clear that our HR is involved from the first step until the end. They play a major role in terms of being the middleman between us and the candidates. If there’s an update to the candidate from our side, it will be an email coming from HR. It’s going to be a full-time job for us if we don’t have them. Thank you Daniel for the hard work from your side 🙏

Regarding the Discovery Call: It’s actually a phone call our HR makes to the candidate to get more details on these topics:

  • Your work experience
  • Your interest and understanding of the crypto industry
  • Your hobbies/interests
  • Goals
  • Expectations

You might be wondering, why not just read the CV or the resume? I think having more information about the candidate will help a lot. There are so many ways to write those docs and some information that we want might not be there. Our HR will put a summary and a rating on the candidate’s page on Lever. It does put more work on our HR, but it’s very helpful to us.

The engineering manager may or may not give input at this point. If all is good, we as the Engineers will be selected as the panels to review Stage 1, which is the Personal Statement’s feedback. Normally, the team that asked for the new hire will be part of the panels. It’s not unusual for multiple teams to be involved as there could be more than one team asking for new hires. However, the panels will be limited to three persons which includes the Engineering Manager.

Stage 1 - Personal Statement#

In this stage, we will review all the information gathered by HR including the resume/CV, blog, Github/Gitlab, Twitter, LinkedIn and most importantly, the answers they put when applying for the job. Those answers come from the custom form that we put on our job’s application page. The questions might be different, depending on the job and level.

For this stage, we will be splitting our feedback into three categories:

  • Strength
  • Needs improvement
  • Red flags

Basically, it’s about our first impression regarding this candidate. We are not supposed to take too much time on this stage, but it can take me more than one hour if there is more information about the candidate. I even browse some of their projects on Github just to see how they code. It helps me to put a picture of this candidate so that I can have better understanding which in turn helps me to build better questions tailored to them in the live interview stage.

This is also the place where we will highlight their strong points and potential problems based on the resources that we have. As mentioned before, the “Discovery Call” notes help us a lot in making sure we understand the candidate better. So, our answers will be based on multiple sources, not just the questions that we posted in the application page.

Once we have worked on the first impressions, we also need to fill in two more points which are:

  • Whether this candidate raises the bar
  • Feedback for Stage 2

What does it mean to raise the bar? This is quite subjective, but for me, it’s about whether the candidate is someone who could offer something new/better compared to the other candidates. It’s just a Yes or No option.

For Stage 2 feedback, we will put a summary of all the points. Sometimes, I’ll put a note on how this candidate compares to the others. It’s a free text field, so we can put anything we want there. We will also leave some notes on what we want to know in detail which we may or may not be asking during the live interview. It’s possible that we will change the panels after that, so the notes that we put may help others too. This is also where we will suggest which assignment is suitable for the candidate.

Then, we will put a Rating (we can’t skip this part):

  1. Strong No Hire
  2. No Hire
  3. Hire
  4. Strong Hire

If you noticed, you can’t choose something like 50-50. You have to pick a hire or no hire. The rating system is quite simple, but it forces us to really think it through. Kudos to the Engineering Manager who thought of this, preventing me from being on the fence LOL. The same rating system will be used throughout. Oh, it’s possible for a candidate to be rejected at this stage.

Once the candidate passes Stage 1, HR will contact the candidate with the assignment that needs to be delivered within a certain time period. Our EM will help to answer if there’s any question regarding the assignment. At this point, we’ll just wait for the assignment to be delivered.

Stage 2 - Take-home Assignment#

This is the stage where we will review the assignments. Yes, there can be two of them, but normally, the other one is just a small task that doesn’t require the candidate to code. Once HR receives the email from the candidate about the complete status of the assignment, they will block two hours of our calendar based on our free time to review it. HR does more work here as they need to ensure we don’t review two assignments at the same time. They also know that we still have to code and do other tasks 😅 It’s not uncommon to have a combination of Stage 1, 2 and 3 happening in the same week. Again, I’m so thankful to our HR. Yes, I’m just going to keep on thanking them in this post.

So, what do we do at this stage? This is where we will look into the code submitted which is normally done through Github. During my early years, I downloaded them and made sure they were working locally. But, these days, it’s very rare for me to do so unless I have a good reason. Besides, one of our requirements is to have the application running on the Internet, either through free or paid hosting. So, I don’t exactly have to ensure it is actually working locally.

I believe this is where I’ll spend most of my time compared to the other stages. We also allowed candidates to use different programming languages before. It’s going to take more time for me to review whenever I get a stack that I’m not familiar with. Recently, we asked candidates to do it in Ruby on Rails. They can use React as the front-end if they want to. Other than reviewing the code, we will also prepare the questions that will be asked during the live interview. For some people, they can make up the questions on the fly, but for me, I need to prepare the basic questions first and then I can improvise from there.

You might be wondering what kind of questions we would ask? It depends on the assignment and the candidate. We will use our knowledge from previous stages to construct good questions to test their knowledge. For example, we won’t be asking basic Ruby questions even though the assignment was built using Ruby on Rails if the candidate is not a Ruby developer. Most of the questions are about the assignment itself, why did you choose to go with this approach and not the others, etc. Most of our questions are open-ended, which means there might be more than one correct answer. The main thing that I want to see is if they can justify why they chose that kind of answer.

Similar to the previous stage, we will put a rating on 10 criteria for the assignment submission. Six of them are compulsory and the other four are extra credits. No, I’m not going to reveal what the points are 🤣. Suffice to say, the good ones won’t have a problem. Then, we’ll also give a rating as the overall, same system as the previous stage. That rating will be the final score for us to decide whether we should move forward with the candidate or not. Again, we also have a free text field where we will put some notes that will help us the reviewers in the next stage.

Sometimes, we ask the candidate to rework their submission. It really depends on the candidate and the submission. We can also just reject the candidate at this stage if we want to, same as other stages.

Then, our HR will take over and communicate with the candidate in order to ensure all parties, engineers and the candidate will be available for two hours on a certain day for the live interview. This requires our HR to look into our calendars (the panels) and the candidate too.

Stage 3 - Virtual Interview with the Engineers#

Virtual interview is where everyone will be in the same video call for the interview. Normally, there will be four people involved: 1) EM 2) The team lead 3) The engineer from the team that is hiring 4) The candidate. It is not always the same though, for example, it’s quite normal to see two team leads and the EM sit together in the call. It happens when there is more than one team hiring. It is also possible for a candidate to be offered a position in the Web team even though they applied for a job in the API team. We will decide based on the outcome of the interviews.

For me, this is the most challenging stage among the processes even though I’m not the candidate. Before experiencing it on my own, I assumed only the candidate would be challenged; apparently the interviewer is too.

Normally, we will set the session to two hours which splits into:

  • 5 to 10 minutes for introduction
  • 50 minutes to discuss the previous assignment
  • 50 minutes about the candidate’s working experience
  • 10 minutes for the candidate to ask us any questions

As usual, we’ll change it depending on the candidate. If we don’t think we got enough information about the coding skills, we’ll just extend the discussion on the assignment a little bit more by taking time from other sessions. However, we will limit the total time to two hours. It’s very rare for us to go beyond that.

Let’s talk about the preparation. In order to have a fruitful conversation, we will prepare the questions before the call. For the panels, one of them will create a doc to summarize all the feedback from earlier stages. Then, we will create a new section to talk about the assignment and the work experience. The doc will be shared among us so that everyone can edit the file and add their questions. Normally, it’ll be done on the same day or the day before.

For the assignment questions, we will split the questions based on the functionality. We try to make the flow as smooth as possible so that we don’t have to jump back and forth between code unless it’s unavoidable. So, if there are different questions about some of the code, we’ll just keep on adding them in the doc. Imagine a tree from the big to small branches. Sometimes, we will combine the questions together when needed. Everything will be done asynchronously as we don’t need everyone to be online to build the questions. Basically, everyone doesn’t have to be in the same room to work on the doc.

What would we ask? It depends on the candidate. Other than the time taken to review the questions, this is the stage where it takes most of my time. The questions will be tailored to the submitted assignment and the candidate. We won’t be asking basic questions on Ruby if they haven’t done Ruby before. However, we will be asking generic programming questions to verify their understanding. For example, I’d look into the code patterns, testing, etc. and see whether the candidate understands what they were doing in the first place.

Then, we have the work experience questions. Depending on the level of the candidates, we would most likely ask about their past projects. Basically the details from the planning until the post-deployment. This is the most “wildcard” section as we don’t have the full details other than what the candidates put in their resume/CV. So, for the preparations, we’ll just outline the very basic questions and we will adapt from there.

Now, let’s jump to what is happening during the interview. Normally, the EM will start by introducing the panels and explain what is going to happen in the next two hours. Then, we’ll ask the candidates to make an introduction. Yes, we have the basic details, but we are going to ask them to do it anyway. It will help the candidate to be at ease for a little bit before we go into the more challenging sessions.

Then, we will start the technical questions with the assignments. The panels will ask based on the document that was prepared earlier and make the adjustments as they go. Normally, there will be one lead and others will add more if needed. The questions can get quite challenging. We will relate the questions to the challenges that we have in the current work environment. The key is to challenge the candidate out of their comfort zone and see how they would adapt and react. Most of our questions are based on real problems. We do make some imaginary problems, but they are not too far-fetched from reality (our existing work).

Due to the limited time that we have, we would try to control the flow as much as possible. It’s pretty normal for the candidates to randomly go out of context when they couldn’t answer the question properly. We will just cut them off even though it feels a little bit rude. One thing that I realized when I started to take over this session is that I started to talk faster than normal. We also can’t give too much time to the candidates to answer the questions. However, we will take a pause here and there when the candidates are asked to give them a little bit more time to answer the questions. It’s kind of expected since we have limited time to ask questions. The main advantage is that we can cover more questions, but, at the same time, it would put more stress on the candidate.

These would be my top tips if I have to give tips based on what I have gone through before:

  • Review the assignment again and prepare. You don’t want to forget about what you did in your assignment and also the WHY.
  • Ask for time if needed in between questions.
  • Use online drawing board and screen sharing to make your point clearer.
  • Don’t be afraid to ask for clarification. It will make you look like someone who can communicate and not just work without questions when given tasks.
  • Minimize the interruptions. Try your best to ensure your Internet, speaker, mic and everything else is working properly before the call. Of course, don’t be late.
  • Have some water ready. You are going to need it, a lot. It is also a good trick to get some extra time in between questions. It will force your interviewers to take a pause too.
  • Look at the camera. Personally, I think it is kind of rude if someone doesn’t look at the camera when someone is talking to you. Try talking to someone on the street while looking somewhere else.
  • Relate the question with your past experience. Do not just answer the question. Think about the time where the answer was something that you implemented before. It will show you have the experience and not just taking some points from a random blog.
  • Be aware of the timing. Do not go out of context too much. Ask if it is ok if you want to elaborate further since you think it will help your interview session.

Asking about work experience is probably the most challenging part of the session for me. Yes, asking non-technical questions is more challenging for me. As mentioned, I can’t prepare much about it. I need to understand the candidate’s project based on what their resumes or discovery calls say. What helps is to have some basic questions as your guidance. Having objectives like what you are looking for from a candidate is good as well. Work backwards from there. Additionally, writing the summaries of what the candidate said while adapting my questions to their answer is quite challenging.

I think this part of the interview is more relaxing for the candidate as they only need to talk about what they have done. It is also important that a candidate can reflect on their past projects so that they are able to mention what they could do better if they had to do it again. A good engineer always reflects so that they can learn and grow. Actually, that applies to everyone.

These are my top tips when it comes to work experience:

  • Pick the most impactful project that you have worked on instead of talking about all of them. Remember, there is a time limit.
  • Depending on the position you are applying for, take the most suitable to show your skills: leadership, technical skills, etc. It doesn’t make sense to talk about managerial skills when applying for individual contributor positions.
  • Be concise in your answers. We are developers as well, don’t put too many buzzwords. Again, time is limited for us.
  • Ensure you can back up your decisions from your past projects. Why did you choose that method? Why not this method?
  • Oh, be respectful to your interviewer, please. I have interviewed a candidate who laughed back at us thinking that we asked stupid questions even though it was not.

I think that’s pretty much it. Towards the end of the interview, we will give the chance for the candidates to ask questions about our work and it can be anything. Best to prepare the questions even before the interview so that you would not waste time thinking about what to ask.

Summary#

Phew, I think this post is too long already. I guess for the summary, I would like to say that it is not just the candidates who have to spend time to do the preparation and even the interviewer can be nervous too. We take our interview sessions very seriously. I hope that the next person who will interview me will do the same. Good luck everyone.