Failing a programming interview at Toptal

A few weeks ago the guys at Toptal reached me and offered the opportunity to apply to its workforce. I was a little skeptical at first due to the fact that they contacted via email and not the typical LinkedIn message. Before going further I asked them how I was chosen and got my email address, because I didn’t remember signing up or something. They told me about a tool they use, on which I’ll talk in another post.

In case you didn’t know, Toptal is an employment portal that connects companies and freelancers. It’s similar to other on the Internet, like Workana, UpWork and such. However, it has the peculiarity of accepting only the top 3% of freelancers, which is a great distinguishing factor. For a company which cannot afford, or doesn’t have the time for, testing waters with freelancers this means good business. It not only reduces the time for screening the candidates, but also guarantees a minimum of quality among them.

From the freelancer perspective, it’s really attractive working with world-class companies in game-changing projects that are really well-paid. However, it requires having great skills to be a part of this gang. They have a three-step screening process broken down as follows: initial interview via Skype where your credentials, experience and English proficiency is tested, technical exam consisting of a programming test with problems very similar to those found in programming contests, and finally a technical interview with a lead developer or technical leader so they can evaluate train of thought and things that a programming test cannot say about us.

After the first interview via Skype, we receive an email containing a link to the the programming test, which can be taken in the next 48 hours. It’s  a set of three problems to be solved in 90 minutes. What I really like is the fact that the problem statements change depending on the selected language (yes, there are also several languages to choose from). The window is split in half vertically, having the statement on the left and the code editor on the right. The algorithm is evaluated in terms of time and memory complexity.

No matter if you’re applying for being a game developer, web developer, or any other specific role, the programming interview tests our algorithm analysis and design. It means always going back to our roots and practicing those techniques. I didn’t pass the programming test and it was a great opportunity to remember that even when the game or prototype works, it is important not to forget that efficiency is king.

I’m not ashamed of sharing this with you, it’s a reminder that I need to practice and study more. It’s a reminder that no matter how much I know about Unity, Phaser or Game Maker features, tips and tricks, it is always important [re]learning algorithm analysis and design.

What to do if we hit the wall like this? Luckily enough, there’s material online to which we can refer to and use in order to get ready for round two:

In a next post I’ll talk more about how I’ve increased programming skills, and what you can do in order to avoid making my mistakes. The most important thing is not getting discouraged. Just practice, take challenges, and always learn to make it better the next time.

I’d love to hear from you. Have you failed a programming test or passed with flying colors? Your opinion is important and could help other readers.

EDIT: Thanks to a friend’s feedback, I’ll give you a little bit more about the test in another post.

PS: They told me I could try again in a couple of months, so I’m gonna study to do so and let you know about my improvements.

It’s fine to celebrate success but it is more important to heed the lessons of failure.

-Bill Gates

By | 2017-01-26T20:24:49+00:00 June 2nd, 2015|Articles, General|41 Comments

About the Author:

41 Comments

  1. Juan Campa June 6, 2015 at 11:06 - Reply

    “Cracking the code interview”, you should read that book

    • pctroll June 7, 2015 at 07:14 - Reply

      Juan!! Wazup, man? Yeah, I have a pdf version of the book and I hope to start reading it next august after finishing other projects I’m working on right now.

      It means a lot having you here. Big hug, my friend.

  2. top-pal June 8, 2015 at 16:00 - Reply

    cracked the process (last round is a simple app development + interview around your code), just to find out I’m not interested in this kind of work were you’re yet again degenerated to a piece of some-thing that fills out a profile and buzz words to sell himself so that the portal can make money of you … but still the interview part was fun although they reflect nothing about my capabilities in real-world work with real-world projects/clients 🙂 !

    • pctroll June 9, 2015 at 06:56 - Reply

      Thank you for sharing your experience!

      I agree that the programming test doesn’t necessarily reflect our competence in complex real-world scenarios, but it’s a glimpse to our skills and knowledge in algorithms, and it’s definitely a way to test ourselves some way or another.

      Hope you liked the blog and can keep you around as reader. Kind regards 🙂

  3. Klevis Miho October 26, 2015 at 14:11 - Reply

    I did the test today and didn’t like it and did’t pass it. I applied as a front end web developer and thought that the test was too geeky for me.

    • dan February 10, 2017 at 08:05 - Reply

      Yeah and what does solving algorithms have to do with being a front-end developer. How would any front-end dev pass that test?

      • pctroll February 27, 2017 at 13:32 - Reply

        Sometimes you just know how to act accordingly. I mean, when doing some Single-Page Applications you may receive a great JSON and use it to give real-time responses to the user. It’s our job to make the most out of that data. The more tools we have under our belt, the better.

  4. Berneck November 6, 2015 at 22:34 - Reply

    Hi! Did you pass the test in your second chance? I just fail today and I’m curious about. I think that I’ll apply again in the next month

    • pctroll November 18, 2015 at 20:14 - Reply

      Hi,

      I haven’t tried again because I found a stable job to finish the year with. However, I’ll try again first quarter 2016.

      Another place to train is HackerRank. Good luck! Let me know how it goes =)

      • Marcelo June 19, 2016 at 14:26 - Reply

        Did you do it already?

        • pctroll June 21, 2016 at 14:53 - Reply

          No, I haven’t applied again.

          For better of worse, I found a stable gamedev freelancing job last august and I’m now on my way to a stable job at a studio in Germany.

          However, every single experience has helped along the way and I’ve been practicing my algorithms once in a while with problems like the ones from HackerRank, SPOJ and such.

  5. Yuan November 17, 2015 at 03:14 - Reply

    Thank for sharing this one.

    I completely agree with you opinion.

    You saved much time of others to avoid failed way.

    • pctroll November 18, 2015 at 20:15 - Reply

      Thank you, Yuan. That’s why I try to share what I learn.

  6. Nadia Kerr (@keepsmilyn) November 17, 2015 at 22:59 - Reply

    If the sample test is anything to go by then I already feel I’m going to fail. 🙁 I couldn’t get my head around the sample test as it wasn’t like any real-world programming stuff I have dealt with in the past.

    The Skype call, blog post and other preliminary tests were straight forward, but the programming test doesn’t really show my true knowledge and skills in the industry.

    I agree that its a test of skills and knowledge, but I wonder whether its really finding the right candidates based on the algorithms. I agree with Klevis, perhaps to geeky for me too.

    • pctroll November 18, 2015 at 20:18 - Reply

      Hello Nadia, thanks for stopping by.

      The test could be too geeky for you, but it’s worth practicing. Don’t let it take the best of you. I’ll try to improve the post or create a new one. In the meantime, try visiting HackerRank and its section about algorithms, it’ll be worth your time and effort. It’s similar to codility, but I think the content is better organized.

  7. Sidharth J Dev November 24, 2015 at 04:41 - Reply

    HI.. I too just failed the test. I am happy about the opportunity I had and will try harder to learn. How soon can I apply for a retry? Is there a minimum time gap? Is there a retry attempt limit?
    thanks

  8. Sebastian November 27, 2015 at 01:27 - Reply

    Hi there, nice post, very enlightening, and thanks for the links to coursera. I also wanted to say that as long as I was reading I was expecting to find more details about the tests. What algorithm did they ask you to complete? I heard (and read) toptal takes tests like the ones on college, science computer related. That was the case? Sorry to tell this but unless you spend some years at college, it’s a little difficult to learn a few stuff without someone explaining you, the axioms related in the proper complexity order with the proper exercises to fix that knowledge. Anyway we can also try to learn it from google, but it’s a little bit more difficult. Please keep updating your post, I really want to know if you could enter toptal and tell us what feels like to work on those projects.
    Thanks

  9. Siddharth December 2, 2015 at 13:27 - Reply

    Just gave the test and got a miserable score. Though I knew how to solve the problems I attempted my inexperience with the edge cases and the time constraints let me down. I’m planning on polishing these skills up to even if its not a very practical skill set to have, seems like a nice challenge to take up and attempt again whenever they allow me

  10. Just another Developer December 5, 2015 at 07:45 - Reply

    I took this test today and I must say it was really tough. I managed to get only 13% and I don’t think I stand a chance of getting hired at Toptal. Yes,I agree that practicing more would have improved my scores.

  11. KK February 16, 2016 at 23:59 - Reply

    Thanks Jorge. I m giving algorithm test today. I will share my experience if i fail or pass.

  12. Friendly March 2, 2016 at 19:38 - Reply

    Thanks for sharing it! You are really a good competitor 🙂 Luck

  13. Mufaddal Nagpurwala March 24, 2016 at 23:57 - Reply

    Heyyy great post! I just finished a call with toptal and they will be sending me the algorithmic test. Thanks for the post.

  14. Laszlo Papp March 27, 2016 at 01:42 - Reply

    I wonder what the passing rate is at the codility stage? Does anyone know that? Also, I remember to have achieved some codility test in the past ifor a company in London in which case they offered 120 minutes for the three tasks. Not that I needed to use all the time, but 90 minutes can be tough, especially if one tries to avoid googling.

  15. SG May 13, 2016 at 10:38 - Reply

    Hi Jorge..thanks for the great post & links for study material.My query is same as Sidharth’s – What’s the minimum time gap between attempts?It would be really helpful if you could answer the query. Thanks!

  16. ketaki patil May 20, 2016 at 01:29 - Reply

    great post…..informative
    I have to give programming test today

  17. Kenji May 31, 2016 at 10:32 - Reply

    Hey all, just took the test and failed with a 30/300, yippee! My contact said that their passing score is at least 120/300 and that I could retake at a later time. Its tough and while it does test only your specific algorithm skills, I feel it weeds out many competent coders who have great day to day practices. Will prolly try again after more HackerRankfoo.

  18. Bans July 21, 2016 at 16:33 - Reply

    I took it first and scored 0%. Tried after a couple of months of studying and writing code in C++. When I came back I was prepared with answers for the previous questions. I wanted to wow the googlers at TopTal. Holy Shit! They changed the questions! Only if I knew where they lived!

    Anyways I tried and scored 122/300. I don’t know if I passed with this.Still waiting for their communications.

    Actually I only took two questions and got 100% for correctness and 0% performance in one of them…so it lowered my average score badly.

    I should be angry that getting to TopTal is harder than getting a real job elsewhere…even with 4 years experience actively programming and pushing to Github.com everynight!

    • Wang September 29, 2016 at 12:19 - Reply

      Hey guys, I have the solution for 3 problems. I got 92% for all of them.

    • Basanth Verma January 28, 2017 at 07:32 - Reply

      Hey Bans, I tired it for the 2nd time too and Coincidentally got the 122/300 !!! I took the test last night and they haven’t gotten back to me yet.. Did u get the job there? Also how long did it take for them to reply to u?

      • pctroll February 27, 2017 at 13:27 - Reply

        They took around 3 days to reply. As I said, I didn’t get the job and I haven’t had the opportunity to try again because thankfully I’ve found clients on game development, which is my primary focus right now.

  19. marcelo oblan August 5, 2016 at 02:24 - Reply

    The test for me is kind of hard as i try it in demo mode and i fail. I’ll take d chance anyway…

  20. Princewill Nforchu August 5, 2016 at 04:56 - Reply

    hello, I took the algorithms test last week and successfully went through. now looking forward to the next face of the test. I will strong agree that the test does not test our real abilities in the real world. However, I believe the test as design to give a little glimpse on our approach to different problems.

    I say so because, of the three questions, I only answer the first with a 100% for both performance and correctness. I started the second but did not finish and i did not even start the third. I had 0% for the last two questions, thus and overall score of 33%.

    However, though the system does not test our true abilities it can significantly help improve our skills. I say so because, before I took that test, I used to be very fund of nesting loops. But now every time i find myself nesting loops, i try to rethink the problem and for like 80% of the time, there is a better way out

  21. Better than Toptal August 25, 2016 at 10:06 - Reply

    Toptal’s test does not appear to be a good indicator of competence. I have 20 years experience and I feel that the test does not take into account the specifics of a platform. You could be an expert on a particular platform, but their test appears to assume that all platforms have the same performance and that the same solution works for all. Even though a solution may work perfectly on one platform, the test may fail you. Maybe if you were an expert in the coding environment or engine they use for the test, you could ace it.

  22. Moises Bassan October 6, 2016 at 12:28 - Reply

    Those Codility tests are f**ked up. I realize it’s hard to find an objective measure for coders, but I have 10+ years experience coding complex financial apps (such as trading engines or banking applications) that work flawlessly in the real world and withstand months time of multiple hackers slamming them all day – and I was only able to get a 3% on my first try.

    Codility filters for people that are only able to solve puzzles and math problems. Those are useless skills for programmers. I’ve seen plenty of “math wizz”es and all sorts of these kinds be abysmal programmers in the real world. While creative people that are good at writing, visual reasoning and design can be excellent programmers.

    I do not agree with this measure one bit – I’ve been excellent at solving real-world problems for over a decade and I know my worth.

    Codility and TopTal can go fuck themselves.

  23. anujparikh1112 November 6, 2016 at 15:34 - Reply

    Just took a test. Attempted 2 out 3 tasks.
    Scored 100% in first task and 66% success and 0% performance with overall 50% in 2nd task.
    As I didn’t attempt third question, my overall score was 50%.
    After 20 mins of my test, I got a mail from Toptal to schedule technical interview. Does that mean, I pass the test?
    Task2 and Task3 seems to very difficult IMO.

  24. shilpa ck November 21, 2016 at 05:53 - Reply

    iam gonna attend toptal test.do u have any suggestions for me regarding preparing test and all other stuffs.iam really happy to hear from you…..

  25. Criss December 20, 2016 at 01:38 - Reply

    I failed the thing too last night. I gotta admit it did piss me off to a certain extent, even though I went with the Toptal process just out of pure curiosity to see what happens and how companies recruit people now days (been a freelancer for over 10 years now, completely out of touch with the “job” world).

    I can only imagine the people that were actually interested in working with Toptal and got failed by Codility are seriously aggravated and depressed by this. If you’re one of those people, don’t be sad dude, it’s NOT you, it’s Codility. Don’t loose confidence in yourself because you fail these automated coding tests, they have absolutely no relevance to how good of a programmer you are.

    Giving a Codility test to a programmer is like asking a painter to draw you a timed on-scale technical drawing of an engine block, then saying they suck as a painter cause they couldn’t do it.

    Codility-like tests are just a tool for companies to help them identify human-robots. The kind of human robots that can sit in a cubicle 10 hours a day for them to throw hundreds of little tasks at them and have pretty lines of codes returned as fast as possible. Don’t feel bad because you failed such test, feel good about it, because it proves you’re a real person and not a robot.

    Here’s an interview I found on another somewhat related thread about how employers and HR people fail.
    https://channel9.msdn.com/Events/ALM-Summit/ALM-Summit-3/Technical-Interviewing-You-re-Doing-it-Wrong

    Enjoy! And, don’t waste your time doing Codility lessons thinking that will make you a better programmer. Those tests are pretty much the same as all the “intelligence” tests you find all over the Internet, the more tests you do the more you memorize their solutions and the better your results are, but that doesn’t make you any smarter.

  26. sutharphp January 24, 2017 at 04:33 - Reply

    On the three exercises, you scored a total of 145/300. The normal threshold for passing is 150/300, but please let me know if any reasonable issue came up during the test that would be important to note.

    but there was passing score only 120 out of 300 and When we spoke on first interview. This was the high score and now it’s changed to 150 ? Can you please let me know how this happened suddenly ?

    • pctroll January 24, 2017 at 06:09 - Reply

      I’m sorry I cannot give you an answer for that :(, and I’m really happy you got that score.

    • ssamayoa February 3, 2017 at 16:07 - Reply

      They may raised the bar because there are too many candidates?

      • pctroll February 27, 2017 at 13:30 - Reply

        That’s an option. It’s like when a pop star hires a great drummer to play some “basic” beats. Yeah, the beats are basic, but the artist and the producer are making sure to reduce risks to zero; specially if the main artist wants to do some improvised show. That drummer (or any other musician) is trained to react and keep the show going. It’s more or less the same here.

Leave a Reply