The Joel on Software Discussion Group (CLOSED)

A place to discuss Joel on Software. Now closed.

This community works best when people use their real names. Please register for a free account.

Other Groups:
Joel on Software
Business of Software
Design of Software (CLOSED)
.NET Questions (CLOSED)
TechInterview.org
CityDesk
FogBugz
Fog Creek Copilot


The Old Forum


Your hosts:
Albert D. Kallal
Li-Fan Chen
Stephen Jones

Whiteboard coding (long post, sorry)

During a recent interview I was asked to remove duplicates from a sorted array (on the whiteboard).  I basically used two pointers and swapped duplicate items in-place, giving an O(n) algorithm with no additional memory/storage usage (outside the couple of pointers). 

This, IMO, was the most optimal solution; however I was rejected from additional interviews.  So I’m kicking myself and wondering what went wrong.  Here’s how it went down:

1) Asked some softball resume questions, I answered.
2) Asked me to write code to remove duplicates from a sorted array.
a. I asked a few clarifying questions and then wrote input array on the whiteboard, asking if this was correct.  He said yes.
b. I talked for a few minutes about creating a new array and copying non-duplicates, but then realized I could simply use two pointers and remove duplicates by copying them to the back of the array.
c. Using the input, I sketched out a pseudo algorithm: if (prev == curr), increment curr to non-prev item and copy curr to prev.  The entire solution was like 10 lines of code and it works (I tested when I got home).
d. I made some adjustments and asked what to do with the items at the end of the array, and we decided to return the new size.
e. I wrote the function on the board and wrote the code, however I did not comment anything.
f. I tested the code using the input of 3,3,4,4,5,5,6,6,7 and determined it worked.
g. The entire time I was turning around and asking him for verification, “is that what you were thinking, etc, etc..”  I did this because I read you’re suppose to involve the interviewer in the process.
h. He stood up and basically said, “ya ya, that works.”
i. This took me like 20-25 minutes to solve (code, talking, everything).

3) Asked me if I had any questions.
4) I received the wonderful “We’ve decided to move with other candidates” email.

So, anyone that has experience interviewing, please tell me how to handle these whiteboard questions, please.  More specifically:

-How much time should I spend standing up there clarifying the question?  I mean I could have spent an hour asking him questions like “does this function need to be generalized to other input, etc..” or "Is this used in a performance critical environment," or "are you saying XYZ and really mean ABC?"

-How much should I involve the interviewer?  Should I turn around every minute and talk to him?  I basically talked the entire time (I'm doing this, doing that.. here's the trade-offs, etc).

-How much should I test my code? 

-And finally, how much time should it have taken me to solve and code this problem?  The entire time I felt a need to hurry and finish.  I didn't want to stand there and look like someone who over analyzed the problem.

Any advice would be appreciated. Oh, no flames please, I'm on the rebound here :->
TheWhiteboardTookMyLunchMoney
Thursday, March 31, 2005
 
 
You did fine, and no matter how much you think about it you will never figure out why you got rejected. It probably has absolutely nothing to do with your technical skills. It could be your brand of mouthwash (or lack thereof), the way you shake hands, eye contact, whatever. Maybe the other two candidates were both summa's from MIT and the only reason you were brought in is in the off chance that the MITers decided they really wanted to pursue that PhD after all instead of joining Cool GenX Startup. Just move on.
Icct Hedral
Thursday, March 31, 2005
 
 
I think you're completely over-looking the obvious--that some fucking lamo managers nephew got the job and was going to get it all along--you just had to fill the "we looked far and wide" qualifier for that to happen. Don't beat yourself up too much and keep your chin up. You sound like a real catch!

Anon
Anon
Thursday, March 31, 2005
 
 
One point, you don't have to swap the elements, you only have to overwrite the duplicate.  Here's my solution, assuming you don't have to do anything with the elements in the end of the array.

int array[SIZE];
int * onePastEnd = std::unique(array, array+SIZE);

I'd love to see an interviewer after getting an STL algorithm for an answer.
Ryan Phelps Send private email
Thursday, March 31, 2005
 
 
Perhaps the interviewer WAS looking for an STL answer.
.
Thursday, March 31, 2005
 
 
Is this actually what interviewing for programming jobs is like today? You are expected to think exactly like the interviewer?
Bored Bystander Send private email
Thursday, March 31, 2005
 
 
TheWhiteboardTookMyLunchMoney, it sounds like you did fine. As others have said, the rejection was probably unrelated to your performance solving that problem and was likely outside of your control.

Ryan, I (and most interviewers I know) would simply acknowledge that the candidate knows a bit about the standard library (a good thing) and ask them to code the full solution, including any helper functions they use.
MS Anonymous
Thursday, March 31, 2005
 
 
Ryan -

yeah you're right, I'm not sure why I said swap, but my algorithm actually overwrote prev+1 with first non-duplicated item, pushing the duplicates to the end of the array and stopping when no dups are found.  something like:

curr = array+1;
prev = array;

while (!end of array)

if prev == curr, increment curr to first non-duplicate item and overwrite prev+1, then increment prev.. etc, etc.. new size will be indicated by where prev points and all dups will be after prev. 

I'm not sure how much this proves about my programming skills, to be honest.  I like the STL answer, thanks, I'll try it next time :)
TheWhiteboardTookMyLunchMoney
Thursday, March 31, 2005
 
 
what was the job for? 20-25 minutes to solve a problem seems long to me.
Tom Vu
Thursday, March 31, 2005
 
 
I agree that it's impossible to know why you didn't get the job if you felt you did well in the interview. It might have been a bad vibe; it might be that the other candidate was really amazingly good-looking; it might have been that you did a good job but the interviewer thought you were too slow; it might have been that you had a bad interviewer who dinged you for not thinking like them.

If you have a recurring problem not getting jobs after interviews, find someone who would never consider hiring you to give you a practice interview and give you honest feedback. (You're never going to get honest feedback from the person who actually dinged you because they'll be too afraid of lawsuits). Every campus career office can arrange for practice interviews; failing that try SCORE and a nice retired executive desperate to get out of the house will be happy to coach you.
Joel Spolsky Send private email
Thursday, March 31, 2005
 
 
Software Developer job.

I don't know, maybe it was less time, but the 20-25 minutes was basically spent like this:

*2-3 minutes to clarify problem and discuss 2-3 solutions (copy to new array, etc), before settling on the in-place pointer manipulation.

*5 minutes writing pseudo code and working through the solution with my input array.

*4-5 minutes writing the actual code.

*5 minutes testing the code with input.

*More talk at end, discussing solution.

I could have turned my back and wrote the code in 5-6 minutes; however I hear the word on the street is you're suppose to involve the interviewer and "talk" through the solution. 

The interviewing game is driving me nuts.  There's such an insanity about weeding out the bottom 99%, people are nuts over it. 

Anyway, thanks for the encouraging comments guys.  I don't feel so bad now :>
TheWhiteboardTookMyLunchMoney
Thursday, March 31, 2005
 
 
Thanks for the comments/advice, Joel.
TheWhiteboardTookMyLunchMoney
Thursday, March 31, 2005
 
 
Whiteboard, you were rejected because you were meant to use the exact same technique the interviewer read about in Coding for Geeks. He didn't understand the technique but Coding for Geeks said it was a sign of an expert programmer.

They hired a guy who had been taught the technique by a lecturer who also happens to read Coding for Geeks. In fact, the whole team is full of people with the same narrow backgrounds. They're currently looking to hire more expert programmers because their projects are falling behind and users are starting to complain about bugs.

This has confirmed to them the importance of hiring experts, although they can't actually recognise experts.
Been there
Thursday, March 31, 2005
 
 
I personally loathe the idea of "coding auditions" during interviews.

Let me spell out the poor logic of this exercise:

It's pretty widely believed (and frankly, true) that people do their best (most creative, precise, etc.) programming during a state of "flow," which is intense concentration at which one arrives after a period of quiet, uninterrupted thinking.

Therefore, the value that a programmer provides comes mostly from his "flow" work, not from his "ad hoc" work.

And yet--

In order to test a programmer's abilities, we consider it acceptable practice to put him in a situation that is the absolute opposite of "flow"--a hot-seat "audition."

Absolutely nothing can justify this contradiction to me.  It's totally illogical to ask people to do ad-hoc programming tests; because of the nature of proper thinking.

I strongly believe that programming and mental exercises during interiews have about as much value as asking people about their favorite hobbies.  Probably less.

If you want to test I personally loathe the idea of "coding auditions" during interviews.

Let me spell out the poor logic of this exercise:

It's pretty widely believed (and frankly, true) that people do their best (most creative, precise, etc.) programming during a state of "flow," which is intense concentration at which one arrives after a period of quiet, uninterrupted thinking.

Therefore, the value that a programmer provides comes mostly from his "flow" work, not from his "ad hoc" work.

And yet--

In order to test a programmer's abilities, we consider it acceptable practice to put him in a situation that is the absolute opposite of "flow"--a hot-seat "audition."

I think programmers should be treated more like graphic designers than sales people during interviews.  Salespeople should very well be expected to do hot-seat auditions, since talking on their feet is important.  Programmers should have verifiable portfolios of work; and interviewers should be willing to spend the time reviewing them.

After all, the stakes are _that_ high.
indeed Send private email
Friday, April 01, 2005
 
 
I'd like to add, on the idea of "portfolios":

I think there ought to be an industry of "programmer portfolio management," which would be the closest we have to a professional guild.  Professionals sign up and submit work to public, independently-verified portfolios, which are available to interviewers.  These portfolios could perhaps be based on the solutions to problems in contests; e.g., topcoder-style contests which are more realistic in scope and working style.
indeed Send private email
Friday, April 01, 2005
 
 
^^

Wow, that post was garbled.  Speaking of ad hoc...:)

---

I personally loathe the idea of "coding auditions" during interviews.

Let me spell out the poor logic of this exercise:

It's pretty widely believed (and frankly, true) that people do their best (most creative, precise, etc.) programming during a state of "flow," which is intense concentration at which one arrives after a period of quiet, uninterrupted thinking.

Therefore, the value that a programmer provides comes mostly from his "flow" work, not from his "ad hoc" work.

And yet--

In order to test a programmer's abilities, we consider it acceptable practice to put him in a situation that is the absolute opposite of "flow"--a hot-seat "audition."

Absolutely nothing can justify this contradiction to me.  It's totally illogical to ask people to do ad-hoc programming tests; because of the nature of proper thinking.

I strongly believe that programming and mental exercises during interiews have about as much value as asking people about their favorite hobbies.  Probably less.

I think programmers should be treated more like graphic designers than sales people during interviews.  Salespeople should very well be expected to do hot-seat auditions, since talking on their feet is important.  Programmers should have verifiable portfolios of work; and interviewers should be willing to spend the time reviewing them.

After all, the stakes are _that_ high.
indeed Send private email
Friday, April 01, 2005
 
 
"Been There" -- you interviewed for a position with Business Objects too, eh?
GUI Joe Send private email
Friday, April 01, 2005
 
 
> I personally loathe the idea of "coding auditions"
> during interviews.

Unfortunately, you absolutely have to do some kind of coding audition to weed out the tons of people who lie about their experience on their resume.

I've done a bunch of interviews where there was a huge disconnect between what the person claimed to know on their resume, versus what they actually were able to do.

But I agree that the technical interviews are often botched, you don't really need to pose an especially difficult problem for them to solve and just dismiss them if they can't jump into fully creative problem solving mode right off the bat. Logic puzzles and riddles are rather silly too.

But it is necessary to test their technical skill level - one of the things I used to do was to present very small code snippets with bugs such as buffer overruns and returning local stack arrays and see if they understood what the problems were with these snippets. This doesn't require you to get into a full-steam creative mode right off the bat, it just requries you to have a certain level of base technical knowledge about how to code.

If you don't test if the person knows how to code, then you will end up getting people who don't understand how to code, and you'll spend a huge chunk of time training them the basics and cleaning up all the crappy stuff that they do. That's terrible because that person becomes a net drain on your team instead of being useful.
Michael
Friday, April 01, 2005
 
 
^

The problem is that there are tons of people who _DON'T_ lie about their experience on resumes, and so the auditions become competitions between them.

And in my view, it is patently unfair to base that competition on "hot seat" rather than "flow" coding, since it is _empirically proven_ that programmers work an order-of-magnitude better in the latter environemnt.

After all, the game here is to weed out the "top 1%," at least in an ideal environemnt.

As I've said, I think there needs to be some serious reform in this area.  And I also think there should be some consistency.  Organizations are heavily biased towards "who you know," even in technical environments.  I personally have gotten most of my jobs without doing _any_ tehnical interviews, including offers from certain big companies.

My general belief is that the "technical interview" is a game, more than a serious act of deliberation.  Serious programming is voluminous, hard-thought-out work, not individual acts of genius.  Any realistic evaluation of a candidate will take this basic fact into account.
indeed Send private email
Friday, April 01, 2005
 
 
The array duplicate prompts is one of the questions I always ask during an interview. And based on your reponse, I would recommend "no hire".

The main thing I want to learn during an interview is whether the applicant can solve programming problems. And the best approach I've found, despite the flaws other posters have mentioned, is to ask them to solve a clean self-contained programming task.

Some people answer the array duplicate problem immediately, telling me a clean and correct algorithm as soon as they hear the task. Those are the people I recommend hiring.

Also, don't "involve the interviewer in the process". I'd interpret that as meaning that you couldn't solve the problem on your own. Thinking aloud is fine, if it helps your thought process, as is asking for a hint or clarification if you're stuck.

This is kind of harsh, and I'm generally less enthusiastic about potential hires than my coworkers. However, I don't want to be stuck with someone who isn't up to the job.
Julian
Friday, April 01, 2005
 
 
*And based on your reponse, I would recommend "no hire".*

Please explain why?
OneFlew
Friday, April 01, 2005
 
 
This is a particularly interesting post for me, as I'm about to jump on the interviewing rollercoaster-go-round.

It's a shame you never received any feedback as to why they didn't hire. For me, such feedback has been useful in improving interview technique.

My instant response to such a problem as this would be to look in the language library first to see if there's an existing function to do it. Don't reinvent the wheel if there's a perfectly good array.removeDuplicates() function.

Then I'd search the web for an algorithm. I've found many algorithms this way. Of course, there are also the Big Books Of Software Algorithms. Why tear your own hair out trying to re-solve something that somebody originally solved in the 1950's? Your hand-rolled algorithm may be bugged and/or perform less efficiently than a previous solution.

I realise that such a 'post-modern' response would offend many interviewers, which is good as hopefully I've eliminated companies I wouldn't want to work for.

Most truly nasty development problems of the kind that waste weeks are not at *this* low level, anyway. Being able to correctly code such an algorithm is obviously a vital skill, but knowing when to use such which algorithm is much more important.
Mantissa Send private email
Friday, April 01, 2005
 
 
I've yet to receive any feedback, other than the typical "no hire" email three months after the interview.

Mantissa - your post really hit a chord with me. It seems that most people tend to reinvent the wheel for some stupid reason, when an existing tool or algorithm exists. It's a waste of time.

Hell, I probably spend as much time on MSDN, CodeProject and other web sites digging up information as time spent actually coding. It's not only useful, but it's also very educational to see how others have tackled the problem. You just might find out that there's more than one way to skin a cat.
QADude
Friday, April 01, 2005
 
 
The other question I'd ask is: "How did the unwanted duplicates get in there in the first place?". How about looking at where the array gets populated and not entering duplicates in the first place, hence solving the problem at source?

I guess they feel obliged to ask *some* sort of techy question at a developer interview, this sort of question seems straight out of the 1970's, to me and shows a lack of savvy on the interviewers behalf.

Once I had an interview where the techy part involved a low level made-up language that they used to set questions in a written paper. I think this is a more reliable way of measuring aptitude than testing how many algorithms the interviewee can a) carry in their head and b) reproduce in public.

Of course, none of this is going to measure your aptitude for putting together a 'big E' Enterprise system in 2005, which is where most of the work appears to be.
Mantissa Send private email
Friday, April 01, 2005
 
 
"Some people answer the array duplicate problem immediately, telling me a clean and correct algorithm as soon as they hear the task. Those are the people I recommend hiring."

interesting, because those are also the people who have either studied that exact problem just a short while before coming to the interview, or who have spent the last few months re-solving it over and over again to commit it to their memory.

in the first case it proves that they are both lucky and good at preparation...no bad thing certainly, but hardly evidence of good programming skills.....in the second it means that they are stupid enough to rewrite code over and over again.
FullNameRequired
Friday, April 01, 2005
 
 
Call up the guy who interviewed him and ask him why he didn't  hire you. Really.  I've done it and it works.

You will find out one of two things:

- You did something wrong in the interview that lost you the job.  This hurts to find out but it's great to know what it is so you can fix things on your next interview.

- You didn't do anything wrong.  They hired someone with more experience than you, or his nephew or something.  Then at least you can stop second guessing yourself and move on.
John Senior Send private email
Friday, April 01, 2005
 
 
"Some people answer the array duplicate problem immediately, telling me a clean and correct algorithm as soon as they hear the task. Those are the people I recommend hiring."

In other words, you're a fan of groupthink.  Unless a developer solves the problem _exactly the way you would_, he's a no-hire...

Sounds like a fantastic environment you're cultivating.  It consists of:

Julian, his co-workers, Julian, Julian, Julian, and...Julian.  And if anything goes wrong, the in-house guru is...Julian.

:)
indeed Send private email
Friday, April 01, 2005
 
 
"Absolutely nothing can justify this contradiction to me.  It's totally illogical to ask people to do ad-hoc programming tests; because of the nature of proper thinking."

There are ways to give a simple programming test that does allow for proper thinking. We tell people that there will be a short programming test and that they can bring a reference book of choice (C language guide etc.). After the oral interview we give them a simple programming task. The task is always something really simple to guage whether or not they really know the language and that they have basic problem solving skills. Our typical test is to sort an array of names. They can write their answer on paper and take as long as they need.

You would be surprised at how many applicants can't even do the test or take 4 hours to complete it. We have only hired one person who did lousy on the test (but great on the oral interview) and we really lived to regret it. I swear by the test! If done fairly it can be invaluable.
matt
Friday, April 01, 2005
 
 
indeed:

> I think there ought to be an industry of "programmer
> portfolio management,"

Don't just "Think". Do it :)
DEBEDb Send private email
Friday, April 01, 2005
 
 
I once had an interview where they gave me a test that one of the developers was using for his Java class he was teaching at the local university.

It had questions on which was the correct calling syntax of a JDBC method, what the <<< operator did, and some other obscure Java questions. Needless to say I did not pass, I don't memorize JDBC calls and parameter order unless I am working in it day in and day out. <<< (unsigned shift by the way) is something I have never needed in Java (nor have I needed the << operator(s)).

The programming piece where they wanted you to write a method for converting a string of numbers into a number, I failed as well (I had a friend there who told me what I messed up on) because I used Integer.decode() they wanted me to write a loop etc. So in one instance I was dinged for not memorized needless method signatures and in another for using one I had memorized.

They were a consulting company that has since lost almost all thier Java jobs because they could never get things done.

Bottom line, you could do everything right, and it will still be wrong, let alone if you miss parts.......
Lorad
Friday, April 01, 2005
 
 
"Some people answer the array duplicate problem immediately, telling me a clean and correct algorithm as soon as they hear the task. Those are the people I recommend hiring." - Julian

Fair enough, but you're testing for speed of coding. Perhaps you should inform your interviewees of this fact when asking the question -- I, for one, would expect an algorithm question to be marked on the space/time complexity of the solution.

By the way, there is another solution to this problem that is more efficient than the one already mentioned IF there are lots of duplicates in the array -- instead of moving through the array elements one by one, use a binary search to find the next biggest non-duplicate.

This algorithm takes time order D log N, where D is the number of distinct elements in the array and N is the total number of elements in the array. Note that in the case that there are no duplicates this algorithm would be slower than the one already discussed, but it would win if there were lots of duplicates.
Jules
Friday, April 01, 2005
 
 
If you are hiring specifically for TDD then watching someone solve a problem would be important.

BTW, i would use a hash table to get rid of the duplicates. I lose on performance but i know it works at least.
son of parnas
Friday, April 01, 2005
 
 
"I've done a bunch of interviews where there was a huge disconnect between what the person claimed to know on their resume, versus what they actually were able to do.
"

But the point about ad-hoc versus flow points out that you never really determined what they were able to do, did you? 

When you evaluate "what they actually were able to do", how did you do that?  By asking them an obscure coding question?

I've watched other interviewers make assumptions about what a person did or did not know based on only a few questions.  When the questions were very detailed low-level stuff, they tended to come from the interviewers experience.  If the interviewee didn't have EXACTLY that experience, they tended to have a different point of view, which gave them a different answer.  Not a wrong answer, necessarily, just different.  And I watched the interviewer down-grade that candidate just on that.

The goal is finding people who can produce.  I think it's a valid point that ad-hoc coding skills may not address whether they can produce or not.
AllanL5
Friday, April 01, 2005
 
 
To those who suggest an STL or library solution - that's not the point of the question. It's not about how you would do this in your everyday coding. It's about seeing how you write code. This simple excercise is a proxy for the actual work you'll be doing, but it usually takes too long to explain the nuances of the real world problems.

Personally, I'm not a big fan of remove duplicates - I don't know what it shows to the interviewer. I prefer to ask somewhat more complex, but still self-contained problem that involves more design, data-structure selection, and basic algoritm development.

To me if a candidate has trouble dealing with simple coding problems, he'll have a lot more difficulty dealing with complexity of working on the product.
igor Send private email
Friday, April 01, 2005
 
 
> But the point about ad-hoc versus flow points out that
> you never really determined what they were able to
> do, did you?

I wasn't able to find out the full extent of their abilities (it would take a much longer interview process to determine that), but I was able to rule out that they didn't possess knowledge of basic C++ functionality, such as understanding why this is bad:

int x[10];

for ( int i = 0; i <= 10; i++ )
    x[i] = i;


> When you evaluate "what they actually were able to do",
> how did you do that?

First, I present a few code snippets such as the above with basic errors in them. If they can't explain the errors, it shows that they don't have sufficient basic knowledge. Then I ask a larger design oriented question after that. But you'd be surprised at how many people fail the very simple basic knowledge stuff.


> By asking them an obscure coding question?

Does understanding how loops and arrays work seem obscure to you?
Michael
Friday, April 01, 2005
 
 
Bloody hell. indeed seems most on mark.

When a writer is hired, does the interviewer describe a scenario and ask them to write it?

A writer has a history of writing whether published or not. Ditto for a programmer. Ask to see their code. Talk through it. That's what I'd do. If they have no code then I would not hire them. I'd hire for people who enjoy their work so even if they've worked at proprietary places they write code in their spare time, among other things.

About the coding questions. I was once asked such a question in an interview. It was for a Perl developer (not Perl/C module development) position. The guy asked me some sort of sorting question, this was on the phone, I said I'd sort it with the spaceship operators. In 8  years of Perl coding I've only use Perl's builtin sort. Maybe I should learn machine code.

Dude, there's no way you can know why they didn't hire you.

About the 99%. Hello .... Hello ... you know if  you are there are not. Did you not get all those tests when you were young that placed in you relation to your peers. I did. I consistently got 80%. Darn ... what can I do. I'm not in the 99%. This 99% is crock. First you take all the people who aren't smart or motivated enough to go into medicine, or law school ... places where there is more money than programmers make ... maybe that brings you down to the 95%  .. what about engineers ..., scientists ... that will whittle down the pool ... and then there simply are not enough people in that category to fill 5% of the jobs. So then what happens? The key is, imho, to simply hire bright, ambitious people. Thinking that you only  hire 99% is a complete joke.
me
Friday, April 01, 2005
 
 
> My general belief is that the "technical interview" is
> a game, more than a serious act of deliberation.
> Serious programming is voluminous, hard-thought-out
> work, not individual acts of genius.

"Serious programming" also involves a large amount of localized, small-scale problem solving. You need to loop through an array and add up the contents, you need to create something that does a calculation and returns the result, etc.. etc..

When someone doesn't understand how to code, then all of these small-scale problems are loaded with bugs, stuff like buffer overruns, using uninitialized variables, returning stack arrays back outside of a function and so on.

It is a terrible experience to work with someone who does not have the technical knowledge to successfully solve these individual small problems during the course of creating software.

During an interview, I felt it was my primary responsibility to make sure that we never hired someone who did not have the knowledge of the basics.


Sure, a big part of programming is "voluminous, hard-thought-out work", but that's not the only part. Just a sheer amount of hard thinking and hard work will not help you out if you don't have a certain base level technical skill set.
Michael
Friday, April 01, 2005
 
 
"BTW, i would use a hash table to get rid of the duplicates. I lose on performance but i know it works at least."

I'd ding you for that.  I don't go for the guys that create complex objects to solve simple algorithms.  I would at least say, "Suppose this was performance or memory critical and we couldn't afford to create a hashtable?" before I dinged you though.  I don't mind folks that use objects; I mind folks that can't *not* use objects.

I do give technical questions in my interviews.  Finding all of the perfect numbers is the one I use most frequently, although I do use others.  I'm not looking to see if you get it right with no bugs.  I'm looking to see if you can break down a problem.  I'm looking to see if you can correct your bugs when I point them out.  I had a guy once whose solution looped n-squared times (where n is the max number you'll consider to see if it is a perfect number).  I pointed this out (3 different ways) and he couldn't figure out how to reduce it, so I no-hired him.

And regarding flow, I've found that the really good people can get into a flow even in an interview.
OneArmedBandit Send private email
Friday, April 01, 2005
 
 
"Just a sheer amount of hard thinking and hard work will not help you out if you don't have a certain base level technical skill set."

Yes, but you (and others) are missing the basic flaw in this logic:

"If you don't have the technical skills, then you will be unable to perform."

So in doing an interview, you are NOT looking to see if the candidate is capable of doing the job.  You are looking to see if he is...NOT capable of doing it! :)

Because logically, if he DOES have the basic technical skills, that still does not establish him as qualified for the position.

Yes, basic technical skills are a necessary condition of doing the job.  They are NOT a sufficient condition, too.

And what's worse is, the technical tests are NOT "basic," because the whole process of interviewing is in essence a compressed psycho-drama wherein the WHOLE candidate is being evaluated, NOT just his work.  Even if you don't _want_ to do it, you will evaluate him on some level other than "basic technical" during these whiteboard exercises.  Anyone who's given interviews (that includes me) knows this.

What really gets me, especially in the original poster's example and some of Joel's writing, is the additional pressure that interviewees feel to "optimize" solutions.  That is because they are eager to please.  And the ego of the interviwer will lead him to hire the cleverest optimizer.

Is optimizing array duplication a basic technical skill, or even a microcosm of one?  I don't think so.  In fact, I explicitly want somebody whose creativity is NOT directed at algorithm optimizations, but at design decisions.  These are different modes of thinking, and the latter is worth WAY more ($$$) in this industry.  The latter is also much harder to test in a "basic technical test," but it shows up very well in voluminous "real work."

I personally want one kind of programmer as a co-worker: an ambitious, bright, enthusiastic person who can go into a semi-dark room for hours and code like an animal; and emerge with a properly thought-out solution.  He has an intuitive feel for design decisions (especially low-level ones), but he is not an architecture astronaut.

Small-scale problem solving is the essence of line-by-line coding; but using it as a singular measure of technical skill will NOT find that kind of programmer.  That's like trying to find a great artist by looking at his brush strokes, rather than his composition.  And THEN, to make it worse, hiring the guy who brushes the fastest!

So--are we as an industry content to hire people based only on the necessary, but not sufficient conditions?  AND worse, exclude people because of the artificial nature of these evaluations, which don't even touch the tip of the iceberg?
indeed Send private email
Friday, April 01, 2005
 
 
"I've found that the really good people can get into a flow even in an interview"

According to Joel:

"The trouble is, getting into "the zone" is not easy. When you try to measure it, it looks like it takes an average of 15 minutes to start working at maximum productivity."

(which is actually originally from Peopleware)

Ok, so both Joel and Peopleware are wrong; anecdotal evidence rules the day over empirical studies of productivity.

Or, you are only hiring the few good people who DO get into "flow" sooner, or happen to be less nervous, or whatever.  None of which has anything to do with actual programmer skill or ability.
indeed Send private email
Friday, April 01, 2005
 
 
> So in doing an interview, you are NOT looking to see
> if the candidate is capable of doing the job.  You
> are looking to see if he is...NOT capable of doing
> it! :)

Yes, but just in that one segment of the interview. Like I mentioned above, I also ask a larger design question after that, the design question is where I got to see some more details of their problem solving ability and design skills.


> but using it as a singular measure of technical skill
> will NOT find that kind of programmer.

I'm not sure where you got this from, I never claimed that it should be used as the "singular measure" - it is just one of the important elements to measure during the interview.

But it's also one of the easiest to quantify. Conducting good interviews is difficult, there isn't a lot of time and there are various factors such as nervousness, etc, that make the process difficult.

That's why doing some simple tests are important, though - they are one test that you can do that eliminates several problems such as "getting in the flow", that you were complaining about before.

It is often a problem that an interviewer bases too much of the interview on a single complex trick type question, where they fail you if you don't "get it" quickly. But it is possible to conduct a higher quality interview that attempts to reduce these problems, I've been trying to describe some of the elements of conducting a higher quality interview.


> So--are we as an industry content to hire people
> based only on the necessary, but not sufficient
> conditions?

I am content to not hire someone for C++ coding who doesn't possess basic knowledge such as using loops and array indexing.


About 30% of the people I interviewed failed the basic coding questions such as recognizing an off-by-one loop in 3 lines of code or returning stack arrays out of a function, and that's after they passed a phone screen process.


It's clear that there are many problems with the interviewing process, it's definitely an artifical situation that does not do a truly accurate test of the person. However, it's also absolutely necessary if you want to have any chance at hiring someone who won't be a  total drain on your organization.

If you're going to expect for the person to do a lot of coding for the job, then you'd better make them do some coding in the interview to make sure they actually have some knowledge about it.
Michael
Friday, April 01, 2005
 
 
Now, how fair are some of these questions to those who just happen to have trick knowledge?

e.g. I "just happen to know" that perfect numbers are of the form P=N(N+1)/2 where N is a Mersenne prime. A Mersenne prime is of the form N=2**k-1.  A necessary, but not sufficient condition is that k be prime.

My solution would iterate over _k_. I will first do a primality test on k, and if it passes, do a (slower) primality test on N.  If that passes, output P.

It doesn't take a very large k to get a _very_ large P.  In fact, anything after k==31 exceeds the capability of a 64-bit integer.

I just tried this, and on a 2 GHz Athlon, I get up to P=2305843008139952128 instantaneously.  Am I hired?
David Jones Send private email
Friday, April 01, 2005
 
 
"I also ask a larger design question after that, the design question is where I got to see some more details of their problem solving ability and design skills."

I'm sure you can imagine how I feel about design questions during interviews.  Honestly, I'm serious about avoiding pablum tests, even if they follow each other in sequence. :)

"I'm not sure where you got this from, I never claimed that it should be used as the "singular measure""

Well, it is, though--how else are programmers able to show off their bad-ass coding skills during your interviews? :)

And I'd argue that no design question during an interview really elicits any technical design skill, at least not the kind I'm familiar with--ie, the frustrating and difficult process of making designs that actually work, in reality.  Anything else is mental masturbation. :)

'I am content to not hire someone for C++ coding who doesn't possess basic knowledge such as using loops and array indexing."

Well, ok--but I am driving at a key question here.

What interviewing technique finds REALLY GOOD PEOPLE, consistently?  Weeding out the bare minimum is pretty easy; as you've said yourself, the questions are "simple."  But valuable work on-the-job is not simple.

I am serious--there are better ways to do this.  I think topcoder has the prototype implementation of one.  They basically have "programming contests" where people compete under time constraints to solve pretty well-spec-ed problems.  The problems are relatively small; but the important end result is a series of verified code submissions from programmers.

Of course, topcoder is mostly results-oriented, not so much content-oriented.

Let's say you had a web-based environment that was more of a "work simulation."  People are verified (using a credit card number or some such), and they are given a simulated "real world problem"--eg, "you have two weeks to add this feature to this large codebase."  After two weeks they submit their results and those are kept on file, in a "portfolio."  People are allowed to submit "supporting documentation" (eg UML) as part of the portfolio, for simulations.  The system verifies their actual solutions, and the code and such is kept on file.

If you're an interviewer, you log in with the programmer (who controls who can and cannot see the portfolio) and you go over his portfolio, and he explains his solutions to the various simulations to you.

Am I full of shit in thinking this is a better way to do interviews?  The web has made everything else better, why not technical hiring processes?
indeed Send private email
Friday, April 01, 2005
 
 
> After two weeks they submit their results and those
> are kept on file, in a "portfolio."

I can see problems with people wondering how much of the work in those 2 weeks came from the candidate, and how much from the candidate's best friend who is helping him out...

If you really want to know if someone knows how to do some coding, I just don't see how you can avoid making them write some code in the interview to prove it.


> If you're an interviewer, you log in with the programmer <...>

This seems like it could put a lot of additional burden on the interviewer, if they will have to examine portfolios of work solving problems that they aren't familiar with themselves, especially if this is to be their only data in evaluating the candidate.

It really seems to help when the interviewer is conducting an interview over material that is familiar to them, they can compare the candidate's responses to others and they have a better yardstick to evaluate them...

And if the portfolio was standardized, I think you'd end up seeing a lot of canned answers...


> Am I full of shit in thinking this is a better way to do interviews?

It's an interesting idea, and there's definitely some value there... But I myself wouldn't really trust it to totally replace having some actual on-the-spot coding done, though.
Michael
Friday, April 01, 2005
 
 
"I can see problems with people wondering how much of the work in those 2 weeks came from the candidate, and how much from the candidate's best friend who is helping him out..."

Don't Computer Science departments in universities already deal with this problem?  They use automated tools to detect cheating; this system could as well.  And the problems would be at least reasonably significant to demand a lot of effort for a "friend"; and no employer would base his opinion on a single portfolio piece.  Plus there would obviously be accounts, tied to a credit card or some such. :)

Plus I don't think the problems in such a system would be "genius-oriented," unline topcoder.com.  They should be straightforward business-ish problems.  The volume of the work will speak for the programmer, not the quality of individual bits.

"This seems like it could put a lot of additional burden on the interviewer"

No, the idea is that the burden is on the interviewee to explain his design, his thought process, what he did, etc.  The correctness of the code is already verified by the system. :)

All the interviwer has to understand is the "spec."

And think about the potential--you could implement period "spec changes" into the programming simulations that force the programmer to fix and resubmit his work.

"And if the portfolio was standardized, I think you'd end up seeing a lot of canned answers..."

I don't think the portfolio should be standardized; I think the problems should be varied, reasonably straightforward, and the specs should be pretty easy to understand by anyone.

The point is just to make the programmer work on something significant, so I don't think it's that hard to come up with a multitude of problems.
indeed Send private email
Friday, April 01, 2005
 
 
First of all, it's very difficult to determine a person's abilities by chatting with them for 30-60 minutes. Asking them to solve self-contained programming problems is the best approach I've found, despite its limitations. In any case, it's more helpful than discussing their prior jobs.

The problems I pose are, putting aside the time and interview pressures, much easier than the problems my coworkers deal with every day. My philosophy is that people who have difficulty will algebra won't be very good at calculus.

I do focus on one narrow skill, though I try to pick up cues regarding communication ability, personality, etc. However, since other interviewers focus on different concerns, they can balance out my emphasis.
Julian
Saturday, April 02, 2005
 
 
Hello,

I have another approach to an interview.

Background:
we programm with XP (Extrme Programming, nit MicroSoft :-)

So the candidate has to fit in our team.

What I want to see from the candidate is:
0. Is he asking for the "12 steps for better code" by Joel (f.e. Source/Version Control, Working Conditions, Tests)?
1. Does he first think in tearms of unit-tests
(narrowing the testband to the border-cases and one standard-case)
2. Does he comment
3. Does he show any signs of coherence in his naming and coding conventions, is he willing to change his coding style to match ours?
4. Does he beautify his code (indentation, full-brackets)
5. Does he fit int my exsisting group, or is he a individuallistic (selfish ?) programmer
6. Can he Team-Program?
7. is he communicativ (and not information-hiding)?

So these are the things I would like to see from the candidate, not the actual coding techniques. Languages change, Tools Change, but Coding Techniques will last a much longer time.

Peter
Peter Miehle Send private email
Saturday, April 02, 2005
 
 

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics
 
Powered by FogBugz