TopTal, Codility, and skill testing
-
Remember our discussion on how developers are like players on a sports team? Here we go again! Haven't yet found an objective way to measure a developer's abilities so the whole exercise of trying to test candidates is pointless. Developers respond to different problems in different ways. There isn't always the 'right' answer. Developers have good days and bad days; their programming form comes and goes. Some are really good at parsing strings. Others are really good at implementing user interfaces. Some developers do well in a particular team. Others crash and burn because of a personality clash. Some enjoy being under pressure, or turning out a solution quickly. Others want to take the time to get the best solution possible. Some are good at testing. Others at finding and fixing obscure faults. Most are crap at writing good documentation. Nearly every one has their own interpretation of what requirements mean. Or what makes good UI design .... The most corrosive aspect of testing, or trying to compare developers, is that it can erode a developer's confidence in their own skills. Just like a striker on a soccer team, confidence is a major component of success. You can't measure that.
-
Marc Clifton wrote:
How exactly does solving an arbitrary algorithm test coding skills? How does it demonstrate good OOP practices, or DB architecture skills, or an understanding of Linq, or really much else other than "getting the trick?"
This is a great question. Is there any way for a site to determine your level of ability without having some human intervention? Can a test be given which is graded by a computer and accurately tests someone's skills?
There are only 10 types of people in the world, those who understand binary and those who don't.
(Automated) Function Point Analysis as a measure of the "scope" / magnitude of the applications one has designed and / or developed. Is designing and building entire mission-critical "systems" "better" than being good at (just) algorithms? Who needs who?
-
Just to get this out of the way, I failed the TopTal[^] application. How did I even find TopTal? In the CP newsletter! TopTal's primary screening process is to use Codility[^] to see how good your skills are. Now, the 90 minute timed test at Codility asked me to solve three problems: 1. the point in which in an array, the count of X from the left != count of X from the right. 2. given some bit encoding scheme, convert N to -N with the least number of bits. 3. the optimum number of moves a chess knight must make to get from (0, 0) to (m, n) Regarding the last, someone here at posted about that idiocy of that actually testing coding skills. In fact, they all are extremely poor tests of skill. Each (particularly #2 and #3) probably involve a simple trick to figure it out, and if you don't know the trick (like me) you're spending a lot of time just thinking about the problem. At least I was. :) How exactly does solving an arbitrary algorithm test coding skills? How does it demonstrate good OOP practices, or DB architecture skills, or an understanding of Linq, or really much else other than "getting the trick?" After working hard at #1, writing assertions, commenting the code, testing edge cases, optimizing the algorithm for O(1) performance, I realized I had spent an hour on one stupid problem. That left 30 minutes for the remaining two. Riiiight. Now, Codility says something like "don't worry if you don't complete all three tests, just show your best work, even if you only complete one test." And then the test results are amusing. The requirements don't state what to do with incorrect inputs into the "solution" method, and they clearly can't handle exceptions being thrown -- I noticed my score in their example test dropped dramatically when I used exceptions. Now to TopTal. I got an email rejecting me because my score was low. It was pretty darn obvious that no one had even bothered to look at my code! That REALLY :mad: me. TopTal: :thumbsdown: Codility: :thumbsdown: Anyone that uses Codility: :thumbsdown: Sadly, this sort of crap testing methods is probably going to be used more and more. Oh, and Codility has an "honor system" where you won't talk about their tests. F*** them. They should come up with better tests! Mar
Marc Clifton wrote:
the point in which in an array, the count of X from the left != count of X from the right.
Count of X as in the number of times X appears?
Regards, Nish
Website: www.voidnish.com Blog: voidnish.wordpress.com
-
Just to get this out of the way, I failed the TopTal[^] application. How did I even find TopTal? In the CP newsletter! TopTal's primary screening process is to use Codility[^] to see how good your skills are. Now, the 90 minute timed test at Codility asked me to solve three problems: 1. the point in which in an array, the count of X from the left != count of X from the right. 2. given some bit encoding scheme, convert N to -N with the least number of bits. 3. the optimum number of moves a chess knight must make to get from (0, 0) to (m, n) Regarding the last, someone here at posted about that idiocy of that actually testing coding skills. In fact, they all are extremely poor tests of skill. Each (particularly #2 and #3) probably involve a simple trick to figure it out, and if you don't know the trick (like me) you're spending a lot of time just thinking about the problem. At least I was. :) How exactly does solving an arbitrary algorithm test coding skills? How does it demonstrate good OOP practices, or DB architecture skills, or an understanding of Linq, or really much else other than "getting the trick?" After working hard at #1, writing assertions, commenting the code, testing edge cases, optimizing the algorithm for O(1) performance, I realized I had spent an hour on one stupid problem. That left 30 minutes for the remaining two. Riiiight. Now, Codility says something like "don't worry if you don't complete all three tests, just show your best work, even if you only complete one test." And then the test results are amusing. The requirements don't state what to do with incorrect inputs into the "solution" method, and they clearly can't handle exceptions being thrown -- I noticed my score in their example test dropped dramatically when I used exceptions. Now to TopTal. I got an email rejecting me because my score was low. It was pretty darn obvious that no one had even bothered to look at my code! That REALLY :mad: me. TopTal: :thumbsdown: Codility: :thumbsdown: Anyone that uses Codility: :thumbsdown: Sadly, this sort of crap testing methods is probably going to be used more and more. Oh, and Codility has an "honor system" where you won't talk about their tests. F*** them. They should come up with better tests! Mar
I hire online developers (in a small way). My teenage son (a keen developer), a couple years back, tried to get me to hire people from TopTal. "No way", I said, "they are all pose".