Screaming Penguin - Source Does Matter

Sadly though, recruiters probably eat that @!#$ up, especially when you are in the top 90%. People who aren't technical or haven't really used Brainbench probably will give the certifications more credence than they deserve. And interestingly enough, having done several rounds of hiring lately, I have started seeing people list these on their resume. What's interesting was that at first I sorta laughed them off but after checking a couple of resumes vs the scores listed, I really did see a good correlation between scores and experience. I am of course talking about morons who listed Brainbench scores that weren't at the master level and down in the 40-60th percentiles. When comparing them with resumes that reflected good skill, the scores were much higher.
I am not saying that a brainbench certification is nearly as good as an MCSE or whatever, but for areas like "internet concepts" and other areas where there is a diverse amount of information that lies outside the realm of a general MCSE certification, I would trust the brainbench exam I took to tell me that an MCSE I was hiring had more than just what he was spoonfed in MCSE courses. You, Atrox, now being a web developer and being an MCSE should do excellent on the exam as you did. I myself ran across topics I didn't really know about such as details about how HTTP Headers work and details about MIME types, which of course was reflected in my score.
On the other hand I do see the inherent problems with Brainbench. Someone could easily grab a couple of knowledgeable friends and sit around the computer and 'put their heads' together to achieve top scores. People can constantly retake the tests until they do well. Some tests are not suited for 'unmonitored testing', such as the English Vocabulary test. I took that test this morning for fun and although it would have been very difficult had I not had access to internet dictionaries, I was able to achieve a 98% percentile by cheating. When taking a test like 'Internet Concepts', I don't care if you 'cheat' because if you can figure the answer out in 180 seconds for a test, I figure you can figure it out in 180 seconds for a job.
But thats just my .63 cents worth.

On November 15th of 2006, I took the Brainbench Java 2 test.

When I went into the test, I was expecting something that would test my general knowledge of the Java language
and object oriented programming. The test that I took did not do that.

There were a large number of questions on special purpose API's that I have never used. There were some questions on development tools that I have never used. And there were a large number of "brain teaser" questions on code snippets which I could have answered, if I would have had more time.

As a result, my test score was very low (2.60).

That score shows that I'm not good at guessing at API's and tools that I've never used. And it shows that I'm not
good at brain teasers with a three-minute time limit. However, it shows ABSOLUTELY NOTHING about my Java
and object oriented programming skills.

In 2002, I studied a book on Java, and then I took a college course on it. I got an A in the course. According to
my Brainbench score, I should not have been an "A" student.

For over a year, I was the sole designer and developer of web software that my employer believes is marketable. That software involves thousands of lines of Java code, a large number of API's, and a large number of advanced object oriented constructs. According to my Brainbench score, I should not be able to accomplish what I have just accomplished.

When I was in college, I took secondary education courses. Later, while I was working for a former employer, I
designed two computer courses which I taught for several years. As a trained and experienced trainer and tester,
it is my opinion that the Brainbench Java 2 test has a number of very serious flaws.

It appears that there have been no sound scientific studies regarding the Brainbench claim that their tests predict employee success. And it appears that there are a large number of companies that are blindly accepting these unsubstantiated claims.

It seems to me that a sound scientific study for the Java 2 test would include the following elements: have
thousands of working and successful Java programmers take various Java 2 tests; have thousands of inexperienced people with Java knowledge take the same tests; for individual test takers, have tests with a large number of questions on API's and tools that they have never used; for individual test takers, have tests with a large number of questions on API's and tools that they have used; for the latter, follow their careers as Java programmers for at least five years.

It appears to me that the creators of the Brainbench Java 2 test do not know what a typical Java programmer
does, and they have no understanding of the art of testing.

I wonder how many careers have been derailed as a result of flawed Brainbench tests. I would like to see a scientific survey on that.