Screaming Penguin - Source Does Matter

The recent Artima post: A Set of Unit Testing Rules, has sparked some debate.

As soon as I read the 'rules' Michael Feathers penned I was sort of confused:
A test is not a unit test if:
# It talks to the database
# It communicates across the network
# It touches the file system
# It can't run at the same time as any of your other unit tests
# You have to do special things to your environment (such as editing config files) to run it.

Tests that do these things aren't bad. Often they are worth writing, and they can be written in a unit test harness. However, it is important to be able to separate them from true unit tests so that we can keep a set of tests that we can run fast whenever we make our changes.

Those are good guidelines to keep your tests simple, but I personally do not think they are a good indicator of what is and what is not a "unit test".

As Cedric Buest commented on that story I think the definition of what really is a "unit test" is a bit more broad and frankly even simpler. A unit test tests a "unit".

Cedric put it this way in his comments "a unit test is a test that only exercises one class in isolation of all others". I would say thats accurate but that the distinction it test a "class" is not really a requirement. Sure your classes should be modeled so things end up there, but its not a requirement. A unit test should test an isolated "unit" of stuff (which should be a class - sure).

After I read the Artima article I then noticed way down in the comments that Cedric pointed to his own distilled version of the the whole discussion in a blog entry on his own site Why unit tests are disappearing.

In that article Cedric again notes the definition and distinction and also makes the astute observation that "these semantic games are fun to play but not very useful at the end of the day. The bottom line is that the success of JUnit has shown that the line between unit and non-unit tests is increasingly being blurred".

The bottom line is make sure things are being tested. Whether you are "unit" or "integration" or "functional" testing is less important, its still important to be aware and have an understanding of such, but the main thing is to have the automated tests.

Cedric also notes that TestNG now has the capability to annotate to "tag" and "group" tests so that concerns over performance (used as some of the original justification for this discussion) along with other motivations can be satisfied. I havent used groups for tests yet but it does seem like "a very powerful and addictive feature".