The Trouble With Too Many Functional Tests

Don't get me wrong. Functional tests have great value. I think apps that are well tested would have a regression suite of functional tests and a collection of non-regression functional tests. Usually, for every pound of production code, I would like to see about two pounds of unit tests and two ounces of functional tests (a little bit goes a long ways). The problem I see in too many shops is zero unit tests and a pound of functional tests.

The following two images demonstrate classes using classes that use classes. There are functional tests that exercise these classes working together. Fixing a bug in one class breaks many functional tests ....

I have seen this happen many times. In one case, a small change broke 47 tests. Meetings were held to decide if the bug should just be left in the code! In the end, it was decided to turn off all tests until the time could be set aside to fix all of the tests. Months passed. Things got mighty stinky ....

The solution is to use unit tests instead:

The result is that the project is more flexible.

Functional Test Misnomers

"By writing only functional tests, I write less test code and exercise more production code!" True! But at the price of making your project brittle. Plus, there are some finer points of your application that will be a lot harder to test without using unit tests. The best coverage and flexibility can only be found through a mix of unit and functional testing that is heavy on unit testing and light on functional testing.

"My business logic is all of these classes working together, so testing just one method is pointless." I'm suggesting that you test all of the methods - just separately. Further, I am not suggesting that you have zero functional tests - they have their value.

"I don't mind if my unit test suite takes a few minutes to run." But do the other people on your team mind? Does your team lead mind? Your manager? If it take a few minutes instead of a few seconds, do you still run the full suite a dozen times a day? At what point do people stop running the tests at all?

"A unit test is anything run by JUnit." It is true that in our industry, the term "unit test" is subjective. I think that what I'm expressing here is the most popular interpretation of the term "unit test".

Comments? Questions? Rude gestures? Click here!