#
For the most part, testing is a great thing that is a surefire way to improve software quality. However, I’ve found that there can be three minor pitfalls when it comes to testing. I’m sure there are more, but these are the ones that I’ve experienced.
When running the tests, your current runtime environment is based off the testing class path. This means that when you are using refactoring, the classes that the test cases are contained within can appear to be valid packages within your application. For example, when recursively searching for a listing of classes by using reflection, the test classes will show up. This is an important thing to note, and to remove the test classes in your cases, when you’re confirming the results of this process. In addition, it is possible to use reflection to see and initiate test cases from the application logic, something that should not be a valid operation in unit testing. For example: If you had a Factory class to initiate only the classes within your JAR file, you could pass in a reference to a test class, and it would initiate it. This would produce undesired behavior.
IDEs consider test cases to be a usage of code. This makes it more difficult to spot dead code in your project. Dead code slows down production time, tends to be non-conforming with the current code base/philosophy, and may be used by an ill-informed developer later.
As of the current JUnit version (4.11), there is no way to disable a test and have a warning created for this state. The only two options for this are to let the unused test case fail intentionally, or to use the ignore annotation. Intentional test case breakages performed by an intentional fail method within the test case, or leaving the functionality broken. Unfinished test cases [which would warrant this situation], should not be left blank, they will show up as a passed test case. Empty test cases should be shown as an error, but are not. This StackOverflow question discusses the Ignore annotation more; however using @Ignore leaves unused test cases, which the developer is responsible for keeping up with. Having a broken test case inflates the failed test case count, and may cause valid test failures to be overlooked.