java hosting


JavaRanch Newsletter Articles in this issue :
Hacking JUnit - Adding a simple extension to the JUnit test frameworkby E.J. Friedman-Hill
Printable Version
A Case Against EJBChris Mathews
Printable Version
Java Data Objects - An IntroductionThomas Paul
Printable Version
The Coffee House - The First Annual JavaRanch Technical ConferenceSolveig Haugland
Printable Version
Introduction to Code CoverageLasse Koskela
Printable Version
How my Dog (almost) learned a Lesson about measuring Project VelocityDirk Schreckmann
Printable Version
Announcing the JavaRanch Radio!Dirk Schreckmann
Printable Version
Book Review of the Month - Enterprise Integration PatternsLasse Koskela
Printable Version
Book Promotions coming in January and February Thomas Paul Printable Version

Hacking JUnit

Hacking JUnit

Adding a simple extension to the JUnit test framework

by E.J. Friedman-Hill

JUnit, if'n you don't know, is a popular unit-testing framework for Java (See http://www.junit.org/.) Using JUnit, you can easily write automated tests for individual Java classes and methods. A typical large project may have thousands or tens of thousands of individual JUnit tests. Here at the Ranch, we use JUnit for testing durn near everything. We like to run our tests often, to make sure our code is alive and kickin'.

Many projects that use JUnit run their automated test suites as part of a nightly build process on multiple machines. Testing on multiple platforms is obviously a good idea, because it helps you find platform-dependent problems. Most projects should test on Windows, Linux, and Solaris as a bare minimum. For some software, testing on workstations from SGI, HP, and IBM makes sense, too -- and perhaps multiple versions of all of these operating systems, as well.

Not every project can afford a whole herd of dedicated test machines, however. Often, test machines are shared between projects, and sometimes test machines are simply desktop or server machines that have some other primary purpose.

In this situation, it may not be possible to set up all the test machines with an ideal testing environment. Indeed, some of the tests might not pass on all the machines, for predictable reasons. This is the problem Tex was facing. Pull up a camp stool and let me tell you about it.

Tex was worried about his herd. He was riding out on the range, rounding up the scruffy herd of test servers. Every one was different, and truth be told, they didn't even all belong to him. He had to beg, borrow, and steal time on many of these machines so that he could test his Java software on many different platforms. His prize cattle, ah, servers, had ideal test environments set up. But the others -- the rare exotic breeds like the IRIX and HP-UX servers -- didn't have the right JDK versions. Other little dogies were missing some of the external software his application used -- it was just too durn expensive to buy for every machine in the herd.

As a result, some of his JUnit tests failed on each machine. All the failures were expected, and due to the variation between the different breeds. But he was always scratchin' his head trying to keep all of these pecadillos straight.

I'm fixin' to help Tex out. Saddle up and come give us a hand, as we extend JUnit so that each test case gets to tell JUnit whether or not it should be run on any given heifer, err, server.

How JUnit works

To write a test in JUnit, you extend the junit.framework.TestCase class and implement one or more methods named testXXXX, where XXXX describes the functionality under test. By default, the JUnit framework will create one instance of your class for each testXXXX method, and invoke these methods via reflection. Each invocation represents a single test. The TestCase class also contains methods called setUp and tearDown. You can override these to set up and dismantle a test scaffold for each test; JUnit will call them immediately before and immediately after calling testXXXX, respectively.

Your class will inherit a fairly large API from TestCase. The inherited methods fall into two categories: methods that let you make testing assertions, which are documented elsewhere (see the book JUnit in Action,) and the lesser-known methods which let you extend the functionality of JUnit itself. We'll look at one such method here:

  public void run(TestResult result);

JUnit calls this method to run the TestCase. The default implementation turns around and calls a method

  public void run(TestCase test);

on the TestResult object. The TestResult arranges to record the success or failure of the test, keeps track of the number of tests actually run, and then turns around and calls

  public void runBare() throws Throwable;

on the TestCase. This method, finally, is the one that actually calls the setUp method, invokes the testXXXX method by reflection, and calls tearDown.

Customizing JUnit

This sequence of calls is complicated, but it give us lots of chances to stick in our own customized code. Because JUnit is so flexible, it's surprising how little code you need to write to add your own features.

What we want to do is, somehow, help Tex out by having JUnit ask each TestCase object whether or not it should be expected to pass in the current environment. Let's extend TestCase to provide our own customized base class for tests. We'll add a method canRun which returns true or false according to whether the test should be run or not. We also need to provide a constructor that takes a test name as an argument; JUnit needs this constructor to put the tests together.

package com.javaranch.junit;

import junit.framework.*;

public class JRTestCase extends TestCase {

    public JRTestCase(String s) {
        super(s);
    }

    public boolean canRun() {
        return true;
    }
}

Now, when you write a JUnit test case, you can extend this class and override canRun. Here's a test case that only makes sense when run under Windows:

package com.javaranch.junit;

public class WindowsOnlyTest extends JRTestCase {

    public DemoTest(String s) {
        super(s);
    }

    public boolean canRun() {
           return System.getProperty("os.name").
                  toLowerCase().indexOf("win") != -1;
    }

    public void testWindowsFeature() {
      // Should run this test only on Windows
    }
}

That's great, but of course JUnit doesn't yet care that we've defined the canRun method, and will run these tests on all platforms anyway. We can make running a test conditional on the result of calling canRun by overriding the run method from TestCase in JRTestCase:

    public void run(TestResult result) {
        if (canRun())
            super.run(result);
    }

That's it! Now if we run this test on Windows, we'll see

C:\JavaRanch> java -classpath .;junit.jar \
    junit.textui.TestRunner com.javaranch.junit.WindowsOnlyTest
.
Time: 0.006

OK (1 tests)

But if we run it on Linux, we'll see

[ejfried@laptop JavaRanch] % java -classpath .:junit.jar \
    junit.textui.TestRunner com.javaranch.junit.WindowsOnlyTest

Time: 0.001

OK (0 tests)

You can use this test class as part of a TestSuite and run it on the whole herd of test servers, and the Windows-only tests will only run on the runty little calves.

Fancying things up

When you override canRun, you can make the implementation as complicated as you want. In particular, you can make it return true for some tests and false for others. The getName method of TestCase returns the name of the testXXXX method that a particular TestCase object will run. You might write a canRun method like this:

   public boolean canRun() {
     if (getName().equals("testOracle"))
       return OracleUtils.oracleIsAvailable();
     else
       return true;
   }

Using this canRun method, the testOracle test method will only be run if the Oracle database is available; otherwise, it will be skipped.

You can make all of your test classes extend JRTestCase, or only some of them. Because only JRTestCase itself knows about the canRun method, you can freely mix and match JRTestCase and TestCase objects in the same TestSuite. You can use JRTestCase only for those tests that are picky about when they should run.

Summary

Using JRTestCase, Tex can write tests that know about their external dependencies. His test suite will now pass at 100% on every server in the herd -- although on some servers, it'll be a mite smaller. Note that JUnit will report the actual number of tests run on each server.

It's easy to add features to JUnit by taking advantage of the rich API provided by its framework classes, and especially by TestCase. This article is barely a drop in the bucket compared to what's possible. I hope I've got you fired up to have a look at the JUnit API documents and see what all else you might have a hankerin' to cook up.

Download all of the code from this article.

Discuss this article in The Big Moose Saloon!


Return to Top
A Case Against EJB

A Case Against EJB

by Chris Mathews

These days, it is becoming harder and harder for me to find cases where I think EJB is appropriate. However, that says as much or more about my personal changes in thinking and development practices as it does about EJB. This article will explore the reasons that are most often cited for using EJBs and discuss how practical they are in real-world solutions.

Remotability

This one is fairly easy to understand. However, most pragmatic developers now try to stay away from large distributed systems unless it is absolutely necessary. The reason is that they don't usually pay off. The perceived decoupling advantage never really pulls it weight and these systems always seem to perform like dogs. Martin Fowler had a nice article (requires registration) describing the evils of unnecessarily distributed systems in Software Development Magazine last year (from content in his PEAA book). Therefore, it is questionable how useful the distributed nature of EJBs actually is.

Security

I don't know about you but I have never really considered J2EE Security a proper solution. There are huge gaps missing from the J2EE Specification for things such as user management and Single Sign-On (SSO). This all adds up to the fact that it is hard to pull off using J2EE Security for real-world solutions without pervasive use of vendor extensions. I thought the whole reason for including security in the spec was to allow for a portable solution. Unfortunately, reality tells a different story and, for that reason, most projects that I have ever worked on have relied third-party solutions such as Netegrity SiteMinder for security.

Persistence

Does anybody actually like working with CMP Beans? I certainly do not. They are hard to develop (almost bearable if you are using XDoclet), hard to test (in-Container testing is the only option here), and not very flexible. Luckily, there are many other persistence options that are more than a match for CMP. My current favorite is Hibernate. This tool just rocks! It supports everything that was good about CMP and has none of the ugly warts that make using CMP such a pain. Sure you still have configuration files, but only one and it is easily managed. You can even use XDoclet to generate your Hibernate configuration file if you want. The best thing about Hibernate (and JDO for that matter) is that it works off of Plain Old Java Objects (POJOs) which make development and testing very easy.

Caching

I actually think this one is a bit of a red herring, always tossed in by the big players to throw us off their trail. While some form of caching is implemented in any decent EJB Container... none of the major EJB Containers support a truly distributed cache out-of-the-box. Therefore, in a clustered environment, which is the norm for large EJB projects, a third-party solution is still required for good caching. Also consider that any good persistence tool supports some form caching as well, so EJBs still don't bring anything additional to the table. For example, Hibernate and most commercial JDO implementations also support distributed caching thru the use of third-party products such as Jakarta JCS, Tangosol Coherence, and EHCache. Whenever the JCache Specification is finally released (hurry up Cameron!) then I am sure that most will add support for that as well.

Scalability

To many people, EJB is synonymous with scalability. I am here to tell you that they are wrong! There is nothing about EJB that is inherently more scalable then any other technology. The real secret to scalability is that good software scales well and bad software scales poorly. This is universally true regardless of the technology that is used. An EJB Container can achieve scalability based on its ability to be clustered and load balanced. However, the same is true of most standard Servlet/JSP Containers and most other web technologies (Perl, PHP, and ASP) as well. Therefore, once again, EJB offers nothing in terms of scalability that cannot be achieved with a simpler solution. In fact, it is my experience that simpler solutions such as Servlet/JSP applications are much easier and cheaper to scale than full blown EJB applications. Remember, scalability is never free; you pay for it in the cost of designing good software.

So far things aren't looking very good for our old friend EJB... but we now come to the light at the end of the tunnel: Messaging and Transactioning.

Messaging

One of the very good things to come out of the EJB 2.0 Specification was the addition of Message Driven Beans. MDBs are quite simply fantastic. This is one area of the specification that they really nailed! MDBs are easy to use, develop, and test (in this case they are easy to mock out). I don't think I ever want to work on a large-scale messaging project without the use of MDBs.

Transactioning

Plain and simple - demarcating transactions in EJB is dead-easy. Using Two-Phase Commit in EJB is dead-easy. These two things hold a lot a weight for systems that have large-scale transactional requirements. In these cases using EJB makes a whole lot of sense, especially if you ever have to deal with 2PC and multiple transactional resources. I know that I am not about to try to reinvent the Transaction Manager wheel on any of my projects, it is just too risky. However, it is important to note that lightweight Frameworks such as Spring and even JBoss's Aspect Oriented Middleware are now showing up with transactional support similar to EJB that can be demarcated on POJOs. In these cases all you need is a JTA implementation and you are good to go. I don't think that this type of solution is quite ready for prime time in a high transactional volume system today but I have no doubt that at some point in the near future they will be. At that point, EJBs will lose their number one advantage.

Conclusion

So to recap... unless your project is making heavy use of transactions, messaging, or both then I think choosing EJB is a mistake. There are too many good alternatives out there to ignore. Even if EJB is chosen, I recommend keeping its use to a very limited capacity. Business logic should be encapsulated in POJOs and the persistence layer should be strictly decoupled from EJB (please no CMP). All of the above benefits of EJBs (with the exception of persistence and caching) can be easily achieved by using EJBs as a thin Decorator on top of POJOs. This type of architecture is easy to develop and test and takes full advantage of the services provided by EJB Containers.

In its current form it looks to me that EJB is on its way out (except in a legacy capacity), though I admit that it is still too early to tell for sure. I guess we will just have to wait and see what is in store for us with EJB 3.0. Suffice it to say, the Expert Group has a lot of catching up to do and I just hope they have the vision and desire to do bring EJB back to the promised land.


Discuss this article in The Big Moose Saloon!


Return to Top

Java Data Objects - An Introduction

by Thomas Paul

The Story Begins

Java Data Objects (JDO) is a specification designed to provide a standardized way to make objects persistent in a database. JDO allows you to treat rows in your database as if they were Java objects. This is similar to other "object-relational mapping" implementations such as Hibernate and Jakarta OJB, however JDO is designed to do what these products do in a standard way.

From an application developers point of view JDO serves the same role as JDBC but it has two main advantages over writing programs using JDBC:

  1. It simplifies accessing the underlying datastore by isolating business application developers from the intricacies of JDBC.
  2. JDO implementations can keep a global cache of objects. Calls to the physical database can be eliminated by using objects that are already in the JDO cache.

Coding JDO

There are actually two parts of mapping objects to rows in a database. The first part is the code that you write in your programs to create objects, get objects, update objects, and delete objects. The second part is the actual mapping of these objects to an actual database. JDO only provides implementation details for the first part. The second part is database specific and how you do the actual mapping will vary from implementation to implementation. JDO is a specification so "out-of-the-box" it doesn't actually support any database. The JDO reference implementation (JDORI), which we will take a look at, supports a pseudo-database that doesn't show how to implement with a real database but it will at least give us an idea of how to write code that uses JDO.

JDO uses a datastore, which represents whatever is the source of the data that JDO will be using. This could be a relational or object database, for example. To access the datastore, JDO uses a class which implements the PersistenceManager interface. This class will be vendor specific. A class that implements the PersistenceManagerFactory interface is used to get the correct PersistenceManager for the datastore that you are accessing. Getting the correct PersistenceManagerFactory is dependent on properties that are supplied to the JDOHelper class.

An example of the JDO properties (used in the example code):

javax.jdo.PersistenceManagerFactoryClass=com.sun.jdori.fostore.FOStorePMF
javax.jdo.option.ConnectionURL=fostore:database/fostoredb
javax.jdo.option.ConnectionUserName=root

The example above specifies that the JDOHelper should find a PersistenceManagerFactory named com.sun.jdori.fostore.FOStorePMF, that it will access a datastore named fostoredb, which is found in the database directory, and that it should connect with the user id of root.

The properties are passed to the getPersistenceManagerFactory( ) static method of the JDOHelper class in the form of a Properties object. An example:

PersistenceManagerFactory pmf =
               JDOHelper.getPersistenceManagerFactory(properties);

A PersistenceManager can then be obtained from the PersistenceManagerFactory. An example:

PersistenceManager pm = pmf.getPersistenceManager();

JDO uses an implementation of the Transaction interface to control transaction flow for all of its functions so you will need to obtain a Transaction object in order to use JDO. A Transaction object can be created using the PersistenceManager.

Transaction tx = pm.currentTransaction();

Configuring JDORI

Before we continue, let's set up an environment that we can use to test some of these JDO commands. You will need to download the JDO reference implementation (JDORI).

The download contains three jar files that you will need to add to your classpath:

  • jdo.jar - standard JDO API
  • jdori.jar - JDORI specific classes
  • btree.jar - used by JDORI to store data

In addition to these you will also need three other jar files to use the JDORI:

In order to create the datastore used in the examples, you will need to write a simple program that creates the datastore. If you are using a real implementation then you would not do this step and the properties you use to access the datastore will be different. Assuming that you have stored the properties listed above in a file called datastore.properties, your program would look something like this:

import java.io.*;
import java.util.*;
import javax.jdo.*;

public class CreateDataStore {
    public static void main(String[] args) {
        try {
            Properties properties = new Properties();
            InputStream stream = new FileInputStream("datastore.properties");
            properties.load(stream);
            properties.put("com.sun.jdori.option.ConnectionCreate", "true");            
            PersistenceManagerFactory pmf =
                    JDOHelper.getPersistenceManagerFactory(properties);
            PersistenceManager pm = pmf.getPersistenceManager();
            Transaction tx = pm.currentTransaction();
            tx.begin();
            tx.commit();
        } catch (Exception e) {
            System.out.println("Problem creating database");
            e.printStackTrace();
        }
    }
}

Notice that we are importing the package javax.jdo. All programs that use JDO will import this package. Notice that in addition to the properties listed above we added one more entry to the Properties object that causes the datastore to be created. As per the javax.jdo.option.ConnectionURL entry in the datastore.properties file, the datastore will be created in a directory called datastore and will be given the name fostoredb.

Creating a Class to Persist

Now that we have a datastore, we need to create a class that can be stored in this datastore. This class will represent a row in a table in a database. For our example, we will create a class called Book that will represent rows in a table called Book. The code for the Book class is similar to many simple JavaBean classes. The class must have a default constructor and it must have get and set methods for each of the instance variables of the class. Here is our sample class:

package com.javaranch.jdotest;

public class Book {
    private String title;
    private String author;
    private String isbn;
    private float price;
    
    public Book() {}
    public Book(String title, String author, String isbn, float price) {
        this.title = title;
        this.author = author;
        this.isbn = isbn;
        this.price = price;
    }
    
    public String getTitle() { return title; }
    public String getAuthor() { return author; }
    public String getIsbn() { return isbn; }
    public float getPrice() { return price; }
    
    public void setTitle(String title) { this.title = title; }
    public void setAuthor(String author) { this.author = author; }
    public void setIsbn(String isbn) { this.isbn = isbn; }
    public void setPrice(float price) { this.price = price; }
}

Before we can use this class we need to run the class file through a JDO enhancer. The enhancer will add methods and variables to the class in order to make it manageable by the JDO implementation. Running through the enhancer can be done from the command line or as part of an ANT script. I will show the commands for each method.

Before you can enhance the class file you will need to create an XML file that is used by the enhancer. This file identifies the classes that you will use and also allows you to specify special field requirements if required. The file should be stored in the same directory as the class file and should be named package.jdo. For our example, the file would look like this:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE jdo PUBLIC "-//Sun Microsystems, Inc.//DTD Java Data Objects Metadata 1.0//EN" 
        "http://java.sun.com/dtd/jdo_1_0.dtd">
<jdo>
    <package name="com.javaranch.jdotest" >
        <class name="Book" />
    </package>
</jdo>

An entry must be placed in this file for each class you wish to enhance. Once you have created this file and compiled your source, you are ready to start the enhancement. Enhancement in JDORI is done using a class named com.sun.jdori.enhancer.Main. The enhancer in another implementation will be different (or may not be required).

From the command line, you would do the following:

java com.sun.jdori.enhancer.Main -s classes 
              classes/com/javaranch/jdotest/Book.class

This assumes that we have stored our class file in a directory called classes. The result will be that the class file has been overwritten with an enhanced version.

To do the same thing from an ANT script we would add an ANT task:

<!-- ===================================== -->
<!-- Enhance                               -->
<!-- ===================================== -->
<target name="enhance" depends="compile" >
    <java fork="true" classname="com.sun.jdori.enhancer.Main">        
        <arg line="-s classes classes/com/javaranch/jdotest/Book.class" />
    </java>
</target>

I have found that you need to run the task with fork="true" because of a problem with Xerces that causes the task to fail otherwise.

Persisting Objects

Now that we have a datastore and an enhanced class to place in our datastore, we can write some code to get our PersistenceManager and create a Transaction object. Here is the code:

InputStream stream = new FileInputStream("datastore.properties");
Properties properties = new Properties();
properties.load(stream);
PersistenceManagerFactory pmf =
              JDOHelper.getPersistenceManagerFactory(properties);
PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();

This code will be the same for any JDO implementation. The only difference will be the properties used to create the PersistenceManagerFactory. We can now look at some examples of using JDO.

First, let's create a Book object and add it to our datastore:

Book book = new Book("Benjamin Franklin : An American Life", 
       "Walter Isaacson", "0684807610", 18.00f);
tx.begin();
pm.makePersistent(book);
tx.commit();

The makePersistent( ) method will cause the Book object to be stored in our datastore.

Suppose we want to fetch this object from the datastore:

tx.begin();
Query q = pm.newQuery(Book.class, "isbn == \"0684807610\"");
Collection result = (Collection) q.execute();
Book book = (Book)result.iterator().next();
q.close(result);

A Query object is used to select objects from the datastore. Creating the Query object requires that we pass a Class object representing the table we wish to select from and a filter representing the criteria used to select objects from the datastore. JDO Query Language (JDOQL) is used to create filters.

Notice that we did not commit the transaction. This was deliberate. Since we have the object, we can make changes to the object and when we commit, those changes will be persisted to the database. For example, suppose we want to update the title of this book:

book.setTitle("Paul Wheaton : An American Life");
tx.commit();

This change will be automatically written out to our datastore. Deleting our book is also simple. While we have an active transaction we can simply do this:

pm.deletePersistent(book);
tx.commit();

In addition to the commit( ) method of the Transaction object, there is also a rollback( ) method that can be used to cancel any transaction.

Conclusions

JDO has several limitations that keep it from being a replacement for JDBC in all cases. The query language is implementation specific so it may not be as robust as SQL. DDL may or may not be supported. Many of the best features of JDO such as caching are also vendor specific and may or may not be available in the JDO implementation that you use. Also, JDO is still a new technology and has not been nearly as well adopted as JDBC. Will JDO take the Java world by storm? So far the verdict is still out but it is certainly a technology to keep your eye on.
Discuss this article in The Big Moose Saloon!


Return to Top

The Coffee House

coffee house

The First Annual JavaRanch Technical Conference

by Solveig Haugland

We don't need to tell you that it can get boring out at the ranch sometimes.

Even through the buzz of a good hard night's debugging and a triumphant finale. Even through the joy of convincing another rancher to stop flirting with .NET and come over to the Light. Even after you've canned your rhubarb and put up a whole mess o'jerky for the winter, and found a great new herbicide for that darned Canadian thistle. Things can get boring.

"And kinda short in the cash flow region, too," added Lacey. (We join this conversation in progress.) "We got this place set up as an Internet Cafe, and a darned nice one too. But these cowpokes just won't go visit the pay sites, if you take my meaning, and the Ella From the Massage Parlor down at the other end of town is givin' it away for free. Dialup, that is."

Zeke piped up from the other end of the cafe, where he was reclining about as languorously as a cowboy can on the green velvet sofa. "You know, all this is because we ain't got no indigenous industry no-how and we don't embrace our inner creative worker."

Brenda sighed. "Zeke, why do you keep reading Kucinich's web site? He ain't gonna win and I think all those years out in the desert made him just a bit tetched."

"Well, you know we ain't got no industry here. We got a bunch of ranchers doin' their own thing, which don't exactly bring out the holiday shoppers or the high-tone clubbers. The only hotel in town closed down two years ago--"

"We had a hotel??"

"--and selling lattes and, well, companionship, are the biggest businesses we got."

"All right, you have a point," sighed Brenda, while Lacey rolled her eyes and busily started writing the coffee specials of the day on the blackboard. "What's your bright idea?"

Sid piped up. "Why don't we put on a circus! I can whip round a rope real good, and I got a real perty cowboy costume in the basement with lots o'spangles, back when I used to ride for the Richard Simmons Ridin' to the Oldies show."

The kind of silence that everyone wants to end very very quickly ensued, and very nearly everyone else in the coffee house started talking busily about the weather, and became fascinated with the buttons on their coats.

Zeke coughed. "That's, um, great, Sid. But what I was thinking is that we got a lot o' space, and a lot o' ranchers, and a lot o' folks who can cut code like there's no tomorrow. Why don't we host a technical conference?"

Sid was annoyed. "And just how is that so different from my circus suggestion?" Everyone ignored him.

"That just might work, Zeke!" said Brenda, surprised even more than she was showing. "I hate to admit it but that's a darned fine idea. Let's get together tonight after the coffee house closes and we'll work on some ideas."

After closing time that night, the midnight soft light bulbs burned, ideas were instantiated and resulting verbal exceptions thrown, and in the morning a tired but happy Brenda wrote a very different list of items on the blackboard and propped it up outside the coffee house for all to see.


LINGONCON 2004
First Annual JavaRanch
Linux
-Klingon Localization Conference



The Javaranchers were frightened out of their wits as they passed by the sign that morning.  "Klingon speakers?? Klingon speakers who are also into Linux? Do you have any idea what kind of person you are inviting out here? I don't care how much money they have--the sight of somebody in a Klingon forehead and parading around in a wearable Linuxdistro is going to sour the milk! I know we Javaranchers ain't exactly shopping at the Gap and buying Britney Spears albums, but you have no idea what you are getting into!"

These and other comments--thought, yelled, and bombarded at one-minute intervals from anonymous email boxes--gave the planning committee a bit of pause. But they soldiered on. Brenda and Lacey sent out press releases, contracted the Generic Suites hotels extremely quickly and even a riverboat for overflow attendees on the river that flowed through town. Zeke pulled some strings at O'Reilly and got them to publish a new book, Forehead First Klingon Linux Ancient Weapons, to be announced at the conference.  Sid got SCO to sue Klingon.org for infringement on something or other and that got a lot of extra publicity. Lacey
invited all 179 small bluegrass bands from Boulder, Colorado to play at the After Dark parties, and Sid got his friends on the Swedish Swing-Dancing Team to come over and do some workshops at the end of the day, calling out the directions in Klingon (Swedish dialect). By the end of the week, the entire Klingon Linux conference was doing the Charleston with both flair and panache. (Flair and Panache are two Klingon warrior-gods and, by way of coincidence, very powerful Linux scripts.)

The week of the conference passed with incident, as might be expected. More than one unfortunate Klingon speaker was invited out on a snipe hunt by some cruel local Javaranchers, and was left pantsless and alone in the middle of snipeless north 40. The inevitable intra-Linux fights broke out about the merits of SuSE vs RedHat and indeed the pronunciation of SuSE. The 117 attendees were outnumbered by the number of Boulder bluegrass bands, but they promised to spread the word for the next year's conference. A few slightly disgruntled attendees suggested that perhaps next year a companion conference among Precious Moments collectors might be held in order to make the dancing a little more gender balanced.

But the core planning team of Brenda, Lacey, Sid, and Zeke were darned excited about the whole thing, and now had indigenous industry and no longer felt alienated from their inner creative workers. Lacey discovered she had a flair for languages, and got the head of the Klingon delegation from Fargo to set up a branch office resulting in several new jobs. Brenda met a very nice fellow from the Redhat folks who promised to show her his fedora next time. Zeke discovered that he was a negotiations whiz, talking on three different cells phones at once when the delegations from LA disagreed on the pronunciation of the Klingon for Linux. And Sid, in an event best left to the imagination, even got to wear his spangled cowboy costume.



Solveig Haugland is an independent trainer and author near Boulder, Colorado. She spends her time creating learning materials and doing training through her company GetOpenOffice.org for those switching from Microsoft Office; Techwriter Stuff: The Single Source, tshirts and posters for techwriters and those who love them; and of course Dating Design Patterns, the original reusable solutions to recurring problems (available, as she is wont to say, through Softpro, Powells, Nerdbooks, and Amazon).  And she is subscribed to the mailing list for the OpenOffice.org Klingon localization project.






Discuss this article in The Big Moose Saloon!


Return to Top
JavaRanch Journal - January 2004 - Introduction to Code Coverage

Introduction to Code Coverage

Lasse Koskela
Accenture Technology Solutions
Copyright © 2004 Lasse Koskela.

Abstract

You might have heard of the term code coverage. If not, this article will introduce you to this fine subject. We'll also take a look at some of the tools available for Java developers and how these tools fit into a build process using Ant.

What is code coverage

Code coverage, in short, is all about how thoroughly your tests exercise your code base. The intent of tests, of course, is to verify that your code does what it's expected to, but also to document what the code is expected to do. Taken further, code coverage can be considered as an indirect measure of quality -- indirect because we're talking about the degree to what our tests cover our code, or simply, the quality of tests. In other words, code coverage is not about verifying the end product's quality.

So, how does code coverage differ from other types of testing techniques? Code coverage can be classified as white-box testing or structural testing because the "assertions" are made against the internals of our classes, not against the system's interfaces or contracts. Specifically, code coverage helps to identify paths in your program that are not getting tested. It is because of this distinction that code coverage shows its usefulness when testing logic-intensive applications with a lot of decision points (versus data-intensive applications which tend to have much less complex decision paths).

Measures

As always, there is more than one way to measure code coverage. We'll cover the essential measures under the broad scope of code coverage one by one, looking at where they're derived from and what do they tell to the developer. I have to warn you though: the list is long, and I have probably omitted plenty of existing code coverage algorithms out there. Furthermore, if you find my descriptions of the measures ambiguous or otherwise incomprehensible, I'm sure Google will help you in finding more thorough explorations on the details of each.

Statement coverage, also known as line coverage, is a measure which indicates the degree to which individual statements are getting executed during test execution. The statement coverage measure is possibly the easiest to implement considering that it can be applied over bytecode, which is a lot simpler to parse than source code having a touch of human handiwork in it. Statement coverage is also probably the most used by developers because it is easy to associate with source code lines -- "ah, that setter really doesn't get executed when my test calls it likes this!" However, the simplicity comes with a cost: statement coverage is unable to tell too much about how well you have covered your logic -- only whether you've executed each statement at least once. Here's an example of that problem:

/**
 * The statement coverage measure will report this method being fully covered
 * as long as we always call the method with condition true. In fact, if our
 * tests never call it with false, we have missed a serious runtime error
 * even though our code coverage was at 100 percent...
 */
public String statementCoverageSample(boolean condition) {
    String foo = null;
    if (condition) {
        foo = "" + condition;
    }
    return foo.trim();
}
			

There is a slight variation of the statement coverage measure called basic block coverage. Basic block coverage considers each sequence of non-branching statements as its unit of code instead of individual statements. This helps in overcoming such result-extorting scenarios where one branch of an if-else is much larger than the other -- statement coverage would easily report the coverage being close to 100% if the tests execute the significantly larger branch but never visit the smaller one.

/**
 * Assuming that our tests invoke this method only with condition false,
 * statement coverage would be approximately 98%, almost "perfect", if you will.
 * Yet, we have missed a small but very important branch and possibly
 * a source of lot of pain later on... Basic block coverage would consider each
 * branch "equal" and report a 50% coverage, indicating that there's probably
 * room for improvement in our tests.
 */
public void bigBranchSample(boolean condition) throws ApplicationException {
    if (condition) {
        System.out.println("Small branch #1");
        throw new ApplicationException("You should have tested me!");
    } else {
        System.out.println("Big branch #1");
        System.out.println("Big branch #2");
        System.out.println("Big branch #3");
        System.out.println("Big branch #4");
        ...
        System.out.println("Big branch #98");
    }
}
			

Decision coverage (also known as branch coverage) is a measure based on whether decision points, such as if and while statements, evaluate to both true and false during test execution, thus causing both execution paths to be exercised. Decision coverage is also relatively simple, which is both its pro and its con. The downside is that the measure doesn't take into consideration how the boolean value was gotten -- whether a logical OR was short-circuited or not, for example, leaving whatever code was in the latter part of the statement unexecuted.

if (amount > 100 || someCode() == 0) {
    // Did someCode() get executed? Decision coverage doesn't say...
    doSomething();
} else {
    doSomethingElse();
}
			

This deficit of the decision coverage measure is tackled to some degree by condition coverage, which extends the boolean evaluation of decision coverage into the sub-expressions (separated by logical ANDs and ORs) as well, making sure each of them is evaluated to both true and false. However, condition coverage is not a true superset of decision coverage because it considers each sub-expression independently, not minding about whether the complete expression is evaluated both ways. But wait, there's more. We have a measure called multiple condition coverage, which also takes into consideration the whole expression as well as the sub-expressions. This, however, often causes the number of required test cases to explode if the code under test employs complex boolean expressions. Making multiple condition coverage high does lead to thorough testing, but one really must consider whether the line of decreasing returns has been reached already...

Moving on, path coverage represents yet another interesting measure. Path coverage measures whether each possible path from start (method entry) to finish (return statement, thrown exception) is covered. Note that this is different from checking whether each individual decision is made both ways. Path coverage has its problems as well. For example, it is sometimes impossible to rate well with regard to path coverage if the code under test includes subsequent branching statements with such expressions that always evaluate the same -- it's impossible to test all theoretical combinations of those paths if the expressions have constraints with each other. The example below illustrates this problem. Another possible problem scenario is loops because they represent a huge range of possible paths (does this loop execute 0, 1, 5, or 1000 times?). There are some variations to the path coverage measure that try to tackle this problem, mostly by simplifying the problem to a manageable size, for example by considering loops to have only two possible paths; zero or more rounds (or one or more in the case of do-while).

/**
  * Path coverage considers this method having four possible paths but because
  * the branching is constrained into a single boolean, we can never execute
  * "A1, B2" or "B1, A2"
  */
public void pathCoverageSample(boolean foo) {
    if (foo) {
        System.out.println("Path A1");
    } // the else-branch would be "Path B1"

    // ...

    if (foo) {
        System.out.println("Path A2");
    } // the else-branch would be "Path B2"
}
			

Function coverage is a measure for verifying that each function (method) is invoked during test execution. In all its simplicity, function coverage is a very easy way to spot the biggest gaps in your code coverage.

One of the more advanced code coverage measures I though I should mention is race coverage, which considers multiple threads executing code simultaneously, thus, helping detect shared resource access problems. These types of measures are seldom used in other than testing software such as operating systems.

Still, we have relational operator coverage, which measures whether expressions with relational operators (<, <=, >, >=) in them are exercised with boundary values. For example, if the expression (a < 4) is executed with values of 2, 3 and 5, the relational operator coverage would report the expression as not covered -- the boundary value a == 4 was not tested. Yes, you guessed correct. This coverage was invented to catch those oh-so-common one-off errors...

So, after listing some of the different algorithms out there, "what are the ones I need to know," you might ask. "Which of those should I use?" Let me post-pone answering that one until later, ok? Let us just move on to what existing tools have to offer to us, and we'll get back to that question in the wrap-up.

Tools

Today, the developer community is in a relatively good situation regarding the availability of high-quality code coverage tools. We'll only take a look at a few, but there are lots more out there -- most are for sale, but some are free or even open source.

First, let's start by looking at how code coverage measures are generally implemented.

Implementation techniques

To my knowledge, the implementations can be categorized into two distinct implementation approaches:

  • Instrumentation
    Instrumentation is all about manipulating the application code by injecting reporting code into strategic positions. In fact, the art of instrumentation has two schools: class instrumentation and source instrumentation. Not surprisingly, the difference is that class instrumentation injects the reporting code directly into compiled .class files while source instrumentation creates an intermediary version of the sources which are then compiled into the final, source-instrumented .class files. Most code coverage tools have chosen one of these two instrumentation techniques.
  • Custom JVM
    Another alternative to injecting reporting code to the bytecode is to move that responsibility into the JVM itself. Code coverage analysis could be implemented by having the virtual machine keep count of which parts of the loaded classes are executed. In practice, however, I haven't heard of any popular tools taking this approach.
Actually, I lied. There is a third category, which I hesitated listing above. The third way to instrument the application code with reporting code is to do it explicitly. Hansel, for example, takes this approach by requiring the developer to encapsulate his JUnit tests into Hansel's wrapper classes. Under the hood, these wrappers use a custom class loader to inject the reporting code at runtime instead of pre-processing the source code or bytecode during the build process. This approach is seemingly unpopular because of the tight coupling into one particular coverage tool (there are other problems as well, related to the use of a custom class loader but that's really out of this article's scope).

So, we have some idea how the code coverage tools at our disposal. Are you itching to find out what kind of features these tools typically boast? (Except for running those boring algorithms, of course)

Typical features

The feature sets, quality and usability of code coverage products vary significantly. I have listed here some of the most essential features from the point of view of usability.

  • Ant integration
    Probably most Java projects today are using Ant (or Maven) to manage their build process, including running unit tests (or functional tests). Thus, Ant integration is one of those features a code coverage tool cannot afford not to have. However, there are subtle differences in how nicely the code coverage related targets fit into an existing build script. Yes, we will see some examples later on. Of course, most tools also provide an alternative, standalone way of running the analysis, either from the command line or via a GUI application.
  • Report formats
    Another obvious feature a code coverage tool must have is reports. Again, there are differences in the type and quality of supported reports. Some tools provide only textual summaries in Ant console output, others produce huge tables of names and numbers in HTML, and others produce nice pictures, others offer to render all this in PDF as well. We'll see examples of the main types of reports in the next section.
  • Source code linking
    Somewhat related to the previous item, source code linking is something one can't live without once having gotten the taste of it. In practice, source code linking means that, as a part of the code coverage report, the tool has generated annotated copies of the actual source code, highlighting the parts which are not covered by your tests. I wouldn't be surprised if this particular feature was the single biggest benefactor in code coverage tools reaching critical mass. Seeing the code block that is causing your coverage to stay away from the "green" is a lot more efficient than seeing that a particular method contains some code that isn't covered by the tests. All of our selected examples include source code linking in their feature set.
  • Checks
    It doesn't take too long after someone has introduced a rule when someone else introduced the idea to enforce that rule. That has also happened to code coverage. Some tools provide a means to pull up the red flag if code coverage drops below a given level. In the context of Ant integration, the build script might typically fail the build until the tests have been reinforced to cover the naked parts of the code.
  • Historical reports
    Few tools provide a way to collect a history of coverage data and produce historical reports illustrating how your project's code coverage has been fluctuating in the course of time. This is also a nice feature, although some might consider it rather irrelevant. In the end, it's not the past we're interested in, but the future.

Now that we've covered some of the core features in modern code coverage tools, let's take a look at how they fit into our existing Ant build script and what does the outcome look like.

Code coverage in action: samples with Clover, JCoverage and GroboUtils

Sample application

Our sample application is something I recently wrote on my spare time. The actual application, however, is of little importance. Our main focus is the structure of the project directory and the Ant build script, the build.xml file.

The project folder structure

As you can see from the above screenshot, the project directory contains only three directories (of which "classes" is generated by the build) and one Ant script. The initial version of our Ant script, which we will extend by introducing three different code coverage tools, is listed below:

<project name="CodeCoverageArticle" default="all" basedir=".">

  <property name="src.dir" value="src"/>
  <property name="lib.dir" value="lib"/>
  <property name="lib.common" value="${lib.dir}/common"/>
  <property name="build.classes.dir" value="classes"/>

  <path id="compile.classpath">
    <fileset dir="${lib.dir}/common">
      <include name="*.jar"/>
    </fileset>
  </path>

  <path id="test.classpath">
    <pathelement location="${build.classes.dir}"/>
    <path refid="compile.classpath"/>
  </path>

  <target name="init">
    <mkdir dir="${build.classes.dir}"/>
  </target>

  <target name="compile" depends="init">
    <javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="true">
      <classpath refid="compile.classpath"/>
    </javac>
  </target>

  <target name="clean">
    <delete dir="${build.classes.dir}"/>
  </target>

  <target name="all" depends="clean, compile, test"/>

  <target name="compile.tests" depends="compile">
    <javac srcdir="${src.dir}" destdir="${build.classes.dir}">
      <classpath refid="test.classpath"/>
    </javac>
  </target>

  <target name="test" depends="compile.tests">
    <junit printsummary="true">
      <classpath location="${build.classes.dir}"/>
      <classpath refid="test.classpath"/>
      <batchtest fork="yes" todir="${reports.tests}">
        <fileset dir="${src.dir}">
          <include name="**/*Test*.java"/>
          <exclude name="**/AllTests.java"/>
        </fileset>
      </batchtest>
    </junit>
  </target>

</project>
			

Common integration steps

Regardless of which of the three demonstrated products are used, there are a number of similarities that can be handled uniformly, separating the minor details from the actual code coverage tasks. The common changes to our initial build.xml are as follows:

  • Obtain the libraries for the selected code coverage tool, and create a directory for them under ${project.home}/lib (for example, Clover's clover.jar would go under ${project.home}/lib /clover while JCoverage's jcoverage.jar and 3rd party .jar files would go under ${project.home}/lib /jcoverage)
  • Introduce a directory for instrumented files (either source code or class files)
  • Introduce a directory for the generated reports
  • Include the code coverage libraries into the classpath used by the JUnit task

I've highlighted the changes in the following, extended version of our Ant script:

<project name="CodeCoverageArticle" default="all" basedir=".">

  <property name="src.dir" value="src"/>
  <property name="lib.dir" value="lib"/>
  <property name="lib.common" value="${lib.dir}/common"/>
  <property name="build.classes.dir" value="classes"/>

  <property name="build.instrumented.dir" value="instrumented"/>
  <property name="coverage.toolname" value="XXXXX"/>
  <property name="lib.coverage" value="${lib.dir}/${coverage.toolname}"/>
  <property name="reports.dir" value="reports"/>
  <property name="coverage.report" value="${reports.dir}/${coverage.toolname}"/>

  <path id="compile.classpath">
    <fileset dir="${lib.dir}/common">
      <include name="*.jar"/>
    </fileset>
  </path>

  <path id="coverage.classpath">
    <fileset dir="${lib.coverage}">
      <include name="*.jar"/>
    </fileset>
  </path>

  <path id="test.classpath">
    <pathelement location="${build.instrumented.dir}"/>
    <pathelement location="${build.classes.dir}"/>
    <path refid="compile.classpath"/>
    <path refid="coverage.classpath"/>
  </path>

  <target name="init">
    <mkdir dir="${build.classes.dir}"/>
    <mkdir dir="${build.instrumented.dir}"/>
    <mkdir dir="${reports.dir}"/>
  </target>

  <target name="compile" depends="init">
    <javac srcdir="${src.dir}" destdir="${build.classes.dir}" debug="true">
      <classpath refid="compile.classpath"/>
    </javac>
  </target>

  <target name="clean">
    <delete dir="${build.classes.dir}"/>
    <delete dir="${build.instrumented.dir}"/>
    <delete dir="${reports.dir}"/>
  </target>

  <target name="all" depends="clean, compile, test"/>

  <target name="compile.tests" depends="compile">
    <javac srcdir="${src.dir}" destdir="${build.classes.dir}">
      <classpath refid="test.classpath"/>
    </javac>
  </target>

  <target name="test" depends="compile.tests">
    <junit printsummary="true">
      <classpath location="${build.instrumented.dir}"/>
      <classpath location="${build.classes.dir}"/>
      <classpath refid="test.classpath"/>
      <batchtest fork="yes" todir="${reports.tests}">
        <fileset dir="${src.dir}">
          <include name="**/*Test*.java"/>
          <exclude name="**/AllTests.java"/>
        </fileset>
      </batchtest>
    </junit>
  </target>

</project>
			

The only thing left to do is the tool-specific part, the proprietary Ant tasks for performing the instrumentation and reporting at suitable stages of the build process. This is what the following sections are focused on.

Integrating Clover

Clover