Netbeans 6.9 - Adding Basic Code Coverage

Netbeans 6.9 - Adding Basic Code Coverage

Description

Netbeans provides (and has done so for quite some time) good JUnit test functions allowing you to with the click of a button generate unit tests. Due to the ease of using and generating unit tests it has encouraged me to use them and so Ive been adding them into my recent code. However, not long after adopting Unit tests on a broad scale I find myself wanting to know how well I'm doing with the use of tests. I need to have some sort of code coverage information. There are at least two main contenders in this area (in the Open Source market of-course) and these are:

• EMMA - http://emma.sourceforge.net
• Cobertura - http://cobertura.sourceforge.net

These are both aimed at slightly different areas but both provide code coverage info. However, before going ahead and implementing these within Netbeans (and especially Im interested in Cobertura) I decided to use Netbeans 6.9 inbuilt plugin for code coverage (which is based on EMMA anyway).

Implementation of Code Coverage

It was very simple and what follows are the steps to get it to work:

1. Go to the plugin manager in Netbeans and download and install the Code Coverage tool.
2. Run your tests across the whole project.
3. Right click on the project, go to the new 'Coverage' link and 'Activate Coverage Collection' which will allow your code to be instrumented.
4. Re-run your tests.
5. Then go to the 'Coverage' and 'Show Project Coverage Statistics'. These will only work after the code has been instrumented and the tests re-run with the instrumentation in place.

This will then give you a report which will show the code coverage across your project for each class and also for the lines within the class and how many are covered by tests (i.e.: it'll look at the branch points like any coverage tool and check you have tests for each case). Furthermore it'll also highlight in your code, when you activate the coverage collection, highlighting which will paint covered code in light green and non-covered in light red so that at a glance you can see what is and isn't covered by tests in your code. Personally I switch this on, run the project stats, view them, see what classes are poorly covered, and then use the colour highlighting to see where I need more tests. However I don't leave it on all the time as I don't like my code painted red/green all the time.

Example Screenshot

The following is a screenshot of the code coverage project stats page along with how to activate it. The numbers indicate:

1. The new 'coverage' link which appears as part of the plugin install.
2. Shows where to start gathering stats (by getting the code to be instrumented) and where to trigger the project stats page to be displayed.
3. The project stats page itself and some of the stats it displays showing class coverage, individual line level coverage etc.

Final Overview

In a future post Id like to look into implementing / using Cobertura in Netbeans and viewing the reports that Cobertura generates through Netbeans in some simple way, although this tool/plugin for viewing code coverage will be quite ample for the time being as it provides me with the ability to see code coverage across the project, the class level and even down to the line level through the coloured highlighting it provides.  This is enough to show me were I need to test more at this stage, with the future updates I think being more a fancy-shmancy kinda thing.

Adding JUnit / Hamcrest Functionality & Sources To Netbeans

Adding JUnit / Hamcrest Functionality & Sources To Netbeans

Im using JUnit4.5 in Netbeans for my unit testing (which in itself I wont go into detail in this blog post but JUnit and Unit Testing are things that are highly important and that I am all for the use of). Now while the latest JUnit version is up to 4.8.1/4.8.2 currently (20100712) Netbeans is not yet by default implemented to use this so for now Im using the version Netbeans comes with, 4.5, and It may be a future blogpost to upgrade Netbeans to the latest JUnit version (not tonight tho cos its 2:44am, Im still writing about Hamcrest and I got work tommorrow). Furthermore JUnit as of version 4.4 uses a library called Hamcrest internally. This is for performing a number of cool things whereby the assert* set of methods in JUnit are extended by the method assertThat():

http://www.junit.org/apidocs/org/junit/Assert.html#assertThat%28T,%20org.hamcrest.Matcher%29

using a parameter called a Matcher. Using the Hamcrest tutorial as a starting point Im going to go through and describe the Hamcrest library and its functionality. The tutorial I mention along with a number of other links I found interesting are at the end of this post....though you first have to read it before you get to them of course :-).

Now Matchers as provided by Hamcrest allow us to test in a more fluent, spoken word declarative sort of a way and the Hamcrest library comes with a number of useful matchers. Here are some of the most important ones (and its important to note that a lot of these only come with the latest version of hamcrest)

Core

• anything - always matches, useful if you don't care what the object under test is
• describedAs - decorator to adding custom failure description
• evaluate - matches if a specified Boolean condition evaluates to true.
• given - conditionally matches a value or Matcher
• isA - matches a specific type
• throws - matches if a function throws the given exception

Logical

• allOf - matches if all matchers match, short circuits (like &&)
• anyOf - matches if any matchers match, short circuits (like ||)
• not - matches if the wrapped matcher doesn't match and vice versa

Object

• equalTo - compares objects using ==
• sameInstance, strictlyEqualTo - compare objects using ===
• hasProperty - checks that a property exists and, optionally, that it matches the provided matcher
• hasProperties - similar to hasProperty, but accepts a dictionary of properties and their associated matchers
• hasPropertyChain - checks that a chain of properties exist and optionally that the value matches a provided value
• instanceOf - test type
• notNullValue, nullValue - test for null

Collections

• array - test an array's elements against an array of matchers
• arrayWithSize - match an array's size against a number
• hasItem, hasItems - test a collection contains elements
• everyItem - test that a matcher matches every item in an array
• sortedBy - test an array's elements are sorted a single field
• sortedByFields - test an array's elements are sorted by multiple fields

Number

• closeTo - test that values are close to a given value
• greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo - test ordering
• between - test that a value is between two given values

Text

• containsString, endsWith, startsWith - test string matching
• re - text matches a regular expression

Date

• dateAfter, dateAfterOrEqual, dateBefore, dateBeforeOrEqual - date comparisons
• dateBetween - test that a date is within a given range
• dateEqual - tests dates for equality

This allows for JUnit tests which are a lot more clearer in their intent. Ill give some examples of this below:

assertThat("chocolate chips", theBiscuit.getChocolateChipCount(), equalTo(10));
assertThat(someObject, notNullValue());



As you can see these are much clearer than the usual case of simply comparing to get a boolean result. In these cases it is much clearer as to the intent of the test even though it For example, the 'is()' matcher is a wrapper that doesn't add any extra behavior to the underlying matcher. The following assertions are all equivalent:


assertThat(theBiscuit, equalTo(myBiscuit));
assertThat(theBiscuit, is(equalTo(myBiscuit)));
assertThat(theBiscuit, is(myBiscuit));



The last form is allowed since is(T value) is overloaded to return is(equalTo(value)). These allow the test statements to 'read' like proper english.

Now Hamcrest has other features, including extensions to Matchers through your own customer matchers, and a bunch of other features (see links below). However, when I went to look at a JUnit test and I added some code to it to use these cool features, using the Netbeans functionality to go to source didnt work as there is no source attached to the code. What gives? Well I did the following.

1. Download hamcrest-all.jar and junit4.8.2.jar
2. Then first of all i customised my junit4.5 lirary. I went to sources and added from junit4.8.2 the directory temp.hamcrest.source to the sources for junit4.5
3. Now while this is fine for the Unit Matches and using Hamcrest in JUnit I was curious about ALL the Hamcrest code and so I also then went and created a new Library called Hamcrest1.1 and added the download jar as the path, source and javadoc as this contains all 3 (the .class files are ther alongside the .java) and then added this to my project which now allows me to play around with Hamcrest.

So with these changes I can now not only use Hamcrest with JUnit, but I can now go and browse to the source as well which I always want to be able to do as I love to be able to go digging around in the implementation and see others code and how tools I love work.

Links

Project -http://code.google.com/p/hamcrest/wiki/Tutorial
JUnit Release Notes - http://old.nabble.com/JUnit-4.4-released-td11677562.html
Tutorial on Hamcrest - http://github.com/drewbourne/hamcrest-as3