JAW Speak

Jonathan Andrew Wolter

Archive for the ‘testability’ Category

Improving developers enthusiasm for unit tests, using bubble charts

with 12 comments

Reading time: 4 – 6 minutes

The Idea
Visualize your team’s code to get people enthused and more responsible for unit testing. See how commits contribute to or degrade testability. See massive, untested checkins from some developers, and encourage them to change their behavior for frequent, well tested commits.

Several years ago I worked with Miško Hevery, who came up with an unreleased prototype to visualize commits on a bubble chart. The prototype didn’t make it as an open source project, and Misko’s been so busy I don’t know what happened to it. I was talking to him recently, and we think there are lots of good ideas in it that other people may want to use. We’d like to share these ideas, in hopes it interests someone in the open source community.
This is a standard Flex Bubble Chart. This was a rich flash widget with filtering, sorting, and other UI controls.

  • Each bubble is a commit
  • The X axis is a timeline
  • The size of bubbles is the size of each checkin
  • The colors of bubbles are unique per developer
  • The Y axis represents the ratio of (test code / total code) changed in that checkin.

Size is measured in lines of code, an imperfect measure, but it proved useful for his internal use.

Interpreting It
We found that showing this to a few people generally got a big interest. All the developers wanted to see their bubbles. People would remember certain commits they have made, and relive or dispute with each other why certain ones had no test code committed.

testability-bubbles-big commits

Best of all, though, when used regularly, it can encourage better behavior, and expose bad behavior.


Using It
Developers often leave patterns. When reading code, I’ve often thought “Ahh, this code looks like so-and-so wrote it.” I look it up, and sure enough I’ve recognized their style. When you have many people checking in code, you often have subtly different styles. And some add technical debt to the overall codebase. This tool is a hard to dispute visual aid to encourage better style (developer testing).

Many retrospectives I lead start with a timeline onto which people place sticky notes of positive, negative, and puzzling events. This bubble chart can be projected on the wall, and milestones can be annotated. It is even more effective if you also annotating merges, refactorings, and when stories were completed.

If you add filtering per user story and/or cyclometric complexity, this can be a client friendly way to justifying to business people a big refactoring, why they need to pay down tech debt, or why a story was so costly.

While you must be careful with a tool like this, you can give it to managers for a mission control style view of what is going on. But beware of letting anyone fall into the trap of thinking people with more commits are doing more work. Frequent commits is probably a good sign, but with a tool like this, developers must continue to communicate so there are no misunderstandings or erroneous conclusions made by non-technical managers.

Some developers may claim a big code change with no test coverage was a refactoring, however even refactorings change tests in passing (Eclipse/Intellij takes care of the updates even if the developer is not applying scrutiny). Thus the real cause for the large commit with no tests deserves investigation.

Many other features existed in the real version, and a new implementation could add additional innovations to explore and communicate what is happening on your project.

  • Filter by user
  • Filter by size or ratio to highlight the most suspicious commits
  • Filter by path in the commit tree
  • Showing each bubble as a pie chart of different file types, and each of their respective tests.
  • Display trend line per team or per area of code
  • Use complexity or test coverage metrics instead of lines of code
  • Add merge and refactoring commit visualizations
  • Color coding commits to stories, and adding sorting and filters to identify stories with high suspected tech debt.
  • Tie in bug fixes and trace them back to the original commits’ bubbles

We hope some of you also have created similar visualizations, or can add ideas to where you would use something like this. Thanks also to Paul Hammant for inspiration and suggestions in this article.

Written by Jonathan

July 16th, 2011 at 7:35 pm

Fast and Easily Testable GWT JUnit Tests

with one comment

Reading time: 4 – 6 minutes

The principle we were trying to follow last November during a short Google Web Toolkit project was to test our controllers as much as possible outside of GWTTestCase. (It’s too slow). So that means never instantiate or reference a GWT widget in the controller. They must always be decoupled by interfaces. Here’s some of what we learned. Formerly my controller had an async event handler in it that showed a pop up message on a failure.
private AsyncCallback asyncCallback = new AsyncCallback() {
  public void onFailure(Throwable caught) {
    Window.alert("An error occured, please try again. \n" + caught.getMessage());
  public void onSuccess(PackageDto pkg) {
    firePackageUpdate(pkg); // throw an event in here that is handled by a
                            // listener, which is the gui code and which has no logic
                            // (logic is in the controller!)
But since I want to run my controllers with regular JUnit, I can’t have the Window.alert call in there. If I did and ran a junit test, it would get this stack trace:
Caused by: java.lang.UnsupportedOperationException: ERROR: GWT.create() is only usable in client code!
   It cannot be called, for example, from server code.  If you are running a unit test, check that your test
   case extends GWTTestCase and that GWT.create() is not called from within an initializer or constructor.
at com.google.gwt.core.client.GWT.create(GWT.java:91)
at com.google.gwt.user.client.Window.(Window.java:230)
... 28 more
I saw two initial solutions to removing the Window.alert(), and then the third recommended solution:
  1. Implement the AsyncCallback<T> interface with BaseAsyncCallback<T> , and implement the onFailure() to call Window.alert(). In tests, I would subclass and override onFailure() in tests, preventing the Window.alert() from executing. I generally don’t like subclassing for tests, so I wasn’t too keen on this.
  2. Implement the AsyncCallback<T> interface with a class BaseAsyncCallback<T>, but this time take in a constructor parameter, FailureHandler (which would be an interface or class of my own.) Then in onFailure(), delegate to the FailureHandler field. In tests, pass a different FailureHandler implementation or subclass, to avoid calling the GWT widget code.
  3. The recommended solution is to treat this the same way I deal with onSuccess(). Tell a listener that a failure event occurred, and let it handle it.
What is key is the notifyAboutFailure() method takes a String of the message to the user (possibly using internationalization along the way). So anything interesting (error processing) still happens on the Controller side. Right now our view can handle the error by calling Window.alert(), but later when we implement the nicer “flash or gmail yellow bar” style UI, that can be dropped in (and tested with GWTTestCase or Selenium).
Principle to remember:
  • Separate the Controllers from all references to Views (GWT UI objects) by using event listener / notifier interfaces.
You also should read:

Written by Jonathan

July 30th, 2009 at 1:05 am

Posted in code, java, testability

Test Driven log4j Logging Code Example

with 3 comments

Reading time: 3 – 4 minutes

Update: Added a snippet so that you can easily assert a message was contained in the logged messages.

Frequently logging is not viewed as “important” by engineers as, say, the logic for refunding a customer’s purchase. The customer’s won’t mind, after all, if we forget to log something right? Maybe so, until there are problems and the operational team supporting your application has very little ability to diagnose problems.

I propose logging should be viewed as another user interface, and thus needs its’ own set of acceptance and unit tests. This makes for more tedious and up front work for development teams — however I want to survive in production without a pager going off, and with fewer late night scrambles to ship patches.

Imagine a class that has logic associated with logging. Below you will see a unit test verifying logging statements are correct. Often logging uses statics, and as Misko has said, statics are a death to testabilty. With Guice you can easily inject loggers automatically for the class under construction. This method gets around most of the issues with static loggers, although I still despise statics everywhere.

This is a basic example, but the point is to understand how to hook into appenders and add/change one to use a test appender.

package com.jawspeak.common.testing;
import com.google.common.collect.Lists;
import org.apache.log4j.AppenderSkeleton;
import org.apache.log4j.spi.LoggingEvent;
import java.util.List;
* Use me for spying on test logs and making asserts against them.
* Example how to use:
*        final SpyLoggerAppenderForTesting spyAppender = new SpyAppenderForTesting();
*        final Logger logger = Logger.getLogger(MyClass.class.getName());
*        try {
*            logger.addAppender(spyAppender);
*            // do something
*            assertEquals(Lists.newArrayList("Attempted to read such-and-such, but could not: oops"), spyAppender.getMessagesLogged());
*        } finally {
*            logger.removeAppender(spyAppender); // clean up global state
*        }
public class SpyLoggingAppenderForTesting extends AppenderSkeleton {
  private List messages = Lists.newArrayList();
  protected void append(LoggingEvent loggingEvent) {
  public List getMessagesLogged() {
    return messages;
  public boolean doMessagesContain(String snippet) {
      boolean isFound = false;
      for (String message : messages) {
          if (message.indexOf(snippet) >= 0) {
              isFound = true;
      return isFound;
  public void close() { }
  public boolean requiresLayout() { return false; }

Here’s an example test where we make use of it. Note that we didn’t assert the lists are exactly the same, because in this project we were also using aspects to do logging, and depending on if you ran with or without the aspects enabled, a different number of logging messages would be created.

package com.jawspeak.common.utility;
import com.google.common.collect.Maps;
import com.jawspeak.common.testing.SpyLoggingAppenderForTesting;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.junit.After;
import static org.junit.Assert.*;
import org.junit.Before;
import org.junit.Test;
import java.util.LinkedHashMap;
import java.util.List;
public class SessionSizeLoggingTest {
private SpyLoggingAppenderForTesting spyAppender = new SpyLoggingAppenderForTesting();
private Logger logger = Logger.getLogger(SessionSizeLoggingTest.class.getName());
private Level oldLoggingLevel = logger.getLevel();
private SessionSizeLogging sessionSizeLogging = new SessionSizeLogging();
  public void setUp() {
    logger.setLevel(Level.TRACE); // set the level in case a log4j.properties or log4j.xml disables this level of logging
  public void tearDown() {
  public void logBytesWhenUnprintable() throws Exception {
    LinkedHashMap map = Maps.newLinkedHashMap();
    map.put("key", new String(new byte[] { 1, 31, 127, -97}, "ISO-8859-1"));
    String sessionId = "abc123";
    sessionSizeLogging.logSessionSize(logger, sessionId, map);
    List messagesLogged = spyAppender.getMessagesLogged();
    assertTrue(spyAppender.doMessagesContain("Session Size for session id: " + sessionId + " (approx. total " + (7 + 3 + 1 + 4 + 1) + " bytes)"));
     // or use
    assertTrue(messagesLogged.contains("  key[   3 bytes]                            key  -->  value[   4 bytes] '\\u0001\\u0031\\u0127\\u0159'"));
  // .. many more tests

This is a trick that I don’t see written about too much, but I recommend it often. And, I’d love to have some readers post enhancements, links, and take testability further with respect to logging.

Written by Jonathan

June 21st, 2009 at 8:17 pm

Posted in code, java, testability

Tagged with , ,

Wrestling the Untestable Into Testable Code: Example with Test Driven SimpleTagSupport for custom JSP Tag

with one comment

Reading time: 3 – 5 minutes

Let’s say app servers must be stateless. There is no affinity for a user pinned to a particular app server, so two requests will likely be handled by different Tomcats, Jetty’s, etc. This means nothing can go in HttpSession (we also can’t use clustering). And when it’s ecommerce, we can’t require cookies or that might cost millions in lost revenue. Given these constraints, you have one valid option left: custom url rewriting over all our links (GET’s) and with hidden form fields (POST’s).

You can edit every link and every form to pass some unique id around. But what if you have two or three id’s that need to pass around for legacy reasons? Sounds like a custom tag would be nice. Something you can put in your jsp’s (oh yeah, let’s say we’re using jsp’s too).

This introduces me to the testability challenge. Write your own tag extending SimpleTagSupport that would ensure parameters were always in a link, and write another tag to write out the three hidden input fields inside every form.

But, SimpleTagSupport is part of the good old servlet and jsp API, so it’s very bloated with context objects, deep inheritance, callbacks, and cruft that makes for difficult unit testing. Just look at the public interface, one method: public void doTag() throws JspException, IOException.

Here’s my test driven test case (with 100% coverage, through the public API), which takes advantage of Spring MVC’s MockHttpServletRequest and MockHttpServletResponse objects (which are really fakes). I’m also using Mockito, which I prefer over EasyMock and JMock.

package com.jawspeak.dotcom.tag;
import static org.junit.Assert.*;
import org.junit.Before;
import org.junit.Test;
import static org.mockito.Mockito.*;
import org.mockito.invocation.InvocationOnMock;
import org.mockito.stubbing.Answer;
import org.springframework.mock.web.MockHttpServletRequest;
import org.springframework.mock.web.MockHttpServletResponse;
import org.springframework.mock.web.MockPageContext;
import javax.servlet.jsp.tagext.JspFragment;
import java.io.StringWriter;
import java.io.Writer;
public class StatefulLinkTagTest_SettingSessionId {
	private StatefulLinkTag statefulLinkTag = new StatefulLinkTag();
	private MockHttpServletRequest request = new MockHttpServletRequest();
	private MockHttpServletResponse response = new MockHttpServletResponse();
	private static final String SESSION_ID = "ADFHE13";
	public void given() throws Exception {
		request.setAttribute("sid", SESSION_ID);
		statefulLinkTag.setJspContext(new MockPageContext(null, request, response));
		JspFragment jspBodyFragment = mock(JspFragment.class);
		doAnswer(new FakeJspBodyAnswerer()).when(jspBodyFragment)
                        .invoke((Writer) anyObject());
	public void shouldRenderLinkWhenThereIsNoParameter() throws Exception {
		String contentAsString = response.getContentAsString();
		assertEquals("&lt;a href=\"/myPage.html?sid=" + 
                     SESSION_ID + "\"&gt;link text body&lt;/a&gt;", contentAsString);
	public void shouldRenderLinkWhenThereIsAlreadyAParameter() throws Exception {
		String contentAsString = response.getContentAsString();
		assertEquals("&lt;a href=\"/myPage.html?my_param=foo&amp;sid=" + 
                    SESSION_ID + "\"&gt;link text body&lt;/a&gt;", contentAsString);
	/** Mockito Answer implementation to manipulate the parameters we pass 
	 * into JspFragment.invoke() due to the ugly servlet API. */
	private static class FakeJspBodyAnswerer implements Answer {
		public Object answer(InvocationOnMock invocationOnMock) throws Throwable {
			StringWriter writer = (StringWriter) invocationOnMock.getArguments()[0];
			writer.write("link text body");
			return null;

As for the implementation, it’s pretty straightforward after writing the tests. It doesn’t show covering multiple ID’s appended to the link, but it is now easy to test drive implementing them.

package com.jawspeak.dotcom.tag;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.jsp.JspException;
import javax.servlet.jsp.PageContext;
import javax.servlet.jsp.tagext.SimpleTagSupport;
import java.io.IOException;
import java.io.StringWriter;
public class StatefulLinkTag extends SimpleTagSupport {
	private String href;
	public void setHref(String href) {
		this.href = href;
	public void doTag() throws JspException, IOException {
		PageContext pageContext = (PageContext) getJspContext();
                      (HttpServletRequest) pageContext.getRequest()));
	private String createTagText(HttpServletRequest request)
                       throws IOException, JspException {
		StringWriter stringWriter = new StringWriter();
		stringWriter.append("<a href=\"");
		if (href.indexOf("?") > 0) {
		} else {
		return stringWriter.toString();

Questions, comments? Want to bash Java and tell me why I should be using Ruby, Erlang or Scala? For more about the SimpleTagSupport interface check here.

Written by Jonathan

March 24th, 2009 at 9:06 pm

Using (These) Anonymous Inner Classes is Probably Too Clever for Your Own Good

with 5 comments

Reading time: 3 – 5 minutes

Sometimes it is tempting to have less verbose Java code and be clever. Say you have an object you need to build and set several properties on it, then use it. This way looks clever, but is a bad idea – especially in production code. Read on for why.

  	TripChoice myTrip = new TripChoice() {{
                        // look at these clever initialization blocks!
			setDestination("San Francisco");
			setAvailableRoutes(new HashSet() {{

This is a huge problem because it can cause a NotSerializableException or create a memory leak. How? Because with that clever anonymous class and initialization block, there is an implicit reference to the outer class’ instance. That outer class will not be garbage collected so long as there is one reference to the anonymous class. Worse, if we ever

Here’s the typical way to do it.

                // blah blah, this is typical, and a rather boring looking great big block of text.
		// (That's also probably scattered over several places in your code when you build these objects).
		TripChoice myTrip = new TripChoice();
		myTrip.setDestination("San Francisco");
		Set availableRoutes = new HashSet();

Here’s why this improved version is even not the best:

  • It uses setters. Setters allow your object to be constructed in an inconsistent state. There are some situations setters are fine (i.e. form backing objects) but often your code is more clear without setters. In this particular case the setters aren’t egregious, but I still consider them a smell and much more verbose way that constructor injection. Read more about the problem with setter injection here
  • It is verbose.
  • It is also mutable (often a negative), again a problem due to the use of setters.

So what’s the best way to do it? I have found constructor injection and in our particular example use of Google Collections. As for seeing an implementation of the best solution – just leave a comment.

Below is a full example you can run that illustrates the hard to catch serialization bug.

package com.jawspeak;
import com.google.common.collect.Lists;
import static org.junit.Assert.*;
import org.junit.Test;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.NotSerializableException;
import java.io.ObjectOutputStream;
import java.util.ArrayList;
import java.util.List;
 * Understands the danger of anonymous subclasses for lists.
public class TooCleverForOurOwnGoodTest {
        @Test(expected = NotSerializableException.class)  // UPDATE: Thanks for the reminder from Dennis, below
	public void serializableDangerBugFromPatricksCleverness() throws IOException {
		// you think a List would be serializable, which it is. And so are Strings.
		final String myString = "3";
		List strings = new ArrayList() {
			// However, because there is an anonymous subclass of ArrayList created,
			// there is an implicit reference to the outer class, which is **not serializable**.
			// This is a nice feature, which lets us have closure-ish syntax in java, such as
			// the following reference to the outer myString.
			add(myString); // It will throw the exception even without this reference.
		// call your method that requires args to be serializable
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		ObjectOutputStream oos = new ObjectOutputStream(baos);
		// The particular bug I had was even worse, because we were creating leaking a reference
		// through that List's implicit reference to the outer object.
	public void serializableDangerBugSafeButNotClever() throws IOException {
		// A List is serialiazable. And so are Strings.
		List strings = new ArrayList();
		// No longer is there a reference to the outer class.
		// call your method that requires args to be serializable
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		ObjectOutputStream oos = new ObjectOutputStream(baos);
	public void serializableDangerBugSafeAndNiceLooking() throws IOException {
		// A List is serialiazable. And so are Strings.
		List strings = Lists.newArrayList("one", "two");
		// call your method that requires args to be serializable
		ByteArrayOutputStream baos = new ByteArrayOutputStream();
		ObjectOutputStream oos = new ObjectOutputStream(baos);

Written by Jonathan

March 10th, 2009 at 11:33 pm

Posted in Uncategorized, code, java, testability

Tagged with , ,

One Unit Test should have Prevented Google from Categorizing the Entire Internet as Malware

with 14 comments

Reading time: 2 – 3 minutes

The google wide massive glitch which this morning, categorized nearly every search result as malware could have been prevented by unit testing. This is a wonderful example why even “silly little scripts” should be test driven.

This is what happened. A file was checked in with a ‘/’ in it. This file listed some or all of the sites Google warns to be dangerous malware sites. When it rolled out to different data centers it it caused search results to be rendered with warning messages next to almost every search result.

One could have written unit tests to assert that typos (such as ‘/’) are not parsed, accidentally causing the entire web to be miscategorized. I really hope to read more about this on the Google Testing Blog, it’s a prime example for this company to take leadership and further promote Unit Testing, and even the opportunity for Test Driven Development. Miško, care to take that post up?

They handled it well, and Marissa Mayer made a post on the Official Google Blog:

We periodically update that list and released one such update to the site this morning. Unfortunately (and here’s the human error), the URL of ‘/’ was mistakenly checked in as a value to the file and ‘/’ expands to all URLs.

I say let’s use this as a publicity moment for testing. I am certain that testing has literally prevented many disasters in complex systems, however because the tests did their work — they got no publicity. Let’s see promotion of “lessons learned,” Google!

Where have you been saved by tests? Or, where was your big blunder that a test could have fixed?

Written by Jonathan

January 31st, 2009 at 4:30 pm

Posted in code, testability

Linking to Miško’s, Russ’ and my Testability Guide

with 3 comments

Reading time: 4 – 6 minutes

I had the great pleasure of collaborating with Miško Hevery and Russ Rufer in creating the Guide for Writing Testable Code. Please feel free to check it out, and leave comments here or on his blog.

Here’s a peek at the different flaws to watch out for, and warn your co-workers about. If you see this in code that is getting checked in, bring it to the attention to your team. See Miško’s post for the full list.

Flaw #1: Constructor does Real Work (link)

  • If you have a constructor that is doing “work” you’ve established a contract that everyone who wants to create this object also is forced to wait for that “work” to happen. This becomes a huge problem in small unit tests. Frequently unit tests create many, many instances of the object under test – and any work you do in the constructor slows down tests.
  • The solution? Break up the responsibility into two objects: a builder of factory to do all the heavy lifting to construct the object, and whatever the original object was that had the heavy constructor. When you split the responsibilities up you will have a better object oriented design, and make it easy for more flexible construction. If you’re tempted to create an init() method and put the work in there – avoid this siren song. It’s a smell of mixed responsibilities and a poorly behaving object that isn’t really ready when the constructor completes.
  • Warning signs are:
    • new keyword in a constructor or at field declaration
    • Static method calls in a constructor or at field declaration
    • Anything more than field assignment in constructors
    • Object not fully initialized after the constructor finishes (watch out for initialize methods)
    • Control flow (conditional or looping logic) in a constructor
    • Code does complex object graph construction inside a constructor rather than using a factory or builder
    • Adding or using an initialization block

Flaw #2: Digging into Collaborators (link)

  • Jumping from one object to another to another to get what you want makes for hard to test and hard to refactor code. This is commonly viewed as the Law of Demeter violation, however you can dig into collaborators without breaking the letter of the law. This is hard to work with in tests, because your tests need to do a great deal of setup stuffing objects in objects into other objects, or else you’ll get null pointers.
  • Warning signs are:
    • Objects are passed in but never used directly (only used to get access to other objects)
    • Law of Demeter violation: method call chain walks an object graph with more than one dot (.)
    • Suspicious names: context, environment, principal, container, or manager

Flaw #3: Brittle Global State & Singletons (link)

  • Global state (usually made possible through the static keyword in Java) creates hard to test and modify code. Parallelizing tests is often impossible, and tests that forget to clean up the global state after running interact undesirably with other tests (causing unexpected failures).
  • Warning signs are:
    • Adding or using singletons
    • Adding or using static fields or static methods
    • Adding or using static initialization blocks
    • Adding or using registries
    • Adding or using service locators

Flaw #4: Class Does Too Much (link)

  • Object oriented design allows you to split responsibilities into individually tested objects. Using an object oriented language does not prevent people from writing procedural code. In fact, most times when someone does not want to write tests it is because their code is hard to test.
  • Warning Signs:
    • Summing up what the class does includes the word “and”
    • Class would be challenging for new team members to read and quickly “get it”
    • Class has fields that are only used in some methods
    • Class has static methods that only operate on parameters

There’s typically no magic wand to wave and make hard to test code instantly testable. Engineers need to have an open mind and learn how to write code designed for testability. (In my experience 90% of the time this involves test driven design and test driven development.) These ideas above, and many of Miško’s other posts are a fantastic starting point for adopting that new way of thinking.

Please link it up with your favorite testability mindset readings.

Written by Jonathan

November 29th, 2008 at 5:26 pm