JAW Speak

Jonathan Andrew Wolter

Archive for the ‘thoughtworks’ Category

Improving developers enthusiasm for unit tests, using bubble charts

with 11 comments

Reading time: 4 – 6 minutes

The Idea
Visualize your team’s code to get people enthused and more responsible for unit testing. See how commits contribute to or degrade testability. See massive, untested checkins from some developers, and encourage them to change their behavior for frequent, well tested commits.

Several years ago I worked with Miško Hevery, who came up with an unreleased prototype to visualize commits on a bubble chart. The prototype didn’t make it as an open source project, and Misko’s been so busy I don’t know what happened to it. I was talking to him recently, and we think there are lots of good ideas in it that other people may want to use. We’d like to share these ideas, in hopes it interests someone in the open source community.
testability-bubbles-overview
This is a standard Flex Bubble Chart. This was a rich flash widget with filtering, sorting, and other UI controls.

  • Each bubble is a commit
  • The X axis is a timeline
  • The size of bubbles is the size of each checkin
  • The colors of bubbles are unique per developer
  • The Y axis represents the ratio of (test code / total code) changed in that checkin.

Size is measured in lines of code, an imperfect measure, but it proved useful for his internal use.

Interpreting It
We found that showing this to a few people generally got a big interest. All the developers wanted to see their bubbles. People would remember certain commits they have made, and relive or dispute with each other why certain ones had no test code committed.

testability-bubbles-big commits

Best of all, though, when used regularly, it can encourage better behavior, and expose bad behavior.

testability-bubbles-frequency

Using It
Developers often leave patterns. When reading code, I’ve often thought “Ahh, this code looks like so-and-so wrote it.” I look it up, and sure enough I’ve recognized their style. When you have many people checking in code, you often have subtly different styles. And some add technical debt to the overall codebase. This tool is a hard to dispute visual aid to encourage better style (developer testing).

Many retrospectives I lead start with a timeline onto which people place sticky notes of positive, negative, and puzzling events. This bubble chart can be projected on the wall, and milestones can be annotated. It is even more effective if you also annotating merges, refactorings, and when stories were completed.

If you add filtering per user story and/or cyclometric complexity, this can be a client friendly way to justifying to business people a big refactoring, why they need to pay down tech debt, or why a story was so costly.

While you must be careful with a tool like this, you can give it to managers for a mission control style view of what is going on. But beware of letting anyone fall into the trap of thinking people with more commits are doing more work. Frequent commits is probably a good sign, but with a tool like this, developers must continue to communicate so there are no misunderstandings or erroneous conclusions made by non-technical managers.

Some developers may claim a big code change with no test coverage was a refactoring, however even refactorings change tests in passing (Eclipse/Intellij takes care of the updates even if the developer is not applying scrutiny). Thus the real cause for the large commit with no tests deserves investigation.

Enhancements
Many other features existed in the real version, and a new implementation could add additional innovations to explore and communicate what is happening on your project.
testability-bubbles-enhancements

  • Filter by user
  • Filter by size or ratio to highlight the most suspicious commits
  • Filter by path in the commit tree
  • Showing each bubble as a pie chart of different file types, and each of their respective tests.
  • Display trend line per team or per area of code
  • Use complexity or test coverage metrics instead of lines of code
  • Add merge and refactoring commit visualizations
  • Color coding commits to stories, and adding sorting and filters to identify stories with high suspected tech debt.
  • Tie in bug fixes and trace them back to the original commits’ bubbles

We hope some of you also have created similar visualizations, or can add ideas to where you would use something like this. Thanks also to Paul Hammant for inspiration and suggestions in this article.

Written by Jonathan

July 16th, 2011 at 7:35 pm

Book Review: The McKinsey Mind. Focusing on Mutually Exclusive Collectively Exhaustive thinking

with 3 comments

Reading time: 4 – 6 minutes

The McKinsey Mind: Understanding and Implementing the Problem-Solving Tools and Management Techniques of the World's Top Strategic Consulting FirmThe McKinsey Mind: Understanding and Implementing the Problem-Solving Tools and Management Techniques of the World’s Top Strategic Consulting Firm by Ethan M. Rasiel
My rating: 4 of 5 stars

Worth reading. Especially if you are in consulting. I like the beginning of the book especially, and will be turning back to some of those pages for reference.

Thinking logically
The book starts strong by introducing “MECE: Mutually Exclusive Collectively Exhaustive.” I use it often in my teams. Think of it as building a decision tree, where you cover every option, and none are overlapping. Each branch in the tree also has more MECE sub-branches. When deciding or investigating something, draw the tree on a whiteboard, and walk everyone through the options, and sub-tree options until you have a decision, or clear actions to take. Very logical. Be sure to encourage other people to contribute to the branches, and if you are leading it, ideally you team will volunteer the branches, and then they have more commitment to the options.

Or, as Wikipedia says:

[MECE] says that when data from a category is desired to be broken into subcategories, the choice of subcategories should be
1. collectively exhaustive — i.e., the set of all subcategories, taken together, should fully characterize the larger category of which the data are part (“no gaps”)
2. mutually exclusive — i.e., no subcategory should represent any other subcategory (“no overlaps”)

This is desirable for the purpose of analysis: mutual exclusivity avoids the risk of double counting information, and collective exhaustion avoids the risk of overlooking information.

Two areas for MECE thinking are in logic-trees and issue trees.

Logic trees help you identify components of a problem. Start at the 20,000 foot view and move progressively downward. You may want to build multiple trees, for instance by business unit (organizational hierarchy) and functionally (production, sales, marketing, etc.) to see which leads you to the next step, the hypothesis.

Form a hypothesis of what component of the logic tree may be causing the problem. Run it by the Quick and Dirty Test: ask what assumptions you are making that must be true. Are any false? If it passes the QDT, gather data and do analysis to disprove it. This is the same as the scientific method. If you fail to disprove it, you may be on to something. Predict what could happen if the identified root cause was changed.

Issue trees let you rigorously test the hypothesis. They are different from logic trees. Logic trees are a hierarchical grouping of elements. Issue trees are the series of questions or issues that must be addressed to support or disprove a hypothesis. It becomes you roadmap for analysis.

Presenting
Present with the conclusion at the start. This was a good lesson. Do not use inductive reasoning to build up from details into a specific conclusion for your audience. They may already agree with it. You would then waste their time. Instead make you conclusion, and progressively drill into details, broadly covering each level before drilling down further. Stop/skip forward if they do not need the convincing. (A refresher on inductive/deductive reasoning).

Presentations are all about getting buy-in. It is important to “pre-wire” the meeting so that there are no surprises, and people already know your conclusions. The act of pre-wiring will identify gaps you need to work on, or build allies for you proposal.

Interviewing clients/stakeholders is a common activity I have done at ThoughtWorks. The authors advised scheduling time with people, and sending them an agenda. This lowers their apprehension of why consultants want to talk to them. Also at the end, during small talk before you walk out, ask “is there any thing else we did not cover that you think we should?” You’ve built rapport by now. Their answer can uncover important, previously unmentioned issues.

Note: as a software engineer, I appreciate the MECE thinking style for its logic. Also, it is the exact way we approach performance tuning an application. Think about the logic tree of slow spots. Build a hypothesis. Test it by profiling the running program under load. Let the data of time spent in each component show the root cause.

View all my reviews

Written by Jonathan

May 1st, 2011 at 3:14 am

Sukrupa in Bangalore, an AMAZING organization, and ThoughtWorks

without comments

Reading time: 3 – 5 minutes

I’ve been living in Bangalore since October. First training at ThoughtWorks University (TWU), now working for a UK client. The people and work has been amazing. Best of all is what we have done with Sukrupa.

Six years ago next month I was in Bangalore, volunteering with the non profit Sukrupa, a slum school founded by the amazing Krupathala Martin Dass. [1] And in January this year ThoughtWorks began working for the school in rebuilding their website and launching a student records project. I’m very proud for ThoughtWorks for investing with many dozen people to work on this, and for the honor to enable such a wonderful, high impact activity. This is part of the culture here. What a great place.

Below is a write up from TWU XX, which completed in February, and compiled by my colleague Pallavi from Australia.

TWU XX project for Sukrupa went live Feb 17 – Release 1 consisted of a redesign of the Sukrupa website www.sukrupa.org and a Student Record System.

Who is Sukrupa?
Sukrupa is a Bangalore-based NGO that works to transform the lives of children from the slums and integrate them into mainstream society. Heard enough and want to help? You can donate or explore volunteer opportunities at Sukrupa.

What did TWU-XX do?

  • We met the most amazingly talented children, who made us realize that every child deserves an opportunity to shine no matter where they come from.
  • We revamped and rebranded www.sukrupa.org to showcase the great work done by Sukrupa.
  • We enabled online donations and are looking for ThoughtWorkers to QA our online donation widget. Any volunteers? ;-)
  • We created an offline donations guide that offers the donor a variety of options to maximize donation.
  • We produced an events summary to help track the many events hosted by Sukrupa (events are an avenue to showcase the talents of the children, and also to increase visibility with sponsors/donors).
  • We are proud to say that we successfully delivered software in a one-click-deploy environment!
  • We got completely upstaged by the Sukrupa children in ‘TWU’s Got Talent (or not!)’, witnessed by Roy, Trevor, Dan and Dave. Stay tuned for embarrassing videos and pictures.

In Closing

  • Sukrupa needs your help! Please donate and/or volunteer!
  • This project with Sukrupa will continue with future TWU batches.
  • ThoughtWorks’ involvement with Sukrupa is set to grow, with Rohit exploring other opportunities for us to help – Thank you Rohit.
  • TWU XX has been an incredibly rewarding experience. Not only did it prepare us to be consultants but it also enabled us to carry forward ThoughtWorks values in our very first project.
  • A big thank you to our trainers and clients for their endless patience.
  • Thank you to the Bangalore office for hosting us and particularly the Admin Team.

My backstory on Sukrupa
[1] Back in 2005, a friend and I were on a five week project to have an art camp with several non profits in India. We attempted to film a documentary and capture children’s answer to “what do I want to be when I grow up.” We met with many non profits and schools in Chennai and Bangalore. You can read the old blog here, ArtView India. Sukrupa was one the organization that left a deep impact on me and I returned in 2007, as well as have worked with Krupa from the US in grant writing and publicity. I’ve been directly involved since 2010, connecting her with ThoughtWorks.

Sukrupa children

Written by Jonathan

April 18th, 2011 at 2:49 pm

Posted in career, thoughtworks

Some Problems with Branch Based Development, And Recommendations

with one comment

Reading time: 3 – 4 minutes

As I have previously written, I dislike branch based development (especially when it involves subversion and long lived feature branches). Sometimes projects have multiple teams concurrently working in the same codebase, each on different release schedules. For example, team “Delta Force” will go live in a year, with some super top secret and amazing functionality. (Do not get me started on how bad an idea this usually is. I believe in short, frequent releases and rapid customer feedback). All the while, team “Alpha Squadron” is working on periodic releases into production every few weeks or months. Nothing from Delta can go live with Alpha’s releases. How do you enable these teams to cooperate? Some may suggest a long lived feature branch for Delta Force’s code. And then “just” merge the changes from Alpha down to Delta. I believe this is a Bad Idea.

But, if you’re forced into this scenario, at least explain to people the importance of frequently merging (i.e. on a daily basis). The longer you wait between merges, the greater the danger of increasing complexity and merge difficulties.
why-branch-and-merge-is-bad-1

Rather than using a long lived branch, I suggest the following solution: Trunk Based Development. This means you do not create actual branches. Everyone is on the same trunk. As my colleague Paul Hammant phrases it, you Branch By Abstraction. My other colleague Martin Fowler calls this Feature Toggles. First, this enables refactoring, because everyone is working in one codebase. Second, as feature toggles become permanently turned on, you can remove them, and the conceptual divergence between the two “branches” drops. This is a Good Thing.

why-branch-and-merge-is-bad-2

Once again, if you have a long lived branch, and even if you frequently merge, the divergence will grow as the not-reintegrated changes accumulate in the long lived branch. This divergence is RISK. It inhibits refactoring and encourages technical debt.

Prefer Trunk Based Development. It is not perfect, as there still is additional complexity in the codebase. But you can mitigate this with polymorphism instead of if conditionals, and have multiple continuous integration pipelines for all deployable feature toggle combinations.

why-branch-and-merge-is-bad-3
By using Trunk based development, we make it easier to do the right thing. This is an example of the Boy Scout Rule. The Boy Scouts of America have a simple rule that we can apply to our profession. “Leave the campground cleaner than you found it.”

If we all checked-in our code a little cleaner than when we checked it out, the code simply could not rot. The cleanup doesn’t have to be something big. Change one variable name for the better, break up one function that’s a little too large, eliminate one small bit of duplication, clean up one if statement.

Long lived feature branches make this difficult, because any changes you make to one branch needs to be replicated, or you may have merge difficulties. This is especially a problem for a “long lived receiving” branch, which by it’s nature does not reintegrate its changes into the mainline. (Thus that branch is limited in what it can refactor).

Written by Jonathan

November 7th, 2010 at 8:00 am

Subversion Parallel Multi-Branch Development And Merging

with 4 comments

Reading time: 2 – 2 minutes

As discussed in my previous post, I dislike merging-based-development, preferring Trunk Based Development instead. But, sometimes you’re stuck with a long-lived development branch, and you need to merge changes (subversion tree-conflicts and all). At the end of the post, I have several scripts I used to make this easier. Not the prettiest, but saved a lot of pain when we had major refactorings in trunk, and needed to locate and merge the changes to those files in a long lived (read: horrible) dev branch.

Imagine this scenario: Multiple streams of development, with a long-lived “3.0 dev” branch that has never reintegrated with the trunk. (Because 3.0 has new features that won’t go into production for many months).

branch-and-merge-problems-1

There are substantial dangers in this approach. This diagram only touches on the surface of the areas of risk in which a merge could fail. Solution? Trunk based development / branch by abstraction.

branch-and-merge-problems-2

Given this required scenario, I developed a few best practices and scripts for merging. The best practices involved having multiple branches checked out into different directories. And then we would find equivalent files that have moved and merge the tree-conflicts.

Scripts to assist in Subversion 3 way merging.

Custom diff3-cmd configuration setting in svn:

Written by Jonathan

November 3rd, 2010 at 4:21 am

The Management Myth: Management Consulting Past, Present & Largely BogusThe Management Myth: Management Consulting Past, Present & Largely Bogus – Book Review

with one comment

Reading time: 5 – 8 minutes

The Management Myth: Management Consulting Past, Present & Largely BogusThe Management Myth: Management Consulting Past, Present & Largely Bogus by Matthew Stewart

My rating: 5 of 5 stars

A highly entertaining read where Matthew Stewart dismantles the Management Consulting industry. For you with experience in consulting, parts of this are hilarious as Stewart chronicles both the imploding of the consultancy he helped create, and the overall history of the industry.

One by one, he tears apart Fredrick Taylor (the father of “scientific management”), Elton Mayo (of the “famous” Hawthorn Effect), Management Consultants, Strategy (as a science), popular Management Gurus, and offers a harsh critique of MBA’s in general. His main premises is that “Management” is not a profession on the level with Medicine or Law, however management education through business schools have attempted to professionalize it. And to that end, they have manufactured “science” (in a very non-rigorous and untestable sense), truisms, frameworks, and case studies.

Here are a few highlights. I didn’t agree with all that he had to say, but I did enjoy reading it. I was laughing out loud towards the end as he details how his fellow partners are embroiled in litigation with him as he tries to sever all ties. Stewart is genuinely funny, giving characters nicknames such as “The Prince of Darkness”, “The Troll,” “Dr. Bob” the corporate shrink, and others.

As for management consultants, he had less-than-flattering quotes:

“[C]onsultants often serve not to provide new knowledge to their clients but merely to communicate ideas already formed. In many instances, our work amounted to harnessing work performed in one part of an organization and then packaging it all as our own work for the benefit of another part of the same organization.”

I can’t speak for all organizations, but I am extraordinarily fortunate to say at ThoughtWorks (my employer) this has never been my case. We are not a traditional management consulting McKinsey style strategy consulting firm, true. However, we do have consulting projects, some the strategic management consulting types. I’ve seen us bring our outside expertise and influence and avoid acting solely as grease in the wheels. Becoming grease may let others bill lots of money, but isn’t very intellectually fulfilling.

Even better, if you have worked in a traditional pyramid style company, (of which ThoughtWorks is very, very, very, very much not) he has this gem to explain it:

“It’s like being stuck in a dungeon with a bunch of rats and a giant block of cheese. All the rats keep climbing the cheese, two years at level one, two years at level two. The threes shit on the twos and the twos shit on the ones, and everyone shits all the time on the rats at the bottom. All they care about is rat-face-time. As in, please-sir-would-you-stick-your-rodent-butt-closer-to-my-face time. You keep going up until one of the other rats bites your ass off.”

Up or out. Several friends of mine elsewhere have shared this is a fairly accurate description.

Strategy. He also tears apart Porter and his Five Forces [3]. Stewart claims all business strategists describe strategy in hindsight (not so useful if you want to implement “strategy” for, you know… the future). I have a little bit of a hard time accepting that all of modern strategy is hogwash, as instead I think while bounded in utility, different frameworks help one to position a problem and look for solutions in diverse ways.

Due to business schools’ roots in Taylor, Mayo and others that he sequentially defuses of all credibility, the author also suggests that the fundamental underpinning of MBA’s are shaky. The academic and scientific rigor is weak, and the content is easily grasped by otherwise intelligent people.

“After 100 years of fruitless attempts to produce such a discipline, it should be clear that [Business Management] does not exist. preparing managers to manage, in fact, is not different from preparing people to live in a civilized world. Managers to not need to be trained; they need to be educated. And for that purpose, although a certain amount of study of business-related subjects may prove useful, the business schools as they are presently constituted are at best superfluous.”

A jaded view? Yes. But also very fun. The best parts is the parallel narrative that progresses through the book about his firm. I don’t want to spoil much, but it is especially fun when his firm’s acquiring company’s CEO steps down because he wishes to promote full time his beliefs in UFO’s and alien-human technology transfer. Serious. [4] This guy had a crazy consulting journey, and has a great style of writing about it.

If you are interested in this book, first read his article on The Atlantic [1], which gives you a taste of the content here. Second, check out the WSJ book review [2], by Philip Broughton, author of a similar book that I recently reviewed. Also check out the New Yorker’s review [5].

[1] http://www.theatlantic.com/magazine/arch…
[2] http://online.wsj.com/article/SB10001424…
[3] Michael Porter, of Harvard. Father of the 5 Forces, which are: (1) the bargaining power of suppliers, (2) the bargaining power of buyers, (3) the rivalry among existing firms, (4) the threat of new entrants, and (5) the threat of substitute products. Extremely influential, he also advocated all strategy aims for a single, measurable goal: excess profits. (Unsurprisingly, Stewart takes criticism of this).
[4] http://articles.sfgate.com/1999-01-09/ne…
[5] http://www.newyorker.com/arts/cri…

And, please share with me your comments.

View all my reviews

Written by Jonathan

October 31st, 2010 at 5:03 am

Less Hate with Maven Part 2: The Wrapper Script

without comments

Reading time: < 1 minute

I previously wrote about useful debugging techniques with maven. Our maven builds have become complex, with Branch By Abstraction, and about 40 devs working simultaneously on the codebase in 2 continents. We have at least 3 profiles for each of the branch abstractions that are currently running in the codebase. I’m one of the tech leads, and in order to keep the team’s build consistent and easy to remember, we have a wrapper script (thanks to Cosmin).

Here it is:

Written by Jonathan

May 28th, 2010 at 9:03 am

Posted in automation, code, thoughtworks

Tagged with

Hotspot caused exceptions to lose their stack traces in production – and the fix

with 5 comments

Reading time: 3 – 4 minutes

Today I was working with Kumar and Elian where we encountered production logs with dozens of NullPointerExceptions with no stack trace. We identified that the JIT compiler will optimize away stack traces in certain exceptions if they happen enough. The code below reproduces it. We are on jdk 1.6 (I don’t remember the minor version now).

public class NpeThief {
    public void callManyNPEInLoop() {
        for (int i = 0; i &lt; 100000; i++) {
            try {
                ((Object)null).getClass();
            } catch (Exception e) {
                // This will switch from 2 to 0 (indicating our problem is happening)
                System.out.println(e.getStackTrace().length);
            }
        }
    }
    public static void main(String ... args) {
        NpeThief thief = new NpeThief();
        thief.callManyNPEInLoop();
    }
}

Run it as follows and the issue will appear (the stack trace length changes from 2 to 0):
javac NpeThief.java && java -classpath . NpeThief
javac NpeThief.java && java -server -classpath . NpeThief

How to fix it? The following options resolve it and it stays at 2, never going to 0 as length of NPE’s stack trace:
javac NpeThief.java && java -XX:-OmitStackTraceInFastThrow -server -classpath . NpeThief
javac NpeThief.java && java -XX:-OmitStackTraceInFastThrow -classpath . NpeThief
javac NpeThief.java && java -Xint -classpath . NpeThief
javac NpeThief.java && java -Xint -server -classpath . NpeThief

So the solution is to start with -XX:-OmitStackTraceInFastThrow argument to java which instructs the JIT to remove this optimization [1,2], or operate in interpreted only mode [3]. What is going on? [2]

The JVM flag -XX:-OmitStackTraceInFastThrow disables the performance optimization of the JVM for this use case. If this flag is set, the stacktrace will be available. (editor: or if the -Xint is set).

“The compiler in the server VM now provides correct stack backtraces for all “cold” built-in exceptions. For performance purposes, when such an exception is thrown a few times, the method may be recompiled. After recompilation, the compiler may choose a faster tactic using preallocated exceptions that do not provide a stack trace. To disable completely the use of preallocated exceptions, use this new flag: -XX:-OmitStackTraceInFastThrow.” http://java.sun.com/j2se/1.5.0/relnotes.html

How does this impact performance? I did extremely naive timing and time java [args] -classpath . NpeThief. It behaved as expected, with interpreted the slowest. Is that the solution?

No. We weren’t going to change production JVM options to resolve this, instead since our isolated example code above indicated that initial exceptions would throw with the full stack trace, we went back into older logs and grepped. Sure enough, right after we deployed there were exceptions appearing with the full stack trace. That gives us enough information where we can identify and fix the bug.

Notes:
[1] Related sun bug 4292742 NullPointerException with no stack trace
[2] Helpful StackOverflow discussion of NullPointerException missing stack traces
[3] -Xint: Operate in interpreted-only mode. Compilation to native code is disabled, and all bytecodes are executed by the interpreter. The performance benefits offered by the Java HotSpot VMs’ adaptive compiler will not be present in this mode.

Written by Jonathan

May 26th, 2010 at 12:46 am

Posted in code, java, thoughtworks

12 Tips for Less* Hate of Maven

with 4 comments

Reading time: 3 – 5 minutes

We’re (stuck*) using maven on a project, and this means often the build doesn’t act like you’re expecting. Debugging requires knowledge about maven’s capabilities and less documented features. Here are some tips, please add more in the comments.

  1. The -X flag shows debug logging, and tee it to a file which you then search for clues of what happened. This is great for verifying if a plugin actually ran, and what it did. Example: mvn -X install | tee out.txt
  2. mvn dependency:tree (show the transitive dependencies of your dependencies) [1]
  3. -Dtest=MyJavaTest (just use the name of the class, no package necessary)
  4. mvn help:system (show environmental variables and system properties) [2]
  5. mvn help:active-profiles (show the active profiles to be run)
  6. mvn help:all-profiles (show all profiles maven reads)
  7. mvn help:effective-pom (show the logical pom.xml being used)
  8. mvn help:effective-settings (show the logical settings.xml which is being used)
  9. mvn help:describe -DgroupId=org.apache.maven.plugins -DartifactId=maven-resources-plugin -Ddetail (show details about a plugin’s options)
  10. -DskipTests this will skip running tests, but still compile it. or -Dmaven.test.skip=true which will skip compilation and execution of tests (use with judgment, I believe in testing)
  11. If -X doesn’t show you the properties when you needed them, you can also use the ant-run plugin to echo a property value (does anyone else know a better way to do this in maven? Tell me what a property value is at the time of the plugin or goal executing?)
  12. Run your tests paused in debug mode, waiting for a debugger to attach to port 5005 with mvn -Dmaven.surefire.debug test [4]

References:

*Note: Initially I disliked the confusion and long, slow learning curve with maven. After working 8 months with maven and my colleagues Lucas Ward and Paul Hammant, I’m feeling much more comfortable and am generally okay with it. For small open source work, where you want a project quickly, but don’t want to spend a few hours with ant, I like it very much. For our large project, maven is fundamentally broken. Too much file copying is required, and manual intervention on a complex build is impossible, or extremely tedious. Running with our own build script we control could be dramatically more flexible.

On a mailing list Paul recently recommended a few other to consider:

Maven Dependency plugin
mvn dependency:tree
- shows me the transitive deps (say, “how the hell is MX4J being dragged into the build ?”)
mvn dependency:analyze
- show me unused, but declared dependencies.
There’s a ton of free capability to that plugin – http://maven.apache.org/plugins/maven-dependency-plugin/

An Atlassian dependency plugin
Atlassian wrote a plugin ( http://confluence.atlassian.com/display/DEV/FedEx+XI+-+Keeping+track+of+you+maven+dependencies ) that guards against undesired changing dependencies.

Maven Enforcer Plugin
A more intrusive way of guarding against deps that, say, should not make it into a war.  http://maven.apache.org/plugins/maven-enforcer-plugin/

What tips do you have? Please share them in the comments.

Written by Jonathan

May 14th, 2010 at 8:13 am

Posted in code, java, thoughtworks

Tagged with ,

Injecting HttpServletResponse into Spring MVC request scoped interceptors

without comments

Reading time: 3 – 5 minutes

I’ve been heavily refactoring a large team’s codebase that stuffed many critical dependencies into request attributes. You see request.getAttribute() throughout the code, which is counter to dependency injection’s principles. Instead of getting what you want, be given it to you. We are moving to a more idiomatic way of using spring to inject things such as our custom Session object.

However, there was a lifecycle mismatch between some singleton interceptors which needed the session object, which is only request scoped. We had everything working, except one of those interceptors also needed the response object as a collaborator.

I had a spring instantiated non-controller bean that is request scoped, and I want to inject in two collaborators:

HttpServletRequest (which can be injected)
HttpServletResponse (which fails injection)

Here is my bean:
<bean id="sessionFactory" scope="request">
<aop:scoped-proxy/> <!-- scoped-proxy to get around the singleton requests injection of a request scoped object problem -->
</bean>

Here are the autowired fields:
@Autowired HttpServletRequest request;
@Autowired HttpServletResponse response;

Can I inject the response object in this bean?

This is the error I was getting:
[INFO] Caused by: org.springframework.beans.factory.BeanCreationExce ption: Error creating bean with name ’scopedTarget.sessionFactory’: Autowiring of fields failed; nested exception is org.springframework.beans.factory.BeanCreationExce ption: Could not autowire field: javax.servlet.http.HttpServletResponse com.example.web.SessionRequiring$SessionFactory.re sponse; nested exception is org.springframework.beans.factory.NoSuchBeanDefini tionException: No unique bean of type [javax.servlet.http.HttpServletResponse] is defined: Unsatisfied dependency of type [interface javax.servlet.http.HttpServletResponse]: expected at least 1 matching bean
[INFO] at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessAfterIn stantiation(AutowiredAnnotationBeanPostProcessor.j ava:243)

I navigated to GenericWebApplicationContext, which delegates to WebApplicationContextUtils.registerWebApplicationS copes(beanFactory), which then registers ServletRequest.class and HttpSession.class as injectable. But not the response object.

So here’s what I did, which works for me:

In web.xml add a filter that will store every ServletResponse in a threadlocal, which will be accessible only from your FactoryBean.

  <filter>
        <filter-name>responseInScopeFilter</filter-name>
        <filter-class>org.springframework.web.filter.DelegatingFilterProxy</filter-class>
        <init-param>
            <param-name>targetBeanName</param-name>
            <param-value>responseInScopeFilter</param-value>
        </init-param>
  </filter>
  <filter-mapping>
        <filter-name>responseInScopeFilter</filter-name>
        <url-pattern>/*</url-pattern>
  </filter-mapping>

Create the filter:

    /**
     * We need a filter to capture the HttpServletResponse object such that it can be injectable elsewhere in request scope.
     */
   public static class ResponseInScopeFilter implements Filter {
        // not the most elegant, but our spring commiter friends suggested this way.
        private ThreadLocal<HttpServletResponse> responses = new ThreadLocal<HttpServletResponse>();
        public void init(FilterConfig filterConfig) throws ServletException { }

        public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain chain) throws IOException, ServletException {
            HttpServletResponse response = (HttpServletResponse) servletResponse;
            responses.set(response);
            chain.doFilter(servletRequest, servletResponse);
            responses.remove();
        }

        /** Only to be used by the BeanFactory */
        private HttpServletResponse getHttpServletResponse() {
            return responses.get();
        }

        public void destroy() { }
    }

And also create the FactoryBean:

/** We need our own FactoryBean because Spring does not offer a way to
     * inject request scoped HttpServletResponse objects.
     * See http://forum.springsource.org/showthread.php?p=298179#post298179 */
    public static class HttpServletResponseFactoryBean implements FactoryBean {
        @Autowired ResponseInScopeFilter responseInScopeFilter;

        public Object getObject() throws Exception {
            return responseInScopeFilter.getHttpServletResponse();
        }

        public Class getObjectType() {
            return HttpServletResponse.class;
        }

        public boolean isSingleton() {
            return false;
        }
    }

Wire it all together in applicationContext.xml

<bean id="httpServletResponse" class="com.example.web.SessionRequiring$HttpServletResponseFactoryBean" scope="request"/>
<bean id="responseInScopeFilter" class="com.example.web.SessionRequiring$ResponseInScopeFilter"/>

Elsewhere I can now inject an HttpServletResponse object and it will be request scoped appropriately for this request. You can read more about this in my post on the spring forums.

Written by Jonathan

May 6th, 2010 at 9:32 pm

Posted in code, java, thoughtworks