The quiet virtue of thread-based testing

Scholars of testing, and especially context-driven testing, will be very familiar with session-based test management. In many ways this approach - affectionately known to many as SBTM or SBT - is the star child of the context-driven testing movement.

It's a well deserved reputation too - as a framework for exploratory testing that makes visible the structure and coverage of testing and enables the production of some valid metrics, SBT has literally helped many testers to tear up the script and ditch more traditional (not to mention more tedious) testing approaches.

Indeed, I know of its effectiveness in these contexts from personal experience - SBT formed a core part of a strategy that I used to help turn around a project that had stumbled off track and onto the front page a few years ago. So it's no wonder that I, and many others in the context-driven world, are such vocal celebrants of SBT's merits.

What I've come to be surprised by though, is the relative lack of recognition for SBT's little brother - thread-based testing (TBT).

Introduced by James Bach as a variant to SBT way back in 2010, you'll be hard pressed to find too much literature on the topic, or to witness anyone speaking about it at conferences. In fact, many of the top google search results for the term are variations upon "what is thread-based testing?"

For those not in the know, the answer to that question is that TBT is in some respects a leaner sibling of SBT - letting go of the timeboxes and chartered sessions, instead structuring testing via threads which are "a set of one or more activities intended to ... achieve an objective".

The relative anonymity of this leaner sibling initially surprised me, since I have come to lean far more heavily upon thread-based testing than session-based testing in my work, and I could safely say the same for many of the 50+ testers I work with at Trade Me. There's a quiet virtue to thread-based testing that I think is often overlooked thanks to the special heroics that SBT often performs.

The difference, of course, is context.

When I joined Trade Me, I got my first chance to work in an agile environment. Trade Me's tech delivery teams are "all-in" on agile, and are pretty mature in their agile mindset. While I'd been stealthily trying to introduce agile practices to stubbornly waterfall organisations for some years, it was a definite context-jump to be able to embrace it openly.

As well as this, Trade Me is purposefully context-driven in its approach to testing. But when I arrived and started chatting to testers, I noticed that while they had heard of SBT and were sort-of using it, they weren't quite getting full value from it - so I leaped into action and ran an internal workshop to help clarify the approach.

Unfortunately though, in doing so I actually fell into the common trap of failing to analyse the value of a practice in context - because I was blinded by its effectiveness in earlier contexts.

Had I waited a while and properly weighed up the situation, I would have recognised that SBT probably wasn't the right approach for the majority of testing projects at Trade Me - in the rapid agile environment, SBT was actually too structure-heavy. Instead, I realised, thread-based testing would probably split the difference nicely.

Indeed, what I came to realise was that the mangled-version of SBT I'd witnessed when I arrived was a result of testers who were trying to be context-driven feeling compelled to adopt SBT since it was the poster-child of that testing movement, but then discovering that it didn't really suit them and so starting to try to adapt it to better fit them.

They weren't even aware that thread-based testing existed as an alternative with a slightly more flexible structure, and so were struggling to throw off the shackles of SBT that they simply didn't need.

This highlights the huge importance of considering your context when deciding on testing practices.

Session-based testing gets talked about a lot because it is a fantastic tool for helping introduce more exploratory style testing in environments where structure and metrics are valued above all else; or where accountability and audit-ability of testing is paramount, such as testing mission-critical systems. And introducing context-driven testing in these sorts of places are the blockbuster achievements of the CDT movement - big wins we like to shout about because it represents progress in how people think about our craft.

On the other hand, thread-based testing flourishes more so in environments where that revelation has already occurred. Where the value of adaptability and flexibility in practices is already recognised and valued, and where formal structures for the sake of formal structures are recognised as a largely wasteful enterprise.

For that reason, it initially appears harder to justify shouting about.

And yet, I think that thread-based testing deserves a bit more adulation than that. Because what it can give you is a way of recognising the structures inherent in good exploratory testing that occurs within an effective agile team. Of communicating the coverage and progress of testing in a rapidly changing environment.

Indeed, at Trade Me I have spoken to a number of testers whose instinctive approach to testing very closely resembled thread-based testing, but they didn't really have the language with which to think about and communicate their work. When I introduced TBT, they suddenly were able to think about their testing with more clarity, and as a result were better able to involve other members of their cross-functional teams in testing.

And this leads me back to something James said in that first post introducing thread-based testing to the world:

Like many of my new ideas, it’s not really new. It’s the christening (with words) and sharpening (with analysis) of something many of us already do.

The real beauty here is not necessarily in the approach itself - since it's something that seems like simple common sense - but in arming testers with a more powerful way of organising their thinking when it comes to performing testing.

This can be especially useful in contexts where testers have happily abandoned highly formalised or cripted testing but find themselves at something of a loss when they are faced with complex and challenging testing projects. If, in such a context, session-based testing feels heavy handed, testers may find themselves unaware of an alternative reference for structuring their work. That's where thread-based testing can save the day.

Indeed, I've talked to, and seen, many testers wallow somewhat in loosey-goosey agile contexts - lost without the demand for structure and scripts, sometimes to the point that they revert right back to writing and executing traditional test cases within every sprint. These testers would benefit massively from an ability to apply a lighter structure to their testing.

So, ultimately, what I'm doing with this post is highlighting the quiet virtue of thread-based testing. It may not get the same level of limelight as its big brother, but it's an effective and powerful way of structuring exploratory testing in its own right - and it's about time it got its moment in the spotlight.