KWST4: An Experience Report

Slinking out from my seat and standing at the front of the room I felt I had left the relative safety afforded by the desks, arranged like so many trenches, rigid structures from behind which verbal artillery could be launched upon me - me, the blundering enemy, exposed and alone and defenceless in no man's land. So I gathered my armour and readied my weapons...

***

It's safe to say, I was a bit nervous about delivering an Experience Report (ER) at the fourth iteration of the Kiwi Workshop on Software Testing (KWST). An annual peer conference that invites a select crowd of testers from around New Zealand to discuss and debate a topic, this year it was how to speed up testing... and why we shouldn't, KWST is a place where reputations are on the line.

Of course, it was an honour to be invited. That I was meant that someone thought I had something useful to contribute to the conference, some experience relevant to the topic that others might learn from. This was great, because it meant that I had to try to live up to that expectation. I work really hard to be good at testing, and I saw this as another challenge to test myself against that goal.

Unfortunately I had to miss the first morning of the conference due to a prior commitment that couldn't be re-arranged, and so on walking into the SoftEd rooms where the event was hosted at lunch time, felt a little bit like the kid who joins school half way through term. With lunch over, the other attendees settled back into their seats confident and content, while I perched nervously, slightly unsure of the rules of this world into which I was now intruding.

I did know some of the rules though. KWST is organised as a peer conference, and each attendee prepares a 10-15 minute experience report relevant to the topic. Throughout the two days, a selection of the attendees are called upon to give their ER, and after each is a period of Open Season - a moderated discussion where the other attendees are invited to question and debate the report and the speaker to defend it.

First up on that afternoon was Thomas Recker, who spoke about some of his experiences where he was asked to introduce automation late in a project as if it were a silver bullet to speed up testing. He instead gave his opinions on some useful strategies that actually can make testing more efficient, a topic that extended into a lively Open Season discussion.

Next, Andrew Robins gave a fascinating talk on his experience in the complex domain of developing software for radios. He described how, through careful planning and preparation, he was able to identify and remove a number of environmental bottlenecks to help the testing of these products proceed more quickly than had ever been possible before.

This completed the ER roster for the first day, and upon finding out that I would be the third (and final) speaker called upon the next day if there was time, my nerves immediately doubled. The ERs I'd witnessed had been confident and quite technically focused, and the subsequent discussions had really demonstrated the level of expertise and knowledge accumulated by those in the room.

To close out the first day, we got into groups to complete a series of interview questions which would then be externally assessed by volunteer test managers around the world. Here, finally, was the confidence boost I badly needed. Working with Nigel Charman and Viktoriia Kuznetcova, getting my teeth stuck into some testing related challenges reassured me that I belonged here, and could readily deal with questions thrown my way.

As I fought the remonstrations of a body that routinely manages to forget that time actually exists before 8am on a Saturday, the second day of KWST4 was kicked off by Viktoriia Kuznetcova giving a nice ER detailing her strategies for having to complete days' worth of testing in just a few hours. She highlighted three main ideas that drove this - cluster, prioritise and parallelise - and I recommend having a read of her own KWST4 recap here for more detail on this.

This was followed by another great talk from Rachel Carson on her experience as her team transitioned to a more agile mindset and the impact that this had on their testing. While she noted that the way they did their testing didn't change a great deal, some of the surrounding processes and expectations did change in a way that allowed them to work faster and more effectively.

Again, both talks were followed by energetic periods of Open Season, with a lot of passionate debate and more than a few red cards - when someone just had to say something immediately - and blue cards - when committed debate took us a little too far off topic - were flaunted, before we broke for lunch and a chance to refresh and re-energise ahead of the last leg.

Upon returning from lunch the original plan had been to first recap the interview activity from the previous evening and then play some testing games, before closing out with a final ER - which would be mine - if there was time. However my good friend and colleague Katrina Clokie, wanting to drop me in it not wanting me to miss out on a great experience, suggested we do the ER first and then move onto the lighter stuff to finish the day. So up I got...

Slinking out from my seat and standing at the front of the room I felt I had left the relative safety afforded by the desks, arranged like so many trenches, rigid structures from behind which verbal artillery could be launched upon me - me, the blundering enemy, exposed and alone and defenceless in no man's land. So I gathered my armour and readied my weapons...

...but as I started talking, explaining my experiences, I realised that this was not war... it was sport. Those assembled did not want to kill me, to brutal me with their words and their questions; they wanted me to succeed, their verbal faints and jostles designed to test me and to push me to grow and succeed. Suddenly the nerves (mostly!) went away and I was having fun.

So I talked about my experience working in a highly complex, highly pressurised environment where we had been tasked with the impossible. I described the approaches we implemented - visual test modelling, exploratory testing structured using an adapted form of Session-Based Test Management, and co-located teams - to speed up not just our testing, but also the whole delivery cycle.

I highlighted the successes we had, but also focused on the problems we encountered; where the speed of change meant that while we delivered short term success, we didn't succeed in changing the mindset of the client, which had been my larger goal.

Then came Open Season, and a huge stack of topics that people wanted to discuss. But by this time I had found my stride, and enjoyed the rigorous questioning. Some people wanted to discuss how we took on the role of a BA by driving a shared vision of the solution via visual models, some wanted to discuss the role of auditors in challenging how we delivered our findings, some questioned the technicalities of how we implemented Session-Based Test Management.

What I found though, was that I could answer all of these questions.

While I'm sure not everyone would agree with my approaches and ideas 100%, I was confident in my own justifications and experiences, and in my deconstruction and analysis of those experiences. Venturing out into no man's land, opening myself up to challenge and critique, reminded me that I have the knowledge and, importantly, the language to discuss and debate my craft with a room full of skilled and experienced testers.

Discovering this, under the heat of interrogation, and returning to my seat amidst a flurry of (controversial) orange "like" or "+1" cards from people who agreed with a comment from one attendee that I had achieved significant success through my experience, was very rewarding. It meant that I could confidently take another step along my testing journey.

And I think that was what was really valuable about KWST4.

As a workshop, we didn't develop any ground-breaking new ideas or solve a significant problem, but as a community I think we expanded our horizons somewhat. I wasn't the only first-time attendee there, and I think that by encouraging new voices to test themselves we strengthen our community and allow each other to return to our day-to-day work of changing the testing world with renewed vigour and confidence.

And that's what good testers have to do. We have to test ourselves before we can lay any sort of claim to that accolade. We naturally question and challenge in our work, and so until we have the opportunity to adequately test our own belief and confidence, we won't be able to take ourselves as seriously as we need to in order to keep challenging the status quo.

My next challenges lie immediately ahead of me, as I will take to the stage to speak at Let's Test Oz and AgileNZ. Both of those talks currently terrify me, and that's just the way I like it. Because to be a great tester, I must test myself greatly.

Many thanks to all those who made KWST4 happen: Oliver Erlewein for organising, Rich Robinson for facilitating, SoftEd for providing the venue, food and refreshments, and Assurity for hosting the Friday night beers/pizza event.