DEWT 2 – Drinks, dinner, more drinks and lots and lots of talk (a.k.a. day 1)

I was very pleased to be part of the 2nd DEWT workshop. This Peer conference took place on October 5 and 6 at Hotel Bergse Bossen in Driebergen, the Netherlands. The Dutch Exploratory Workshop on Testing (DEWT) is a workshop that falls into the series of peer workshops on testing like LAWST, LEWT, SWET and GATE. The main theme of this workshop was:

 Experience Reports: Implementing Context-Driven Testing.

Friday evening started with gathering in the grand cafe, dinner and a lightning talk by myself in the conference room. I had planned and prepared the lightning talk to be about the summarized content of my EuroSTAR conference talk “Seven Questions to Help You on the Path of Testing”.

A talk that I have also entered for the next Dutch Testing Conference. The Dutch Testing Conference has a process of sending review comments on your abstract and one of the remarks in the review, a couple of days before, made me change my mind about the lightning talk. That particular remark reminded me that the context driven approach to testing is far away from being common knowledge in the software testing world.

I wondered how we communicate our view on testing and why it is that we are not really effective in bringing the message of context driven testing across?

The section below describes what my thoughts were about the content of my lightning talk mixed and enhanced with parts of the discussions arose after the discussion and later that evening when we were having drinks and the next day during the conference.

Since I started to see myself as a follower of the context driven testing (CDT) approach I have seen many a tweet and blog pass by proclaiming that if you follow context driven testing results will be much better than if you are using the old school / traditional / factory / waterfall / IEEE/ ISTQB process approaches to testing. And roughly parallel with my start of being context driven I also started to work in an agile environment. And like CDT many agilists proclaim superiority on traditional or waterfall approaches of software development. And sure enough if applied consciously and thoughtfully both approaches often yield better results than the more process oriented ‘traditional’ approaches. But even so I believe both followers of CDT and agile are making a few fundamental mistakes in their communication.

First of all the agile manifesto and the context driven testing principles have been around for over ten years now and at that time it probably was a good idea to react against the then main stream ideas of software testing. However a lot of time has passed since then and most people who had (or still have) the same feeling towards the ‘old’ approaches have changed to following the ideas of CDT and / or agile by now. So this rhetoric will have no effect on them anymore. Well maybe it could create some kind of group identity, but that can hardly be the purpose in my opinion.

Second I wonder how many people actually work in a true traditional / waterfall assignment. Personally I admit to having worked in an environment based on so-called waterfall process principles, but even then I saw myself as being in a true waterfall assignment. And in that particular case hardly anybody fully committed to doing waterfall (or to be more specific TMap) by the book. And obviously this underlines the flaw of such process and deliverable oriented approaches, but it also shows that a lot of people in such an environment do not commit to the approach as such. So trying to sway them to follow your approach because their approach is ‘bad’ might look appealing, but kind of misses the point as a lot of people in that environment will feel you are not talking about them. And even if they feel addressed by your comments how will this make them change. For the better part they already agreed with you so what your telling is either no news or not adding helpful information.

Finally, and this is for me personally the main conclusion of the discussions. Being context driven is not about being against something. It is about being a (self) educated, thoughtful, skillful, open to your context testing craftsman. A craftsman that is likely to engage in to an open discourse with his peers and team mates, who’s drive it is to learn what is necessary to do the job and then learn some more just for the fun of it. As Joris Meerts aptly pointed out “Yes that does make us elitist.” For unfortunately there are not that may software testers that invest in maintaining or even extending their software testing (and general) skills, let alone seeking to collaborate with others while they are doing that.

Even after one and a half day of conferring the question on how to bring the context driven testing message and mindset across is still not answered. So let me know if you come up with useful ideas and I will collect them and come back to the subject on our next DEWT peer conference (or any time sooner when we meet).

Standing left to right:
Ray Oei, Jean-Paul Varwijk, Adrian Canlon, Markus Gartner (Germany), Ruud Cox, Joris Meerts, Pascal Dufour, Philip Hoeben, Gerard Drijfhout, Bryan Bakker, Derk Jan de Grood, Joep Schuurkes, Lilian Nijboer, Philip-Jan Bosch, Jeroen Rosink, Jeanne Hofmans
Kneeling left to right:
Tony Bruce (UK), Zeger van Hese (Belgium), Ilari Henrik Aegerter (Switserland), Huib Schoots, Peter Simon Schrijver


Exercising agility

Sprint 0 – The idea

While getting in the mood for the Agile Testing Days 2011 in Potsdam I remembered  seeing a XP exercise sometime before. Essence of the exercise was to use the agile manifesto, its principles and to link them to development practices. This really felt as a great practical extension to my previous two posts on agile. Only trouble was that it had forgotten who wrote it so the exact steps were unknown to me. Also I remembered that it originally targeted an XP development audience and  that I felt that it needed some adjustment to be useful for (non-coding) testers.

At the same time I was getting ready to participate in the TestNet workgroup theme night with the Agile Testing workgroup. While e-mailing with the workgroup chair, Cecile Davis, I offered to use a recreation of the exercise suitable for testers and to translate the manifesto and principles into Dutch along the way.

Sprint 1 – Proof of Concept

At the following meet the other workgroup members liked my idea and draft version of the exercise. So at the TestNet workgroup night on November 8 I have presented and executed the exercise in front of some 30 people.

All in all the execution of the exercise was a success and I received positive feedback. Several participants were interested in doing the exercise with their team. But personally to be honest I quickly saw after the introduction that in spite of the good idea the six exercise steps I had written were not going to work with such a large group. So the exercise was saved by improvising, getting some goodwill and by offering additional explanation during and after the execution. Also I discovered that trying to keep it inside the period of an hour was a real challenge.

In good agile tradition I concluded the evening with a retrospective. Combining feedback from the participants with my own feedback I put new items on the (virtual) backlog. First one was to write this post, second to do a re-write of the exercise in general and thirdly to make an adaptation of the exercise steps in order to make them suitable both for smaller and larger groups. Both the content and the ‘new’ exercise steps of the exercise will be part of the following sections. The Dutch version, containing I believe the first complete translation of both the agile manifesto and the principles, will be placed on the TestNet ‘Agile Testen’ workgroup page [TBD].

Sprint 2 – The exercise (published version)


  • Large separately printed sheets of the agile manifesto and the twelve principles
  • As many  A4 / Letter print outs with the practices on them as you have participants
  • Post-its or similar
  • Markers
  • Introductory presentation about the agile manifesto, the principles and the practices
  • Know what you’re talking about
  • A room with possibility to form groups, walk around, and hang the print outs on to the wall.

Step 1:
Give a (short) presentation on the Agile Manifesto and its principles.
Depending on the agile experience of your audience this can be shorter or longer. Check my previous two posts for to get additional information or inspiration.

Step 1B (small groups):
Group members discuss what the principles mean to them and how they map to the Agile Manifesto.
Depending on the time you have and on the familiarity of your audience with the Agile Manifesto you can spent longer or shorter amounts of time on this. In larger groups this tends to take a lot of time and is probably best added into the step 1 presentation or left out all together.

Step 2:
Present the practices and handout a hardcopy version of them to all participants.
Depending on the experience of your audience (and the time you have) you can provide information about all of the practices (takes a lot of time), or let the audience indicate which ones need to be elaborated or skip explanation al together.

Step 3:
Have the group go up to all of the principles and have them discuss  and map the practices that they think fitting to the principle. Mapping should be visualised by writing the practice on to a post-it and sticking it to the top of the sheet if it is a good fit, to the bottom of the sheet if it only fits partially.
The group should form a consensus on the which practices fit and to what extend. The aim here is to learn by actively linking practices to agile principles by discussion and argumentation. When necessary the facilitator should provide additional information and elaboration about the practices. 

Step 3B (Large groups):
Divide the audience into smaller groups of no more than 6-8 people. Then execute step 3 by letting them go by all principles from a different starting point.
To aid the visual result you can hang multiple versions of each principle (as many as you create groups) next to each other and differentiate the groups choices in this way.

Step 4 (Large groups only):
Have each group pick one or two of the principles and let them explain why they chose those practices and why they placed them higher of lower on the sheet. The other groups should be able to ask questions.
This step is more about sharing information and reasoning then that it is about argumentation and justification. A single group will have done this automatically during the exercise.

Final step:
Have all members evaluate what they have learned and taken away from this exercise. One of the take aways should be to pick one or more practices that they are going to actively work on during the next period. Then execute a (quick) retrospective on the execution of the exercise.
Depending on the group size and time you have left this can be down by letting each member express this to the group or to let them do this individually later.

The agile manifesto, the principles and the practices

To conclude this is what it is all about. Starting of with the practices:


A practice is something that has proven to be valuable in a certain context and offer insight into solutions that may or may not work in your situation.

You can do this with the list below for a start. But preferably create you own shorter, longer or more suitable list. Basically you can do this with any kind of practices not just test related practices.

  • Manage risk
  • Execute your project in iterations
  • Embrace and manage change
  • Measure progress objectively and understandably
  • Test your own test cases
  • Leverage test automation
  • Team change management
  • Everyone can test (and owns quality)
  • Understand the domain
  • Describe test cases from the user perspective
  • Manage versions
  • Co-locate
  • Leverage patterns
  • Actively promote re-use
  • Rightsize your process
  • Continuously reevaluate what you do
  • Test Driven Development
  • Concurrent Testing
  • Pair Testing
  • Specification by example
  • Acceptance Test Driven Development
  • Plan sustainably
  • Stand-up meeting
  • Plan in relative units
  • Configuration management
  • Learn by doing
  • Whole team approach
  • Shared Vision
  • Use Case Driven Development
  • Risk Based Testing
  • Evolutionary (Test) Design
  • …..

The agile manifesto

We are uncovering better way of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The principles:

  • Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
  • Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
  • Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.
  • Business people and developers must work together daily throughout the project.
  • Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
  • The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
  • Working software is the primary measure of progress.
  • Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
  • Simplicity–the art of maximizing the amount of work not done–is essential.
  • Continuous attention to technical excellence and good design enhances agility.
  • The best architectures, requirements, and designs emerge from self-organizing teams.
  • At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

Remembered who the original exercise was from: David A. Koontz