Testing is some value to someone who matters


I have a concern. We online testers have one thing in common: we care enough about our craft to take the time and read these blogs. That’s all very fine. However most testers, and this is based on my perception not research, most testers do not read blogs, or articles, or books or go to conferences, to workshops or follow a course. Well some of those testers do, but only when they think their (future) employer wants them to. And when they do they go out for a certificate that proofs they did so.

Becoming a tester

Regardless of where we start our carreer, be it in software engineering, on the business side or somewhere else, most testers start out with some kind of introductory test training. In the Netherlands, where I live, most of the time that means you’re getting a TMap, or sometimes an ISTQB training. And my presumption is that you get a similar message on what testing is everywhere:

  • Establishing or updating a test plan
  • Writing test cases (design; procedures; scripts; situation-action-expected result)
  • Define test conditions (functions; features; quality attributes; elements)
  • Define exit criteria (generic and specific conditions, agreed with stakeholders, to know when to stop testing)
  • Test execution (running a test to produce actual result; test log; defect administration)

But there are likely to be exceptions. For instance at the Florida Institute of Technology where Cem Kaner teaches testing.

Granted neither TMap nor ISTQB limit testing solely to this. For instance TMap starts of by defining testing as: “Activities to determine one or more attributes of a product, process or service” and up to here all goes well, but then they add “according to a specified procedure”. And there is where things start to go wrong. In essence the TMaps of the world hold the key to start you testing software seriously.  But instead of handing you down the knowledge and guide you to gather your own experiences they supply you with fixed roadmaps, procedures, process steps and artifacts. All of which are obviously easy to use for reproduction in a certification exam. And even this still could still be, as these methods so often promote, a good starting point to move on and develop your skills. Unfortunately for most newbies all support and coaching stops once they passed the exam. Sometimes even facing discouragement to look beyond what they have learned.


To make matters worse the continuing stance to make testing a serious profession has brought line managers, project managers, other team roles and customers the message that these procedures, processes and artifacts not only matter, but they are what testing is about. These items (supposedly) show the progress, result and quality of testing being undertaken. Line and process managers find it easy to accept these procedures, processes and artifacts as measurement items as they are similar to what they use themselves according to their standards. So if the measurements show that progress is being made and that all artifacts have been delivered they are pleased with the knowledge that testing is completed. Customers or end users go along in this but face limits in their belief of these measurements as they actually experience the end product. Like testers they are more aware that testing is never really ended and about the actual information about the product and not about testing artifacts.


New methods and approaches such as agile testing have brought the development roles closer together and have created a better understanding for the need and content of testing to both the team and the stakeholders. Other approaches, like context driven testing, focus more on enhancing the intellectually based craftsmanship of testing, emphasizing the need for effective and efficient communication, the influence of context on all of this and that software development is aimed at solving something. And thus the aim of testing is shifting from checking and finding defects to supplying information about the product. Regardless of this however and inspite of how much I adhere to these approaches I think they have a similar flaw as the traditional approaches. Like TMap or ISTQB neither of them go outside of their testing container enough to change the management and customer perception. They still let management measure testing by the same old standards of counting artifacts and looking at the calendar.


I think we as a profession should seek ways of changing the value of testing to our stakeholders. To make them understand that testing is not about the process or procedure by which it is executed nor about its artifacts, but about the information and value to achieve business goals it supplies.

I myself cannot give you a pret-a-porter solution so I challenge you, my peers, to discuss with me if you agree on this vision and if you do to form a new approach for this together. I will gladly facilitate such a discussion and deliver its (intermediate) results to a workshop or conference at some point.

#AgileTD (2)

Agile Testing Days

From 14 to 17 November 2011 The Agile Testing Days took place in Potsdam (D). Here is my second impression of my visit there.


I am an advocate of being well prepared when going to a conference. This enables me to  make informed choices of which tracks I really want to follow or not. This time I added two additional things to my preparation. First I did a poll on Twitter and LinkedIn. I wanted to know who else would be going to the conference and at what time they would arrive in Potsdam. I myself would be arriving on Sunday and wanted to meet other conference attendees. It always surprises my how well this works. On Sunday I met with Lisa Crispin and went out to have dinner with her and eight other testers, of which four were conference speakers. The second part of the preparation was more pragmatic. I wrote a number of blog posts an agile basics, gave a tutorial and a workshop at TestNet.

Surprise & Tutorial

Monday started with a quick breakfast and registration for the conference at which I got a bag with conference information, some tourist information (incl. a bottle of local beer) and also the program for Testing & Finance. A few months earlier I had entered a proposal for that conference and since I hadn’t gotten a reaction I was curious to see who, if not me, were on the program. To my surprise I found myself having a track at the end of day 1. A brief check learned that they had tried to reach me the day before at my home e-mail address that had not checked since I left home.

The rest of the Monday was filled with a one-day version of Jurgen Appelo’s Management 3.0 course. This course I can only wholeheartedly recommend to both managers and testers alike. Besides the necessary theory, the course also is punctuated with reading tips and practical exercises. My personal take-away from this tutorial is the use of complexity theory and the notion that, despite of the fact that all models are fallible, several weaker models can be as effective and as a strong model, and certainly no better model. I will dig into that at some time soon and combine that with Jerry Weinberg’s books on Systems Thinking.

Some of the highlights of the track days

Keynote by Johanna Rothman – “Agile Testers and test managers”
In the keynote, the changing roles of testers and test managers were discussed. For example, testers will need to cooperate more intensively with developers. Test managers should be leaders in the organization and pursue the following key activities: Monitoring of the project portfolio; Removing organizational obstructions, Create confidential relationships, Leading the hiring process, Increasing the capacity of the organization and finally the Start up communities of practice. I liked the keynote enough to be following her tutorial at the Belgium Testing Days in March 2012.

Track by David Evans – “What testers and developers can learn from each other”
This track showed that testers and developers, while working on the same product, see this with a different perspective. Testers often seem more capable of changing perspective. By being able to do so testers can learn developers that there are different kinds of tests. A good model for showing this is the “Agile Testing Quadrants” as defined by Crispin and Gregory from the book Agile Testing. But I a will keep further description short as you can see the whole presentation on YouTube at, http://skillsmatter.com/podcast/agile-scrum/what-testers-and-developers-can-learn-from-each-other

Track by Rob Lambert – “Do agile testers have wider awareness fields”
This track went in to the need to be aware of your context and to use this awareness to your benefit as a tester. Perhaps it is the process or maybe it is the people? Or is the awareness field of agile testers is not wider at all? Agile testers however seem to display a higher ability to feel, to perceive, to know and to be aware of themselves and the world around them. Traditional testers often seem to have a more limited consciousness in terms of testing, and development roles. Even if it is wider, it is often less versatile than in an agile environment. There is however a distinction between social (situational) awareness and personal awareness. One reason for the difference in perspective among other things, is that the focus of testing in a traditional development environment is narrower than the focus of testing in an agile environment. A greater awareness and a broader focus, often leads to an increase in choices. This allows you to choose one of the possible paths instead of chosing a prescribed path. A greater awareness is also a first step on the path of change. However you can not follow all paths. It is necessary to have sufficient self-knowledge and to know your limits. More on this in this prezo.

Track by Huib Schoots – “So you think you can test”
Huib actually wasn’t on the initial program of the Agile Testing Days. But a few of the presenters were ill and since Huib happened to have his laptop with this presentation on it at the venue he offered to pitch in. The organization graciously  accepted his offer and Huib made his first appearance at an international conference. As the title suggests his talk was on what makes a good tester and how to become one. I really enjoyed his talk but rather than to describe it here I am going to point out a series of columns on this Huib is writing.

Keynote by Liz Keogh – “Haiku, Hypnosis and Discovery: How the mind makes models”
Liz put an extraordinary exercise into her keynote. She let the audience pair up to create Haiku’s. Together with Johanna Rothman and I came up with the following sentences that we combined to the following Haiku:

Foggy breath
An agile journey
Bright blue burst over the rocks

Or a more famous one from Matsuo Basho:
Furuike ya                  Old pond
Kawazu Tobikomu     Frog jumps in
Mizu no oto                The sound of water

Liz continued with a hypnosis session to explain that concentrated and focussed attention on positive experiences can bring a state of mind that widens perception and activates the ability to see patterns and models. Huib Schoots volunteered to go on stage and be hypnotized. It was impressive to see how more than half of the audience participate and was elevated by the experience.

Keynote van Gojko Adzic – “5 key challenges for agile testers tomorrow”
Gojko concluded the track days with an inspiring keynote talk on five challenges agile testers are facing:

#1 Shorter delivery phases
#2 Agile is now main stream
#3 Faster feedback
#4 Large “enterprise” projects
#5 Validating business, not software

His final message was to adopt principles, adapt practices, teach each other how to test, help business to define and validate actionable metrics, visualize risk value areas and to draw up contexts to inform testing. This definitively struck a chord with me. As I am working at a large enterprise transitioning to agile. I can fully understand that the energetic and all present Gojko won the MIATPP Award 2011 as “The most influential Agile Testing Professional Person 2011”

Final day

The final day was a series of parallel sessions with Open Space, Coding Dojo, Testing Dojo and last but certainly not least TestLab. To be honest I was both to actively involved and tired after the previous days to take sufficient notes. But what I can share with you that these possibilities to actively use what you have learned, to spar with your peers and to be coached by the organizers and speakers that are there makes this part of the conference potentially the most valuable part.