Seven questions – What questions do I have?

The previous two questions helped you to find why testing is necessary, what information you need to answer the first question (business value) and which test ideas help you deliver meaningful and relevant information. This post now extends this to areas that help you identify the circumstances in which you will have to do your work. It ends with a little advice that you should not take things for granted especially if you do not understand them.

DID-A-TEST

Originally called Jean-Paul’s test this mnemonic represents a set of surveying questions that helps you identify working conditions. Once you have the answers to these questions you should check if and if so how this influences your ability to test and the ability to give more or less rich information to your stakeholders. You can use these questions to identify  boundaries and constraints to your testing possibilities and address them or at least be and make others aware of them. These questions are by no means exhaustive, but in my opinion they form a good starting point in exploring your test context.

Are the Developers available?

Developers are physically close of far from you. They are more or less available in time or more or less organizationally accessible to testers. The ability or inability to work together with development can influence your risk assessments, your insight into risk areas, your knowledge about development solutions and what is or is not covered by development testing activities. Additionally when addressing developers it is good to know the preferences and willingness of each developer with regard to working with testers.

How soon do you have access to Information?

Of course you can use the FEW HICCUPPS mnemonic (James Bach, Michael Bolton) to improve and expand your test ideas, but gathering information about the intended product or solution is a main starting point and important reference to work with. So getting access to the sources of information or even better being involved in the information gathering should start as soon as possible.

Do you control the test Data?

My interpretation of test data here is wide in the sense that I do not only mean the ability to enter different types of inputs, in different variations and quantities. I also mean the ability to set up and load data sets creating test scenarios. And the ability to set or remove states in the software. Being able to control the data is beneficial in speeding up test execution, creating typical test situations and helps to quickly repeat the test case if necessary.

Having control of the test data is only one side of the story. The other side of the story is that you need to find the right ‘Trigger Data‘ to use. Trigger Data is any data item, set of data or data state specifically created and used to invoke, enable or execute your test case (scenario).

Are the Analysts available?

Like the developers the availability, both physical and in time, of the (business) analysts has an impact on the way you can interact with them. And like the developers analysts will have preferences and are more or less willing to work with testers. The impact of this might however be larger as analysts are often the first source of information about the products intended functionality and its means of satisfying the stakeholders needs and wants. They are often also a sort of gate(keepers) in communicating to business stakeholders. In that sense they can make a testers live more or less easy. Especially if testers are not expected to go outside of the projects boundaries.

Are the (other) Testers available?

In my experience working as the only tester on a project has an impact both on the way you work and to some extend to the quality of your work. Being able to pair, share thoughts or just have a chat with another tester can help you reconsider your work and develop new or different test ideas. The tester doesn’t necessarily have to be in your team to have this effect. Having other testers in your team brings both the benefit (and sometimes burden) of being able to divide work, get fast feedback on test ideas or test results and the possibility to focus or divert away from your strengths and weaknesses as a tester.

Do you have a quiet work Environment?

This question addresses two different aspects. The first aspect is the infrastructure. Do you know what it’s components are? Do you have a separated test environment? And if so are you its only user? Do you know how to get access to it? Are you allowed to change it yourself or do you need others to do it for your? Is your test environment similar to the real production environment?

Secondly it addresses the circumstances of your workplace. Do you work in isolation, in cubicles, or in a large office garden? Is your work uninterrupted or are you (in)voluntarily involved into other work processes and activities? Does that influence your performance and well-being? What the influences are obviously depends on you as person and the real circumstances. But it is wise to take note and consider possible consequences. There are many studies into this field. Here are few articles that might trigger your interest: “Designing the work environment for worker health and productivity” by Jacqueline C. Vischer;  “Interrupt Mood” by Brian Tarbox or “Where does all that time go” by Michael Bolton.

Are the Stakeholders (that matter) available?

Stakeholders come in many forms and shapes, but they have one thing in common. They are in someway involved in the creation and/or use of the software solution. That not only means they need to be informed about the product that also means that they have expectations and opinions about the product itself, what it is used for, and what the products needs to able to do to make it valuable to them. As a tester you should identify these expectations and opinions and tailor your information about the product so that it is meaningful to them.

In theory the effort you put into gathering, tailoring and presenting that information is based on how much the stakeholders matters to the product, the project and to some extend to you the tester. I say in theory because to do so in practice the stakeholders need to be available and accessible. If they are not or if it is difficult you should take the extra time and effort into account of your testing and test reporting.

Is there (mandatory) Tooling?

There are many types of tools available in the market to capture requirements, store test cases, log test execution or manage bugs. And likewise there are many tools available to use during testing. As a tester you need to find out which tools there are, which tools you are allowed to use, and which tools are mandatory to use. You will might not know all the tools you are faced with or are unable to use a tool that you already know and like. In that case you will have to get used to the ‘new’ tooling and learn to use it. Additionally many tools have inbuilt workflows and processes that take away time from actual testing. As a tester you should be aware of this and take this into account when testing.

Poutsma principle

Whenever I start on a new test assignment or pick up a new work item I need to search and find its purpose, its meaning and I need to understand how the chosen requirements offer a solution to the problem that is solved. Sometimes that is really easy.

Say you visit the 36th International Carrot Conference before going to CAST 2013.  You come home and decide to sell carrots for hungry rabbits online and you want to vary the amount of carrots or differentiate the type of carrot for different breeds of rabbits. You will need something like drop down list or input field to identify the different rabbit breeds.  And except for the sudden urge to sell carrots this is fairly easy to understand and test.

If however you are asked to test the software implementation of calculating results for a new Credit Risk Model used by an international bank you will have a lot more to understand. If so I remind myself of the Poutsma Principle:

If something is too complex to understand, it must be wrong.

I use this principle to remind myself to keep asking questions until I either understand it or except the argumentation of it as proof. In either case it helps me to break down requirements to a level that makes me confident enough to start testing and daring enough  so that I can also use my personal addition to the principle

And it is your job (as a tester) to proof it wrong.

If you want to know more about the Poutsma Principle you can follow this link.

Seven Questions

This is the opening blog of a small series of posts in which I elaborate on a test approach heuristic using 7 questions that I have developed over the years.

Thinking about testing

As a tester I have seen many approaches to software testing pass me by. A few of them, like TMap (Next) and ISTQB were picked up by the Rabobank and I have had the mixed pleasure of working with them. But regardless of how different the approaches voice out to be from each other they all seem to have a number of things in common:

  • They are mostly oriented on management (of testing)
  • They focus on processes and deliverables
  • They do not teach you how to actually test something in practice
  • They hardly make any connection to software development in general
  • They are supposedly mastered after certification

I admit that both TMap and ISTQB (initially) helped to give testing a positive foothold in many organisations and have underlined that testing should get its place in software development. Even so the five elements I have described above should also show that there are fundamental flaws in how these approaches apply testing. Following them does not guarantee you to get fully involved into software development nor does it teach you how to test in practice. Usually as a compensation for these flaws testers go to boot camp like courses to teach them more practical testing skills, like determining test coverage and applying test design techniques. Even so for many testers the start-up of their professional life is focused on getting the certification and maybe some introduction to actual testing. And then….well…..for of them most it ends here and they go out to work and follow the processes and deliver a bunch of documents. If your new to software testing this will probably keep you busy for a while, but eventually you (should) start to ask yourself questions like: Is there no better way to test this? Do I really have to write these elaborate test plans / test scripts / test cases that nobody seems to really care about? Why don’t the developers agree with me on my defects? Why is my work not valued?

I have asked myself these and similar questions and over the years I have come up with a set of alternative questions whose answers guide me through a development / test cycle. These questions demand creativity, knowledge and skilled experience to answer them. And any answer you can come up with this time will differ the next time you ask yourself that same question again.

The seven questions I use are:

I have done a talk on these seven questions at EuroSTAR 2012 and will do the at Belgium Testing Days 2013. Contact me there if you want to meet and talk, or sent me a tweet @arborosa .

Here are some twitter reactions that I got from talking at EuroSTAR 2012

250 hours of practice – February/March

It has been a while since my last post. It was not that I have been procrastinating. No I actually was too busy to write a post. There were many reasons for this and most of them are covered in this post. So lets start with the biggest time consumer.

BBST Foundation

For those who do not know what this is I can almost hear you thinking.

“A foundation course? Doesn’t this guy have enough experience already and now he does Black Box Software Testing Foundation course. ”

Well let me put it this way. There are foundation courses and foundation courses. The Black Box Software Testing (BBST) Foundation course is as it says a foundation for further education and it is a good place to start your testing career. It’s just that it has a distinctive approach to it. First of all it is a four-week online course. Which is longer then almost any other course on software testing. Second it focusses on getting the participants to become critical and thinking software testers. It does this not by handing you any predefined recipe. It gives you the ingredients and the possibility to work with them. Let me, as an example, give you a short overview of the content:

Basic definitions                                              Programming fundamentals and Coverage
What is a computer program                               Numbering
Types of testing                                                   Storage
.                                                                           Representation
Strategy                                                               Data and Control structures
What is testing                                                     Coverage
Testing missions
Testing strategy                                              The impossibility of complete testing
                                                                          The basic combination rule
Oracles                                                                Paths and sub paths
System under Test                                              Data flows
Heuristics                                                             Sequences
Consistency oracles
More types of oracles                                       Measurement

All captured in :

  • 20+ articles and several books to read
  • 306 slides
  • Over 3 hours of video lecture
  • 5 quizzes
  • multiple individual and group assignments
  • Exam and exam grading
  • Over 70 hours of thinking, preparing and answering either individually or with team members from multiple time zones

Given that I also have worked these four weeks and maintained a family life I have made long days in February. But it was all worth it. Even if you have, like me, some experience in software testing it is a splendid (re)sharpener of your critical thinking skills with regard to software testing. Not only with the content of the course itself, but also with the interaction you have with the twenty or so other participants. Especially the group assignments and the assignments that need reviewing each others work give you a good insight in cultural and thinking differences people have. And from this and from the comments you can get a lot of more wisdom.

So some seventy of so hours spent on BBST Foundation. Well on my way towards the end goal of two hundred and fifty. But you might have noticed that this covers only February and this post is written half way March. So what more have I done.

Proposals

The end of February and beginning of March was also the time in which several deadline for calls for papers came in to view. I do not know if this really is considered practice, but it gets you to think and write about software testing anyway. Within one week I entered five proposals for a track and one for a tutorial for EuroSTAR, TestNet spring and autumn event and the Agile Testing Days. And I already am more than happy to tell you that the tutorial on the use of mind maps in testing, that I thought of together with testing buddy Huib Schoots, already got accepted.

But not only did I spent time on my own proposals. Zeger van Hese, this years program chair, invited me to help review some of the many proposals that EuroSTAR has gotten this year. And even if the amount of, anonimized, text is not so much I did want to do a serious review and evaluation of the proposals. (Like I would like others do with mine.) Some of them were good, some were bad and for some I was indecisive.

TestNet book

And there was more time spent reviewing. A couple of months ago I had committed to reviewing the TestNet jubilee book on the future of software testing. Obviously at the time I had not imagined to be this busy. But at the cost of some sleep I managed to finish the review prior to my next challenge.

The book itself is a great reference to over think where testing is going to and what choices you as tester need to make to make yourself both comfortable and future proof the coming five years.

Conferences

Although I have given talks and workshops before I had never been a speaker at a major international test conference. March 14 I made my debut on the international stage at the Belgium Testing Days. As I will be spending a separate post on the content of my talk I will limit for now with the remark that it was great fun to do and that it was great to meet both familiar and new  faces.

Meanwhile during this period Markus Gärtner published an interview with me on his blog as prequel to Europe’s first context-driven testing conference “Let’s Test“. The only downside for me seems to be that I am actually not able to attend, due to a lack of funds.

To complete the conference experience for the coming period I am invited to be a test expert at the Dutch Testing Conference (agile, context-driven testing and exploratory testing) and I invite participants to ask me questions during the conference.