The CDT Brigade

On August 26, 2014 @perftestman posted the following tweet:
“I think a lot of the CDT brigade just like the sound of they’re  own voice –
everything’s CONTEXT driven! ”

It was her reaction on my earlier tweet, that got some attention:
Dear @pt_wire I use effective, documented systematic testing processes and methods. But not generic one size fits all. #CDT vs #ISO29119

For a while both James Bach and I reacted to it but the limitation of 140 characters, twittering on a mobile phone and not the least work made me stop engaging in the twitter feed. But it wasn’t the first time that people resort to this kind of fallacy thus avoiding discussing the actual content. This made think about it and in this post I will share my little thought exercise.

Addressing the first part of her tweet an answer to what is a brigade?

  • A group of people who have the same beliefs
    I wouldn’t compare CDT to a belief system but one cannot deny that the CDT community shares values, talks about it, sometimes feels strongly about them and expresses them out in the open. This part doesn’t apply specifically to CDT as e.g. the ISTQB brigade has similar behavior.
    I also do not think that the CDT community is out to convert people to be context-driven testers. To convince by pointing out alternatives and different approaches -yes, answer questions – yes, share knowledge – yes, share experiences – yes, but testers are allowed to decide for them selves how and if they want to use it.
  • A group of people organized to act together
    The CDT community certainly regularly confers, meets each other (live or online) and challenges each other. They are however not organized as a single group or organization nor do I think they want to be. They are mostly independently and critical minds that have discovered that taking into account and using the context helps them to deliver more value and that building skill, acquiring and sharing knowledge and experience with others helps them to get better at doing that.
  • A large group of soldiers that is part of an army
    I fear that @perftestman intended to make use of this definition. This comparison however lacks credibility. The CDT community might seem to go battle figuratively over some subject and if feeling strongly about engaging in fierce discussion. And unfortunately occasionally a few of them even lash out to individuals that oppose CDT values or cannot handle the challenging style of discussion. But even if we form a community we are not so organized that we form a specific group of sorts nor do we intend to destruct or conquer. The community, essentially is a collection of likeminded professionals who rather aim to convince with facts, ideas and experience with the intend to advance the craft.

In a follow up post I will address an earlier tweet: “Too many of these so called test guru are impostors – detached from the realities of software development and #lovethesoundofyourownvoice” and compare the contributors to ISO 29119 to the CDT brigade and thus attempt to discover who are meant to be the potential impostors in this tweet and other occasions that spawned remarks like this.

 

Advertisements

Seven Questions – Why do I test?

Reasons for testing

The question why something is tested has kind of a schizophrenic nature to it. Its answer is either so obvious that the question itself is ignored or it is so cumbersome that testers rather avoid to answer it. The latter is mostly the case if testers have to defend why testing is done in the first place. I cannot provide you with ready made answers for this, because the answer depends too much on the circumstances and context of your situation. What I can tell you is that it is worth while for you to figure out why your specific subject under test needs testing. If you can answer this for your specific situation, then the contextually acceptable general answer should be able to be derived from it.

What to take into consideration?

One of the more common ideas on why software testing is conducted is that it’s done to find bugs. The idea is that the fact that you do or do not find more then a certain amount of bugs in time is a measurement for release readiness. Obviously such a amount of bugs doesn’t say anything about how serious the remaining bugs are nor does it say anything about the bugs you did not find. Already more than 40 years ago Edsger W. Dijkstra (1969 p.16 and 1970) discovered that software testing can show the presence of bugs, but never their absence .

Another reason to do software testing is that there is some internal or external reason to do it. A standard, law or regulation exists that either states that you have to test or whose interpretation makes management believe that you should do tests, often in a certain predefined way, to meet the rules. This is not a bad thing as an external reason for testing.

A better, more internal, reason for testing is that the software is tested to provide information. Better still Information that is meaningful with regard to the product itself, its intended use, its real use, its potential (miss)use and related to the value this has to which stakeholders.

Your Challenge

It is your challenge as a tester to find out what the information is that the stakeholders value. Then extend this with the information they should value, even if for reasons thus far unknown to them. And finally to find a way how to provide that information so that its relevance and value is delivered to them in a meaningful way.

With some well-directed extra effort the value of testing can grow. Both to the tester and to the stakeholders.

Finding the right reason

Way back in 1998, with the introduction of the Euro to the financial markets I came in contact with software testing for the first time. As a business acceptance tester I was responsible of judging whether the new programs actually had the desired functionality and if we could work with them. Especially that latter part had the focus of my attention. Being one of the users myself and being involved in the requirements design of the product I found it easy to understand why this had to be tested and what value to look for.

More often than not software testers are not so familiar with the everyday practical needs and demands of the product they are working on. In this case I have two approaches that I prefer to use and that have served me well in the past. The first is to approach testing heuristically, with for instance the Heuristic Test Strategy Model,  and explore the product with helpful mnemonics like FEW HICCUPPS . The second approach is to converse and to keep on conversing with the stakeholders and ask them all they need to know.

Who else would know better what matters to them than the people who matter (to the product and/or the project) themselves.

Why do I test, summarized.

Being challenged by James M. Bach

On March 31st I joined Huib Schoots and Pascal Dufour in picking up James Bach from Schiphol Airport and driving him to his hotel in Dordrecht. Having met Cem Kaner (briefly) and Michael Bolton (several times) I was looking forward to meeting James in person and I was not disappointed. Conversation en route was vivid and fruitfull  and continued during dinner at Villa Augustus and later on April 3rd after an Agile Meetup with presentation by James at Codecentric. During dinner James had offered Pascal a challenge and later that evening he offered me the following challenge on twitter:

Explain dendogram-based testing

My first thought was “What’s a dendogram?” So I googled it and saw something I had long forgotten from when I studied Social Sciences and certainly had never used in testing.

So for those of you who requested me to blog on testing and the social sciences this is my first post on the subject.

For all the others with this post I want to share my answer to the challenge with you and some of the reactions I got from James on it.

What is dendogram-based testing?

To answer this question I will start with a short description of what I understand a dendogram is. Followed with a description of how I think this could be used in testing.

Dendogram

A dendogram is a visual representation of data that shows hierarchical clusters of, potentially before unknown, groups of data in a cluster tree diagram. A picture of such a diagram is shown below.

Cluster analysis, for a dendogram, starts with a data matrix, where objects are rows and observations are columns. From this beginning, a table is constructed where objects are both rows and columns and the numbers in the table are measures of similarity or differences between the two observations.

  X1 X2 X3
O1      
O2      
O3      

 From measurements to relative observation results 

  O1 O2 O3
O1      
O2      
O3      

 You can then place the results in a cloud of n points or in a matrix and put each point in a class of its own (having n classes, each containing a single point). Then, look for the closest two classes (for a given distance, for instance the relative distance between measurement of a feature– but other distances will give other results, perhaps more appropriate for some data sets) and join them into a new class. You now have n-1 classes, all but one containing a single element. Then iterate this activity until you reach a single set containing al the classes. The result is presented as a dendogram: the leaves are the initial 1-element classes and the various “levels”) of the dendogram are various clustering’s of the data, into a decreasing number of classes.

It is also possible to calculate the distances and their relationships with the use of Euclidian algebra. This uses a number of formulas to calculate hierarchical distance between the data (clusters). But besides the relative complexity of the calculation I prefer to use the more visual and intuitive approach.

Dendogram-Based Testing

The concept of clustering  data in a hierarchical form and thus grouping it based on some measured characteristic opens a number of possibilities for testing. All of these possibilities have in common that both the activity of clustering of data and the end result, be it in numbers or a visual representation, form an additional source of information about the subject under test. The kind of information will largely differ based on the chosen measurement. With regard to testing I see the following, extendable list of areas for observation and measurement:

  • The relationship between functionality and code

Where code is differentiated into several elements such as (that are more or less commonly used):

  • Type of input
  • Used variables
  • Classes
  • Functions
  • Methods
  • Etc.
  • The relationship between test cases and code
  • The relationship between functionality and GUI elements
  • The relationship between defects and code
  • The relationship between defects and their cause of failure

The clusters of information that you get out of it provide the following information and activities for testing:

  • The different clusters of information can suggest different testing techniques
  • The different clusters of information can suggest different testing missions
  • The different clusters of information can suggest different elements of reporting

In essence the information derived from a dendogram is dependent from the kind of observation and measurement you choose, the skill you have in identifying the clusters and patterns and the skill to adapt your testing activities. In any case it forms a new oracle to be used in software testing.

The above description was made before I read the following document, that I had avoided on purpose:  http://melindaminch.com/docs/thesis.pdf

In my, opinion the thesis is a nice academic exercise that adds insight to testing, but lacks sufficient practicality for use in every day testing. But I will study it more carefully at sometime in the future and I will see if there is an opportunity to use HCE 3.0 that I have downloaded.

For now I prefer the more general intuitive approach as described above and instead of the more elaborate HCE program I would use a reverse mind map.

Giving me the below, originally not in my answer included, graphical steps:

Which translates into the following Dendogram

James response to my answer

Below I have pasted the, unedited, response James gave my to my answer.

Jean-Paul, I am impressed. This is an excellent answer. I concur with your reasoning. I’d like to try using dendograms in testing, when the opportunity arises.

The reason I gave you this challenge is that we who aspire to excellence must be comfortable describing test techniques that don’t yet exist– in other words, we must be confident in our ability to invent test techniques as needed. This is a challenge that cannot be solved well using Google, since almost nobody has applied dendograms to testing, and the one reference you cite does a poor job of it.

By researching this and replying as you did, you have marked yourself as a leader in the Context-Driven community. Thank you.

— James

Needless to say that I am proud to receive such a response. And I feel strengthened in my drive to grow and learn as tester and as a member of the context-driven community.