This sixth post in the series of software testing overviews introduces the first of some 80+ different test types. That number itself is completely arbitrary. While searching for and investigating testing definitions I found over a hundred definitions of test types and I have chosen to leave out a number of ‘test types’. My choice to do so is based on the interpretation that some test types rather described a test level or a test technique and I could not see how to make a useful test type out of them.

So what then do I call a test type?

To me a test type is a particular subject of testing that has an approach, goal and/or use of oracle(s) that provides information that is typical to that test type.

While going through my overview you might find that some of the test types I mention do not entirely fit the narrow description of a test type as provided above. The reason that they are mentioned in spite of this is that I felt that they were so often mentioned as a test type that they should have a mention in this post just for that reason.

This post differs somewhat from the earlier posts as the definitions used are often rewritten by me to form a single aggregate definiton as many different ones for the same term exist. Where useful I have added comments as additional information. Also since a post with over 80 descriptions would be too long I have split up the overview into alphabetical sections. To begin with the letter A.

Just as a reminder these are not necessarily my definitions but a collection of definitions I encountered. Finally there are no sources or attributions for the individual test types as this would make this a totally different exercise.

A/B testing

A/B testing originates from marketing research used to investigate the more efficient of two possible solutions by presenting them to two different groups and measuring the ‘profits’. In software testing it is mostly used to compare two versions of a program or website, often of which only one contains changes on a single or a few controllable criteria.

Acceptance testing
During acceptance testing requirements, variables, parts of a program or specific behaviour of a program is compared against measurable aspects of predetermined acceptance criteria. This requires at least four things.
First identification of the requirements, variables and parts or behaviour (coverage). Second expressing these in measurable aspects.
Third the aspects need to represent the defining elements of the acceptance criteria. Finally the acceptance criteria themselves should represent the needs and wants of the stakeholder. The goal of this interpretation of acceptance testing is to provide stakeholders the possibility to accept the software. And provide the possibility of sign-of.

A more pragmatic way to look at acceptance testing is to allow the stakeholders, often end users, to evaluate the software and see if it meets there expectations and (operational) needs. In practice this is often done by proxy by stakeholder representatives. Sometimes the testers are selected as the representatives. I do not think that is a good idea as testers then step outside of there boundary of information provider, are often commited to the solution and their role in creating it and more essentially testers are not the end users. 

Active testing
Testing the program by triggering actions and events in the program and studying the results. To be honest I do not consider this a test type as in my opinion this describes nearly all types of testing.

Ad-hoc testing

Ad-hoc testing is software testing performed without explicit prior planning or documentation on direction of the test, on how to test or on which oracles to use.
Some definitions see this as informal, unstructured and not reproducible. Informality and being unstructured (if seen as unprepared) is certainly true as this would be the point of doing it ad-hoc. The not reproducible part depends on whether you care to record the test progress and test results. Something that is in my opinion is not inherently attached to doing something ad-hoc but highly advisable. Why elso would you be testing if you are able to tell the testing story.

Age testing
It is a testing technique that evaluates a system’s ability to perform in the future. As the system gets older, how significantly the performance might drop is what is being measured in Age Testing.
To be honest I found only one reference to this test type but I find the idea interesting. 

Agile testing
Agile testing is mentioned often as a test type or test approach but I have added no definition or description here. In my opinion agile testing is not a software test type. Agile testing rather is a particular context in which testing is performed that may have its particular challenges on test execution, on how tests are approached and on choices of test tooling but not a specific test type. 

Alpha testing
Alpha testing is an in-house (full) integration test of the near complete product that is executed by others than the development team but still is executed in a development environment. Alpha testing simulates the products intended use and helps catch design flaws and operational bugs.
One could argue that this is more of a test level than a test type. I care to view it as test type because it is more about the type of use and its potential to discover new information than that it is part of the software development itself. I specifically disagree with the idea that this is an intermediary step towards, or is part of, handing over software to a Test/QA group as some definitions propose. In my opinion testing is integrated right from the start of development up until it stops because the product ends its life-cycle or due to some other stopping heuristic. 

API testing
API testing involves testing individual or combined inputs, outputs and business rules of the API under investigation.
Essentially an API is a device-independent, or component-independent access provider that receives, interprets/transforms and sends messages so that different parts of a computer or programs can use each other’s operations and/or information. Testing an API is similar to testing in general albeit that an API has a smaller scope has, or should have, specific contracts and definitions that describe the API’s specific variables, value ranges and (business) rules. Testing an API should however not be limited to the API alone. Sources, destinations (end-points), web services (e.g. REST, SOAP), message types (e.g. JSON, XML), message formats (e.g. SWIFT, FIX, EDI, CSV), transport- (e.g. HTTP(S), JMS, MQ)  and communication protocols (e.g. TCP/IP, SMTP, MQTT, TIBCO Rendezvous) all influence the overall possibilities and functionality of the API in relation to the system(s) that use(s) the API. Typically API testing is semi- or fully automated and requires sufficient tool, message type, and transport- and communication protocol knowledge to be executed well.

An alternative meaning of API testing is testing by using an API. In this case the API is a means to an end in gaining access to the subject under test and feeding it with data or instructions and gathering responses.