Test Types – N, O, P

Test Type
A particular type of testing that has an approach, goal and/or use of oracle(s) that provides information that is typical to that test type.

This is the sixth post in a (sub) series on Test Types. Please add any additions or remarks in the comment section.

Negative testing
Negative testing ensures that your application can gracefully handle invalid input or unexpected user behavior. It is the process of validating the application against invalid data.
Although most testers will agree on the importance of negative testing it sometimes is difficult to put the necessary time into it. Especially when many developer or business stakeholder argue against too many “hypothetical” tests in favor of confirmation tests

Neighbour testing
Testing the connectivity and data exchange between the application and its direct ‘neighbours’.
A colleague of mine used this type of testing in a Chain Test Plan that he wrote. I found it a useful expression to describe limiting the (initial) scope of a chain test to its immediate surroundings.

Network conditions simulation testing
Testing traffic, exchange of data and behaviour of systems and interfaces while simulating different network conditions.
Think of introducing latency, interruptions, data package loss etc.

Non-functional testing
Testing how a software application or system operates.
A very short description that covers a world of possibilities. While behaviour is the subject of many requirements documents, user stories and the like, non-functionals are much less intensively described. Often they are only identified by a number of constraints and boundaries describing what the system should be able to handle or what is specifically not allowed to do. Don’t be fooled by their absence. Non-functionals have the possibility of influencing the behaviour of applications so that it looks as if the behaviour is incorrect or correct while it is not. Moreover non-functionals can have a major impact on user experience. There are many quality attributes and testing types connected to non-functional testing that can be used for investigation.

Operational testing
Operational testing is testing focussed on usage of the application or system in its intended environment, its intended usage and by its intended users.
Environment here extends beyond the hardware, tooling or user interface and such. It includes applicable standards, procedures, processes and culture. Similarly for users this not only relates to consumers or end users but also to operations, customer service, engineers, etc. And usage extends beyond day-to-day use but also relates to maintenance, backup and restore, etc.

Packaged application testing
Testing application packages or application suites on the inter functionality of the separate parts in their interaction and their behaviour as a whole.
There are a  number of suppliers offering package application testing services for products like SAP and Oracle. Although I see the benefit of testing process flow and behaviour throughout such a group of applications that more or less work together I am not quite sure how this differs from any other testing.

Penetration testing
Investigating a system and its environment on the possibilities of gaining unwanted access to a environment or system or the possibility to retrieve not be disclosed information.
Penetration testing together with performance testing (see next item) is in my opinion one of the specialist fields in software testing. While every testing should have some basic understanding and skill in this area it involves both continuous renewed skill, knowledge and practice to excel in penetration testing. 

Performance testing
Determining how a system performs in terms of responsiveness and stability under various work loads.
While the focus is often particularly on high loads or peak loads special attention should also be given to low loads, intermittent loads, loads addressing only specific parts of the system. Also load should point towards both load given to the system as load produced by the system.
There are several sub-areas that fall under performance testing that each have a specific area of performance that they address: Load testing, Stress testing, Soak testing, Configuration testing, Spike testing, Isolation testing, etc..

Test Types – J, K, L, M

Test Type
A particular type of testing that has an approach, goal and/or use of oracle(s) that provides information that is typical to that test type.

This is the fifth post in a (sub) series on Test Types. Please add any additions or remarks in the comment section.

Keyword-driven testing

Keyword-driven testing is a methodology used for both manual and automated testing but is best known in combination with automated testing. In this method of testing the documentation and design of testing is separated from the execution. Keywords or action words are defined for each of the actions to be executed. These words are then used in a pre-defined format (often a table) to enable a combination of actions to be executed (and possibly evaluated) by either a person or a previously set up test automation framework.

Load testing

Load testing is the process of putting demand on a software system or computing device and measuring its response.
Although closely related to stress testing load testing has the aim to measure behaviour for different loads and not necessarily for peak loads.

Localization testing

Localization is the process of adjusting (internationalized) software for a specific region or language by adding locale-specific components and translating text.
Localization testing then is performing checks and validation that the translation and locale-specific components are functionally correct and understandable.

Manual scripted testing

Manual scripted testing is the process of manually executing previously designed test scripts in search for software behaviour that does not match the behaviour as described in the script.
On paper this type of software testing is still relatively popular as it is based on the early descriptions of structured testing and as such appeals to non-testers. A fair number of testing practitioners however judge executing test scripts as inefficient, ineffective and boring.

Manual support testing

Oddly enough there are two definitions in circulation for this.
“Testing technique that involves testing of all the functions performed by the people while preparing the data and using these data for automated system.
This one I believe should be discarded as being part of test automation preparation. Although I have to admit that at the start of test automation this activity is often under estimated. The second one I think is much more useful.
Testing manual support systems involves all the functions performed by people in preparing data for, and using data from, automated applications. The objectives of testing the manual support systems are to verify that the manual support procedures are documented and complete, support responsibility has been assigned, determine that the manual support people are adequately trained and to determine that the manual support and the automated segment are properly interfaced.

Memory profiling

Memory profiling is the process of investigating and analyzing a program’s behavior to determine how to optimize the program’s memory usage.

Migration testing

Migration testing is the activity of testing data, functionality or behaviour of software after migration to a new platform, database or program for intended and unintended differences.

Model based testing

Model based testing is the automatic generation of software test procedures, using models of system requirements and behavior. Once created these test procedures can be run repeatedly.
Model based testing has been around for several years now but has never really made a big impact on testing as such. The reasons for this I believe are that the “automatic generation” and repeatability of the tests require a considerable technical- and time investment and are not nearly as easily achieved as portrait by model based testing tool vendors. Next to this the success and quality of the tests is highly dependent on the ability to formulate the software behaviour into a model and on the quality of the then created model itself. In practice model based testing will find bugs but not as many as often promised and certainly not all.

Mutation testing

Mutation testing (or Mutation analysis or Program mutation) is used to design new software tests and evaluate the quality of existing software tests. Mutation testing involves modifying a program in small ways. Each mutated version is called a mutant and tests detect and reject mutants by causing the behavior of the original version to differ from the mutant. This is called killing the mutant. Test suites are measured by the percentage of mutants that they kill. New tests can be designed to kill additional mutants. Mutants are based on well-defined mutation operators that either mimic typical programming errors (such as using the wrong operator or variable name) or force the creation of valuable tests (such as dividing each expression by zero). The purpose is to help the tester develop effective tests or locate weaknesses in the test data used for the program or in sections of the code that are seldom or never accessed during execution. (Wikipedia)

Test Types – G, H, I

Test Type
A particular type of testing that has an approach, goal and/or use of oracle(s) that provides information that is typical to that test type.

This is the fourth post in a (sub) series on Test Types. Please add any additions or remarks in the comment section.

Glass box testing
Testing or test design using knowledge of the details of the internals of the program (code and data) (BBST definition)
Too many this might sound familiar to White box testing. I think however that the perspective is quite different. Here the perspective is from a tester rather than a developer.

Gorilla testing
Gorilla testing is testing on particular module, component or functionality heavily and with large variety.

Grey box testing
Grey box testing is testing with, limited, knowledge and access to the code and internal structure and details of the software.
It is my opinion that most testers, safe pure programmers and pure black box tester, are doing grey box testing. As such they stand somewhere on a scale between the two extremes. Their place is sometimes determined by (enforced) choices of role or scope but more often by their programming and design knowledge and capabilities.

GUI testing
Graphical User Interface testing is testing the application’s user interface to detect if the functionality of the interface itself and the functionality that is directly influenced or dependent by the user interface functions correctly.
With the current growing attention to automated testing, and especially to testing on API level, the GUI is quick to be overlooked in its importance. It remains however know more than ever the first point of contact of any user with the system.

Incremental integration testing
Incremental integration testing is an approach in which you first test each module of component individually and then add them one by one together and test the integration. You can do this top down, bottom up or functionally incremental.

Installation testing
One side of installation testing is aimed at ensuring that all the necessary components are installed properly and are working as required once installed. The other side of installation testing focusses on what users need to do to install and set up new software successfully.

Integration testing
Integration testing is testing where, previously tested, individual modules and components are combined and tested as a group. It tests not only interactions between individual components but also between different sets of components and parts of the system within its direct environment. Integration testing focusses on different aspects such as functionality, performance, design and reliability.

Inter systems testing
Inter systems testing focusses at testing on interconnection and integration points of separate systems but working together.

Interface testing
Interface Testing is performed to evaluate whether systems or components pass data and control correctly to one another. It is to verify if all the interactions between these modules are working properly and errors are handled properly.

Internationalization testing
Internationalization is designing software systems in such a way that they can be adapted to different languages and regions without engineering changes, loss of functionality, loss of data or integrity issues. Internationalization testing is aimed at uncovering these potential problems.