The previous blog post had a number of definitions of software testing in it that focus on testing having bug finding as a purpose. As such this is not so strange since many people see finding or rather calling out bugs is the most visible part of software testing. Whether you agree with this or not one thing to me seems certain bugs are an important subject in software testing. This post then is all about bugs or rather it is about, the many definitions of (software) bugs.
- “This thing gives out and then that. ‘Bug’—as such little faults and difficulties are called—show themselves, and months of anxious watching, study, and labor are requisite before commercial success—or failure—is certainly reached.” Edison letter to Pushkas [Edison, 1878]
- “A software error is present when the program does not do what its end user reasonably expects it to do” Myers, 1976, p. 6 The art of software testing [1976]
- “There can never be an absolute definition for bugs, nor an absolute determination of their existence. The extent to which a program has bugs is measured by the extent to which it fails to be useful. This is a fundamentally human measure” Beizer, p. 12). Software Systems and Quality Assurance [1984]
- Fault: “An incorrect step, process, or data definition in a computer program. Note: This definition is used primarily by the fault tolerance discipline. In common usage, the terms ‘error’ and ‘bug’ are used to express this meaning.”
Failure: “The inability of a system or component to perform its required functions within specified performance requirements. Note: The fault tolerance discipline distinguishes between a human action (a mistake), its manifestation (a hardware or software fault), the result of the fault ( a failure), and the amount by which the result is incorrect.” IEEE Standard 610.12-1990 [1990] - “A mismatch between the program and its specification is an error in the program if and only if the specification exists and is correct.” Kaner, Falk and Nguyen, Testing Computer Software [1993]
- “Defect. Any state of unfitness for use, or nonconformance to specification.
Error. (1) The difference between a computed, observed, or measured value and the true, specified, or theoretically correct value or condition. (2) An incorrect step, process, or data definition. Often called a bug. (3) An incorrect result. (4) A human action that produces an incorrect result. Note: One distinction assigns definition (1) to error, definition (2) to fault, definition (3) to failure, and definition (4) to mistake. Fault. An incorrect step, process, or data definition in a computer program. See also: error. Failure. Discrepancy between the external results of a program’s operation and the software product requirements. A software failure is evidence of the existence of a fault in the software.” Software Error Analysis, NIST [1993] - “A software defect is a “material breach” of the contract for sale or license of the software if it is so serious that the customer can justifiably demand a fix or can cancel the contract, return the software, and demand a refund.” Kaner, Software QA Magazine [1996]
- “The difference between the actual test result and the expected result can indicate a software product defect” Koomen and Pol, Test Process Improvement [1999]
- “An observed difference between an expectation (prediction) and its actual outcome. A finding can be made on different objects, such as the test base (upon intake), while testing the system, on the test infrastructure, etc.” (translation) Pol, Teunissen, van Veenendaal, Testen volgens TMap 2e druk [2000]
- “An error is clearly present if a program does not do what it is supposed to do, but errors are also present if a program does what it is not supposed to do.” Meyer, The art of software testing (2nd edition) [2004]
- “A generic term that can refer to either a fault (cause) or a failure (effect). (IEEE 982.1-2005 IEEE Standard Dictionary of Measures of the Software Aspects of Dependability, 2.1) Example: (1) omissions and imperfections found during early life cycle phases and (2) faults contained in software sufficiently mature for test or operation” [2005]
- “A bug, also referred to as a software bug, is an error or flaw in a computer program that may prevent it from working correctly or produce an incorrect or unintended result.” The Linux Information Project [2005]
- “A defect (fault) is the result of an error residing in the code or document.”Broekman, van der Aalst, Vroon and Koomen, TMap Next for result driven testing [2006]
- “A bug is anything about the program that threatens its value.” Bach and Bolton, Rapid Software Testing [2007]
- Anything that causes an unnecessary or unreasonable reduction of the quality of a software product. BBST Bug Advocacy [2008]
- “Defects are observations that the test results deviate from expectations” Bouman, SmarTEST [2008]
- “A finding is an observed difference between an expectation or prediction and the actual outcome” Van der Aalst, Baarda, Roodenrijs, Vink and Visser, TMap NEXT Business Driven Test Management [2008]
- “something that went wrong,” Alan Page. Ken Johnston. Bj Rollison, How we test software at Microsoft [2009]
- “Situation that may cause errors to occur in an object” (ISO/IEC 10746-2:2009 Information technology — Open Distributed Processing — Reference Model: Foundations, 13.6.3) [2009]
- Imperfection or deficiency in a work product where that work product does not meet its requirements or specifications and needs to be either repaired or replaced (IEEE 1044-2009 IEEE Standard Classification for Software Anomalies, 2) [2009]
- An attribute of a software product that reduces its value to a favored stakeholder or increases its value to a disfavored stakeholder without a sufficiently large countervailing benefit. BBST Foundations [2010]
- A software error is the failure to comply with an assured characteristic (translation)
Ein Softwarefehler ist die Nichterfuellung einer zugesicherten Eigenschaft (eigenschaftsbezogene Fehlerdefinition). Basiswissen Software Testen [Spillner 2010] - A product has an error if it does not provide the security that when, taking into account all circumstances, in particular its performance, it can be used as reasonably can be legitimately expected, at the timing in which it was placed on the market. (translation) Ein produkt hat einen Fehler, wenn es nicht die Sicherheit bietet, die unter Berücksichtigung aller Umstande, ins besondere seiner Darbietung, des Gebrauchs, mit dem billigerweise gerechnet werden kann, des Zeitpunktes, in dem es in den Verkehr gebracht wurde, berechtigterweise erwartet werden kann [BGB01, ProdukthaftungsG. par. 3] (risikobesogene Fehlerdefinition). Basiswissen Software Testen [Spillner 2010]
- Defect in a hardware device or component (ISO/IEC/IEEE 24765:2010 Systems and software engineering–Vocabulary) [2010]
- Manifestation of an error in software (ISO/IEC/IEEE 24765:2010 Systems and software engineering–Vocabulary) [2010]
- A Software Defect / Bug is a condition in a software product which does not meet a software requirement (as stated in the requirement specifications) or end-user expectations (which may not be specified but are reasonable). Software Testing Fundamentals [2010]
- Incorrect step, process, or data definition in a computer program (ISO/IEC 25040:2011 Systems and software engineering–Systems and software Quality Requirements and Evaluation (SQuaRE)–Evaluation process, 4.27) [2011]
- Defect in a system or a representation of a system that if executed/activated could potentially result in an error (ISO/IEC 15026-1:2013 Systems and software engineering–Systems and software assurance–Part 1: Concepts and vocabulary, 3.4.5) [2013]
- Software bugs can originate from every step in translation from natural language, when:
- the Syntax-rules were not followed
- the translation falsified the Semantics, or
- if the Pragmatics are unclear
Software-Fehlertoleranz und Zuverlässigkeit [2013]
- A software bug is an error, flaw, failure, or fault in a computer program or system that causes it to produce an incorrect or unexpected result, or to behave in unintended ways. Wikipedia [2015]
- “the people who wrote the program told the computer to do something it couldn’t do”. Practical Programming [Gries, Campbell, Montojo 2015]
- An imperfection or deficiency in a project component where that component does not meet its requirements or specifications and needs to be either repaired or replaced (A Guide to the Project Management Body of Knowledge (PMBOK(R) Guide) — Fifth Edition)
- A software bug is a problem causing a program to crash or produce invalid output. Techopedia
-