1. Appendix A: Common Software Errors Testing Computer Software Cem Kaner, Jack Falk, Hung Quoc Nguyen Please do not reproduce without this notice http://www.testingeducation.org/BBST/testdesign/kaner_common_software_errors.pdf
2. USER INTERFACE ERRORS
   1. FUNCTIONALITY
      1. Excessive functionality
      2. Inflated impression of functionality
      3. Inadequacy for the task at hand
      4. Missing function
      5. Wrong function
      6. Functionality must be created by the user
      7. Doesn't do what the user expects
   2. COMMUNICATION
      1. Missing information
         1. No onscreen instructions
         2. Assuming printed documentation is readily available
         3. Undocumented features
         4. States that appear impossible to exit
         5. No cursor
         6. Failure to acknowledge input
         7. Failure to show activity during long delays
         8. Failure to advise when a change will take effect
         9. Failure to check for the same document being opened more than once
      2. Wrong, misleading, or confusing information
         1. Simple factual errors
         2. Spelling errors
         3. Inaccurate simplifications
         4. Invalid metaphors
         5. Confusing feature names
         6. More than one name for the same feature
         7. Information overload
         8. When are data saved?
         9. Poor external modularity
      3. Help text and error messages
         1. Inappropriate reading level
         2. Verbosity
         3. Inappropriate emotional tone
         4. Factual errors
         5. Context errors
         6. Failure to identify the source of an error
         7. Hex dumps are not error messages
         8. Forbidding a resource without saying why
         9. Reporting non-errors
      4. Display bugs
         1. Two cursors
         2. Disappearing cursor
         3. Cursor displayed in the wrong place
         4. Cursor moves out of data entry area
         5. Writing to the wrong screen segment
         6. Failure to clear part of the screen
         7. Failure to highlight part of the screen
         8. Failure to clear highlighting
         9. Wrong or partial string displayed
         10. Messages displayed for too long or not long enough
      5. Display layout
         1. Poor aesthetics in the screen layout
         2. Menu layout errors
         3. Dialog box layout errors
         4. Obscured instructions
         5. Misuse of flash
         6. Misuse of color
         7. Heavy reliance on color
         8. Inconsistent with the style of the environment
         9. Cannot get rid of onscreen information
   3. COMMAND STRUCTURE AND ENTRY
      1. Inconsistencies
         1. "Optimizations"
         2. Inconsistent syntax
         3. Inconsistent command entry style
         4. Inconsistent abbreviations
         5. Inconsistent termination rule
         6. Inconsistent command options
         7. Similarly named commands
         8. Inconsistent capitalization
         9. Inconsistent menu position
         10. Inconsistent function key usage
         11. Inconsistent error handling rules
         12. Inconsistent editing rules
         13. Inconsistent data saving rules
      2. Time-wasters
         1. Garden paths
         2. Choices that can't be taken
         3. Are you really, really sure?
         4. Obscurely or idiosyncratically named commands
      3. Menus
         1. Excessively complex menu hierarchy
         2. Inadequate menu navigation options
         3. Too many paths to the same place
         4. You can't get there from here
         5. Related commands relegated to unrelated menus
         6. Unrelated commands tossed under the same menu
      4. Command lines
         1. Forced distinction between uppercase and lowercase
         2. Reversed parameters
         3. Full command names not allowed
         4. Abbreviations not allowed
         5. Demands complex input on one line
         6. No batch input
         7. Can't edit commands
      5. Inappropriate use of the keyboard
         1. Failure to use cursor, edit, or function keys
         2. Non-standard use of cursor and edit keys
         3. Non-standard use of function keys
         4. Failure to filter invalid keys
         5. Failure to indicate keyboard state changes
         6. Failure to scan for function or control keys
   4. MISSING COMMANDS
      1. State transitions
         1. Can't do nothing and leave
         2. Can't quit mid-program
         3. Can't stop mid-command
         4. Can't pause
      2. Disaster prevention
         1. No backup facility
         2. No undo
         3. No Are you sure?
         4. No incremental saves
      3. Error handling by the user
         1. No user-specifiable filters
         2. Awkward error correction
         3. Can't include comments
         4. Can't display relationships between variables
      4. Miscellaneous nuisances
      5. Inadequate privacy or security
      6. Obsession with security
      7. Can't hide menus
      8. Doesn't support standard O/S features
      9. Doesn't allow long names
   5. PROGRAM RIGIDITY
      1. User tailorability
         1. Can't turn off the noise
         2. Can't turn off case sensitivity
         3. Can't tailor to hardware at hand
         4. Can't change device initialization
         5. Can't turn off automatic saves
         6. Can't slow down (speed up) scrolling
         7. Can't do what you did last time
         8. Can't find out what you did last time
         9. Failure to execute a customization command
         10. Failure to save customization commands
         11. Side-effects of feature changes
         12. Infinite tailorability
      2. Who's in control
         1. Unnecessary imposition of a conceptual style
         2. Novice-friendly, experienced-hostile
         3. Artificial intelligence and automated stupidity
         4. Superfluous or redundant information required
         5. Unnecessary repetition of steps
         6. Unnecessary limits
   6. PERFORMANCE
      1. Slow program
      2. Slow echoing
      3. How to reduce user throughput
      4. Poor responsiveness
      5. No type-ahead
      6. No warning that an operation will take a long time
      7. No progress reports
      8. Problems with time-outs
      9. Program pesters you
      10. Do you really want help and graphics at 300 baud?
   7. OUTPUT
      1. Can't output certain data
      2. Can't redirect output
      3. Format incompatible with a follow-up process
      4. Must output too little or too much
      5. Can't control output layout
      6. Absurd printed level of precision
      7. Can't control labeling of tables or figures
      8. Can't control scaling of graphs
3. ERROR HANDLING
   1. ERROR PREVENTION
      1. Inadequate initial state validation
      2. Inadequate tests of user input
      3. Inadequate protection against corrupted data
      4. Inadequate tests of passed parameters
      5. Inadequate protection against operating system bugs
      6. Inadequate version control
      7. Inadequate protection against malicious use
   2. ERROR DETECTION
      1. Ignores overflow
      2. Ignores impossible values
      3. Ignores implausible values
      4. Ignores error flag
      5. Ignores hardware fault or error conditions
      6. Data comparisons
   3. ERROR RECOVERY
      1. Automatic error correction
      2. Failure to report an error
      3. Failure to set an error flag
      4. Where does the program go back to?
      5. Aborting errors
      6. Recovery from hardware problems
      7. No escape from missing disk
4. BOUNDARY-RELATED ERRORS
   1. NUMERIC BOUNDARIES
   2. EQUALITY AS A BOUNDARY
   3. BOUNDARIES ON NUMEROSITY
   4. BOUNDARIES IN SPACE
   5. BOUNDARIES IN TIME
   6. BOUNDARIES IN LOOPS
   7. BOUNDARIES IN MEMORY
   8. BOUNDARIES WITHIN DATA STRUCTURES
   9. HARDWARE-RELATED BOUNDARIES
   10. INVISIBLE BOUNDARIES
   11. CALCULATION ERRORS
   12. OUTDATED CONSTANTS
5. CALCULATION ERRORS
   1. IMPOSSIBLE PARENTHESES
   2. WRONG ORDER OF OPERATORS
   3. BAD UNDERLYING FUNCTION
   4. OVERFLOW AND UNDERFLOW
   5. TRUNCATION AND ROUNDOFF ERROR
   6. CONFUSION ABOUT THE REPRESENTATION OF THE DATA
   7. INCORRECT CONVERSION FROM ONE DATA REPRESENTATION TO
   8. ANOTHER
   9. WRONG FORMULA
   10. INCORRECT APPROXIMATION
6. INITIAL AND LATER STATES
   1. FAILURE TO SET A DATA ITEM TO 0
   2. FAILURE TO INITIALIZE A LOOP-CONTROL VARIABLE
   3. FAILURE TO INITIALIZE (OR REINITIALIZE) A POINTER
   4. FAILURE TO CLEAR A STRING
   5. FAILURE TO INITIALIZE (OR REINITIALIZE) REGISTERS
   6. FAILURE TO CLEAR A FLAG
   7. DATA WERE SUPPOSED TO BE INITIALIZED ELSEWHERE
   8. FAILURE TO REINITIALIZE
   9. ASSUMPTION THAT DATA WERE NOT REINITIALIZED
   10. CONFUSION BETWEEN STATIC AND DYNAMIC STORAGE
   11. DATA MODIFICATION BY SIDE-EFFECT
   12. INCORRECT INITIALIZATION
   13. RELIANCE ON TOOLS THE CUSTOMER MAY NOT HAVE OR UNDERSTAND
7. CONTROL FLOW ERRORS
   1. PROGRAM RUNS AMOK
      1. GOTO somewhere
      2. Come-from logic errors
      3. Problems in table-driven programs
      4. Executing data
      5. Jumping to a routine that isn't resident
      6. Re-entrance
      7. Variables contain embedded command names
      8. Wrong returning state assumed
      9. Exception-handling based exits
      10. Return to wrong place
          1. Corrupted stack
          2. Stack under/overflow
          3. GOTO rather than RETURN from a subroutine
      11. Interrupts
          1. Wrong interrupt vector
          2. Failure to restore or update interrupt vector
          3. Failure to block or unblock interrupts
          4. Invalid restart after an interrupt
   2. PROGRAM STOPS
      1. Dead crash
      2. Syntax errors reported at run-time
      3. Waits for impossible condition, or combination of conditions
      4. Wrong user or process priority
   3. LOOPS
      1. Infinite loop
      2. Wrong starting value for the loop control variable
      3. Accidental change of the loop control variable
      4. Wrong criterion for ending the loop
      5. Commands that do or don't belong inside the loop
      6. Improper loop nesting
   4. IF, THEN, ELSE, OR MAYBE NOT
      1. Wrong inequalities (e.g., > instead of >=)
      2. Comparison sometimes yields wrong result
      3. Not equal versus equal when there are three cases
      4. Testing floating point values for equality
      5. Confusing inclusive and exclusive OR
      6. Incorrectly negating a logical expression
      7. Assignment-equal instead of test-equal
      8. Commands belong inside the THEN or ELSE clause
      9. Commands that don't belong inside either clause
      10. Failure to test a flag
      11. Failure to clear a flag
   5. MULTIPLE CASES
      1. Missing default
      2. Wrong default
      3. Missing cases
      4. Case should be subdivided
      5. Overlapping cases
      6. Invalid or impossible cases
8. ERRORS IN HANDLING OR INTERPRETING DATA
   1. PROBLEMS WHEN PASSING DATA BETWEEN ROUTINES
      1. Parameter list variables out of order or missing
      2. Data type errors
      3. Aliases and shifting interpretations of the same area of memory
      4. Misunderstood data values
      5. Inadequate error information
      6. Failure to clean up data on exception-handling exit
      7. Outdated copies of data
      8. Related variables get out of synch
      9. Local setting of global data
      10. Global use of local variables
      11. Wrong mask in bit field
      12. Wrong value from a table
   2. DATA BOUNDARIES
      1. Unterminated null terminated strings
      2. Early end of string
      3. Read/write past end of a data structure, or an element in it
   3. READ OUTSIDE THE LIMITS OF A MESSAGE BUFFER
      1. Compiler padding to word boundaries
      2. Value stack under/overflow
      3. Trampling another process' code or data
   4. MESSAGING PROBLEMS
      1. Messages sent to wrong process or port
      2. Failure to validate an incoming message
      3. Lost or out of synch messages
      4. Message sent to only N of N+1 processes
   5. DATA STORAGE CORRUPTION
      1. Overwritten changes
      2. Data entry not saved
      3. Too much data for receiving process to handle
      4. Overwriting a file after an error exit or user abort
9. RACE CONDITIONS
   1. RACES IN UPDATING DATA
   2. ASSUMPTION THAT ONE EVENT OR TASK HAS FINISHED BEFORE ANOTHER
   3. BEGINS
   4. ASSUMPTION THAT INPUT WON'T OCCUR DURING A BRIEF PROCESSING
   5. INTERVAL
   6. ASSUMPTION THAT INTERRUPTS WON'T OCCUR DURING A BRIEF INTERVAL
   7. RESOURCE RACES: THE RESOURCE HAS JUST BECOME UNAVAILABLE
   8. ASSUMPTION THAT A PERSON, DEVICE, OR PROCESS WILL RESPOND
   9. QUICKLY
   10. OPTIONS OUT OF SYNCH DURING A DISPLAY CHANGE
   11. TASK STARTS BEFORE ITS PREREQUISITES ARE MET
   12. MESSAGES CROSS OR DON'T ARRIVE IN THE ORDER SENT
10. LOOP CONDITIONS
    1. REQUIRED RESOURCE NOT AVAILABLE
    2. DOESN'T RETURN A RESOURCE
       1. Doesn't indicate that it's done with a device
       2. Doesn't erase old files from mass storage
       3. Doesn't return unused memory
       4. Wastes computer time
    3. NO AVAILABLE LARGE MEMORY AREAS
    4. INPUT BUFFER OR QUEUE NOT DEEP ENOUGH
    5. DOESN'T CLEAR ITEMS FROM QUEUE, BUFFER, OR STACK
    6. LOST MESSAGES
    7. PERFORMANCE COSTS
    8. RACE CONDITION WINDOWS EXPAND
    9. DOESN'T ABBREVIATE UNDER LOAD
    10. DOESN'T RECOGNIZE THAT ANOTHER PROCESS ABBREVIATES OUTPUT
    11. UNDER LOAD
    12. LOW PRIORITY TASKS NOT PUT OFF
    13. LOW PRIORITY TASKS NEVER DONE
11. HARDWARE
    1. WRONG DEVICE
    2. WRONG DEVICE ADDRESS
    3. DEVICE UNAVAILABLE
    4. DEVICE RETURNED TO WRONG TYPE OF POOL
    5. DEVICE USE FORBIDDEN TO CALLER
    6. SPECIFIES WRONG PRIVILEGE LEVEL FOR A DEVICE
    7. NOISY CHANNEL
    8. CHANNEL GOES DOWN
    9. TIME-OUT PROBLEMS
    10. WRONG STORAGE DEVICE
    11. DOESN'T CHECK DIRECTORY OF CURRENT DISK
    12. DOESN'T CLOSE A FILE
    13. UNEXPECTED END OF FILE
    14. DISK SECTOR BUGS AND OTHER LENGTH-DEPENDENT ERRORS
    15. WRONG OPERATION OR INSTRUCTION CODES
    16. MISUNDERSTOOD STATUS OR RETURN CODE
    17. DEVICE PROTOCOL ERROR
    18. UNDERUTILIZES DEVICE INTELLIGENCE
    19. PAGING MECHANISM IGNORED OR MISUNDERSTOOD
    20. IGNORES CHANNEL THROUGHPUT LIMITS
    21. ASSUMES DEVICE IS OR ISN'T, OR SHOULD BE OR SHOULDN'T BE
    22. INITIALIZED
    23. ASSUMES PROGRAMMABLE FUNCTION KEYS ARE PROGRAMMED
    24. CORRECTLY
12. SOURCE, VERSION AND ID CONTROL
    1. OLD BUGS MYSTERIOUSLY REAPPEAR
    2. FAILURE TO UPDATE MULTIPLE COPIES OF DATA OR PROGRAM FILES
    3. NO TITLE
    4. NO VERSION ID
    5. WRONG VERSION NUMBER ON THE TITLE SCREEN
    6. NO COPYRIGHT MESSAGE OR A BAD ONE
    7. ARCHIVED SOURCE DOESN'T COMPILE INTO A MATCH FOR SHIPPING CODE
    8. MANUFACTURED DISKS DON'T WORK OR CONTAIN WRONG CODE OR DATA
13. TESTING ERRORS
    1. MISSING BUGS IN THE PROGRAM
       1. Failure to notice a problem
       2. Misreading the screen
       3. Failure to report a problem
       4. Failure to execute a planned test
       5. Failure to use the most "promising" test cases
       6. Ignoring programmers' suggestions
    2. FINDING "BUGS" THAT AREN'T IN THE PROGRAM
       1. Errors in testing programs
       2. Corrupted data file
       3. Misinterpreted specifications or documentation
    3. POOR REPORTING
       1. Illegible reports
       2. Failure to make it clear how to reproduce the problem
       3. Failure to say that you can't reproduce a problem
       4. Failure to check your report
       5. Failure to report timing dependencies
       6. Failure to simplify conditions
       7. Concentration on trivia
       8. Abusive language
    4. POOR TRACKING OR FOLLOW-UP
       1. Failure to provide summary reports
       2. Failure to re-report serious bugs
       3. Failure to verify fixes
       4. Failure to check for unresolved problems just before release