-
Some Important Questions
-
How much testing have we done?
- Status
- Scope
- Pending (To Do)
- In Progress
- Risks
- Blockers
-
How thorough has our testing been?
- Quick
- Practical
- Deep
-
How effective has our testing been?
- Product Elements
- Quality Criterias
- Coverage, etc.?
-
How much testing is enough?
- For Alpha Release
- For Stable Version
- For Making any Decision
- Are we done yet?
-
Key Components of a Test Report
-
Report Overview
- Summary
- Mission
- Scope
- Key Metrics
-
Status
- Completed?
- In-Progress?
- Test Environment Details
- Version of Product
- Duration of Test Cycle
-
Issue Overview
- Blockers
- Recommended to Fix
- Can be postponed
-
Anything else?
- Change Requests
- Known Issues for Release Notes
-
Test Notes
- Test Process
- Coverage
-
Notable Test Results
- Contains key issues (worth noting)
-
Consists of:
- Issue Title | ALM Link
- Issue Summary
- Risk
- Implications
-
Pitfall:
Surrogate or Proxy Measures
- Traditional test reports often showcase test metrics
- Many of the attributes we wish to study do not have generally agreed methods of measurement.
- To overcome the lack of measure for an attribute, some factor which "can be" measured is used instead.
- These alternate measures are called surrogate measures.
- A widely used opportunity for disaster !
-
Ex: Measuring testers productivity via:
- Bugs
- Test Case Count
- Test Script %
- Pass vs Fail %
-
Pitfall
- Wrong Assumption
- Used because they are easy to count !