QA | tobywatkins1

Regression Testing
1. Define when regressions are needed
  1. Revision control process
    1. Revisions
      1. Topic
      2. The revision control process for regressions is based on dates and time lines. This means regressions must be completed within the time frame of that release, what ever extends this time period is assessed for risk and omitted. For almost no reason will these dates be comprimised.
      3. Topic
    2. Benifits
      1. Topic
      2. The Benefits of this is less pressure on the testing group. The programmers will not have to scramble to fix dead versions. Issues that arrive will be resolved by the next planned released.
      3. Topic
    3. Risks
      1. Topic
      2. The risk here is that if the code fails , all testing will cease until the next version is available. This means that the new features implemented in this version of the code will not be checked until the next revision, possibly pushing back dates.
      3. This also method also has a nasty habit of bloating backlogs, and piling more coding on to the programmers adding pressure
  2. Issue driven process
    1. Issue
      1. Topic
      2. Using the Issue driven process will be based on the progression of the testing group. This means that time lines are generalized to incorporate the assesment of the testing group before the revision is released.
      3. Topic
    2. Benifits
      1. Topic
      2. The positive attributes of working with the issue driven process is that everything planned in that release, including new features will be addressed before the next revision is completed.
      3. Topic
    3. Risk
      1. Topic
      2. Depending on how many issues are found during testing this method could exponentially expand existing time lines, pushing completion dates farther out than intitiall expected.
      3. Topic
2. Define how to automate regressions
  1. Automation paths
    1. Path 1
      1. The first path that can be followed, starts by incorporating an automation period to the test phase.
      2. Risk - will blow out time lines attempting to get the the automation correct
      3. Risk - will need to add an extra step to review the automation
      4. Benefit - will be available for the next regression cycle after the inital revision is completed
    2. Path 2
      1. Topic
      2. The second path that could be traversed is setting up automation after milestones.
      3. Risk - creates a backlog of automation that would need to be completed before the next phase of the application
      4. Risk - There will be no automate regressions for the that entire phase.
      5. Benefit - A sizeable portion of the code has been automated and will allow for faster regressions if that coded phase needs regression
    3. Path 3
      1. Topic
      2. The third path that could be used is to wait till the completion of the project
      3. Risk - No autiomation for the entire life cycle
      4. Benefit - There will be no impacts to the schedule due to latency in the progress of the automation sweet
      5. Topic
  2. Automation tools
    1. tool type 1
      1. QTP - Quick Test Pro
      2. unknown
    2. tool type 2
      1. RFT - Rational Functional tool
      2. 2k
    3. tool type 3
Test Plan
1. Define testing phases
  1. The Phases
    1. Phase 1
      1. Requirements negotiation
      2. If the requirements shake out, then inital test cases will be made based on this information.
    2. Phase 2
      1. Approach Review
      2. Minor interaction with development but being able to know the possible choices to overcome the problem
    3. Phase 3
      1. Design Review
      2. Once the decision has been made the documenation needed from development should be provided
    4. Phase 4
      1. Test Plan Creation
      2. Based on the information the test plan is created
    5. Phase 5
      1. Test plan Review
      2. Development and QA get together to revise the test plan to make sure everything is being addressed
    6. Phase 6
      1. Regression Testing
      2. The regression testing will be completed first to ensure stability of the code. Then the new functionality. (These two phases could be switched)
    7. Phase 7
      1. New Features Testing
      2. Once the new features and regression testing has been completed the results will be given to the development
    8. Phase 8
      1. Testing results
      2. The results will be reviewed with development if everything is satisfied then we will allow it to be released to production or we can move on to the next revision or phase.
    9. Topic
    10. Phase 4.5
      1. Automation Creation
      2. Once the test plan has been created, the tester will then create the automation based on the test plan
    11. Phase 4.6
      1. Automation Review
      2. Development and QA will walk through a demo of the automation, and if it's solid it will be used to run regressions.
2. Define parts of the test plan
  1. The Parts
    1. Topic
      1. Test plans will vary depending on the input from development and what is needed to complete the test of the applications features
      2. Listed here are theoretical 'plans' for this section
    2. Confirguration
      1. This section of the test plan would be if the application has any configuration files that need to be tested or deployed for the application to function properly
      2. Topic
      3. There would be both negative and positive test cases for this section that the system would have to be able to handle and or exit gracefully.
    3. Installation
      1. This section of the test plan would focus on the Install and Uninstall of the program
      2. Topic
      3. The focus here would be to ensure that the system will install the correct files / avoid rewriting exiting files / remove files / remove registry entries
    4. Boundary
      1. This section of the test plan would attempt to go out of bounds by attempting to break the page doing what the client may attempt
      2. Topic
      3. The objective here is to break the page by keying in longer than expected character sets / erroneous character sets / attempting to override drop-downs
    5. Functional
      1. This section of the test plan would allow for all the new functionality to be tested
      2. Topic
      3. At this point in the testing phase, the tester should have the design specs needed to test out the new features that are in the release, making sure that the positive path is validated.
    6. Validation
      1. This section of the test plan will validate the data that was created by the functional tests.
      2. Topic
      3. The user will need to access the database or repository where the data is sent and validate that the information has been inserted correctly
    7. Negative
      1. This portion of the test plan will enable the tester to do malicious things to the code.
      2. Topic
      3. At this point the tester is being a tester, they will attempt to do whatever they can do destroy the normal flow of the product. By keying in special characters, utilizing the browsers features of back and forward to see if things will break.
    8. Third Party
      1. Depending on the code, this section will incorporate browser testing / third party applications / stress testing / performance testing
3. Define process for implementing test plan
  1. Implementing a Test plan
    1. How To:
      1. Implementing the test plan will be a shaky process at the start. The inital test plan will usually be a skeleton
    2. Skeleton:
      1. A skeleton test plan is basically a frame for what will be in the completed test plan, you can think of this portion as an outline to the plan of attack. Usually once the skeleton is vetted out and there is no documentation then the Test Plan Review Proccess takes place.
    3. Skeleton Plan Review:
      1. The test plan review process can take more than one meeting. In this particular review, the development group is asked on a macro level what is missing in the plan. Once this is skeleton has been completed then the tester will go back and add the 'meat' to the plan.
    4. Test plan Review
      1. Once all the information needed to test the code, then once again the development group is brought in and thoroughly go through the test plan looking for holes / redundancy / or anything that is amiss.
    5. *Automation Review
      1. If there is a need for automation, then at this phase after the Test plan review, then the automation is created. Development will give their insight to the coding and pass off this process to the Demo
    6. *Automation Demo
      1. Once the coding has been completed for the automation, then the development and certain stake holders will sit down and 'watch' the demo to be sure that first the code is working and second if something is being done incorrectly through the stake holders eyes.
    7. Test Plan Execution
      1. After all the above is completed the tester will walk through the test plan documenting success and failures of the code.
    8. *Automation Execution
      1. While the 'manual' testing is being completed, the created automation could be running through it's code to ensure that it will pass and fail the sections it's coded for.
4. The testing cycle
  1. Testing cycle
    1. Intro:
      1. The testing cycle is based on the software cycle. Testing would consist of, a cycle of revisions and regressions until the goals for that phase are met.
    2. Alpha phase
      1. Topic
      2. Path
      3. Initial Test plan creation
      4. Revisions / Regressions
      5. Goals for alpha phase met
      6. Topic
    3. Beta Phase
      1. Topic
      2. Path
      3. Test Plan Revisions
      4. Revisions / Regressions
      5. Goals for Beta phase met
      6. Topic
    4. GC Phase
      1. Topic
      2. Path
      3. Revisions / Regressions
      4. Staging / Demo for clients
      5. Goals for Gold Candidate met
      6. Topic
    5. MC Phase
      1. Topic
      2. Path
      3. Regressions
      4. Continue Staging Phase
      5. Goals for Master Candidate met
      6. Topic
    6. Inital Release
      1. Topic
      2. Path
      3. Staging Phase Complete
      4. Production roll out
      5. Topic
    7. Revision / Feature Add
      1. Topic
      2. Path
      3. Production issues found
      4. New features requested
      5. Topic
    8. New Release
      1. Topic
      2. Path
      3. Test plan Revisions
      4. Regressions
      5. Staging Phase
      6. Production roll out
      7. Topic
Knowledge Tree
1. Define if we need a knowledge tree
  1. Knowledge tree
    1. Topic
      1. Why would we need to have a knowledge tree to complete the task at hand?
    2. Pro's
      1. Topic
      2. Topic
      3. Business Logic will be explained and defined in the tree.
      4. Possible Work arounds will be illuminated and the problems that they cover will be known
      5. Anyone can reference this material
      6. Can be used to consider new features being implemented to the code
      7. Topic
    3. Con's
      1. Topic
      2. Topic
      3. Added work to do, to keep the tree updated - support
      4. Tree can be bloated and be hard to access
      5. Topic
2. Leverage the Access program
  1. What should we use if we decide to create a knowledge tree. What should be added?
    1. Knowledge tree
      1. Topic
      2. Programs
      3. Excell
      4. Access
      5. IssueView
      6. wiki
      7. Other
      8. Topic
    2. Content
      1. Topic
      2. What should be added
      3. Business Logic
      4. Definitions of 'options'
      5. Version updates
      6. Interfaces for Third Party
      7. Web Interface
      8. Topic
      9. What shouldn't be added
      10. Bugs
      11. Development documentation
      12. Easter Eggs
      13. Topic
Test Environment
1. Identify tools needed for test environment
  1. Machines
    1. Windows Machine
      1. This machine is to test out the application on a windows setup
    2. *Apple Machine
      1. This machine is to test out the application on the apple platform (future)
  2. VMware
    1. XP
      1. browsers
      2. IE6.0
      3. Safari
    2. Vista
      1. browsers
      2. IE7.0
      3. FireFox3.0
    3. Windows 7
      1. browsers
      2. IE8.0
      3. Chrome
    4. Xp 64
      1. browser
      2. IE8.0 (64)
      3. Safari
    5. Vista 64
      1. browsers
      2. IE8.0
      3. Chrome
    6. Windows 7 Ultimate (64bit)
      1. browsers
      2. IE8.0 (64)
      3. Firefox3.0
  3. *Mobile Devices
    1. smart phones
      1. Windows Mobile
      2. Mobile 6
      3. Moblie 7
      4. HTC Android
      5. Eris
      6. Iphone 3Gs
      7. Blackberry
Bug Tracking
1. Streamline processes for Bug Fixes
  1. I'm a bug from test
    1. Initial Bug Found
      1. step 1
    2. Bug is documented
      1. step 2
    3. bug is fixed
      1. step 3
    4. Bug is tested
      1. step 4
    5. New code released
      1. step 5
  2. I'm a bug from production
    1. Client finds bug
      1. step 1
    2. Tech support reports bug
      1. step 2
    3. QA/Tech support verifies Bug
      1. step 3
    4. Bug is documented
      1. step 4
    5. Bug is fixed
      1. step 5
    6. Bug is tested
      1. step 6
    7. New code released
      1. step 7
  3. An overview for how QA will be integrated into the bug finding process.
2. Modify the bug entering process
  1. The way bugs are generated I think is spot on, the only thing I would change here is how the actual defect is being documented. example below:
  2. Description: Provider - The initial phone field will accept spaces - C Steps: 1. Enter into the Provider page. 2. On the Phone field place in the area code. 3. Once the area code is finished, press the space bar. 4. Now key in the rest of the number. Result: The phone field will cap out and the remaining numbers will be placed into the ext. field
  3. By simply adding this to the existing process will enable the developer to go right to the issue instead of having to play around with the system to get the issue to occur.
3. Bug Matrix / Tracking
  1. Revision 1.0.1
    1. Test section 1
      1. 100
      2. 100
      3. 0
      4. 0
      5. 0
      6. 0
      7. 0
      8. Topic
    2. Test section 2
      1. 20
      2. 10
      3. 10
      4. 0
      5. 0
      6. 0
      7. 0
      8. (4231,4232)
    3. Test section 3
      1. 150
      2. 25
      3. 1
      4. 124
      5. 0
      6. 0
      7. 0
      8. (4233)
    4. Test section 4
      1. 0
      2. 0
      3. 0
      4. 0
      5. 50
      6. 50
      7. 0
    5. Test section 5
      1. 0
      2. 0
      3. 0
      4. 0
      5. 50
      6. 30
      7. 20
      8. (5201)
    6. Totals
      1. 270
      2. 100
      3. 80
      4. 20
      5. 135
      6. 11
      7. 124
      8. 4
  2. This grid below is an example matrix that will be used to not only track the amount of test cases by section, but also what bugs have been placed in due to the testing. As you can see the numbers aren't always the same based on the results of the tests these values will change. The example that may be a bit confusing is the 'blocked' status. When a test cases are set to blocked this means that a particular issue is stopping testing from continuing, the code is essentially broken and no valuable testing can continue. For the 'automated' section, this 'blocked' status is not needed as if a script fails, then it will fail all the tests after if a similar issue is encountered.
Change Management
1. Design/Define change request
  1. Floating Topic
    1. Change?
      1. What change is being proposed? This explanation should incorporate the business gain from this request.
    2. Difficulty?
      1. This section should explain from the developers insight on how much effort would be involved in this change.
    3. Impact?
      1. This section should explain what will be pushed or omitted in that remediation cycle.
    4. Resolution?
      1. This section should be where stake holders sign off on if this change will be put in immediately or but on hold for a later release.
  2. I could create a Change Document in Excell in a few minutes, let me know if this is that path we want to choose and it will be created. Below is what would be put into the Change request.
2. Assess current knowledge base and make recommendations
  1. In my opinion we should leave the current knowledge base alone, and allow tech support to manage it since it's their device for resolving issues on their side. Instead we should utilize the tool we have already started and customize it to be the knowledge tree that we want for development
3. Review and assess IssueView for Change Requests
  1. It is my opinion that we create less work for ourselves. We should use a Word document to call out what we want in a change request and store these documents on the network. The process of these Requests should be a group effort that should be either discussed through and email or a meeting where all parties sign off that this is the new direction going forward.