When an application environment is built, they are built in many steps (e.g., hardware installs, operating system install, SAN space allocation, database setup, etc.), and there will be tests along the way for each step. Most of technical steps go through vendor recommended or the company specific tests to validate each component of the environment. When all the infrastructure components are complete, a “shakedown” test should be performed.
The purpose of the “shakedown” test is to validate whether all the infrastructure components are setup to work together correctly, so, the application test (aka business acceptance test) can start.
Once an application (and its severs) pass the shakedown test, the application can start. The application test should have test scripts developed and the actual test should follow the test scripts exactly. All the actual test data entry and results should be logged. There is a risk of potentially corrupting the production data, if the test environment and its control is not properly setup. If this situation occurs, logged test transaction data can be used to recover the corrupted data.
Perform all appropriate testing as defined in the test strategy and test plans using the test scripts and following the defect management process
Defect Management Process Flow
The tester, during the course of executing tests, might encounter defects. These defects will be logged using the defect management report spreadsheet.
The status of the defect will automatically be set to “Open” once the defect has been created.
The following information will be captured at the point the defect is encountered.
- Test Id – the reference by which the test is identified.
- Summary – a brief description of the defect
- Severity/Priority – e.g. Major, Minor, etc.
- Status – the current status of the defect (Assigned, Closed, In Progress, Open, Reopened, resolved)
- Due Date – date the fix is required (if applicable)
- Component – e.g. function, sub-system, or system
- Affects Version – version of code/system being tested
- Reporter – defaults to the person logged on
- Assign to – name of the individual or team
- Wave – Which migration wave does this affect
- Environment – environment in which the defect was encountered
- Description – a detail description of the defect
- Attachments – supporting evidence (screen shots, data, actual results, etc)
- Root Cause – used for reporting and prevention of repeated defects
Review, Assign, Defer or Reject
During this stage open defects will be reviewed in order to assess and confirm the impact (priority) and validity. Defects that are agreed as genuine will be assigned to the relevant team for resolution. Defects that have been raised in error will be closed accordingly. Defects that are deemed to be duplicates will be linked to the relevant defect and closed. Defects that are deemed to be change requests will be updated accordingly and linked to the relevant Change Request (where applicable). The valid defect will be assigned to the relevant team for resolution.
During this stage assigned defects will be reviewed, resolved and re-tested accordingly by the relevant support team within the relevant environment.
In the following instances progress will stop;
- the defect cannot be reproduced, or
- more information is required, or
- the assignee feels that the defect is not a defect and that the system / component is working as designed, or
- The defect has been assigned incorrectly.
If it is agreed that the system / component is working as designed, the defect will be updated (with resolution details) and closed as “working as designed” as agreed.
If it is agreed that the defect has been incorrectly assigned, the defect will be re-assigned to the relevant team / individual.
Upon successful resolution (fixed and retested by the relevant team in the relevant environment), the defect is passed back to the tester for re-test.
Once the defect has been resolved (fixed), packaged into a release and delivered into the relevant testing environment, the tester will re-test the defect. Upon successful re-test the tester will close the defect.
If the defect fails re-test, the defect will be updated with additional information and re-assigned back to the relevant team (or individual) for further investigation.
If, as a result of re-test, a further defect is encountered a new defect will be raised.
As well as re-testing the fix, a level of regression testing should be carried out. This however, is dependent on the impact of the defect. That is, regression testing should be carried out in all cases where a Major defect has been encountered. For defects of a Minor nature, the level of regression testing to carry out may be minimal (or not required) depending on the level of impact of the defect.
Below are criteria to use to select test candidates for regression testing:
- Functionality that supports a high volume of data
- Additional information captured at this stage is as follows:
- Assign To
- Update Comments
During the final stage, following successful retest and closure, the defect is assessed in terms of its impact to other areas (if any). The defect can, at this point, be reopened if it is deemed that the resolution is incorrect or incomplete.
- Execute infrastructure test scripts.
- Execute application test scripts.
- Record all identified defects and their resolution.
- Update IT room migration data repository with test results.
Hints and tips
- A dedicated defect manager could be required who is familiar with defect tracking tools such as quality center.
- Resolver groups need to be clearly defined with an agreed hierarchy of expertise.
- Be careful not to overwhelm the technical infrastructure team with issues such as access and authentication.
- Test completion reports