Test Levels / Levels of Execution / Phases
of Testing
Unit, Integration, System & Acceptance are 4 common test levels.
Unit Testing
Execution of test cases on smallest testable piece of software which is either a
module / unit / component / program. Smallest testable piece of software is tested,
in isolation, with an intention to uncover & find all the deviations in smallest piece
software.
Unit testing is accomplished within the development environment by developer
class people.
Functional testing/ Non-Functional testing/ Structural testing/ Testing Related to
Changes is possible at Unit Level. However, Unit testing is dominated by structural
testing.
Robustness test, Dynamic analysis, Code reviews for control flow analysis, data flow
analysis, as well as Code coverage are common tests at Unit level. Stress test is not
done at this level.
Component Integration testing
When more than one unit is combined & checked to expose errors that might exist
in interfaces or in communication between them.
Integration of components may follow a strategy of either Big Bang Integration or
Incremental integration. Big bang integration is when all the units are integrated
at once & tested for interfacial & communication errors. Whereas Incremental
integration combines available components at the beginning & incrementally
combines more units as & when they are available.
Incremental integration strategy is preferable as the isolation of defects is much
easier & further time management is better.
Incremental integration may be accomplished either as Top-down integration
approach or Bottom-up approach or a mixed approach called Hybrid / Sandwich
approach.
Top-down approach is possible when the main module or the modules (at the top of
the Control flow / architecture diagram) are available for integration & more units
that fall below such top modules are incrementally integrated.
Bottom-up approach is possible when the bottom modules are available & modules
that fall above it are incrementally integrated.
A mixed approach may involve random integration involving some of the top &
some of the bottom modules incrementally integrated.
Incremental integration requires temporary programs such as Stub or Driver to
facilitate test execution.
Stub: is a temporary program that replaces an unfinished Called function or Called
module. Stub reacts when called with a message & returns back the control
Driver: is a temporary program that replaces an unfinished Calling function or
Calling module. Driver can initiate control
A test execution environment which facilitates test execution with the help of Stub /
Driver is referred to as Test Harness. Sometimes a Stub / Driver is referred to as
Test Harness.
Integration testing is executed in development environment by developer class
people or Specialist integrators. Any type of test is possible at any level.
System Testing
When all the components are integrated and deployed into a customer expected
environment & subjected to rigorous testing, such testing is called System testing.
System testing is with an intention of uncovering as many defects that exist in the
system as a whole. Defects are then referred to the development & once correction
is proper then the quality of the system gets improved.
A rigorous evaluation of the system from one end to another involving many
negative tests is how system testing can be characterized.
System testing, also called End to End testing, is carried out by Independent testers
in a customer expected environment.
Any type of test may be done at this level & must include evaluation of Functional &
Non-Functional characteristics.
The source artifacts for System testing can be SRS/ FRS/ Use Cases/ Business
processes.
Acceptance Testing
When a system is assessed as to whether it meets user purpose & reasonable
expectations such a test is referred to as Acceptance testing.
Acceptance testing is the responsibility of Users / Client & doesn’t involve Negative
testing.
The source artifacts for Acceptance testing can be BRS/ Use Cases/ Business
processes.
Acceptance testing done at Development company environment is known as Alpha
test whereas Acceptance test done at Client / User environment is known as Beta
test.
Alpha test is also known as Factory test & Beta test is known as Field test or Site
testing.
When acceptance test is done as per contractual criteria such a test is called
Contractual acceptance test
When acceptance test is done as per Regulatory standard such a test is called
Regulatory acceptance test
UAT: User Acceptance Test is to assess whether the software meets user needs &
reasonable expectations.
OAT: Operational Acceptance Test is to assess whether UAT approved software can
be made live. In operationalizing a software criteria such as training to users, data
backup & restore procedures, security permissions, disaster recovery procedures
may need to be defined & achieved. Network admin of the client verifies if such
criteria is satisfied before making the application live.
System Integration testing:
When our system combined with a third party component in forming the total
solution then an additional test level is involved which is called System Integration
Testing.
An external software component called C.O.T.S (Commercial Off The Shelf
Software) + our system = Complete Solution
It is required to assess if system of systems has any errors at interfaces or in
communication between them.
Maintenance Testing
When a live system (released system) is tested such testing is referred to as
Maintenance testing.
A released system may need to be tested because of the following triggers,
1. Modification: to correct failures noticed by the client or to implement change
request made by the client or implementation of additional features (enhancement)
2. Migration: change in the environment from one to another
3. Retirement: existing application is being retired & possibly being replaced with
a new technology application. Such about-to-be-retired application needs to be
evaluated as to whether the data can be safely backed up / restored, data can be
migrated to another technology, or data can be retained for longer duration.
Workflow:
Customer reports an issue >> Support staff >> Support staff escalates the issue to
higher level called CCB as they are unable to address this issue >> Change Control
Board, comprising of the people who have worked on this project, shall analyze
for what exactly needs to be changed, extent of the change & impact of proposed
change on other aspects of software (called Impact analysis). If CCB approves the
changes >> Such changes are implemented by developers >> Such changes are
tested by testers >> Patch / Update / Hot fix / New version is released.
Maintenance testing is dominated by Regression testing & influenced by Impact
analysis study.
of Testing
Unit, Integration, System & Acceptance are 4 common test levels.
Unit Testing
Execution of test cases on smallest testable piece of software which is either a
module / unit / component / program. Smallest testable piece of software is tested,
in isolation, with an intention to uncover & find all the deviations in smallest piece
software.
Unit testing is accomplished within the development environment by developer
class people.
Functional testing/ Non-Functional testing/ Structural testing/ Testing Related to
Changes is possible at Unit Level. However, Unit testing is dominated by structural
testing.
Robustness test, Dynamic analysis, Code reviews for control flow analysis, data flow
analysis, as well as Code coverage are common tests at Unit level. Stress test is not
done at this level.
Component Integration testing
When more than one unit is combined & checked to expose errors that might exist
in interfaces or in communication between them.
Integration of components may follow a strategy of either Big Bang Integration or
Incremental integration. Big bang integration is when all the units are integrated
at once & tested for interfacial & communication errors. Whereas Incremental
integration combines available components at the beginning & incrementally
combines more units as & when they are available.
Incremental integration strategy is preferable as the isolation of defects is much
easier & further time management is better.
Incremental integration may be accomplished either as Top-down integration
approach or Bottom-up approach or a mixed approach called Hybrid / Sandwich
approach.
Top-down approach is possible when the main module or the modules (at the top of
the Control flow / architecture diagram) are available for integration & more units
that fall below such top modules are incrementally integrated.
Bottom-up approach is possible when the bottom modules are available & modules
that fall above it are incrementally integrated.
A mixed approach may involve random integration involving some of the top &
some of the bottom modules incrementally integrated.
Incremental integration requires temporary programs such as Stub or Driver to
facilitate test execution.
Stub: is a temporary program that replaces an unfinished Called function or Called
module. Stub reacts when called with a message & returns back the control
Driver: is a temporary program that replaces an unfinished Calling function or
Calling module. Driver can initiate control
A test execution environment which facilitates test execution with the help of Stub /
Driver is referred to as Test Harness. Sometimes a Stub / Driver is referred to as
Test Harness.
Integration testing is executed in development environment by developer class
people or Specialist integrators. Any type of test is possible at any level.
System Testing
When all the components are integrated and deployed into a customer expected
environment & subjected to rigorous testing, such testing is called System testing.
System testing is with an intention of uncovering as many defects that exist in the
system as a whole. Defects are then referred to the development & once correction
is proper then the quality of the system gets improved.
A rigorous evaluation of the system from one end to another involving many
negative tests is how system testing can be characterized.
System testing, also called End to End testing, is carried out by Independent testers
in a customer expected environment.
Any type of test may be done at this level & must include evaluation of Functional &
Non-Functional characteristics.
The source artifacts for System testing can be SRS/ FRS/ Use Cases/ Business
processes.
Acceptance Testing
When a system is assessed as to whether it meets user purpose & reasonable
expectations such a test is referred to as Acceptance testing.
Acceptance testing is the responsibility of Users / Client & doesn’t involve Negative
testing.
The source artifacts for Acceptance testing can be BRS/ Use Cases/ Business
processes.
Acceptance testing done at Development company environment is known as Alpha
test whereas Acceptance test done at Client / User environment is known as Beta
test.
Alpha test is also known as Factory test & Beta test is known as Field test or Site
testing.
When acceptance test is done as per contractual criteria such a test is called
Contractual acceptance test
When acceptance test is done as per Regulatory standard such a test is called
Regulatory acceptance test
UAT: User Acceptance Test is to assess whether the software meets user needs &
reasonable expectations.
OAT: Operational Acceptance Test is to assess whether UAT approved software can
be made live. In operationalizing a software criteria such as training to users, data
backup & restore procedures, security permissions, disaster recovery procedures
may need to be defined & achieved. Network admin of the client verifies if such
criteria is satisfied before making the application live.
System Integration testing:
When our system combined with a third party component in forming the total
solution then an additional test level is involved which is called System Integration
Testing.
An external software component called C.O.T.S (Commercial Off The Shelf
Software) + our system = Complete Solution
It is required to assess if system of systems has any errors at interfaces or in
communication between them.
Maintenance Testing
When a live system (released system) is tested such testing is referred to as
Maintenance testing.
A released system may need to be tested because of the following triggers,
1. Modification: to correct failures noticed by the client or to implement change
request made by the client or implementation of additional features (enhancement)
2. Migration: change in the environment from one to another
3. Retirement: existing application is being retired & possibly being replaced with
a new technology application. Such about-to-be-retired application needs to be
evaluated as to whether the data can be safely backed up / restored, data can be
migrated to another technology, or data can be retained for longer duration.
Workflow:
Customer reports an issue >> Support staff >> Support staff escalates the issue to
higher level called CCB as they are unable to address this issue >> Change Control
Board, comprising of the people who have worked on this project, shall analyze
for what exactly needs to be changed, extent of the change & impact of proposed
change on other aspects of software (called Impact analysis). If CCB approves the
changes >> Such changes are implemented by developers >> Such changes are
tested by testers >> Patch / Update / Hot fix / New version is released.
Maintenance testing is dominated by Regression testing & influenced by Impact
analysis study.
No comments:
Post a Comment