Saturday, June 18, 2011

Test to Learn, Test to Debug, Test to Evaluate

The lot of discussion is going on about the differences between Systems Integration, Systems Testing, Development Testing, Operational Testing etc. A lot of confusion... totally unnecessary in my opinion.

The problem with the above definitions that they reference the types of testing to the methods, responsibility etc. but not to the purpose.

I'd like to propose more natural (for me) distinction - Test to Learn (or to Discover), Test to Debug (or to Solve Problems) and Test to Evaluate (or to Judge).

Learning is the process of acquiring knowledge that previously was not in the posession of the learner. The testing that is focused on learning is always explorative, with the path that may wind and twist and loop. More than that, Test to Learn will never end - there will be always missing knowledge.

Debugging is the process of finding anomalies, investigating the causes and fixing the system. Integration is essentially System Debugging.

Evaluation is the judging the system against some criteria in order to accept or reject some proposition. Testing is the activity that may be a part (but not the whole thing) of either the Debugging or the Evaluation. (System Testing is essentially System Evaluation.

When the managers say "Start Testing. When you'll find severe problems, investigate them and change the system. When the problems are not severe - proceed" they ask to start in the Evaluation Mode and change to Debugging Mode amidstream. Start as ST and change to SI if needed. No wonder that it's difficult for me to explain the difference between SI and ST - the managers frequently perceive the pair-wise encounters of some sub-systems that are done as part of sub-system testing combined with interface checking as the sufficient SI activity and try to drive the project into ST mode without conclusion of all SI activities such as E2E thread testing - they afterwards wonder why the ST phase takes as long as SI+ST phase that was proposed in the initial project plan and rejected as "too long ST".

I'd be delighted to forfeit talking about "System Testing" or "System Integration" and change to "System Debugging" (Subsystem Debugging, Interface Debugging, Whole System Debugging") and "System Evaluation" (or "System
V&V").

Learning, Debugging and Evaluation are not just Testing - there is a lot more that's going on - discussions, arguments, reading, eriting, thinking - but when the Testing is concerned the disctinction of the type of Testing by purpose has its merits and for me is more useful.

No comments: