Tips for Obtaining Objective Test Evidence: Document What You Observe

Q: When testing an Interactive Voice Response System (IVRS) for deployment, what method would you recommend for obtaining objective evidence of the failure test steps? These are steps where you intentionally enter erroneous data to test the IVR’s edit checks. Problem is, the system won’t accept them by design, and therefore won’t trigger a fax report.

For the good data test steps, we receive a report from the IVRS via fax, so we’re good there. For the bad data test steps, I told them our subjective results (i.e., “As Expected/Pass”) would have to be sufficient, because that’s all we could do. Any suggestions?

A: I think you can go a step beyond the subjective (“As Expected/Pass”) determination without driving yourselves crazy in the effort to obtain objective evidence. When the tester has been instructed to try to enter erroneous data, it would be easy to have the tester record what actually happened — for example, that data were typed into the field and were unable to be saved (perhaps the field went blank?).

It’s difficult for me to come up with other examples of objective evidence without understanding exactly how the system has been designed and how the tests have been set up. But the key point is the tester can and should write down what was actually observed in terms of the results of the test step, instead of your having to rely solely on the tester’s subjective judgment.

Answered by Gordon B. Richman, Assistant General Counsel, Quality and Regulatory Affairs, for Beckman Coulter and former Vice President of Strategic Compliance Consulting at EduQuest.


Leave a Reply

Your email address will not be published. Required fields are marked *