The recent discussions about checking vs testing have been interesting. My spontaneous response, here, looked at the label (or container) of a test/check but also alluded to the fact that an automated batch of tests are much more than checks.
Yesterday, I saw a clarification of checking from Michael Bolton, here, and a tweet from James Bach:-
Did I do this? Well, if you accept that you wouldn't run a test/check for no reason then the answer is yes.The other item: An example. I was thinking of a re-running of test cases - in my view this requires the objectives of running those cases to be understood beforehand.
In this respect, the title of the post, sapient checking, is nonsense.
Thinking before Testing?
I'm still of the view that all testing requires thought beforehand - otherwise how can you interpret results and try to help with their meaning? This has several implications.
One implication is that any automated batch of test cases to be used for re-running should always go through a selection process (review for relevance and effectiveness.) I know this might sound radical to some people but it shouldn't be!
Suppose an existing/old test case is to be executed to ensure/check/verify that a new feature has not negatively impacted the existing feature?
Possible reasoning that was an input to the selection of the test case:
1. If the old feature was expected to be impacted then the test case would be opened for review/modification (I'm not distinguishing between new design and maintenance here.)
2. If the old feature had no expected impact then the test case is expected to "work", as is, then that's the measure of evaluation to make this a test. (The underlying software of the product has changed - the new feature - and the same inputs and outputs are to be verified in this new configuration/build.)
3. If we don't know if it's reason 1 or 2 then it may be for one of two main reasons: 3a) It's an exploratory/learning exercise -> testing, 3b) It's a test case that has "always" been run or never been reviewed for relevance -> time for reassessment?
Room for Testing?
For old/legacy test cases then the tester is making an assumption/assertion that a test case (or group of test cases) should work (on a new build/configuration/env.) The tester tests that hypothesis.
It's very important to keep the test base up-to-date and relevant! Filter out redundancies and keep them under review. In terms of legacy cases this should be budgetted as part of the maintenance of the product.
All of this work with legacy test cases is what I consider testing - it isn't anything without the thought process to guide it.
The problem with trying to define checking is that it is so tightly intertwined with testing that I don't think it adds value into the communication about a product or project.
If, as a tester, you're trying to help your project understand the risks associated with a project and communicating issues found then I don't think there is any need to say x issues have occurred during testing and y issues have been observed during checking.
I can think of checking as an element of testing, just like a test step might be an element of a test case. But on its own there is no value in communicating it to stakeholders.
I think there have been many communication problems tester<->non-tester in the past (and probably still continue today) - the information needs to be tailored to the audience, which is a great skill for a good tester!
I can't see the example where a test/check can be selected without any thought process. To distinguish a group of test steps, under a given context, as either a check or a test is totally unecessary in my working environment.
Keeping the storytelling good and the quality of the reporting and feedback high and consistent will lead to reduced misunderstandings and communication improvement. I know, easier said than done!
I suspect communication problems (and not a lack of terminology) have been the root causes for some of the cases where a distinguishment between testing and checking has perceived to be needed.
Checking without sapience doesn't hang together. Why would you do something without a purpose?
This post is my effort to understand the discussion - using my combination of learning, critical thinking and past experience.
What does your experience and analysis of the discussions tell you?