Queuing for the ride (anticipation)
A short post today triggered a lot of thoughts - just before going to bed, hence the need to get them off my chest - or at least unburden my "concious" mind before going to bed...
Paul's post, here, triggered a whole whirl of questions in my head and I started writing a comment on his blog.
Unfortunately, I noticed that my comments were turning into double the length of the post and thought it a bit unfair to unload on his site... (I had @testobsessed's comment about doing the same in my head.)
The thing that caught my attention was the idea of a test plan being judged afterwards as being sound or optimistic.
The ride begins
I've written many formal and informal test plans - some have been very accurate and some have been way off. In most cases I've tried to reflect on what led to them being good, bad or just ugly! I will leave the various reasons for another post (it's not the point of this post!)
With regard to planning assumptions being "sound" or "optimistic" I feel some words of caution are necessary.
Planning assumptions and being able to get guidance on when the test project will finish are very useful tools - or at least the PM likes to know when the testing will finish (potential for mental detour here!) However, the plan is only as good as the feedback into the plan, whether this is an iterative refinement of the plan or waiting until many of the unknowns are resolved before "fixing" the plan...
In both of these cases there is no guarantee that some "must work" feature is not found wanting until all the "high prio/must work" test cases are executed successfully (I'm being simplistic for the sake of argument - there are other potential show-stoppers also.)
Note, I'm not making any allowance for exploratory cases here (for simplicity), which I would normally allow for.The point is that we can't know everything about the nature of the product from a planning certainty perspective (unless the plan is not fixed until we know nearly everything about the product) to determine afterwards whether the plan was sound or optimistic.
If we don't set the plan (point at when it's not updated anymore) until all the unknowns about the product are known (this is after we've concluded testing of the product), then what's the point of the plan?The plan might be fine but miss a fault found 6-months down the line that should reasonably have been found in your test scope. However, the scope was too restrictive or "defective" in some other way.
When the testing is finished it might look like the plan was "sound" - that you made all the right assumptions...Or the plan could fail due to some unforeseen issue being found late in the day (for whatever reason) - technically if it's a late req change or new 3rd-party-product in the baseline and then one might consider that the plan should change to reflect this. But, it could also be a basic interaction problem that takes longer to fix than your planned execution time (which you or the developers/architects might not be able to predict straight-away)...
In any of these cases it would be harsh to consider the plan as being optimistic if it's not met due to conditions that are not under the plans "control".
What to do?
So, does this mean there's no such thing as a good or bad plan? No, of course, bad plans exist that do not take into account vital factors needed to complete test execution.
Bad plans are not all due to bad luck and good plans are equally not all due to good luck.
My point is that a timescale where the exit/release criteria does not meet those set out in the plan does not necessarily make it a bad plan or that the plan was optimistic.
The project stakeholder may change those criteria at any time - or indeed ignore the criteria when a release decision is needed...
Conclusion
Sometimes there are factors outside the control or purview (I've recently seen In The Loop :)) of the test plan that do not make it a bad test plan or that the assumptions for the plan are incorrect or optimistic.
Photo finish
Anyway, many thanks to Paul for triggering this latest mental roller-coaster ride for me. It was fun, I'm a bit giddy, am looking forward to the photo as I reached the bottom of the fall, but now I can go to bed, mind unloaded :)
What triggered your last mental roller coaster ride?
No comments:
Post a Comment