Standards, Straw Man Arguments and Superstition

I listened to a webinar in November 2013, ref [1], about the testing standard, "ISO 29119".

At the time I made some notes on the recording, but there was one item that particularly struck me as slightly intimidatory. However I happily assigned it to my mental trash bin and just remembered the meta-labels, "Straw man, scare tactic".

I had forgotten about the recording... until today when I saw this on twitter from James Christie:

james_christie @michaelbolton What happens when the press asks your CEO, "did you follow standards?" when there's been a fiasco? 2014-05-28 15:33

Note: These were not James' words.

Gut feeling: Straw man argument!

Why?

1. Why would the press be asking about a software testing standard when there is no evidence of the testing standard "working" or preventing similar "fiasco's"? It seems equally plausible for the press to ask if the omens in the CEO's tea leaves were followed!

2. If you have a CEO, that's worth his salt₁, and there has been a fiasco (presumably a software catastrophe), then I'd be very surprised if the thing that's highest on his mind is some internal witch-hunt.

He's probably gonna:

1. Put people on fixing the issue
2. Put people/teams on liaising with the customer (if it's one big and, understandably, angry one)
3. Put people on understanding what went wrong
4. Understand if the company did what they thought was right / good enough

I'd be surprised if a company that used technically competent people would have the/a root cause being that a template (or group of templates / standard) was not followed - especially if there is no empirical study (evidence) to demonstrate how the standard (template) might avert such a fiasco.₃

If a catastrophic failure has been found to be caused₂ by not following a template then I would love to hear about it.

I've been in those discussions with responsible managers and execs - and (maybe this is my good fortune of dealing with competent managers)₄ they're more interested in that the technical influential/responsible people do what is right / good enough for the needs of the business, rather than be concerned about being fully or partially compliant to a particular template.₅

Yes, I'm not naive, I know some people, projects and company cultures follow some template as an ass-covering exercise. I've been there.₁

So... The straw man argument is constructing a need to explain whether or not a particular template is used, when the need for that template is not established.

Reasoned argument?

Ok, so it was time for me to revisit the webinar, where at timestamp ~33-34mins, one hears:

"If you are still doubtful about using these standards, then I have a question for you.  

Can you afford not to? 

Imagine you're responsible for the testing of an important application. It could even be business critical, or safety critical, and something goes noticeably wrong with the application in use.  

Even if really good testing could’ve missed the bug, how easy will you find it to explain to the business your testing missed the bug and no, your test processes do not comply with international testing standards; that they are the ones we've used for years and no, you don't have them fully documented and justified.  

So, can you afford not to use them?"

First impression: You should follow a template.... just in case.

Second impression: Slightly sinister tone. 

Ok, let's take a closer look. Closer inspection:

1. The first two sentences are plain rhetoric, trying to establish doubt in the listener - doubt due to not following a standard. This is "rhetoric" because there's no argument/justification about why a standard would help you avoid this situation.

2. "if really good testing could’ve missed the bug" - good testing can miss bugs. This is not necessarily anything to do with a process. Good processes can miss bugs.

3. "how easy will you find it to explain to the business your testing missed the bug and no, your test processes do not comply with international testing standards" - this statement is intending to sow doubt. But the international testing standard has no evidence of being useful, appropriate or a proven track record.

I could go on and pick it apart, displaying the parts which are negative evaluations, argument precursors etc, but you get the picture. It says very little more than an innuendo of repercussions if you don't use the standard.

In Summary

• If someone tries to scare you, or generate doubt and worry, ask to see the evidence, the track record, the money. Look out for straw man arguments, misleading and off-the-point comments and allusions to superstition or unjustified belief.
• Successful CEO's don't usually look for scape goats in their staff, don't usually follow checklists without a good business reason. 
• For me the jury is still partially out on the new software testing standard - I will publish some thoughts on its content at a later stage. But, just because it's been labelled a standard does not make it comparable to other ISO or IEEE standards where interoperability or conformance to a standardised result is usually the goal - here the goal seemed to be to create a standard (job done! Is that the business case? WTF! )

Notes
1. If you have a CEO (or responsible manager) that doesn't trust the technical competence of the organisation then either (1) change the organisation, or (2) change organisation. And Yes! I have done both in the past!

2. Note, if a template is intended to replace competence then this should be declared also. (This puts an additional burden on the template, guidelines and usage of the template.)

3. The existence of a standard, template or checklist does not stop mistakes in following such a checklist

4. Competent managers (in my experience) usually do not chase paper trails to a template, and usually trust their staff. I'll give some tips on how to deal with managers that are tempted by reliance on checklists over competent staff in a separate post.

5. Businesses tend to adopt (or use) the standards that make business sense, rather than adopting a standard purely because there is a standard.

References
[1] Eurostar webinar: Webinar 75: ISO 29119 - The New Set of International Standards on Software Testing