Yet more Monty Python & Software Testing

I drift in and out of Monty Python analogies for Software Testing now and then. Here's a previous reference <link>.

Whilst falling asleep the other night I remembered a Phil Kirkham post about Monty Python and my comment on it, so I thought I'd indulge myself and do a little bit of Monty Python - Software Testing analogising(?) - just for therapy :)

Here's the "Life of Brian" based comment I made:

Coordinator: Crucifixion? 

Prisoner: Yes. 

Coordinator: Good. Out of the door, line on the left, one cross each. 

[Next prisoner] 

Coordinator: Crucifixion? 

Mr. Cheeky: Er, no, freedom actually. 

Coordinator: What? 

Mr. Cheeky: Yeah, they said I hadn't done anything and I could go and live on an island somewhere. 

Coordinator: Oh I say, that's very nice. Well, off you go then. 

Mr. Cheeky: No, I'm just pulling your leg, it's crucifixion really. 

Coordinator: [laughing] Oh yes, very good. Well... 

Mr. Cheeky: Yes I know, out of the door, one cross each, line on the left. 

Or, as applied to testing...

Coordinator: Scripted test?

Tester: Yes

Coord: Good. Other there on the shelf, one scripted test case each.

[Next Tester]

Coordinator: Scripted test?

Tester: Er, no, an exploratory approach please.

Coord: What?

Tester: Yes, they said I could come and do some testing with my eyes open.

Coord: Oh, I say, that's sounds very nice. Well, off you go then.

Tester: No, I'm just joking my PM gets scared if we don't follow the script.

Coord: Oh, well in that case...

Tester: Yes I know, over there, one scripted test case each.

Life of Brian is a rich source:

Spectator I: I think it was "Blessed are the cheesemakers". 

Mrs. Gregory: Aha, what's so special about the cheesemakers? 

Gregory: Well, obviously it's not meant to be taken literally; it refers to any manufacturers of dairy products.

Or, as applied to testing...

Tester I: I think it was "Blessed are the certified".

Tester II: What's so special about the certified?

Tester III: Well, obviously it's not meant to be taken literally; it refers to any arbitrary label or categorisation.

There's lots of potential in the Meaning of Life too:

Three Project Managers (PM) and a management consultant (MC) discuss the state of affairs:

PM#1: Ah! Morning Perkins.

PM#2: Morning.

PM#1: What's all the trouble then?

PM#2: Test reports in disarray. During the night.

PM#1: Hm. Not nice numbers eh?

PM#2: Yes.

PM#1: How's it feel?

PM#2: Stings a bit.

PM#1: Mmm. Well it would, wouldn't it. That's quite a lot of extra information you've got there you know.

PM#1: Yes, real beauty isn't it?

All: Yes.

PM#1: Any idea how it happened?

PM#2: None at all. Complete mystery to me. Woke up just now... one piece of detailed analysis too many.

:

PM#1: Hallo Doc.

MC: Morning. I came as fast as I could. Is something up?

PM#1: Yes, during the night old Perkins (PM#2) had his test progress reports disrupted.

:

MC: Any headache, bowels all right? Well, let's have a look at this test report of yours then. [Looks at sheet] Yes... yes... yes... yes... yes... yes... well, this is nothing to worry about.

PM#2: Oh good.

MC: There's a lot of it about, probably a virus, keep warm, plenty of rest, and if  you're reporting progress remember to stick to statistics.

PM#2: Oh right ho.

MC: Be as right as rain in a couple of days.

PM#2: Thanks for the reassurance, doc.

:

MC: Jolly good. Well, must be off.

PM#2: So it'll just sort itself out then, will it?

MC: Er... I think I'd better come clean with you about this... it's... um it's not a virus, I'm afraid. You see, a virus is what we doctors call very disruptive. So it could not possibly have made a positive impact on the quality of these reports. What we're looking for here is I think, and this is no more than an educated guess, I'd like to make that clear, is some multi-cellular life form the genu *bonus extertus*. What we management consultants, in fact, call a good tester.

All: A good tester...!!

:

PM#3: A good tester - on this project?

PM#1: Hm...

PM#3: A good tester on this project...?

PM#1: Ah... well he's probably escaped from a zoo.

And remember don't be complacent in testing:

A tester has been asked for his certification in testing:

Tester: "I didn't expect a kind of Spanish Inquisition." 

Certification advocate: "Nobody expects the Spanish Inquisition! Our chief weapon is surprise...surprise and fear...fear and surprise.... Our two weapons are fear and surprise...and ruthless efficiency.... Our *three* weapons are fear, surprise, and ruthless efficiency...and an almost fanatical devotion to the certification syllabus.... Our *four*...no... *Amongst* our weapons.... Amongst our weaponry...are such elements as fear, surprise.... I'll come in again."

I was working on the dead parrot and test tool vendors, but it got messy - so I'll stop there!

Test Case Counting Reflections

James Christie wrote an interesting piece on some issues with test case counting, here. I started writing some comments but realised it was turning into a post in itself. So, here it is...

I recognise the type of scenario that James writes about - of a project manager, stakeholder or even fellow tester being obsessed with test case statistics.

I'm even writing about this from the other side - why the non-tester (manager / stakeholder) might be interested in tc counting... But I'll leave that to another post.

I think it's a case of the test case being the least common denominator - it's an easy measure to ask for - to get a handle on "what's happening". The key for the tester, though, is to convey the value (lack of, or limited) of such a number/measure (although the first step for the tester is to understand the problems and limitations with tc counting..)

What are the figures saying, but also what pieces of the picture are they not showing?

I purposely tell my teams not to talk to me in terms of tc numbers - this is quite challenging at first - but once they understand the information I'm interested in (the feature, aspects of coverage, risk areas, limitations with the model/environment, built-in assumptions and other aspects of 'silent evidence') then it actually generates a lot more creative and searching thinking (I believe.)

How might a conversation with a stakeholder on test case statistics go? Let's do a thought example and see how it might go and problems it may hide/show.

Stake Holder: "How many TC's have been executed, and how many successfully?"

Tester: "Why?"

SH: "So I can gauge progress..."

T: "What if I said all but 1 TC had been executed successfully?"

SH: "Sounds good. What about the 1 remaining one?"

T: "That /could/ be deemed a show-stopper - a fault in installation that could corrupt data"

SH: "Ok, have we got anyone looking at it?"

T: "Yep, 2 of the best guys are working on it now."

SH: "Good"

T: "But..."

T: "It might cast a question mark over a bunch of the 'successful' TC's that were executed with this potential configuration fault"

SH: "Mmmm, what's the likelihood of that?"

T: That's something we're investigating now. We've identified some key uses cases we'd like to re-test, but it really depends on the fix and extent of the code change.

SH: Ok, thanks.

T: During our testing we also noticed some annomalies or strange behaviour that we think should be investigated further. This would mean some extra testing.

SH: Ok, that we can discuss with the other technical leads in the project.

The stakeholder has now got a (more) rounded picture of the problem/s with the product - he's also getting feedback that it's (potentially) not just as simple as fixing a bug so that the last remaining TC will work. Concerns have been raised about the tests already executed as well as the need for more investigation (read maybe more TC's) - all this without focussing on test case numbers

Not all examples will work like this, of course - but maybe it's a case of not talking about test cases, or talking about test cases and saying this is only a fraction of the story.

There is a whole different angle about getting the stakeholders to understand the limitations about units that are called test cases. One of James Bach's presentations comes to mind, here.

Got any strategies for telling the whole story rather than just the numbers?

Creative Thinking Challenge

I've been reading about a form of alternative education (here) recently, and been thinking about the need for creativity (research) in my daily work. I've also been reading about approaches to thinking, and creative thinking.

I saw an example of creative thinking recently to add quickly the numbers 1 to 10, or 1 to 100. The method was quick but not necessarily intuitive - which is part of the point.

I started thinking about this to see if I could adapt the method to add the even numbers between say 1 and 300. I did the mental arithmetic fairly quickly and then checked my working with pen and paper afterwards.

There have been some interesting tester challenges doing the rounds recently, and I thought that a thinking challenge was in order.

So, the challenge is to add the even numbers (2, 4, 6,...) from 1 to 300.

Bonus points if you can do it in less than 30 seconds! Less than 10 seconds if using a calculator. (I'm not counting alternative thinking time here - that I'll leave to you to work on.)

If you want to post a comment with the answer here then you'll need to show your thinking!
(I'll delay comment publication a few days in case there's any latecomers that wants to try it - without being tempted by the comments.)

There will be a follow-up post - where I identify the book and some other things I've learnt from it.

Presenting at Iqnite Nordic 2010

The Iqnite Nordic 2010 programme was just released, here.

I'm happy to be presenting "Test Reporting to Non-Testers".

It'll be primarilly an experience report with some real data and problems they have caused in their presentation and interpretation.

The presentation will build on themes around assumptions (in the report author and receiver), statistics (their use and misuse) and reading too much into the reported information.

By looking at problems that the examples have caused I'll be able to suggest some alternative approaches for testers when communicating with non-testers.

Communication with non-testers is a big interest of mine - not only do testers have to deal with modelling and assumptions of the product and stakeholders, they also have to handle the communication channels towards non-testers. This has lots areas for pitfalls (a whole different type of modelling and assumption awareness!)

Unfortunately, the mind-map I made when drafting this is huge - it needs to be printed on A3 to be readable. The result is that I'll only be touching on some areas in the presentation - but I think they're some of the key ones and worthwhile.

So there's room for a part II of the presentation!

Testing Refresher from Sanderson of Oundle

Today I was "philosophising" with my 5 yr old daughter - wondering about why she thought the sun was so big if I could make a circle of it with my hand (she had a good answer) - and then we started discussing which of the moon and sun was bigger and reasons why one might "appear" bigger than the other. This generated a great number of questions and hypotheses.

This activity reminded me of "Sanderson of Oundle", I'd recently read an article and a booklet on the subject. Just the title sounds intriguing doesn't it?

Who?
F.W. Sanderson was headmaster of Oundle public school from 1892-1922 - and the story of his approach to teaching sounded both revolutionary and visionary and generated interest from pupils and parents alike.

After I read the article I searched and managed to track down a copy of a booklet that the school produced on the seventieth anniversary of Sanderson's death. It's only 24 pages but gives a great insight into a great enabler.

Sanderson said: 

"We shall see what changes should come over shcools. They must be built in a large and spacious manner, the classrooms being replaced by halls or galleries, in which the children can move in the midst of abundance, and do and make research: not confined to a classroom."

"The methods will change from learning in classrooms to researching in the galleries; from learning things of the past to searching into the future; competition giving place to co-operative work."

He had two main methods:

• Let there be no work which is not in some sense creative.

To him, classrooms were just tool-sharpening rooms and the real work was done in the laboratories, library, museum, art room or power station. The creative work took part in research outside of the classroom.

• Let all work be co-operative rather than competitive.

He held creative work (through research) to be higher than examined work and he believed the best way to achieve this was by co-operation. His opinion was that all had areas in which they can excel and it's for the teacher to help the pupil to find those areas - not to constrain them to a norm. BTW, he didn't believe in bad students - just students that the teacher hadn't found the right angle or area of interest yet.

Why the interest to me?
I've recently "regressed" into the scientific approach - or maybe I'm a born-again scientist (mathematician actually)? I'm on a little bit of an evidence-based refresher... So I found the work that he was doing (and the way he was doing it) to be so enlightening - and with a lot of common sense.

Just take his two main methods:

Creative work:
This is so true for me as a software tester - I don't do things just because they're there or that's the way it's always been done. I try to evaluate and understand what I'm doing, the selections I'm making, the assumptions that are there (both obvious and less obvious) to be able to give a full account of my actions. That gives me the piece of mind that I'm trying to do a good job.

Take an automated test case (or test suite) - I wouldn't run it without considering the reasons for running it, what information I'm getting out of it, what it's not telling me and even considering if this selection is still relevant. If I can cover those bases - and be able to discuss them with colleagues - then I'm making a creative input into the decision around that test.

Co-operative vs Competition:
Co-operation in project environments revolves very much around communication. That's not just writing a report on time or emailing someone to say that all plans have changed. It starts before that - framing the expectations around how the communication is going to work - a sort of trust and confidence-building exercise. If X knows I'll treat all information in a professional way and be a good ball-plank for him then I'm much more likely to know about any potential changes/problems sooner.

Research and being creative is very much key to most work connected with software development and testing - and co-operation is a natural part of this.

Competition in project environments can have several connotations. Most of the "competition" whether it be one-up-manship or the CYA syndrome is usually connected with non-cooperation. Non-cooperation in the sense of a non-functioning or not-open two-way dialogue does not help the project.


Takeaways
I loved reading about Sanderson of Oundle - I wish I'd gone to a school like that and with a teacher like that.

It was also a reminder for me about the importance and relevance of creativity and cooperation in my daily work. Without creativity there is no thinking (and vice-versa) and if I'm not being creative (or thinking) about my testing then I'm not doing good testing.

Discussing testing with my colleagues (cooperation) is a basic need for good testing.


Got any Oundle-like opinions?

EuroSTAR BlogSTAR Candidate Roll-Call(?)

Is the popularity/talent contest format crossing over to the testing world now? Maybe, and maybe that's a good thing. It has the potential to generate interest and involvement, publicity for the conference and a way-in and impetus for testers whose company budgets can't stretch to the conference.

EuroSTAR's search for a BlogStar sounds interesting - and worth taking a look.

Is this the next X-factor or "The Testing Blogger's got Talent"? It's got the popular vote factor - and that's a really interesting angle.

There is the potential for a great showcase! There's a lot of great bloggers - some who didn't make it to Eurostar via the VideoStar competition that I'd be really interested in seeing blogging.  Note, I haven't re-freshed myself on the video entries (and maybe my memory is a bit hazy) - but I voted for the 3 mentioned below.

So here's my short wishlist for entries.

Possible Line-Up?

Rob Lambert - always got something topical to say. His video entry was slick - something about roadsigns? :)

Pradeep Soundararajan - his "joker" video gave me willies! His writing has many learning observations.

There are other Indian bloggers that could easily enter and generate a lot of interest - Parimala Shankaraiah and Ajay Balamurugadas to name but two!

Then there's the group that are regularly involved in WTANZ - Marlena Compton, Trish Khoo, Keis & Oliver Erlewein - all generating good work - with opinions aplenty!

Anne-Marie Charrett - her "don't vote for me" video was fun (or was it "yes but, no but"?) There's always a lot of intelligence in her posts.

I saw a tweet that Adam Goucher might think about entering - cool!

Peter H-L always has an interesting take on topics - I liked his suggestion about a group entry - but not sure the prize budget would stretch to that!

Markus Gärtner - he's already going but it could be fun to see an entry from him - although his blog output would probably be more than anyone else ;-) But always worth reading!

This isn't an exhaustive list of interesting test bloggers - just the ones not in my current inattentional blindness blind-spot!

And From Leftfield

The search has been promoted as looking for: "Not for the faint hearted", "with an opinion", "not being afraid if everyone doesn't like what you've got to say": Well prompted by the national origins of the eurostar team I'd say that calls for a "Father Jack" blog! Oooh, tempting...

Or maybe we should follow Peter's train of thought with a team entry - who wants to be Dougal, Ted and Mrs Doyle????

(In case folks are getting confused - these are Father Ted references.)

There's actually been many an absurd observation in "FT" that you could draw a testing angle on - "near, far away..."

Final Thoughts & Question

Interesting competition and a clever way to get people involved. Some folks might wonder about the opportunity cost involved in generating posts on another site - but just think of it as it being another sounding board for all those posts you have in prep! I know, it can be an effort though...

I do have a question about the voting though - I wonder how that will be done - and do hope it's not a simplistic poll. I mean if I get my friends and family (that are not testers) to vote for me does that really show my blogging as being more interesting to testers?

I'd be inetersted in seeing other factors in the voting: the most-read blog post, most read blogger, most comments, most retweets, etc, etc,  Just thinking...

I'm tempted to enter - but have other things to sort out first. So, for me, the jury's out just now.

If I enter and somehow reach the conference I promise not to come to the gala in the persona of Father Jack! Maybe ;-)

Carnival of Testers #11

June was a busy month - not only were there a range of good blog posts but the World Cup in South Africa kicked off. Many people heard about the vuvuzela for the first time - and a host of football cliches and analogies (not new for many!) I'll try to maintain part of the spirit with the odd footy cliche or analogy.

Testing is a funny old game. Enjoy!

Markus Gärtner kicked-off the month with a good write-up from XP2010 of a Lean presentation by Mary Poppendieck.

She shoots, she scores! Parimala Shankaraiah with a good retrospective, here.

Some hard work in the middle of the pitch! Jon Bach and Lisa Crispin (amongst others) tackled the subject of Diversity in Agile and Women in Agile.

A dazzling run down the wing with a nice piece on Jon Bach's observations around approaching a combination lock problem.

A long throw-in from Trish Khoo continuing the gender diversity in IT, here.

Alan Page shows it's a game of two halves with his recount on the wide wide world of testing sources.

There are no easy games from James Bach and his views on the need for challenge (and why he does it) in the testing culture.

If you missed it the first time then the replay has plenty of highlights from James Christie's call for testers to challenge the culture.

A great advert for the game from John Stevenson and his account of a training session in India.

Andréas Prins was strong on paper with his post on his learning and sources around test profession-related learning.

With a few minutes of the game left Maura van der Linden gave a great account on her experience with transpection.

A strong challenge from Abe Heward and his assertion that good testers are not robots.

A late corner from Michele Smith with a testing question on a "bug free" claim.

Gatecrashing the top 4 was Simon Morley and his thinking about showing his thinking post.

Zeger Van Hese's ran his socks off with his perspective on collateral product features turns up.

Into extra-time and illuminating perspectives and observations on a WTANZ session from Keis. What a lovely finish!

Until next time, enjoy!