Our survey said...

" #interpretation #fun #context #cognition "

Whilst compiling material for some other work I stumbled across some old Family Fortunes and Family Feud 'funny/strange' answers on YouTube*.

I've recently being doing a lot of thinking around framing, ref [5], and the problems it can cause and solve and I started thinking about different causes for the unexpected answers.

For communication analysis I use two types of exercise, (1) frame analysis and (2) word and meaning substitution.

Frame analysis

• What are the aspects that might be important to each person involved in the communication? This usually revolves around situational context of either the one asking the question (presenting the problem) or the one answering the question (presenting a solution). Here there is scope for a range of cognitive and interpretation mistakes.

Word and meaning substitution

• A well-known example of this is the "Mary had a little lamb" exercise, described in "Are you lights on", ref [1], and is a demonstration of how changing the emphasis of a word in a sentence, or replacing a word with a similar meaning (from a dictionary or thesaurus), can change the meaning of the sentence. Therefore, if both parties in the communication intend different word emphasis (in word placement or interpretation) then there is a possibility of confusion.

So, what can appear as confusing or even amusing answers can, with the right perspective, have a certain logic. In the list** below I've made an attempt at finding the perspective behind the answer, in red.

The questions are typically prefixed with "We asked 100 people to name..."

Q: Something a husband and wife should have separate of

A: Parents

Logical answer(?) but maybe not along the intended lines of the questioner.

Q: A planet you recognize just by looking at a picture of it

A: The Moon

Confusion of definition of planet with 'celestial body' (something in space with an orbit)

Q: A month of spring

A: Summer

Slip of the ear, of->after(?), ref [4]

Q: A word that starts with the letter Q

A: Cute

Q. Name a part of the body beginning with 'N'

A. Knee

Phonetic interpretation

Q: The movie where John Travolta gave his most memorable performance

A: The John Travolta Biography

'Most memorable performance' to the questioner had a potentially different meaning. To the answerer either it was interpretted as a film where he featured the most, or he wanted to give an amusing answer.

Q: Something you wouldn't use if it was dirty

A: Toilet paper

Amusing and logical answer(?)

Q: A signer of the Declaration of Independence

A: Thomas Edison

Slip of the tongue, specifically noun substitution, ref [4]

Q: Something that comes in twelves

A: Dozens 

Could be logical interpretation but not something the questioner was intending(?)

Q: A sophisticated city.

A: Japan

Misinterpretation (or even slip of the ear) of city for destination.

Q: A kind of bear

A: Papa Bear

Recency effect(?) - had recently been reading or exposed to children's stories (?)

Q. Name a number you have to memorise

A. 7

Misinterpretation of 'memorise' as 'favourite' or 'memorable'(?)

Q. Name something in the garden that's green

A. Shed

Context-specific to the answerer(?)

Q. Name something that flies that doesn't have an engine

A. A bicycle with wings

'Logical' and specific answer - but the questioner could have maybe clarified the question with a 'commonly known item'.

Or, recency effect - flugtag, ref [6].

Q. Name something you might be allergic to

A. Skiing

'Alergic' -> 'don't like'(?)

Q. Name a famous bridge

A. The bridge over troubled waters

Interpreted as 'something well-known with bridge in it'(?)

Q. Name something you do in the bathroom

A. Decorate

Specific to the answerer's context.

Q. Name an animal you might see at the zoo

A. A dog

Generics. Potential that the answerer has not interpreted the the question as 'generally seen and residing in the zoo'.

Q. Name a kind of ache

A. Fillet 'O' Fish (?)

Brain-freeze or 'slip of the ear'(?)

Q. Name a food that can be brown or white

A. Potato

Answerer framed the question as a food which could be presented as brown or white(?)

Q. Name a famous Scotsman

A. Jock

'Slip of the ear' -> 'a common nickname'(?)

Q. Name a non-living object with legs

A. Plant

Maybe thinking of a plant on a plant stand(?)

Q. Name a domestic animal

A. Leopard

Misinterpretation of 'domestic'(?)

Q. Name a way of cooking fish

A. Cod

'way' misinterpreted as 'type'(?)

Analysis Notes

• Context - some answers are specific to the answerer and not the questioner. Example traps might be (1) Understanding and interpretation, (2) Word association problems or (3) Relating everything to ones own experience or circumstances.

• Recency effects, ref [3] - the interpretation associated with a word was used in a different context, giving a skewed answer. In testing this occasionally results in skewed emphasis of the risk determination - see tester framing problems in ref [5].

• Skipping and changing words in sentences - to actually hear a different question - sometimes grouped under 'slips of the ear'. In testing this might result in an incorrect solution application, similar to framing problems but can also be 'straightforward' slips that result in some faulty analysis - missing some key input parameter for example.

• Other framing effects can be caused by the previous question, previous answer or even some realisation that a previous answer was wrong/silly and so inducing more stress in the answerer.

• Stress can mean that sometimes when you're trying to react you don't actually listen to the whole message or question. This can be time pressure or other stresses. Be aware of this potential problem.

• Anchoring effects, ref [2] - focusing on a word and giving an association with that word (rather than focusing on the whole question). In testing this typically results in confirmatory testing.

• Generic statements can create confusion. These are generic statements as part of the answers - this is where the question can be confused between giving an example of a specific kind and categorizing the answer into a grouping. Opposite of the answerer-specific problem. More on this in another post...

• Don't rule out brain-freezes either - these can be multi-word substitution or paragrammatism, ref [4], which result in nonsense responses.

And finally...

This is a good exercise and quite instructive for those working in software testing - it's a good illustration of how what might be seen as an obvious or simple answer can actually diverge from the expectations of the stakeholder or even customer.

Be alert for not just for confusing messages but also the potential for confusing answers. In this way you might know when to re-affirm your interpretation back to the stakeholder or customer.


References

* If you want to see the clips you can search youtube for "family fortunes answers" or "family feud answers" or "game show stupid answers".
** Lists compiled from

http://www.funny-haha.co.uk/Joke.asp?J=283

http://www.businessballs.com/familyfortunesanswers.htm

http://www.stupidgsa.com/american/family-feud/

[1] Are Your Lights On?: How to Figure Out What the Problem Really Is (Gause and Weinberg, Dorset House, 1990)
[2] Anchoring effects: http://en.wikipedia.org/wiki/Anchoring
[3] Recency effects: http://en.wikipedia.org/wiki/Serial_position_effect

[4] Um...: Slips, Stumbles, and Verbal Blunders, and What They Mean (Erard, Panteon, 2007)

[5] http://testers-headache.blogspot.com/2011/08/framing-some-decision-and-analysis.html

[6] http://en.wikipedia.org/wiki/Red_Bull_Flugtag

Book Frames and Threads Updated

" #softwaretesting #testing "

Earlier in the year I tracked some of the influences to my software testing learning, here. I say some, as I exclude blogs and online articles - that is an update for a later time.

The map is intended to track the influences - whether it is something I have in my active reference library or something that sits in the anti-library, reminding me that however much I absorb and use there is always a great deal more out there that I haven't looked at that could be useful. It's a very physical way of reminding me that I don't know everything - and that's good!

In the map I try to track

• whether the publication was discovered during my own research or via the test community (either recommendation or discussion), 
• if the book/article was directly influenced/referenced by another publication (linked by a blue line)
• if it's ongoing whether it is active or not
• changes from the last version (in pick/read) -whether it has moved (with a line to track) or a new addition

This is a useful reference for me to

• track some elements of my learning, influences and research
• track what has been current (the changes under the latest period and what that might mean for me)
• remind me that I don't track all my influences (blogs, some other online material and publications)
• remind me that there is still plenty to research
• show that items are added and removed from the anti-library

Future additions that I'm thinking about:

• include some dates (addition and/or completion)
• include more community influences
• include more notes around the publication (what I learnt or felt about it - you can learn without agreeing with the publication)
• include a frequency of re-use (how often I re-use it for reference) - a type of star rating for use/re-use

The map is below - I'm more than happy to receive comments on recommendations for publications and sources and even additional information that I might find useful to include.

If it encourages you to do something similar I'd love to hear about it.

Enjoy!

Testing: Do you train like an athlete?

" #softwaretesting #testing "

I just read this analysis piece on the BBC site about the pressures involved in sprinting, especially the start and run-up to the start.

As I read through it I found myself mentally ticking off links to the testing world:

• There is no perfect start
• Appearance and presentation is part of the message
• Pressure kills concentration
• Reacting at the right time
• Distractions affecting focus

No perfect start?
The interview contrasts two sprinters with different techniques and how their physical make-up presents different problems at starting and how they handle that. In the software development world this translates to there is no best practice. Every problem and solution is unique - what works for one athlete (or product) does not necessarily work for another.

Presentation of the message
In the article the example is given of Linford Christie displaying his superior physique to other competitors before a race. The intention was partly - I'm prepared and ready.

In the testing world, the message and form in which we give that message is important. Successful message styles are usually truthful, not unduly biased by numbers and consistent. Part of this goes towards building your brand.

Building this trust in your team and stakeholders is vital to successful story-telling.

Pressure kills concentration
All athletes respond and cope with pressure in different ways. Some pressure is exerted by other athletes, some is created by the athlete themselves (their own expectations) and their surroundings.

In the testing world - sometimes the pressure is external - created by teams or stakeholders, sometimes with unrealistic expectations of testing and sometimes because they don't handle/distribute the pressure so well.

This reminded me of the chapter in Pragmatic Thinking and Learning, "pressure kills cognition" which gives examples of overt pressure disrupting thinking. Be on the look-out for this - it's not the easiest thing to deal with when you're on the receiving end, but if anything, don't pass on the pressure - or at least understand what effect it might have.

Reaction times
In athletics the false start is feared - especially for the sprints. The athletes are coiled like a spring and are trying to react as quickly as possible. But, they don't want to react too quickly, and not too late either - they want something that's good enough for them. The interview demonstrates the difficulty here with the game of slapsies (look here and here).

In testing, the similarity might be when to raise or highlight a problem, call in help for the investigation. You don't want to be crying wolf for every issue, just as you don't want to be doing some lone investigation "too long" (and potentially getting stuck). Here is where pairing or having a colleague (tester or developer), that you can run something by, is very useful. Even at a daily stand-up meeting mention what you're currently investigating - sometimes there is someone who says "check X" or "have you got Y set in your configuration".

There is a routine and balance to find here - something that takes both practice and goes hand in hand with your message / brand.

Distractions affect focus
Athletes get distracted by the antics of other athletes, sometimes by the crowd or acoustics in the stadium. They have different ways of dealing with this - some try to get into and stay 'in the zone', some go and lie down and stare at something, whilst others continuously move around to avoid tensing up.

Distractions play a big part in any work environment also, as well as the ways we try and remedy them. It might be the problem of multi-tasking - getting many high-priority demands on your attention and not being able to decide which to devote attention to - or really how to stay focussed on the task at hand.

Sometimes, it's more efficient to simplify the problem into smaller component parts, and so have a feeling of manageability and being able to see progress quicker. Small wins keep attention and focus rather than long drawn-out slogs wading through mud.

Sometimes it's about removing distractions - don't check email for the next hour, close unnecessary browsers (with their flash animations that catch the eye - or use a flash blocker) - reduce the number of open windows to the minimum needed for the task.

Sometimes it's right to de-focus on the problem, step back and look at the wider picture. This helps you relate the problem to it's situational context, re-evaluate why you're doing something, maybe even bring in a fresh pair of eyes to help. This sometimes gives new information or re-affirms the original scope, then you can re-focus on the problem (with any new insights and information).

Are you a person that whilst talking to someone must always answer a phone call (no matter who it's from)? Or do you treat phone calls like someone coming up to you in the corridor whilst you're already in a conversation - usually they'd wait to interrupt your ongoing conversation - so why should it be different with a phone call? (The exemption here is if you're waiting for some urgent or important information which would lead you to interrupt the conversation.)

Awareness
There are lessons to learn all around - especially from non software testing disciplines. The key is to be able to recognise your potential problem areas - whether it's to do with message presentation, knowing when to react, handling distractions or being aware that pressure can have a detrimental affect on performance.

Awareness is an important first step in problem solving - whether you can solve the issue or not - understanding factors that affect your "testing performance" is key!

Carnival of Testers #25

" #softwaretesting #testing "

Wow!

August was busy - partly due to CAST2011 and Agile2011 conferences triggering the creative and writing juices...

Journey and Influence
Many testers draw influences and experience from outside of the traditional software testing field.

• Rob Lambert pondered the Peltzman effect and how it might apply to testing, here.
• A journey so far in exploratory testing is given, here, by Albert Gareev.
• John Stevenson reviewed different views on the need for product knowledge (or not) when exploratory testing.
• The Bach Brother's Legion of Test Merit premiered this month, with the first recipients presented by James Bach here.
• Brian Osman looked at some of the areas lacking in good testing, here.
• Two interesting perspectives, here, on success and failure from Joel Montvelisky.
• A crime series triggered Henrik Emilsson to start thinking about testers and criminals(!), here.
• I started thinking about the chapter in Slack when I read Joe Strazzere's post on being fungible (or not), here. Good points!
• The what-if heuristic from Daniel Berggren, here, shows how useful a tool it can be for a tester.
• Good testing journey in search for mushrooms from Zeger Van Hese, here.
• Original and very interesting ideas on creating testing perspectives from Shmuel Gershon, here.
• Parimala Shankaraiah continued journal on public speaking is here. Good observations!
• Oliver Erlewein writes about a new proposed testing standard on a new collaborative blog, here. Interesting new site!
• Important points from Alan Page on test design for automation, here.

CASTing an eye...
Many testers jotted down their thoughts and observations, some being:

• The use of Ustream allowed Claire Moss to make some notes on Paul Holland's lightning talk, here. Daniel Woodward made some interesting observations, here.
• Day 1 summaries from Pete Walen and Michael Larsen, here and here.
• Good rapid blogging from Markus Gärtner, here and here.
• Post-conference reflections from Jon Bach, here, Elena Houser, here, Ajay Balamurugadas, here, Michael Hunter, here, Ben Kelly, here, Matt Heusser, here, and Pete Walen, here. Some important observations in them all.   
• Some more important point were made by Eric Jacobson, here.
• Important observations from James Bach on the test competition, here. 
• And finally, some tips on presenting at conferences, here, from Lanette Cramer.

Until the next time...