On Being Replaceable, Role Traps and Adding Value

I said to one of my managers several years ago that one my main goals towards the team and organisation was to make myself replaceable.

He nearly fell off his chair....

As I was a leader within the organisation - if I become replaceable what does it mean for the position I have or the value of the work I'm doing? Oh, I say this to most managers (partly as a test :) )

It is important to distinguish how people are viewed within teams, projects and organisations. In this context there are two main views:

1. What they bring in terms of personal skill, knowledge and capability.
2. What they bring in terms of operational responsibility or ability.

Traps

Where teams and organisations have roles or specialized people (e.g. testers) then there is a risk of attaching someone's behaviour to that of the role.

Or the other way around, 

The skill of a role is often thought of in terms of a person/s that perform it.

(or have performed it). 

Or:

Experience is implicitly labelled as an exemplar. 

Example
In the past I have sat in a coordination meeting with a team and asked an incisive question such as, "Customer X have feature Y adapted/changed in their network and so your development of feature Z should also look into interactions between features Y & Z". This means that a potential risk is reduced - or at least some investigation is started to reduce a potential risk.

Now, if I miss a similar meeting with another team, there might be such a question not asked that might mean such a risk is not caught or it is thought about later, meaning some additional work. (Note: These coordination meetings might be some form of reference/project or expert aid to the teams.)

Question: Who was the target audience for my question? 

Many people would think it was the team doing development of feature Z. My target audience is everyone though - it's not just about the question, it's also why it is asked (some might call that the context) - in this case to help all realize something they hadn't seen beforehand.

I don't ever see myself as the person who asks questions others don't - but more as someone who might help others ask better/different questions next time. If I haven't passed on some of that capability then I've failed.

Adding Value
There are two main ways I add to the team, project or organisation.

• I point out the differences between what I am bringing as me, and what I am bringing as "performing role X". They might overlap at times, and not at others - and it's important to help others with that difference.
• I ask "why?" a lot. Not to be a pain (even if that's how it might sometimes be seen), but to help understanding.

But, how is this "adding value"? It's highlighting behaviours that others can adopt that are not "owned" by a role.

The "Why?" Question
One of the most common questions I ask is "why?". It's a sign of wanting to learn what someone else is thinking. It's a sign of wanting to flush out, and clarify, assumptions. The "why" question is one of the cogs of dialogue and understanding.

So, if I am the only one who is looking out for dialogue and understanding then there might be a problem in the organisation. However, in my experience, someone has attached the "why" question or "the types of question that I ask" to the role I had - so it becomes "role ABC asks those type of questions".

That's where I need to remind people around me that these are not my questions, I don't get territorial about such questions, and that if I'm not around and the question pops into your head, go ahead and ask it.

Learning Organisations
Organisations, teams or projects that want to grow and learn must be very careful about roles -> sure, if someone is the designated decision maker let them make it. Until that point, there's usually plenty that can be done without the decision-maker - including asking questions.

Sometimes people (teams) need to be given permission to think for themselves - strange as that may sound.

Lessons

• It's not what skills you bring to the table, it's what you leave behind for others after you've gone.
• It's not what attitude you bring to the party, it's the positive change in attitude you leave behind that's important. (Sometimes, that means more people are prepared to ask, "why?") 

Doubt
A typical question I get about achieving replaceability is: don't you do yourself out of a job, or no one needs you after you've improved others?

In a team, or organisation, that has a constant ambition of improvement that is never a problem - there is always a new problem to work on. As Weinberg said (I think), when the most important problem gets solved, problem #2 gets a promotion. Sure, it's a different problem and may take you out of your comfort zone, but ultimately, that's how you improve.

Some people treat knowledge as power and hold onto it. Unfortunately, those are the folks that can become one-trick ponies or get bypassed by progress...

And finally...
So, to me, being replaceable is positive - it means I've added value - it means I've given others a tool for thinking more clearly - it means I can carry on improving, learning and adding value.


Are you adding value? 
Are you leaving something on the table for others when you move on?

Examining the Context-Driven Principles

At Let's Test 2013 James Bach had a keynote about the Context Driven Testing principles, some of their origin, usage and implications of them. An interesting piece of trivia that James revealed was that the basis of the principles were written down very quickly, with some refinement afterwards.

He discussed a little about the implicit meanings behind the 7 principles, but the slides were skipped through so I didn't see the details. However, by this point at least one question occurred to me.

My question in the open season went along the lines, "The principles were mentioned as a litmus test to gauge if someone was context driven. However, adopting a scientific approach (ie one of searching for information and not settling on an assumption), there may well be more than 7. How do we add number 8?"

An underlying question was (for me):

1. If the principles were written so (comparatively) quickly why haven't they been challenged from testers claiming to follow or adhere to them?
2. Indeed, isn't this a good exercise for a tester - context-driven or otherwise?

Therefore, I thought I would take a look at them - as a (comparatively) quick exercise. James did mention that Markus Gärtner had extended the principles as an exercise previously. I remember seeing a discussion on the software-testing yahoo list some time ago and checked it out - and sure enough I'd discussed the possible extensions there, and thought it was a useful topic to revisit.

The 7 published principles, ref [1], 

1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

My Approach
In assessing the principles I used two approaches, (1) Critical/Logical consistency - do the propositions support conclusions or implications?; (2) Are the principles testable (falsifiable)?

Assessment
#1 [Unchanged] I tried to extend this one, but I think it is rhetorically good - succinct and a good start to the principles. This can not be tested but is inferred by induction.

#2 [Remove] I could think of  removing/replacing this one. It is a derivation of #1. The essential content here is that "Practices evolve over time" - I thought of making this my #2, partly to remove the distraction of "best practices" (that would be a derivation of these principles), and partly to emphasize the evolution of practices. However, I finally settled on having no #2.

#3 [Rephrase] The principle is an assertion. For this one I want to emphasize that the project, the processes and people form a system and that they change over time. The importance here is to illustrate that the system is not static, it is complex because of people (and their emotions) and as such, working with such a system is not a trivial activity. Therefore I would re-phrase.

#4 [Remove] This is a derivation / implication of #3, an application of the principles.

#5 [Unchanged] Again, succinct and difficult to refute.

#6 [Unchanged] An assertion that where good software testing happens there is a high intellectual demand.

#7 [Remove] This is unnecessarily wordy and sticks out as too different from the previous principles. It also drifts into certainty and seems like an untestable principle, and so I would remove it.

Addition
#8 Here I want to tie the system mentioned in the rephrased #3 and good software testing.

Result

1. The value of any practice depends on its context.
2. -
3. Projects and people form part of the system that work together, where products and practices evolve over time
4. -
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. -
8. Good software testing encapsulates the system of project, people and practices that work on building the product.

So, I ended up with 5 principles. The application of these would then produce variants and clarifications. For instance, the best practice item is derived from #1. Understanding systems of people, emotions and time is derived from these and many, many more.

I stopped there, but if I spent more time could probably refine them somewhat. But they are good enough for this exercise. And as James suggested - if you started from scratch, you might well not end up with the original 7 principles.

So, how would your interpretation look?


Reference
[1] http://context-driven-testing.com/

Book Frames and Threads - Jan 2013 Update

This is an update to the map I use to track some of the book and reference material influences.

Changes since the last edition, ref [1], are marked in dark grey. Some changes I've put in this version:

• Some online resources included
• A re-use indication (books/texts that I refer back to are in larger font - the font isn't proportional to its popularity, but maybe for the next update)
• Some author references are included
• Some articles are referenced - but I'm aware that many are not included

This map will be printed out (to replace the previous version) on an A3 sheet by my desk. It sometimes generates curiosity and questions - which is then a useful medium for discussion about testing, and the many many influences from outside of software development.

Any recommendations for additions - whether books, publications or format?

References

[1] Book Frames and Threads Updated

Map

Book Frames and Threads Updated

" #softwaretesting #testing "

Earlier in the year I tracked some of the influences to my software testing learning, here. I say some, as I exclude blogs and online articles - that is an update for a later time.

The map is intended to track the influences - whether it is something I have in my active reference library or something that sits in the anti-library, reminding me that however much I absorb and use there is always a great deal more out there that I haven't looked at that could be useful. It's a very physical way of reminding me that I don't know everything - and that's good!

In the map I try to track

• whether the publication was discovered during my own research or via the test community (either recommendation or discussion), 
• if the book/article was directly influenced/referenced by another publication (linked by a blue line)
• if it's ongoing whether it is active or not
• changes from the last version (in pick/read) -whether it has moved (with a line to track) or a new addition

This is a useful reference for me to

• track some elements of my learning, influences and research
• track what has been current (the changes under the latest period and what that might mean for me)
• remind me that I don't track all my influences (blogs, some other online material and publications)
• remind me that there is still plenty to research
• show that items are added and removed from the anti-library

Future additions that I'm thinking about:

• include some dates (addition and/or completion)
• include more community influences
• include more notes around the publication (what I learnt or felt about it - you can learn without agreeing with the publication)
• include a frequency of re-use (how often I re-use it for reference) - a type of star rating for use/re-use

The map is below - I'm more than happy to receive comments on recommendations for publications and sources and even additional information that I might find useful to include.

If it encourages you to do something similar I'd love to hear about it.

Enjoy!

Fake Learning: Shortcuts Don't Always Exist

" #softwaretesting #testing "

Excuse the mini-rant...

I seem to have a love-hate relationship with different Q&A forums.

Why?
I like to give input on them, to share my perspective, with the hope that my take, however left-field or mundane, might help the questioner and all those taking part. I don't consider myself the guru - I'm just providing one piece in a thousand piece jigsaw - rather than have the whole picture on the front of the box!

Some people are out there asking questions in a genuinely investigative way, searching, pushing the boundaries of their own experience and forming opinions on what they find. That is fantastic, I love the energy (I won't say passion :) ), the willingness to go on a mental adventure, get out of your own mental comfort zone and see what you find and how you might use it.

For examples of thought-provoking questions look at the questions from Phil, here, or Shmuel, here, or Shrini, here.

There are no right or wrong answers - but every answer might help you learn something new.

Comfort Zone
But, on the other side of the coin there is the search for information without the searching or learning. This is definitely a case of not pushing ones own mental comfort zone.

Examples of staying within ones own mental zone - questions of the form:

What is the (insert one of: "best, most effective") (insert one of: "tool, process, method, etc") for (insert a general non-specific context or situation)?

These types of question want to bypass the learning journey. It's a pay-per-view approach - without the payment!

That may be ok if all you want to do is compile a list (the 21 capital coach journey in one week), but you must ask yourself how that information might be used - whether to get a flavour of something or to claim to "know" about something are two different purposes.

It's sort of the difference between learning by rote and information gathering (with hypothesis formation) -> ooh, scripted learning vs exploratory learning, or testing with your eyes wide open vs eyes wide shut!

Fake Learning?
The essence of these types of questions - for me - is an attempt to shortcut learning - to shortcut the experimentation,  digesting the information, hypothesis (ideas) forming and making new discoveries (at least to yourself). Shortcutting this (and purporting to have gone through it) is something I think of as Fake Learning.

Several people highlighted the problem recently:

• Myself on twitter, here.
• Phil Kirkham on twitter, here.
• Michael Bolton on twitter, here.
• Rosie Sherry in her blog, here.

Depressed?
No, not really. Am I (and maybe others) going through that "hate" part of the cycle right now. Maybe yes.

But it doesn't stop my own searching and willingness to help others along the way - so I'm still on the lookout for interesting and thought-provoking questions - that doesn't mean I always have the time to respond and get involved myself!

But if I'm asked directly for help, I'll always try and help. It fits in with my testing motto too!

Mini-rant over. :)


Beware of Fake Learning!

Framing, Threads and Books

" #softwaretesting #testing #learning "

Inspired by various blogs reflecting on the past year and especially one from Rob Lambert mapping his reading and Geir Gulbrandsen comtemplating his upcoming reading I started thinking about some of the influences on my reading choices.

I then realised that there is a connection between my various reading choices, some of the ongoing study or work areas I'm involved with and discussions in the software testing community. So I started thinking of a picture of my reading as a framing exercise - to help elicit the thinking and reasoning behind my choices. This became an interesting mapping exercise - quite suited for a mind map.

The exercise would also help show up holes in my linkage (maybe I had several alternative sources and picked one - it would be interesting to show this - I haven't done that yet!) and also areas that I'd specifically excluded (this could be in the form of annotations or comments about progressing or not along a particular path - this isn't included in my current map.)

I split the groupings into books "read once" (meaning read at least once), "ongoing" and the "antilibrary" (from Taleb's The Black Swan - an unread source that demonstrates what you don't know - or in this case a subset of that.) As a first step the main sources were added as labels (whether via own research or via community recommendation or discussion - blogs, articles, test forums and twitter) as well as a linkage between books - which was read due to another book (either it was referenced or some research from that book pointed to another).

I'm going to leave my thoughts about the individual books to another time & place, as well as some coding (perhaps colour) about which ones are more frequently used and also some timescale (purely for my own benefit to help put the reasoning into context).

Other useful additions that may be made would be of magazine articles, e-books and other online literature (Rob Lambert's idea!) - as well as the aforementioned information about what was excluded and why.

The interesting aspect of this exercise was that I can see the framing behind parts of my learning journey but also that the map is a set of threads. Some of them are ongoing, some will be re-visited and no doubt extended and some will be closed down. Plus there's always room for new threads.

The way that I read books I do it in a very "thread-like" way - I have many "ongoing" at any one time and I revisit read ones. That's ok for me and charting them like this helps to record my (where I have and haven't visited).

Just like testing!

Update: I've also intentionally left out what I've learnt from each of these threads - that's a whole series of posts!

What's on my current reading list?

I take inspiration and ideas from a wide range of literature, other testers and sometimes triggered by something from leftfield.

I thought I'd jot down my current reading list (I last did this for my summer09 reading, here) for a couple of reasons:

• It's good to walk-through what you're currently doing (reading) and why - sometimes you might be reading something obscure, but for a particular reason.
• If you're like me, I get ideas from what other people are reading - so this is part "here's some tips" for other people, but also a hope that readers will send in tips to complement my reading - so that's your challenge at the end!

The Scientific Corner
After attending the RST course in March I had the urge to rediscover the scientific method. So as a step in that direction I started reading the following.

• Philosophy of Science: A Very Short Introduction. Well, I'm into science and philosophy, so why not start with "What is Science?" Great stuff! 
• The Oxford Book of Modern Science Writing. This is a collection of excerpts from some outstanding scientific papers and books of the 20th century. I've devoured a few already - a combination of good writing and interesting science.
• Can a Robot be Human? As a tester and wanabee-layman-philosopher this is right up my street. There are interesting slants and many questions - the sort that make you think! Great for a critical thinker, lateral thinker, divergent thinker and a tester!

The Systems Thinking Corner
Looking at complex problems and trying to understand their complexity is interesting to me - it also helps in my daily work.

• What The Dog Saw. This collection of Gladwell articles (I think most can be found on the NewYorker site) is an eclectic mix with his distinctive take on them. Very interesting and insightful.
• What Every Body is Saying. Observation is a necessity (as a tester) and so why not try it on people around me? I'm certainly no expert but it gives me a few insights and maybe it will help in the odd discussion in the future.
• The Black Swan. Enjoying this slightly-scholarly book and I'm taking my time with it. Lots of good things for testers to think about. These types of events are all around us - just think volcanos!
• The User Illusion. I can't seem to finish this book - I've been reading it for years - it has tricky parts - I read and re-read parts. It's an exploration of conciousness and how the unconcious mind processes so much information. It introduced me to entropy, information theory and to Gödel - satisfying the mathematician in me and giving me another insight on testing problems.

The Testing Corner
I have plenty of software testing books - some of which are constant reference material and I've written about before. However, I don't think I've mentioned these two before:

• Beautiful Testing. I started dipping into this last autumn and it then fell by the wayside - will re-visit before long. Some interesting chapters - I'll wait until I've finished it before giving a verdict.
• The Art of Software Testing. This is a pure reference material for me. I picked up a second hand first edition last autumn - did the triangle self-test and enjoyed some of the "phycological" aspects of software testing.

The Recently Finished Corner
I think I have more than one, but this is the one that stands out.

• Blink. Well, I'm into how the mind processes information, why that sometimes works and sometimes doesn't. As a tester I can relate to why people can be hampered by too much information - so I liked the war-games description!

The Not Started-Yet Corner
Next up...

• A Devil's Chaplin: Selected Writings. I like Richard Dawkins, his grounded scientific approach and this is a way into broadening my exposure to his writing.
• The Secrets of a Buccaneer Scholar. I read the free version last summer. I liked it and now want to re-read it. Nuff said. The only downside is that I've got the UK cover - but hopefully it has UK English spelling :)
• Are Your Lights On? Really looking forward to this...

The Weinberg Corner

I'm a latecomer to Jerry's work. After finding his Perfect Software book last summer I'm gradually going through a swathe of his work. The current ones are:

• Quality Software Management vol II. Great insights into observations and interpretations and the pitfalls that go with them. I'm enjoying working through this book.
• The Gift of Time. Whilst this is not a Weinberg tome it's related to his body of work with reflections from people that have worked with him or been inspired by his work. Easy to dip into essays. I've nearly read through all and I like the description of the fieldstone method - with possible applications in my day job!
• Exploring Requirements: Quality Before Design. The test for attentiveness was illuminating. Ongoing! 

My reading is typically driven by my emergent learning and divergent thinking style. I've written about this before, here. Some of these I bought second hand from Alibris - a handy site for getting "hard-to-get" books.

And now - if you're still with me - a couple of challenges for you dear reader, yes you!

Quick Test!
Without looking - How many corners did I mention? Were you paying attention?


Have you got any great recommendations? Let me know!

Testing Lessons from a Clown

Last weekend saw a visit to a children’s theatre where a famous local clown was performing.

I sat on the sideline out of the way, or so I thought, and ended up both enjoying the performance and observing some tester's lessons.

Clowns are perfect models of communication analogies for many tester-related areas, whether it is issues with communication ambiguity, communicating with non-testers or looking at the problem from a non-expert (domain knowledge) angle.

Lessons

One of the early problems was the clown’s hat falling off. 

"It's behind you." 

The clown walks in a semi-circle to face the other direction, no hat. "No, turn around." Ahh... Clown steps towards and bends down to pick up hat and kicks it away with oversized boot. Repeat.

"No, you're kicking it!" Clown, "Should I kick it? Ok." Hat is kicked into the front row - one of the children hand it back.

Clown, "Ah, so if I kick it away I get the hat back!" 

Lesson: Not every result/explanation is the only explanation!

Hat falls off again. This time, when it's regained, it's put on upside down.

"It's the wrong way round, turn it around"

Hat is rotated in the horizontal axis.

"No the other way!"

Hat is rotated in the horizontal axis in the opposite direction.

Lesson: Ambiguous input leads to unexpected/undesired output.

Lesson: Sometimes we get tripped-up by language. 

There were lots more of these types of children interactions leading to misunderstandings all following a similar theme. What was interesting to see was that the children began modifying the way they gave their instructions - they became a little more precise as the show went on!

It's great to see them learning these ideas as well as having fun. They're exactly some of the skills that any future tester would be proud of.

I got dragged into the act - used as a prop in various ways. So, for the second time in a week I became the stage masters assistant - maybe I've got the perfect circus face...

Have you learnt anything from an unexpected source recently?

Trigger-Happy vs Rapid-Fire Testing

 #qa #softwaretesting

 So, what's that then? First impressions? Is it a desirable or less desirable action?

In General

Let's assume that Trigger-Happy has more "undesirable" associations and Rapid-Fire has more "desirable" associations.

When it's an undesirable activity or approach to testing it can be associated with lack of direct questioning, or lack of processing of the results to determine the next steps. It might be a sign that the tester is "lost" and isn't asking for help or it could be a sign of a misunderstanding about an action, procedure or step.

Alternatively, it could also be purposeful action to generate data/results without interpreting them, i.e. "I'll only need to interpret them if I see something interesting - if nothing interesting happens then I won't care too much."

So what is it?

Trigger-Happy Testing is an approach to testing where the thought process is put on hold unconsciously. It might be:

• The pound-the-keyboard approach (sending in several strings of data to an input buffer without any connection between them or having a specific question in mind). [This is trigger-happy when there's no "time to stop" or "stop to change" guideline.]
• Performing some action because "I saw someone do this and it always works for them."
• "This step is written in my notebook and I always do this." Without any meaning or understanding behind the activity or its expected result.

Rapid-Fire Testing is the approach to testing where the feedback loop of result-analysis-decision is consciously suspended. Examples might be:

• Let's try and break it. When we've broken it then we'll analyse if the way we did it was "reasonable".
• "The pound-the-keyboard approach" when used to generate a batch of results which might be analysed as a group. Analysing the results may help determine a "test step" grouping, determine a pattern in the behaviour of the product or probing for certain responses.

The essence of trigger-happy testing is a warning sign and it's useful to recognise them either in you or someone else.

Indicators

This approach can be a sign of:

• Running out of ideas - try something new. This is a positive approach, when you're at a dead end - "do something" - but try to know when to understand the results and have an idea about when to change approach.
• Not interpreting the results so far from the application/platform and forming a hypothesis about what to do next. If this is done for a purpose ("I'm going to analyse later", fine) but if not then it could be a warning sign ("try anything", rather than "ok let's get my bearings and what shall I do next?")
• Receiving a new/unseen response/result from the application/platform and resorting to a "this usually works or gets me back to a known state". This could indicate not using defined operating procedures (for certain HW) or knowing if what I'm doing is appropriate.

So, why have you named something Trigger-Happy Testing or Rapid-Fire Testing?

Because I could!

But, more importantly, after taking James' Rapid Testing (read previous post) I was triggered to re-look at some of my own assumptions and observations.

I've observed this type of activity on several occasions and asked the testers what their thinking behind it was, what led them down this path.

Many times it's frustration, reaching a dead-end in ideas and not knowing what to do next. From the perspective of "doing something" and not freezing, it's a good idea. But it can be dangerous if it becomes a general approach - i.e. getting stuck in the "pound the keyboard" mode without necessarily evaluating, "why did I do that and what shall I do next?"

Other times it's a "let's just break it mentality". Also good as long you operate within some criteria of "when to stop" and to understand the results.

My motivation is to highlight when it's a potential benefit (rapid-fire) and when it's a potential danger (trigger-happy) to a tester and help testers identify some of the warning signs.

Understanding your own approach and when each is applied is fundamental to learning how to be a better tester the next time around.

Oh, I haven't finished exploring this area yet...

Do you recognise a trigger-happy or rapid-fire approach?

Roundtables and Being Promiscuous

This post was triggered by a post on "007 Unlicensed to Test" about "bounce buddies" - people to bounce ideas off.

I use and really appreciate the use of bouncing ideas off of others. It's a good way to learn and refine your thoughts. I do this in my daily work. In another way blogging is a form of the idea bouncing - putting thoughts and ideas out to generate a discussion (I very much believe in blog discussion rather than blog preaching!)

I have two approaches to my "idea bouncing" in my work environment - one is informal and the other is slightly less informal (but not formal).

The Round Table
The seed for the idea came from my manager - a regular meeting of test leaders. I developed this idea into a roundtable discussion.

If you google for "round table discussion" you'll see different flavours of how they should work. The pattern we follow closest is here.

One of the ideas behind the roundtable discussion is to have a discussion of equals - all having an equal contribution - anyone can contribute with a discussion topic and all have the option to have a say.

The semi-informal gathering is a enabler for bouncing ideas - all are busy with different schedules of meetings and so having an opportunity of a small oasis to bounce ideas around is great. We only started this a few weeks ago, but it's been very positive so far.

I'd started a similar weekly meeting - with similar intentions - in a different group some years ago. Due to the nature of the discussions this became known as the "weekly complaints meeting". The current version is staying truer to the idea of a round table discussion so far.

Promiscuous?
I don't use this term in the sexual context but rather the idea of having an indiscriminent approach to who I bounce ideas off. This is infectious (again not sexually!) If you behave in a very open way about asking questions or views on ideas then the receiver is more ready to do the same back - ask their own questions or say "what do you think about xyz?"

That is a very positive result!

Firstly, you might not have the polished idea when starting to talk to somebody - but by the end of the discussion you're going to have clarified some things - maybe it was a bad idea, there were some points overlooked, you get a lot of positive feedback or you come up with a new idea on an unrelated topic.

Secondly, the other person is encouraged to ask you their own questions (assuming you respond) - re-inforcing the importance of a two-way flow in any discussion.

For me, this type of discussion typically happens by the coffee machine, in the corridor, a visit to somebody's desk or an email.


These two ways of bouncing ideas around and getting feedback have a very positive impact on my working life (even if the feedback isn't positive!) They are great ways to communicate and great ways to encourage communication.

Blogging is another way - although less fire-sure about getting the feedback.


So, do you actively bounce ideas around and if so what methods do you use?

Real-life maps & Testing

Two things triggered me to write this post...

I saw an amplified post from Jerry Weinberg on maps, here, and the ongoing Agile Testing Days in Berlin. I had a weekend in Berlin a couple of weeks ago - and I made a note in my notebook at the time.

After one day/night there I made a note "Map chaos: Scale and position on many Berlin maps do not correspond". Yes, I also had a testing perspective in mind.

I had looked at 4 different maps (from different sources) and none seemed to tie in or they all gave me problems of different kinds.

• On the plane I had been reading an flight magazine with a feature on Berlin nightlife - unfortunately their map didn't correspond to any of the other Berlin maps I had! I guess the map maker of that one was still under the influence when making the map - so yes, that was useless.

• After landing I decided to "go native" and travel as much local transport as possible. However, interpretting the map for the link between the airport and the u/s-bahn system was not straightforward (to my non-Berlin-trained eyes!) I eventually got the message by the time I was leaving!

• In my Time Out guide to Berlin all there detailed maps of the city centre (I was staying in Mitte - the centre) didn't have the street that my hotel was on - even though it was just around the corner from a main U-bahn station. This made even getting to my hotel tricky - as the last bit I did on foot. I got there by deducing which of the streets it must be based on the information I had of surrounding streets. Yes, I would've asked a passer-by but there weren't any...
  

• Then there was the map I got from the hotel reception - at least that one had the street of the hotel on it! This was the most heavilly used map. But even this one had problems - when trying to get to a local landmark we asked a taxi driver (you'd think they'd have a good grasp on the sites) and he was non-plussed. When we showed him the map and pointed to the place with the name, he called it something else as though it was obvious!

So what's the testing tie-in?

Yes, this works on many levels. Perspective (or context): You understand the map quickly if 1) you understand how the map maker made it, 2) you have a link to a common understanding of the mapping principles.

Take contours as an example - if you don't know anything about contour lines then they're not going to mean anything on a map - and you could end up trying to climb some big hills unintentionally!

Underground/subway maps are typically drawn from the perspective of having a neat/even distribution of the lines & stations to be able to show that in a "nice/tidy" way. They're not meant for guides to "how to get to the station if you're walking above ground".

So, what were the bad assumptions with each of my maps?

Map #1: It turned out that this map was not to scale and only vaguely correct where the rivers junctions and districts were concerned - it was very scant on detail. Lesson: If the detail isn't there (and you need it to make sense of the situation) be suspicious - start asking questions.

Map #2: No real guide how to use for the bus-ubahn connections - for the un-initiated. I don't mind "using to learn" but it would be good if the map told that's how I'd learn about it... Lesson: Sometimes you just have to suck it and see!

Map #3: Although the book was a 2009 edition my hotel straight was on a line with the "wall" - meaning that the street wasn't useable on the first maps that they'd based their guide on. Lesson: Use your own "fuzzy belief" or "fuzzy logic" - use the information you can see and confirm - where there are gaps use the surrounding information to help interpolate/deduce.

Map #4: This seemed to be the most accurate and useful map but it still was understood by some of the locals. They obviously had their own map (whether real or mental). Lesson: Synch'ing your perspective to someone elses is important (and probably underrated in communication.)

What more did I learn from the exercise?
These apply everywhere, but including our daily testing lives:

• Don't take anything for granted - especially when 4 different sources are giving 4 different versions of information. Don't take for granted that whoever you communicate with have a shared perspective - check that you're both "in synch".
  

• Always try and have an assessment of your own meaning of the situation. Take the data you're getting and make your own story - how you'd re-tell the story of all the information you've just taken in.

• Lack of data in a situation where you need it to make sense of the situation? Whether this is map reading or interpretting a customer requirement - it's time to start asking questions!

I'm sure there are more map-making & testing analogies out there. Anyone?

Are you an Inquisitive Tester?

#softwaretesting #testing #qa

 I read an ariticle yesterday, here, about a study of innovators - including characteristics of the main drivers behind Apple, Amazon & Ebay.

They distilled their finding into 5 "discovery skills":

"The first skill is what we call "associating." It's a cognitive skill that allows creative people to make connections across seemingly unrelated questions, problems, or ideas. The second skill is questioning - an ability to ask "what if", "why", and "why not" questions that challenge the status quo and open up the bigger picture. The third is the ability to closely observe details, particularly the details of people's behavior. Another skill is the ability to experiment - the people we studied are always trying on new experiences and exploring new worlds. And finally, they are really good at networking with smart people who have little in common with them, but from whom they can learn."

 They then summarized these into a characteristic - inquisitiveness. A good summary I think.

Initially, I was just browsing but the more I read the more I thought "yes, this is exactly what I'd look for in a tester". A tester has to be inquisitive. Throw the communication and networking skills into the mix and you have a very sound foundation.

So, as far as skill sets go: Great Innovators = Great Testers.

Have you thought about the skills that make you successful? You are inquisitive, right?

Sapient checking?

#softwaretesting

The recent discussions about checking vs testing have been interesting. My spontaneous response, here, looked at the label (or container) of a test/check but also alluded to the fact that an automated batch of tests are much more than checks.

Yesterday, I saw a clarification of checking from Michael Bolton, here, and a tweet from James Bach:-

jamesmarcusbach: We think a check is "an observation with a decision rule that can be performed non-sapiently" example: junit asserts.

about a day ago

Thu Sep 03 23:08:34 +0000 2009

After reading Michael's post and then seeing James' tweet I wanted to test putting sapience (the concious thought process) and checking together, resulting in the following tweet discussion:-

YorkyAbroad: @jamesmarcusbach RT: check is "an observation with a decision rule ... performed non-sapiently". ME: Selection of the observation=sapience?

about 4 hours ago

Fri Sep 04 19:01:06 +0000 2009

jamesmarcusbach: Sapience is whatever cannot be automated about human intellectual activity.

about 4 hours ago

Fri Sep 04 19:24:25 +0000 2009

jamesmarcusbach: I apply to term specifically to focus on things my programs can't do, but that I need to get done.

about 4 hours ago

Fri Sep 04 19:25:11 +0000 2009

YorkyAbroad: @jamesmarcusbach I was thinking of sapience as in http://bit.ly/10797H . You shouldn't make an observation with some thought to select it.

about 3 hours ago

Fri Sep 04 20:04:49 +0000 2009

YorkyAbroad: @jamesmarcusbach Freudian slip? You shouldn't make an observation //without// some thought to select it.

about 3 hours ago

Fri Sep 04 20:06:41 +0000 2009

jamesmarcusbach: That's what I'm talking about, too. So, are you saying that if I don't think about selection, I should keep my eyes closed?

about 3 hours ago

Fri Sep 04 20:08:47 +0000 2009

YorkyAbroad: @jamesmarcusbach I'm saying if you don't think about it then don't make the observation.

about 3 hours ago

Fri Sep 04 20:11:07 +0000 2009

YorkyAbroad: @jamesmarcusbach If you make the observation without thinking then what question is it answering?

about 3 hours ago

Fri Sep 04 20:11:57 +0000 2009

YorkyAbroad: @jamesmarcusbach If you have no question in mind then why make the observation?

about 3 hours ago

Fri Sep 04 20:13:26 +0000 2009

jamesmarcusbach: I'm simply asking then: are you telling me to keep my eyes shut until I specifically know what to look at?

about 3 hours ago

Fri Sep 04 20:22:52 +0000 2009

jamesmarcusbach: I may make observations because they are inexpensive and they supply my mind with fodder for generating questions.

about 3 hours ago

Fri Sep 04 20:23:41 +0000 2009

YorkyAbroad: @jamesmarcusbach I'm thinking of 'checking' and sapience. Would you make a check without applying any thought beforehand?

about 3 hours ago

Fri Sep 04 20:40:35 +0000 2009

YorkyAbroad: @jamesmarcusbach The thinking weeds out and refines which checks you might want to make.

about 3 hours ago

Fri Sep 04 20:41:22 +0000 2009

jamesmarcusbach: Good point. That is one of the little traps of checking-- GOOD checking still must be wrapped in what Michael & I call testing.

about 2 hours ago

Fri Sep 04 20:54:17 +0000 2009

YorkyAbroad: @jamesmarcusbach Precisely! I wouldn't advocate checking without thinking beforehand. I just advocate testing - implying thinking first...

about 2 hours ago

2009-09-04T21:26:18Z

I had two things in mind with the sapience and checking combination. The first thing was to show that, according to the descriptions of sapience and checking in the above links, they are incompatible.

Did I do this? Well, if you accept that you wouldn't run a test/check for no reason then the answer is yes.

In this respect, the title of the post, sapient checking, is nonsense.

The other item: An example. I was thinking of a re-running of test cases - in my view this requires the objectives of running those cases to be understood beforehand.


Thinking before Testing?
I'm still of the view that all testing requires thought beforehand - otherwise how can you interpret results and try to help with their meaning? This has several implications.

One implication is that any automated batch of test cases to be used for re-running should always go through a selection process (review for relevance and effectiveness.) I know this might sound radical to some people but it shouldn't be!


Example
Suppose an existing/old test case is to be executed to ensure/check/verify that a new feature has not negatively impacted the existing feature?

Possible reasoning that was an input to the selection of the test case:

1. If the old feature was expected to be impacted then the test case would be opened for review/modification (I'm not distinguishing between new design and maintenance here.)

2. If the old feature had no expected impact then the test case is expected to "work", as is, then that's the measure of evaluation to make this a test. (The underlying software of the product has changed - the new feature - and the same inputs and outputs are to be verified in this new configuration/build.)

3. If we don't know if it's reason 1 or 2 then it may be for one of two main reasons: 3a) It's an exploratory/learning exercise -> testing, 3b) It's a test case that has "always" been run or never been reviewed for relevance -> time for reassessment?

Room for Testing?
For old/legacy test cases then the tester is making an assumption/assertion that a test case (or group of test cases) should work (on a new build/configuration/env.) The tester tests that hypothesis.

It's very important to keep the test base up-to-date and relevant! Filter out redundancies and keep them under review. In terms of legacy cases this should be budgetted as part of the maintenance of the product.

All of this work with legacy test cases is what I consider testing - it isn't anything without the thought process to guide it.


Cloudy? Foggy?
The problem with trying to define checking is that it is so tightly intertwined with testing that I don't think it adds value into the communication about a product or project.

If, as a tester, you're trying to help your project understand the risks associated with a project and communicating issues found then I don't think there is any need to say x issues have occurred during testing and y issues have been observed during checking.

I can think of checking as an element of testing, just like a test step might be an element of a test case. But on its own there is no value in communicating it to stakeholders.

I think there have been many communication problems tester<->non-tester in the past (and probably still continue today) - the information needs to be tailored to the audience, which is a great skill for a good tester!


And Finally...
I can't see the example where a test/check can be selected without any thought process. To distinguish a group of test steps, under a given context, as either a check or a test is totally unecessary in my working environment.

Keeping the storytelling good and the quality of the reporting and feedback high and consistent will lead to reduced misunderstandings and communication improvement. I know, easier said than done!

I suspect communication problems (and not a lack of terminology) have been the root causes for some of the cases where a distinguishment between testing and checking has perceived to be needed.

Checking without sapience doesn't hang together. Why would you do something without a purpose?



This post is my effort to understand the discussion - using my combination of learning, critical thinking and past experience.

What does your experience and analysis of the discussions tell you?

Feedback: Friend or Foe?

#testing #feedback

I'm currently making notes on learning opportunities and one aspect that I thought needed separate treatment was on how testers give feedback to each other.

If you think about your experience in the work environment, on testing forums, twitter and blog comments are you keeping it professional - with the emphasis on professional courtesy?

I've seen examples on forums where sometimes a "seemingly innocent" comment or question is met with annoyance, condescension, rebuttal or frustration. Let's call it "unfriendly feedback".

Some Unfriendly Feedback Types

Annoyance: Maybe the responder had a bad day?

Condescension: What's the responder trying to show? Hopefully they are not falling into the "knowledge is power" syndrome. This is not a team-player response.

Rebuttal: This can be a legitimate "critical thinking" response. However, remember that the essence of critical thinking is about understanding and clarifying communication and not about being "critical".

Frustration: Maybe the question was something that could have been googled or someone has just asked a re-formulation of a previous question.

My Take?

One of my professional mottos is that "I'll help anyone but won't do the work for them."

This can be a fine line - especially for newbies - sometimes they don't know where to start, don't know which questions to ask first, how to formulate the question and are just keen to "start". That enthusiasm is good; it's an ingredient for future interest and development.

So, yes, answering questions from newbies can sometimes be hard work - the key is to direct and discuss threads for investigation. But remember, questions from non-testers can also sometimes be hard work!

These feedback types are equally visible in many peoples' work environment. They are not limited to tester<->tester, it occurs tester<->non-tester and non-tester<->non-tester also.

I've seen consultants treated less than courteously precisely because they were consultants - to their credit they responded professionally. This falls into a wanabee-superiority response and I don't think of it as a feedback type - more a cooperation problem.

Give it to 'em

If you're giving feedback keep it professional, courteous or light/humorous. If you can't then count to ten and try again. The important thing is not to withhold feedback just because you're biting your lip or counting to 10, 100, 1000,.... Give the feedback - it helps the receiver and it helps you!

If you're on the receiving end of the above type of feedback then rise above it. Don't bite back and lead by example. Give it back to 'em - an example of professionalism that is!

Note. When I was a new tester I was on the receiving end of the condescension type of response from a fellow tester in my organisation. I persevered - and saw that tester subsequently sidelined. If you're not a teamplayer then you're heading for a team of one!

I know! Easier said than done at times!

Have you been on the receiving end of unfriendly feedback? How did you respond?

Have you recently given unfriendly feedback? How do you think it made the receiver feel?

A Plague for Critical Thinkers?

#softwaretesting

Are you a critical thinker? Do you know? Do you care?

If you’re a software tester then the answer to the first question is probably yes.

Plagues

It’s with this aspect in mind that I’ve been reading James Whittaker’s series on the 7 plagues of software testing. I enjoy reading James Whittaker – whether in book or in blog form. He has great insights and viewpoints.

I have become increasingly perturbed by the “7 Plagues..” series of blogs. Yes, there has been the research, analysis and insightful observations, but the thing that has been nagging at the back of my mind has been the use of “plague” in the titles.

Critical Thinking

I believe most testers are critical thinkers – they need to be due to the nature of the job – whether they realize it or not. It’s with this in mind that I feel uneasy about the use of plague – this is more of a rhetorical device (in the parlance of critical thinking.)

Critical thinking (to paraphrase) is the study of arguments/suggestions to analyse the underlying proposals and conclusions to determine whether the argument “holds together”.. In this field the idea of a rhetorical device is one where a loaded/weighted phrase is used without contributing anything to the proposal to advance an argument. I.e. it’s emotional weight is intended to sway the decision rather than a clear argument.

Clarity

Clarity is king in software testing. Yes, there is room for analogy, but when a series of articles pounds the idea of a plague affecting software testing then it is reasonable to assume that some people will think of software testing being “plagued” by certain problems…

I am not going to go through the individual plagues here – I’ve made comments on some of the posts (both agreeing and disagreeing with the contents.) However, in summary, I can say that James highlights some potential pitfalls that testers (and the industry at large) can fall into.

I suspect the meaning was “plights” that can affect tester/industry rather than “plague or epidemic” – maybe even meaning that something was endemic. However, I was tripping up on the use of “plague” and that partly distracted from the content.

“7 plagues of software testing” sounds much sexier and has a better sound-bite than “7 pitfalls for testers and the software testing industry”. So that’s probably the main reason for the titles.

A Plague for a Critical Thinker?

The problem for testers is anything that distracts from the root cause, the underlying problem - the "reduced" argument. Rhetorical devices "get in the way" - they cloud the argument and don't help the analyser understand what a person/statement is trying to say.

So to repeat, "clarity is king".

Critical mass?

Sound-bites should not be what we strive for in the industry. Consistency, clarity and honesty will win more friends and supporters in the long run.

Software testing needs to achieve a critical mass of critical thinkers to carry the profession forward.

So: critical thinking – if you didn’t care before, do you care now?

Moonlighting #2

#softwaretesting

Reflections on the Test Republic Test Challenge

(Not the 80's series with Bruce Willis.....)

My look at the testing challenge on Test Republic. It was interesting for a number of reasons. It re-affirmed the need for some key skills and highlighted that learning opportunities are all around.

I liked the challenge because it gave me a couple of learning opportunities. One was to try something new (explicitly testing a website) and the other was to write an example of a bug report that would be a good/useful example to others. It's sometimes easy to preach about technique, but sometimes just demonstrating is equally instructive.

I approached it from the perspective that it would give me some reminders about things to have in mind during our daily work.

The challenge didn't give any guideline on how the testing should be done (it's aim was on the fault reporting and motivation of fault fixing), so I approached the test subject from a usability angle using an ad-hoc/exploratory method.

Usability gave me the starting point and the ad-hoc approach limited my test session (30 minutes) and gave me the thread for the test step development. My raw test notes (to be published later - I want to add some additional thought explanation) show the test steps with observations and comments and how they lead to the successive steps.

The challenge didn't give me any groundbreaking moments but it did throw up some very useful reminders of skills and techniques to have in mind.

Take a break

#1. Emphasied the importance of walking away, taking a break, unplugging

When you finish a piece of work during your working day (test session, writing a report or reading a document) take a break. This might be taking a coffee, a five minute leg stretcher, talking to a colleague or walking the dog.

This take-a-break activity allows you to re-charge (resting), re-focus on your next activity and allowing a little non-focussed-thinking - this might be drifting into problem solving whilst doing something else (for me this was pushing a pram with my sleeping daughter.) Some of my best brainstorming happens this way.

The challenge didn't make me do this - but it's something I consciously employed as the challenge was an "on-the-side" activity.

The Power of Review

#2. Emphasised the importance of review, inspection and re-review

This is something I've been thinking about recently in preparation for a different post - the amount of time spent on reviews and analysis of work, reports and feedback. This is a huge resource for any testing professional.

Review and re-review was demonstrated as being very important in this exercise with two examples. One was to do with the actual fault decription/reporting and the other was to do with reviewing the test session notes.

Fault/Bug Report

Reviewing the reporting. When writing a report - whether a bug/issue report, a test session/execution summary or some presentation of findings it's always useful to take a break after it's "finished" and go back and re-read. Look at it from one of two perspectives: 1) proof-reading - does it make sense, does it say what I intended?; 2) will my intended audience/receiver understand it?

Test Session Notes

A re-review of the testing notes threw up a fault that I didn't pick up as an obvious fault the first time around. Whilst writing a report for another fault and explaining the steps involved (for reproduction purposes) the fault became obvious.

This emphasizes the need to review test notes, environment changes, lab set-up now and again. There is always information to be found. Sometimes it might show that all is as expected. Other times it might raise a question "why is this step skipped, this step inserted, this env variable set, etc, etc."

Keep it clear and understandable

#3. Illustrated that there are useful learning opportunities for testers in the most common of places.

A well-known website. Go find the errors... Ok, so you find an error, now describe it...

Sounds simple? Yes, but doing the simple simply is something everyone should be practising now and then. Describe something in a clear and straightforward way that's directed at your audience.

In this challenge this meant describing the faults found in a way to get the message across - that the fault is understandable, along with all the basic data, how to reproduce and some motivation for the fix to be implemented.

I'm a great advocate of "keeping it simple" or "simplify, simplify,..." - it's very easy to preach, but putting your words into practice is something we should all be able to do.

Push your own boundaries

#4. Learning is (should be) fun. Don't be affraid of mistakes.

Don't be affraid of doing something new, different or unfamiliar. This is something where you can always find a learning opportunity. You might "fail", achieve or exceed your expectations but you'll always learn something.

Whether this is to do with a new test technique, approach or a way of analysing a problem there are always opportunities to pick something up and improve.

Maybe the test technique didn't work for you. Why? Was there some preparation missing or do you need to go back through the basics?

Tip: Think about how you might explain that technique to someone not familiar with it. This challenges you to understand what you're talking about and is a great way to clarify your own thoughts. The next step is to do it in person - then you might get challenged for real.

If you never make a mistake you'll never demonstrate that you've improved. The Japanese work ethic is very open to making mistakes as long as you learn from them. If you're learning to ski or ride a horse you'll be told that if you're not falling over / falling off then you're not trying hard enough.

The "What's in it for me?" Priniciple

#5. Influencing factors - the perspective of the receiver or the report

The challenge called it bug advocacy. Some might know it as motivating your request. But however you understand it you can think of this as an example of the "what's in it for me?" scenario. I wrote a post recently, here, where I touched on influencing skills that are useful for all testers.

The bottom line for any motivational argument is how to answer the other person (the one you're selling to) when they're thinking "what's in it for me?" If you just tell them about you're problems (bug #n fails due to xyz, please fix) without any motivation then you're likely to be disappointed.

(For best results do the influencing indirectly - let them come up with the idea.)

So the problem might then be presented as: bug #n fails due to xyz, impact of failure "abc", estimated cost of solution "def", benefit to customer "ghi", benefit to supplier "jkl". If the supplier/developer then turns around and says we'll fix this as a priority to be delivered xxx, then you've done your job - you've convinced them of the need for the solution. Then the bargaining can start about actual delivery times, emergency/interim fixes etc.

This principle is an influencing skill - it's something all testers should have (to a greater or lesser extent) and it's something that needs continuous practice to stay sharp.

The above five areas occurred to me during the course of the challenge and during the susbsequent assessment. They are skills/techniques useful for all testers during their day-to-day work.

I'll post my actual entry at a later date with some extended notes on the approach and steps for deduction and reporting.

When did you last think about the skills you're using and which need improving?