Testing as Feedback

Reflections and a thought experiment connected to testing

I saw this tweet from Martin that triggered my thinking. I noticed it because the tweet was getting several responses and Martin poses an interesting question, as he usually does.

I looked at a few of the replies. The testing community is full of insightful thinkers. These two caught my eye:

Of course, I had to comment on this because it goes to the heart of any hypothesis based and social interaction development.

Then I wondered how the replies were being formulated, and even how they were being received.

Maybe I'm over-loading Martin's original intent, but I'm trying to extend his line of thinking, so bear with me.

Based on Martin and Jari's questions, an approach to analyzing a situation is to consider an item (artefact), activity or set or interactions and consider what would the situation/system look like if you flipped the meaning 180 degrees; an opposite, or antonym. Then look at that resulting situation and consider do you learn anything about the original.

Stumbling block?
The idea of what testing is and isn't and who does what for what purpose, is not clear. That was part of the reason for my "data question" - who is answering from which perspective and how does one know. If this was an anthropological or qualitative analysis study the original question might have been set up differently - but then it might not have gotten the same engagement or attention.
Actually, I think the biggest stumbling block with the replies to the question posed is that the respondents are not indicating what they think of as testing for the "opposite of testing" answer, which is always going to be limited in twitter.

As interesting as many of the replies are, it's difficult for me to use them to understand "testing" or the system of software development from the replies.

That triggered my own thought experiment.

For this experiment, I will not try to define testing directly, but I will use another element in software and product development to understand the potential impact of "no testing". So, the simplistic assumption is that the opposite of "testing" is "no testing" and that this will be observed by looking at other characteristics in a product development lifecycle, in this specific case feedback.

Approach
A system can be analysed by altering a meaning, or purpose, for one of its components and see how the resulting new system might perform. Another way is to remove that component.

Compare to a picture of a group of people - what does the picture tell you about the group of people, their interaction, their context or their potential. Now remove a person or an item. What changes about the whole system or story do you now see (search for photoshopping, green screening or photo manipulation for examples) - e.g. do you see things that were obscured by the now-removed object or person.

Why Feedback?
It is difficult to look at a system of interactions that makes up product development and isolate components, activities and elements of output. For me, feedback is an interaction between people or information about a system or product - it can be internal within the system (company, teams) or external (information received or gleaned from outside the company or teams). Internal feedback - to me - has a value, care and thought attached to information or data points, maybe even discussion or talking points.

Compare, "login doesn't work" to "login doesn't work under circumstances X, Y & Z" or "login performance has changed since version a.b.c". These are all potential forms of feedback - I claim that the second two have more value than the first. A team or product owner can react on all of these, but my claim is that it's easier to make decisions based on #2 & #3. Note, I'm making no claims on who or what generates this type of feedback.

Note, my experience is that good testing generates good feedback in systems of product development, which aid a variety of decisions and analysis  - sometimes that comes from individuals and sometimes from systems that individuals have put into place to facilitate feedback and analysis.

And so, riffing on testing being removed from software development and the impact on feedback:

Reflections
There are other aspects of product development that one could look at besides feedback.

The timing is implied in these tweets - the implication is that if it takes longer to get feedback and be able to make a decision about it then you (team, product owner or business manager) miss the opportunity to adjust course, make a different decision, conclude that an experiment doesn't / didn't work and adjust investment, etc.

My implication in these tweets is that testing provides valuable feedback to teams and companies. 

Another implication is that feedback is in the system of product development and that ultimately it is a social interaction between individuals.

I do not make a claim that testers own feedback. Good testers can contribute to good feedback systems and delivery.

Extending that: the proximity of feedback to when code is written is vital for hypothesis or experiment-based development.

I think I've probably generated a few question marks that need explanation and extension, but I will leave that to another post.

Quality-Value: Heuristic or Oracle?

The other week James Thomas wrote down some thoughts related to “Quality is value to some person”. I added some diverse thoughts - a mini-brain dump. This post is an expansion on one of those thoughts.

Start point

I made the assertion that the heuristic/statement (“quality is value to some person”) had been primarily used in a software testing context as a tool to illustrate that stakeholders are the gatekeepers of quality and not necessarily testers.

To me this is ok - but ultimately not a proactive approach helping the team, company or organisation work out how to delight it’s current and potential customers with good/acceptable/superior value and quality.

Point to explore

The point I made on James’ blog that I want to expand on is:

“4. In what scope is the “quality is value to some person” used in SW testing? I don’t know if it really matches Jerry’s original intent. I think it has been used to find a responsible stakeholder to discuss test results & objectives with, and probably also to aid testers to explain that they are not the sole arbiters of quality.  

4.1. I read the intent (from Jerry’s QSM) as highlighting a relationship and a perspective (i.e. subjective rather than objective) - which by nature can’t (usually) be static. I haven’t really seen/read of anyone applying it from this perspective to SW testing. I wonder how it would look…”

So, the question I will explore- if this is a transient and subjective statement - what use can it have to software testing or software development as a whole? I make some claims (statements and opinions) and questions - partly re-anchoring the context in which the statement* could be used:

1. Who is making the statement*?
2. When is it useful to make the statement*?
3. To what scope does it apply?
4. It’s bigger than software testing

Who is making the statement*?

I assert that it makes no sense for only certain people in a development team or project to have this view (“quality is value to some person”) - therefore, this is a team or project view of quality, or preferably a team together with a product owner or even customer view on quality.

This starts to imply a synchronised view on acceptable goals for a product, or feature, or vertical slice of a feature - as a goal for the team/group rather than individuals using it as a reminder (or in the worst case a defensive and passive position).

When is it useful to make the statement*?

I assert it is a useful reminder at the start of work (goals of a product, or feature) - these may be preliminary goals used in a prototyping activity or a hypothesis on what a customer might want. Doing this at the start is establishing a common goal for the team (or project or program, etc). This is not a statement useful for gatekeeping but useful for goal alignment - alignment of subjective views if you will.

To what scope does it apply?

I assert this applies to the whole development and delivery chain. Therefore, this implies that synchronisation between development and delivery teams (or even a DevOps set-up) would be desirable. Again, the alignment is about aligning common goals, not gatekeeping.

It’s bigger than software testing

Hopefully, this point is obvious?

Applying the statement* to product development (and delivery) I assert that it soon becomes clear(?) that it is much more than software testing and should be running through the whole chain. It doesn’t need to be a defence mechanism used by testers if alignment on goals has been achieved within the team, program, organisation, company.

Flip side? From Heuristic to Oracle

Suppose individuals or teams are using the statement* as a reminder/defence mechanism to illustrate that one or more stakeholders need to take a view on quality - what could this mean? I’d interpret it as a symptom of the team/organisation and its maturity with regards to delivering synchronised development & deployment quality. 

Another way to look at it: it’s a canary call for silos and local-optimisations. You could say it could be used as an oracle for spotting an organisational problem! More on that in another post…

*Statement: "Quality is value to some person"

Some Software Development & Testing Challenges 2016

So it's 2016 and I have been reflecting on some of the challenges I see for Software Development, with emphasis on Software Testing.

Continuous Integration

Everyone knows what this is right? The concept has been around a while* - everyone has been there and done that, if you read the hype. But who is innovating?

A lot has been written about CI and its place in support of testing... Or has it?

Some Challenges

1. Massive parallel script execution against the same target drives a re-think on test framework design, modelling and creation - impacting data modelling and needs for flexibility in frameworks and harnesses
• This is a move away from single isolated and independent "tests" on a stateful application. It will trigger a change in test script approaches. Where is the work on that? Pointers gratefully received...
• I have seen some research on "multi-session testing of stateful services", but more is needed.
2. CI script execution and studies showing the effectiveness of dynamic test script selection strategies for team or testing support
• I see that as a rule-driven approach to setting a series of checks on commits, e.g. (1) which checks cover my updated code-base (execute and result), (2) which whitespots in my codebase are there now (report)
• Where are the studies or experience reports, where is the work?
3. There are socio-technical challenges with CI use and implementation. Technology is the easy part, the soci-technical part comes in when organisational issues and preferences distort the technology choices. This might range from "we have always done it this way" to "the language or framework of choice here is X so everyone must adapt to that".
• CI is a development approach, and is distinct from testing. It's like an extension to compiler checks**. Thinking around selecting and adding to those "compiler checks" needs to be dynamic. Experience reports, empirical studies for this?
• There is a danger that "testing" is driven into a TDD/Acceptance Test-only mode.
• I would like to see more research on organisational and soci-technical challenges around software development...
4. Are people really going all-in on cloud and virtualization technologies to solve their CI related bottlenecks? Mmmm...

Software Testing

Some Challenges

1. Detachment from Software Development
• This can be seen in various forms
1. Distillation down to process on "testing" only - the ISO 29119 papers are a classic example of this. This is the "reductionist" approach to a wicked organisational problem - not very sophisticated and with a high risk of solving the wrong problem.
2. Other examples are some/many software testing only books - yes, it can be good to highlight the testing and testers role and special challenges there, but until you start from software development as a whole (systems thinking approach) then there is a high risk that you are making a local optimisation. Another reductionist approach, liable to solving the wrong problem.
• So, software testing focus -> good; software testing interplay and interaction and CONNECTION to the creative process -> better.
2. Mis-understanding of the software testing challenge - how to link a creative process (software creation and positive and confirmatory tests and checks) to a challenging process (testing for issues, highlighting problems, testing for risks)
• Many organisations focus on confirmatory tests - a proxy for customer Acceptance Tests - as an MVP (minimum viable process), i.e. a proxy "get out of gaol card". See Myers [2] example of testing in an XP approach is an example here.
• Myers [2] first wrote about the psychology of software testing. However, Martin et al [4] make the case for reframing this as an organisational approach/problem. Rooksby et al [5] observe the cooperative nature of software testing.
• More studies on satisficing the organisational needs please!!
3. Lack of soci-technical studies and research into software testing and its part in software development. Rooksby & Martin et al [4] & [5] performed ethnographic studies of software testing to highlight its cooperative and satisficing nature. This called for further research
• Sommerville et al [6]:
• "An over-simplification that has hindered software engineering for many years is that it is possible to consider technical software-intensive systems that are intended to support work in an organization in isolation. The so-called ‘system requirements’ are the interface between the technical system and the wider socio-technical system yet we know that requirements are inevitably incomplete, incorrect and out of date."

The sooner we stop treating software development, and especially testing, in reductionist approaches, consider the socio-technical aspects - especially for large and complex systems - the better. And, today, most systems are inevitably complex.

Got any pointers to recent advances? I'm all ears...

References

[1] Li, Chou, 2009, IEEE; A Combinatorial Approach to Multi-session Testing of Stateful Web Services

[2] Myers, 2011, 3rd ed; The Art of Software Testing

[3] Rethinking Experiments in a Socio-Technical Perspective: The Case of Software Engineering

[4] Martin, Rooksby, Rouncefield, Sommerville, 2007, IEEE; ‘Good’ Organisational Reasons for ‘Bad’ Software Testing: An Ethnographic Study of Testing in a Small Software Company

[5] Rooksby, Rouncefield, Sommerville, , Journal of CSCW; Testing in the Wild: The Social and Organisational Dimensions of Real World Practice

[6] Sommerville, Cliff, Calinescu, Keen, Kelly, Kwiatkowska, McDermid, Paige, 2011, Communications of the ACM; Large Scale Complex Systems Engineering

*I led the architecture work on a multi-stage CI system in ¨2010

**yes, a big simplification.

On Being Replaceable, Role Traps and Adding Value

I said to one of my managers several years ago that one my main goals towards the team and organisation was to make myself replaceable.

He nearly fell off his chair....

As I was a leader within the organisation - if I become replaceable what does it mean for the position I have or the value of the work I'm doing? Oh, I say this to most managers (partly as a test :) )

It is important to distinguish how people are viewed within teams, projects and organisations. In this context there are two main views:

1. What they bring in terms of personal skill, knowledge and capability.
2. What they bring in terms of operational responsibility or ability.

Traps

Where teams and organisations have roles or specialized people (e.g. testers) then there is a risk of attaching someone's behaviour to that of the role.

Or the other way around, 

The skill of a role is often thought of in terms of a person/s that perform it.

(or have performed it). 

Or:

Experience is implicitly labelled as an exemplar. 

Example
In the past I have sat in a coordination meeting with a team and asked an incisive question such as, "Customer X have feature Y adapted/changed in their network and so your development of feature Z should also look into interactions between features Y & Z". This means that a potential risk is reduced - or at least some investigation is started to reduce a potential risk.

Now, if I miss a similar meeting with another team, there might be such a question not asked that might mean such a risk is not caught or it is thought about later, meaning some additional work. (Note: These coordination meetings might be some form of reference/project or expert aid to the teams.)

Question: Who was the target audience for my question? 

Many people would think it was the team doing development of feature Z. My target audience is everyone though - it's not just about the question, it's also why it is asked (some might call that the context) - in this case to help all realize something they hadn't seen beforehand.

I don't ever see myself as the person who asks questions others don't - but more as someone who might help others ask better/different questions next time. If I haven't passed on some of that capability then I've failed.

Adding Value
There are two main ways I add to the team, project or organisation.

• I point out the differences between what I am bringing as me, and what I am bringing as "performing role X". They might overlap at times, and not at others - and it's important to help others with that difference.
• I ask "why?" a lot. Not to be a pain (even if that's how it might sometimes be seen), but to help understanding.

But, how is this "adding value"? It's highlighting behaviours that others can adopt that are not "owned" by a role.

The "Why?" Question
One of the most common questions I ask is "why?". It's a sign of wanting to learn what someone else is thinking. It's a sign of wanting to flush out, and clarify, assumptions. The "why" question is one of the cogs of dialogue and understanding.

So, if I am the only one who is looking out for dialogue and understanding then there might be a problem in the organisation. However, in my experience, someone has attached the "why" question or "the types of question that I ask" to the role I had - so it becomes "role ABC asks those type of questions".

That's where I need to remind people around me that these are not my questions, I don't get territorial about such questions, and that if I'm not around and the question pops into your head, go ahead and ask it.

Learning Organisations
Organisations, teams or projects that want to grow and learn must be very careful about roles -> sure, if someone is the designated decision maker let them make it. Until that point, there's usually plenty that can be done without the decision-maker - including asking questions.

Sometimes people (teams) need to be given permission to think for themselves - strange as that may sound.

Lessons

• It's not what skills you bring to the table, it's what you leave behind for others after you've gone.
• It's not what attitude you bring to the party, it's the positive change in attitude you leave behind that's important. (Sometimes, that means more people are prepared to ask, "why?") 

Doubt
A typical question I get about achieving replaceability is: don't you do yourself out of a job, or no one needs you after you've improved others?

In a team, or organisation, that has a constant ambition of improvement that is never a problem - there is always a new problem to work on. As Weinberg said (I think), when the most important problem gets solved, problem #2 gets a promotion. Sure, it's a different problem and may take you out of your comfort zone, but ultimately, that's how you improve.

Some people treat knowledge as power and hold onto it. Unfortunately, those are the folks that can become one-trick ponies or get bypassed by progress...

And finally...
So, to me, being replaceable is positive - it means I've added value - it means I've given others a tool for thinking more clearly - it means I can carry on improving, learning and adding value.


Are you adding value? 
Are you leaving something on the table for others when you move on?

Examining the Context-Driven Principles

At Let's Test 2013 James Bach had a keynote about the Context Driven Testing principles, some of their origin, usage and implications of them. An interesting piece of trivia that James revealed was that the basis of the principles were written down very quickly, with some refinement afterwards.

He discussed a little about the implicit meanings behind the 7 principles, but the slides were skipped through so I didn't see the details. However, by this point at least one question occurred to me.

My question in the open season went along the lines, "The principles were mentioned as a litmus test to gauge if someone was context driven. However, adopting a scientific approach (ie one of searching for information and not settling on an assumption), there may well be more than 7. How do we add number 8?"

An underlying question was (for me):

1. If the principles were written so (comparatively) quickly why haven't they been challenged from testers claiming to follow or adhere to them?
2. Indeed, isn't this a good exercise for a tester - context-driven or otherwise?

Therefore, I thought I would take a look at them - as a (comparatively) quick exercise. James did mention that Markus Gärtner had extended the principles as an exercise previously. I remember seeing a discussion on the software-testing yahoo list some time ago and checked it out - and sure enough I'd discussed the possible extensions there, and thought it was a useful topic to revisit.

The 7 published principles, ref [1], 

1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.

My Approach
In assessing the principles I used two approaches, (1) Critical/Logical consistency - do the propositions support conclusions or implications?; (2) Are the principles testable (falsifiable)?

Assessment
#1 [Unchanged] I tried to extend this one, but I think it is rhetorically good - succinct and a good start to the principles. This can not be tested but is inferred by induction.

#2 [Remove] I could think of  removing/replacing this one. It is a derivation of #1. The essential content here is that "Practices evolve over time" - I thought of making this my #2, partly to remove the distraction of "best practices" (that would be a derivation of these principles), and partly to emphasize the evolution of practices. However, I finally settled on having no #2.

#3 [Rephrase] The principle is an assertion. For this one I want to emphasize that the project, the processes and people form a system and that they change over time. The importance here is to illustrate that the system is not static, it is complex because of people (and their emotions) and as such, working with such a system is not a trivial activity. Therefore I would re-phrase.

#4 [Remove] This is a derivation / implication of #3, an application of the principles.

#5 [Unchanged] Again, succinct and difficult to refute.

#6 [Unchanged] An assertion that where good software testing happens there is a high intellectual demand.

#7 [Remove] This is unnecessarily wordy and sticks out as too different from the previous principles. It also drifts into certainty and seems like an untestable principle, and so I would remove it.

Addition
#8 Here I want to tie the system mentioned in the rephrased #3 and good software testing.

Result

1. The value of any practice depends on its context.
2. -
3. Projects and people form part of the system that work together, where products and practices evolve over time
4. -
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. -
8. Good software testing encapsulates the system of project, people and practices that work on building the product.

So, I ended up with 5 principles. The application of these would then produce variants and clarifications. For instance, the best practice item is derived from #1. Understanding systems of people, emotions and time is derived from these and many, many more.

I stopped there, but if I spent more time could probably refine them somewhat. But they are good enough for this exercise. And as James suggested - if you started from scratch, you might well not end up with the original 7 principles.

So, how would your interpretation look?


Reference
[1] http://context-driven-testing.com/

The Linear Filter Trap

Or: Illustrating Systems Thinking with Proximate and Distal Analysis

I was having a discussion the other day where we were looking at some problems and discussing whether it was sufficient to treat symptoms... I remarked that by treating symptoms, rather than root causes [notes n1], and not allowing time to find the real problems then we would, in many cases, be fooling ourselves [notes n2].

Why? (I was asked)
People like to solve problems that they can see - which means we have a tendency to "fix" the problem we see in front of us. This occurs more often if the problem appears to have a "straightforward" fix -> I think of this as a form of cognitive ease in action. Digging for root causes is a challenging activity, and we sometimes want to believe that the cause we identify is good enough to fix. For another example of cognitive ease, with best practices, see ref [2].

An illustration of this - that I have seen in one form or another - is that we settle for the first solution without understanding (or trying to) the root cause. There is no guarantee that fixing a symptom will make the problem better. Many times, the problem improves for a while, but then re-occurs in another form. Now there is a "new" problem to solve, which usually has the same (or similar) root cause, so from the system perspective it's ineffective [notes n3].

Ineffective, because many problems in processes and organisations are often non-linear, but we often try to solve "linear" problems, and...


Linear Filter
I expanded, that I think of this from a systems thinking perspective as applying a "linear filter" to a "non-linear system".

What? (I was asked again) Linear vs Non-Linear?
Non-linear -> multiple interactions affect the output vs linear -> the output is directly proportional to the input. So the application here is that there are usually multiple causes for a problem -> when I perform an assessment after a root cause analysis (RCA) activity I usually take them root cause by root cause, in the order that we think will have the biggest impact on improvement.

Ok, time for a....

A Real-life example
A fault (bug) report was written by a customer -> initial RCA shows that some "test" was not executed that would (in theory) have caught the problem. This is a symptom and a "linear" view of the problem. The "linear filter trap" is to then consider this as the root (or most important) problem. [notes n6]

Digging deeper shows that the team had it's priorities changed during execution, to make an early drop (resulting in some negative and alternative use-cases being delivered later). This, in itself, is not a problem but the communication that was associated with the "early drop" didn't reflect the change.

In this case, some of the underlying problems were a set of communication issues:

• Partly in the team connected with their story [notes n4] (especially their testing silent evidence [notes n5]), 
• Partly connected with the stakeholder that changed priorities and may have had a duty to follow the change of expectations through the delivery chain and what that might mean at those different stages, and 
• Communication with the customer to ensure that they are aware of the staged delivery, and what means to them.

Another example of a root cause analysis can be found in ref [3].

And finally
Tackling and fixing symptoms is a very natural activity - very human. But it is not always enough. Sometimes it is enough - it depends, of course, on the scale of the problem and the cost of investigating the underlying problems and tackling those. Sometimes, the underlying problems cannot be fixed and it is sufficient with easing the symptoms.

But I believe, as good testers, it is important to understand the difference between symptoms and root causes, especially where it affects either the testing we do or affects the perception of the testing we do. This is important where a perception of "testing or testers missed something"... So,

Be aware of the linear filter trap!

Notes
[n1] In philosophy and sociology, root causes and symptoms are usually referred to as distal and proximate causes, see ref [1] for more background.

[n2] Slightly naughty of me, playing on the fact that most people don't like to think that they are fooling themselves, but that's a different story...

[n3] The times when it might be effective are when we (some stakeholder) is prepared to take the cost of fixing some problem now. A problem here is that stakeholders that are project-driven have, by the nature of the task, a propensity to see only as far as the end of the project. A product owner may have a different perspective - bear this is mind when someone is deciding whether it's a project problem or a product problem - or even a line organisation problem.

[n4] Story here means the story about the product and the story about the testing of the product.

[n5] Here, testing silent evidence refers to the elements not tested and thus not reported - their significance is assumed to be not important. For further background see ref [4].

-->edit-->
[n6] I should add that the problem with the trap in this example is that I have seen this in the past trigger one of two responses: (1) A perception that the testers are at fault, which becomes a myth/rumour with a life of its own ; (2) A knee-jerk reaction to implement some extra oversight of the test displicine or team as a whole -> in the worst case it becomes a desire to introduce some additional "quality gate". This is a good example of when reacting to the perceived symptom is both ineffective and counter-productive for the organisation.

References
[1] Wikipedia: Proximate and ultimate causation
[2] The Tester's Headache: Best Practices - Smoke and Mirrors or Cognitive Fluency?
[3] The Tester's Headache: Problem Analysis - Mind Maps and Thinking
[4] The Tester's Headache: Mind The Information Gap

What's on my current reading list?

I take inspiration and ideas from a wide range of literature, other testers and sometimes triggered by something from leftfield.

I thought I'd jot down my current reading list (I last did this for my summer09 reading, here) for a couple of reasons:

• It's good to walk-through what you're currently doing (reading) and why - sometimes you might be reading something obscure, but for a particular reason.
• If you're like me, I get ideas from what other people are reading - so this is part "here's some tips" for other people, but also a hope that readers will send in tips to complement my reading - so that's your challenge at the end!

The Scientific Corner
After attending the RST course in March I had the urge to rediscover the scientific method. So as a step in that direction I started reading the following.

• Philosophy of Science: A Very Short Introduction. Well, I'm into science and philosophy, so why not start with "What is Science?" Great stuff! 
• The Oxford Book of Modern Science Writing. This is a collection of excerpts from some outstanding scientific papers and books of the 20th century. I've devoured a few already - a combination of good writing and interesting science.
• Can a Robot be Human? As a tester and wanabee-layman-philosopher this is right up my street. There are interesting slants and many questions - the sort that make you think! Great for a critical thinker, lateral thinker, divergent thinker and a tester!

The Systems Thinking Corner
Looking at complex problems and trying to understand their complexity is interesting to me - it also helps in my daily work.

• What The Dog Saw. This collection of Gladwell articles (I think most can be found on the NewYorker site) is an eclectic mix with his distinctive take on them. Very interesting and insightful.
• What Every Body is Saying. Observation is a necessity (as a tester) and so why not try it on people around me? I'm certainly no expert but it gives me a few insights and maybe it will help in the odd discussion in the future.
• The Black Swan. Enjoying this slightly-scholarly book and I'm taking my time with it. Lots of good things for testers to think about. These types of events are all around us - just think volcanos!
• The User Illusion. I can't seem to finish this book - I've been reading it for years - it has tricky parts - I read and re-read parts. It's an exploration of conciousness and how the unconcious mind processes so much information. It introduced me to entropy, information theory and to Gödel - satisfying the mathematician in me and giving me another insight on testing problems.

The Testing Corner
I have plenty of software testing books - some of which are constant reference material and I've written about before. However, I don't think I've mentioned these two before:

• Beautiful Testing. I started dipping into this last autumn and it then fell by the wayside - will re-visit before long. Some interesting chapters - I'll wait until I've finished it before giving a verdict.
• The Art of Software Testing. This is a pure reference material for me. I picked up a second hand first edition last autumn - did the triangle self-test and enjoyed some of the "phycological" aspects of software testing.

The Recently Finished Corner
I think I have more than one, but this is the one that stands out.

• Blink. Well, I'm into how the mind processes information, why that sometimes works and sometimes doesn't. As a tester I can relate to why people can be hampered by too much information - so I liked the war-games description!

The Not Started-Yet Corner
Next up...

• A Devil's Chaplin: Selected Writings. I like Richard Dawkins, his grounded scientific approach and this is a way into broadening my exposure to his writing.
• The Secrets of a Buccaneer Scholar. I read the free version last summer. I liked it and now want to re-read it. Nuff said. The only downside is that I've got the UK cover - but hopefully it has UK English spelling :)
• Are Your Lights On? Really looking forward to this...

The Weinberg Corner

I'm a latecomer to Jerry's work. After finding his Perfect Software book last summer I'm gradually going through a swathe of his work. The current ones are:

• Quality Software Management vol II. Great insights into observations and interpretations and the pitfalls that go with them. I'm enjoying working through this book.
• The Gift of Time. Whilst this is not a Weinberg tome it's related to his body of work with reflections from people that have worked with him or been inspired by his work. Easy to dip into essays. I've nearly read through all and I like the description of the fieldstone method - with possible applications in my day job!
• Exploring Requirements: Quality Before Design. The test for attentiveness was illuminating. Ongoing! 

My reading is typically driven by my emergent learning and divergent thinking style. I've written about this before, here. Some of these I bought second hand from Alibris - a handy site for getting "hard-to-get" books.

And now - if you're still with me - a couple of challenges for you dear reader, yes you!

Quick Test!
Without looking - How many corners did I mention? Were you paying attention?


Have you got any great recommendations? Let me know!

Summer Reading: Busman's Holiday

#softwaretesting #books

A long holiday this year – with plenty of traveling and sometimes sporadic internet access, so I decided to take a few books with me.

I changed focus from my usual pick of biography, philosophy and science to something a little closer to the profession and ones I know will probably help in the next couple of months – so a little bit of homework and refresh at the same time.

The selection of books was some that I’ve read and dipped into many times and a couple of new reads - some classics and some potential classics.

Software Test Specific

Agile Testing: A Practical Guide For Testers and Agile Teams, Crispin & Gregory, Addison Wesley, 2009

First-time read.
A very good and comprehensive guide to hands-on agile testing for testers and team leaders. Can be used as a dip-into read (using the summary as a guide.) The use both non-Agile and Agile-specific observations and take in other well-established work (eg Marricks).

A good collection of research, observation and experience – follows one of the golden guidelines for good test literature (see below).

Lessons Learned in Software Testing, Kaner, Bach & Pettichord, Wiley, 2002

A re-read.

A potential classic. The series of lessons includes experiences that many readers will recognize – “yes, done that, seen that, been burnt by that etc, etc.” This is a “dip-into” book for me – almost a coffee-table book.

I don’t agree with all the observations – but that’s not the point of reading it – it’s there for the “on-tap” observations – having an observation that you don’t agree with can help clarify your own thoughts.

Perfect Software and other illusions about testing, Weinberg, Dorset House, 2008

First-time read.

A book that highlights the importance of information in testing – both information used in testing and how testers represent information. Some great thought-provoking comments around the typical everyday questions that testers face, e.g. “why don’t we test everything?”

Other

An Introduction to General Systems Thinking, Weinberg, Dorset House, 2001 (Silver Anniversary Ed.)

A popular recent classic. This is a great introduction to how science looks at different types of problems and ultimately gains from the system approach – giving an important different perspective to the problem/model. Read this for the first time and enjoyed it.

Critical Thinking: A Concise Guide, Bowell & Kemp, Routledge, 2005

A re-read.

This is a great book for outlining and defining structures of arguments and how to distinguish valid contributors from erroneous ones. Not a “dip-into” book, but worth the work. Going over certain chapters as a refresher and for the exercises.

How to Win Friends and Influence People, Carnegie, Vermilion, 1981

A classic from the 30’s that’s always worth browsing and re-browsing. Some of the ideas around influencing people and teams are valid today even when some of the references and quotations are 00’s years old. You’ll see similar principles being taught in modern-day child psychology books and docu-soaps on childcare and relationships…

I paraphrase some of the lessons and call them the “What’s in it for me?” principle. Before asking anyone (or a team) to do something different you must look at the proposition from their perspective and understand how that change/difference is going to benefit them (from their perspective, not yours!)

Secrets of a Buccaneer Scholar, Bach, Simon & Schuster, 2009

I managed to squeeze in a copy of the free download. Interesting read. I pick up the essence of a challenge here: “Dare to fail”.

The Mythical Man-Month: Essays on Software Engineering, Brooks, Addison Wesley, 1995

I had read parts of the original classic but bought the revised version for the additional essays – however, haven’t had the chance to open it so far.

I like this book for some of the great lessons that you can hear in SW engineering – “Throwing people at projects does not give the numerical pay-back” and “Avoid over-engineering”. I don’t find all the books thoughts relevant today, but there are still some thought-provoking observations. Looking forward to the newer essays when I get the chance.

A Golden Rule?

One of the golden rules about test literature relating to “handbooks or guides” is the way in which observations, lessons and ideas are presented.

Sometimes good literature actually re-states the obvious, whether it’s something you’ve heard or experienced before or something new that “clicks” with you and sometimes it’s a great collation of ideas and information – that is the research has been done, collated and summarized in a useful way.

If a handbook does this it has a greater chance of being more accessible. Think of this as the simplicity idea: “State the ideas clearly”, “State the relevant research and ideas around them”, “State the observations and arguments connected to the ideas” and “Summarize.”

When did you last take a busman’s holiday? What other books would you take and why?