ISO 29119 Questions: Part 1

Reason & Motivations?

There is an ISO software testing standard (ISO/IEC/IEEE 29119). Currently parts 1-3 are published, and parts 4-5 are in draft. I have read parts 1-3 and the draft of part 4 and am and have been forming several questions. Some questions might be regarded as problems, some as issues and others as a need for clarification.

I will use a series of 3 posts to document. 

Part 1: Looks at the reasoning and motivation for the standard - based on the information publicly available.

Part 2: Looks at some of the content.

Part 3: Looks at the output and usage of the standard and some implications.

Note, in this post I’ll only refer to publicly available versions and parts of the standard - the part I am considering here is the introduction - that is viewable in the preview part of the IEC webstore, see references.

Why?

Whenever someone comes to me with a new idea - I usually hear about all the reasons why it’s a great idea. What isn’t usually obvious is to say why the change is needed - or what problem is being solved.

29119

And so I had the same question in my head with 29119. I wanted to know the reasons and motivations behind it - especially ones that might support its usage or adoption.

According to 6.1.4 of the ISO Guide for Standards, ref [3], the introduction should state:

The introduction is a conditional preliminary element used, if required, to give specific information or commentary about the technical content of the document, and about the reasons prompting its preparation.

So, there should be reasons in the introduction of 29119. Looking at the introduction, there are three potential reasons given.

Reason 1?

The purpose of the ISO/IEC/IEEE 29119 series of software testing standards is to define an internationally-agreed to set of standards for software testing that can be used by any organization when performing any form of software testing [see refs 1 & 3]

Looking more closely at that:

• “internationally”: ISO implies this already - so this part is redundant
• “agreed”: the drafting of a standard requires consensus/agreement of 75% of the drafting members, so this is also redundant when the standard is published.

Note: One could ask questions about the representativeness of those drafting 29119, but that’s not a question I’m looking at here.

So “internationally-agreed” is redundant.

• “set of standards for software testing” -> seems to be a duplication of “software testing”, redundant.

• “can be used”: possible - this probably relates to how the standard can be tailored (more about this in part 2 of this post). 
• “by any organisation”: quite a bold claim, but linked with the previous point on tailoring this could be read as any company can tailor the standard.

So - “can be used by any organisation” -> could be generously interpreted as “can be tailored”. But then this is still problematical. 

Standards application and conformance are usually full compliance, partial (or tailored) compliance or non-compliance. Therefore, the fact that you can declare conformance or not goes with the territory of a standard. Therefore, this is redundant information - in terms of reasoning for a standard. 

An alternative reading could be that the standard is “useable” by any organisation. Again, this is redundant information - either the standard is adopted and conformance declared against it, or not. And why would effort be placed on a non-useable standard? Therefore, this is redundant information.

• “when performing any type of software testing”: again another bold claim. But part 1 of the standard “defines” the types of testing it is talking about.. The “performing” part is also implied as why else would you use the standard if not for software testing? So these parts are also redundant - by implication if you’re following the standard you’re following the definitions of the standard. (more about the definitions in part 2 of this post).

So, the paragraph becomes:

The purpose of the ISO/IEC/IEEE 29119 series of software testing standards is to define an internationally-agreed to set of standards for software testing that can be used by any organization when performing any form of software testing

Correcting the grammar for the parts that are stripped out would look like:

The purpose of the ISO/IEC/IEEE 29119 series of software testing standards is to define [a] … set of standards

Reason 1: Verdict?

So, when you strip out the redundant parts of the statement the purpose of the standard is to create a standard. That, in grammar, is called a tautology -> redundant information. That’s not much of a purpose - it still does not tell me why the it was produced.

Reason 2?

“This series of international standards can support testing in many different contexts.“ [see ref 1]

• “can” - I think this claim is a moot point - I suspect the reason is that, “if you claim conformance then it’s supporting testing” - so it’s not actually a reason /for/ the standard. It’s like saying, “if you follow the standard then you are standard-compliant”.

• “support testing” - well as part 1 is attempting to define testing and part 2 is attempting to define a process then in a way it could be claimed to support testing. On the other hand, it could be read that if you perform testing in accordance with parts 1-3 then the standard is supporting your testing - actually this is wrong, because it has defined what is “your testing” and potentially “what is excluded” - in this sense it’s not supporting at all. 

Reason 2: Verdict

My reading of this sentence is, “if you follow the standard then the standard will support your testing”. Unfortunately, this is a circular argument, ref [4], i.e. not really a supporting reason for the standard at all.

Reason 3?

“Together, this series of international standards aims to provide stakeholders with the ability to manage and perform software testing in any organization.”

• “stakeholders” - this is ambiguous who is meant. I’ll be generous and assume the user of the standard is meant here.

• “ability to manage … software testing in any organisation”
• “ability to … perform software testing in any organisation”

Interesting. This could be interpreted as anyone - absolutely anyone - by following this standard, can perform (and/or manage) software testing in any organisation. In fact, I’m not sure how to interpret it in a different way - there is no guidance (clarification) in the text.

Ok - I have seen (quite a few) people in my time that (i) can’t manage software testing and (ii) are pretty mediocre at software testing. In my judgement I have a hard time understanding how this standard would give the ability for “anyone” to become “testing managers” or “good testers” - except by creating a lowest common denominator. 

I can almost hear the explanation, “Find the least able person and we’ll calibrate to that person.” Really?

I have met and worked with - and I’m sure others will have similar experience - people (managers and testers) in various companies that are “going through the motions” - they tend to work /within/ a box (set of definitions or practices) and are not the ones who think or challenge assumptions (i.e. the ones who “think on their feet”). These are the people that spend more effort defining boundaries than communicating - they are plodders. If the aim of the standard is to produce a set of test managers or testers that either (a) think inside the box, (b) don’t think about or question what they’re doing, (c) plod along; then it is not something that can be associated with good (or efficient) work.

A Possible Reason 3 Verdict

The statement is telling - and probably says more than it really should.

Come on! If the aim of 29119 is to set the bar so low that anyone can look as though they are performing good work - BECAUSE they appear to be following 29119 - then that Mr Regulator, Mr Customer, Mr Potential-Stakeholder, I repeat, that is a “bad smell” - it’s a sign the company claiming conformance hasn’t got their eye on the ball! In a sense - you’d want to be extremely careful of anyone claiming conformance…

Introduction verdict?

According to 6.1.4, ref [2], I didn’t see any reason for the standard.

The skills I used to analyse the text were reasoning, logical and critical thinking - basic skills in any good testers toolkit - it comes into play (in my experience) when discussing requirements, understanding needs from stakeholders and reacting to and understanding conflicting needs in a project situation. Or even trying to understand standards. Actually, it’s difficult to do well and takes practice and concentration.

I had the impression that a number of testing experts were involved in the drafting and review of the standard. I think the resultant communication in the introduction has room for improvement - and doesn’t bode well for the rest of the documents. I’ll leave it to the reader to judge if the people involved in drafting the introduction did a good job of explaining the reasons and motivation for 29119.

Other Sources

Looking elsewhere for reasoning for the standard. Ok, let’s try some other sources.

Web #1

If I look at the “softwartestingstandards” web page, ref [5], I see this:

“By implementing these standards, you will be adopting the only internationally-recognised and agreed standards for software testing, which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world.”

Ok, the part about “adopting the only internationally-recognised and agreed standards for software testing” is covered above - it’s the same logic - and reduces to a redundant phrase (says nothing in terms of motivation).

The next part is interesting though, “which will provide your organisation with a high-quality approach to testing that can be communicated throughout the world”.

I’m interested that “high-quality approach to testing” is used - this is not stated in 29119, either as a motivation for using the standard or as a result of using the standard. As far as I can see there is no evidence (whether as case-study, or something else) to support this claim. Therefore, it is rhetoric - an attempt to influence without any evidence (proof).

Web #2

A Presentation given at the BCS, page 8 ref [6], states the motivation for 29119 as:

• Demand for existing 'standards’

• Conflicts in current definitions and processes

• Gaps in the current standards provision

• A Baseline for the Testing Discipline

• Current industry practice is lacking

• Buyers unclear on what is 'good test practice'

“Demand for existing 'standards’” - it’s not stated (or referenced) in the presentation where or what this demand is, therefore it’s an unsupported claim. Rhetoric.

“Conflicts in current definitions and processes” - 29119 is replacing some standard, so there is scope to believe that this replacement is reducing conflict between definitions and processes. However, the case for why this needs to happen is not made - it’s not demonstrated (in this presentation or elsewhere) what this conflict looks like or what problems it causes. Therefore, this seems to be a pretty weak argument. In terms of support in the presentation this claim is unsupported. Therefore in the scope of the presentation it is rhetoric.

“Gaps in the current standards provision” - it’s not stated (or referenced) in the presentation where or what these gaps are, or if or why they are relevant to being a motivation for a new standard. Rhetoric.

“A Baseline for the Testing Discipline” - it’s not stated (or referenced) in the presentation where or what this need is, therefore it’s an unsupported claim. Rhetoric.

“Current industry practice is lacking” - I could probably agree to this (but maybe for completely different reasons!) Without more information it’s not clear what this means - and isn’t stated (or referenced) in the presentation. Therefore, the claim is unclear and appears to lack any supporting evidence. Rhetoric.

“Buyers unclear on what is 'good test practice'” - I like the idea of presenting and distinguishing good test practices. However, the argument about buyers is not supported in the presentation (or referenced) with evidence. Therefore it’s an unsupported claim. Rhetoric.

Original Proposal for the Standard & ISO Directive #1

According to Annex C of ISO Directive 1, ref [7], a proposal for a new standard just have a justification. Specifically:

“C.3.2 The documentation justifying new work in ISO and IEC shall make a substantial case for the market relevance of the proposal.C.3.3 The documentation justifying new work in ISO and IEC shall provide solid information as a foundation for informed ISO or IEC national body voting.”

Now to documentation proposing the work for the new standard, ref [8], produced in 2007:

The market requirement section states:

“The national standards on which the new international standard is to be based are widely used both as the basis for commercial contracts and international qualification schemes. However, their national origin often results in them being ignored by potential users in some countries. At present there are a number of gaps (and overlaps) in the coverage they provide. A coherent set of international standards for the complete life cycle is required.”

The first two sentences state that some national standards are used, but not widely. The third sentence states there are gaps/overlaps between existing standards. 

The forth sentence is interesting: “A coherent set of international standards for the complete life cycle is required.” But where is the case made according to C.3.2 & C.3.3? According to point C.3.3 there is no evidence to support this point.

Ok, no help there.

In the “purpose and justification” section it does state:

“The purpose is to produce an integrated set of international standards to cover the software testingprocess throughout the development and maintenance of a software product or system.”

and

“In overall terms, the purpose of the project is to unify and integrate the currently fragmented corpus of normative literature regarding testing that is currently offered by three distinct standards-makers: BSI, IEEE, and ISO/IEC JTC 1/SC 7. The result of the project will be a consistent, unified treatment adopted by all three organizations.”

These two purposes seem complimentary, but there are problems. 

In the first says that “the software testing process” will be covered - but this seems undefined. The part 1 of 29119 outlines concepts and definitions, doesn’t it? Well there’s a problem there - that I’ll cover in part 2 of this post. But to summarise here - no parts 1-3 do not define “the software testing process” - part 1 is informative, meaning it has examples and not definitions - there is nothing to stop someone else creating their own definition. Part 2 has problems with conformance meaning that it can’t (and doesn’t) define THE software testing process. More details in part 2 of this post.

So this purpose might be real, but it’s also unrealistic. Unjustified.

The second purpose - to integrate information from different sources (BSI, IEEE & ISO) might seem reasonable - but this is more of a paper exercise than anything to do with standardising. There is no justification why this should be done or what problem is caused to the users of those individual sources. So, this is an unjustified purpose.

Summary

I wanted to understand the motivations and justification behind 29119. 

I looked in the introduction of 29119-1, where it is supposed to be stated according to ISO’s directives. I didn’t find anything that didn’t reduce to tautology and rhetoric. Now, I don’t know personally any of the members of the workgroup that produced 29119. But they are supposed to be experts in their field, and yet they produced an introduction to describe the motivation of 29119 that does not stand up to scrutiny. 

Did they all have an off-day - that lasted 6 years? Groupthink? Or what was it?

I looked elsewhere - presentations given to the BCS and the softwaretestingstandards site. There were a lot of claims without supporting evidence or references. This is also known as rhetoric.

Then I searched in the original proposal for the workgroup that would produce the standard. There, I found claims without evidence, some were unrealistic and some were almost a wish list.

After reading this I wondered about the people that reviewed the proposal. I’m sure there’s a bunch of intelligent people that looked at this and yet they seemed to have missed a lot. Was it some form of groupthink, I don’t know.

Well, I've been looking for reasoning and motivations for 29119 that will stand up to scrutiny - so far without success. In the next part I'll look at some of the content of 29119 in more detail.

References

[1] IEC/ISO/IEEE 29119-1 preview http://webstore.iec.ch/preview/info_isoiecieee29119-1{ed1.0}en.pdf

[2] ISO/IEC Directives Part 2 ISO/IEC 

Directives Part 2

[3] The Art of Software Testing Standards http://www.softwaretestpro.com/Item/5895/The-Art-of-Software-Testing-Standards

[4] Wikipedia: Circular Reasoning http://en.wikipedia.org/wiki/Circular_reasoning

[5] Webpage: ISO/IEC/IEEE 29119 Software Testing Standard http://www.softwaretestingstandard.org/index.php

[6] BCS - British Computer Society, page 8: http://www.bcs.org/upload/pdf/sreid-120913.pdf

[7] ISO/IEC Directives, Part 1 http://www.iso.org/sites/directives/directives.html

[8] ISO/IEC JTC1/SC7 3701

On Thinking about Heuristic Discovery

How do you spot new heuristics?

How do you identify new heuristics?

These were questions that were “touched on” during parts of SWET6* (a small part in the open season of James Bach’s discussion and during a car journey from SWET6).

“Pause & Reflect” / “System 2 Re-Insertion”

During the open season of James Bach’s topic on documenting heuristics from a test activity I remarked that I’d spotted an heuristic that he didn’t appear to notice. It was the activity of pause and reflection: putting down the work for a while, revisiting, correcting and re-working. This can cycle through after bouncing ideas around with colleagues or getting their feedback, followed by then further ‘pause and reflection’ cycles until the result was deemed good enough. 

Reflection

Due to the nature of how the activity and report had been made it incurred a series of pauses (interruptions or breaks) which then (I assert) meant that some of the parameters of the context (or frame through which you look at the work) has to be remembered, reviewed or picked-up again - this can mean that the frame gets slightly altered, i.e. “you look at it with a fresh pair of eyes” and, hey presto, see something different or new.

I do not claim to be the first to spot the power of this activity - but I think of it as re-activating the System 2 mode of thought (according to Kahneman, ref [1] this is more deliberate and takes conscious effort to use). The action of putting the work (report) aside for a time and then revisiting means that some of the context is forgotten and so has to be remembered when the work is picked up again. This is the re-analysis in the system 2 mode of thought.

Noticing New Heuristics

The pause and reflection about the observations is very important. It gives the basis for new pattern recognition. Either you notice something different that you don’t recognise or that is a little different. Typically this might be a procedure that you followed, for example:

• I tend to find problems of type X when I do A, B & C, 
• I tend to find race condition problems when I alter (shorten and lengthen) the timing between certain test steps.
• I tend to notice new/different patterns when I pause and reflect on a set of actions and compare with previous times I did some similar action.

Blink Comparator / Trawl and Compare

The activity of reflection and noticing new patterns is something I think of as a blink comparator test. A blink comparator test is something astronomers used to use to find new stars. A reference pattern is used to compare with a current observation. It’s like trawling through previous experience (observations) and comparing with the latest experience. In the example above it might be:

• When I find problems of type X (in the past) what was the same/similar to the current actions (A, B & C)?
• When I have found race conditions in the past were there any timing differences in the actions I made in the test steps? (When I change the timing between steps - for the same types of steps - I find race conditions more often.)
• When I notice new patterns is there some key step that is common? (Pause and reflect,)

Visualization

Sometime in January I tried to sketch this connection between Pause & Reflect and how I noticed new heuristics. This is my latest version that I think reflects the process I tend to follow:

A larger version of this model can be viewed here.

"Pause & Reflect Heuristic"

The model is best read entering the “Pause & Reflect Heuristic” box. 

Here observations of actions (or reviewing a test report) is done - with a pause & reflection in-between - so the two “Solution / Result” clouds may be different. The “Evaluate” cloud is where a difference is noticed. In some cases this feeds back as re-work, in other cases it causes a question which then feeds into the “Proto-pattern” cloud. 

Blink Comparator Test / Trawl and Compare

This is the step where a search of the difference (comparator item) is made. The search might be a re-analysis of similar experiences to understand what caused the different result. The result / assessment might be that it was a random action, or something outside my control, that caused the difference**. At other times it might be, “step X was made when the system was in state Y” - that might be enough to give me a new useful rule of thumb (heuristic).

The next step would be to search if it was something in use elsewhere or known already. If new, then classify (name and describe) and publish (talk about or discuss it).

Question, Test, Evaluate

But the work doesn’t stop there. When using this heuristic new observations about it’s use should be gathered. Does the heuristic work as before? Is there a new (maybe more subtle) pattern or action that makes it useful. This might result in a new heuristic, a refined heuristic or a restriction to fewer applications. This is the “Question, Test, Evaluate” yellow box. It is analogous to the scientific method where the heuristic is the object (hypothesis) tested.

Testing the model

I’m currently collecting new observations of noticing new patterns and trying to construct ways I can test this model.

In the meantime, comments and suggestions are very welcome.

References

[1] Thinking, Fast and Slow [Kahneman; 2012; Farrar]

*SWET6 was at Hönö Hotell, Öckerö, Sweden, 19-20 October 2013, Attendees: Martin Jansson, Steve Öberg, Saam Koroorian, Mikael Jönsson, Anders Bjelkfelt, Marcus Möllenborg, Klaus Nohlås, Simon Morley, Henrik Emilsson, James Bach

**Note, a “different result” might be, when I tackled the problem now I got a different result than previously. What made the difference? This can applies to systems of people interaction too.

The Linear Filter Trap

Or: Illustrating Systems Thinking with Proximate and Distal Analysis

I was having a discussion the other day where we were looking at some problems and discussing whether it was sufficient to treat symptoms... I remarked that by treating symptoms, rather than root causes [notes n1], and not allowing time to find the real problems then we would, in many cases, be fooling ourselves [notes n2].

Why? (I was asked)
People like to solve problems that they can see - which means we have a tendency to "fix" the problem we see in front of us. This occurs more often if the problem appears to have a "straightforward" fix -> I think of this as a form of cognitive ease in action. Digging for root causes is a challenging activity, and we sometimes want to believe that the cause we identify is good enough to fix. For another example of cognitive ease, with best practices, see ref [2].

An illustration of this - that I have seen in one form or another - is that we settle for the first solution without understanding (or trying to) the root cause. There is no guarantee that fixing a symptom will make the problem better. Many times, the problem improves for a while, but then re-occurs in another form. Now there is a "new" problem to solve, which usually has the same (or similar) root cause, so from the system perspective it's ineffective [notes n3].

Ineffective, because many problems in processes and organisations are often non-linear, but we often try to solve "linear" problems, and...


Linear Filter
I expanded, that I think of this from a systems thinking perspective as applying a "linear filter" to a "non-linear system".

What? (I was asked again) Linear vs Non-Linear?
Non-linear -> multiple interactions affect the output vs linear -> the output is directly proportional to the input. So the application here is that there are usually multiple causes for a problem -> when I perform an assessment after a root cause analysis (RCA) activity I usually take them root cause by root cause, in the order that we think will have the biggest impact on improvement.

Ok, time for a....

A Real-life example
A fault (bug) report was written by a customer -> initial RCA shows that some "test" was not executed that would (in theory) have caught the problem. This is a symptom and a "linear" view of the problem. The "linear filter trap" is to then consider this as the root (or most important) problem. [notes n6]

Digging deeper shows that the team had it's priorities changed during execution, to make an early drop (resulting in some negative and alternative use-cases being delivered later). This, in itself, is not a problem but the communication that was associated with the "early drop" didn't reflect the change.

In this case, some of the underlying problems were a set of communication issues:

• Partly in the team connected with their story [notes n4] (especially their testing silent evidence [notes n5]), 
• Partly connected with the stakeholder that changed priorities and may have had a duty to follow the change of expectations through the delivery chain and what that might mean at those different stages, and 
• Communication with the customer to ensure that they are aware of the staged delivery, and what means to them.

Another example of a root cause analysis can be found in ref [3].

And finally
Tackling and fixing symptoms is a very natural activity - very human. But it is not always enough. Sometimes it is enough - it depends, of course, on the scale of the problem and the cost of investigating the underlying problems and tackling those. Sometimes, the underlying problems cannot be fixed and it is sufficient with easing the symptoms.

But I believe, as good testers, it is important to understand the difference between symptoms and root causes, especially where it affects either the testing we do or affects the perception of the testing we do. This is important where a perception of "testing or testers missed something"... So,

Be aware of the linear filter trap!

Notes
[n1] In philosophy and sociology, root causes and symptoms are usually referred to as distal and proximate causes, see ref [1] for more background.

[n2] Slightly naughty of me, playing on the fact that most people don't like to think that they are fooling themselves, but that's a different story...

[n3] The times when it might be effective are when we (some stakeholder) is prepared to take the cost of fixing some problem now. A problem here is that stakeholders that are project-driven have, by the nature of the task, a propensity to see only as far as the end of the project. A product owner may have a different perspective - bear this is mind when someone is deciding whether it's a project problem or a product problem - or even a line organisation problem.

[n4] Story here means the story about the product and the story about the testing of the product.

[n5] Here, testing silent evidence refers to the elements not tested and thus not reported - their significance is assumed to be not important. For further background see ref [4].

-->edit-->
[n6] I should add that the problem with the trap in this example is that I have seen this in the past trigger one of two responses: (1) A perception that the testers are at fault, which becomes a myth/rumour with a life of its own ; (2) A knee-jerk reaction to implement some extra oversight of the test displicine or team as a whole -> in the worst case it becomes a desire to introduce some additional "quality gate". This is a good example of when reacting to the perceived symptom is both ineffective and counter-productive for the organisation.

References
[1] Wikipedia: Proximate and ultimate causation
[2] The Tester's Headache: Best Practices - Smoke and Mirrors or Cognitive Fluency?
[3] The Tester's Headache: Problem Analysis - Mind Maps and Thinking
[4] The Tester's Headache: Mind The Information Gap

Problem Analysis - Mind Maps & Thinking

Credit to Peter Haworth-Langford, Christin Wiedemann and Oscar Cosmo, with whom I was chatting back in December...

The topic of mind maps was discussed (in various usages) and I realised that maybe I had a usage that I need to write about and digest...

I frequently get asked to do a root cause analysis type of activity. Usually it's connected with faults reported against a product, but other times it might be to analyse a situation or process. In all cases I usually have a main question in my head, "what is the real problem?"

Below is an example of using these approaches for a root cause analysis of a fault. The analysis is used as a learning opportunity - to see if there is anything that needs to be changed or improved, including project and team structures and aspects of how they work.


Example
A typical root cause analysis (RCA) in my shop might look like the following. This is one example and there are many other variants.

The example "presumes" there was some missing testing (which might reasonably have been expected to happen), but it could just as easily be missing design analysis, design coding error or any of a range of other issues.

I have two main techniques that I use to help reveal information - and so help me on my way to "the real problem". Note, I say this as though there is only one, but I'm well aware there are usually many different competing aspects.

The first is the 5-whys and the second is what I think of as the "are your lights on?" approach and I use them in distinct ways:

• 5-whys - extracting raw data
• Lights-on - attaching meaning to the analysis

The 5-Whys
This is the forensic search for raw data - collecting the different aspects of the problem - what the problem was, its solution, aspects related to team analysis, system design, aspects related to test design and execution, aspects of team dynamics, changes in timing and priorities that might any of these parts.

Actually, I don't restrict the analysis to "5" - it can be 2, 3, or 8 - as many as are needed to get a "good enough" picture or level of granularity (part of this is an understanding of how deep we need to dig).  Extending the above example, it might become:

Lights-On
This approach is heavily based on the Gause & Weinberg book, "Are your lights on?", which I recommended highly. This is an approach that helps answer:

• What is the problem?
• To whom is it a problem?
• Whose problem is it?
• Does the problem need solving?

Usually, by working through these aspects of the problem it is possible to determine the extent of the issue, if it is an issue that should be addressed, if this is something that can be done now and by who.

Sometimes the problem appears due to a change in project priority or timing, something doesn't get completed and, crucially, the customer is not aware that it shouldn't use feature X in this first drop, or that it has certain limitations.

Communication (or the absence of it) is, occasionally, a root cause in itself!


A Third Way
I said I had two main ways, well recently I've realised that I'm using a third way - I say recently, as I've really only attached (or discovered) the terminology for it: Framing.


Framing
Framing is heavily based around the work of Tversky & Kahneman, with some application by Russo.

This is the filter that is used to look at the problem. In the above example, I might look at the issue from the project perspective: what priorities existed on the project (stakeholder or customer), were there aspects outside the control or remit of the project, did any of the project aspects change and with what timing?

There might be team frames: how did the internal and external communication to and from the team work; was there a project change that didn't filter into the team; information availability, assumptions and other limitations.

Finally, there might be aspects that work on project, team or individual level: risk averse or risk taking attitudes; attitudes of uncertainty, lack of support or overconfidence; or a combination of many factors.

So, applying some framing aspects to this example might give:

The framing of the problem - and using multiple frames - helps to put the issues into perspective, the situational context - what did it mean at this particular point in time, in this particular project, in this particular team set-up, for this particular customer, etc, etc.


Full Circle
The framing aspect now fits quite well into my RCA approach, or any problem analysis approach:

• Raw data is gathered: 5-whys is one approach
• Situational context: Framing, ideally using a number of frames.
• Meaning and decision: Lights-on

Conclusion
Mind maps help in a number of aspects here: visualisation and recording of thinking and exploration.

However, the big aspect is the thinking - using a range of tools to explore the problem, not limiting analysis to the problem artifact (a team didn't do xyz...) but adding in the situational aspects and weighing up the problem with the surrounding information to get to a better understanding of it (framing). Then the information can be used to help understand if it really is a problem that needs solving and by who (Lights-On).


References

1. Gause & Weinberg, "Are Your Lights On?: How to Figure Out What the Problem Really Is" (Dorset House, 1990)
2. Kahneman & Tversky, "The Framing of Decisions and the Psychology of Choice" (Science v211, 1981)
3. Russo, "Decision Traps: The Ten Barriers to Decision-Making and How to Overcome Them" (Fireside, 1990)
4. 5 Whys, (wikipedia)