He discussed a little about the implicit meanings behind the 7 principles, but the slides were skipped through so I didn't see the details. However, by this point at least one question occurred to me.
My question in the open season went along the lines, "The principles were mentioned as a litmus test to gauge if someone was context driven. However, adopting a scientific approach (ie one of searching for information and not settling on an assumption), there may well be more than 7. How do we add number 8?"An underlying question was (for me):
- If the principles were written so (comparatively) quickly why haven't they been challenged from testers claiming to follow or adhere to them?
- Indeed, isn't this a good exercise for a tester - context-driven or otherwise?
Therefore, I thought I would take a look at them - as a (comparatively) quick exercise. James did mention that Markus Gärtner had extended the principles as an exercise previously. I remember seeing a discussion on the software-testing yahoo list some time ago and checked it out - and sure enough I'd discussed the possible extensions there, and thought it was a useful topic to revisit.
The 7 published principles, ref [1],
1. The value of any practice depends on its context.My Approach
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
In assessing the principles I used two approaches, (1) Critical/Logical consistency - do the propositions support conclusions or implications?; (2) Are the principles testable (falsifiable)?
Assessment
#1 [Unchanged] I tried to extend this one, but I think it is rhetorically good - succinct and a good start to the principles. This can not be tested but is inferred by induction.
#2 [Remove] I could think of removing/replacing this one. It is a derivation of #1. The essential content here is that "Practices evolve over time" - I thought of making this my #2, partly to remove the distraction of "best practices" (that would be a derivation of these principles), and partly to emphasize the evolution of practices. However, I finally settled on having no #2.
#3 [Rephrase] The principle is an assertion. For this one I want to emphasize that the project, the processes and people form a system and that they change over time. The importance here is to illustrate that the system is not static, it is complex because of people (and their emotions) and as such, working with such a system is not a trivial activity. Therefore I would re-phrase.
#4 [Remove] This is a derivation / implication of #3, an application of the principles.
#5 [Unchanged] Again, succinct and difficult to refute.
#6 [Unchanged] An assertion that where good software testing happens there is a high intellectual demand.
#7 [Remove] This is unnecessarily wordy and sticks out as too different from the previous principles. It also drifts into certainty and seems like an untestable principle, and so I would remove it.
Addition
#8 Here I want to tie the system mentioned in the rephrased #3 and good software testing.
Result
1. The value of any practice depends on its context.
2. -
3. Projects and people form part of the system that work together, where products and practices evolve over time
4. -
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. -
8. Good software testing encapsulates the system of project, people and practices that work on building the product.
So, I ended up with 5 principles. The application of these would then produce variants and clarifications. For instance, the best practice item is derived from #1. Understanding systems of people, emotions and time is derived from these and many, many more.
I stopped there, but if I spent more time could probably refine them somewhat. But they are good enough for this exercise. And as James suggested - if you started from scratch, you might well not end up with the original 7 principles.
So, how would your interpretation look?
Reference
[1] http://context-driven-testing.com/
I stopped there, but if I spent more time could probably refine them somewhat. But they are good enough for this exercise. And as James suggested - if you started from scratch, you might well not end up with the original 7 principles.
So, how would your interpretation look?
Reference
[1] http://context-driven-testing.com/
Very much like your approach to reduce it to the essential and reminds me of my good friend George Carlin, who did the same on a completely different subject: http://www.youtube.com/watch?v=YzEs2nj7iZM
ReplyDeleteThanks for the link, Ilari, I hadn't seen that. Very good!
DeleteI have blogged about the results from the tutorial at CAST 2011 with James here:
ReplyDeletehttp://www.shino.de/2011/08/11/cast-2011-context-driven-leadership/
We ended with these ten:
1. The value of any practice depends on its context and the problem at hand.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project’s context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn’t solved, the product doesn’t work.
6. Good software testing is a challenging intellectual process.
7. Through judgment and skill, exercised cooperatively throughout the entire project, are we able to effectively test our products in an efficient way.
8. A context-driven tester takes the responsibility to continuously nurture and expand his skills by learning new practices.
9. A context-driven testers teaches others about his practices and skills in the context he applied them.
10. We adapt our processes to match their context. If the process does not work in a context, we change the process.
Have you changed anything here, or just chosen different words?
ReplyDeleteI don't feel an urgency to your version. That's my major concern. Number 7, in the original version, is a strong statement and it ties in a few other principles. It's a conclusion and a warning. You've replaced it with something that means almost the same thing, but sounds like soggy bread to my ear.
I want to expand, not contract the first part:
1. The value of any practice depends upon the context, because practices exist for the purpose of solving problems, not gratifying the ego or covering up incompetence.
2. Anyone who thinks there is a best practice independent of context could not have arrived at that conclusion through consideration of evidence, since evidence belongs to context.
3. Anything called a "best practice" could more accurately be called a "locally or temporarily favored practice" or "the only known practice" or a "practice", or "pseudo-scientific bullshit."
4. "Best practice" is not an engineering concept. It is a marketing term used to inhibit innovation.
James
DeleteAs I said I applied two tests to analyze the 7 principles - so rephrase and tweaking was likely to be the outcome.
I wanted to emphasize that the development is made within a system of project, people and processes.
I wanted to introduce the concept of systems changing over time -> evolving
For me, the #8 seems less soggy than the #7 - it ties in the system and testing of the system - which is important. #7 seemed far less succinct to me.
As for the best practices - I'd use the principles to show how they contradict the idea of best practices. And there are many more marketing battles that can be fought with them - but I would want to demonstrate that as a field application guide rather than wrap them in the principles themselves.
Anyway, I took the talk at Lets Test as a challenge to examine the principles and I encourage others to do so - because that will undoubtedly clarify ones own thinking. Whether that results in new principles or not is not the key goal (for me), but the test of ones own assumptions and thoughts about them.
Thanks!
(1) I sometimes hear stories about how quickly the principles were written and about how quickly Lessons Learned itself was written. Sometimes, I hear that it all got done in only a few weeks. My email archive on the development of the book includes 1635 emails (and there are probably hundreds more that got filed in other folders) from 1997 through 2001. It is true that once we started typing the Principles, things went pretty quickly. but the ideas had seasoned for a long time before the typing. That doesn't make them RIGHT or FINAL, but there was a lot more work, over a lot longer time period, with a lot more planning, than is sometimes described.
ReplyDelete(2) I don't think we should be using litmus tests to decide whether someone is "context-driven" or not.
(3) I don't think these principles were developed as a litmus test. If we were going to create a litmus test, I would ask different questions (and make different observations) that addressed behavior and verifiable attitudes rather than relying on affirmation of a broad set of principles. I can think of several such questions, but see #2 above.
(4) I think these principles were a mechanism for three authors of a book to express their understanding of a set of ideas they had been working on for years. They were a simplifying summary of (what turned out to be) a temporary agreement in perspective of three people. They were inspirational (inspire other people to think our way) not legislative or religious. They are not dogma, or they should not be.
(5) I think there were some things that were so obvious to us that we didn't write them down (but in retrospect, I wish we had). For example, I believe that we believed (back then) that the willingness to learn from people we disagreed with was central. That dogmatism is dangerous. That there were many people who disagreed with us who were smart, honest, and had valid experiences. That any testing or process recommendation we would make is subject to strong counter-examples. That (as with most or all scientific knowledge) everything we (think we) know today will turn out to be mistaken.
(6) Rather than polishing the words in a short list that is necessarily oversimplified, I think it makes more sense to inquire into the ideas underlying context-driven testing. We don't need an orthodoxy. We need what evolves out of multiple advocacies of multiple ideas.
- Cem
Cem,
DeleteThere was no intention of implying the groundwork behind the principles, as written, was short or not the result of a deal of preparatory work - even though I alluded to the short time to actually make them explicit or my reference to a quick assessment of the principles.
If anything, my quick look was intended to show other testers - whether they consider themselves aligned to or following the context-driven principles - that they may find some new angle to look at the principles, or new interpretations or alternative ways of phrasing them - if they take "some" time.
My ambition to do this "quickly" was a way of showing that it is possible, accessible and, in my opinion, a useful exercise for testers. The actual result is not the most interesting part - for me - but the process of analysis. In this case I used 2 approaches, but could have easily used more.
It would be an interesting exercise to apply Gause & Weinberg's "Mary had a little lamb" heuristic to the principles - to help gauge ambiguity or alternative meanings.
Another exercise might be to challenge a tester to describe the principles (or their interpretation) in different words or format - this is a process to help illuminate the meaning that someone attaches to them.
The intention behind this examination was not simplification but an exercise in finding potential issues and plausible alternatives.
I agree that the application and implication behind these (or other) principles are the interesting area for real/further research - and would make a very interesting challenge for any tester. Indeed I believe James alluded to the implications of the principles as being an interesting area during his keynote.
As a skeptic I don't believe in orthodoxy, and that everything is open to questions and reassessment - hence the exercise.
Many thanks for the insights and thought-provoking comments!
Thanks for the post, Simon!
ReplyDeleteI understand the desire and need to take a closer look on the principles. There is a lot to discuss around them. However, what I find more important is to have a mission, vision and values for CDT; in a collection, not spread here and there. What do you think about this kind of approach?
Best regards,
Jari
There are many principles that guide Software Testing Principle. Before applying methods to design effective test cases, a software engineer must understand the basic principles that guide software testing. The following are the main principles for testing
ReplyDelete