MandaViews: modeling of testing techniques
We’ve all attended training, presentations or seminars where we saw various diagrams or graphs illustrating the ideas and concepts presented. Do we really remember them and did they help us to understand and above all to learn?
From a pedagogical point of view, the question is essential. Everybody knows the adage “a picture is worth a thousand words”, but when patterns are too many, do they remain effective?
Preamble to the first part: The first part of the article is the translation and adaptation by the authors of the original article published in December 2011 in Testing Experience # 16, The Future of Testing, p112-115 ” MandaViews : modelising test techniques “by Philippe Roux-Salembien, Damien Mathieu, Franck Launay and Gregory Heitz. Testing experience is an English-language journal dedicated to tests, which broadcasts from Germany across Europe and beyond. In the original article of TE, we proposed the main testing techniques, as they were mainly in the syllabus at the end of 2011, or possibly in the books we quoted. We introduced a modeling of White Box and Black Box testing techniques. But given the changes in CFTL and ISTQB syllabi in the meantime, we decided to present only the Black Boxes here.
Bearing this in mind, we tried to develop a new concept: MandaViews (MDVs). The MDVs offers a family of models to illustrate the testing techniques. The objectives are, first, to provide ideograms to facilitate understanding and long-term remembering of different testing techniques (black box, based on experience and defects, and white boxes) and then to offer a project reporting tool for managers.
Why propose a modeling of test techniques?
A teaching aid
Testers wishing to improve their knowledge not only must understand and know how to apply the different testing techniques but they also must be able to sort them into family or rank them according to their aim. This can sometimes be a tedious job of analysis, compilation … A synthetic illustration thus facilitates learning but also prioritization of techniques.
A documentary approach
Furthermore, an overview of techniques actually used in each step or level of a test project could provide relevant feedback to stakeholders on the panel of in-use testing techniques.
Mandalas: from an ancient tradition to an applied pedagogy
In Sanskrit, mandala means “circle”. These concentric diagrams are often used as meditation tools. For decades these antiquarian charts have been adapted and used by teachers to illustrate concepts or connected notions to make them more understandable and easier to remember. I attended a lecture given by Armelle Geninet teacher in my daughter’s high school. This lecture was entitled “How can we increase our effectiveness in helping our children with their homework?”. Mrs Geninet explained among other concepts and ideas, how Antoine de la Garanderie, a philosopher and pedagogue, had developed the concept of “mental management” and its 5 “mental gestures” to describe the learning process: Attention, Memorizing, Understanding, Reflection and Imagination.
This research identified the need for the learner to “spatialize knowledge”. The above-described Mandalas are used to contribute to the assimilation of knowledge in areas as diverse as mathematics or English. Mandalas’ specificity, in such educational context, compared to other graphic representations lies on the use of symmetries, links and graphic codes between the different presented items.
The combination of these three points aims at improving the spatialization, comprehension and memorization of the concept visualized by the mandala. During the lecture, I had the idea of applying these educational-oriented mandalas to my own domain: software testing, specifically to represent the testing techniques. The idea was supported by the wide variety of test techniques described in the syllabi of CFTL and ISTQB. So I designed a first model for the black boxes (BN) and techniques based on experience and called it: MandaViews. Some time later, during a meeting with the consultants and trainers of Factory Consulting (a company specialized in coaching and training in software testing methods and tools), I talked through this concept with them. They agreed about the originality of the idea and discussed the potential of MDVs in test activities. So we decided to continue exploring other uses of MDVs we could share with the community of testers. Thus arose the idea of this article.
Like the mandalas, the MDVs relies on pie charts. The circles are divided into sections, each containing a pictogram illustrating the test technique it represents. The first principle of understanding MDVs is based on a clear association between the icon and the principle of the technique it depicts.
Example: boundary value analysis
The long term remembering process of the test technique underlying principle is thus facilitated. The clustering and the choice of the position of each technique according to its category and affinities also contributes to improve understanding of their relationships and their potential use in everyday life.
MandaViews: The model for effective teaching.
The three objectives of MDV model rest on the process shown in Fig. 1.
First, spatialization allows us to have a global understanding of all the techniques from the same class thanks to clustering of related techniques.
Secondly, visualization is improved due to a theme of colors and ideograms facilitating comprehension and memorization of each technique principles and its relationships.
Third, the homogeneity of the resulting models facilitates long-term memorization of information while considering the model.
Figure 1: Key Principles of MDV Model, inspired by the research of Anthony de la Garanderie
Authors of the part stemming from Testing Experience: Philippe Roux-Salembien (ACIAL), Damien Mathieu (Factory Consulting), Franck Launay (Factory Consulting), Grégory Heitz (Sopra Steria).
The MandaViews Model for modeling techniques based on the and techniques based on experience
Figure 2: First MDV: example for specifications-based techniques and experience-based techniques
NB : les schémas ne sont pas présents et j’ai un doute sur la description en trois points ci-dessous
Why combine black box techniques and techniques based on experience ?
Black box techniques and techniques based on experience and defects are probably the most commonly used testing techniques so it seemed logical to group them. Furthermore, according to ISTQB and CFTL and they cover a dozen of techniques, which is ideal for the construction and the symmetry of the first MDV.
The circular shape allows quick viewing of exposed testing techniques, clustering of specific techniques and insertion of comments.
Moreover, the division into three concentric zones facilitates knowledge spatialization and indicates:
1. The name of the testing technique outside;
2. The pictograms designed to illustrate the principle of the techniques in the circle in the middle;
3. The categories of techniques in the central zone.
The pictograms associated with testing techniques are easy to insert in MDVs and help to understand and memorize the principle of each technique. The first two icons of techniques based on the specifications illustrate the role of equivalence partitions and their link with boundary value analysis. Then the pictogram of the decision table technique offers a sample of table easier to identify than the cause-effect graph. The state transition test pictogram illustrates the starting point, three different states and their transitions.
For combinatorial techniques, the classification tree proposes an example of a tree then pairwise testing is exemplified by a sample from a table made with this technique. The pictogram of Use Case shows a simplified view of the usual UML diagrams. Regarding techniques based on experience, we have chosen to offer examples of categories in the test pictogram taxonomy (NB je n’ai pas compris la phrase même en français) consistent with the definition of CFTL and ISTQB. The error estimate which comes next is probably fairly easy to recognize, the test based on checklists too.
The exploratory testing shows a magnifying glass zooming in on the word “Charter” (test chart), recalling the use of the testers’ intuition around predetermined themes (or charters). Last, the pictogram of the attack-based testing represents with flashes of lightning attempts to force failures.
One can notice that the spatialization of the twelve techniques from both categories enables to group them visually, which helps both understanding and memorization.
– Firstly equivalence partition and boundary value testing are placed side-by-side since both are based on equivalence classes
– Secondly attack-based testing may use techniques based on experience and defects, which explains that their lastly position amongst these techniques.
The MDVs propose a new approach to the representation of test techniques. First, the MDV model provides a useful graphical means of spatializing, understanding and memorizing each testing technique thanks to the specific characteristics of mandalas. Second, the “MDV Project” (currently being defined) could provide a practical illustration encompassing all the techniques used in a project. This would provide a model to graph the identifiable part of test techniques in an ongoing project to help stakeholders making decisions. It would also help focusing on the issue of testing techniques and their relative effectiveness in order to constitute a database by company and / or customer to capitalize on the subject and to guide strategic choices of coverage.
The MDVs are still under development (eg for project samples or review). We would be happy to share our designs with you and to know your impressions. Draw meaningful and sufficiently intuitive pictograms is a real challenge for some techniques, therefore further suggestions will be welcome.
Authors of the part “summary tables”: Philippe Roux-Salembien (ACIAL), Zied Bouhalli (ACIAL), Bruno Drahi (ACIAL)
Note by Philippe Roux-Salembien: Special thanks to Ms. Geninet for inspiring me the concept of MDVs with mandalas and having kindly agreed that her work is referenced in this article. Also thank you to Thierry Charles for his help in the first sketches of the paper model I had designed, and once again thank to Eileen Basgallop for her kind assistance in correcting the first version of this article published in English in the magazine Testing Experience (ref. above). And of course, special thanks to Damien, Franck and Gregory for their support and significant assistance to develop, test and improve the concept of MDVs and this article.
I do thank my colleagues Zied and Bruno for their responsiveness, availability and enriching contribution during the drafting of the “synthetic tables” of this article. Many thanks also to Damien Mathieu for his very careful proofreading of these summary tables.