After a four year hiatus, we have resumed recording the Watir Podcast. It’s not at the same site. You can now find it on SoundCloud. Please listen and provide feedback.
Chapter 3: Control Flow Testing
This chapter introduces a basic graph, control flow. I found the tax documents for this exercise.
I read this in two parts. The first part, after the standard definitions section, focused on the relationships of a control-flow model, how they vary, and how to draw them. I liked the use of the truth table to show the relationships in a compound predicate.By the end of this section, I knew (remembered might be a better term) how to use the tactics of this technique. I could differentiate when the nodes would be serial instructions and when they would be selection as an if or case statement..
Dr. Beizer presented a step-by-step instruction set for building and using the control-flow model for testing. The model development seemed to be as if I were modeling a script or coded program. but showed what happened in the methods as well. The steps within the tax instructions gave that feel. The space of each instruction gave us a memory register to store the result. Each instruction did one thing… well, mostly. While I didn’t like the idea of reviewing 18 year old tax instructions, it worked well with this lesson.
The coverage, which is beginning to be developed in the previous chapter, starts to become more relevant. Where is the risk? In some places, the decisions seem somewhat arbitrary; to the whim of the author. Most of the coverage decisions are direct to the risks, and causes of the risks. Even within those, some of the reasons appear to be “old” ways of thinking.
In summary, this chapter gives the tester something they can use right now. It is the first finished feature of model-based testing. It is releasable, usable, and helpful.
Chapter 2: Graphs and Relations
This chapter introduces graphs as the central tool of testing.
Just running through the definitions got me pumped for testing. Every relationship Dr. Beizer described brought about examples in my head from software that I commonly test in my job. Then I saw the Question/Answer of the book. What do you do when you see a graph? Cover it! In 20 years, that has never left me or my test methodology.
I am not an expert on graphing so I did not see anything blatantly left out. I noticed in the undirected links section of definitions that the author didn’t think symmetric relations are not common in testing – I thought that modern software models include load balanced services, as well as multiple threads, which may have those relationships. I have tested those relationships many times.
Regarding the graph types used in testing, I took interest to the those. I felt that the transaction flow model he described doesn’t get enough attention. The finite-state model of menus… modern menus often change based on the current task or context – in the author’s word processing example, one may see a different menu set when editing a table.
While I enjoyed the discussion of transitive/intransitive relationships, I really liked the end. To paraphrase, If specification or programmer is confused about transitivity, then you have a fertile ground for testing.
The examples of the relationship types are helpful. Except the part about equivalence classes. I had to look at 3 web definitions and two YouTube videos before I felt comfortable. Then I reviewed a single partition example and said “oh yeah, I know that.”
The concept of the matrix of relationships between nodes was completely boring, but so much more efficient. If I were modeling with intention to automate coverage of a model, I would use the matrices in some form. But I get why Dr. Beizer says “…the format is cumbersome and error-prone for large matrices.”
The generic testing principles outlined have covered a great deal of my testing lifetime. It’s also the ToC for the rest of this chapter. Each instruction expands to a cookbook for creating a testing tool.
- Define the graph
- Define the relation
- Node Cover
- Link Cover
- Test All Weights
- Design Loop Tests
The instructions are simple, and mostly straight-forward. I didn’t particularly like calling properties as ‘weights’ because I think of weight as a particular type of property. Like the weight might show the likelihood one branch of the graph is used compared to an alternate branch. The author also describes the purpose & significance of each step.
I am not sure the ‘test all weights’ is necessary as a separate step. The properties can be tested within the node cover & link cover. They should be tested there.
In summary, I think this chapter sets up the following chapters by creating vocabulary and tools for them.
“What did they say?” “That it’s supposed to work that way.” “And?” “I wasn’t sure what else to say. What do you think?” “I think that you need to develop some trust. First, develop some trust in yourself. Then develop some trust from the dev to you.” It’s a common conversation I have with junior testers. As testers, our goal should be to build trust. Trust enables us to our jobs.
What is trust?
The asset most required in our jobs as software testers is trust. That is because trust is knowledge of a person or group. The currency of that knowledge is the combination of competence, character, and commitment.
The first, competence, is how well we can do our jobs. People may think we have character, but we I say a feature has a problem, do people believe that I know what I am talking about. Competence varies for all of us depending on the domain. I can know much about restful web services (Rest) and not know much about User Experiences (UX). Or somebody may trust my knowledge in Rest but not UX. I could understand the technologies used but lack understanding of the business use.
The second measure of trust is character. I consider character to cross domains. I guess that somebody could have a character weakness, for instance if they are a covered alcoholic, but not relevant for this kind of trust. Do people know you give an honest answer? That would be integrity. Do they know you see things through to the end, or do you drop things when you get resistance? That would be courage. Other character traits of a tester include ability to communicate, and dealing with ambiguity.
And the third measure of trust is commitment to causes. A person could be honest and earnest, but wouldn’t lift a finger to save a spider if they don’t like spiders. Others will trust me or not if I am dedicated to those values that the team shares. This can vary from place to place. I have worked where the tester was ‘supposed to be the police’, as well as places where the tester was supposed to describe the quality level to management so they could make a proper decision. Popular commitments in agile include delivering value to the customer, continuous improvement, engineering for current (not potential needs), and always-working software.
Why do We Need Trust?
We do not hear many arguments against trust. But we do not always hear for a call to gain trust. Without trust, everything is more expensive in both money and time. For example, terrorist attacks in Mumbai and New York in the past decade created new, more expensive agencies to combat the potential for attacks. I have personally witnessed the cost in time every time I travel from an American airport. The example is extreme, but it scales down to our work. For instance, if I am not trusted to create tests then I will have additional review processes, sign-offs, etc.
Without building that trust, even when the competence, character, and commitment are present, you will pay the no-trust taxes. In a new relationship, the trust must be built. An assumed trust, one that is assumed upon meeting, can be duplicitous – other assumptions come with it.
How do We Build Trust?
How to build trust isn’t a simple question. That comes having competence, character, and commitment. When the currency is low then we will find building trust difficult. Those things can grow in time. If we study our craft, if we study our domain, we will gain the competence that convince our stakeholders that we ‘know what we are doing.’ As we mature, we build character from making the right decisions. As we question our beliefs and motives, we develop the commitment that other respect.
On the other hand, we build trust from others as we improve ourselves. We work in various domains – we have different levels of knowledge in those areas. We start at different trust levels in our relationships. We work with people who have different general tendencies for trust based on their experiences. Domain risk also plays into trust. I will be careful no matter who I am working with to build a pacemaker.
Lowest Trust – When we work with somebody that does not trust us, deserved or not, we can build a foundation of that trust by being polite, and simply helping them. Do not ask questions except for those that will help you do the task correctly.
Low Trust – Learn about them. Questions should be inquiring but not challenging. Your interest in them, and sharing some common experiences. Though you should focus more on them. What are their motives for what they are working on? What are their commitments?
Medium Trust – When you are at good level of trust, you can make suggestions to help them. THEN BACK OFF. Expect them to ignore your suggestion. In the occasion that they take your suggestion, be available to answer questions about details. Success isn’t necessary, but a big investment into a failure can lower the trust level.
High Trust – Once you have a high level of trust, you can act on their behalf if you tell them appropriately about what is done. Give them a report on what you will do, what you have done. For those working on Agile teams, this can sound like a ‘standup meeting’.
Highest Trust – When you are at the highest level of trust, you can just act on behalf of somebody. You know their interests and how they would decide themselves.
You can be at different levels of trust at different times of a relationship. It can go up or down for various reasons. If you haven’t interacted with somebody for a while, you might treat the trust level as if it atrophied. If you are a hired expert, you can probably skip the lowest trust level and start somewhere else.
Planning to Be Trusted
By evaluating the trust levels that your stakeholders have in you, you can begin planning on how to build a higher level of trust. The work you do when implementing a trust-building plan will actually make you better and more trustworthy. That trust is a cornerstone to becoming more effective as a software tester.