Book Review Pt4: Black-box testing: techniques for functional testing of software and systems

Chapter 3: Control Flow Testing

This chapter introduces a basic graph, control flow. I found the tax documents for this exercise.

I read this in two parts. The first part, after the standard definitions section, focused on the relationships of a control-flow model, how they vary, and how to draw them. I liked the use of the truth table to show the relationships in a compound predicate.By the end of this section, I knew (remembered might be a better term) how to use the tactics of this technique. I could differentiate when the nodes would be serial instructions and when they would be selection as an if or case statement..

Dr. Beizer presented a step-by-step instruction set for building and using the control-flow model for testing. The model development seemed to be as if I were modeling a script or coded program. but showed what happened in the methods as well. The steps within the tax instructions gave that feel. The space of each instruction gave us a memory register to store the result. Each instruction did one thing… well, mostly. While I didn’t like the idea of reviewing 18 year old tax instructions, it worked well with this lesson.

The coverage, which is beginning to be developed in the previous chapter, starts to become more relevant. Where is the risk? In some places, the decisions seem somewhat arbitrary; to the whim of the author. Most of the coverage decisions are direct to the risks, and causes of the risks. Even within those, some of the reasons appear to be “old” ways of thinking.

In summary, this chapter gives the tester something they can use right now. It is the first finished feature of model-based testing. It is releasable, usable, and helpful.

Book Review Pt3: Black-box testing: techniques for functional testing of software and systems

Chapter 2: Graphs and Relations

This chapter introduces graphs as the central tool of testing.

Just running through the definitions got me pumped for testing. Every relationship Dr. Beizer described brought about examples in my head from software that I commonly test in my job. Then I saw the Question/Answer of the book. What do you do when you see a graph? Cover it! In 20 years, that has never left me or my test methodology.

I am not an expert on graphing so I did not see anything blatantly left out. I noticed in the undirected links section of definitions that the author didn’t think symmetric relations are not common in testing – I thought that modern software models include load balanced services, as well as multiple threads, which may have those relationships. I have tested those relationships many times.

Regarding the graph types used in testing, I took interest to the those. I felt that the transaction flow model he described doesn’t get enough attention. The finite-state model of menus… modern menus often change based on the current task or context – in the author’s word processing example, one may see a different menu set when editing a table.

While I enjoyed the discussion of transitive/intransitive relationships, I really liked the end. To paraphrase, If specification or programmer is confused about transitivity, then you have a fertile ground for testing.

The examples of the relationship types are helpful. Except the part about equivalence classes. I had to look at 3 web definitions and two YouTube videos before I felt comfortable. Then I reviewed a single partition example and said “oh yeah, I know that.”

The concept of the matrix of relationships between nodes was completely boring, but so much more efficient. If I were modeling with intention to automate coverage of a model, I would use the matrices in some form. But I get why Dr. Beizer says “…the format is cumbersome and error-prone for large matrices.”

The generic testing principles outlined have covered a great deal of my testing lifetime. It’s also the ToC for the rest of this chapter. Each instruction expands to a cookbook for creating a testing tool.

  1. Define the graph
  2. Define the relation
  3. Node Cover
  4. Link Cover
  5. Test All Weights
  6. Design Loop Tests

The instructions are simple, and mostly straight-forward. I didn’t particularly like calling properties as ‘weights’ because I think of weight as a particular type of property. Like the weight might show the likelihood one branch of the graph is used compared to an alternate branch. The author also describes the purpose & significance of each step.

I am not sure the ‘test all weights’ is necessary as a separate step. The properties can be tested within the node cover & link cover. They should be tested there.

In summary, I think this chapter sets up the following chapters by creating vocabulary and tools for them.

Book Review Pt2: Black-box testing: techniques for functional testing of software and systems

Chapter 1: Introduction

This chapter introduces testing purposes, roles, and methods.

The first thing that hits me are the definitions. Is this book like 20 years old or something? The Performance Bug and Security Bug definitions have grey hair. I like how Dr. Beizer dances around the unit, component, and integration tests. I can now understand it as well as I’ve ever understood it – but I won’t try explaining this to a child because they’ll tear me apart. People don’t try to break software anymore. They find where it’s already broken, it’s weaknesses, and it’s limitations. Otherwise, I generally agree with the definitions.

I agree with the numbered reasons for testing software. Most of these reasons can be incorporated into a cucumber scenario. The “penultimate objective” implies that using some kind of find-rate will help make a decision about readiness – I disagree because it could be a problem with the pesticide paradox.

In the Test Strategies section, test strategy is so much more than test techniques. I’m surprised to see them treated an synonyms. But there… BAM! Behavioral testing IS black box testing. I also like that the author doesn’t assume that unit tests are white box tests – one could test units and methods without knowing the code within them, only the promise of the interfaces.

I noticed that when the author talks about the progressive cost of testing units, to components, to integration, to system testing, he leaves out that the biggest cost of those bugs found late are because the time passed since the bug’s introduction is greater AND the number of people involved in repeating past work is higher (building, deploying, and possibly releasing).

In the testing isn’t everything section, I am wondering what happened to Formal Inspections. I’m fully trained in them, though I haven’t participated in one since probably 1997. I like how he says don’t try to justify automated tools as if that ever worked. I hate to do it but I have to do it.

The process section seems pretty old school as well. Old fashion we must have group discipline vs. current day we must have personal discipline. Or maybe I’m just remembering the command & control days poorly, and the short-cuts of today as not poorly enough.

I did not love this chapter. It’s basic and out of date. If I were to teach this chapter to younger testers/developers, I would focus on what has expired, and what replaced it. Then I would ask for reflections of what the students wish we had retained from the days this book was written.

Book Review Pt1: Black-box testing: techniques for functional testing of software and systems

On the 20th anniversary of Black-box testing: techniques for functional testing of software and systems (1995) by Dr. Boris Beizer, the first book I ever read about… actually, the first thing I read about software testing, I review the book to introduce it to the generation of testers that never heard of it.

First, I have to admit that I forgot the premise of the book was to build knowledge about modeling in order to support development of software tools. I generated tests with a product in the late 90’s called TestMaster by Teradyne Software. Yes, I just linked to GeoCities. I also developed my own generator after being inspired by Harry Robinson’s Intelligent Test Automation.

The section about Missing Models explains this book in relation to software testing books from before this book’s existence, focusing mostly on Myers’ The Art of Software Testing. It describes different types of models including logic-based models and language-based models. Thanks for reminding me about how much I have not read.

Right after the preface, BAM! I see the readme.doc chapter. Now I know that I’m dealing with a softare professional, in the greatest sense of the word. Read this, it’s quick. The books is efficiently organized, so it’s to have a good map. The readme explains the structure of the chapters in relation to each other, and the structure of each chapter.

I am already excited about this.

A Quick Way to Test Rest Web Services


Photo “Seagulls on the American River” courtesy of Vince Mig.

Testing a Rest API

When I think of rest, it’s usually the type that requires a bed or comfortable couch. But these days I am spending more time verifying intermediate components of a larger system that take advantage of the synchronous behavior known as a restful web service than working on web GUI interfaces.

Continue reading

My Customizations in ZSH

I have added the following as oh-my-zsh custom plugins. I did this because I like to sync my customizations between computers, but I don’t want them all to be active on every computer. I can tailor that with my .zshrc
plugins=(git ruby gem mvn rvm sublime safepaste autojump ushare pipe directory hardware ackgrep  command-not-found)

directory.plugin.zsh – this is supposed to be mostly directory stuff, but I added in my autojump statistics, disk space, and tar/zip commands. Seemed relevant to me.

alias as='autojump -s'
alias po='popd'
alias pu='pushd'
alias home='cd ~/'

alias md='mkdir -p'
alias mkdir='mkdir -p'
alias RF='rm -rf '
alias path='echo -e ${PATH//:/\\n}'

alias ..='cd ..'
alias ...='cd ../..'
alias ....='cd ../../..'
alias .....='cd ../../../..'
alias lsa='ls -a'
alias ll='ls -l'
alias lla='ls -la'
alias la='ls -a'
alias lf='ls *(.)' #just files
alias ld='ls *(/)' #just directories
alias df='df -kTh'
alias du='du -kh'
alias tj='tar -xvjpf '
alias t='tar xvfz '
alias tz='tar -xvzpf '
alias tb='tar -xvfj '

ackgrep.plugin.zsh – I called it ackgrep (my favorite seach tool), but I use it for searching just about anything that would otherwise be annoying.

alias a='ack-grep '
alias apage='ack-grep --pager=less -r'
alias ainvert='ack-grep -v '
alias ag='which '
agg() { ack-grep -H $* $ZSH_CUSTOM }

alias f='find . |grep -v svn |grep '
alias findjar='find . -type f -name  *.jar |xargs -n1 -i -t  jar tvf {} |grep '

alias h='history'
alias hg='history|grep -v grep|grep ' 
alias hgg='history|grep '

alias pg='ps -eaf|grep -v grep|grep ' 
alias pgg='ps -eaf|grep '

alias lg='lla|grep '

pipe.plugin.zsh – I found these in a customization, either on the internet or on a VM that somebody created for me (probably from Steve G). I can’t even remember I have them, but when I do it’s easy living.

alias -g DN=/dev/null
alias -g EG='|& egrep'
alias -g EH='|& head'
alias -g EL='|& less'
alias -g ELS='|& less -S'
alias -g ETL='|& tail -20'
alias -g ET='|& tail'
alias -g F=' | fmt -'
alias -g G='| egrep'
alias -g H='| head'
alias -g HL='|& head -20'
alias -g LL="2>&1 | less"
alias -g L="| less"
alias -g LS='| less -S'
alias -g MM='| most'
alias -g M='| more'
alias -g NE="2> /dev/null"
alias -g NS='| sort -n'
alias -g NUL="> /dev/null 2>&1"
alias -g PIPE='|'
alias -g R=' > /c/aaa/tee.txt '
alias -g RNS='| sort -nr'
alias -g Sk="*~(*.bz2|*.gz|*.tgz|*.zip|*.z)"
alias -g S='| sort'
alias -g TL='| tail -20'
alias -g T='| tail'
alias -g TF='| tail -f '
alias -g TFN='| tail -f -n '
alias -g US='| sort -u'
alias -g VM=/var/log/messages
alias -g X0G='| xargs -0 egrep'
alias -g X0='| xargs -0'
alias -g XG='| xargs egrep'
alias -g X='| xargs'

ruby.plugin.zsh – some of these are in the standard ruby plugin. I wanted more because rvm doesn’t cooperate as much as it should on my zsh.

alias sgem='sudo gem'

# Find ruby file
alias rfind='find . -name "*.rb" | xargs grep -n'

# #RVM & Ruby
alias rvm_login='/bin/zsh --login'
alias rl='rvm_login && rvm use 2.0.0'
alias ru='rvm gemset use '
alias rv='rvm use '
alias gg='gem list|grep '
alias bu='bundle update'

# Add RVM to PATH for scripting
export PATH=$PATH:$HOME/.rvm/bin

alias yd='yardoc features/**/*.feature features/**/*.rb lib/**/*.rb'

hardware.plugin.zsh – this is just to help me with my laptop annoyances. Like that stupid touchpad that my wrist rests on while I type.

getID() {
	xinput list|grep "Synaptics TouchPad"|awk '{print $6 }'|cut -d'=' -f2

alias laptopprimary='xrandr --output LVDS --primary'alias tpf='xinput set-prop `getID` "Device Enabled" 0'
alias tpn='xinput set-prop `getID` "Device Enabled" 1'