Tag Archives: tests

Improving the Value of Automated Testing

I have an idea. I will not get too high on it except that it is intriguing to me. Maybe the idea is not new to other people. The idea came to me while I was thinking about the test automation pyramid (or ice cream cone as automation expert Alister Scott recognized the typical shape). I am fond of the three-layer concept – enough so that I made myself learn how to write unit tests and service-layer tests, in addition to the GUI-based tests that are commonly practiced by software testers.

I am thinking about the shape of the triangle, which part should be how big, etc. Suddenly the problem hits me. The problem is not isn’t the investment in automated tests. It isn’t the maintenance (which I thought maintenance was a big issue). Alright, I am lying. Investment is half of the equation – the return on investment calculation. The problem is the (lack of) focus on the return for the investment.

So much is invested in tests that will find what? Low severity defects that do not halt releases? I do not care about low severity defects until I have sussed the high severity defects. I care even less about them when I am faced with making a substantial investment to find them. Where does that leave me? Focus on the big bugs. Automation for big bugs. Which bugs? The ones we do not want to ship with them in. The ones that require a patch if you miss them. The ones that make you slip release dates if you catch them late.

“I don’t care about low severity defects until

I have sussed the high severity defects.”

I expect to get criticized for this idea. Why? I have no experience on this. I have never tried it. It is an untried idea. What could be less valuable than a hypothesis? I can try it, but this kind of thing would take a while to prove itself out. But I cannot worry about that while I am brainstorming an idea. The idea will be flawed in many people’s minds. Automated tests are really just automated checks. That is not new. People are not particularly suspicious that the results are false positives, they do not like that the test are little positives – that is they test so little compared what a person can do. I believe this concept too.

What I really believe in is the ends, not the means. The means is how I will get it done. The ends is what I want to accomplish. I care about the automation only when it affectedly helps me get the good defects. What are those? In my world, those are the defects that reduce my organization’s ability to meet agreed or assumed service level agreements. I call those Enterprise Readiness defects. Examples include poor performance, poor performance after time (example: caused by a memory leak), poor resilience during or after high loads, problems with failover, and problems with data retention.

How can I accomplish that? By remembering that automation is just a tool in the toolbox.

More on this later…

Advertisements

There is an App for That – or Ruby Library

We have heard the ads and promotions for the iPhone. There’s an app for that! Of course there is. Sometimes the app is even good. I am bummed when it is not, but I will keep trying and evaluating them – yes, I have an iPhone.

I realized that this phenomenon also occurs with the Ruby scripting/programming language. I was teaching my co-workers how to use Ruby & Watir, and the benefits of those. I wanted to demonstrate the usefulness of Ruby as a scripting language so I started to show different libraries that could be used. Although it had been around me for years, I finally realized that Ruby covers some major ground thanks to all of the developers and hacks (I use that term in a loving way)  that spent hours, days, weeks, and months extending Ruby to do more.

First, when I read through the Pickaxe book, I learned about many of the installed libraries I can use with Ruby. Sure, we expect to have the string, array, and hash types. But with a simple require ‘net/http’ statement, I can request web pages over the internet – without a browser! I can create my own app server using webrick – I did just that as a target for my squid server test. I can work with file systems, command line parameters, cgi scripts, yaml streams, xml streams, and test it all with test unit.

But wait, that’s not all you get! I learned Ruby to build a test framework. After that, I found myself in a manager position where I could not spend so much time working on automated checking and frameworks. Wanting to keep in practice, I started using Ruby to solve other computer problems for me. I created scripts to help me manager my music library. I automated tedious tasks at home.  For all these, I found websites that could help me identify the best one to use for solving my problem including Ruby Toolbox, RubyGems, and GitHub where many of them are created. On GitHub, you can even fork the project that you kinda like but want to change some stuff. I have seen it done! And you can get it for the low, low price of Free. You will never get that kind of free stuff and you can change it on an iPhone app. And if you order now, you can get an additional Ruby installation, just pay storage fees.

If I had to pick a favorite, it would be Watir (Web Application Testing In Ruby) because that library enticed me to learn Ruby. Because I have learned more libraries and how to use them, I have ventured away by using database client libraries, web service libraries, and faster-performing web libraries.

I would like to know – what is your favorite Ruby library?