San Diego Quality Meetup 5/4/2016

I have long hoped that San Diego testing community would develop a user group/meetup. My recruiter, Jon Barton of TEKSystems, started a meetup group. I attended a meeting in April. Then I volunteered to speak at a future meeting. That future meeting was last night.

http://www.meetup.com/San-Diego-Quality-Engineering-User-Group/events/230308712/

The periscope video:
https://www.periscope.tv/JonBartonTEK/1yoKMmanbQeKQ

The slide deck:
https://drive.google.com/open?id=1oJyGQ2Id9kVPJ7Qlbn79mnuDBbx6MInWy0qkOsuy5Iw

The book I recommend:
https://leanpub.com/cucumber_and_cheese

I am hopeful for the future of software testing in San Diego if they group becomes a place for people to step out of their circle of comfort, to mentor each other, to learn and grow.

There is an App for That – or Ruby Library

We have heard the ads and promotions for the iPhone. There’s an app for that! Of course there is. Sometimes the app is even good. I am bummed when it is not, but I will keep trying and evaluating them – yes, I have an iPhone.

I realized that this phenomenon also occurs with the Ruby scripting/programming language. I was teaching my co-workers how to use Ruby & Watir, and the benefits of those. I wanted to demonstrate the usefulness of Ruby as a scripting language so I started to show different libraries that could be used. Although it had been around me for years, I finally realized that Ruby covers some major ground thanks to all of the developers and hacks (I use that term in a loving way)  that spent hours, days, weeks, and months extending Ruby to do more.

First, when I read through the Pickaxe book, I learned about many of the installed libraries I can use with Ruby. Sure, we expect to have the string, array, and hash types. But with a simple require ‘net/http’ statement, I can request web pages over the internet – without a browser! I can create my own app server using webrick – I did just that as a target for my squid server test. I can work with file systems, command line parameters, cgi scripts, yaml streams, xml streams, and test it all with test unit.

But wait, that’s not all you get! I learned Ruby to build a test framework. After that, I found myself in a manager position where I could not spend so much time working on automated checking and frameworks. Wanting to keep in practice, I started using Ruby to solve other computer problems for me. I created scripts to help me manager my music library. I automated tedious tasks at home.  For all these, I found websites that could help me identify the best one to use for solving my problem including Ruby Toolbox, RubyGems, and GitHub where many of them are created. On GitHub, you can even fork the project that you kinda like but want to change some stuff. I have seen it done! And you can get it for the low, low price of Free. You will never get that kind of free stuff and you can change it on an iPhone app. And if you order now, you can get an additional Ruby installation, just pay storage fees.

If I had to pick a favorite, it would be Watir (Web Application Testing In Ruby) because that library enticed me to learn Ruby. Because I have learned more libraries and how to use them, I have ventured away by using database client libraries, web service libraries, and faster-performing web libraries.

I would like to know – what is your favorite Ruby library?

Exploratory Testing with Interactive Ruby

I want to premise this post with two things. First, I didn’t think of this. I heard Jim Knowlton interviewed on the Watir Podcast. I also attended a web cast with Michael Bolton on using automated testing in a manual form. Both inspired me to try this myself. As some of my software friends are aware, I dig testing with Ruby and Watir (Web Application Testing In Ruby). I didn’t know anything about Ruby or Interactive Ruby (IRB) before working with it, starting in about 2007-2008. But since starting to use it, I found I am more productive by “trying” my code out in the IRB interface before adding it to my functions. Very handy!

My eyes were opened even more when I heard Jim talking about using IRB to facilitate his testing (and his co-workers) by creating libraries that assisted him. I started to envision things I could use it for: pumping data into a system for create conditions for testing business logic; grabbing near-real time information from log files; quickly getting to the ‘state’ in which I want to explore; and grabbing unseen information about the interface that I am looking at such as the state of objects and making them ‘seen’. I remember being so excited that I asked other people to listen to the podcast.

The webcast with Michael was revealing too. I could see with a simple command line IO, inputing numbering and getting numbers back helped me quickly learn about a system that I knew nothing about. He talked strategies to use exploration, as well as tactics that supported the strategy.

Those lessons came to fruition at my new job. I started with needing to test a web service. I have tested SOA web services before. But I had not tested the with Ruby before. I saw the team I was joining using a RestClient. They were modifying the ‘get’ and ‘delete’ calls based on the data set from the ‘create’ call. I asked myself how many times I wanted to copy that data, the answer was three times. So I pulled out the restful web services book that I had opened one or two times in the past, and found a snippet of net/http to call web services. Once I figured out how to add the pieces I needed, in less than a day, I had a class that could perform all three calls and retain the data, allowing me to test the web services api until I was satisfied that it was working according to the design. Running these from IRB allowed me to run manual tests. I kept changing the tests. Even though they were written in QC with instructions on parameters, I kept asking myself “what if I did it this way?” which allowed me to learn and satisfy myself that the API was strong.

I also saw  the other team members copying and pasting data from the create api call to use in a lynx (text-based command line browser) for testing business logic. I found all the typing and copying unnecessary and mistake-prone so I ‘printed’ the lynx command out allowing me to copy and paste the entire thing. This allowed me to more quickly get to the business of testing. I used that gained time to add a direct command from Ruby, reducing a step to run quicker, and finally incorporating the Mechanize library with that saved time.

The fruit of that labor was to allow me to test more quickly from the web service call to the browser call. That in itself did not find any defects, but it enabled me to see the need for faster (virtual) browser calls which did find three major defects and a minor defect. In spite of these gains, my manager was concerned about the cost of maintenance of early automation – a lesson I learned from James Bach’s Test Automation Snake Oil that says “Manual testing, on the other hand, is a process that adapts easily to change and can cope with complexity. Humans are able to detect hundreds of problem patterns, in a glance, an instantly distinguish them from harmless anomalies.”  I gave her a demonstration to show that my mind was in control of these tests. She felt at ease and asked me to continue.

The final culmination was today when I discovered a problem in a caching algorithm that would not be detected with smaller slower data-flows. I had no intention to test for load. I wanted to test functionality in a system environment and more realistic volumes. At this point, I did create a Ruby “script” but even then it took command line arguments to allow me to modify the test cases as I saw fit. There were no magical red or green colors that filled a test management system with results. Just tests that I ran as I felt were needed to learn if this system was ready to release or not.

– Dave McNulla