Continous Integration (Jenkins)

I fancy myself an automated tester. What’s so automatic about it if I have to run a command to make the test run? I guess it’s not. So how to I change that? In the past, I have created schedulers and such to kick off tests. Then I made listeners that would figure out if a new build existed.

Those were Mickey Mouse (please Disney, don’t sue me) solutions compared to the ones I started working with for the past four years – integrated continuous build systems, including Cruise Control, Hudson, Jenkins, and Bamboo (my chronological experience). None of those were my doing. Even when my framework was integrated, somebody smarter than me created that solution.

So in favor of not being left in the dark, I felt I should be able to create the same kind of integration. Step 1 is to install a server. I chose Jenkins, not for any particular reason other than I thought that I would be spending more time using it than figuring it out. I think that I was right.

Basic Set Up

At first I started installing the Ubuntu native Jenkins, then I decided it would be easier to download the jenkins.war file. First feature was to get it running. OK, that’s not a feature, but it was important. I hate apps/app servers that use port 8080 so I changed it.

java -jar /devroot/tools/jenkins/jenkins.war --httpPort=5050

I did not have to install any database for it. I will research that more. I added Jenkins GIT plugin to support integration with GIT. That was easy. Then I was able to link my learning project (downloaded Jeff Morgan’s Puppies app) via file link as:

file:///home/dave/dev/puppies/

I set it to poll every minute (hourly doesn’t seem like continuous integration to me).

Advanced Set Up

Finally, I created a command script (what happens when something in the GIT repository changes) with something to do:

cd /home/dave/dev/puppies
rails s &
cucumber --tags @full

To verify, I checked in a change. Here was my ultimate output (I didn’t mind the failures because the cukes are in disarray):

8 scenarios (8 failed)
45 steps (9 failed, 6 skipped, 1 undefined, 29 passed)
2m41.479s

You can implement step definitions for undefined steps with these snippets:

Then /^I should see the following error messages: "(.*?)"$/ do |arg1|
  pending # express the regexp above with the code you wish you had
endBuild step 'Execute shell' marked build as failure
Finished: FAILURE

After I proved to myself that the command scripts ran for check-ins, I added a little bit more. I set up basic authentication for users, allowing everybody to do anything (it’s just me). I also started working on an ssh key, but I didn’t finish that yet.

Summary

Jenkins was easy to install, set up for my repository (even easier for svn and cvs, though I am not sure why), and easy to create mock scripts. I found it easy to integration my test framework. Jenkins even knew that it failed.

I would recommend that any tester that is working on automation should feel comfortable installing and integrating with Jenkins.

Test Stack

Earlier this year I started thinking more about the test stack. The test stack is heavily integrated with the dev/CM stack. The automated tests run using a language, library, or languages and libraries. People edit the test files in editors or IDEs. The test code is stored in source control systems. Test are run by continuous integration environments, the results are stored. Deployment systems create VM’s for the system under test and for running tests. Testers tie their changes checked into source control to requirements and features in the issue tracking system.

I use these parts, these components, in my job. I have used different versions of them, different brands/sources, and in different ways. But until last week, almost all of my energy was focused on learning about the automated test languages & libraries. In some cases, I used a “stack in a box” that we can get from a vendor like Compuware or HP. Other times I was provided those components by IT departments and CM professionals.

But I decided earlier this year to learn more about that stack. I will not become an expert on those components. But I will install some of them, and integrate them, and get them all working together on my laptop (or from my laptop).

I will try to drop some insight into my experience with them. I have already worked with Windows, Mac, and Linux, including the on VM’s that software under test and test software can run on. I set up Git as a source control repository. This past weekend, I installed Jenkins that runs deployments and tests everytime code is checked into my Git. Last, I installed Redmine that I can record features that I will create. Those will be the parts I write.

I may install Chef, which deploys VM’s and configures them by deploying SUT and test software. I haven’t decided on that, or what kind of monitoring software (such as Splunk). If you have suggestions, I would appreciate reading them and possibly being influenced by them. I can’t do it all, but I want to know enough that I can better build one should I ever be in the position of a small office.

Update: I want to add database, of which I am using sqlite3 right now. Of course there is data loading.

Update 2: Somebody told me about flyaway, which is a database (scheme?) versioning application.

Continuous Integration (Jenkins)

Monitoring (Graylog2)