Category Archives: Security

Test Automation Curriculum

Two things happened to me lately. First, I was trying to find a career tester in the San Diego area that knows at least a little bit about automated testing. It isn’t going well. I’ve reviewed a lot of resumes. all the submitters are career manual testers.

Surely somebody sometime must have wondered if they need to learn more about automation. Elisabeth Hendrickson once asked Do Testers Have to Write Code? They did a survey to figure out what companies were looking for from tester skills. In our case, we aren’t looking for somebody to write the test code, but to write and review the cucumber scenarios. Just the same, even on a light desire level, I was disappointed.

Second, a younger person asked me what he should learn in test automation last week. I had already been contemplating writing this curriculum, so I was resolved to do it. Srini, here it is.

Other people who don’t work with LAMPs, such as .Net environments etc. will probably not appreciate this list. Make your own list on your own blog and put the link in a comment here. I don’t begrudge anybody doing something else. I just don’t want to go there.

I created this curriculum for testers learning test automation. While some addresses how and why, most of the list is about tools that can help create a full solution. Anyway, here is my list in priority order:

  • An open source tool such as Watir-Webdriver or Selenium/Java – do not mess around with the QTP and TestComplete. The cargo cults that buy those tools will expect “anybody can automate”.  With open source tools, you can download your own learning playground and incorporate that with the other products.
    • Learn how to create page objects. Even if you take advantage of a library like WatirMark or Page-Objects, you will have to do some tailoring yourself. I have been working with Selenium/Java so I am developing my skills on that combination now. Either way, you need to know how to work on that in an efficient way. In fact, you can address a lot of the entries in here just be using Cheezy’s book Cucumbers and Cheese (well worth the $15). I swear that I do not get a dime from it or Cheezy’s work, it’s just such a big benefit for anybody learning that I cannot miss the chance to say how good it is.
    • An open source framework such as Cucumber, Cucumber-jvm, or RSpec.
  • Github and Git – there are other good source control tools out there, including subversion. Git is easy to use locally for managing your own practice code. It’s easy go get copies of other people’s public projects onto your own system (how did they do that?). CodeSchool has a free course on git. There is also a nice paper on the differences between git/mercurial and subversion so you can understand the differences.
  • Ant and Maven if you use Java. Most of what I learned was through osmosis, but being able to shoehorn cucumber into your project is good to know.
  • Jenkins or Hudson, CruiseControl, or some other open source continuous integration tool. If you ever work at a place that will be introducing automated testing for the first time, this is great to know how to set it up.
  • Performance testing in JMeter – I think you can find a ruby alternative (BlitzIO or Grinder) but you don’t really need this tool to be in a ruby language. The importance is to learn the different kinds of testing you find under this umbrella (incorrectly) called Performance Testing. The other important skill is creating the right monitors so you can discover where things are bottled up.
  • Owasp‘s ZAProxy – learn how to capture the http calls between your browser or simulator and the server under test. You will learn a lot there. While you are there, download the GoatWeb project where you can learn about security vulnerabilities through practice.
  • Monitoring tools (Splunk or Graylog2) – One way to find the errors that are occurring on the system under test is through logging. Those are deleted nearly every time the server is redeployed. You can monitor those logs and server performance much better through a monitoring server.
  • A true startup is probably not going to hire a newb unless they are cost-control-centered. But if you find you get there are there is no issue tracking, it would be good to know how to set up issue tracking and integrate to your version control and continuous integration server. I’ve tried RedMine and it was fine.

If you see that you think should be on the list that is not there, please add a comment.

Improving the Value of Testing – Security!

Do what you say and say what you do.

I think I got that from an ISO 2001 audit preparation meeting in the mid-90’s during an effort to sell fax machines that we were manufacturing at HP to the EU. I like that so I try to use it.

Do What You Say

I said that I was going to try Improving the Value of Testing. What would be better than security testing? A bunch of things, you might say. But the reality is that security is the highest risk you are facing in your products. The bad guys understand more than you do, and probably more than the people who make the security tools you use already. For me, I do not even understand much about what the tools do, or know the difference between sql injection and cross-site scripting. 

Say What You Do

So I am going to venture into this a little by trying to do some security testing with tools that I get from where ever. I will even make some home-grown tools if possible because I like to build and I like control. That would help amp my understanding to a higher level, in my opinion.

My first attempt was to crack open an old book How to Break Web Software by Mike Andrews and James A. Whittaker. Things change a lot in 6 computer years. All the web services are in SOAP – yuck. That’s like getting your mouth washed out. And almost all the tools are for Windows, but I primarily use a Mac. Still, I think I can get some concepts out of this. I try the paso proxy, but it’s not working for me yet.

So I move on to SoapUI. That’s an old friend, but I have never used it for security nor Rest. I spent some time on trying to simply send a request (POST) to my system under test but the SoapUI crashed. And crashed. And crashed. I tried five times before I went to their forum and found an recent unanswered post called Clean Install: Mac OSX beach ball of death. Oh dear. 

I spent a lot of time on those without getting anywhere. Edison would have said that I learned some ways that it doesn’t work. I will add more as I have time and additional information!