Motivation:
Recently I decided that it was time that I give Selenium RC a shot. I was attracted to the idea of recording scripts with Firefox and then working with them in a language of my choosing. All the cool kids at Defcon seem to think Python is the answer to everything, but since that’s still on my ToLearn list, I figured I’d default to the more familiar Java where I’m already comfortable. The option to do that was a big selling point for me. Since I’m looking at a JavaScript (using Strophe.js) jabber client, I was also happy to let Firefox deal with that. Another coworker is developing LoadRunner scripts for performance tests on the ejabberd server. Essentially his scripts must duplicate the basic functionality of the JavaScript code in LoadRunner’s C. Gross. Initially I thought I might borrow his scripts and tweak them for my own testing, but I realized that I could jump well ahead of him if the Selenium idea panned out. No redoing the JavaScript logic. No C code. Win.
A few days of puzzled stares at Eclipse and a hundred Google searches later, I think I’m still winning. It was not quite the effortless leap I had envisioned however. What follows is my adventure in getting Selenium to a point where I could start producing some actual tests. I found answers to many of these problems all over the place. When I can recall where I found them, I will link to the page, but I viewed countless pages wandering around and pleading to the Googles for help and don’t plan to retrace all of that. Hopefully someone else on a similar quest comes across this post and explains to me how I’m a total idiot, where to find these answers clearly written in one place, and how I could’ve accomplished the same thing in a smarter, easier way. If not that, perhaps some other lost chump like myself will be grateful to find a bunch of solutions in one place.
Problem:
The problems started early. I recorded a simple script. I converted it to Java (please don’t bother to tell me that this was my problem). I tried to run it with Selenium RC. At this point I found that Firefox opened up in a strange configuration, barfed a bunch of popups at me and then the test killed itself. Some poking around revealed that *chrome as the browser mode was at fault. Certain extensions were being loaded that just didn’t want to play nice and the profile would not remember the login for our company’s Blue Coat Proxy.
Solution:
Selenium allows for a *custom browser mode. This allows you to get your browser configured manually. http://www.borngeek.com/firefox/profile-tutorial/ me helped get my Firefox profile straightened out. I created a profile with minimal extensions (Firebug and McAfee’s SiteAdvisor were particularly rude to my Selenium tests). The profile (named selenium) is also configured to use the Selenium RC server as a proxy (localhost:4444). I made sure to disable warnings, popup blocking etc as these features may interfere with the test scripts. Since it’s a separate profile, I can easily return to my normal, more securely configured settings by restarting Firefox in it’s regular profile. The browser string in my Selenium setUp call now looks like: “*custom C:\\Program Files\\Mozilla Firefox\\firefox.exe -P selenium”
Problem:
Having defeated the popups, extensions and proxy settings I hit some SSL certificate issues.
Solution:
First, I added the cybervillainsCA certificate to Firefox as a trusted authority (only for the selenium profile!). That didn’t quite do the trick. It turns out the Selenium RC server doesn’t care for the certs in our test environment. Some more Googling revealed that restarting the selenium server with -trustAllSSLCertificates.
At this point I’m starting the server with
java -jar selenium-server.jar -trustAllSSLCertificates
That’s actually in a .bat file which has a shortcut in my Quick Launch bar so that it’s started with a single click.
Problem:
I figured by this point I’d done the configuring that I’d need to and now the coding party would start. When I recorded the script I had done a simple login, navigated to a second page, logged in to the test chat client and then stopped. This failed and I noticed that the script had not recorded the second open, so it was looking for the second login form on the wrong page. That was weird I thought, but I added it in. The test still broke. Here I learned that I was not allowed to start on one domain and then open another. I had logged in at sub1.testenvironment and the second page was at sub2.testenvironment. The problem relates to cross domain restrictions browsers enforce on JavaScript. Selenium controls the actions with JavaScript so it has a meltdown here. It turns out that the *chrome browser mode operates in a way that avoids this restriction. Here “chrome” is a Firefox mode, not a reference to Google’s browser. That’s a bitch when you’re searching to understand what it is and how it works. I already found back at the beginning that the *chrome mode didn’t work for me, so then what?
Solution:
Some google wandering turned up this very useful document: Selenium Tutorial. It indicates that this problem can be addressed by running the Selenium server in proxyInjectionMode. Ok, fine. Now I’m starting the server with:
java -jar selenium-server.jar -proxyInjectionMode -trustAllSSLCertificates
Problem:
That victory celebration was cut very short as I learned that this mode was broken. Before making that change I was able to have my script open the browser, load a page, login, load a page and then fail on a cross domain navigation attempt. After the change, the whole test failed right at the start. A step back! Bummer. ”onXhrStateChange.bind is not a function on session” WTF does that mean? A couple resources essentially said “it means you’re using proxyInjectionMode, don’t do that.” Well shit, I kinda have to don’t I?
Solution:
http://groups.google.com/group/selenium-users/browse_thread/thread/bc7030cd44730d4f
A clue! Put “true” as an extra argument! There’s a slight problem with this however… Java doesn’t really care for additional arguments. You can’t go around forcing extra values on methods. Previously I was just using the provided selenium-java-client-driver.jar. Thankfully, the source jar is also there, so digging into that we get down to DefaultSelenium.java. There’s an array that I can put “true” into. Let’s see if it works:
public void open(String url) {
commandProcessor.doCommand(“open”, new String[] {url,});
}
becomes
public void open(String url) { commandProcessor.doCommand(“open”, new String[] {url, “true”}); }
Booya! Cross domain problem is gone for me.
Problem:
Now the script is logging in on sub1.testenvironment, navigating to sub2.testenvironment and apparently loading the chat client page, but the script cannot find the elements on the page. Actually worse than that, it claims it can’t find the window. Somehow after the second selenium.open() call, it completely loses track of the window and can’t do anything else.
Solution:
Rather than open() for the chat test page, I used openWindow() which allows me to name the new window which Selenium is now able to find. This seems to be a rather inelegant solution, to just start popping up new windows, but the second should be as far as I go so it’s not a huge deal.
Problem:
IT IS STILL NOT SUCCESSFULLY SUBMITTING VALUES ON THE CHAT PAGE!!!!!
Solution:
Timing. You can’t take anything for granted about Selenium’s timing. While to my eyes, I’d see the browser window load with all of the necessary fields, Selenium would claim that they were not there. I used waitForPopUp() after openWindow(), but that does not ensure that the JavaScript client has successfully executed it’s initialization. Some additional code was added to wait for Selenium to find the element before trying to type in it.
System.out.print(“Waiting on login form”);
while(!browser.isElementPresent(“username”))
System.out.print(“.”);
browser.type(“username”, “testUser2″);
Sometimes it spins for awhile, other times it goes almost immediately, but it always succeeds now. After all of that I don’t have any actual tests to show for it, but in my naive view, I’ve finally cleared the hard part and can actually throw in the test cases from here.