In the team I’m currently working with, the need for regression testing was becoming obvious. The team is working on a web MVC-type framework, and has been running into the results of limited testing: quality wasn’t just low, it was unknown. The usual excuses were there, pressure to release, and interruptions by unplanned work. Usually, the unplanned work is caused by the same lack of quality that is probably caused by pressure to release.
Since this is a web-intensive project where the front-end is not just important, but actually a core part of the software, we decided the first order of business would be to get Selenium tests running on the test/showcase applications. I’ve worked with Selenium before, but not with the GWT front-end technology, and not with the Selenium’s WebDriver APIs. I thought I’d share the process, and some of the issues encountered. In this post, I’ll describe how to get selenium running from a local maven build while still being able to run the unit/micro-tests separately.
Integration into the build
The project is built using Maven, so the first step is to get both the selenium dependencies in there, and be able to run a simple Hello World test from the maven build. The Hello World test was simply an empty Selenium JUnit test that opened the test-showcase, the hello world applet within that, and checked the window title.
Getting this to compile requires you to add the selenium-java dependency into your maven pom.xml file:
Note: I initially used the 2.3.0 version of selenium. That version does not work together with FireFox 6, though, and that broke my tests for a bit when I installed that update.
At this point, as long as the application under test is running outside the build, just doing a mvn clean install will run you selenium test, and you’ll see Firefox starting up, doing whatever you’ve defined in the test, and quitting again. Success!
Next, I would really like to get this running without depending on the external process of starting your application outside of your build. I won’t go into the troubles you can have if you’ve tailored your build process for deployment on some proprietary application server (check out the cargo plugin, as a starting point). But luckily, in this case, the system started without any problems by simply doing:
mvn jetty:run
This starts jetty with the default configuration, which mean port 8080, which I don’t want happening when we’ll be running this later on our build server, since that one uses port 8080 itself. More importantly, I don’t want to run these tests with every build. Integration tests using a deployed application are simply too slow to run with every build. Developers should be able to run them locally, but not when they’ve only added new unit-test (and any production code that came with those).
The way to deal with this is to use a maven profile where you can switch certain phases of the build on and off. Another way would be to create a separate maven (sub) project that is there only to run the integration test build, but that was not the direction I chose. So what does such a profile look like?
That’s quite a lot of XML, but in short the integrationt-test profile does the following:
- Activate the integration-test profile when the -Dintegration.test property is set
- Start jetty (jetty:run) before doing the integration-test phase
- Stop jetty after the integration-test phase
- Do NOT run any unit tests if the integration-test profile is enabled
- DO run all the integration tests in the package
com.example.seleniumwhen the integration-test profile is enabled
Which is great, and works. But when we then run the normal build, the selenium tests are still run (and fail). So that’s where the micro-test profile in the snippet above comes in: it excludes the selenium tests from the normal surefire testrun.
So now if we do mvn clean install all our normal micro-tests are run, and if we do mvn -Dintegration.test clean install the micro-tests are skipped and the selenium tests are run.
Now we are ready to write a real test, but I’ll save that for a future post.