Recently I’ve been working on a website that is a single page AJAX driven application that’s essentially just a frontend to an API, rather than doing any of it’s own processing.
For testing it I wanted to be able to test without hitting the API but since the application is so javascript heavy and makes use of various complex workarounds for various browser issues, testing it using the usual methods never worked and only “real” browsers cut the mustard.
I’ve tried a number of headless browsers such as zombie.js but they’ve all had issues so I’ve settled on using selenium for testing in a real browser. The other issue is that most web application testing systems assume at least some kind of backend system that you can then mock the http calls of. However in my case the entire app is client side and has no server at all, so doing that wasn’t an option.
The solution I came up with was a cobbled together selection of rake tasks, shell scripts, a custom node.js server and some hooks in cucumber to set up fixtures. I run my tests against a live server through a recording proxy to get the results of the API calls, then format the results into little fixture files to get loaded by the fixtures server based on tags on my cucumber scenarios. It’s quite a flexible setup now it’s running, the only chore is writing the fixtures, though hopefully I can knock up a quick script to generate the fixtures automatically from my proxy recordings.
I wont go into the details of running cucumber with capybara and selenium, since those are already well documented elsewhere, such as on the capybara github page. However, I’ll run through my scripts.
First off, I have my bash script to actually setup my server and run all my tests, it goes a little something like this:
UPDATE: Made some amendments thanks to suggestions from Ralph Corderoy in the comments.
UPDATE 2: Removed set -eu line since breaking out on errors means not shutting down the fixture server when tests fail.
|
|
#!/bin/bash echo "Starting fixtures server..." node ./fixture-server.js & PID=$! sleep 1 echo "Running tests..." rake features echo "Shutting down fixtures server..." curl http://127.0.0.1:7357/_fixtures/shutdown &> /dev/null sleep 1 echo "Force shutdown of fixtures server..." kill $PID &> /dev/null || true |
What is happening in the above should be fairly obvious, but essentially I’m running my fixtures server and capture it’s PID so I can shut it down later. I wait for a bit, then execute my tests via a Rakefile, then I send a command to the fixture server to shutdown, wait for a bit, then force a shutdown in case it’s not done it already. Simples!
The fixture server can be found in my random scripts repository on github. It’s hard-coded to run on port 7357 but that’s easy to change. I’ve tried to document how it works in the code but a gist of it is that the fixture server will server static assets but fall through on all other requests to look at whatever fixtures it has loaded. By default, it will have loaded no fixtures so will 404. However, it has some special URLs for loading in fixtures. On /_fixtures/load/:fixture the server will load a json fixture file with the name :fixture.json. Multiple fixtures can be loaded for the same path and the server will work it’s way through them, serving each one once until it has no more where it will then 404 again. Fixtures can be cleared from memory with a call to /_fixtures/clear and you can inspect the currently loaded fixtures by calling /_fixtures/inspect. A call to /_fixtures/shutdown will instruct the server to shut itself down. At the moment the fixture server only supports loading fixtures for GET and POST requests as that’s all I’ve needed, but it’s easy to extend that.
The fixture files themselves are very simple. Here is an example one:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
[ { "method": "GET", "path": "/test", "status": 403, "headers": { "Content-Type": "application/json; charset=utf-8" }, "content": "{\"success\":false,\"error\":true,\"error_code\":403,\"error_message\":\"Unauthorized\"}" }, { "method": "POST", "path": "/test/post", "status": 302, "headers": { "Location": "/" }, "content":"" }, { "method": "GET", "path": "/test", "status": 200, "headers": { "Content-Type": "application/json; charset=utf-8" }, "content": "{\"success\":true,\"message\":\"Authorized\"}" } ] |
In the above fixture file, the server is setup to respond with a failure message the first time it is accessed with a GET on /test, a redirect the first time it is accessed with a POST on /test/post and a success message the second time it is accessed with a GET on /test.
In cucumber, using the stuff I’ve mentioned above, we can now do cool stuff like configuring fixtures for each scenario. I use something like this to setup my fixtures:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 |
require 'net/http' module FixtureHelpers def load_fixture(fixture) Net::HTTP.get(::TEST_HOST, "/_fixtures/load/#{fixture}", ::TEST_PORT) end def clear_fixtures Net::HTTP.get(::TEST_HOST, '/_fixtures/clear', ::TEST_PORT) end end World(FixtureHelpers) Before('@notauthed') do load_fixture(:notauthed) end Before('@authed') do load_fixture(:authed) end After do clear_fixtures end |
I slap that in a .rb file in my features/support directory and I can now use the @authed and @notauthed tags to load the appropriate fixtures into the server, like in the following feature:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 |
Feature: Login page In order to access the site As a customer I want to be able to login @notauthed Scenario: Should get redirected to the login page when not logged in Given I am on the home page Then I should be on the login page @notauthed Scenario: Login without any details Given I am on the login page When I press "Login »" Then the "Username" field should have errors And the "Password" field should have errors And I should see "Whoops!" And I should see "Your login details could not be authenticated" @notauthed Scenario: Login with bad details Given I am on the login page And I fill in "Username" with "invalid-user" And I fill in "Password" with "invalid-pass" When I press "Login »" Then the "Username" field should have errors And the "Password" field should have errors And I should see "Whoops!" And I should see "Your login details could not be authenticated" @notauthed @authed Scenario: Login with valid details Given I am on the login page And I fill in "Username" with "valid-user" And I fill in "Password" with "valid-pass" When I press "Login »" Then I should be on the home page |
You’ll notice that I can do cool things like load multiple fixtures by specifying multiple tags. The fixtures get loaded in the order that that tags appear so I can create fairly complex fixture scenarios from simple individual fixtures.
All in all I’ve found it a reasonably nice simple way of being able to test client-side only apps that drive APIs without having to hit the actual API server. Obviously if you are developing your own API server as well, you should be writing tests for the API server in addition to the client, but this way to can keep them nicely de-coupled in your tests so you don’t have to hit live data, which based on your project might be impossible to test against anyway.
Like this:
Like Loading...
Damn, I could not be more excited about Numenera. For those of you unaware, 
The other projects I’ve been working on are 
Lorcian are discussing. 
). There are many stories I want to finish and that a few people on Protagonize want to see the conclusions of, so I need to get back on the saddle. I had said to myself last year I would try and seriously pursue publication, but that hasn’t happened. I’m hoping this year, now I have more time due to driving to work, things will be different, but if I’m entirely honest with myself, I doubt I’ll put the required time or effort into making a serious attempt. Not only is it a lot of work, it’s quite intimidating. I do want to finish at least one novel though by the end of the year and hopefully get into the meat of editing it with an eye to publish it independently, for kicks, since I’m not sure I’m even interested in mainstream publication or actually making a living from writing.
Check out our interview with Monte on the game! http://www.puresophistry.com/2012/09/16/kickstarter-profile-numenera-bringing-innovation-to-table-top-rpgs/