I saw a purple cow
Just before Christmas we got some unsuspecting members of the public in to test parts of our website. We gave them some tasks to complete like commenting on a planning application, applying for a job and ordering a mattress to be picked up.
What I found fascinating was the different ways each person navigated the site. One used only the site navigation, one only used the A-Z, one relied on the site search and the other used a combination of everything, including the breadcrumb which I didn’t think anyone used. And that’s just it – we think we know how people will use council websites but you really don’t have a clue until you watch them. We keep trying Camtasia screen capture software but when you’re online it runs painfully slowly so we film the screen and just watch it back.
That test was ungainly – the scenarios took about an hour to complete. No member of the public would realistically sit on a council website for an hour and the testers admitted that had they been at home they would probably have picked up the phone to the call centre but because they’d been asked to test the site they were determined to see it through.
The next round we made the tasks to be tested much quicker and we made it around a single issue we were developing – our new bin collection calendar. We also added some customer journey mapping at the end which has turned out to be an absolute gem and something that shows you instantly, not only where the faults are in a process but also the good bits.
We did another round last week looking at reporting road and lighting faults and although I haven’t written the full report yet, the customer journey mapping has shown us the parts of the process that need ‘fixed’. The point of customer journey mapping is to capture people’s emotions – how they feel at each stage of a process. It doesn’t have to be web-based, it can be anything with a process – arranging a civil wedding ceremony, applying for council tax benefit, reporting a pothole, the list is endless.
Customer journey mapping sounds complicated and expensive but it really is low-fi. There’s no software involved, no expensive training and no special equipment. All you need is some squared paper, a pencil and a ruler. Create a graph where the X axis contains each stage of the process. The Y axis ranges from 0 to 10, where 5 is neutral – draw a line right across the graph at five. Ask the user how they felt at each stage of the process and plot it on your graph – anything below the neutral line they felt frustrated, sad, angry, bored etc, and anything above the line they felt satisfied, happy, chuffed, excited. Join the dots and you’ll see clearly where your process is causing problems.
Anything below the line you want to change to get above the line. Get enough of the same points below the line and it’ll be obvious things have to change.
Sometimes there will be things above the line that aren’t necessary for the process but your public like them. These are known as purple cows and you should keep them, even if they aren’t essential to you – they may even offset some bad points in the process you can’t fix for one reason or another.
If you need to use the graphs for a report you can tidy the graphs up in some kind of graphics programme and add smiley faces, little hearts and even purple cows if you like but I wouldn’t go spending a fortune on any fancy software because you’d still be best doing the original on a piece of paper while you sit next to your tester having a chat at the end of the session.
We’re now starting a rolling programme of user testing both on existing bits of the site we want to improve, or before and after comparisons of things we are developing. None of it is expensive, none of it involves consultants or external agencies and it doesn’t take up much time.
Top tips
- Keep the scenarios short. It will surprise you how long it takes some people to complete a process you think takes five minutes. We forget we know the website inside out and others don’t.
- Time how long it takes people to complete a task – if it’s longer/more complex than a phone call why would they use the web?
- Use a mixture of the public and internal staff who don’t work on the website. On average 70% of a council’s staff live in the area – make use of them.
- If you get email feedback about your site ask those who have contributed if they’d like to take part in user testing – create a directory of participants.
- Use ready made groups such as residents associations, tenants forums, licensing groups, community councils, young carers groups.
- Film the test so you can see exactly how they navigate the site – there will be surprises every time!
- Do customer journey mapping at the end of the test
- Offer to do customer journey mapping of the same process at your customer call centre and face-to-face one-stop shop. The results can be used to highlight common failures, to compare channels or even be used for business process re-engineering
- Before the test starts tell the testers that you’re not testing them, you’re testing the process
- Be impartial
- Explain to the testers that you can’t help them during the test
- Ask them to note on the scenarios if they would have given up on the website and picked up the phone at any point
And that’s about it. I really enjoy these exercises and I always learn loads, even just talking to the testers about what they use the site for, what they’d like to see on the site and what other sites they use.
Please let me know if you want to know anything else or see any reports from the tests we’ve done so far.