I saw a purple cow

Just before Christmas we got some unsuspecting members of the public in to test parts of our website. We gave them some tasks to complete like commenting on a planning application, applying for a job and ordering a mattress to be picked up.

What I found fascinating was the different ways each person navigated the site. One used only the site navigation, one only used the A-Z, one relied on the site search and the other used a combination of everything, including the breadcrumb which I didn’t think anyone used. And that’s just it – we think we know how people will use council websites but you really don’t have a clue until you watch them. We keep trying Camtasia screen capture software but when you’re online it runs painfully slowly so we film the screen and just watch it back.

That test was ungainly – the scenarios took about an hour to complete. No member of the public would realistically sit on a council website for an hour and the testers admitted that had they been at home they would probably have picked up the phone to the call centre but because they’d been asked to test the site they were determined to see it through.

The next round we made the tasks to be tested much quicker and we made it around a single issue we were developing – our new bin collection calendar. We also added some customer journey mapping at the end which has turned out to be an absolute gem and something that shows you instantly, not only where the faults are in a process but also the good bits.

We did another round last week looking at reporting road and lighting faults and although I haven’t written the full report yet, the customer journey mapping has shown us the parts of the process that need ‘fixed’. The point of customer journey mapping is to capture people’s emotions – how they feel at each stage of a process. It doesn’t have to be web-based, it can be anything with a process – arranging a civil wedding ceremony, applying for council tax benefit, reporting a pothole, the list is endless.

Customer journey mapping sounds complicated and expensive but it really is low-fi. There’s no software involved, no expensive training and no special equipment. All you need is some squared paper, a pencil and a ruler. Create a graph where the X axis contains each stage of the process. The Y axis ranges from 0 to 10, where 5 is neutral – draw a line right across the graph at five. Ask the user how they felt at each stage of the process and plot it on your graph – anything below the neutral line they felt frustrated, sad, angry, bored etc, and anything above the line they felt satisfied, happy, chuffed, excited. Join the dots and you’ll see clearly where your process is causing problems.

Anything below the line you want to change to get above the line. Get enough of the same points below the line and it’ll be obvious things have to change.

Sometimes there will be things above the line that aren’t necessary for the process but your public like them. These are known as purple cows and you should keep them, even if they aren’t essential to you – they may even offset some bad points in the process you can’t fix for one reason or another.

If you need to use the graphs for a report you can tidy the graphs up in some kind of graphics programme and add smiley faces, little hearts and even purple cows if you like but I wouldn’t go spending a fortune on any fancy software because you’d still be best doing the original on a piece of paper while you sit next to your tester having a chat at the end of the session.

We’re now starting a rolling programme of user testing both on existing bits of the site we want to improve, or before and after comparisons of things we are developing. None of it is expensive, none of it involves consultants or external agencies and it doesn’t take up much time.

Top tips

  • Keep the scenarios short. It will surprise you how long it takes some people to complete a process you think takes five minutes. We forget we know the website inside out and others don’t.
  • Time how long it takes people to complete a task – if it’s longer/more complex than a phone call why would they use the web?
  • Use a mixture of the public and internal staff who don’t work on the website. On average 70% of a council’s staff live in the area – make use of them.
  • If you get email feedback about your site ask those who have contributed if they’d like to take part in user testing – create a directory of participants.
  • Use ready made groups such as residents associations, tenants forums, licensing groups, community councils, young carers groups.
  • Film the test so you can see exactly how they navigate the site – there will be surprises every time!
  • Do customer journey mapping at the end of the test
  • Offer to do customer journey mapping of the same process at your customer call centre and face-to-face one-stop shop. The results can be used to highlight common failures, to compare channels or even be used for business process re-engineering
  • Before the test starts tell the testers that you’re not testing them, you’re testing the process
  • Be impartial
  • Explain to the testers  that you can’t help them during the test
  • Ask them to note on the scenarios if they would have given up on the website and picked up the phone at any point

And that’s about it. I really enjoy these exercises and I always learn loads, even just talking to the testers about what they use the site for, what they’d like to see on the site and what other sites they use.

Please let me know if you want to know anything else or see any reports from the tests we’ve done so far.



  1. Amanda Burgauer (@amshru) · August 29, 2012

    I was amazed the other day when I called the Council, asked for info and was told yes, it was there, but you had to “know how to find it”. Mostly focused around the mapping tool, which once you’re into it, is awesome, but it’s not intuitive to find it.

    • carolynemitchell · August 29, 2012

      Ahh, the mapping tool . . . Like you say, for me maps are great once I get my head around them but some people just don’t like using them. As your comment proves if you’re quicker phoning why would you continue to struggle with a website? I’ll DM you my number for future SLC web questions and comments if you like.

  2. Pingback: I saw a purple cow | weeklyblogclub
  3. Pingback: Tech, culture, caring – and a purple cow | weeklyblogclub
  4. Ben Darlington · September 3, 2012

    We started focussing on the customer journey visualisation problem about a year ago and developed an approach which maps real customer journeys in real-time – not just ‘test’ journeys.

    The principle is the same – pages in rows, journeys in columns – but the visualisation is more advanced and, of course, because it’s real data and real journeys, it’s accurate.

    A coloured box in the intersection shows when a journey touched a page: the size of the box shows the time spent on the page and the colour shows at what point in the individual journey the page view occurred (redder/hotter = nearer the centre event)

    The visualisation also define a filter ‘event’ and puts that at the centre of the visualisation, showing before-event and after-event page views for every journey. That means you can focus on an event you really want to understand (a specific page view, a transaction completed or an interest in a particular service area) and then visualise all the relevant journeys to and from that event. Or, if you want, you can just map journeys around a specific point in time.

    We can get up to 400 visualisations on any one page and the results are startling – because it’s all based on real data of real citizens doing real things on your website, you get a much better understanding of real behaviours.

    If you’re interested, a sample, anonymous visualisation (real data but page names change to conceal which local government website traffic is being visualised) is available here:

    Click to access LGN_SampleJourneyVisualisation.pdf

    • carolynemitchell · September 5, 2012

      Sorry it’s taken so long to reply – firewall at work!
      I will certainly take a closer look at this – thank you for sharing 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s