Dear Search: Reading Between the Lines of Search Data

Welcome to the first Dear Search article, an occasional series where the DigitalGov Search team addresses common search questions.

Dear Search,

Right now, I am building up user research services that can be offered to product owners on a regular or as-needed basis. So, being able to look at search trends and offer advice to teams seems like a good start.

When you are trying to understand user behavior based on search data, how do you typically go about it? Do you look for terms, find trends, and then talk to the teams involved in those words and see if the content matches the search terms?

Or do you look at it from a higher level and see if the terms fit into a content strategy?

Is there more to search data than just the terms people search for – does it matter which page people are on when they search? Or can you tell how many times a person is searching to see if they like / don’t like the results?

And finally, do you have a formal method of evaluating the search experience that you can share?

Best,

UX Vexed

Dear Vexed,

There is a natural progression in thinking about search trends and user experience as they fit within the overall content strategy of a website.

1. What terms are people searching for?
2. What are they clicking on after they run those searches?
3. Why am I seeing a click-through rate over 100% on some of these searches?
4. Wait, where were these people when they ran these searches?
5. Wow, there are so many ways people are searching for the same kind of thing using different words, how can I look at this more holistically?

And a question that can branch off of any of these stages is where to make improvements, whether any updates are needed to the search configuration, the content on the site, the structure of the site, or all of these working together.

There is a line of thinking that says if a site is designed optimally, then search is not necessary. However, usability research has shown that about half of people start with search, and a good number of the people who start out browsing resort to search if their sensemaking process doesn’t jive with the site’s architecture. Besides, providing multiple paths of discovery has been a core tenet of information science for over 125 years.

DigitalGov Search, like any search tool worth its salt, provides robust search analytics. We give views into to the top queries your site gets and the top pages clicked on out of your results pages. We also provide click-through rates for all queries: for all the times a query was run, how many clicks did that search results page get? You can see this for the top queries, or you can search for a particular term and see all the queries that contain that term.

A really low click-through rate, or one well above 100%, is a sign that some sort of change would be helpful to searchers, but what that change is will depend on what they were looking for.

Does a particular page need to be boosted in the search results, or do you need to respond to queries on a jargony keyword? Then setting up a Best Bet in our Admin Center may be your answer. Or does the page need a plain language rewrite? A click-through rate over 100% means that searchers are, on average, clicking on the same results page a bunch of times, finding results that look promising, but then the content doesn’t end up serving their needs. This kind of bouncing is called pogo sticking – fun for kids, frustrating for searchers. It could indicate your site has a lot of similar-looking content on a given topic that could benefit from disambiguating the page titles.

The next important question is: Where were the searchers when they ran a particular query? I’ve seen cases where a query was run directly from a page that, from the look of the url, should have contained the information the person was looking for. In our referrers report you can see the pages where the most queries were run on your site, and from there you can see all the queries that were run on any particular page. This is a gold mine for determining whether it’s your search configuration or your site content – or structure! – that needs tweaking.

Because the relationships between all these data points can be complex, we gave a webinar in December 2015 on analytics. Michelle Chronister, the former user experience team lead for USAgov, described how that team looks at search data for trends that can guide USAgov’s content strategy. They group query terms into topics, and analyze at the topic level. A webinar recap is available that includes a video recording, slide deck, and data tracking/report template as well as an infographic of USAgov’s 2015 search traffic.

By looking at trends for individual queries, query topics, locations, and destinations, you’ve begun the journey towards using data to better inform your search configuration and content strategy. We also recommend looking at your website data from the Digital Analytics Program or any other analytics software you have running on your site.

If you have further questions, Vexed, you can always email the DigitalGov Search team.

Happy searching,

Dawn

Program Manager, DigitalGov Search