Measure Performance to Make Continuous Improvements

Welcome to This Week’s IDEA, where we talk about one essential topic around 21st Century IDEA and share resources and tools that you can use to start making small, incremental changes to your websites and digital services.

One of the design principles of the U.S. Web Design System (USWDS) is to listen, or evaluate and improve your product by listening to your audience and learning from what you hear.

Continuous feedback drives continuous improvement. We should measure customer experience—determine how well what we’ve built is working for our audience—at every stage. And we should listen to what people say, and observe how they interact with our products or services, through direct observation and analytics data.

If we’re not listening, we’re not learning.

Below, we share a case study from the U.S. Department of Health and Human Services (HHS) on how they’re listening through their performance measurement program.

HHS Performance Measurement Program

The digital performance measurement program at HHS has changed the way the agency manages its sites and content, driving better understanding of customers and their needs. Performance measures guide how they allocate resources, and inform their vision for the future, including where they innovate. They are committed to listening to users, and gather both quantitative and qualitative data to empower teams across HHS to have deeper dialogues about service delivery. Listening to customer feedback has changed how HHS communicates with, and provides services to, the public.

To determine key performance indicators (KPIs), HHS first identifies their goals, then they tailor their KPIs to reflect the value they aim to bring to users, and what they measure against. They’ve adopted a few universal KPIs, which they’ve dubbed “Satisfaction KPIs.” These are based on 21st Century IDEA and the USWDS design principles, and include page load time, readability, search satisfaction, and search engine optimization (SEO).

At the same time, they recognize that every site is different, with different business goals. As an example, a metric such as bounce rate would not be a good common KPI when comparing sites such as time.gov and IRS.gov. Time.gov is expected to have a relatively high bounce rate (a visitor checks the time and leaves the site), whereas IRS.gov is expected to have a much lower bounce rate (visitors browse across many pages, checking tax information, changes to tax laws, etc.). Because these two sites have very different goals, KPIs must be tailored to fit each site.

However, when sites have goals that align, some KPIs could apply across the board. Because many sites across the HHS network focus on information dissemination, many will use common KPIs such as pageviews and time on page to measure performance or success. They can customize these, based on targeted user actions such as downloading a form or clicking a link to an external site.

Customer Satisfaction Surveys

HHS uses a common survey platform, and runs surveys on all of their websites. Feedback is collected at both the site and page level, and they have developed a common baseline to allow comparison around similar topics such as, Was this page helpful? They also incorporate additional questions based on current customer experience strategies and specific experiences, and leverage different triggering mechanisms to reach users at different interaction points along the journey, based on agency goals and user paths. Surveys may appear as an on-page survey, a feedback slider, or a popover survey.

To analyze and benchmark customer satisfaction data across sites, HHS uses the Was this page helpful? metric as the key benchmark for cross-page and cross-site satisfaction comparison.

Customer feedback and satisfaction data are shared internally with both employees and managers, primarily through dashboards that consolidate data into easily digestible information. They also host regular meetings to brief staff on high-level trends, and dive deeper into specific inquiries. They don’t currently share customer satisfaction data publicly, but are considering doing so in the future.

Actionable Data

“Actionable” is key for any analytics team. HHS has the capability to track data from a variety of sources, all tied to clearly defined actions. Their goal is to make HHS.gov a flagship government website. When they flag a spike or drop, they first look at the quantitative metrics that may be related to, or impacted by, the change. They also use qualitative survey data to complete the story, to understand the underlying issue. They analyze both qualitative and quantitative data to determine how to address any underlying issues that impact users, such as improving SEO language to ensure people are finding the right information, or improving information architecture to deliver a better customer experience.

Insights

Recently, the most surprising finding for HHS has been the flip in user searches from coronavirus to covid. Since the pandemic hit the U.S. in February 2020, coronavirus was the top search term across HHS.gov, as well as across the web. With this information, HHS made the decision to use coronavirus in site copy (as opposed to covid) to improve search engine rankings for the more popular term. This remained a constant through March, April, and even into May, but toward the end of May they began to notice a shift. By the beginning of June, covid searches had surpassed coronavirus searches, not only across HHS.gov search, but also on external search engines. By monitoring this data on a consistent basis, HHS was able to make easy adjustments to their content and SEO, to ensure they continued to deliver relevant information based on the most popular search terms.

Data has also played a key role in improving and informing the coronavirus content strategy on HHS.gov. Over the past several months, HHS played a major role in the distribution of coronavirus-related information to citizens and healthcare providers. With news and guidance in the early days of the pandemic often changing several times a day, they used data to identify content gaps and areas of confusion, and worked to ensure visitors could easily find and understand important information.

Governance

The HHS Digital Council consists of individuals from across the department who worked together to develop a digital strategy to define and coordinate their digital communications. The council works together to outline the way ahead for HHS as a whole, and develops the best practices to carry out their mission of reaching the American public. While everyone on the council works in different parts of HHS, each with their own mission and target audiences, they were able to find commonality and work towards shared goals.

Their current challenge, along with many others in government right now, is adapting their roadmap to tackle their new role in responding to the pandemic. However, they acknowledge that COVID-19 has helped to highlight areas of improvement that they would not have anticipated prior to this experience, and so they continue to learn and respond proactively to new challenges as they arise.

Up Next

Attend any or all webinars in the Digital Analytics Program’s three-part analytics case study series:

• Part 1: Assessment—How to assess your analytics strategy to ensure you’re pulling impactful and relevant data (August 26)
• Part 2: Analysis—How to pull reporting and conduct analysis through the lens of your analytics strategy (September 24)
• Part 3: Action—How to take actions using the analysis and insights you created using DAP and Google Analytics (October 21)

Listen to Analytics Lessons From Canada.ca and learn how our northern neighbors approach analytics.

From the Field

“In the past two years, the HHS.gov team has vastly improved page load speeds, working to make sure that our pages load quickly on phones. Now, you’re going to start to see more of the visible changes that will continue to make it easier for our users to find what they are looking for. Today, 80 percent of HHS.gov visitors consistently report they found or partially found what they were looking for. Just five years ago, it was less than 20 percent. With improved understanding of user goals, needs, and behavior driving our design and functional decisions, we expect to see customer satisfaction continue to rise.”—via HHS.gov

Do you have a 21st Century IDEA-related comment or question? Or would you like to give a shout out to your colleagues? Send it to us at digitalgov@gsa.gov, and we’ll work to incorporate it into the next newsletter.