{
    "version" : "https://jsonfeed.org/version/1",
    "content" : "news",
    "type" : "single",
    "title" : "Product lessons from the front lines of COVID-19 civic tech response |Digital.gov",
    "description": "Product lessons from the front lines of COVID-19 civic tech response",
    "home_page_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/","feed_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2020/10/16/product-lessons-from-the-front-lines-of-covid-19-civic-tech-response/index.json","item" : [
    {"title" :"Product lessons from the front lines of COVID-19 civic tech response","summary" : "Two Presidential Innovation Fellows share how we can build digital services quickly without compromising quality for the public in times of crisis.","date" : "2020-10-16T00:00:00-05:00","date_modified" : "2025-01-27T19:42:55-05:00","authors" : {"likhitha-patha" : "Likhitha Patha","dr-kaeli-yuen" : "Dr. Kaeli Yuen"},"topics" : {
        
            "communication" : "Communication",
            "product-and-project-management" : "Product and project management",
            "research" : "Research"
            },"primary_image" : { "uid" : "hero-image", "alt" :
  "Headshots of Kaeli Yuen and Likhitha Patha, who are both smiling. The quote reads, We wanted to step up and serve people in a time of need, so we asked ourselves, how do we build useful digital services quickly without compromising the quality and value to the public?", "width" :
  "960", "height" :
  "540", "credit" :
  "", "caption" :
  "", "format" :
  "png" },"branch" : "bc-archive-content-3",
      "filename" :"2020-10-16-product-lessons-from-the-front-lines-of-covid-19-civic-tech-response.md",
      
      "filepath" :"news/2020/10/2020-10-16-product-lessons-from-the-front-lines-of-covid-19-civic-tech-response.md",
      "filepathURL" :"https://github.com/GSA/digitalgov.gov/blob/bc-archive-content-3/content/news/2020/10/2020-10-16-product-lessons-from-the-front-lines-of-covid-19-civic-tech-response.md",
      "editpathURL" :"https://github.com/GSA/digitalgov.gov/edit/bc-archive-content-3/content/news/2020/10/2020-10-16-product-lessons-from-the-front-lines-of-covid-19-civic-tech-response.md","slug" : "product-lessons-from-the-front-lines-of-covid-19-civic-tech-response","url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2020/10/16/product-lessons-from-the-front-lines-of-covid-19-civic-tech-response/","weight" : "1","content" :"\n\n\n\n\n\n\n\u003cdiv class=\"image\"\u003e\n  \u003cimg\n        src=\"https://s3.amazonaws.com/digitalgov/hero-image.png\"alt=\"Headshots of Kaeli Yuen and Likhitha Patha, who are both smiling. The quote reads, We wanted to step up and serve people in a time of need, so we asked ourselves, how do we build useful digital services quickly without compromising the quality and value to the public?\"/\u003e\u003c/div\u003e\n\n\n\u003cp\u003e\u003cstrong\u003eLikhitha Patha\u003c/strong\u003e: When Kaeli Yuen and I joined the federal government as part of the 2020 cohort of \u003ca href=\"https://pif.gov/\"\u003ePresidential Innovation Fellows\u003c/a\u003e (PIF), we did not anticipate a global pandemic on the horizon. For us, this unprecedented time turned out to be a call to step up and serve the American people in a time of need. We prioritized helping agencies respond to COVID-19 by building new digital services that addressed public needs. I was detailed to the Department of Health and Human Services (HHS) and Health Resources and Services Administration (HRSA) where I led the launch of \u003ca href=\"https://telehealth.hhs.gov/\"\u003etelehealth.hhs.gov\u003c/a\u003e, a website that centralizes telehealth policy updates and resources for health care providers and patients. Kaeli was detailed to the Department of Veterans Affairs (VA), where she led the launch of the \u003ca href=\"https://www.va.gov/coronavirus-chatbot/\"\u003eVA coronavirus chatbot\u003c/a\u003e to speed Veteran access to information on the impact of COVID-19 to their VA benefits and services, and the \u003ca href=\"https://www.va.gov/covid19screen/\"\u003eVA COVID-19 screener tool\u003c/a\u003e to help assess COVID-19 risk factors at VA medical centers’ building entrances.\u003c/p\u003e\n\u003cp\u003eDeveloping products during this time of crisis was uniquely challenging for a couple of reasons:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eEvery problem we came across was urgent in its own sense\u003c/li\u003e\n\u003cli\u003eVisibility and scrutiny on work delivered was higher than usual, which also meant that the appetite for risks and tolerance for hiccups was very low\u003c/li\u003e\n\u003cli\u003eWe didn’t have the luxury of time on our side and results needed to be delivered “yesterday”\u003c/li\u003e\n\u003cli\u003eWe were working 100% remote for the first time while coming together as new teams\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThese constraints led us to ask ourselves:\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHow do we accelerate innovation and cultivate a culture of rapid prototyping and iteration in the government? More importantly, how do we build useful digital services quickly without compromising the quality and value to the public in a time of crisis?\u003c/strong\u003e\u003c/p\u003e\n\u003cp\u003eDespite intense timelines, we ended up launching new digital products and services from ideation to production in less than two weeks. Looking back, lean and agile product development proved to be more essential than ever in these constraints. Specifically, Kaeli and I arrived at the following lessons that product managers and technologists can apply to effectively build products and services and bring them to production quickly. We go into details with anecdotes from our experiences below but here’s the short story:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eStart by digging into the problem space\u003c/li\u003e\n\u003cli\u003eBuild a strong coalition of support and partnerships — especially with those on the front lines\u003c/li\u003e\n\u003cli\u003eBuild your “2-pizza” team to stay nimble\u003c/li\u003e\n\u003cli\u003eDon’t let perfect be the enemy of the good\u003c/li\u003e\n\u003c/ol\u003e\n\u003ch2 id=\"1-start-by-digging-into-the-problem-space\"\u003e1. Start by digging into the problem space\u003c/h2\u003e\n\n\n\u003cdiv class=\"image image-right\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/likhitha_w200.png\"  alt='Likhitha Patha stands smiling in a government building. The United States flag hangs on the wall in the background.'\n    srcset=\"https://s3.amazonaws.com/digitalgov/likhitha_bu.jpg 48w,https://s3.amazonaws.com/digitalgov/likhitha_w800.png 800w,https://s3.amazonaws.com/digitalgov/likhitha_w400.png 400w,https://s3.amazonaws.com/digitalgov/likhitha_w200.png 200w\"\n    sizes=\"(max-width: 600px) 40vw, 400px\"\n  /\u003e\u003c/div\u003e\n\n\n\u003cp\u003e\u003cstrong\u003eLikhitha\u003c/strong\u003e: When I initially started working on an effort to accelerate telehealth adoption and was still early in the exploration phase, I heard several ideas and wish lists spanning from wanting to build two-sided marketplaces to automatic triage bots from invested stakeholders. It was tempting to pick something that a highly influential stakeholder was asking for or something involving a latest tech trend because it was exciting to work on. However, taking a pause from reviewing/prioritizing solutions presented to me and instead investing time to dive deep into the problem space paid off. We started by talking to doctors and patients on the front lines to understand broader problems they’re dealing with to deliver and receive care. After validating a few hypotheses with user testing, we learned that the crux of the problem that the government was positioned to solve was providing easy access to trusted \u003ca href=\"https://plainlanguage.gov/\"\u003eplain-language guidance\u003c/a\u003e on evolving telehealth policies. This narrowed down the problem to “need for a centralized place to get plain language telehealth policy guidance for health care providers, and scale access to trusted information for the public.” And that’s exactly what we did with \u003ca href=\"https://telehealth.hhs.gov/\"\u003etelehealth.hhs.gov\u003c/a\u003e — it provides trusted, plain language resources on telehealth (telemedicine) to patients and healthcare professionals.\u003c/p\u003e\n\n\n\n\n\n\n\n\u003cdiv class=\"image\"\u003e\n  \u003cimg\n        src=\"https://s3.amazonaws.com/digitalgov/telehealth-website.png\"alt=\"The homepage of telehealth.hhs.gov. White text on a dark blue background reads, Telehealth: Health care from the safety of our homes. Illustrated in bright blue, a blue hand is holding a phone and is on a video conference with a health care professional.\"/\u003e\u003c/div\u003e\n\n\n\n\n\u003cdiv class=\"image image-right\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/chatbot-phone_w200.png\"  alt='An illustrated mobile phone has the VA coronavirus chatbot webpage open. The chatbot has 2 questions populating the screen.'\n    srcset=\"https://s3.amazonaws.com/digitalgov/chatbot-phone_bu.jpg 48w,https://s3.amazonaws.com/digitalgov/chatbot-phone_w400.png 400w,https://s3.amazonaws.com/digitalgov/chatbot-phone_w200.png 200w\"\n    sizes=\"(max-width: 600px) 40vw, 400px\"\n  /\u003e\u003c/div\u003e\n\n\n\u003cp\u003e\u003cstrong\u003eKaeli\u003c/strong\u003e: In early March, there was a big surge in VA call center volume due to coronavirus-related questions from Veterans, leading to frustrating hold times for Veterans, staffing challenges for VA, and delayed access to clinical care for callers with urgent healthcare questions. Given multiple challenges, it was challenging for our VA team to prioritize where to focus our efforts, and tempting to jump at the many solutions being offered both from inside VA and from external third parties. However, taking time to understand the problem before jumping to solutions is counterintuitively more important in a crisis, given that there isn’t time to lose on solving the wrong problem.\u003c/p\u003e\n\u003cp\u003eBy spending time on user research, including deep dives with VA call centers and multiple rounds of interviews with Veterans, our team at VA Office of the Chief Technology Officer (VA CTO) learned that the majority of calls to VA were about a handful of the same, VA-specific frequently asked questions related to COVID-19 (e.g., “how can I switch my scheduled VA healthcare appointments to telehealth appointments?”). We developed the VA coronavirus chatbot as an automated way to deliver this information to Veterans without requiring a phone call. During peak usage, the VA coronavirus chatbot handled twice as many COVID-19 related queries as the VA’s national call center.\u003c/p\u003e\n\u003ch2 id=\"2-build-a-strong-coalition-of-support-and-partnershipsespecially-with-those-on-the-front-lines\"\u003e2. Build a strong coalition of support and partnerships—especially with those on the front lines\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eLikhitha\u003c/strong\u003e: Given the short amount of time to address the identified challenge, we couldn’t wait for all of our insights to be learned solely via user research. We had to diversify our sources of insight by getting connected to entities directly plugged in with frontline workers. By sharing our understanding of the problem space, we were able to seek people on the front lines who can partner with us and support us. We got creative and tapped into second-degree learnings by identifying non-profits and academic entities who work regularly on problems faced by healthcare providers and patients. Overall, our coalition of support and partnerships included stakeholders for buy-in, leaders for approval, and partners for collaboration. What makes a coalition strong is how you earn trust and build relationships, as that is crucial for influencing without authority. We had to also accelerate internal processes to ship faster, and having this strong support from leaders and stakeholders from involved agencies enabled us to cut through a lot of red tape.\u003c/p\u003e\n\n\n\u003cdiv class=\"image image-right\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/kaeli_w200.png\"  alt='Dr. Kaeli Yuen stands smiling outdoors. She is on the steps of a government building.'\n    srcset=\"https://s3.amazonaws.com/digitalgov/kaeli_bu.jpg 48w,https://s3.amazonaws.com/digitalgov/kaeli_w1200.png 1200w,https://s3.amazonaws.com/digitalgov/kaeli_w800.png 800w,https://s3.amazonaws.com/digitalgov/kaeli_w400.png 400w,https://s3.amazonaws.com/digitalgov/kaeli_w200.png 200w\"\n    sizes=\"(max-width: 600px) 40vw, 400px\"\n  /\u003e\u003c/div\u003e\n\n\n\u003cp\u003e\u003cstrong\u003eKaeli\u003c/strong\u003e: As part of VA’s central office, our team relies on strong partnerships with our colleagues in the field to help us identify and solve the most important problems, especially in a crisis. For example, the VA CTO team’s partnership with the VA Medical Center is Durham, North Carolina surfaced a problem we were well-equipped to help solve: when VA medical centers began to reopen their non-emergency services (e.g., outpatient visits), they began implementing pre-entry screenings for COVID-19 symptoms and risk factors for all patients and employees. However, these manual in-person screenings created long lines and unnecessary crowding at entrances. The VA Medical Center in Durham had begun piloting a digital screening tool, and came to our team for help with improving and scaling the tool. This time crunch made the most of our technical skills and our network. While the team at VA CTO improved the usability and dependability of the tool, the field medical center shared a clear use case and call to action with their peers at medical centers across the country. After pilot success in North Carolina, the \u003ca href=\"https://www.va.gov/covid19screen/\"\u003eVA COVID-19 screener\u003c/a\u003e is being used by VA medical centers across the country, and has provided over 2 million screening results.\u003c/p\u003e\n\u003ch2 id=\"3-build-your-2-pizza-team-to-stay-nimble\"\u003e3. Build your “2-pizza team\u0026quot; to stay nimble\u003c/h2\u003e\n\u003cp\u003e\u003cstrong\u003eLikhitha\u003c/strong\u003e: In order to move fast and deliver quickly, building a small, lean team of empowered contributors is essential. The “2-pizza team” rule is an infamous rule of thumb in the technology world, and it\u0026rsquo;s a handy measure for this — if the team can’t be fed with two pizzas, then it’s too big. The rationale is that as your team size grows, so do the number of links that require managing. Keeping the immediate team small allows for more effective communication, coordination and execution, and staves off symptoms of inefficiency like “too many cooks in the kitchen.” In order to keep the team effective, ensuring there is transparency, and establishing clear roles and responsibilities was also critical. Our immediate team of eight used a number of virtual collaboration tools to consistently workshop and communicate no matter our location; all of which also played a big role in maintaining transparency. Establishing who does what by when and identifying owners, reviewers, collaborators, and approvers upfront kept things clear as we moved forward.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eKaeli\u003c/strong\u003e: Key to the speedy delivery (the VA COVID-19 screener was launched in just 12 days!) of multiple COVID-19-related tools from VA has been not only the “2 pizza” team size but also the fact that we had those tech teams in-house. VA has a dedicated tech team (the Digital Experience Product Office, made of designers, content strategists, engineers, product managers) focused on delivering digital products with a modern, shared VA.gov platform. Having a team like this on hand means you don’t have to worry about contract management, waiting for task orders, onboarding new teams, negotiating scope, or waiting for access to systems. Plus, the entrepreneurial mindset of the team was key for quick delivery. The heavy-lift required to get this infrastructure and in-house tech team in place turned out to be a great investment that allowed the VA CTO team to respond rapidly to the urgent needs of Veterans.\u003c/p\u003e\n\u003ch2 id=\"4-dont-let-perfect-be-the-enemy-of-the-good\"\u003e4. Don’t let perfect be the enemy of the good\u003c/h2\u003e\n\n\n\n\n\n\n\u003cdiv class=\"quote-block \"\u003e\n    \u003cblockquote\u003e\n      \u003cspan class=\"quote-block__quotation-mark\"\u003e“\u003c/span\u003e\n      If you’re not embarrassed by the first version of your product, you’ve launched too late.\n      \u003cspan class=\"quote-block__quotation-mark\"\u003e”\u003c/span\u003e\u003ccite\u003e— Reid Hoffman, an American entrepreneur\u003c/cite\u003e\u003c/blockquote\u003e\n  \u003c/div\u003e\n\u003cp\u003e\u003cstrong\u003eLikhitha\u003c/strong\u003e: It’s easy to get hung up on wanting to ship a perfect product from the get-go, be it for precision or for risk aversion. In the wise words of Reid Hoffman, an American entrepreneur, “If you’re not embarrassed by the first version of your product, you’ve launched too late.” By waiting to ship the perfect product, we’d be wasting time instead of actually getting the product in the hands of real users through which we can further learn and iterate. Creating a culture for iteration rather than striving for perfection was critical — this mindset had to be adopted by our immediate team and leaders supporting us. We knew going in that what we put out on Day 1 wouldn’t be perfect; we prepared ourselves to iterate quickly and employed mechanisms where we could collect and incorporate user feedback. As we kicked off the effort, we declared a set of guiding principles and “ship-it must-haves” to align towards this mindset. Having these criteria pre-established also helped us stay focused on priorities and stick to our immediate goals. Our “ship-it must-haves” were the following:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003eWe have validated user needs and have reasonable confidence that the right content is in place.\u003c/li\u003e\n\u003cli\u003eWe meet HHS and HRSA compliance and launch requirements.\u003c/li\u003e\n\u003cli\u003eWe adhere to the latest web design standards. In order to create a beautiful and modern website, we leveraged the \u003ca href=\"https://designsystem.digital.gov/\"\u003eU.S. Web Design System\u003c/a\u003e (USWDS) to easily adopt mobile-friendly design standards.\u003c/li\u003e\n\u003cli\u003eWebsite content including links are reviewed and signed off by source-of-truth owners.\u003c/li\u003e\n\u003cli\u003eWebsite can withstand high-traffic, and we are operationally ready.\u003c/li\u003e\n\u003cli\u003eWe ensure there is a way for website users to send us feedback.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003e\u003cstrong\u003eKaeli\u003c/strong\u003e: An important part of this piece of advice is “don’t reinvent the wheel.” Even though we had the in-house tech team available who could have built the VA coronavirus chatbot themselves, we saved precious time by starting with a commercial off-the-shelf chatbot framework. There were some minor trade-offs in customizability, but it got the job done and addressed the pressing need to get information to Veterans quickly.\u003c/p\u003e\n\u003chr\u003e\n\u003cp\u003eAre you passionate about the civic impact of human-centered design and product development? Interested in using your expertise for public good? Learn more about PIF \u003ca href=\"https://pif.gov/\"\u003eat our website\u003c/a\u003e, and follow us on \u003ca href=\"https://www.twitter.com/PIFgov\"\u003eTwitter\u003c/a\u003e and \u003ca href=\"https://www.linkedin.com/company/white-house-presidential-innovation-fellows/\"\u003eLinkedIn\u003c/a\u003e to find the latest work from our fellows.\u003c/p\u003e\n"}
  ]
}