{
    "version" : "https://jsonfeed.org/version/1",
    "content" : "news",
    "type" : "single",
    "title" : "Taking the Pulse of the Federal Government's Web Presence |Digital.gov",
    "description": "Taking the Pulse of the Federal Government's Web Presence",
    "home_page_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/","feed_url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2015/06/03/taking-the-pulse-of-the-federal-governments-web-presence/index.json","item" : [
    {"title" :"Taking the Pulse of the Federal Government's Web Presence","summary" : "The U.S. federal government is launching a new project to monitor how it’s doing at best practices on the Web.","date" : "2015-06-03T13:10:57-04:00","date_modified" : "2025-01-27T19:42:55-05:00","authors" : {"eric-mill" : "Eric Mill","julia-solorzano" : "Julia Solórzano","gray-brooks" : "Gray Brooks","john-tindel" : "John Tindel"},"topics" : {
        
            "analytics" : "Analytics",
            "product-and-project-management" : "Product and project management",
            "security" : "Security"
            },"branch" : "bc-archive-content-3",
      "filename" :"2015-06-03-taking-the-pulse-of-the-federal-governments-web-presence.md",
      
      "filepath" :"news/2015/06/2015-06-03-taking-the-pulse-of-the-federal-governments-web-presence.md",
      "filepathURL" :"https://github.com/GSA/digitalgov.gov/blob/bc-archive-content-3/content/news/2015/06/2015-06-03-taking-the-pulse-of-the-federal-governments-web-presence.md",
      "editpathURL" :"https://github.com/GSA/digitalgov.gov/edit/bc-archive-content-3/content/news/2015/06/2015-06-03-taking-the-pulse-of-the-federal-governments-web-presence.md","slug" : "taking-the-pulse-of-the-federal-governments-web-presence","url" : "/preview/gsa/digitalgov.gov/bc-archive-content-3/2015/06/03/taking-the-pulse-of-the-federal-governments-web-presence/","content" :"\u003cp\u003eThe U.S. federal government is launching a new project to monitor how it’s doing at best practices on the Web.\u003c/p\u003e\n\u003cp\u003eA sort of health monitor for the U.S. government’s websites, it’s called \u003ca href=\"https://pulse.cio.gov/\"\u003ePulse\u003c/a\u003e and you can find it at \u003ca href=\"https://pulse.cio.gov/\"\u003epulse.cio.gov\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-410-pulse.jpg\"\n    alt=\"600-x-410-pulse\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/\"\u003ePulse\u003c/a\u003e is a lightweight dashboard that uses the \u003ca href=\"https://github.com/GSA/data/blob/gh-pages/dotgov-domains/2015-03-15-federal.csv\"\u003eofficial .gov domain list\u003c/a\u003e to measure two things:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://pulse.cio.gov/analytics/domains/\"\u003eAnalytics\u003c/a\u003e: Whether federal executive branch domains are participating in the \u003ca href=\"https://digital.gov/guides/dap/\"\u003eDigital Analytics Program\u003c/a\u003e (DAP) that powers \u003ca href=\"https://analytics.usa.gov/\"\u003eanalytics.usa.gov\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://pulse.cio.gov/https/domains/\"\u003eHTTPS\u003c/a\u003e: Whether federal domains have deployed the \u003ca href=\"https://https.cio.gov/faq/\"\u003eHTTPS protocol\u003c/a\u003e, and how well they’ve done it.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThese two things are just a start—there are a lot of other important things worth measuring! It’s also important to note that Pulse is currently only measuring parent domains (e.g. agency.gov) and is \u003cem\u003enot\u003c/em\u003e measuring subdomains (e.g. portal.agency.gov).\u003c/p\u003e\n\u003cp\u003eThe project will hopefully expand over time to measure more best practices and more websites. In the meantime, Pulse is a commitment by the U.S. government to build a world-class analytics program and to transition entirely to HTTPS.\u003c/p\u003e\n\u003ch2 id=\"background\"\u003eBackground\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/analytics/agencies/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-170-analytics-agencies.jpg\"\n    alt=\"600-x-170-analytics-agencies\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003ePulse is a collaboration between 18F and the \u003ca href=\"http://www.gsa.gov/portal/content/104550\"\u003eOffice of Government-wide Policy\u003c/a\u003e (OGP).\u003c/p\u003e\n\u003cp\u003eLike 18F, OGP is an office of the U.S. General Services Administration. Among many other things, OGP operates the \u003ca href=\"https://www.dotgov.gov/\"\u003e.gov domain registry\u003c/a\u003e and the \u003ca href=\"https://cio.gov/\"\u003eCIO Council\u003c/a\u003e, an interagency forum of Chief Information Officers.\u003c/p\u003e\n\u003cp\u003e18F previously partnered with OGP in December to \u003ca href=\"https://18f.gsa.gov/2014/12/18/a-complete-list-of-gov-domains/\"\u003erelease the complete .gov domain list\u003c/a\u003e. Since then, 18F has worked with the Digital Analytics Program \u003ca href=\"https://18f.gsa.gov/2015/03/19/how-we-built-analytics-usa-gov/\"\u003eto build analytics.usa.gov\u003c/a\u003e, and has coordinated with a number of agencies to \u003ca href=\"https://18f.gsa.gov/2015/02/09/the-first-gov-domains-hardcoded-into-your-browser-as-all-https/\"\u003estrengthen HTTPS for federal .gov domains\u003c/a\u003e. We’re deeply gratified that we’ve had the opportunity to work with OGP to create a platform that continues this momentum.\u003c/p\u003e\n\u003ch2 id=\"how-pulse-works\"\u003eHow Pulse Works\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/https/agencies/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-168-https-agencies.jpg\"\n    alt=\"600-x-168-https-agencies\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003ePulse was created in around six weeks. We built the project \u003ca href=\"https://github.com/18f/pulse\"\u003ein the open from day 1\u003c/a\u003e, obtained our domain name and relevant cybersecurity approvals in our first couple weeks, and released new versions of the dashboard to \u003ca href=\"https://pulse.cio.gov/\"\u003epulse.cio.gov\u003c/a\u003e early and often throughout the process.\u003c/p\u003e\n\u003cp\u003eWe also gathered usability feedback throughout development from users both inside and outside of the government, and repeatedly incorporated the results of that feedback into our work. Even though Pulse is only a handful of pages and puts most of its data into a simple table, we wanted to pay attention to detail and take the same user-centered approach 18F takes with our larger projects.\u003c/p\u003e\n\u003cp\u003ePulse is a static website whose data is created from a combination of sources:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe \u003ca href=\"https://github.com/GSA/data/blob/gh-pages/dotgov-domains/2015-03-15-federal.csv\"\u003eofficial .gov domain list\u003c/a\u003e. This is currently exported manually by GSA staff on a roughly quarterly basis.\u003c/li\u003e\n\u003cli\u003eThe \u003ca href=\"https://analytics.usa.gov/data/sites.csv\"\u003elist of websites which participate in DAP\u003c/a\u003e. This is also currently exported manually by GSA staff on a roughly quarterly basis.\u003c/li\u003e\n\u003cli\u003eData collected from a public scan of how federal domains respond to HTTP and HTTPS, using an open source tool by \u003ca href=\"https://twitter.com/benbalter\"\u003eBen Balter\u003c/a\u003e called \u003ca href=\"https://github.com/benbalter/site-inspector\"\u003esite-inspector\u003c/a\u003e.\u003c/li\u003e\n\u003cli\u003eData collected from a public scan of HTTPS configuration details for federal domains, using the \u003ca href=\"https://github.com/ssllabs/ssllabs-scan/blob/stable/ssllabs-api-docs.md\"\u003eSSL Labs API\u003c/a\u003e.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eTo coordinate the data collection process, we created \u003ca href=\"https://github.com/18F/domain-scan\"\u003edomain-scan\u003c/a\u003e, a small Python command line tool that runs domains through site-inspector and the SSL Labs API and produces CSV reports.\u003c/p\u003e\n\u003cp\u003eWe then run these CSVs through a \u003ca href=\"https://github.com/18F/pulse/blob/master/data/data.py\"\u003efinal step\u003c/a\u003e, where we take the low-level primitives we gathered during the scanning process and create some higher-level conclusions and save them in a format that Pulse can automatically render into a table.\u003c/p\u003e\n\u003cp\u003eThe process is not fully automated, and so Pulse isn’t updated every day. There’s work to do on all of the above to get to the point of showing fully up-to-date data without human intervention.\u003c/p\u003e\n\u003ch2 id=\"measuring-participation-in-the-digital-analytics-program\"\u003eMeasuring Participation in the Digital Analytics Program\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/analytics/domains/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-181-analytics.jpg\"\n    alt=\"600-x-181-analytics\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eThe \u003ca href=\"https://digital.gov/guides/dap/\"\u003eDigital Analytics Program\u003c/a\u003e (DAP) is a free, shared Web analytics service for U.S. federal agencies.\u003c/p\u003e\n\u003cp\u003eTo participate, agencies place some JavaScript on their websites that report to a combined analytics account. DAP has \u003ca href=\"https://digital.gov/guides/dap/common-questions-about-dap/\"\u003eprivacy controls\u003c/a\u003e that anonymize visitor addresses and restrict data sharing.\u003c/p\u003e\n\u003cp\u003eAccess to the account is shared within the federal government, and much of its data is shared publicly on \u003ca href=\"https://analytics.usa.gov/\"\u003eanalytics.usa.gov\u003c/a\u003e. DAP also regularly publishes a list of around 4,000 participating websites that have reported visitor data in the preceding 2 weeks.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/analytics/domains/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-221-analytics-domains.jpg\"\n    alt=\"600-x-221-analytics-domains\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003ePulse measures participation in the simplest way possible: by comparing the .gov domain list to the list of participating websites published by the Digital Analytics Program. It’s not rocket science, but in the future we’d like to automate this process using the \u003ca href=\"https://github.com/18F/analytics-reporter\"\u003eanalytics-reporter\u003c/a\u003e tool \u003ca href=\"https://18f.gsa.gov/2015/03/19/how-we-built-analytics-usa-gov/\"\u003ewe created for analytics.usa.gov\u003c/a\u003e.\u003c/p\u003e\n\u003ch2 id=\"measuring-https-in-gov\"\u003eMeasuring HTTPS in .gov\u003c/h2\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/https/domains/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-194-https.jpg\"\n    alt=\"600-x-194-https\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eEnforcing strong HTTPS is an \u003ca href=\"https://18f.gsa.gov/2014/11/13/why-we-use-https-in-every-gov-website-we-make/\"\u003eimportant baseline\u003c/a\u003e for government websites, and is in the \u003ca href=\"https://blog.mozilla.org/security/2015/04/30/deprecating-non-secure-http/\"\u003eprocess\u003c/a\u003e of \u003ca href=\"https://www.chromium.org/Home/chromium-security/marking-http-as-non-secure\"\u003ebecoming\u003c/a\u003e the baseline for the Web at large.\u003c/p\u003e\n\u003cp\u003eHTTPS is simple enough to detect, but characterizing HTTPS support for a domain, precisely and reliably, is trickier than you might expect.\u003c/p\u003e\n\u003cp\u003eWe lean heavily on the open source \u003ca href=\"https://github.com/benbalter/site-inspector\"\u003esite-inspector\u003c/a\u003e, a command line tool written in Ruby. Site-inspector measures various useful things about websites, and was originally written by Ben Balter to \u003ca href=\"http://ben.balter.com/2015/05/11/third-analysis-of-federal-executive-dotgovs/\"\u003eanalyze .gov domains\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003e\u003ca href=\"https://pulse.cio.gov/https/domains/\"\u003e\u003cdiv class=\"image\"\u003e\n  \u003cimg\n    src=\"https://s3.amazonaws.com/digitalgov/_legacy-img/2015/06/600-x-211-https-domains.jpg\"\n    alt=\"600-x-211-https-domains\"/\u003e\u003c/div\u003e\n\n\u003c/a\u003e\u003c/p\u003e\n\u003cp\u003eTo get the precision we wanted, we needed to take into account several subtle things about domains:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDomains have 4 possible “endpoints”—\u003ccode\u003ehttps://www\u003c/code\u003e, \u003ccode\u003ehttps://\u003c/code\u003e, \u003ccode\u003ehttp://www\u003c/code\u003e, and \u003ccode\u003ehttp://\u003c/code\u003e—which may each exhibit very different behavior.\u003c/li\u003e\n\u003cli\u003eDescribing a domain’s HTTPS support means detecting which endpoint is “canonical,” as well as looking holistically at which endpoints redirect to others.\u003c/li\u003e\n\u003cli\u003eA domain’s HTTPS certificate might be issued for an invalid hostname (e.g. a248.e.akamai.net). In this case, HTTPS is likely an \u003cstrong\u003eunsupported\u003c/strong\u003e way to access the domain.\u003c/li\u003e\n\u003cli\u003eA domain’s HTTPS certificate might have an incomplete or untrusted chain (e.g. missing intermediates, or a private root certificate), in which case HTTPS is likely a \u003cstrong\u003esupported\u003c/strong\u003e way to access the domain.\u003c/li\u003e\n\u003cli\u003eA domain might set an \u003ca href=\"https://https.cio.gov/hsts/\"\u003eHSTS\u003c/a\u003e policy for www, but neglect to apply one to the bare domain, negating HSTS policy for its other subdomains.\u003c/li\u003e\n\u003cli\u003eA domain might support HTTPS with a valid certificate, but have a policy of “downgrading” users by redirecting away from HTTPS to HTTP.\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eWe use site-inspector to look at all of the above factors (and many more) and calculate a bunch of helpful things about a domain’s HTTPS support. If you really want to dive deeply into the methodology, you can read the \u003ca href=\"https://github.com/benbalter/site-inspector/pull/24\"\u003eoriginal work discussion\u003c/a\u003e.\u003c/p\u003e\n\u003cp\u003eTo measure HTTPS quality, we lean on \u003ca href=\"https://www.ssllabs.com/\"\u003eSSL Labs\u003c/a\u003e. SSL Labs’ \u003ca href=\"https://www.ssllabs.com/downloads/SSL_Server_Rating_Guide.pdf\"\u003egrading system\u003c/a\u003e [PDF] has become a widely respected, universally referenced gauge of HTTPS quality. (Here’s the \u003ca href=\"https://www.ssllabs.com/ssltest/analyze.html?d=pulse.cio.gov\"\u003ereport for Pulse itself\u003c/a\u003e).\u003c/p\u003e\n\u003cp\u003eWe used \u003ca href=\"https://github.com/ssllabs/ssllabs-scan\"\u003essllabs-scan\u003c/a\u003e, an open source client for the \u003ca href=\"https://github.com/ssllabs/ssllabs-scan/blob/stable/ssllabs-api-docs.md\"\u003eSSL Labs API\u003c/a\u003e, to collect the top-level grade along with some common relevant issues that are worth addressing (such as forward secrecy, or the use of SHA-1 signatures).\u003c/p\u003e\n\u003ch2 id=\"looking-forward\"\u003eLooking forward\u003c/h2\u003e\n\u003cp\u003eWe’re still in the process of fully documenting the tools we used. If you’re interested in using any of it in your own work, and you have questions about how to get started, \u003ca href=\"https://github.com/18f/pulse/issues/new\"\u003ering in on GitHub\u003c/a\u003e. We’re an open source team, and we’d love your contributions!\u003c/p\u003e\n\u003cp\u003ePulse is clearly a small and simple website, but we think it’s a promising foundation for celebrating (and motivating) the U.S. government’s progress on making world-class websites and online services.\u003c/p\u003e\n\u003cp\u003eWe’re thrilled we had the opportunity to work with the Office of Government-wide Policy here at GSA to get Pulse started, and we hope others find it useful. Feel free to \u003ca href=\"https://github.com/18F/pulse/issues/new\"\u003eleave feedback\u003c/a\u003e on the project so far, and where to take Pulse next!\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eEric Mill, Julia Solórzano, and Gray Brooks work at 18F. John Tindel works in the Office of Government-wide Policy.\u003c/em\u003e\u003c/p\u003e\n\u003cp\u003e\u003cem\u003eThis article was originally posted on the \u003ca href=\"https://18f.gsa.gov/2015/06/02/taking-the-pulse-of-the-federal-governments-web-presence/\"\u003e18F blog\u003c/a\u003e.\u003c/em\u003e\u003c/p\u003e\n"}
  ]
}