Methodology

How we grade news organizations on HTTPS

Secure The News looks at a variety of factors related to HTTPS deployment best practices and give each news site a score of 0-100, which is then converted to a more familiar letter grade from A-F, according to the table below.

Our site security grading methodology was inspired by SSL Labs’ SSL Server Rating Guide. Ideally, we would reuse SSL Labs' incredibly detailed results, but unfortunately we cannot due to their terms of service. Qualys, which owns SSL Labs, refused to grant us permission to use their API.

Our criteria will instead mirror General Services Administration (GSA) Pulse’s current criteria for a modern and secure HTTPS deployment. Most sites start by deploying HTTPS alongside the insecure HTTP version of their site, and then gradually harden their deployment over time using standard techniques such as: enforcing HTTPS via redirect, setting an HSTS response header, using HSTS preloading, and so on.

Grading scale

Score Letter Grade Explanation
0 F No HTTPS or invalid HTTPS
30 D HTTPS available, but downgrades to HTTP
50 C HTTPS available but not default
70 B Enforces HTTPS by default
71-100 B+ through A+ Best possible deployment, based on bonus factors below

Once a site is enforcing HTTPS, for a grade of a B, it may continue to earn points by implementing one or more of the following measures to harden their HTTPS deployment, which can take the site anywhere from a B+ grade to an A+. Since some of these measures can be implemented independently of each other, and sites may choose to do things differently depending on their unique requirements, the point values are added independently.

Note that while our metric will add these values independently, in practice some of these measures depend on each other. For example, a site cannot be preload-ready unless it has HSTS max-age >= 18 weeks and has includeSubDomains. This does not conflict with adding the bonus points independently, but it is good to keep in mind.

Bonus Points Explanation
+5 Site deploys HSTS
+5 HSTS max-age >= 18 weeks
+10 HSTS with includeSubdomains
+5 HSTS is preload-ready
+5 HSTS is preloaded

A site that implements all of these best practices earns a total of 30 bonus points. When added to the baseline score of 70 points for a deployment that enforces HTTPS, this gives such a site the maximum score of 100.

Future Improvements to the Secure The News Metric

HTTPS and website security experts will notice that this rubric is not 100% complete. It does not take into account numerous issues that may adversely affect the security of an HTTPS deployment and would be mitigated according to best practices in a modern deployment, such as:

  • Support for out-of-date and insecure SSL/TLS protocols (e.g. SSLv2, SSLv3)

  • Support for insecure cipher suites that are vulnerable to well-known cryptographic attacks (e.g. POODLE, CRIME, BEAST, etc.)

  • Support for insecure cipher suites which could be vulnerable to downgrade attacks, aka FREAK.

  • Support for insecure ciphersuites that use Diffie-Hellman key exchange with keys that are not large enough to be considered secure, aka Logjam.

  • Use of weak public key ciphers or insufficient key lengths (e.g. 1024-bit RSA)

  • Use of signature algorithms that are vulnerable to collision attacks (e.g. MD5, SHA1)

  • And so on…

Over time, we plan to gradually update and improve our grading methodology to provide a more detailed and nuanced evaluation of the security of a given HTTPS deployment. However, doing so is a substantial and time-consuming project. If you're interested in getting involved, the Secure The News source code is public on GitHub (under the AGPLv3 License). If you don't know what to work on, we suggest taking a look at issues that are tagged with "contributors-welcome".