A hole in Cloudflare’s shield: how the attack on Jabber.ru exposed a problem no one has talked about since 2023

I raised this issue on the Cloudflare forum 23 days ago, but, as with other similar requests, I received no response. So I’m here to break it down for you.

Flight review: the jabber.ru incident

For those who missed it or forgot, here’s a quick recap:

What happened? For several months (at least from July to October 2023), all traffic to Russia's largest XMPP service jabber.ru was intercepted and decrypted. The attackers managed to obtain fully valid TLS certificates from Let's Encrypt for the domains jabber.ru and xmpp.ru.

How did they do it? This wasn’t a server hack. The attack was carried out at the hosting providers' network level — Hetzner and Linode. The attackers (presumably part of a "lawful interception" operation by state actors) reconfigured routing so that all traffic meant for the servers of jabber.ru, including requests for domain validation from Let's Encrypt, passed through their infrastructure.

By intercepting validation requests (most likely http-01 or tls-alpn-01), they successfully proved to the CA (Let's Encrypt) that they controlled the domain and received legitimate certificates. The attack was discovered accidentally when one of the fraudulent certificates expired and wasn’t renewed in time, causing connection errors for users.

In my opinion, the key point here is that the vulnerability is not in the XMPP protocol or Let's Encrypt itself. The vulnerability lies in the fundamental domain ownership verification (Domain Validation, DV) mechanism.

Technical enlightenment: CAA, RFC 8657, and how this should save us

It might seem like if your traffic can be intercepted at the provider level, you're defenseless. But that's not the case. To protect against such scenarios, there's the Certification Authority Authorization (CAA) mechanism, described in RFC 8659.

It’s a simple DNS record that says: "Only this CA can issue certificates for my domain."

# Allow only Let's Encrypt
example.com. IN CAA 0 issue "letsencrypt.org"

But, as the case with jabber.ru showed, this is not enough. The attackers did use the allowed CA, which issued them the certificate.

Post factum, thanks to the incident analysis by an engineer named Hugo Landau, he came to the conclusion that the standard he had proposed back in 2019 under the number RFC 8657could have prevented such an incident. This standard extends CAA with two critically important parameters:

1. accounturi
   This is a "second factor" for certificate issuance. This parameter binds certificate issuance to a specific account at the CA. Your account URI in Let's Encrypt is a unique identifier.

   # Allow Let's Encrypt, but ONLY from the account with ID 12345678
   
   example.com. IN CAA 0 issue "letsencrypt.org; accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/12345678"

   Even if an attacker intercepts your traffic, they won't be able to obtain a certificate, since their request will come from a different Let's Encrypt account, and LE has supported this standard since January 2021. The CA will check the accounturi from the request against what's in your CAA record, see the mismatch, and deny issuance.

2. validationmethods
   This parameter allows you to restrict the methods the CA can use to verify domain ownership.

   # Allow Let's Encrypt, but only via DNS validation
   
   example.com. IN CAA 0 issue "letsencrypt.org; validationmethods=dns-01"

   Since the attack on jabber.ru was possible due to intercepted network traffic, limiting validation to the dns-01 method (which requires changes to the DNS zone, not control over network traffic) would have made the attack significantly harder, and the use of DNSSEC by the domain could have made it impossible.

3. Combo

   # Maximum protection
   
   example.com. IN CAA 0 issue "letsencrypt.org; validationmethods=dns-01;
   accounturi=https://acme-v02.api.letsencrypt.org/acme/acct/12345678"

   Hugo Landau, author of RFC 8657, stated in his incident analysis that based on what we know about this attack, it would have been prevented by deploying this extension.

Elephant in the china shop: what does Cloudflare have to do with this?

And now, the main point. Cloudflare, as a leader in internet infrastructure and security, does something very strange for its no-alternative Universal SSL service for free accounts—when enabled, Cloudflare automatically adds CAA records in your DNS zone for its partner CAs (Let's Encrypt, Google Trust Services, etc.).

Here's what this record looks like:

# Records for regular certificates (issue)
example.com. IN CAA 0 issue "letsencrypt.org"
example.com. IN CAA 0 issue "pki.goog; cansignhttpexchanges=yes"
example.com. IN CAA 0 issue "digicert.com; cansignhttpexchanges=yes"
example.com. IN CAA 0 issue "comodoca.com"
example.com. IN CAA 0 issue "ssl.com"

# Records for wildcard certificates (issuewild)
example.com. IN CAA 0 issuewild "letsencrypt.org"
example.com. IN CAA 0 issuewild "pki.goog; cansignhttpexchanges=yes"
example.com. IN CAA 0 issuewild "digicert.com; cansignhttpexchanges=yes"
example.com. IN CAA 0 issuewild "comodoca.com"
example.com. IN CAA 0 issuewild "ssl.com"

See the problem? They still don't use accounturi and don't let you overwrite the records!

This leaves the same security hole that affected jabber.ru. Cloudflare is basically saying: "Any account in Let's Encrypt can obtain a certificate for this domain if validation passes."

Your own, stricter CAA record with accounturi for your server not hidden behind a proxy becomes almost useless. Because according to CAA processing rules, if there is at least one permissive record without accounturi, the CA is allowed to issue a certificate for any account. And if you decide to manually add records with accounturi in CF, CF will kindly not apply them and will keep its own records. And if you want to disable Universal SSL, you'll have to pay some real money.

Thus, Cloudflare not only fails to use modern security standards for its clients, but actively prevents them from doing so on their own, effectively nullifying the whole idea of RFC 8657.

Conclusions and what to do?

The current situation is a stalemate. On one hand, we have a working standard, supported by leading CAs (Let’s Encrypt implemented this back in 2021), which solves a real and well-documented problem. On the other—the largest infrastructure provider, which not only ignores this standard and ignores reports about this standard, but with its implementation also creates risks for millions of users.

For users:

1. Conduct an audit. Check your CAA records. If you use Universal SSL from Cloudflare, know that you are vulnerable to attacks like jabber.ru.

2. Demand change. If you care about your project’s security, support my topic on the Cloudflare forum. The more attention this gets, the higher the chance we’ll get heard. This is not the first such request, but previous ones were simply ignored.

3. Consider alternatives. For critical projects, it might be worth ditching Universal SSL in favor of manual certificate management and full control over CAA records—either in Cloudflare or in a different service.

What Cloudflare should do:

1. Start using accounturi and validationmethods for all certificates they issue under Universal SSL. This would immediately and by default raise security for all their customers. After all, they certainly know the ID of their profiles when requesting certificates, don’t they?

2. Provide full CAA record management for users, so they can add their own records with accounturi and validationmethods, and so those records properly coexist with Cloudflare’s records.

3. Update the documentation and honestly tell users about the current risks and how they handle CAA.

Yes, the white fluffy fox will not reveal itself for some time, but as practice shows, it will surely come. And it’s better to patch this hole now, rather than deal with the consequences later, as the administrators of jabber.ru once had to.

David Osipov

ISNI: 0000 0005 1802 960X