The Private Key Was the Number 4: Test Your Estate Before the eMudhra Long Tail Bites

October 21, 2026 · 14 min read

The Private Key Was the Number 4: Test Your Estate Before the eMudhra Long Tail Bites

TL;DR - Eight eMudhra (emSign) intermediate CA certificates were generated in April 2025 with private keys so weak that anyone can recover them from the public certificates, in some cases instantly: the keys are integers smaller than 256, and one of them is literally the number 4. A researcher proved it on 17 October and the browsers had all blocked the keys within three days, but no fraudulent certificates ever hit Certificate Transparency, so CT-enforcing clients were never realistically exposed. The exposure is everything else: servers, CI runners, containers, language runtimes and appliances that do not check revocation and need software updates to stop trusting the compromised keys. What you need to do: run one curl test against your build box, patch your OS trust stores and container images, update your language CA bundles, and reissue any emSign certificate you hold before Chrome drops the parent root on 27 October.


By The Numbers

ItemValue
Compromised intermediate CAs8 (6 TLS, 2 code signing)
Size of the private keysIntegers under 256 (P-384 should be 384 bits)
Smallest private key4
Keys created30 April 2025, single key ceremony
Time from report to Chrome block100 minutes (18:05 to 19:45 UTC, 17 October)
Time from report to Firefox block2 hours 53 minutes (20:06 to 22:59 UTC)
Fraudulent certificates on CT logs0
Certificates chaining to the root Chrome is removing~30,000, about 97% issued via ACME
Chrome removal of emSign ECC Root CA - G327 October 2026

I opened the Bugzilla thread on Tuesday morning with coffee, expecting the usual CA incident shape: a misissued certificate, an embarrassed apology, a boring afternoon. What I found instead was a Mozilla engineer explaining that the private keys of eight intermediate certificates are integers smaller than 256, and a table elsewhere showing the smallest of them is the number 4.

I have spent close to two decades working in cloud and security architecture, and the TLS trust store is the substrate everything I build sits on, so a compromised intermediate CA is not an abstract story for me. It is the layer I have to assume holds. Reading that a working intermediate key fits in a single byte took a minute to process.

Here is what actually happened, because the shape of the failure matters more than the number. On 30 April 2025, eMudhra, the Indian certificate authority that sells under the emSign brand, ran a key generation ceremony to create a batch of intermediate CAs. According to the CA's own account in Mozilla's incident tracker, a flaw in the ceremony script created a second, weak key pair on the hardware security module during a key-wrapping test. The weak keys had labels similar to the intended keys. They were the ones that got used for the certificate requests. eMudhra's own summary of the ceremony is blunt: there were human errors in multiple steps of the process.

The result was eight intermediate CA certificates whose private keys are recoverable from the public certificates in milliseconds. Six of them sign TLS certificates, covering the DV, OV and EV tiers. Two sign code-signing certificates. All eight were cross-signed by Sectigo, and the chains reach roots trusted by Mozilla, Microsoft, Apple and Chrome. From the moment those certificates were created, anyone who fetched one and ran the recovery could have minted a valid certificate for any domain on the internet.

Nobody appears to have done so, which is the part worth understanding before reaching for the panic button.


What Happened After the Report

On Friday 17 October, a security researcher going by Wayne sent every root program the same proof: a message, and for each of the eight intermediates, a valid signature on that message made with the private key. The message read, in full, "We have a problem here - Wayne".

The response timeline is the fastest I have seen for an incident of this class, and it is worth reading slowly:

Time (UTC)What happened
17 Oct, 18:05Researcher reports to Apple, Chrome, Microsoft, Mozilla and eMudhra with proof
17 Oct, 19:45Chrome corroborates and pushes a CRLSet block - 100 minutes total
17 Oct, 22:59Firefox OneCRL block complete for all standard configurations
18 OcteMudhra revokes all eight intermediates
19 OctApple devices block the keys; Sectigo revokes its cross-signs; downstream trust stores notified
20 OctMicrosoft platforms block; Mozilla removes emSign ECC Root CA - G3 from NSS; public disclosure
27 OctChrome removes the same root from its trust store (stable rollout)

Neither browser needed an update or a restart. OneCRL and CRLSets are revocation lists the browsers fetch on their own schedule, so the protection landed on user machines the same evening. Apple and Microsoft followed within three days. If your exposure is a browser, this story ended for you before the weekend did.

Why the Browsers Were Never Really in Danger

There are no entries in the Certificate Transparency logs for any certificate issued by these intermediates. In eighteen months, the compromised keys never issued a single publicly logged certificate.

That matters because Chrome, Firefox, Edge and Safari all enforce CT: a certificate that is not logged simply does not validate. An attacker holding these keys could have generated all the fraudulent certificates they wanted, and none of them would have worked against a CT-enforcing client. The eighteen-month window was real, but it was closed at the browser layer the whole time.

The incident page puts it carefully: this does not rule out unlogged certificates that would be trusted by clients which do not enforce CT, and it does not rule out abuse in the future against clients that do not check revocation. That is exactly the right level of worry, and it points the story somewhere other than your browser.

The Long Tail Is the Story

Most software that uses TLS does not consume revocation information, and never has. Revocation checking is the perennial unfinished business of the web PKI: clients fail open when they cannot check, because failing closed breaks the internet in practice. That design trade-off was a reasonable bet when the worst case was one misissued end-entity certificate. It is a worse bet when eight intermediate keys are public knowledge and the software has no mechanism at all for untrusting them short of a software update.

That is the actual exposure list: your servers, your CI runners, your container images, your language runtimes, your command-line tools, and the appliance in the corner that nobody has touched since 2023. Mozilla's guidance to trust-store maintainers was direct: if your clients reliably check revocation and fail closed, you are fine. If they do not, you need to act, either by blocking the intermediates specifically or by removing trust in the roots that signed them.

Four days after disclosure, the patch state of that long tail is genuinely uneven. Node.js and Bun are still pending as I write this. That fact is the thesis of this post in one table row.

The Ten-Second Test

Dennis Jackson, the Mozilla cryptography engineer coordinating the disclosure, stands up twelve test hosts that serve certificates chaining to the compromised keys. Point any client at one:

curl https://dv-g3b-emudhra.we-have-a-problem.com/

A certificate error means that client refuses the compromised chain. You are protected. A clean connection with no errors means that client still trusts keys the entire internet has known about since 17 October.

Run the boring test first, on your browser, to see what protected looks like. Then run the interesting ones: inside the container your builds execute in, from your jump boxes, from whatever your monitoring uses to hit APIs, and from that appliance. The offline variant matters too - the site offers the full chains in a zip you can verify with OpenSSL against any trust store you cannot reach from the internet.

The Patch List

Work through it in this order, because the first two items cover the most software.

Patch your OS trust store. Ubuntu, Debian, Fedora, Arch and Gentoo all shipped updated ca-certificates packages in the days after disclosure, and NSS 3.131.1 carries the removal for anything linked against it. On Enterprise Linux, until your vendor ships, the incident page publishes a workaround that blocklists the emSign root directly. This one update fixes curl, apt, and everything else that defers to the system bundle.

Rebuild your container images. Patching the host does nothing for images that baked a stale bundle weeks ago, and your registry cache will happily keep serving them. Pull fresh bases and rebuild. Then run the curl test from inside a rebuilt image, because that is where your software actually runs.

Update language bundles. The versions that matter: Go 1.26.10 and 1.27.3, Python certifi 2026.10.19, the curl CA bundle dated 2026-10-20, NSS 3.131.1, Rust webpki-roots 1.0.10, Electron 44.9.1, Elixir castore 1.0.22, and the Mozilla CA module for Perl. Node.js and Bun are pending. If your platform has not shipped yet, that is not a reason to wait on the rest - it is a reason to run the curl test against that platform and file the issue internally.

Check appliances and network gear. Firewalls, load balancers and VPN concentrators ship their own bundles, frequently derived from NSS. Vendor advisories went out on 19 October. Work through yours, and expect the oldest box in the fleet to be the last one fixed, because it always is.

Go took an unusual extra step worth knowing about: the Go releases put the hashes of the eight compromised keys directly in code, on its urgent security track. Filippo Valsorda, who maintains Go's cryptography packages, said the quiet part out loud - the keys are compromised and revoked, but Linux root stores are too poorly maintained for revocation to reach applications, so embedding the block in the runtime is the only option that actually works. When the Go team starts hard-coding individual CA keys into releases, the revocation story has failed at every layer beneath them.

If You Hold an emSign Certificate

Chrome is going further than blocking the eight intermediates: it is removing emSign ECC Root CA - G3, the root that signed them, from its trust store entirely, reaching stable Chrome on 27 October US time. Mozilla has already done the same in NSS. Some legitimate certificates chaining to that root will simply stop working in Chrome, and eMudhra has committed to migrating affected subscribers.

The scale is manageable: about 30,000 certificates chain to that root, and roughly 97% were issued through ACME, so most will reissue automatically. The manual check is still worth ten minutes this week. Look at your certificate inventory for emSign issuers - your CA management console, or a quick sweep with openssl s_client across your public hosts, or crt.sh if you want the outside view. Anything on the affected hierarchy gets reissued from another CA before Tuesday, or the sites behind it go dark in Chrome with no error message that points back at you.

The Warning Sign Nobody Read

Here is the part of this incident that I keep coming back to. In May 2025, six months after the ceremony, the certificate revocation lists for the six TLS intermediates started failing signature checks, and Sectigo reported it to Mozilla. eMudhra re-signed the CRLs with the key that matched the certificates - which was the weak key - and closed the incident as a process error.

Everything needed to catch this was inside that ticket. A CRL signature that does not verify means the signing key does not match the certificate it belongs to. When that happens in the weeks after a key ceremony, the question "why was the signature wrong?" has exactly one interesting answer, and it is the answer that ends this incident sixteen months early. Instead the mismatch got re-signed away, the ceremony records presumably matched the intended keys rather than the real ones, and the weak keys stayed trusted for another year and a half.

A failed signature verification is a signal, not a chore. Before you re-sign and close the ticket, ask why the signature was wrong. That discipline costs ten minutes and would have caught the most embarrassing CA failure of the year in May 2025 instead of October 2026.

The certificate warning that closes as a process error is not closed. It is parked, and it is waiting for a researcher with a proof to reopen it on a Friday night.


What To Do Right Now

The deadline that actually bites is Tuesday 27 October, when Chrome's removal of the emSign root reaches stable. Everything on this list fits inside a normal working day.

  1. Run the curl test against your build runners, jump boxes and monitoring hosts. Log which ones still trust the chain.
  2. Patch OS trust stores everywhere the package manager reaches: apt update && apt install ca-certificates, dnf update ca-certificates, pacman -Sy ca-certificates, or your equivalent.
  3. Rebuild container images from fresh bases and re-run the curl test from inside them.
  4. Bump language bundles: certifi, webpki-roots, castore, curl's bundle, NSS wherever it is vendored.
  5. Sweep your certificate inventory for emSign issuers and reissue anything on the affected hierarchy before 27 October.
  6. Chase your appliance vendor if the fleet includes firewalls or load balancers, and note which boxes have no patch yet, because those are your residual risk register.

Key Takeaways

  • Eight emSign intermediate keys were compromised from birth. The ceremony script swapped weak keys in during a key-wrapping test, and the private keys are integers under 256.
  • The browser layer was protected by CT, not by luck. Zero certificates ever hit the logs, so CT-enforcing clients had nothing to attack them with.
  • The exposure is the non-browser long tail. Servers, CI, containers, runtimes and appliances that fail open on revocation and need software updates to untrust the keys.
  • One curl command tells you the truth about any client: a certificate error at dv-g3b-emudhra.we-have-a-problem.com means protected.
  • Chrome drops the parent root on 27 October. If you hold an emSign certificate, reissue before Tuesday or go dark in Chrome.
  • The May 2025 CRL failure was the early warning. Re-signing without asking why the signature was wrong cost the ecosystem sixteen months.
  • Go hard-coding CA key hashes into the runtime is what a broken revocation ecosystem looks like from the inside.

FAQ

Am I affected if I only use Chrome, Firefox, Safari or Edge? Effectively no. All four blocked the compromised intermediates between 17 and 20 October via their own revocation mechanisms, and no fraudulent certificates that would pass CT enforcement were ever issued. Update your browser if you have disabled automatic updates, and otherwise carry on.

Why does Certificate Transparency get the credit here? Because an attacker needs a certificate that a client will accept, and CT-enforcing clients only accept logged certificates. The compromised keys never issued a logged certificate, so the eighteen-month window produced zero usable attacks against those clients. The Android 17 CT post in Further Reading covers what CT does and does not enforce for internal CAs.

What exactly was wrong with the keys? The affected keys were P-384, which should carry 384 bits of randomness. The ceremony flaw produced keys that are integers smaller than 256 instead. Recovering a private key from the public certificate is instant arithmetic rather than cryptanalysis, which is why Mozilla described multiple people recovering them independently within days.

My curl test passed on my laptop. Am I done? No. Your laptop is the client that was already fine. The value of the test is running it from the places where your software actually runs: build containers, CI runners, servers and appliances. A clean pass on your workstation and a clean connection from your build runner is the failure mode this post is about.

What breaks on 27 October? Certificates chaining to emSign ECC Root CA - G3 stop validating in Chrome as the root removal reaches stable. Legitimate certificates are caught in this, not just attacker forgeries, which is why eMudhra is migrating subscribers and why the inventory check matters this week.

Should I distrust eMudhra's other roots too? Chrome's root program is reviewing the status of eMudhra's remaining root in its store, and Mozilla took the emergency step of removing the affected root outright. I would not preemptively rip out a vendor's whole hierarchy on my own estate, but I would not add new emSign-issued certificates this week either, and I would watch the root programs' follow-ups before renewing anything on that hierarchy.

Is my internal PKI affected? Your private CA is not in any trust store, so the compromise does not touch it directly. The reasons to care are adjacent: the incident is a worked example of why ceremony scripts, key custodianship and signature-verification failures deserve your attention, and Android 17's CT enforcement is separately squeezing internal CAs that issue publicly-suffix names. Different mechanisms, same audit window.

My Take

The number I keep coming back to is not 4. It is sixteen months.

The CRL signature checks failed in May 2025. That ticket had the whole story inside it - a signing key that did not match the certificates, six months after a ceremony that everyone involved now agrees had human errors in multiple steps. Re-signing the CRLs and closing it as a process error was not malice; it was a Tuesday. Somebody had a queue, the re-sign made the check pass, and the box got ticked. That is exactly how the worst incidents of my career have all started, and I would bet it matches yours too.

The second thing worth sitting with is Go's response. A programming language shipping with specific CA key hashes welded into its security releases is an ecosystem admitting something: revocation does not reach the long tail, the long tail is where the software actually runs, and the only mechanism that reliably does reach it is the software update you were doing anyway. That is not a workaround, it is a diagnosis.

And the optimistic read is genuine. A researcher proved the compromise on a Friday night, three root programs had blocks out within hours, Apple and Microsoft followed inside three days, and no attack ever materialised at the browser layer because the CT safety net held. The immune response worked. The work left is on us, in the unglamorous places: rebuild the images, patch the bundles, chase the appliance vendor, and ask why the signature was wrong next time one fails.


Mathew Clark Founder, SecureInSeconds Currently: running the curl from a build container, because that is where the answer lives Drafted with AI assistance - read, verified, and fixed by a human.


Further Reading

Share:
Buy me a coffee

You might also like