EuroMakers Logo
SoftwareJournalMy stackSubmit
SoftwareBrowse the European directoryJournalInsights on European softwareMy stackProducts you saved on this deviceSubmit Software
Back to the Journal
Editorial

Proton Chose the Hardware

A midnight filter swap in Frankfurt took Mail, VPN, Drive, and Pass down together. Proton's on-call powered the servers off first. Primary databases do not fail over without a person.

EM

The EuroMakers Editorial Team

Researching European software

29 August 20264 min read
Editorial cover for Proton Chose the Hardware
Editorial

The Journal · Made in Europe

At 00:09 CEST on 27 August, Proton's status page said Investigating. Twenty-nine minutes later it named a room: a critical cooling failure in Frankfurt. By morning the headline was Global Outage. Proton Mail, VPN, Drive, Pass, Calendar. Just over two hours. No mail lost.

Proton status page Past Incidents: 29 and 28 August show no incidents, 27 August is a red Global Outage marked Resolved with no data lost
status.proton.me, 29 August 2026. The green rows are today. The 27th is still Global Outage.

The incident report landed the next day, under Bart Butler's name. First paragraph, their line:

The root cause was a total failure of the cooling system in our Frankfurt datacenter.

— Bart Butler, proton.me, 28 August 2026

Just after 23:00 on the 26th, cooling in the main hall failed. From about 23:15 the air went from 21.8°C to 51.9°C in less than half an hour. Some probes read 60°C. Servers and switches died one by one.

Proton incident report Timeline: cooling failed just after 11 p.m., temperature rose from 21.8°C to 51.9°C in less than half an hour, some probes 60°C
proton.me/blog/august-27-outage-incident-report, 29 August 2026. 21.8°C to 51.9°C before midnight.

The primary waits for a person

A listing will say Swiss, encrypted, own infrastructure. It will not say this: almost every Proton system fails over on its own. Primary databases do not. They keep a person in that loop on purpose, so a blip does not split the brain — replicas that missed writes and cannot be reconciled. Standard procedure is still to fail over inside the same building, for latency. A building that is cooking is a bad replica.

Around midnight both the primary and the backup network switch on one rack died. That rack held several primary database copies. User-facing damage started there.

Proton incident report: both primary and backup network switches on a critical rack failed, primary database failovers are not automatic, on-call had to choose hardware versus service and Frankfurt versus Zurich
proton.me, 29 August 2026. Split brain is the reason a human still holds the failover.

The on-call had three questions while the hall heated. Bring the inbox back, or save the metal. Fail over inside Frankfurt, or jump to Zurich. Move everything, or only what was already dead. Random machines dying one by one is not the complete-hall script their automation expects.

They powered servers off. AI-era CPUs and GPUs had squeezed the old three-to-four-hour thermal window down to about twenty minutes. Replacement kit is scarce. Losing the boxes would have lasted longer than a delayed inbox.

105°C on the network cards

Cooling returned at 00:45 CEST. Then the NICs: 105°C against a 45°C normal. Temperature protection had disabled cards until a cold reset. Proton's security posture limits out-of-band access, so they had to wake more staff. Most services were back by 01:30. Push and payments around 02:00. Status called the rest done at 02:27.

Both compressors lost their filters

The attacker was a maintenance window nobody announced. The facility operator replaced air filters on both of the redundant compressors in the middle of the night, without notice, and did not say the cooling had failed. Proton's redundancy assumed the hall would tell them.

Proton incident report Root cause: air filter replacement on both redundant air compressors at night without prior notice, operator failed to communicate the cooling failure, some servers suffered heat death
proton.me, 29 August 2026. Both legs of the redundant cooling were opened at once.

The series of events that led to this incident are highly improbable — yet they happened.

— proton.me/blog/august-27-outage-incident-report

Some servers took heat death. They do not yet know what that does to the lifespan of the rest. Database work that would shorten this recovery is due by the end of the year. New hall space in the next few weeks. Until then, a thermal event in one Frankfurt room can still put a person between you and the replica in Zurich.

EuroMakers files Proton under Communication and Security because that is the job on a calm Tuesday. The night of the 27th the job was a rack. They run their own halls in Frankfurt and Zurich. They do not rent Amazon. Independence is the product. It is also why a filter change in one room became a global status incident.

Who this is not for

Do not read the outage as a reason to leave if you picked Proton because a Swiss company holds the keys and the metal. The two hours are that choice under heat. Do not read Proton as a five-nines hyperscaler if you cannot sit through a window while a human decides whether Frankfurt or Zurich gets the primary. That person already wanted AWS.

  • You need automatic multi-site failover with no one on the call. Primary databases wait for a person, on purpose.
  • You cannot accept delayed mail for two hours. Nothing was lost. Delivery waited.
  • You thought own-hardware meant the facility operator could not take both compressors at once. They did.
  • You wanted the catalog line — Swiss, encrypted — without the thermal one. The thermal one showed up at 51.9°C.

If you want encrypted mail from a company that will power the rack off rather than split a database, start with the listing, then read the 27 August report. The catalog will say Swiss. The report will say 51.9°C.

New to the EuroMakers Journal? Start with how the Journal works.

#Proton#Frankfurt#Switzerland#Datacenter#Incident#Email

Find your next European tool

Browse a curated directory of software made in Europe across every category your team relies on.

Explore the directory

Keep reading

Editorial cover for Stelae Runs on a Single Machine
Editorial
28 August 2026

Stelae Runs on a Single Machine

Leon Becker's WordPress editor in Türkheim sleeps when you stop typing. The public beta lasts until 31 October 2026. Then €19.90 a year, no grandfathering — because one person on one host is the product.

5 min readRead
Editorial cover for Germany Paid to Keep VLC on Windows XP
Editorial
27 August 2026

Germany Paid to Keep VLC on Windows XP

The orange button on videolan.org still ships VLC 3.0.23. The release notes credit Germany's Sovereign Tech Fund — and they still list Windows XP.

4 min readRead
EuroMakers

European software, found when you need it. Origin on every listing.

Explore the directory

Discover

  • All software
  • Journal
  • My stack

Contribute

  • Submit software
  • Contact

Our standard

  • Why we list it
  • Privacy

© 2026 EuroMakers. Made with conviction in Europe.

Logos provided by Brandfetch and Logo.dev.

EuroMakers logo rising behind an illustrated Alpine valley, European town, railway, and cyclists