Google just made "the server never sees your data" a realistic product pitch. The company open-sourced HEIR, a compiler that converts pre-trained AI models so they can run on encrypted data. It demonstrated four working applications, from fraud detection to audio hotwords, all computed on ciphertexts the server cannot read. For founders building AI on sensitive data, this is the first credible open-source path to private inference that does not require a team of cryptographers.
Why It Matters
Until now, building an AI feature on user data meant choosing between two bad options. Send data to the cloud and you own the breach risk, the compliance burden, and the "trust us" conversation with enterprise buyers. Process locally and you are stuck with device limitations, plus you risk leaking your proprietary model if it ships to the device. Homomorphic encryption removes that tradeoff: the server computes on encrypted inputs and returns encrypted results, so it never sees the underlying data.
The catch was always cost and complexity. Converting a model to homomorphic encryption required cryptographers and days of manual work. Google's HEIR automates the conversion, and the latency overhead is falling fast. That is why this announcement matters more than a research demo.
Background
HEIR stands for Homomorphic Encryption Intermediate Representation. It is an open-source compiler toolchain and development platform built by Google researchers and engineers, added to Google's Private Computing Toolkit. The project has been in the works since Google signaled its intentions in 2023, and the homomorphic encryption community has been building on it since.
Today's showcase includes four compiled applications with latency numbers on a single-threaded CPU:
- A deep learning recommendation model built with Belfort Labs, LG, and NYU that serves private content recommendations.
- A credit card fraud detector compiled with Niobium and hardshell.ai.
- The Kitsune intrusion detection system that spots anomalies in encrypted network traffic without seeing packet contents.
- A hotword detector built with Belfort Labs that lets an audio-triggered AI agent recognize wake words while the audio stays encrypted.
Hardware accelerator partners include Belfort, Niobium, Cornami, and Optalysys. Academic collaborators span Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, Edinburgh, and Tsinghua. Four peer-reviewed papers are already built on HEIR.
Key Insights
The cost curve is the real headline. Homomorphic encryption has existed for decades. The reason it never shipped in production was overhead: encrypted operations are orders of magnitude slower than plaintext ones. Google's compiler narrows that gap, and hardware accelerators from its partners aim to cut latency further. When the tradeoff shifts from "impossible" to "measurable cost," founders start doing the math, and the math is improving every quarter.
Open sourcing is a strategic move, not charity. Google already owns differential privacy, private information retrieval, and secure enclaves on Google Cloud. HEIR extends that moat: if HEIR becomes the default compiler for encrypted inference, Google becomes the default place to run it. Founders get free tooling, Google gets the ecosystem. That is the same playbook as Kubernetes, and it worked.
Privacy is becoming a moat, not a checkbox. Regulated industries like healthcare and finance are the clearest buyers. They cannot share data across institutions under current rules, and they are terrified of breach liability. A vendor that can say "we process your data without ever seeing it" skips the hardest part of the enterprise sales cycle: the security review. For startups, that is a competitive weapon against incumbents who are stuck on the "trust us" model.
Audio agents are the most exposed category. AI agents that record or process meeting audio are walking into wiretap-style lawsuits, and Otter.ai is a live example. A hotword detector that never decrypts the audio turns a legal liability into a product feature. If you build voice or agent products, encrypted inference is risk management you can sell.
What This Means for Founders
Three moves are worth making now:
- Test HEIR on your own model. The repo is public and the toolchain is free. Run a proof of concept on your most sensitive feature and measure the latency delta before you dismiss it.
- Revisit your enterprise pitch. If you sell to healthcare, finance, or any regulated buyer, "we never see the data" is now a claim you can back with cryptography instead of policy. Lead with it.
- Watch the accelerator partners. Belfort, Niobium, Cornami, and Optalysys are building hardware to cut encrypted inference latency. If they deliver, private inference stops being exotic by 2027.
The window is real but not infinite. Google is giving the tooling away, which means the moat moves up the stack: to the applications you build, the data you already own, and the trust you earn in regulated markets. Start the proof of concept this quarter, not next year.
