Google this week formally showcased HEIR, its open-source compiler for fully homomorphic encryption, in a detailed post added to the company's security blog. HEIR is an open-source compiler that unlocks cryptographically secure private AI inference, the latest addition to Google's Private Computing Toolkit. The timing coincides with the August 14, 2025 posting of a formal research paper to arXiv. The paper presents HEIR as a unified approach to building homomorphic encryption compilers, aiming to support all mainstream techniques in the field and integrate with all major software libraries and hardware accelerators.
The core problem HEIR addresses is a longstanding mismatch between cryptographic capability and engineering accessibility. By the authors' own account, all publicly described production systems using homomorphic encryption have been implemented with hand-optimized subroutines, required teams of cryptography experts to build, and have utilized comparatively simple circuits. HEIR attacks that bottleneck directly. The compiler can convert pre-trained AI models that operate on unencrypted data to operate on encrypted inputs, with Google's stated vision being a one-click solution enabling non-experts to incorporate encrypted inference into production applications. Technically, HEIR is built on the MLIR compiler framework and introduces homomorphic-encryption-specific abstraction layers at which existing optimizations and new research ideas may be easily implemented. The project aims to support all modern FHE schemes, code generation for hardware accelerators including GPU, TPU, FPGA, and custom ASICs, standard FHE libraries such as OpenFHE and Lattigo, and front-end languages including Python and Torch. Demonstrated applications include a deep learning recommendation model and a hotword detector, with demos showing single-threaded CPU latency for four applications, including the recommendation model.
The sectoral stakes are substantial. Standard end-to-end encryption forces a trade-off in which user data can be protected from breaches but the service provider cannot perform features that depend on the data; critical sectors like healthcare and finance are especially averse to these risks, and strict regulations limit data sharing across institutions. Local processing, the most common alternative, is constrained by device capabilities, while shipping a proprietary AI model to a device risks leaking the model itself. HEIR directly dissolves both horns of that dilemma by allowing the server to process ciphertexts and return encrypted results without exposing any underlying information.
Skeptics have a substantial empirical counterpoint. The honest overhead of fully homomorphic encryption remains roughly 1,000x to 10,000x versus plaintext, placing anything interactive or frontier-model-sized out of scope; widely quoted minutes-per-token encrypted-LLM figures actually come from multi-party computation research, not FHE. Independent benchmarks confirm a more nuanced picture: on CPU-only implementations, FHE operations can be 10,000 times slower than plaintext equivalents, though with batching, optimized CKKS implementations, and GPU acceleration, that gap narrows to roughly 10x to 100x for throughput-oriented workloads. Researchers in the field note that FHE is practical in 2026 for narrow workloads: private lookups, small-model ML inference, and encrypted analytics, but that the confidence interval around frontier-scale encrypted inference remains very wide. Work on the first FPGA backend for the HEIR compiler was also presented at the 2026 FHE.org conference, suggesting the hardware gap is actively narrowing, though no production timeline has been announced.
The practical ceiling for encrypted AI inference is still set by physics and clock speed, not by the quality of the compiler, so the question HEIR actually answers is whether a hospital data scientist can try it at all.