{"schemaVersion":1,"id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","title":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","description":"A practical guide to TFHE, Concrete-ML by Zama, computing on encrypted data, real ML inference on ciphertext, and the honest performance reality of homomorphic encryption.","language":"en","datePublished":"2026-02-24","dateModified":null,"contentFormat":"text/markdown","content":"Cryptography\n\n# Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data\n\n2026-02-249 min read\n\n[Lucio Durán](<https://lucioduran.com/>)\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/homomorphic-encryption-tfhe-concrete-ml>)\n\n## Homomorphic Encryption Principles\n\nRegular encryption is like a locked safe. You can store things in it and take things out, but you can't modify the contents without unlocking it first. Homomorphic encryption is like a safe with built-in manipulator arms, you can perform computations on the contents while they're still locked inside.\n\nFormally: given encryptions E(a) and E(b), you can compute E(a + b) and E(a × b) without knowing a or b. That's it. Every computation can be decomposed into additions and multiplications, so this is Turing-complete. The catch is performance.\n\nThere are three levels:\n\n- **Partially Homomorphic Encryption (PHE)**: Supports unlimited operations of ONE type (addition OR multiplication). RSA is multiplicatively homomorphic. Paillier is additively homomorphic. Fast, but limited.\n- **Somewhat Homomorphic Encryption (SHE)**: Supports both addition and multiplication, but only a limited number before the noise overwhelms the result.\n- **Fully Homomorphic Encryption (FHE)**: Supports unlimited additions AND multiplications via \"bootstrapping\", a technique that refreshes the ciphertext noise. This is what TFHE provides.\n\n## TFHE: The Scheme Behind Concrete-ML\n\nTFHE (Torus FHE) represents encrypted data as polynomials over a torus (a circle group). The key innovation is fast bootstrapping, TFHE can refresh ciphertext noise in ~10ms, compared to minutes for older schemes like BGV/BFV.\n\nEach TFHE ciphertext contains:\n\n1. The encrypted value (a polynomial)\n2. Noise (essential for security, but grows with computation)\n3. Parameters (determines security level and noise capacity)\n\nThe noise grows with every operation:\n\n- **Addition**: Noise grows linearly (cheap)\n- **Multiplication**: Noise grows exponentially (expensive)\n- **Bootstrapping**: Resets noise to a fixed level (costs ~10ms but enables further computation)\n\nThis is why multiplicative depth matters. A model with 10 sequential multiplications needs 10 bootstrapping operations minimum. At 10ms each, that's 100ms of pure bootstrapping overhead before counting the actual computation.\n\n## Concrete-ML: FHE for Machine Learning\n\n[Zama](<https://www.zama.ai/>)'s [Concrete-ML](<https://docs.zama.ai/concrete-ml>) wraps TFHE in a scikit-learn-compatible API. You train a model on plaintext data, then \"compile\" it into an FHE circuit:\n\n```python\nfrom concrete.ml.sklearn import LogisticRegression\nimport numpy as np\n\n# Train on plaintext data (this step is normal ML)\nX_train = np.load(\"features_train.npy\") # Shape: (10000, 20)\ny_train = np.load(\"labels_train.npy\") # Shape: (10000,)\n\nmodel = LogisticRegression(n_bits=8) # Quantize to 8-bit integers\nmodel.fit(X_train, y_train)\n\n# Evaluate in plaintext first (sanity check)\nplaintext_accuracy = model.score(X_test, y_test)\nprint(f\"Plaintext accuracy: {plaintext_accuracy:.4f}\")\n# Output: Plaintext accuracy: 0.9234\n\n# Compile the model into an FHE circuit\nfhe_circuit = model.compile(X_train)\n\n# Now we can do inference on encrypted data\n# Step 1: Client encrypts their data\nencrypted_input = fhe_circuit.encrypt(X_test[0:1])\n\n# Step 2: Server runs inference on encrypted data\nencrypted_result = fhe_circuit.run(encrypted_input)\n\n# Step 3: Client decrypts the result\ndecrypted_result = fhe_circuit.decrypt(encrypted_result)\nprint(f\"FHE prediction: {decrypted_result}\")\n# Output: FHE prediction: [1] (same as plaintext prediction)\n```\n\nThe ` n_bits=8 ` parameter is critical. It controls quantization, how many bits represent each value in the model. Lower bits = faster FHE but lower accuracy:\n\n| n\\_bits | Plaintext Accuracy | FHE Inference Time | Speedup vs. 16-bit |\n| --- | --- | --- | --- |\n| 4 | 0.8912 | 45ms | 18x |\n| 6 | 0.9156 | 95ms | 8.5x |\n| 8 | 0.9234 | 180ms | 4.5x |\n| 12 | 0.9241 | 520ms | 1.6x |\n| 16 | 0.9243 | 810ms | 1x |\n\nGoing from 16-bit to 8-bit, we lose 0.09% accuracy but gain 4.5x speed. That's the tradeoff that makes FHE practical.\n\n## The Credit Scoring Pipeline\n\nHere's the architecture I built for the bank:\n\n```\n[Customer App] → [Encrypt features locally] → [Send ciphertext to API]\n ↓\n [FHE Inference Server (AWS)]\n ↓\n [Encrypted result]\n ↓\n[Customer App] ← [Decrypt result locally] ← [Return encrypted prediction]\n```\n\nThe customer's financial data (income, debt-to-income ratio, credit history length, etc.) never leaves their device in plaintext. The bank's model weights are embedded in the compiled FHE circuit, but the customer can't extract them, the circuit is a one-way computation.\n\n```python\n# Server-side: Load the compiled model (no access to cleartext data)\nfrom concrete.ml.deployment import FHEModelServer\n\nserver = FHEModelServer(path_dir=\"./deployed_model\")\n\n@app.post(\"/predict\")\nasync def predict(request: Request):\n encrypted_input = await request.body()\n\n # Run inference on ciphertext, server never sees plaintext\n encrypted_result = server.run(\n serialized_encrypted_quantized_data=encrypted_input,\n serialized_evaluation_keys=get_evaluation_keys(request.headers[\"client-id\"]),\n )\n\n return Response(content=encrypted_result, media_type=\"application/octet-stream\")\n```\n\n```python\n# Client-side: Encrypt, send, decrypt\nfrom concrete.ml.deployment import FHEModelClient\n\nclient = FHEModelClient(path_dir=\"./client_model\", key_dir=\"./keys\")\nclient.generate_private_and_evaluation_keys()\n\n# Prepare the data\nclear_input = np.array([[65000, 0.32, 7, 3, 720, 12000, 0, 1]])\nencrypted_input = client.quantize_encrypt_serialize(clear_input)\n\n# Send evaluation keys (public, needed by server for computation)\nevaluation_keys = client.get_serialized_evaluation_keys()\n\n# ... send encrypted_input and evaluation_keys to server ...\n\n# Receive and decrypt\nencrypted_result = response.content\ndecrypted_prediction = client.deserialize_decrypt_dequantize(encrypted_result)\nprint(f\"Credit score category: {decrypted_prediction}\")\n```\n\n## Custom Neural Networks with Concrete-ML\n\nFor more complex models, you can compile PyTorch modules:\n\n```python\nimport torch\nfrom concrete.ml.torch.compile import compile_torch_model\n\nclass CreditNet(torch.nn.Module):\n def __init__(self):\n super().__init__()\n self.fc1 = torch.nn.Linear(20, 64)\n self.fc2 = torch.nn.Linear(64, 32)\n self.fc3 = torch.nn.Linear(32, 2)\n\n def forward(self, x):\n x = torch.relu(self.fc1(x))\n x = torch.relu(self.fc2(x))\n x = self.fc3(x)\n return x\n\n# Train normally in PyTorch\nmodel = CreditNet()\n# ... training loop ...\n\n# Compile for FHE\nquantized_model = compile_torch_model(\n model,\n X_train,\n n_bits=6, # Aggressive quantization for speed\n rounding_threshold_bits=6, # Additional optimization\n)\n\n# Check the circuit complexity\nprint(f\"Circuit size: {quantized_model.fhe_circuit.size}\")\nprint(f\"Max bit-width: {quantized_model.fhe_circuit.graph.maximum_integer_bit_width()}\")\nprint(f\"Estimated FHE time: {quantized_model.fhe_circuit.complexity}ms\")\n```\n\nThe model architecture matters enormously for FHE performance. Rules I learned the hard way:\n\n1. **ReLU is expensive**, each ReLU requires a comparison, which in FHE means a programmable bootstrapping operation (~10ms each). A network with 96 ReLU activations has ~960ms of bootstrapping overhead.\n2. **Depth over width**, a 3-layer network with 64 neurons per layer is much slower than a 2-layer network with 128 neurons, because depth adds sequential bootstrapping.\n3. **Avoid max/min/argmax**, these require multiple comparisons. Use average pooling instead of max pooling.\n4. **Batch normalization is free**, it fuses into the preceding linear layer during compilation. Always use it.\n\n## The Performance Reality\n\nLet me be completely transparent about what's fast and what's not:\n\n```python\nimport time\nfrom concrete.ml.sklearn import (\n LogisticRegression, DecisionTreeClassifier,\n XGBClassifier, LinearSVR\n)\n\nmodels = {\n \"Logistic Regression (8-bit)\": LogisticRegression(n_bits=8),\n \"Decision Tree (depth=5, 8-bit)\": DecisionTreeClassifier(n_bits=8, max_depth=5),\n \"XGBoost (10 trees, depth=4)\": XGBClassifier(n_bits=6, max_depth=4, n_estimators=10),\n \"Linear SVR (8-bit)\": LinearSVR(n_bits=8),\n}\n\nfor name, model in models.items():\n model.fit(X_train, y_train)\n circuit = model.compile(X_train)\n\n # Benchmark FHE inference\n encrypted = circuit.encrypt(X_test[0:1])\n start = time.time()\n result = circuit.run(encrypted)\n fhe_time = time.time() - start\n\n # Compare with plaintext\n start = time.time()\n _ = model.predict(X_test[0:1])\n plain_time = time.time() - start\n\n accuracy = model.score(X_test, y_test)\n print(f\"{name}: acc={accuracy:.4f}, FHE={fhe_time*1000:.0f}ms, plain={plain_time*1000:.2f}ms\")\n```\n\nResults on an [AWS](<https://aws.com/>) c6i.2xlarge (8 vCPUs, no GPU acceleration):\n\n| Model | Accuracy | FHE Inference | Plaintext | Slowdown |\n| --- | --- | --- | --- | --- |\n| Logistic Regression | 0.923 | 85ms | 0.02ms | 4,250x |\n| Decision Tree (depth=5) | 0.941 | 210ms | 0.01ms | 21,000x |\n| XGBoost (10 trees, depth=4) | 0.956 | 1,240ms | 0.08ms | 15,500x |\n| Linear SVR | 0.917 | 72ms | 0.01ms | 7,200x |\n| Custom NN (2-layer, 64 units) | 0.948 | 3,400ms | 0.15ms | 22,667x |\n\nThat 4,250x slowdown for logistic regression translates to 85ms, completely viable for real-time API calls. The XGBoost at 1.2 seconds is fine for batch processing. The neural network at 3.4 seconds is borderline, acceptable for some use cases, too slow for others.\n\n## What's Not Possible (Yet)\n\nLet me save you weeks of wasted effort:\n\n- **Large language models**: A GPT-2 size model would take hours per token. Not happening.\n- **Image classification with deep CNNs**: ResNet-50 inference would be ~45 minutes. Even a tiny MNIST classifier takes 8-12 seconds.\n- **Anything requiring dynamic control flow**: FHE circuits are static. You can't branch on encrypted values (that would leak information).\n- **Large feature spaces**: 1000+ features means 1000+ encrypted values to process. Memory becomes the bottleneck before compute.\n\n## Key Sizes and Network Transfer\n\nFHE has a data expansion problem. A 20-dimensional feature vector that's 160 bytes in plaintext becomes:\n\n```\nEncrypted input: ~42 KB\nEvaluation keys: ~158 MB (sent once per client, cached by server)\nEncrypted output: ~4 KB\n```\n\nThat 158 MB evaluation key is the elephant in the room. It needs to be sent from the client to the server once per session. On a fast connection, that's a few seconds of upload. On mobile, it's a deal-breaker without aggressive key compression:\n\n```python\n# Enable key compression, reduces eval keys by ~4x\ncircuit = model.compile(\n X_train,\n configuration=fhe.Configuration(\n enable_key_compression=True,\n compress_evaluation_keys=True,\n ),\n)\n# Compressed eval keys: ~40 MB (still large, but manageable)\n```\n\n## The Architecture That Works\n\nAfter iterating through several designs, here's what worked for the bank deployment:\n\n```python\n# Deployment architecture\n\"\"\"\n1. Client generates keys once, stores locally\n2. Evaluation keys uploaded to server once (cached with TTL)\n3. Each inference request sends only encrypted input (~42 KB)\n4. Server returns encrypted result (~4 KB)\n5. Client decrypts locally\n\nTotal per-request data transfer: ~46 KB\nLatency budget:\n - Network (encrypted input): 15ms\n - FHE inference (server): 340ms\n - Network (encrypted result): 5ms\n - Client decrypt: 2ms\n Total: ~362ms\n\"\"\"\n```\n\nThe 340ms inference time is for the XGBoost model that the bank actually chose, 15 trees at depth 4, quantized to 6 bits, processing 20 financial features. Accuracy is 95.2% on their holdout set, compared to 96.1% for the plaintext model. The bank accepted that 0.9% accuracy trade-off because FHE let them process customer data from a partner institution without data sharing agreements, which would have taken 8 months of legal review.\n\n## Where This Is Going\n\nTFHE hardware acceleration is coming. Zama is building custom ASICs that promise 100-1000x speedup over CPU-based FHE. Intel's HEXL library already provides 2-4x speedups using AVX-512 instructions. GPU acceleration through CUDA-based TFHE implementations shows 10-50x improvements.\n\nWhen hardware catches up, and it will, because the market demand is clear, a 340ms inference will become 3ms. At that point, FHE becomes viable for real-time applications across the board: encrypted search, private recommendation systems, confidential medical diagnosis, and yes, even large model inference.\n\nFor now, the sweet spot is clear: small-to-medium models (logistic regression, gradient boosting, shallow networks) on structured data with fewer than 100 features. If your use case fits that profile and you need mathematical privacy guarantees stronger than \"we promise not to look at your data,\" homomorphic encryption is ready.\n\n## Frequently asked questions\n\n### Is homomorphic encryption actually usable in production?\n\nFor specific, bounded use cases, yes. ML inference on small models (logistic regression, small decision trees, shallow neural networks) runs in 100ms-10s range on modern hardware, which is acceptable for batch processing and some real-time scenarios. Large neural networks (BERT, GPT-class) are not feasible, they'd take hours per inference.\n\n### How much slower is FHE compared to plaintext computation?\n\nFor TFHE (the scheme Concrete-ML uses), expect 1,000x to 1,000,000x slowdown depending on the operation and bit-width. Addition is cheap (~1,000x). Multiplication is expensive (~10,000x). Deep neural network inference compounds these costs through many sequential multiplications.\n\n### Can the server learn anything about my data during FHE computation?\n\nNo. The server only sees ciphertext throughout the computation. It cannot distinguish between two different encrypted inputs, cannot extract partial information, and the computation result is also encrypted, only the client with the secret key can decrypt the output. This is provable under standard cryptographic assumptions (Learning With Errors).\n\n## Tips\n\n1. **LuchoTip:** Quantize your model to 8-bit integers or lower before compiling with Concrete-ML. FHE operations on 16-bit values are ~16x slower than 8-bit because the noise growth is exponential with bit-width. Most ML models lose less than 1% accuracy going from float32 to int8, but gain 10-20x FHE speedup.\n2. **LuchoTip:** Use ` fhe.Compiler ` with ` show_mlir=True ` to inspect the generated MLIR intermediate representation. If you see chains of ` tensor.extract ` operations, your model has memory access patterns that TFHE can't parallelize. Restructure to use contiguous tensor ops instead.\n3. **LuchoTip:** Start with Concrete-ML's built-in models (` LogisticRegression `, ` XGBClassifier `) before attempting custom neural networks. The built-in models are optimized for FHE, they produce circuits with minimal multiplicative depth. A custom PyTorch model compiled naively will be 50-100x slower.\n\n- homomorphic-encryption\n- tfhe\n- concrete-ml\n- zama\n- privacy\n- encrypted-computation\n\n[Back to Blog](<https://lucioduran.com/blog>)","headings":[{"level":1,"text":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#title"},{"level":2,"text":"Homomorphic Encryption Principles","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#homomorphic-encryption-principles"},{"level":2,"text":"TFHE: The Scheme Behind Concrete-ML","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#tfhe-the-scheme-behind-concrete-ml"},{"level":2,"text":"Concrete-ML: FHE for Machine Learning","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#concrete-ml-fhe-for-machine-learning"},{"level":2,"text":"The Credit Scoring Pipeline","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-credit-scoring-pipeline"},{"level":2,"text":"Custom Neural Networks with Concrete-ML","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#custom-neural-networks-with-concrete-ml"},{"level":2,"text":"The Performance Reality","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-performance-reality"},{"level":2,"text":"What's Not Possible (Yet)","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#whats-not-possible-yet"},{"level":2,"text":"Key Sizes and Network Transfer","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#key-sizes-and-network-transfer"},{"level":2,"text":"The Architecture That Works","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-architecture-that-works"},{"level":2,"text":"Where This Is Going","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#where-this-is-going"},{"level":2,"text":"Frequently asked questions","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#questions"},{"level":3,"text":"Is homomorphic encryption actually usable in production?","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-1"},{"level":3,"text":"How much slower is FHE compared to plaintext computation?","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-2"},{"level":3,"text":"Can the server learn anything about my data during FHE computation?","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-3"},{"level":2,"text":"Tips","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#article-tips"}],"sections":[{"heading":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","level":1,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","content":"Cryptography"},{"heading":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","level":1,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#title","content":"2026-02-249 min read\n\n[Lucio Durán](<https://lucioduran.com/>)\nEngineering Manager & AI Solutions Architect\n\nAlso available in: [Español](<https://lucioduran.com/es/blog/homomorphic-encryption-tfhe-concrete-ml>)"},{"heading":"Homomorphic Encryption Principles","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#homomorphic-encryption-principles","content":"Regular encryption is like a locked safe. You can store things in it and take things out, but you can't modify the contents without unlocking it first. Homomorphic encryption is like a safe with built-in manipulator arms, you can perform computations on the contents while they're still locked inside.\n\nFormally: given encryptions E(a) and E(b), you can compute E(a + b) and E(a × b) without knowing a or b. That's it. Every computation can be decomposed into additions and multiplications, so this is Turing-complete. The catch is performance.\n\nThere are three levels:\n\n- **Partially Homomorphic Encryption (PHE)**: Supports unlimited operations of ONE type (addition OR multiplication). RSA is multiplicatively homomorphic. Paillier is additively homomorphic. Fast, but limited.\n- **Somewhat Homomorphic Encryption (SHE)**: Supports both addition and multiplication, but only a limited number before the noise overwhelms the result.\n- **Fully Homomorphic Encryption (FHE)**: Supports unlimited additions AND multiplications via \"bootstrapping\", a technique that refreshes the ciphertext noise. This is what TFHE provides."},{"heading":"TFHE: The Scheme Behind Concrete-ML","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#tfhe-the-scheme-behind-concrete-ml","content":"TFHE (Torus FHE) represents encrypted data as polynomials over a torus (a circle group). The key innovation is fast bootstrapping, TFHE can refresh ciphertext noise in ~10ms, compared to minutes for older schemes like BGV/BFV.\n\nEach TFHE ciphertext contains:\n\n1. The encrypted value (a polynomial)\n2. Noise (essential for security, but grows with computation)\n3. Parameters (determines security level and noise capacity)\n\nThe noise grows with every operation:\n\n- **Addition**: Noise grows linearly (cheap)\n- **Multiplication**: Noise grows exponentially (expensive)\n- **Bootstrapping**: Resets noise to a fixed level (costs ~10ms but enables further computation)\n\nThis is why multiplicative depth matters. A model with 10 sequential multiplications needs 10 bootstrapping operations minimum. At 10ms each, that's 100ms of pure bootstrapping overhead before counting the actual computation."},{"heading":"Concrete-ML: FHE for Machine Learning","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#concrete-ml-fhe-for-machine-learning","content":"[Zama](<https://www.zama.ai/>)'s [Concrete-ML](<https://docs.zama.ai/concrete-ml>) wraps TFHE in a scikit-learn-compatible API. You train a model on plaintext data, then \"compile\" it into an FHE circuit:\n\n```python\nfrom concrete.ml.sklearn import LogisticRegression\nimport numpy as np\n\n# Train on plaintext data (this step is normal ML)\nX_train = np.load(\"features_train.npy\") # Shape: (10000, 20)\ny_train = np.load(\"labels_train.npy\") # Shape: (10000,)\n\nmodel = LogisticRegression(n_bits=8) # Quantize to 8-bit integers\nmodel.fit(X_train, y_train)\n\n# Evaluate in plaintext first (sanity check)\nplaintext_accuracy = model.score(X_test, y_test)\nprint(f\"Plaintext accuracy: {plaintext_accuracy:.4f}\")\n# Output: Plaintext accuracy: 0.9234\n\n# Compile the model into an FHE circuit\nfhe_circuit = model.compile(X_train)\n\n# Now we can do inference on encrypted data\n# Step 1: Client encrypts their data\nencrypted_input = fhe_circuit.encrypt(X_test[0:1])\n\n# Step 2: Server runs inference on encrypted data\nencrypted_result = fhe_circuit.run(encrypted_input)\n\n# Step 3: Client decrypts the result\ndecrypted_result = fhe_circuit.decrypt(encrypted_result)\nprint(f\"FHE prediction: {decrypted_result}\")\n# Output: FHE prediction: [1] (same as plaintext prediction)\n```\n\nThe ` n_bits=8 ` parameter is critical. It controls quantization, how many bits represent each value in the model. Lower bits = faster FHE but lower accuracy:\n\n| n\\_bits | Plaintext Accuracy | FHE Inference Time | Speedup vs. 16-bit |\n| --- | --- | --- | --- |\n| 4 | 0.8912 | 45ms | 18x |\n| 6 | 0.9156 | 95ms | 8.5x |\n| 8 | 0.9234 | 180ms | 4.5x |\n| 12 | 0.9241 | 520ms | 1.6x |\n| 16 | 0.9243 | 810ms | 1x |\n\nGoing from 16-bit to 8-bit, we lose 0.09% accuracy but gain 4.5x speed. That's the tradeoff that makes FHE practical."},{"heading":"The Credit Scoring Pipeline","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-credit-scoring-pipeline","content":"Here's the architecture I built for the bank:\n\n```\n[Customer App] → [Encrypt features locally] → [Send ciphertext to API]\n ↓\n [FHE Inference Server (AWS)]\n ↓\n [Encrypted result]\n ↓\n[Customer App] ← [Decrypt result locally] ← [Return encrypted prediction]\n```\n\nThe customer's financial data (income, debt-to-income ratio, credit history length, etc.) never leaves their device in plaintext. The bank's model weights are embedded in the compiled FHE circuit, but the customer can't extract them, the circuit is a one-way computation.\n\n```python\n# Server-side: Load the compiled model (no access to cleartext data)\nfrom concrete.ml.deployment import FHEModelServer\n\nserver = FHEModelServer(path_dir=\"./deployed_model\")\n\n@app.post(\"/predict\")\nasync def predict(request: Request):\n encrypted_input = await request.body()\n\n # Run inference on ciphertext, server never sees plaintext\n encrypted_result = server.run(\n serialized_encrypted_quantized_data=encrypted_input,\n serialized_evaluation_keys=get_evaluation_keys(request.headers[\"client-id\"]),\n )\n\n return Response(content=encrypted_result, media_type=\"application/octet-stream\")\n```\n\n```python\n# Client-side: Encrypt, send, decrypt\nfrom concrete.ml.deployment import FHEModelClient\n\nclient = FHEModelClient(path_dir=\"./client_model\", key_dir=\"./keys\")\nclient.generate_private_and_evaluation_keys()\n\n# Prepare the data\nclear_input = np.array([[65000, 0.32, 7, 3, 720, 12000, 0, 1]])\nencrypted_input = client.quantize_encrypt_serialize(clear_input)\n\n# Send evaluation keys (public, needed by server for computation)\nevaluation_keys = client.get_serialized_evaluation_keys()\n\n# ... send encrypted_input and evaluation_keys to server ...\n\n# Receive and decrypt\nencrypted_result = response.content\ndecrypted_prediction = client.deserialize_decrypt_dequantize(encrypted_result)\nprint(f\"Credit score category: {decrypted_prediction}\")\n```"},{"heading":"Custom Neural Networks with Concrete-ML","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#custom-neural-networks-with-concrete-ml","content":"For more complex models, you can compile PyTorch modules:\n\n```python\nimport torch\nfrom concrete.ml.torch.compile import compile_torch_model\n\nclass CreditNet(torch.nn.Module):\n def __init__(self):\n super().__init__()\n self.fc1 = torch.nn.Linear(20, 64)\n self.fc2 = torch.nn.Linear(64, 32)\n self.fc3 = torch.nn.Linear(32, 2)\n\n def forward(self, x):\n x = torch.relu(self.fc1(x))\n x = torch.relu(self.fc2(x))\n x = self.fc3(x)\n return x\n\n# Train normally in PyTorch\nmodel = CreditNet()\n# ... training loop ...\n\n# Compile for FHE\nquantized_model = compile_torch_model(\n model,\n X_train,\n n_bits=6, # Aggressive quantization for speed\n rounding_threshold_bits=6, # Additional optimization\n)\n\n# Check the circuit complexity\nprint(f\"Circuit size: {quantized_model.fhe_circuit.size}\")\nprint(f\"Max bit-width: {quantized_model.fhe_circuit.graph.maximum_integer_bit_width()}\")\nprint(f\"Estimated FHE time: {quantized_model.fhe_circuit.complexity}ms\")\n```\n\nThe model architecture matters enormously for FHE performance. Rules I learned the hard way:\n\n1. **ReLU is expensive**, each ReLU requires a comparison, which in FHE means a programmable bootstrapping operation (~10ms each). A network with 96 ReLU activations has ~960ms of bootstrapping overhead.\n2. **Depth over width**, a 3-layer network with 64 neurons per layer is much slower than a 2-layer network with 128 neurons, because depth adds sequential bootstrapping.\n3. **Avoid max/min/argmax**, these require multiple comparisons. Use average pooling instead of max pooling.\n4. **Batch normalization is free**, it fuses into the preceding linear layer during compilation. Always use it."},{"heading":"The Performance Reality","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-performance-reality","content":"Let me be completely transparent about what's fast and what's not:\n\n```python\nimport time\nfrom concrete.ml.sklearn import (\n LogisticRegression, DecisionTreeClassifier,\n XGBClassifier, LinearSVR\n)\n\nmodels = {\n \"Logistic Regression (8-bit)\": LogisticRegression(n_bits=8),\n \"Decision Tree (depth=5, 8-bit)\": DecisionTreeClassifier(n_bits=8, max_depth=5),\n \"XGBoost (10 trees, depth=4)\": XGBClassifier(n_bits=6, max_depth=4, n_estimators=10),\n \"Linear SVR (8-bit)\": LinearSVR(n_bits=8),\n}\n\nfor name, model in models.items():\n model.fit(X_train, y_train)\n circuit = model.compile(X_train)\n\n # Benchmark FHE inference\n encrypted = circuit.encrypt(X_test[0:1])\n start = time.time()\n result = circuit.run(encrypted)\n fhe_time = time.time() - start\n\n # Compare with plaintext\n start = time.time()\n _ = model.predict(X_test[0:1])\n plain_time = time.time() - start\n\n accuracy = model.score(X_test, y_test)\n print(f\"{name}: acc={accuracy:.4f}, FHE={fhe_time*1000:.0f}ms, plain={plain_time*1000:.2f}ms\")\n```\n\nResults on an [AWS](<https://aws.com/>) c6i.2xlarge (8 vCPUs, no GPU acceleration):\n\n| Model | Accuracy | FHE Inference | Plaintext | Slowdown |\n| --- | --- | --- | --- | --- |\n| Logistic Regression | 0.923 | 85ms | 0.02ms | 4,250x |\n| Decision Tree (depth=5) | 0.941 | 210ms | 0.01ms | 21,000x |\n| XGBoost (10 trees, depth=4) | 0.956 | 1,240ms | 0.08ms | 15,500x |\n| Linear SVR | 0.917 | 72ms | 0.01ms | 7,200x |\n| Custom NN (2-layer, 64 units) | 0.948 | 3,400ms | 0.15ms | 22,667x |\n\nThat 4,250x slowdown for logistic regression translates to 85ms, completely viable for real-time API calls. The XGBoost at 1.2 seconds is fine for batch processing. The neural network at 3.4 seconds is borderline, acceptable for some use cases, too slow for others."},{"heading":"What's Not Possible (Yet)","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#whats-not-possible-yet","content":"Let me save you weeks of wasted effort:\n\n- **Large language models**: A GPT-2 size model would take hours per token. Not happening.\n- **Image classification with deep CNNs**: ResNet-50 inference would be ~45 minutes. Even a tiny MNIST classifier takes 8-12 seconds.\n- **Anything requiring dynamic control flow**: FHE circuits are static. You can't branch on encrypted values (that would leak information).\n- **Large feature spaces**: 1000+ features means 1000+ encrypted values to process. Memory becomes the bottleneck before compute."},{"heading":"Key Sizes and Network Transfer","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#key-sizes-and-network-transfer","content":"FHE has a data expansion problem. A 20-dimensional feature vector that's 160 bytes in plaintext becomes:\n\n```\nEncrypted input: ~42 KB\nEvaluation keys: ~158 MB (sent once per client, cached by server)\nEncrypted output: ~4 KB\n```\n\nThat 158 MB evaluation key is the elephant in the room. It needs to be sent from the client to the server once per session. On a fast connection, that's a few seconds of upload. On mobile, it's a deal-breaker without aggressive key compression:\n\n```python\n# Enable key compression, reduces eval keys by ~4x\ncircuit = model.compile(\n X_train,\n configuration=fhe.Configuration(\n enable_key_compression=True,\n compress_evaluation_keys=True,\n ),\n)\n# Compressed eval keys: ~40 MB (still large, but manageable)\n```"},{"heading":"The Architecture That Works","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#the-architecture-that-works","content":"After iterating through several designs, here's what worked for the bank deployment:\n\n```python\n# Deployment architecture\n\"\"\"\n1. Client generates keys once, stores locally\n2. Evaluation keys uploaded to server once (cached with TTL)\n3. Each inference request sends only encrypted input (~42 KB)\n4. Server returns encrypted result (~4 KB)\n5. Client decrypts locally\n\nTotal per-request data transfer: ~46 KB\nLatency budget:\n - Network (encrypted input): 15ms\n - FHE inference (server): 340ms\n - Network (encrypted result): 5ms\n - Client decrypt: 2ms\n Total: ~362ms\n\"\"\"\n```\n\nThe 340ms inference time is for the XGBoost model that the bank actually chose, 15 trees at depth 4, quantized to 6 bits, processing 20 financial features. Accuracy is 95.2% on their holdout set, compared to 96.1% for the plaintext model. The bank accepted that 0.9% accuracy trade-off because FHE let them process customer data from a partner institution without data sharing agreements, which would have taken 8 months of legal review."},{"heading":"Where This Is Going","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#where-this-is-going","content":"TFHE hardware acceleration is coming. Zama is building custom ASICs that promise 100-1000x speedup over CPU-based FHE. Intel's HEXL library already provides 2-4x speedups using AVX-512 instructions. GPU acceleration through CUDA-based TFHE implementations shows 10-50x improvements.\n\nWhen hardware catches up, and it will, because the market demand is clear, a 340ms inference will become 3ms. At that point, FHE becomes viable for real-time applications across the board: encrypted search, private recommendation systems, confidential medical diagnosis, and yes, even large model inference.\n\nFor now, the sweet spot is clear: small-to-medium models (logistic regression, gradient boosting, shallow networks) on structured data with fewer than 100 features. If your use case fits that profile and you need mathematical privacy guarantees stronger than \"we promise not to look at your data,\" homomorphic encryption is ready."},{"heading":"Frequently asked questions","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#questions","content":""},{"heading":"Is homomorphic encryption actually usable in production?","level":3,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-1","content":"For specific, bounded use cases, yes. ML inference on small models (logistic regression, small decision trees, shallow neural networks) runs in 100ms-10s range on modern hardware, which is acceptable for batch processing and some real-time scenarios. Large neural networks (BERT, GPT-class) are not feasible, they'd take hours per inference."},{"heading":"How much slower is FHE compared to plaintext computation?","level":3,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-2","content":"For TFHE (the scheme Concrete-ML uses), expect 1,000x to 1,000,000x slowdown depending on the operation and bit-width. Addition is cheap (~1,000x). Multiplication is expensive (~10,000x). Deep neural network inference compounds these costs through many sequential multiplications."},{"heading":"Can the server learn anything about my data during FHE computation?","level":3,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#question-3","content":"No. The server only sees ciphertext throughout the computation. It cannot distinguish between two different encrypted inputs, cannot extract partial information, and the computation result is also encrypted, only the client with the secret key can decrypt the output. This is provable under standard cryptographic assumptions (Learning With Errors)."},{"heading":"Tips","level":2,"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#article-tips","content":"1. **LuchoTip:** Quantize your model to 8-bit integers or lower before compiling with Concrete-ML. FHE operations on 16-bit values are ~16x slower than 8-bit because the noise growth is exponential with bit-width. Most ML models lose less than 1% accuracy going from float32 to int8, but gain 10-20x FHE speedup.\n2. **LuchoTip:** Use ` fhe.Compiler ` with ` show_mlir=True ` to inspect the generated MLIR intermediate representation. If you see chains of ` tensor.extract ` operations, your model has memory access patterns that TFHE can't parallelize. Restructure to use contiguous tensor ops instead.\n3. **LuchoTip:** Start with Concrete-ML's built-in models (` LogisticRegression `, ` XGBClassifier `) before attempting custom neural networks. The built-in models are optimized for FHE, they produce circuits with minimal multiplicative depth. A custom PyTorch model compiled naively will be 50-100x slower.\n\n- homomorphic-encryption\n- tfhe\n- concrete-ml\n- zama\n- privacy\n- encrypted-computation\n\n[Back to Blog](<https://lucioduran.com/blog>)"}],"links":[{"text":"Lucio Durán","url":"https://lucioduran.com/"},{"text":"Español","url":"https://lucioduran.com/es/blog/homomorphic-encryption-tfhe-concrete-ml"},{"text":"Zama","url":"https://www.zama.ai/"},{"text":"Concrete-ML","url":"https://docs.zama.ai/concrete-ml"},{"text":"AWS","url":"https://aws.com/"},{"text":"Back to Blog","url":"https://lucioduran.com/blog"}],"structuredData":[{"@context":"https://schema.org","@graph":[{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},{"@type":"WebSite","@id":"https://lucioduran.com/#website","url":"https://lucioduran.com/","name":"Lucio Durán","inLanguage":["en","es"],"publisher":{"@id":"https://lucioduran.com/#person"}},{"@type":"WebPage","@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#page","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","name":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data | Lucio Durán","description":"A practical guide to TFHE, Concrete-ML by Zama, computing on encrypted data, real ML inference on ciphertext, and the honest performance reality of homomorphic encryption.","inLanguage":"en","author":{"@id":"https://lucioduran.com/#person"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"},"isAccessibleForFree":true,"datePublished":"2026-02-24","mainEntity":{"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#article"},"breadcrumb":{"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#breadcrumb"}},{"@type":"BlogPosting","@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#article","mainEntityOfPage":{"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#page"},"headline":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","description":"A practical guide to TFHE, Concrete-ML by Zama, computing on encrypted data, real ML inference on ciphertext, and the honest performance reality of homomorphic encryption.","datePublished":"2026-02-24","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com"},"publisher":{"@id":"https://lucioduran.com/#person"},"url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog#blog"},"articleSection":"Cryptography","keywords":"homomorphic-encryption, tfhe, concrete-ml, zama, privacy, encrypted-computation","wordCount":1587,"timeRequired":"PT9M","workTranslation":[{"@type":"BlogPosting","@id":"https://lucioduran.com/es/blog/homomorphic-encryption-tfhe-concrete-ml#article","inLanguage":"es","url":"https://lucioduran.com/es/blog/homomorphic-encryption-tfhe-concrete-ml"}],"isAccessibleForFree":true},{"@type":"Blog","@id":"https://lucioduran.com/blog#blog","url":"https://lucioduran.com/blog","name":"Blog","inLanguage":"en","author":{"@type":"Person","@id":"https://lucioduran.com/#person","name":"Lucio Durán","url":"https://lucioduran.com/"},"publisher":{"@id":"https://lucioduran.com/#person"},"isPartOf":{"@id":"https://lucioduran.com/#website"}},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://lucioduran.com/"},{"@type":"ListItem","position":2,"name":"Blog","item":"https://lucioduran.com/blog"},{"@type":"ListItem","position":3,"name":"Homomorphic Encryption in Practice: TFHE, Concrete-ML, and ML Inference on Encrypted Data","item":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml"}],"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#breadcrumb"},{"@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Is homomorphic encryption actually usable in production?","acceptedAnswer":{"@type":"Answer","text":"For specific, bounded use cases, yes. ML inference on small models (logistic regression, small decision trees, shallow neural networks) runs in 100ms-10s range on modern hardware, which is acceptable for batch processing and some real-time scenarios. Large neural networks (BERT, GPT-class) are not feasible, they'd take hours per inference."}},{"@type":"Question","name":"How much slower is FHE compared to plaintext computation?","acceptedAnswer":{"@type":"Answer","text":"For TFHE (the scheme Concrete-ML uses), expect 1,000x to 1,000,000x slowdown depending on the operation and bit-width. Addition is cheap (~1,000x). Multiplication is expensive (~10,000x). Deep neural network inference compounds these costs through many sequential multiplications."}},{"@type":"Question","name":"Can the server learn anything about my data during FHE computation?","acceptedAnswer":{"@type":"Answer","text":"No. The server only sees ciphertext throughout the computation. It cannot distinguish between two different encrypted inputs, cannot extract partial information, and the computation result is also encrypted, only the client with the secret key can decrypt the output. This is provable under standard cryptographic assumptions (Learning With Errors)."}}],"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#questions","url":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#questions","inLanguage":"en","isPartOf":{"@id":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml#page"},"author":{"@id":"https://lucioduran.com/#person"}}]}],"contentHash":"dc8ce0e1ee37990de08aedfddaeb432769e510bab733b1b4634a4f07a510f6fc","provenance":{"method":"Deterministic serialization of the server-rendered main element","source":"https://lucioduran.com/blog/homomorphic-encryption-tfhe-concrete-ml","hashAlgorithm":"sha256","hashScope":"JSON document fields before contentHash and provenance","sourceHTMLBytes":75603,"contentBytes":15217}}