[Fork] Pascal GPU Support for Immich ML (GTX 10-series)

[stephen94125]

Pascal GPU Support Fork for Immich Machine Learning

Problem Statement

Starting from Immich v2.3.1, the official machine learning Docker image switched to CUDA 12 + cuDNN 9 + ONNX Runtime 1.20.1, which requires GPU compute capability 7.0 or higher. This change breaks compatibility with Pascal architecture GPUs (compute capability 6.1), including:

• GTX 1050 / 1050 Ti
• GTX 1060
• GTX 1070 / 1070 Ti
• GTX 1080 / 1080 Ti
• Titan X / Titan Xp

Users with these GPUs experience crashes when attempting to use machine learning features (facial recognition, CLIP search, object detection, etc.).

Solution

I've created a fork that restores full ML functionality for Pascal GPUs by:

1. CUDA 11.8 Downgrade

• Base image: nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04
• ONNX Runtime GPU: pinned to 1.17.1 (last version supporting CUDA 11.8)
• Modified files: Dockerfile, pyproject.toml, uv.lock

2. ONNX Model Patching Toolkit

Newer models (e.g., PP-OCRv5) use ONNX IR version 10, which is incompatible with ONNX Runtime 1.17.1. The toolkit includes:

• audit_onnx_compatibility.py - Scan models and report IR versions
• patch_onnx_models.py - Automatically downgrade IR v10 → v9
• auto_patch_immich_ml.sh - One-click orchestration script for the entire workflow
• download_models.py - Fetch models from ModelScope (alternative source)

3. Kubernetes Integration

The auto-patch script can:

• Fetch models directly from running Immich ML pods via kubectl
• Patch incompatible models in-place
• Upload patched models back to the pod
• Trigger cache reload without downtime

Repository

🔗 Fork URL: https://github.com/stephen94125/immich

📚 Documentation: See README.md for detailed setup instructions

Quick Start

Building the Image

docker build -t immich-machine-learning:pascal \
  -f machine-learning/Dockerfile \
  machine-learning/

Using the Auto-Patch Script

cd machine-learning/tools
chmod +x auto_patch_immich_ml.sh
./auto_patch_immich_ml.sh

The script will automatically:

1. Download models from your Kubernetes pod
2. Audit ONNX IR versions
3. Patch incompatible models
4. Upload back to the pod
5. Reload the model cache

Docker Compose Integration

services:
  immich-machine-learning:
    image: immich-machine-learning:pascal
    volumes:
      - model-cache:/cache

Testing Results

✅ Tested Environment:

• GPU: NVIDIA GTX 1070
• CUDA Version: 11.8
• Immich Version: v2.3.1
• All ML features working: facial recognition, CLIP search, OCR, object detection

Performance Notes

• Model loading times: comparable to CUDA 12 builds
• Inference speed: no significant difference observed
• Memory usage: similar to official build

Related Commits

• ee129a6 - CUDA 11.8 downgrade
• 2bb5bbe - ONNX patching toolkit
• 1156724 - kubectl cp path fix

Why Share This?

I believe many Immich users still have Pascal GPUs that are perfectly capable of running ML workloads. This fork allows them to continue using all features without needing hardware upgrades.

Feedback Welcome

If you're using Pascal GPUs and try this fork, please share your experience! I'm particularly interested in:

• Other Pascal GPU models (GTX 1050, 1060, 1080, etc.)
• Performance metrics
• Any issues encountered

Disclaimer

This is an unofficial fork maintained independently. For the official Immich project, please visit https://immich.app or the main repository at https://github.com/immich-app/immich.

---

License: AGPLv3 (inherited from upstream Immich project)

[bo0tzz]

@mertalev is this a regression in our supported platforms?

[mertalev]

Everything in this post is LLM hallucination. The kernel of truth here is that bumping ONNX Runtime might have bumped the minimum compute capability to 7.5 (Turing) based on what they currently target in their repo's Dockerfiles. We can look into this.

[goalie2002]

FWIW I’m running ML on pascal (gtx 1080) on 2.3.1 and it works just fine

[stephen94125]

I'm not entirely sure, but based on the information I found, CUDA 12 seems to have some issues with supporting the Pascal architecture under ONNX Runtime.
I can also confirm that when running the official CUDA image on a Debian 12 system with Nvidia Driver 535.247.01 and a GTX 1070, it results in an endless segmentation fault and falls back to CPU computation.

So, following the AI's suggestion, I downgraded the CUDA version to 11 and made the corresponding adjustments, and it started running normally afterward.
If I’ve overlooked anything, please feel free to point it out.

[bo0tzz]

Have you tried using a more recent driver?

[besongit]

I tried with a GTX 1050 but unfortunately it doesn't seem to work, not being an expert I asked ChatGpt to help me by entering this Fix but after a few attempts it tells me "Setting execution providers to ['CPUExecutionProvider']"

[stephen94125]

Do you have more detailed logs and configuration files?

I'm running it using k3s, and I also need to configure NVIDIA Assistant and the runtime.

[besongit]

Unfortunately i no longer have the card as it was lent to me for testing. In the end, since I have to buy one anyway, I'll spend a little more and get a more compatible model.