mobile: thumbnails load feels slow, possibly due to extra thumbnail generation step

[Atemu]

The bug

Ever since I've started using Immich, I noticed that thumbnails feel quite slow to load to a usable resolution. Setting the option to always load remote thumbnails works around this issue causing thumbnails to load reasonably quick but that has obvious issues.

This phone is a Fairphone 4 which is by no means a particularly fast phone but it certainly isn't a slow phone either. I remember that Google photos never used to have issues like this on the much slower Oneplus 5 I used to use but I don't know how it fares on the FP4 since I stopped using Google photos years ago.

For reference, the Fossify gallery has no such issues on the same phone with the same pictures. Thumbnails take a short moment to appear but stay the background colour in the short moment until they're there. It's really quite quick though, nothing I'd be annoyed about or really notice unless I looked for it.

Not so much with Immich though unfortunately.

A thing I've noticed in this regard is that Immich has another step in the process: it loads a noticeably low-resolution thumbnail first before loading the actual thumbnail.

This might honestly be just a feeling thing where the jump between placeholder to something appearing does not result in the actual usable image to appear. Instead, a super low resolution placeholder appears and it's so low res that you can't tell what's in it in the short moment it takes to load the actual thumbnail that is high resolution enough to tell what's in the picture.

All it really does is delay the actual thumbnail from appearing and I never had a use for it. I honestly think it'd be better to skip this generation step entirely.

This is how it works when forcing remote thumbnails too and that's with the much higher latencies and lower bandwidth of a wireless connection to a network-local server. This alone might be why that feels so much better.

Would it be possible to make generating this middle step thumbnail at least optional?

Another thought that crossed my mind while writing this: The local image has to be read from disk, decoded, downscaled, encoded again and written somewhere to generate the thumbnail.
This generation step obviously takes a noticeable amount of time. Wouldn't it be better to just skip it and show the image directly, in full resolution? I don't know how feasible this is but, in theory, merely decoding the image and scaling the resulting bitmap should be rather cheap as both of those tasks can be HW-accelerated via the video codec HW (mjpeg/h.265) and generic GPU 2D accel respectively. Perhaps worth checking out as that'd save us from generating thumbnails for local files entirely.
If it turns the view too sluggish but is still faster to load than generating thumbnails, Immich could perhaps show the originals until the thumbnails are generated asynchronously and seamlessly swap those in as they finish generating.

The OS that Immich Server is running on

NixOS

Version of Immich Server

v1.116.2

Version of Immich Mobile App

v1.116.2

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

name: immich
services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: 'no'
    healthcheck:
      disable: false
    logging:
      driver: json-file
  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: 'no'
    healthcheck:
      disable: false
    logging:
      driver: json-file
  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
    healthcheck:
      test: redis-cli ping || exit 1
    restart: 'no'
    logging:
      driver: json-file
  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}'
        || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}'
        --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures),
        0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [
        "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user",
        public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB",
      "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: 'no'
    logging:
      driver: json-file
volumes:
  model-cache: null

Your .env content

DB_DATABASE_NAME=immich
DB_DATA_LOCATION=/var/lib/postgresql/data/
DB_HOSTNAME=immich_postgres
DB_PASSWORD=postgres
DB_USERNAME=postgres
IMMICH_VERSION=v1.116.2
REDIS_HOSTNAME=immich_redis
UPLOAD_LOCATION=/var/lib/immich/

Reproduction steps

1. Open App
2. Scroll through pictures quickly
3. Feel sluggish thumbnail loads with a low res thumbnail step in between

Relevant log output

No response

Additional information

(Not yet using the NixOS module, still plain docker-compose.)

[bo0tzz]

I think this is a dupe of #2128, right?

Immich doesn't generate thumbnails on demand, they're created ahead of time so there's no generation step slowing things down here.

[Atemu]

Hm, no that's not quite what I'm experiencing. It doesn't cause lag or anything or takes extremely long, it just takes noticeably longer to load the thumbnails in after starting to load them than a standard gallery app and has this weird in-between step of a super low res thumbnail that does, to me, does not appear to be helpful.

When exactly does the Immich app pre-generate the thumbnails?

Also surely Immich can't pre-generate the thumbnails of all images you could possibly browse ahead of time as those can range into the 10000s, so there must be a limitation on how much and when thumbnails are generated, right?

[mertalev]

Not a mobile dev, but I don't think the app persists the thumbnails it generates. There's a low-res thumbnail that it makes and switches with a high-res thumbnail, and in-memory caches that store the raw decoded data. I don't think there's an encoding step.

Looking at the code here, it seems like it's effectively reading and decoding from disk twice to make those two thumbnails. There's almost no time saved by resizing to 32, 32 vs. width, height, so the low-res thumbnail seems kind of pointless. I guess it makes better use of the in-memory cache since more thumbnails can fit, but unless the cache hit rate is high it's probably slower than just skipping the low-res thumbnail.

[Atemu]

I don't think the app persists the thumbnails it generates.

Oh interesting, I had assumed it does. Perhaps that would be another optimisation.

Although I believe Android provides a thumbnail caching mechanism of its own for local images if I'm not mistaken. That should probably just be used if possible.

Looking at the code here

Ah thank you, I had attempted to take a look around myself.

I don't think there's an encoding step.

I had a look at what I believe to be the source for thumbnailDataWithSize() and it takes the codec aswell as the quality as parameters. We're calling it with the default of jpeg encoder and set quality to 75, so I assume it's encoding to JPEG.
If we truly never cache these thumbnails anywhere, we shouldn't be encoding anything and should just scale down the bitmap of the asset. That should be much faster than encoding two JPEGs.

[mertalev]

Oh, you're right. It's encoding two JPEGs, then decoding them back...

[edbr-xyz]

I also have this issue on my OnePlus 7 Pro.

I notice that images that are only remote (were uploaded from my PC), have nice looking thumbnails that seem to load almost instantly, even while images that are on the device, above and below, are still just 32x32 blurry blocks.

If I then close the app and turn off my WiFi and data, then re-open Immich, the thumbnails of those remote images are still there, and still load instantly.

[Atemu]

What I experience doesn't persist long enough to take a screenshot. What you experience might actually be #2128.

The root of that issue could be similar though.

[edbr-xyz]

That's probably a factor, but Immich could be caching thumbnails for local images, but instead it seems to only do it for the remote ones.

[EinToni]

Hey, may I chip in here? :)
I looked into it and did a performance profiling of this part of the app and it is actually not the decoding that takes so much time, it is the pure loading of the image from disk and rescaling it in asset.local?.thumbnailDataWithSize(...).
I did a bit more testing and this is the result of the time it takes to execute thumbnailDataWithSize with different options:

Size	Quality	Run1 (77 samples)	Run2 (77 samples)	Run3 (77 samples)
32	100	935.65ms	990.52ms	927.40ms
32	95 (default)	568.32ms	617.22ms	575.66ms
32	75	277.81ms	240.09ms	222.96ms
32	15	207.53ms	172.81ms	185.83ms
256	100	484.39ms	497.78ms	498.56ms
256	95 (default)	402.48ms	415.39ms	407.61ms
256	75	123.96ms	116.05ms	128.13ms
256	15	136.68ms	131.49ms	146.57ms

( I highlighted the currently used configurations)

The rest of the _codec method for local thumbnails uses no measurable time.

I did these tests on my pixel 8 running in profiling mode, so quite realistically results.

I would propose to drop the small thumbnail. This is just costing valuable time. The user is waiting ~0.25s to get from the blur to the still bad looking small thumbnail. Only then after another ~0.4s the user gets the actual thumbnail. That is 0.65s on average in total!!
When dropping the small thumbnail the user would only have to wait for ~0.4s to get from the blur straight to the good thumbnail. I think this is way better for the user experience, because the small thumbnail is (in my opinion) not helpful. It is rather just frustrating, because the image appeared to have loaded (the visual move from blur to something better) but it's still so bad that it is basically a blur.

Furthermore the thumbnails are not cached and therefore always recalculated. I would also propose to implement caching here as equally done for the remote thumbnails. The already existent ThumbnailImageCacheManager can be reused for that.

As I have already implemented it for testing purposes I would open a PR in the next days (after a bit of cleanup) if this sounds like an acceptable solution.

[mertalev]

I looked into it and did a performance profiling of this part of the app and it is actually not the decoding that takes so much time, it is the pure loading of the image from disk and rescaling it in asset.local?.thumbnailDataWithSize(...).

I agree with removing the 32x32 thumbnail, but I don't think this is quite accurate. thumbnailDataWithSize is decoding the original image first before encoding it into a small image. If you just tested the time that method takes, you're measuring loading from disk + decoding + resizing + encoding. Also, the timing numbers you shared aren't necessarily representative of what it would be without the 32x32 thumbnail. The second thumbnail is once again loading the original image right afterwards, so it benefits from better caching in the storage layer.

It's also interesting to see that the quality has such a big impact on the timing. JPEG encoding time doesn't generally change much with different Q values, so I wonder what's causing that difference. It's probably worth looking into.

Caching should definitely help as well.

[EinToni]

Thank you for the input.

True, the measurement is not accurate, but I focused more on the real application than on theoretical measurements and wanted to show a relative difference of the different setting... Although I have to confess I made a blunder. I completely neglected the caching of android itself and was with my head just thinking about explicit implemented caching. The measurements I did were all in one row, the strong decrease in time usage from the top to bottom is therefore mainly from caching.

I redid some tests but this time just took the time for all thumbnails to finish generating. Removing the small thumbnail still speed things up, calculating the 77 thumbnails with:

• Only the small thumbnail: 1240
• Only the big thumbnail: 1550
• Both directly after each other: 2500

As you already noted, they are not accurate and just give a rough estimate of the changes. But it is still very noticeable when using it. I used this changed version to browse through my local images and it felt a lot snappier.

[mertalev]

Thanks for retesting! 1550ms vs 2500ms is a huge difference.