Successfully sync an iCloud Library to self hosted Immich in an airtight env. #21251
Replies: 1 comment 6 replies
-
|
Hi, thanks for a great job. I want to clarify a couple of points. First, you said that
but when I export files unchanged from the macOS Photos application itself, xmp sidecars are also created there. Or they really may not store some important information (adjusted datetime, or manual geotag). Secondly, I have quite a lot of photos taken on the iPhone, but there was incorrect information about the location, and I manually corrected it through macOS Photos. Does this affect the creation of duplicates in Immich when syncing already via an iOS device? And third, I have ~800 GB of photos and videos, that is, 42,000 items in iCloud. Let's say osxphotos is a mandatory part of the library migration, is it possible to split the synchronization anyway, at least for each year? I don't have a drive big enough to hold two copies of the library. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
How I set up my docker to avoid opening ports
Prerequisites:
npm_proxy)note that I am on a Lattepanda with intel chip, some settings might differ on your machine specially related to ML and Accel
docker-compose.yml:.envhwaccel.ml.ymlhwaccel.transcoding.ymlSpin up with
docker compose up -dand setup your instance (admin user and other users). Make sure to set VAAPI in the video transcoding settings on the admin side. Make sureEnable face importis enabled.In your NPM entry for the URL under which you run immich, make sure to add something like this to the advanced tab so to allow larger uploads:
Point the host to the IP you set in docker for it, and port
2283Actual Import process
Download originals to this Mac. If you never did this before, let it first download all things, before you proceed.venv)osxphotospython library and run the following command (adapt as per the manpage, but be sure to understand each option!):osxphotos export "/path-to/icloud-backup" --skip-bursts --not-shared --not-syndicated --not-shared-moment --not-shared-library --filename "{id}" --ignore-date-modified -V --sidecar xmp. NOTE, this skips bursts because I had almost none, but if you have and want them, remove that flag. It also skips shared folders and shared albums because A) if you shared items they are in the main timeline anyway and B) anything shared to you you likely do not want in the library. Finally, this does sync hidden folder of iCloud (the mobile app is not able to see that one album so be prepared for some number mismatch). To dry run, add--dry-run.immich CLIand connect to your server. If you have self signed certificate - before log in, doexport NODE_EXTRA_CA_CERTS=$HOME/your-cert.pemimmich upload /path-to/icloud-backup. For dry run, do--dry-run. This takes about 2 hours on 14k assets using LAN. Let it finish.recents (all)in the app, and enable backup. If you have NO outliers and NO items left in your main timeline that where also in hidden folder and otherwise did export withosxphotosexactly what the app sees too (that includes bursts etc) then it will work just fine. It might be that you get a few outliers that are re-uploaded, for example if you added--skip-editedto the export, or did not export bursts but have bursts in your library, those will be re-uploaded and basically be duplicates. Chances are if that happens, those will appear in immich without meta data and thus easily findable in the "most recent" items. Or, use the utility to de-duplicate.This way, I synced 14k photos approx (some videos being 20gb large) in about 2 days.
The entire process also works without osxphotos/cli, but it takes ages, approximately on a LAN it syncs on my end 1000 pics of medium size per day.
Note that if you do NOT use osxphotos chances are a HIGH number of your assets will miss metadata. Somehow, iCloud Photos does a strange thing with locations and other metadata that the app cannot understand - however osxphotos, when exporting to the sidecar files, does understand and put it in a immicc-compatible format. Don't ask me the technical details on it, I do not understand them, but I know there where hundreds of dislocated images in my lib after a app-only import and that was not the case with the osxphotos sidecar/cli import.
Hope this helps anyone.
Beta Was this translation helpful? Give feedback.
All reactions