Skip to content

Conversation

@etnoy
Copy link
Collaborator

@etnoy etnoy commented Feb 20, 2025

Deleting a large external library is slow and fragile. Often, assets end up getting stuck and the library lingers on in a limbo state. This is annoying for the user because the library isn't listed in the library managment, but yet (potentially thousands) of assets hang around in the timeline.

We have a daily cron job set up to clean up pending library deletions, but there is no way to trigger this manually.

In the backend, the cleanup is triggered by the job that scans all libraries, but not the individual library scans.

In the frontend, we have a button that scans all libraries, but it enumerates all libraries and individually scans them instead of calling the backend which would also queue the cleanup.

This PR makes the scan button call the job for libraries, which in turn calls the refresh all job.

When the user presses "Scan all libraries" in the frontend, it still scans all libraries but also cleans up deletions :)

I also tweaked the library job panel, apparently the two buttons did the exact same thing as the force flag isn't used in the library job:
image

@etnoy etnoy force-pushed the fix/scan-all-libraries branch 2 times, most recently from e5c7bcc to c886ca8 Compare February 20, 2025 10:52
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Feb 20, 2025
@etnoy etnoy changed the title feat(server,web): library cleanup from ui feat(server): library cleanup from ui Feb 20, 2025
Copy link
Member

@danieldietzler danieldietzler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems sensible to me, thanks!

@etnoy etnoy force-pushed the fix/scan-all-libraries branch from c886ca8 to 312b060 Compare February 20, 2025 11:31
@etnoy etnoy force-pushed the fix/scan-all-libraries branch from 312b060 to 5197982 Compare February 21, 2025 09:04
@etnoy etnoy force-pushed the fix/scan-all-libraries branch from 09f94a8 to 7c7fa5c Compare February 21, 2025 09:43
@etnoy etnoy requested a review from jrasm91 February 21, 2025 11:01
"library_settings": "External Library",
"library_settings_description": "Manage external library settings",
"library_tasks_description": "Perform library tasks",
"library_tasks_description": "Scan external libraries for new and/or changed assets",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could expand this description a little more. Giving the details of all the actions of the job. Quite a few people have questions about what this job actually does.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's probably something that should go in the docs not the English translation of something.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When seeing this job and the library's options on the library page, it is unclear when you should use which.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When seeing this job and the library's options on the library page, it is unclear when you should use which.

I agree and I first tried to have no button on the library job page since it wasn't very intuitive. Right now, though, the code requires it to compile.

Maybe I should just do the needed change to remove the button?

Longer term, I would like to move the various and often-confusing buttons from the jobs page to a more dedicated repair page. That page would show if there are assets missing thumbnails, missing metadata, ML etcetera, with a friendly button to fix. We could also stop showing assets without thumbs in the timeline altogether.

Copy link
Member

@alextran1502 alextran1502 Feb 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I should just do the needed change to remove the button?

Does this button ultimately run the scan command on all libraries? If yes, I guess we just need to clarify that in the description.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it will run the scan on all libraries, and also run the check for pending library deletions

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a screenshot to the PR description

@alextran1502 alextran1502 merged commit 869839f into main Mar 3, 2025
39 checks passed
@alextran1502 alextran1502 deleted the fix/scan-all-libraries branch March 3, 2025 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants