Broken repo - don't understand why and how to fix

[davidfrickert]

Description

"The Git data underlying this repository cannot be read. Contact the administrator of this instance or delete this repository."

Have this happen to two repos so far on the last few months. Solved first by deleting the repo and re-pushing the code, but really would be better if this doesn't happen

I can still pull/push to the repo from git tools, but anything involving gitea UI or actions is completely broken on these repos.

(Note: the actual repo name has been replaced with in the snippets below)
The repo seems fine:

/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ git show-ref --tags
bd3da0bb27177a3b191bb84521eef59692b210b6 refs/tags/0.1.0
33989d17e4d94ac386af6f1a1fdeef14b5231ba0 refs/tags/0.1.1
/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ /usr/bin/git -c protocol.version=2 -c credential.helper= rev-parse
/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ echo $?
0

I have this in the app.ini:

[log]
MODE = console
LEVEL = Debug ; please set the level to debug when we are debugging a problem
STACKTRACE_LEVEL = error
COLORIZE = false ; this can be true if you can strip out the ansi coloring

I don't really get anything interesting in the logs when trying to access the repo via gitea (a bit more shown in the gist but probably not useful):

2025/05/10 14:58:19 .../context_response.go:70:HTML() [D] Template: repo/empty

Funnily enough, I am able to clone the repo with no issues (sorry for some PT language):
Pushing new commits / pulling etc also works.

git clone https://git.djf.lol/davidfrickert/<REPONAME>.git
Clonando para '<REPONAME>'...
remote: Enumerating objects: 37, done.
remote: Counting objects: 100% (37/37), done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 37 (delta 10), reused 0 (delta 0), pack-reused 0 (from 0)
A receber objetos: 100% (37/37), 13.67 KiB | 13.67 MiB/s, concluído.
A resolver deltas: 100% (10/10), concluído.

Gitea Version

1.23.7

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/davidfrickert/753cd38cfc1462f92b1863f3b8241c24

Screenshots

Git Version

git --version -> git version 2.47.2

Operating System

Fedora 41 Server

How are you running Gitea?

Helm chart on Kubernetes (3 gitea instances, 3 redis instances)
Using the official docker image from gitea v1.23.7

Database

PostgreSQL

[davidfrickert]

not sure if relevant but i saw this in the logs before the repository broke (3 times in a row the same failure of the snippet below)

2025/05/10 14:51:07 ...dules/git/command.go:292:Run() [D] git.Command.RunDir(/data/git/gitea-repositories/davidfrickert/<REPONAME>.git): /usr/bin/git -c protocol.version=2 -c credential.helper= cat-file --batch
2025/05/10 14:51:07 ...ons/commit_status.go:30:CreateCommitStatus() [E] Failed to create commit status for job 14619: GetCommit[20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34]: object does not exist [id: 20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34, rel_path: ]
	/go/src/code.gitea.io/gitea/services/actions/commit_status.go:30 (0x20dd04f)
	/go/src/code.gitea.io/gitea/services/actions/job_emitter.go:73 (0x20de58e)
``

[badhezi]

Any way to reproduce this issue locally?

[lunny]

Can you visit the directory of that repository from server side.

[davidfrickert]

Any way to reproduce this issue locally?

unfortunately, i don't know what triggers this

[davidfrickert]

Can you visit the directory of that repository from server side.

Yeah I can, in the issue description I do that:

/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ git show-ref --tags
bd3da0bb27177a3b191bb84521eef59692b210b6 refs/tags/0.1.0
33989d17e4d94ac386af6f1a1fdeef14b5231ba0 refs/tags/0.1.1
/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ /usr/bin/git -c protocol.version=2 -c credential.helper= rev-parse
/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ echo $?
0

[davidfrickert]

fsck also doesn't detect anything wrong:

/data/git/gitea-repositories/davidfrickert/<REPONAME>.git $ /usr/bin/git -c protocol.version=2 -c credential.helper= fsck
Checking object directories: 100% (256/256), done.

[lunny]

Is there any branches in this repository except tags?

[davidfrickert]

Is there any branches in this repository except tags?

yes, git show-ref on the repo (in the gitea container server folder)

7e2122c5a1498b417b2eb8d0e104fc7bac7c022a refs/heads/master
564c52b0a4991a9d4c35690f7b0b981a535e9ce4 refs/heads/renovate/softprops-action-gh-release-digest
449e5b11d9c41a860bbfeb01216901f8acba9801 refs/pull/1/head
564c52b0a4991a9d4c35690f7b0b981a535e9ce4 refs/pull/2/head
bd3da0bb27177a3b191bb84521eef59692b210b6 refs/tags/0.1.0
33989d17e4d94ac386af6f1a1fdeef14b5231ba0 refs/tags/0.1.1

[lunny]

Does the commit 20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34 exist? Maybe it's related to the directory permissions. Could you compare the permissions between the normal repository and abnormal repository.

[davidfrickert]

Does the commit 20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34 exist? Maybe it's related to the directory permissions. Could you compare the permissions between the normal repository and abnormal repository.

yes, git show 20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34 (ran inside the gitea container)

commit 20bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34
Author: davidfrickert <david.frickert@protonmail.com>
Date:   Sat May 10 16:50:16 2025 +0200

    add python gitignore

diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..0a19790
--- /dev/null
+++ b/.gitignore

(..... etc .....)

I don't see any issues with permissions, ran find . -type f -exec ls -l {} \;on broken repo and on normal repo and it looks exactly the same.

ran in broken repo (inside the gitea container):

-rw-r--r--    1 git      git             23 May 10 15:22 ./HEAD
-rw-r--r--    1 git      git             41 Apr 28 22:29 ./refs/tags/0.1.0
-rw-r--r--    1 git      git             41 Apr 29 11:58 ./refs/tags/0.1.1
-rw-r--r--    1 git      git             41 May  9 16:39 ./refs/pull/1/head
-rw-r--r--    1 git      git             41 May  9 18:01 ./refs/pull/2/head
-rw-r--r--    1 git      git             41 May 10 15:22 ./refs/heads/master
-rw-r--r--    1 git      git             41 May  9 16:44 ./refs/heads/renovate/softprops-action-gh-release-digest
-r--r--r--    1 git      git            818 Apr 28 22:29 ./objects/f1/fdfaa7c61bb67f79baaffe6e09f5563d634b66
-r--r--r--    1 git      git            277 Apr 28 22:29 ./objects/c8/4b50b3c02c2fa67fd6d2db78ff9a6ad7a45650
-r--r--r--    1 git      git            112 Apr 28 22:29 ./objects/79/3e84f67900f8c452e8bf12f86076deaf8ddda0
-r--r--r--    1 git      git             51 May 10 14:50 ./objects/e4/46399da377f20c71cbbebe8e8ecd6aa6ff44bf
-r--r--r--    1 git      git            170 Apr 29 13:22 ./objects/49/0d13b2908874e7c9a829ab89e7105d57a2b46b
-r--r--r--    1 git      git            314 May 10 14:50 ./objects/14/3ce3f2e36a38b52b03f029348c4f731b171d17
-r--r--r--    1 git      git             77 Apr 28 22:29 ./objects/56/1b454b2dfc8af2c9b15be3fe03403086d2c2de
-r--r--r--    1 git      git            189 May  9 16:44 ./objects/56/4c52b0a4991a9d4c35690f7b0b981a535e9ce4
-r--r--r--    1 git      git           1611 May 10 15:22 ./objects/3b/58d13fccc949bad61def816203c8fd33a49348
-r--r--r--    1 git      git             74 May  9 16:39 ./objects/71/90a60b6422152e768da3a34747a17d956739be
-r--r--r--    1 git      git             51 May  9 16:44 ./objects/8d/26961f3b7435204f4e1de922b40440ae649615
-r--r--r--    1 git      git            283 May 10 14:50 ./objects/8d/5f2eb3bef5e97d4e346a0c50d8a6eb36c55ff6
-r--r--r--    1 git      git             92 May 10 14:50 ./objects/da/63e6c5f225468a29b69bbf8142601b4223140f
-r--r--r--    1 git      git            399 Apr 28 22:29 ./objects/da/e21d8b844a45c24006fff5a0120a5fc268f685
-r--r--r--    1 git      git            402 Apr 29 13:22 ./objects/c4/edb76f1caeecf0491b3eda4e3f45a1322218b0
-r--r--r--    1 git      git            282 May  9 16:39 ./objects/74/a4f92d74ae793f7d987a9cefc7267fd5d1b1cd
-r--r--r--    1 git      git            176 Apr 29 11:58 ./objects/33/989d17e4d94ac386af6f1a1fdeef14b5231ba0
-r--r--r--    1 git      git            282 May  9 16:44 ./objects/47/fba7a2e153d9901a4f87914f9c1ee3bf484d5c
-r--r--r--    1 git      git            245 Apr 29 13:22 ./objects/c5/c7d4d94b94e29b6c532da410f93dbd2c00bd5e
-r--r--r--    1 git      git            172 May 10 14:50 ./objects/20/bb6ac95c03c92b91b3ae26f5fc8f71ff0d9b34
-r--r--r--    1 git      git            620 Apr 28 22:29 ./objects/40/51cec6c869256bb95d0f470b06954730c442f0
-r--r--r--    1 git      git            190 May 10 14:50 ./objects/ec/c2ea5556075dfad0944ecd6bd50ce8be4aeadf
-r--r--r--    1 git      git             48 Apr 28 22:29 ./objects/ab/1033bd37224ee84b5862fb25f094db73809b74
-r--r--r--    1 git      git           1611 May 10 14:50 ./objects/0a/197900e25d259ab4af2e31e78501787d7a6daa
-r--r--r--    1 git      git            304 May  9 16:44 ./objects/ff/5ac4a75c7cd977a80cdc6e8d1a4121437ea020
-r--r--r--    1 git      git            246 Apr 29 11:58 ./objects/ce/23c24c4d4127ef286ecc35ba620894a0907000
-r--r--r--    1 git      git            314 May 10 15:22 ./objects/be/4f359942f864d3253cfe1b9d1946e720329d87
-rw-r--r--    1 git      git              1 Apr 28 22:28 ./objects/info/packs
-r--r--r--    1 git      git           6716 Apr 28 22:29 ./objects/84/4123b455e9e5de13e8ff949ea600747b4a2488
-r--r--r--    1 git      git             92 Apr 28 22:29 ./objects/3d/dfb0d6e12fa57a592eea051efce9d3195af1cb
-r--r--r--    1 git      git            169 May 10 15:22 ./objects/7e/2122c5a1498b417b2eb8d0e104fc7bac7c022a
-r--r--r--    1 git      git            271 May 10 14:50 ./objects/66/8f2882c8f4e232a08139ce707260115523883d
-r--r--r--    1 git      git             52 Apr 28 22:29 ./objects/b8/a10511f18e7f578c61edb0443d74ca97de5eb0
-r--r--r--    1 git      git            303 Apr 28 22:29 ./objects/76/843acfb6a720c0cc9163fe3d6111eebb41ad1b
-r--r--r--    1 git      git            274 Apr 28 22:29 ./objects/db/ef96d0f20bd0751dd7bba6e7b64bf631fc5248
-r--r--r--    1 git      git             92 May  9 16:44 ./objects/36/492ec8b630d19bc584fdca4a30b93e7a793d72
-r--r--r--    1 git      git            162 May  9 16:39 ./objects/44/9e5b11d9c41a860bbfeb01216901f8acba9801
-r--r--r--    1 git      git            416 Apr 29 11:58 ./objects/53/d0830abff85a8422904334ff2f10fa786e1eac
-r--r--r--    1 git      git            246 Apr 28 22:29 ./objects/35/979649c0f999dad2d3fcb3fb0448e9619cc9f7
-r--r--r--    1 git      git            132 Apr 28 22:29 ./objects/bd/3da0bb27177a3b191bb84521eef59692b210b6
-r--r--r--    1 git      git           1127 Apr 28 22:29 ./objects/b5/aa4caee347c374301839b8a5ca2eb66ba47786
-rwxr-xr-x    1 git      git            478 Apr 28 22:28 ./hooks/applypatch-msg.sample
-rwxr-xr-x    1 git      git            416 Apr 28 22:28 ./hooks/pre-merge-commit.sample
-rwxr-xr-x    1 git      git           1492 Apr 28 22:28 ./hooks/prepare-commit-msg.sample
-rwxr-xr-x    1 git      git              0 Apr 28 22:28 ./hooks/proc-receive.d/gitea
-rwxr-xr-x    1 git      git            189 Apr 28 22:28 ./hooks/post-update.sample
-rwxr-xr-x    1 git      git           2308 Apr 28 22:28 ./hooks/sendemail-validate.sample
-rwxr-xr-x    1 git      git            896 Apr 28 22:28 ./hooks/commit-msg.sample
-rwxr-xr-x    1 git      git            134 Apr 28 22:28 ./hooks/post-receive.d/gitea
-rwxr-xr-x    1 git      git            134 Apr 28 22:28 ./hooks/proc-receive
-rwxr-xr-x    1 git      git            376 Apr 28 22:28 ./hooks/post-receive
-rwxr-xr-x    1 git      git            137 Apr 28 22:28 ./hooks/update.d/gitea
-rwxr-xr-x    1 git      git           4898 Apr 28 22:28 ./hooks/pre-rebase.sample
-rwxr-xr-x    1 git      git           1374 Apr 28 22:28 ./hooks/pre-push.sample
-rwxr-xr-x    1 git      git           3650 Apr 28 22:28 ./hooks/update.sample
-rwxr-xr-x    1 git      git           1649 Apr 28 22:28 ./hooks/pre-commit.sample
-rwxr-xr-x    1 git      git            544 Apr 28 22:28 ./hooks/pre-receive.sample
-rwxr-xr-x    1 git      git            356 Apr 28 22:28 ./hooks/update
-rwxr-xr-x    1 git      git            376 Apr 28 22:28 ./hooks/pre-receive
-rwxr-xr-x    1 git      git            133 Apr 28 22:28 ./hooks/pre-receive.d/gitea
-rwxr-xr-x    1 git      git            424 Apr 28 22:28 ./hooks/pre-applypatch.sample
-rwxr-xr-x    1 git      git           2783 Apr 28 22:28 ./hooks/push-to-checkout.sample
-rw-r--r--    1 git      git           1167 May 10 15:22 ./logs/HEAD
-rw-r--r--    1 git      git            795 May 10 15:22 ./logs/refs/heads/master
-rw-r--r--    1 git      git            129 May  9 16:44 ./logs/refs/heads/renovate/softprops-action-gh-release-digest
-rw-r--r--    1 git      git            385 May 10 15:22 ./info/refs
-rw-r--r--    1 git      git            240 Apr 28 22:28 ./info/exclude
-rw-r--r--    1 git      git             66 May  9 16:40 ./config
-rw-r--r--    1 git      git             73 Apr 28 22:28 ./description

[davidfrickert]

there doesn't seem to be any issue with the git repo itself as i can do git clone, push commits, etc...
just the gitea ui says the repo is broken, i wonder if it's some db setting

[davidfrickert]

this seems to be the case, i queried the "repository" table in the postgres db and the field "status" has the value 3, which all the other repos have status 0

[davidfrickert]

ok so forcing status=0 seems to have recovered my repo for now