Collection.unlock() blocks indefinitely

[shuklak13]

Environment

operating system: Debian
Python version: 3.9
SecretStorage==3.3.1

Issue

I am trying to get my keyring configuration via this command: keyring get https://upload.pypi.org/legacy/ <my-username>.

However, this command blocks indefinitely. Terminating the command returns this traceback:

^CTraceback (most recent call last):
  File "/usr/local/home/karanshukla/.local/bin/keyring", line 8, in <module>
    sys.exit(main())
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/cli.py", line 135, in main
    return cli.run(argv)
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/cli.py", line 68, in run
    return method()
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/cli.py", line 76, in do_get
    password = get_password(self.service, self.username)
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/core.py", line 55, in get_password
    return get_keyring().get_password(service_name, username)
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/backends/SecretService.py", line 78, in get_password
    collection = self.get_preferred_collection()
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/keyring/backends/SecretService.py", line 65, in get_preferred_collection
    collection.unlock()
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/secretstorage/collection.py", line 67, in unlock
    return unlock_objects(self.connection, [self.collection_path])
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/secretstorage/util.py", line 153, in unlock_objects
    dismissed, (signature, unlocked) = exec_prompt(connection, prompt)
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/secretstorage/util.py", line 139, in exec_prompt
    dismissed, result = connection.recv_until_filtered(signals).body
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/jeepney/io/blocking.py", line 181, in recv_until_filtered
    self.recv_messages(timeout=deadline_to_timeout(deadline))
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/jeepney/io/blocking.py", line 116, in recv_messages
    msg = self.receive(timeout=timeout)
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/jeepney/io/blocking.py", line 101, in receive
    b = self._read_some_data(timeout=deadline_to_timeout(deadline))
  File "/usr/local/home/karanshukla/.local/lib/python3.9/site-packages/jeepney/io/blocking.py", line 105, in _read_some_data
    for key, ev in self.selector.select(timeout):
  File "/usr/lib/python3.9/selectors.py", line 469, in select
    fd_event_list = self._selector.poll(timeout, max_ev)
KeyboardInterrupt

It seems Collection.unlock() is blocking indefinitely. What is causing this, and how can this be fixed?

[mitya57]

Hi!
Can you please share more details about your configuration?

• What is the desktop environment (GNOME Shell, other DE, headless)?
• Is gnome-keyring installed and running?

It looks like gnome-keyring opened a prompt dialog and waits for you to type a password (or at least it thinks so). Do you see any dialogs when it happens?

[blakjak44]

Just wanted to say I had this same issue on an Ubuntu server and fixed it by configuring the keyring to automatically unlock.
https://wiki.archlinux.org/title/GNOME/Keyring#PAM_step

[abn]

@mitya57 some of the Poetry users have been experiencing this issue as well. I have been trying to recreate conditions to reproduce the issue.

For context, I am using GNOME Shell and gnome-keyring is installed and working as expected. I was able to narrow down reproduction to the following sequence.

1. Kill gnome-kerying (to force a locked keyring scenario).
2. Open a desktop app that requires keyring forcing an unlock dialogue. In my case, Pycharm IDE.
3. Dismiss unlock dialogue.
4. Trigger unlock(), in my case using the keyring package.

This causes no dialogue to appear, and the python session hangs - obviously waiting for the user to interact with a dialogue that never gets displayed.

What I also found interesting was that if I skipped (2, 3), the same (4) will trigger the dialogue successfully and dismissing it without unlocking and retrying (4) still shows the dialogue.

Any insights into debugging the root cause will be helpful.

If it helps, here are the dbus events for when the unlock get stuck.

dbus-events.log

method call time=1736459117.222271 sender=:1.1247 -> destination=org.freedesktop.DBus serial=2 path=/org/freedesktop/DBus; interface=org.freedesktop.DBus; member=AddMatch
   string "sender='org.freedesktop.secrets',interface='org.freedesktop.Secret.Prompt'"
method return time=1736459117.222275 sender=org.freedesktop.DBus -> destination=:1.1247 serial=4294967295 reply_serial=2
method call time=1736459117.222278 sender=:1.1247 -> destination=org.freedesktop.secrets serial=3 path=/org/freedesktop/secrets/aliases/default; interface=org.freedesktop.DBus.Properties; member=Get
   string "org.freedesktop.Secret.Collection"
   string "Label"
method return time=1736459117.222282 sender=:1.1081 -> destination=:1.1247 serial=229 reply_serial=3
   variant       string "Login"
method call time=1736459117.222286 sender=:1.1247 -> destination=org.freedesktop.secrets serial=4 path=/org/freedesktop/secrets/aliases/default; interface=org.freedesktop.DBus.Properties; member=Get
   string "org.freedesktop.Secret.Collection"
   string "Locked"
method return time=1736459117.226750 sender=:1.1081 -> destination=:1.1247 serial=230 reply_serial=4
   variant       boolean true
method call time=1736459117.227217 sender=:1.1247 -> destination=org.freedesktop.secrets serial=5 path=/org/freedesktop/secrets; interface=org.freedesktop.Secret.Service; member=Unlock
   array [
      object path "/org/freedesktop/secrets/aliases/default"
   ]
method return time=1736459117.228304 sender=:1.1081 -> destination=:1.1247 serial=231 reply_serial=5
   array [
   ]
   object path "/org/freedesktop/secrets/prompt/u21"
method call time=1736459117.228706 sender=:1.1247 -> destination=org.freedesktop.secrets serial=6 path=/org/freedesktop/secrets/prompt/u21; interface=org.freedesktop.Secret.Prompt; member=Prompt
   string ""
method return time=1736459117.228923 sender=:1.1081 -> destination=:1.1247 serial=232 reply_serial=6

[mitya57]

Hi @abn!
I cannot reproduce this. I followed your instruction, the only difference is that I used Seahorse and its Unlock button instead of Pycharm.

According to your dbus-events.log, the gnome-keyring daemon returned a prompt path to SecretStorage (/org/freedesktop/secrets/prompt/u21), so SecretStorage started waiting¹ for a Completed signal on it. So it is a bug in gnome-keyring that it did not display the dialog (or, rather, did not instruct gnome-shell using org.gnome.keyring.SystemPrompter interface to display it, unless you did not include this part in your log).

I have a few questions that will maybe help shed light on this:

• What is your gnome-keyring version?
• Does it still happen if you replace step 1 with collection.lock() call from SecretStorage?
• Does it still happen if you use Seahorse like I did in step 2?
• Can you double-check if there were any calls on org.gnome.keyring.SystemPrompter interface when it blocked?

Footnotes

1. SecretStorage waits in blocking mode. I can add asynchronous API, but that is another story — please file a separate issue if you need it. ↩

[abn]

Thanks for responding @mitya57. I have some additional updates I hadn't posted yet as I am still digging into the issue. Based on what I know now, it makes sense you could not reproduce it based on my previous information.

So, some context is in order. Poetry uses secretstorage via the keyring library. I started digging into the issue since a lot of users have historically complained about Poetry hanging due to an unavailable and/or locked keyring.

What I discovered since my last post;

1. The IDE (PyCharm) was triggering poetry commands in the background because it detected the project to use Poetry.
2. The poetry install command uses concurrent.futures.ThreadPoolExecutor to spawn concurrent tasks that needed to access the credentials.

This is likely why you were unable to reproduce the issue. What seems to be happening in my case, was that Poetry spawned N concurrent requests to the keyring library to check if credentials existed for HTTP requests. I would imagine, that this lead to one of those requests succeeding and presenting a prompt that, if cancelled, in some cases leads to the next thread prompting. However, it seems when the process runs in the background (which is the case when the IDE starts), at some point or if unlucky at the very begining, the prompt never gets shown and leads to all threads waiting for a dbus response. This, I am guessing, also makes the prompt inaccessible for any other clients.

Based on this, I am thinking the issue largely is with how Poetry uses thread executor with keyring rather than secretstorage itself. However, I am also curious if there are any safety fixes that can/should be done here to prevent or mitigate the impact of such a scenario.

Regarding an asynchronous call, I do not think this will help in our case since the call is made via keyring package and there it is expected to be blocking.

For completeness, responses to your questions below.

• What is your gnome-keyring version? gnome-keyring-46.2-2.fc41.x86_64
• Does it still happen if you replace step 1 with collection.lock() call from SecretStorage? No
• Does it still happen if you use Seahorse like I did in step 2? No
• Can you double-check if there were any calls on org.gnome.keyring.SystemPrompter interface when it blocked? I could not see any calls in the logs. Tried reproducing it again as well.

My takeaway right now is that for whatever reason gnome-keyring gives up on trying to present the prompt and/or crashes somewhere. This could be a side-effect of how Poetry triggered multiple simultaneous requests. The best thing I can do is the following:

1. Attempt an access in the application pre-flight before threads are spawned.
2. If (1) fails or is cancelled, disable keyring use entirely for the application session.

I would appreciate your insights here. Additionally, how does secretstorate behave when either of the following is true;

• dbus session is not available (eg: containers)
• a prompt cannot be displayed (eg: headless)
• previous is true and there is no tty

[mitya57]

Thank you for the detailed explanation! I managed to reproduce the issue now and it looks like a gnome-keyring bug to me.

When one process makes several unlock requests in parallel, it works fine: if the user dismisses one prompt, the next one is displayed, and the hang does not happen. However, when there are multiple processes (≥ 3) doing requests at the same time, only one prompt is displayed. The other processes do receive some prompt object paths, and gnome-keyring allows to call Prompt method on them, however it does not actually display the other prompts. Furthermore, after that it is not possible to unlock the keyring from any application, including seahorse, so the issue is not specific to Python or SecretStorage.

So I made a small test script to reproduce this problem and filed a gnome-keyring bug.

As a workaround, maybe Poetry can use some lock so that different processes don’t try to make unlock requests in parallel?

Now answering your remaining questions:

Additionally, how does secretstorate behave when either of the following is true;

• dbus session is not available (eg: containers)

SecretStorage will raise SecretServiceNotAvailableException, which will make python-keyring disable the Secret Service backend. If no other backend is available, python-keyring will raise an error too. However for example the keyrings.alt package provides a file-based backend which will work on any system.

• a prompt cannot be displayed (eg: headless)
• previous is true and there is no tty

One can run gnome-keyring-daemon with --unlock argument and pass the password to stdin (as described in python-keyring README). If you do not do that, SecretStorage will raise PromptDismissedException.

[abn]

Glad the explanation helped and thank you for reporting to gnome-keyring.

As a workaround, maybe Poetry can use some lock so that different processes don’t try to make unlock requests in parallel?

Yes. This is what I ended up proposing together with some logic to do a "dummy" retrieval pre-flight.

We also discovered that the issue we had was also exasperated by @functoosl.cached_property not being very thread-safe.

Thank you also for answering the remaining questions. It is very helpful. From my side, I think this issue can be closed, but I cannot speak for the author.