Bug: 'Unable to connect to terminal'

[jordan-bravo]

Describe the bug

When attempting to launch the built-in Lightning node terminal within Polar, an error message is displayed:

Unable to connect to terminal
Docker container 'polar-n2-alice' not found

To Reproduce

Steps to reproduce the behavior:

1. Open Polar
2. Start network (in my case, two LND nodes)
3. Click on a Lightning node (in my case, an LND node called 'alice')
4. Click on Actions
5. Under the section Terminal, click Launch.

Expected behavior

The Polar Terminal should launch and provide the ability to run commands on the node.

Actual behavior

The Polar Terminal launches and shows the error listed above.

Screenshots

Desktop (please complete the following information):

• OS: Fedora 37
• Polar Version: 1.4.0
• Docker Version: 20.10.21
• Docker Compose Version: 2.12.2

[jamaljsr]

Are you able to deposit funds and open a channel from alice?

Does docker ps show the alice container running?

[jordan-bravo]

• docker ps shows the alice container is running.
• I restarted Polar and I'm also experiencing a permission issue with admin.macaroon (see screenshot below).

[jamaljsr]

Setting up docker on Linux can be pretty tricky when it comes to permissions. Did you install Docker Desktop for Linux or install using a package? For the packages or scripts, there are sometimes post-install steps that need to be done. Is Docker running as root or a user account?

Unfortunately, I dont have a Fedora machine to test or troubleshoot with.

[jordan-bravo]

To test it out, I did the following:

• A fresh install of Ubuntu 22.10.
• Installed Docker Desktop v4.15.0 (which installs Docker version 20.10.21 and Docker Compose version v2.13.0)
• Downloaded the Polar AppImage, made it executable, then opened, which shows the following, no matter how many times I click "Check Again":

Any advice? Is Polar compatible with Docker Desktop?

Here is the output of the terminal command docker version:

[jordan-bravo]

Update

I created a symbolic link using the command sudo ln -s /home/${USER}/.docker/desktop/docker.sock /var/run/docker.sock, and now Polar recognizes Docker, but still shows the following error for Docker Compose (see screenshot).

{
  "errno": -2,
  "code": "ENOENT",
  "syscall": "spawn docker-compose",
  "path":"docker-compose",
  "spawnargs": ["version","--short"]
}

It looks like Polar is calling docker-compose, which is the syntax for the deprecated api for Docker Compose V1. Docker recommends using the Docker Compose V2 syntax: docker compose (a space between the words, instead of a hyphen).

I tried creating an alias in my ~/.bashrc file:

alias docker-compose="docker compose"

However Polar is still showing the same error. I'm not sure how to make Polar use the alias.

[jordan-bravo]

Update 2

There is an option in Docker Desktop to create a link from docker-compose to docker compose, this solved the startup issue and now Polar can find Docker and Docker Compose.

However, there is still a permission issue with admin.macaroon. Any suggestions?

[jamaljsr]

Thanks a bunch for all of the details and screenshots. I have an Ubuntu VM with Polar working but I haven't installed from scratch in a long while. Unfortunately, Docker Desktop doesn't work inside a VM, so I won't be able to reproduce your exact scenario myself.

For the permissions issue, can you share what the current permissions and owner are for the bob dir? Polar should create this dir before starting the network in order to make your host user the owner of the dir. This should grant you read/write permission to the subdirs as well when the LND node writes to the Docker volume. It appears something is going wrong there.

[jordan-bravo]

Here's what the permissions look like. My (host) user name is j but the owner is 100999.

[jamaljsr]

Ahh, that is interesting. This isn't behavior I've ever seen before. My hunch is that Docker Desktop is handling the file permissions differently now that it is using a VM internally. Docker Server for Linux doesn't use a VM.

Can you try stopping the network, changing the owner/permissions of the volumes dir to your j user, then restart the network? I'm curious to see if Docker will modify the perms again after it boots up.

[jordan-bravo]

I performed the following steps:

• In Polar, I stopped the network.
• In a terminal, cd to the directory ~/.polar/networks/1/volumes/lnd, then run the commands:

sudo chown -R j:j alice
sudo chown -R j:j bob

• In Polar, start the network.

Result: the network starts, and I can launch the Polar terminal with no permission issues.

Summary

There are three outstanding issues here when using Docker Desktop:

1. Polar is expecting docker at /var/run/docker.sock. A temporary workaround is to created a symbolic link using the command sudo ln -s /home/${USER}/.docker/desktop/docker.sock /var/run/docker.sock, however this does not persist across system reboots.
2. Polar is using the deprecated V1 Docker Compose API which expects the command docker-compose instead of the V2 version docker compose. The workaround for this is a setting in Docker Desktop, check the box for Enable Docker Compose V1/V2 compatibility mode.
3. Inside the Docker Desktop VM, the owner for the items in the lnd directory are not the host user. The workaround for this is to recursively own the directories with sudo chown -R ${USER}:${USER} ~/.polar/networks.

[jamaljsr]

Thanks for testing and confirming that behavior. I agree that Polar needs to be updated to resolve those few issues. I will try to get a native Linux machine setup to work on better compatibility with Docker Desktop. I really appreciate the troubleshooting here.

[jordan-bravo]

I'm a TypeScript and React dev, I'd be happy to submit a bug fix if I'm able to figure it out. Could you point me in the right direction with regards to where in the codebase these Docker related issues might be located?

[jamaljsr]

I appreciate the offer to help.

Polar uses the dockerode and docker-compose NPM packages to communicate with Docker on the machine. All of this interaction is done in the dockerService.ts file. That's where you'd want to start.

Regarding each issue, here are my thoughts on how I would initially attempt to solve them:

• Polar is expecting docker at /var/run/docker.sock. A temporary workaround is to created a symbolic link using the command sudo ln -s /home/${USER}/.docker/desktop/docker.sock /var/run/docker.sock, however this does not persist across system reboots.

By default, dockerode uses multiple methods to attempt to connect to Docker on the machine. You can see this logic in the docker-modem repo. Since it appears that dockerode isn't automatically detecting the socket file at /home/${USER}/.docker/desktop/docker.sock, Polar could check for the existence of this file and if it exists, pass it as the socketPath to the Dockerode constructor. I created #609 which allows the user to specify the paths for the Docker socket and docker-compose executable, but I would really prefer the detection to be automatic. This is why I still haven't merged that in yet.

• Polar is using the deprecated V1 Docker Compose API which expects the command docker-compose instead of the V2 version docker compose. The workaround for this is a setting in Docker Desktop, check the box for Enable Docker Compose V1/V2 compatibility mode.

I haven't found a clear solution to this, but after just a bit of searching I found that the dockerode dev also created the dockerode-compose package which has some functionality to call docker compose commands via the Docker API. Unfortunately, it looks like it doesn't have full API coverage. It only supports up, down, and pull. Polar also needs the ability to start, stop, and remove individual services in the docker network. A thought I had was possibly use the dockerode-compose package for some of the functionality and fill the gaps in the Polar codebase using the dockerode package, which should be possible. There may also be other packages out there but I haven't spent a lot of time researching.

• Inside the Docker Desktop VM, the owner for the items in the lnd directory are not the host user. The workaround for this is to recursively own the directories with sudo chown -R ${USER}:${USER} ~/.polar/networks.

I haven't spent any time troubleshooting this issue since I don't have a machine running Docker Desktop on Linux. I'd need to learn more about why the volume file permissions aren't being set in the same way as Docker Engine. It could just be setting additional flags in the compose file, I'm not sure. One idea is to have Polar reset the file perms after starting the nodes but this feels a bit hacky. There may be a cleaner solution but would require a better understanding of why Docker behaves differently.

[jordan-bravo]

Thanks for the info. I found something that might shed light on the ownership situation.

The Docker Desktop Docs describe this issue but it's not immediately obvious to me what the solution would be.

In particular, the last paragraph of the section is relevant:

In this scenario if a shared file is chowned inside a Docker Desktop container owned by a user with a UID of 1000, it shows up on the host as owned by a user with a UID of 100999. This has the unfortunate side effect of preventing easy access to such a file on the host. The problem is resolved by creating a group with the new GID and adding our user to it, or by setting a recursive ACL (see setfacl(1)) for folders shared with the Docker Desktop VM.