[19.03.8] Binding folders are bind from MSYS2/MinGW

[eine]

Expected behavior

Bind folders work in both PowerShell and MinGW.

Actual behavior

After updating to 17.12.0-ce-win46 (15048) Stable 0ac7325, local folders bind to container volumes are not properly shown if docker is executed from MinGW.

Information

• E9FA3323-7ADF-49BB-A59A-73C64DC4AFC1/2018-01-09_21-24-26.

• I'm pretty sure it is due to the update, because reverting to Version 17.09.1-ce-win42 (14687) Stable 3176a6a "fixes" it.

Steps to reproduce the behavior

This works (17.12.0):

PS C:\Program Files\Docker\Docker\resources\bin> .\docker.exe run --rm -v d:/test:/test alpine ls -la test
total 5
drwxr-xr-x    2 root     root             0 Jan  9 19:48 .
drwxr-xr-x    1 root     root          4096 Jan  9 20:10 ..
-rwxr-xr-x    1 root     root            20 Jan  9 19:48 testfile.md

This also works (17.09.1):

user@host MINGW64 ~
$ winpty docker run --rm -itv //d/test:/test alpine ls -la test
total 5
drwxr-xr-x    2 root     root             0 Jan  9 19:48 .
drwxr-xr-x    1 root     root          4096 Jan  9 20:31 ..
-rwxr-xr-x    1 root     root            20 Jan  9 19:48 testfile.md

This does not (17.12.0):

user@host MINGW64 ~
$ docker run --rm -v //d/test://test alpine ls -la test
total 4
drwxr-xr-x    2 root     root            40 Jan  9 20:05 .
drwxr-xr-x    1 root     root          4096 Jan  9 20:11 ..

[sgammill]

Your mingw64 docker run commands are specifying the host directory in the Moby VM, not Windows paths. In the Moby VM, Windows drives are not mounted at the root directory anymore for some reason; they're now mounted in /host_mnt . As such, the command you're looking for is:

docker run --rm -v //host_mnt/d/test:/test alpine ls -la test

The double // after -v is an escape sequence so your shell interprets it as a single /.

[eine]

Thanks a lot @sgammill. May I know about any reference you might have found related to the change in the mount path inside the Moby VM?

Until now, adding a slash to pwd allowed to use the same scripts in both GNU/Linux and Windows (with MinGW): -v /$(pwd):/test. This change breaks all these scripts!

Might be related to #1284 and moby/moby#35425.

[sgammill]

I don't think it's related to LCOW. It's a change they made to the filesystem structure of the Moby VM. I stumbled on this like you did and then peeked inside the Moby VM and found out where the Windows drives were being mounted.

[sgammill]

With Windows docker.exe in Linux container mode, Windows-style host paths are interpreted as as paths on the Windows host, whereas Unix-style host paths are interpreted as paths in the Moby VM.

I have the same use case as you. As a work-around, you can abstract pwd in your scripts:

wd() {
  if [ "$OS" = Windows_NT ]; then
    cygpath -m -a .
  else
    pwd
  fi
}

docker run -v $(wd):/test ....

I'm not trying to prevent any RFE for host paths on Windows to work differently. I'm just offering the facts.

[eine]

Thank, again, for the workaround. Since I am going to use it in scripts that need to tell Windows and GNU/Linux hosts apart, it is clean enough for me. I will leave this issue opened anyway, so that maintainers can decide if it is worth an enhancement.

[gtardif]

Yes, we had to change where the drives are mounted in the linuxKit VM, but we do some translation of the mounted paths. Windows paths work a mentioned above : -v d:/test:/test.
Unix-style paths are translated, but not with a "//" at the beginning. You should use -v /d/test:/test instead of -v //d/test:/test.

[eine]

@gtardif , thanks for confirming that you had to change it. Unluckily, due to how MinGW/MSYS2 manipulates the paths, the format you mention (d:/test:/test) does not work. These are all the combinations I tried:

user@host $ docker run --rm -v /d/test:/test alpine ls -R test
ls: test: No such file or directory

user@host $ docker run --rm -v /d/test://test alpine ls -R test
bash: /c/Program Files/Docker/Docker/resources/bin/docker: Bad address

user@host $ docker run --rm -v //d/test:/test alpine ls -R test
test:

user@host $ docker run --rm -v //d/test://test alpine ls -R test
test:

user@host $ docker run --rm -v D:/test:/test alpine ls -R test
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: invalid mode: \msys64\test.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.

user@host $ docker run --rm -v D:\test:/test alpine ls -R test
test:

user@host $ docker run --rm -v D:\test://test alpine ls -R test
test:

user@host $ docker run --rm -v 'D:\test':/test alpine ls -R test
test:
testfile.md

user@host $ docker run --rm -v 'D:\test'://test alpine ls -R test
test:
testfile.md

user@host $ docker run --rm -v d:/test:/test alpine ls -R test
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: invalid mode: \msys64\test.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.

user@host $ docker run --rm -v 'd:/test':/test alpine ls -R test
C:\Program Files\Docker\Docker\resources\bin\docker.exe: Error response from daemon: invalid mode: \msys64\test.
See 'C:\Program Files\Docker\Docker\resources\bin\docker.exe run --help'.

user@host $ docker run --rm -v d:/test://test alpine ls -R test
bash: /c/Program Files/Docker/Docker/resources/bin/docker: Bad address

user@host $ docker run --rm -v 'd:/test'://test alpine ls -R test
bash: /c/Program Files/Docker/Docker/resources/bin/docker: Bad address

Note that only these two work as expected: 'D:\test':/test and 'D:\test'://test.

Of course, //host_mnt/d/test://test and //host_mnt/d/test:/test work too. But /host_mnt/d/test:/test does not.

[sgammill]

@gtardif Thanks for the mention that /c/path/ still works... I don't know why that stopped working for me when the Linuxkit-based VM was introduced (possible error occurred between chair and keyboard), but I can confirm it works now.

So I'll amend my observation: With Windows docker.exe in Linux container mode, Unix-style host paths are interpreted as paths on the Windows host if they're of the form /c/path/... but otherwise are interpreted as paths in the Moby VM (e.g., /var/run/docker.sock).

... This appears to be hold for host paths in --volume and --mount on the docker command line and in volume/mount specifications in compose yml files for docker stack, but host filenames for secrets and configs must still be Windows paths (forward-slashes permitted).

@1138-4eb MSYS2 shell interprets a leading / (but not //) as a Unix path that it will autoconvert to a Windows path on the fly before executing the command. You need to set MSYS2_ARG_CONV_EXCL='*' to disable this conversion before running docker.exe in order to get the path syntax to work.

export MSYS2_ARG_CONV_EXCL='*'
docker run --rm -v d:/test:/test alpine ls -R test
docker run --rm -v 'd:\test':/test alpine ls -R test
docker run --rm -v /d/test:/test alpine ls -R test
docker run --rm -v /host_mnt/d/test:/test alpine ls -R test

These 4 docker commands should all accomplish the same thing.

(Also, FWIW, to get cygpath to print backslashes, use cygpath -w -a . instead of -m)

[eine]

@sgamill, according to your comments, I wrote this test script:

opts="/d/test '/d/test' //d/test '//d/test' D:/test 'D:/test' D:\test 'D:\test' d:/test 'd:/test' d:\test 'd:\test'"

echo "Without setting MSYS2_ARG_CONV_EXCL"

for j in 0 1; do
  for i in $opts; do
    echo "$i"
    docker run --rm -v "$i":/test alpine ls -R test
    echo ""
    docker run --rm -v "$i"://test alpine ls -R test
    echo ""
    echo ""
  done
  export MSYS2_ARG_CONV_EXCL='*'
  echo "After setting MSYS2_ARG_CONV_EXCL"
done

Summary of working formats:

• Without setting MSYS2_ARG_CONV_EXCL
  • D:\test
  • d:\test
• After setting MSYS2_ARG_CONV_EXCL
  • /d/test
  • D:/test
  • D:\test
  • d:/test
  • d:\test

It just confirms what you explained. I added it to have a quick test script.

Thanks for the tip about cygpath -w -a ..

[eine]

Closing because this has been fixed in Version 17.12.0-ce-win47 (15139) stable 9c692cd.
winpty docker run --rm -itv /$(pwd):/test alpine ls test works again.