`/etc/dnsmasq.conf` is read and fails due to confinement if it exists

[vay3t]

Describe the bug
Describe what your problem is.

To Reproduce
How, and what happened?

1. reboot
2. multipass launch
3. multipass shell unbeatable-earwig
4. ping google.com

Expected behavior
I need internet in VM

Logs
Sorry for spanish

-- Logs begin at Sat 2020-02-29 03:22:41 -03, end at Fri 2020-04-03 18:45:43 -03. --
abr 02 21:54:47 hydraStation systemd[1]: Started Service for snap application multipass.multipassd.
abr 02 21:54:48 hydraStation multipassd[23638]: Unable to determine subnet for the mpqemubr0 subnet
abr 02 21:54:49 hydraStation multipassd[23638]: Using AppArmor support
abr 02 21:54:49 hydraStation multipassd[23893]: Applying AppArmor policy: multipass.dnsmasq
abr 02 21:54:49 hydraStation multipassd[23638]: 
abr 02 21:54:49 hydraStation dnsmasq[23893]: no se puede leer /etc/dnsmasq.conf: Permiso denegado
abr 02 21:54:49 hydraStation multipassd[23638]: 
                                                dnsmasq: no se puede leer /etc/dnsmasq.conf: Permiso denegado
abr 02 21:54:49 hydraStation dnsmasq[23893]: FALLO al iniciar
abr 02 21:54:49 hydraStation multipassd[23638]: gRPC listening on unix:/var/snap/multipass/common/multipass_socket, SSL:on
abr 02 21:54:49 hydraStation multipassd[23638]: QIODevice::write (QFile, "/var/snap/multipass/common/cache/multipassd/vault/multipassd-image-records.json"): device not open
abr 02 21:58:42 hydraStation multipassd[25172]: Applying AppArmor policy: multipass.dnsmasq
abr 02 21:58:42 hydraStation dnsmasq[25174]: iniciado, versión 2.79 tamaño de caché 150
abr 02 21:58:42 hydraStation dnsmasq[25174]: opciones al compilar: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify
abr 02 21:58:42 hydraStation dnsmasq-dhcp[25174]: DHCP, IP range 10.215.78.2 -- 10.215.78.254, lease time infinito
abr 02 21:58:42 hydraStation dnsmasq-dhcp[25174]: DHCP, sockets bound exclusively to interface mpqemubr0
abr 02 21:58:42 hydraStation dnsmasq[25174]: se usan direcciones locales sólo para dominio multipass
abr 02 21:58:42 hydraStation dnsmasq[25174]: leyendo /etc/resolv.conf
abr 02 21:58:42 hydraStation dnsmasq[25174]: se usan direcciones locales sólo para dominio multipass
abr 02 21:58:42 hydraStation dnsmasq[25174]: se usa el servidor 127.0.0.53#53
abr 02 21:58:42 hydraStation dnsmasq[25174]: /etc/hosts leído - 8 direcciones
abr 02 21:58:42 hydraStation dnsmasq[25174]: no se puede leer /var/snap/multipass/common/data/multipassd/network/dnsmasq.hosts: No existe el archivo o el directorio
abr 02 22:04:30 hydraStation multipassd[25527]: Applying AppArmor policy: multipass.qemu-img
abr 02 22:04:31 hydraStation multipassd[25537]: Applying AppArmor policy: multipass.qemu-img
abr 02 22:04:31 hydraStation multipassd[25543]: Applying AppArmor policy: multipass.qemu-img
abr 02 22:04:32 hydraStation multipassd[23638]: process working dir '/snap/multipass/1784/qemu'
abr 02 22:04:32 hydraStation multipassd[23638]: process program 'qemu-system-x86_64'
abr 02 22:04:32 hydraStation multipassd[23638]: process arguments '--enable-kvm, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/snap/multipass/common/data/multipassd/vault/instances/surpassing-cobia/ubuntu
abr 02 22:04:32 hydraStation multipassd[23638]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
abr 02 22:04:32 hydraStation multipassd[25632]: Applying AppArmor policy: multipass.surpassing-cobia.qemu-system-x86_64
abr 02 22:04:32 hydraStation multipassd[23638]: process started
abr 02 22:04:32 hydraStation multipassd[23638]: QMP: {"QMP": {"version": {"qemu": {"micro": 1, "minor": 11, "major": 2}, "package": "(Debian 1:2.11+dfsg-1ubuntu7.23)"}, "capabilities": []}}
abr 02 22:04:32 hydraStation multipassd[23638]: QMP: {"return": {}}
abr 02 22:04:52 hydraStation multipassd[23638]: QMP: {"timestamp": {"seconds": 1585875892, "microseconds": 843033}, "event": "NIC_RX_FILTER_CHANGED", "data": {"name": "net0", "path": "/machine/peripheral/net0/vi
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPDISCOVER(mpqemubr0) 52:54:00:27:2b:18
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPOFFER(mpqemubr0) 10.215.78.16 52:54:00:27:2b:18
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPDISCOVER(mpqemubr0) 52:54:00:27:2b:18
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPOFFER(mpqemubr0) 10.215.78.16 52:54:00:27:2b:18
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPREQUEST(mpqemubr0) 10.215.78.16 52:54:00:27:2b:18
abr 02 22:04:55 hydraStation dnsmasq-dhcp[25174]: DHCPACK(mpqemubr0) 10.215.78.16 52:54:00:27:2b:18 surpassing-cobia
abr 02 22:07:12 hydraStation multipassd[23638]: No mounts to stop for instance "surpassing-cobia"
abr 02 22:07:12 hydraStation multipassd[23638]: QMP: {"return": {}}
abr 02 22:07:12 hydraStation multipassd[23638]: QMP: {"timestamp": {"seconds": 1585876032, "microseconds": 769106}, "event": "POWERDOWN"}
abr 02 22:07:12 hydraStation multipassd[23638]: VM powering down
abr 02 22:07:13 hydraStation multipassd[23638]: QMP: {"timestamp": {"seconds": 1585876033, "microseconds": 337650}, "event": "SHUTDOWN", "data": {"guest": true}}
abr 02 22:07:13 hydraStation multipassd[23638]: VM shut down
abr 02 22:07:13 hydraStation multipassd[23638]: process finished with exit code 0
abr 02 22:07:43 hydraStation multipassd[29227]: Applying AppArmor policy: multipass.qemu-img
abr 02 22:07:43 hydraStation multipassd[29233]: Applying AppArmor policy: multipass.qemu-img
abr 02 22:07:43 hydraStation multipassd[23638]: process working dir '/snap/multipass/1784/qemu'
abr 02 22:07:43 hydraStation multipassd[23638]: process program 'qemu-system-x86_64'
abr 02 22:07:43 hydraStation multipassd[23638]: process arguments '--enable-kvm, -device, virtio-scsi-pci,id=scsi0, -drive, file=/var/snap/multipass/common/data/multipassd/vault/instances/grand-hake/ubuntu-18.04
abr 02 22:07:43 hydraStation multipassd[23638]: qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.vmx [bit 5]
abr 02 22:07:43 hydraStation multipassd[29323]: Applying AppArmor policy: multipass.grand-hake.qemu-system-x86_64
abr 02 22:07:43 hydraStation multipassd[23638]: process started

Additional info

• OS:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.4 LTS"
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

Name:           primary
State:          Stopped
IPv4:           --
Release:        --
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           --
Disk usage:     --
Memory usage:   --
Mounts:         /home/vay3t => Home
                    UID map: 1000:default
                    GID map: 1000:default

Name:           canny-rattler
State:          Stopped
IPv4:           --
Release:        --
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           --
Disk usage:     --
Memory usage:   --

Name:           unbeatable-earwig
State:          Running
IPv4:           10.75.70.118
Release:        Ubuntu 18.04.4 LTS
Image hash:     fe3030939822 (Ubuntu 18.04 LTS)
Load:           0.00 0.02 0.04
Disk usage:     1001.5M out of 4.7G
Memory usage:   77.2M out of 985.1M

Additional context
My user is in sudo

[Saviq]

Hi @vay3t it looks like something went wrong with routing the traffic into the instances, do you have a firewall running that could interfere?

Here are the relevant bits from iptables that ensure the instances get access to the Internet:

$ sudo iptables -t filter -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps /* generated for Multipass network mpqemubr0 */

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  10.105.67.0/24       anywhere             /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  anywhere             10.105.67.0/24       ctstate RELATED,ESTABLISHED /* generated for Multipass network mpqemubr0 */
REJECT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */ reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */ reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp spt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp spt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp spt:bootps /* generated for Multipass network mpqemubr0 */

$ sudo iptables -t nat -L
# ...
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  10.105.67.0/24      !10.105.67.0/24       /* generated for Multipass network mpqemubr0 */
MASQUERADE  udp  --  10.105.67.0/24      !10.105.67.0/24       /* generated for Multipass network mpqemubr0 */ masq ports: 1024-65535
MASQUERADE  tcp  --  10.105.67.0/24      !10.105.67.0/24       /* generated for Multipass network mpqemubr0 */ masq ports: 1024-65535
RETURN     all  --  10.105.67.0/24       255.255.255.255      /* generated for Multipass network mpqemubr0 */
RETURN     all  --  10.105.67.0/24       base-address.mcast.net/24  /* generated for Multipass network mpqemubr0 */

Does this shed any light?

[vay3t]

I use firewalld but... okay

firewalld

$ sudo firewall-cmd --list-all
public (active)
  target: default
  icmp-block-inversion: no
  interfaces: wlan0
  sources: 
  services: ssh dhcpv6-client
  ports: 
  protocols: 
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 

iptables

$ sudo iptables -t filter -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp dpt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp dpt:bootps /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
INPUT_direct  all  --  anywhere             anywhere            
INPUT_ZONES_SOURCE  all  --  anywhere             anywhere            
INPUT_ZONES  all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  10.75.70.0/24        anywhere             /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  anywhere             10.75.70.0/24        ctstate RELATED,ESTABLISHED /* generated for Multipass network mpqemubr0 */
ACCEPT     all  --  anywhere             anywhere             ctstate RELATED,ESTABLISHED
ACCEPT     all  --  anywhere             anywhere            
FORWARD_direct  all  --  anywhere             anywhere            
FORWARD_IN_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_IN_ZONES  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES_SOURCE  all  --  anywhere             anywhere            
FORWARD_OUT_ZONES  all  --  anywhere             anywhere            
DROP       all  --  anywhere             anywhere             ctstate INVALID
REJECT     all  --  anywhere             anywhere             reject-with icmp-host-prohibited
REJECT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */ reject-with icmp-port-unreachable
REJECT     all  --  anywhere             anywhere             /* generated for Multipass network mpqemubr0 */ reject-with icmp-port-unreachable

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp spt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp spt:domain /* generated for Multipass network mpqemubr0 */
ACCEPT     udp  --  anywhere             anywhere             udp spt:bootps /* generated for Multipass network mpqemubr0 */
OUTPUT_direct  all  --  anywhere             anywhere  

$ sudo iptables -t nat -L
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  10.75.70.0/24       !10.75.70.0/24        /* generated for Multipass network mpqemubr0 */
MASQUERADE  udp  --  10.75.70.0/24       !10.75.70.0/24        /* generated for Multipass network mpqemubr0 */ masq ports: 1024-65535
MASQUERADE  tcp  --  10.75.70.0/24       !10.75.70.0/24        /* generated for Multipass network mpqemubr0 */ masq ports: 1024-65535
RETURN     all  --  10.75.70.0/24        255.255.255.255      /* generated for Multipass network mpqemubr0 */
RETURN     all  --  10.75.70.0/24        base-address.mcast.net/24  /* generated for Multipass network mpqemubr0 */
POSTROUTING_direct  all  --  anywhere             anywhere            
POSTROUTING_ZONES_SOURCE  all  --  anywhere             anywhere            
POSTROUTING_ZONES  all  --  anywhere             anywhere   

Notes

I just realized that I don't have dns but I do have internet.

[Saviq]

I use firewalld but... okay

I just realized that I don't have dns but I do have internet.

That suggests the requests to our internal dnsmasq are blocked.

Can you try, from both your host and the instance:

$ dig google.com @10.75.70.1
# ...
;; ANSWER SECTION:
google.com.             216     IN      A       172.217.16.46
# ...

If it works from the host, but not the instance - you may need to open port 53 from the mpqemubr0 subnet to the dnsmasq host.

Looking at journalctl -u snap.multipass* may also shed a light on what's going if neither work - it would mean that our dnsmasq can't reach out to the internet.

[vay3t]

I have evidence of the logs, I share them with you

[Saviq]

@vay3t it's best to use code blocks (```) to include logs, or attached file rather than screenshots.

I reproduced your problem. Is /etc/dnsmasq.conf something that you use? Our instance tries to read it in and fails on it. It should never have read it in the first place. I'll PR a fix to that effect.

[Saviq]

@vay3t if you can move the /etc/dnsmasq.conf file away, snap restart multipass should make it work again. You can then move the file back.

[Saviq]

@vay3t another way to workaround for now:

$ sudo systemctl edit snap.multipass.multipassd.service <<EOF
[Service]
Environment=DISABLE_APPARMOR=1
EOF