Mounting multiple disks in iPXE mode

Test Environment

Term	Definition
Host OS	MacOS high Sierra
RancherOS	v1.1.3
Platform	Virtualbox
Root Disk(/dev/sda)	4GB
Other Disk(/dev/sdb)	3GB
CPU	1C
MEM	2GB

We use this environment to demonstrate the whole process.

Enable iPXE

Follow the previous instructions to create a VM named IPXE. In the boot order, the hard disk boot is the first. Then we need to create the boot file of iPXE at the host location, the path here should be ~/Library/VirtualBox/TFTP/IPXE.pxe, its content is probably like this:

#!ipxe
dhcp
set base-url http://192.168.1.125/rancheros/ipxe
kernel ${base-url}/vmlinuz printk.devkmsg=on console=tty1 rancher.autologin=tty1 rancher.cloud_init.datasources=[url:${base-url}/cloud-config.yaml] rancher.password=rancher
initrd ${base-url}/initrd
boot

The content of cloud-config.yaml can be defined by the user. Here we configure like this:

#cloud-config
runcmd:
- [ mkdir, -p, /mnt/user-docker]
- [ mount, -t, ext4, /dev/sdb, /mnt/user-docker ]

rancher:
  docker:
    graph: "/mnt/user-docker"

...
...

Mounting multiple disks

Now we can boot the IPXE VM for the first time. After booting is complete, we can ssh to this VM with rancher password:

# There are two disks
[root@rancher rancher]# fdisk -l
Disk /dev/sdb: 3 GiB, 3221225472 bytes, 6291456 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sda: 4 GiB, 4294967296 bytes, 8388608 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


# We haven't mounted any disks yet, so you can see these
[root@rancher rancher]# df -hT
Filesystem           Type            Size      Used Available Use% Mounted on
overlay              overlay       998.1M    180.2M    817.9M  18% /
tmpfs                tmpfs         942.0M         0    942.0M   0% /dev
tmpfs                tmpfs         998.1M         0    998.1M   0% /sys/fs/cgroup
tmpfs                tmpfs         998.1M         0    998.1M   0% /media
none                 tmpfs         998.1M    752.0K    997.4M   0% /run
tmpfs                tmpfs         998.1M    100.0K    998.0M   0% /mnt
devtmpfs             devtmpfs      942.0M         0    942.0M   0% /host/dev
shm                  tmpfs          64.0M         0     64.0M   0% /host/dev/shm
none                 tmpfs         998.1M    752.0K    997.4M   0% /var/run
tmpfs                tmpfs         998.1M    180.2M    817.9M  18% /etc/hostname
shm                  tmpfs          64.0M         0     64.0M   0% /dev/shm
devtmpfs             devtmpfs      942.0M         0    942.0M   0% /dev
shm                  tmpfs          64.0M         0     64.0M   0% /dev/shm

Let's manually format /dev/sdb first, here we need to format all the disks except the root disk:

[root@rancher rancher]# mkfs.ext4 /dev/sdb
mke2fs 1.43.9 (8-Feb-2018)
Creating filesystem with 786432 4k blocks and 196608 inodes
Filesystem UUID: b7c1c6d1-b96b-4783-8c1f-9cd375c404dd
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

Then we have to install ros on the root disk(/dev/sda):

$ wget http://192.168.1.125/rancheros/ipxe/cloud-config.yaml
$ ros install -c cloud-config.yaml -d /dev/sda

When the VM starts up again, we can check the output of ros c export:

...
rancher:
  docker:
    graph: /mnt/user-docker
runcmd:
- - mkdir
  - -p
  - /mnt/user-docker
- - mount
  - -t
  - ext4
  - /dev/sdb
  - /mnt/user-docker
...

Also we can check the user-docker and mount points.

[root@rancher rancher]# docker info
...
...
Name: rancher
ID: 2VW4:R7DN:KPX7:EIGW:XSAM:QPFU:VFPG:LJAJ:GTJX:RZ5H:Z7G3:N2JS
Docker Root Dir: /mnt/user-docker
Debug Mode (client): false
Debug Mode (server): false
...
...

[root@rancher rancher]# df -hT | grep sdb
/dev/sdb             ext4            2.9G      9.3M      2.8G   0% /mnt/user-docker
/dev/sdb             ext4            2.9G      9.3M      2.8G   0% /mnt/user-docker/plugins
/dev/sdb             ext4            2.9G      9.3M      2.8G   0% /mnt/user-docker/overlay

Troubleshooting

If you don't get the expected result, you can check these two points:

1. The output of ros c export, this can make you know if runcmd is actually set.
2. The console logs system-docker logs console, this can make you know if runcmd runs failed.

Others

There was a bug for cloudinit url datasource in v1.1.3, please refer to this issue . In the case of multiple nics, the content of the url sometimes fails to download.

It has been fixed in v1.1.4 and v1.2.0.