packet: Read BGP peer address from metadata service

In some Packet facilities the BGP peer address isn't the same as the
gateway address allocated for a host. Rather, it is a loopback
address that's reachable via the gateway.

The Packet metadata service now exposes BGP info to hosts, so we can
query the metadata service for the BGP peer address.

We currently use the first peer address only since MetalLB doesn't
support multiple node peers yet.

The source address is explicitly specified since when the peer
address is a loopback address, the source IP addresses which ends up
getting selected by the kernel is the node's *public* address which
doesn't work. In cases where the peer address is the gateway address
there is no harm in explicitly specifying the source.

Fixes #1009.

Author: johananl

assets/terraform-modules/packet/flatcar-linux/kubernetes/workers/cl/worker.yaml.tmpl

@@ -82,12 +82,9 @@ systemd:
         ExecStartPre=/usr/bin/bash -c "grep 'certificate-authority-data' /etc/kubernetes/kubeconfig | awk '{print $2}' | base64 -d > /etc/kubernetes/ca.crt"
         ExecStartPre=-/usr/bin/rkt rm --uuid-file=/var/cache/kubelet-pod.uuid
         ExecStartPre=/etc/kubernetes/configure-kubelet-cgroup-driver
-        # TODO: Workaround until https://github.com/coreos/afterburn/pull/358
-        # makes it into Flatcar. Then we can read COREOS_PACKET_IPV4_PRIVATE_GATEWAY_0
-        # from /run/metadata/flatcar and disable the template conditionals below.
         ExecStart=/bin/sh -c \
             "%{~ if bgp_node_labels != "" ~}
-            BGP_PEER_ADDRESS=$(ip route | grep '10.0.0.0/8' | awk {'print $3'}); \
+            BGP_PEER_ADDRESS=$(curl -s https://metadata.packet.net/metadata | jq -r .bgp_neighbors[0].peer_ips[0]); \
             %{~ endif ~}
             /usr/lib/coreos/kubelet-wrapper \
             --node-ip=$${COREOS_PACKET_IPV4_PRIVATE_0} \
@@ -112,7 +109,9 @@ systemd:
             --node-labels=$${NODE_LABELS} \
             --node-labels=lokomotive.alpha.kinvolk.io/public-ipv4=$${COREOS_PACKET_IPV4_PUBLIC_0} \
             %{~ if bgp_node_labels != "" ~}
-            --node-labels=$${BGP_NODE_LABELS},metallb.lokomotive.io/peer-address=$BGP_PEER_ADDRESS \
+            --node-labels=$${BGP_NODE_LABELS} \
+            --node-labels=metallb.lokomotive.io/peer-address=$BGP_PEER_ADDRESS \
+            --node-labels=metallb.lokomotive.io/src-address=$${COREOS_PACKET_IPV4_PRIVATE_0} \
             %{~ endif ~}
             --pod-manifest-path=/etc/kubernetes/manifests \
             --read-only-port=0 \