Add on: Multus

Multus is a CNI manager which enables attaching multiple network interfaces to pods in Kubernetes (details here).

The install is transparent and the current CNI plugin is carried forward as the default in Multus. With Multus enabled, users can easily add multiple interfaces to pods using varying CNI plugins (like bridge, macvlan, ipvlan, ptp, etc.).

This is handy for small local cases where you want to attach to local networks/vlans or in much more complex setups where multiple more fully featured CNI plugins can be used simultaneously.

Enable the Multus add-on by running the command:

microk8s enable multus

You can confirm it’s installed by verifying the daemonset is running:

microk8s kubectl get pods -n kube-system --selector=app=multus

Which should return output similar to the following:

NAME                         READY   STATUS    RESTARTS   AGE
kube-multus-ds-amd64-gtxq2   1/1     Running   0          2m36s

Once enabled you can create network attachment definitions for various networks you want your pods to connect to. Here is an example that creates two networks that attach to Linux bridges (br100 & br200) on the host machine:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: home-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "home-network",
    "type": "bridge",
    "bridge": "br100",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.201",
      "rangeEnd": "192.168.1.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.1.1"
    }
  }'
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: work-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "work-network",
    "type": "bridge",
    "bridge": "br200",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.2.0/24",
      "rangeStart": "192.168.2.201",
      "rangeEnd": "192.168.2.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.2.1"
    }
  }'

Then you can create pods that attach to those networks using annotations. Here is an example of an alpine image that connects to the two above networks:

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: multus-alpine
  name: multus-alpine
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
      {
         "name" : "home-network",
         "interface": "eth1",
         "ips": ["192.168.1.205"]
      },
      {
         "name" : "work-network",
         "interface": "eth2",
         "ips": ["192.168.2.210"]
      }
    ]'
spec:
  containers:
    - name: multus-alpine
      image: alpine:latest
      command: ["sh"]
      args: ["-c", "while [ true ]; do ifconfig; sleep 3; done"]
  restartPolicy: Always

This example uses an assigned IP but if you leave it out one will be assigned from the network range.

Should you need to disable the plugin you can do so by running:

microk8s disable multus

More details can be found at the Multus page. In particular in the how-to-use section of the repository docs:
k8snetworkplumbing multus-cni repository