Most people think you only can have one network namespace per k8s pod. Does that have to be the case? Maybe you want the process to be completely isolated from the pod network. While this sounds a little odd, this is a requirement in a lot of user facing cloud services. This also throws a monkey wrench into the current behavior. The container runtime creates the network namespace and executes the CNI plugin. Multus is useful here for executing additional CNI plugins via the NetAttachDef resource however what is creating the extra network namespace? The CNI plugin executed by Multus is creating and setting up the network namespace. Sound weird? It is!

Lets walk through the yaml below. Notice in the command for the pod below that is using

ip netns exec bgp ....command goes here.....

What makes this creative is that we don’t need to change the network namespace path since the volume mount keeps it static. In this case its path is /run/netns/bgp. You will notice that the Pod and NetworkAttachmentDefinition need to be dynamically so that the network namespace paths line up otherwise the pod will just go into a crash loop. In the case below the netid with a value of 1234567 is passed into the CNI plugin (which creates the network namespace with the name of 1234567) and the hostpath is /run/netns/1234567.

apiVersion: v1
kind: Pod
metadata:
  name: router
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/networks: cust01-network
spec:
  containers:
    - image: frrouting/frr
      command:
        - "ip"
        - "netns"
        - "exec"
        - "bgp" 
        - "/sbin/tini"
        - "--"
        - "/usr/lib/frr/docker-start"
      imagePullPolicy: IfNotPresent
      name: router
      volumeMounts:
        - mountPath: /run/netns/bgp
          name: bgp-volume
        - mountPath: /etc/frr
          name: config-volume
      securityContext: #If you comment this out, then the container doesn't have the ability to do anything priv like 'ip a, ip r'
        privileged: true
  volumes:
    - name: bgp-volume
      hostPath:
        # directory location on host
        path: /run/netns/1234567
    - name: config-volume
      configMap:
        name: frr-daemons

---

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: cust01-network
spec:
  config: '{
  "cniVersion": "0.4.0",
  "name": "customer-1",
  "type": "make-net",
  "netid": "1234567"
}'

link to demo