Identity Plugin RFC

docs/rfcs/0001-identity-plugin.md

@@ -0,0 +1,198 @@
+- Created:
+- Current Version:
+- Status: WIP
+- Contributors: Harshit Gupta ([email protected], [email protected]), Michael Brown ([email protected])
+- Approvers:
+
+# Summary
+
+The identity plugin, running as a daemonset, fetches identity artifacts for each annotated container and mounts the artifact volume into the container.
+
+# Motivation
+
+The current way how we mount identity artifacts into a container is to use a initContainer / sidecar. There is a sidecar for each pod. This sidecar fetches the artifacts from identity providers, injects the volume containing identity artifacts into the pod and mounts the volume into the container. Using a sidecar in each pod for fetching identity artifacts is too resource intensive. Using a initContainer sidecar comes with an additional overhead of increased container start latency. The motivation for using a NRI Identity Plugin is that only one process is enough to fetch the identity artifacts for all of the annotated containers in all the pods on a single host. Therefore the identity PLUGIN as a daemonset can fetch the identity artifacts for all the containers in all the pods on all thee hosts in the cluster that have the appropriate annotations. Running as a single process per host instead of single process per pod we save on host resources.
+
+# Proposed Implementation
+
+## Setup Steps
+
+1. Create a static spiffe-id for the plugin.
+
+```
+spire-server entry create \
+    -spiffeID spiffe://example.org/nri/identity-injector \
+    -selector k8s:ns:backend \
+    -selector k8s:sa:db-writer
+```
+
+PENDING - what selector's will the plugin have?
+
+
+2. https://spiffe.io/docs/latest/deploying/spire_agent/#delegated-identity-api Configure the Plugin's Spiffe ID in the Agent configuration.
+
+```
+agent {
+    trust_domain = "example.org"
+    ...
+    admin_socket_path = "/tmp/spire-agent/private/admin.sock"
+    authorized_delegates = [
+        "spiffe://example.org/nri/identity-injector"
+    ]
+}
+
+```
+
+
+## How the plugin will work
+
+1. Plugin registers itself to the agent and get its spiffe id.
+
+2. Containerd calls the NRI plugin on CreateContainer event. The plugin receives the pod and container metadata. Using this metadata, the plugin executes step 3 below. The plugin has to ensure that the pid is a stable identifier, that the pid is not a recycled identifier. This is a platform dependent step. (e.g. by using pidfds on Linux). https://spiffe.io/docs/latest/deploying/spire_agent/#delegated-identity-api
+
+3. The plugin reads the podspec and parses the annotations. Using spire-api-sdk, the plugin fetches identity artifacts and mounts the volume containing the identity artifacts into the container.  The spire-api-sdk streams artifact updates, the plugin will on receiving updates will overwrite the artifact on the host volume. Since the host volume is mounted into the container, the artifacts in the container are automatically updated. The plugin does not need a separate mount annotation in the podspec yaml. The daemonset specifies the volume that the plugin will use to fetch and download the identity artifacts. The plugin will then mount the host volume to the container volume using the cert_dir from the podspec.
+
+
+
+## DaemonSet yaml
+
+```
+apiVersion: apps/v1
+kind: DaemonSet
+metadata:
+  name: nri-plugin-identity-injector
+spec:
+  template:
+    spec:
+      priorityClassName: system-node-critical
+      containers:
+        - name: plugin
+          image: plugin:latest
+          args:
+            - "-idx"
+            - "10"
+          resources:
+            requests:
+              cpu: "2m"
+              memory: "5Mi"
+          securityContext:
+            allowPrivilegeEscalation: false
+            capabilities:
+              drop:
+                - ALL
+          volumeMounts:
+            - name: nri-socket
+              mountPath: /var/run/nri/nri.sock
+            - name: identity-artifacts                      # Artifact volume
+              mountPath: /var/identity/
+      volumes:
+        - name: nri-socket
+          hostPath:
+            path: /var/run/nri/nri.sock
+            type: Socket
+        - name: identity-artifacts                          # Artifact volume
+          hostPath:
+            path: /var/identity/
+            type: Directory
+
+```
+
+
+
+## Podspec example
+
+```
+apiVersion: v1
+kind: Pod
+metadata:
+  name: bbid0
+  labels:
+    app: bbid0
+  annotations:
+    identity.noderesource.dev/container.c0: |+
+      - spire_agent_address: /tmp/agent.sock
+        cert_dir: /var/certs
+        svid_file_name: svid.pem
+        svid_key_file_name: svid_key.pem
+        svid_bundle_file_name: svid_bundle.pem
+    identity.noderesource.dev/container.c1: |+
+      - spire_agent_address: /tmp/agent.sock
+        cert_dir: /var/certs
+        svid_key_file_name: svid_key.pem
+        svid_bundle_file_name: svid_bundle.pem
+spec:
+  containers:
+  - name: c0
+    image: busybox
+    imagePullPolicy: IfNotPresent
+    command:
+      - sh
+      - -c
+      - |
+        if [ -f /var/certs/svid.pem ]; then
+          echo "svid exists!"
+        else
+          echo "svid does NOT exist."
+        fi
+        sleep inf
+    resources:
+      requests:
+        cpu: 500m
+        memory: '100M'
+      limits:
+        cpu: 500m
+        memory: '100M'
+  - name: c1
+    image: busybox
+    imagePullPolicy: IfNotPresent
+    command:
+      - sh
+      - |
+        if [ -f /var/certs/svid.pem ]; then
+          echo "svid exists!"
+        else
+          echo "svid does NOT exist."
+        fi
+        sleep inf
+    resources:
+      requests:
+        cpu: 1
+        memory: '100M'
+      limits:
+        cpu: 1
+        memory: '100M'
+  - name: c2
+    image: busybox
+    imagePullPolicy: IfNotPresent
+    command:
+      - sh
+      - -c
+      - |
+        if [ -f /var/certs/svid.pem ]; then
+          echo "svid exists!"
+        else
+          echo "svid does NOT exist."
+        fi
+        sleep inf
+    resources:
+      requests:
+        cpu: 1
+        memory: '100M'
+      limits:
+        cpu: 1
+        memory: '100M'
+  terminationGracePeriodSeconds: 1
+
+```
+
+## Handling Openshift / cri-o
+
+Openshift / cri-o specific challenges that the plugin resolves.
+
+## Handling VMs
+
+
+# Alternatives
+
+An alternative to identity plugin is to use sidecar containers or initContainers in every pod for fetching the artifacts and mounting the artifact volume.
+
+# Open Questions