Smart Comparing Specs To Check If Dependent Resource Needs to Applied Again

[csviri]

Problem Statement

For dependent resources we want to have a default behavior to compare k8s resource specs in order to check if the desired resource needs to be reapplied or not. For example if somebody provides a desired definition of a target resource, like a Deployment object , after the apply K8S fills some attributes with default values. So the actual resource on the server will be always different specified in desired. So we cannot simply compare specs.

A sample jsonpatch after deployment is applied (desired vs applied):

{"op":"add","path":"/template/spec/containers/0/ports/0/protocol","value":"TCP"}
{"op":"add","path":"/template/spec/containers/0/imagePullPolicy","value":"IfNotPresent"}
{"op":"add","path":"/template/spec/containers/0/resources","value":{}}
{"op":"add","path":"/template/spec/containers/0/terminationMessagePath","value":"/dev/termination-log"}
{"op":"add","path":"/template/spec/containers/0/terminationMessagePolicy","value":"File"}
{"op":"add","path":"/template/spec/volumes/0/configMap/defaultMode","value":420}
{"op":"add","path":"/template/spec/dnsPolicy","value":"ClusterFirst"}
{"op":"add","path":"/template/spec/restartPolicy","value":"Always"}
{"op":"add","path":"/template/spec/schedulerName","value":"default-scheduler"}
{"op":"add","path":"/template/spec/securityContext","value":{}}
{"op":"add","path":"/template/spec/terminationGracePeriodSeconds","value":30}
{"op":"add","path":"/progressDeadlineSeconds","value":600}
{"op":"add","path":"/revisionHistoryLimit","value":10}
{"op":"add","path":"/strategy","value":{"rollingUpdate":{"maxSurge":"25%","maxUnavailable":"25%"},"type":"RollingUpdate"}}

In addition to that, for resources like Service the ClusterIP is changes for different instances of Service. (If this diff would be the same, we could just cache the diff and apply before comparison).

[csviri]

Solution

It's quite normal that resources are altered during create and/or update, adding default values but also think of mutation hooks (adding side cart containers or similar is quite common for example). One solution is to just compare if that desired state and actual state, if there are only additions accept that. So, this makes sure no value from desired spec is different from the actual spec (otherwise would be applied again), if a mutation hook changes a value of course this would fail, but probably that would be an exception where user can write it's own matcher. So:

  public boolean match(R actualResource, R desiredResource, Context context) {
    var desiredSpecNode = mapper.valueToTree(ReconcilerUtils.getSpec(desiredResource));
    var actualSpecNode = mapper.valueToTree(ReconcilerUtils.getSpec(actualResource));
    var diffJsonPatch = JsonDiff.asJson(desiredSpecNode, actualSpecNode);
    for (int i = 0; i < diffJsonPatch.size(); i++) {
      String operation = diffJsonPatch.get(i).get("op").asText();
      if (!operation.equals("add")) {
        return false;
      }
    }
    return true;
  }

This impl uses jackson, to create a jsonpatch between actual and desired state, if it contains only adds it means the desired values are not changed.

Of course for resources which don't have specs (like Secret, ConfigMap) will have different approach, like comparing data.

[csviri]

Just small additional to note: after created/update and possibly default values and mutation hooks are applied. Detecting further changes on spec can be done by recording the generation after create/update in memory. This might or might be not desirable though.