Feature Request: Support additional resources in ClusterServiceVersion, like StatefulSet and DaemonSet

[27149chen]

Currently in the csv, only deployment, role/rolebinding and clusterrole/clusterrolebinding are supported to describe an operator installation.
However, to gain more control of the application or for other perpose, the operator needs not only a deployment (the controller), but also a StatefulSet (for stateful thing) or a DaemonSet (something like a node agent).
We have an operator which is composed by one deployment and one daemonset, which can not be installed by a csv. It will be great if csv can support it, thanks.

[dmesser]

Looking for some information on the requirements: when your Operator get's installed, it's controller, CRDs and required RBAC will be put in place. It's then ready to provide it's services via it's API (the CRDs).
Is your requirement that you start acting on cluster without user requests? Because what you describe falls outside the definition of what makes up an Operator and should rather be deployed by your Operator instead of with your Operator.

[27149chen]

@dmesser
I want to do the following things in the DaemonSet:

1. Gathering some node info which will be used for all the instances of the CRDs (sounds like "start acting on cluster without user requests")
2. Prepare a node agent which the controller will talk to when an instance is created, so action happens after the instance creation.

What's your suggestion to deploy this DaemonSet? with my Operator or by my Operator?

[dmesser]

With what's possible now you'd deploy the DaemonSet via your Operator. It could do this on start-up using an init-container or as part of a Configuration CR that "configures" your Operator. The Operator could also creates this CR on start up - this would have the benefit, that your Operator will lifecycle it, in case it get's deleted or modified.

[27149chen]

@dmesser Yes, I can deploy it during the controller start-up, but it is just a workaround. I don't think "Configuration CR" is a good idea because it is part of the operator, and should be managed by the olm in openshift, not the operator itself.
So it will be great if csv can support to create resources other than Deployment, which make it possible to install more complex operators.

[dmesser]

I think we would need to look at the consequences of supporting what you ask for.
What you describe (Node agents in DaemonSet, etc), falls outside of the definition of what an Operator is: a controller owning certain CRDs.

If all the other stuff would become part of OLM we would also need to support configuring those - e.g. could you just instantiate the DaemonSet or would it need some configuration from the user/admin?. Normally this would come via the CR. Since the CRDs are not yet reconciled at install time, OLM becomes responsible for providing a user interface. Also, could you reliably create the DaemonSet if the Operator Deployment is not active yet? I.e. do you need ordering?
Now the point where OLM transforms into a fully configurable, YAML manifest template solution is not that distant anymore.

Whereas with the definition now, the boundaries of responsibilities is very clear: OLM deploys your Operator and your RBAC, an operator being one or more controllers in pods owning CRDs.
Everything else is up to the logic and value-add of your Operator. Default configurations can be achieved using Configuration CRs or init containers. It's difficult for OLM to own these since the requirements would be so broad, that OLM would become a general-purpose deploy-my-bag-of-YAML controller. And people would rely on it instead of building the Operator to a higher maturity level.

[27149chen]

@dmesser , thank you for your explanation. I totally agree that the olm should not become a general-purpose deploy-my-bag-of-YAML controller. Now I understand why I should use a "Configuration CR" to deploy it. Thank you.