Support configuration to keep or delete tables when delete host.

[qixiaogang]

Can we support configuration to keep or delete tables when deleting host in https://github.com/Altinity/clickhouse-operator/blob/master/pkg/controller/chi/worker.go#L396

This is our case, when updating the chi, the sts fails, then we restart the operator to trigger creation of the sts. If creation of sts fails, we want to keep the PVC and data in the PVC, which means no deletion of the tables on host.

[alex-zaitsev]

It is possible to keep PVC and PV if you specify retain policy in PVC template:

    volumeClaimTemplates:
      - name: volume-claim-retain-pvc
        reclaimPolicy: Retain

[qixiaogang]

@alex-zaitsev we also want to keep the data in the PVC, so when delete the pod, we don't to execute https://github.com/Altinity/clickhouse-operator/blob/master/pkg/controller/chi/worker.go#L396

[qixiaogang]

@sunsingerus
volumeClaimTemplates:
- name: volume-claim-retain-pvc
reclaimPolicy: Retain
will only keep the PVC, it will execute drop table, which means data in the PVC will be deleted, we want to have the capability to keep the table and data when delete the host.

[alex-zaitsev]

Related #275

[sunsingerus]

volumeClaimTemplates:

• name: volume-claim-retain-pvc
  reclaimPolicy: Retain
  will only keep the PVC, it will execute drop table

it is not expected behavior. Possible bug.
Anyway, we plan to review PVC and PV management in near future

[alex-zaitsev]

Fixed in 0.9.6. If 'Retain' policy is used, replicated tables are not dropped.