Add external mysql option for pipeline and metadata

[Jeffwan]

Which issue is resolved by this Pull Request:
Resolves #905

Description of your changes:
Add two overlays to allow user to use external mysql servers.

Checklist:

• Unit tests have been rebuilt:
  1. cd manifests/tests
  2. make generate-changed-only
  3. make test

---

This change is 

[Jeffwan]

INFO|2020-02-12T20:36:07|/src/kubeflow/testing/py/kubeflow/testing/argo_client.py|31| Workflow kubeflow-manifests-presubmit-kfctl-go-iap-906-18ef263-7328-2a54 in namespace kubeflow-test-infra; phase=Failed

Seems it's unrelated and this PR only add new overlays.

[Jeffwan]

/test kubeflow-manifests-presubmit

[Jeffwan]

Temporarily put username and password to configmap. These will be used to replace placeholder in pipeline configmap. The challenge is the value from secret is encoded string and I can not find way to decode it easily.
secretGenerator can not use StringData and upstream issue is tracked here. kubernetes-sigs/kustomize#1444

[Jeffwan]

/cc @jlewi @kunmingg @zhenghuiwang @kkasravi

[jlewi]

/assign @zhenghuiwang

[zhenghuiwang]

/lgtm

The metadata part looks good. Do you plan to make it the default overlay for aws?

[zhenghuiwang]

/approve

[Jeffwan]

@zhenghuiwang
No, this is an option for users want to use cloud managed SQL service.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zhenghuiwang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [zhenghuiwang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xaniasd]

@Jeffwan thanks for this PR. How could I customize the values in secrets.env?

[Jeffwan]

@xaniasd Customize the values in secrets.env? Do you mean how to apply overlays? You can do like this for metadata and pipeline. Make sure you fill secrets.env and params.env

[xaniasd]

@Jeffwan no my question was about filling in secrets.env. I can customize parameters in params.env with the parameters map like so:

- kustomizeConfig:
    overlays:
    - application
    parameters:
    - name: MY_HOST
      value: db

As far as I can see in kfctl code, this is not possible for secrets.env files (maybe for good reason too). So how can I change the password?

[Jeffwan]

@xaniasd Hi, https://github.com/kubeflow/kfctl/blob/master/pkg/apis/apps/kfdef/v1/application_types.go doesn't have field to set secrets. Currently, only few application use secrets and I think it could be a feature request. You can kfctl build and manually update secrets.env at this moment.

I was trying to integrate to kfctl and expose config to users in manifest file, meet the same problem. That's definitely something we can improve.

[thesuperzapper]

@Jeffwan do you have plans to add support for external DB's in the katib-controller manifests?

See here:
https://github.com/kubeflow/manifests/blob/v1.0.2/katib/katib-controller/base/katib-mysql-deployment.yaml

[Jeffwan]

@thesuperzapper

There's some ongoing discussion there. Katib maintainers has some concerns on the base manifests since there's standalone users. But I think that's the trend, we should have similar pattern there. I will check it later.

for the RDS documentation, please check this website after this PR merged

kubeflow/website#1934