Add AWS Periodic Tests

[PatrickXYS]

Which issue is resolved by this Pull Request:
Resolves #

Description of your changes:
Add AWS Periodic Tests to test orchestration on Kubeflow AWS.

Also removes periodic tests for e2e and unit tests, since no one will check it daily and see if they succeed or fail, thus, it only consume resources without any contribution.

Checklist:

• Unit tests have been rebuilt:
  1. cd manifests/tests
  2. make generate-changed-only
  3. make test

[PatrickXYS]

/cc @Jeffwan

[terrytangyuan]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: PatrickXYS, terrytangyuan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [PatrickXYS,terrytangyuan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[Jeffwan]

em. I know kubeflow minimum distribution is still being discussed, but I hope to keep it until we figure it out If this is not a problem at this moment.

[Jeffwan]

Also removes periodic tests for e2e and unit tests, since no one will check it daily and see if they succeed or fail, thus, it only consume resources without any contribution.

This doesn't make sense to me. It's ok to add AWS or other cloud provider cases here. That's a separate story. AWS test won't guarantee kubeflow manifest wok for other platforms.

[PatrickXYS]

Also removes periodic tests for e2e and unit tests, since no one will check it daily and see if they succeed or fail, thus, it only consume resources without any contribution.

This doesn't make sense to me. It's ok to add AWS or other cloud provider cases here. That's a separate story. AWS test won't guarantee kubeflow manifest wok for other platforms.

I agree with the idea AWS test is separate story. The reason I removed minimum distribution periodic job is presubmit should be able to detect any failures, only situation I can come up with is establishing Kubeflow-Kubernetes Version Compatibility Matrix, but I don't think we want to support before 1.2.

[Jeffwan]

@PatrickXYS

Kubeflow compability matrics is a separate story.. We will give some matrix in 1.2 but that's not related.

I would suggest to bring it back since this is cloud agnostic and only have some core components and can reflect master/branch control plane status. Presubmit does help but this won't cover all the cases. For example, some dependency change (like kfctl, kuebrnetes cluster version), Github update (tar.gz or https issue in the past) or GCR outage, etc. This help us make quick response to Kubernete users.

We already had the pain in 1.1 and hope at least we can run it during release cycle.

[PatrickXYS]

@Jeffwan

Thank you for explanations! That sounds good to me, I'll add those periodic test back

[PatrickXYS]

Created PR #1615