Get Rid Of the Standalone Metadata UI

[jlewi]

/kind feature

Right now there are two metadata UIs.

1. Built into KFP (i.e. the Execution and Artifacts view inside KFP)
2. A standalone version of the UI for metadata
   • https://github.com/kubeflow/metadata/tree/master/frontend
   • This is what the ArtifactStore in central dashboard links to

The standalone UI has fallen behind and per #225 it doesn't look like anyone will be maintaining it. So it looks like we should get rid of it. It looks like the work items are

• Delete https://github.com/kubeflow/metadata/tree/master/frontend
• Remove all the metadata-ui resources from https://github.com/kubeflow/manifests/tree/master/metadata/base
• Get rid of the central-dashboard link to the Artifact store

@yanniszark @kimwnasptd I believe you will be the ones maintaining the centraldashboard could you take a look at removing the artifact store link?

@kubeflow/wg-pipeline-leads Is there someone on the KFP side that could looking into cleanup up the metadata K8s manifest?

Related to #225

[Jeffwan]

@jlewi

em. How does user check artifact if user uses metadata outside KFP? Should we redirect them to KFP pipeline UI?

I think there're some users use SDK to log_params like this way. https://github.com/kubeflow/metadata/blob/master/sdk/python/sample/demo.ipynb

[Bobgy]

@Jeffwan do you have contact with such users?
I think if the existing UI works for them, we can keep it until KFP can provide sth that's a superset of its features and replace it.

But I kind of doubt if users can successfully use that UI, because it suffers from the same scalability issue KFP artifact list is having: there was no pagination for the artifact list request, so the UI could only show everything in one list. When artifact count is getting larger, the page will start to break.

[jlewi]

Should we redirect them to KFP pipeline UI?

Yes. I checked and Executions and Artifacts not logged through KFP still show up there.

[Jeffwan]

@Jeffwan do you have contact with such users?
I think if the existing UI works for them, we can keep it until KFP can provide sth that's a superset of its features and replace it.

But I kind of doubt if users can successfully use that UI, because it suffers from the same scalability issue KFP artifact list is having: there was no pagination for the artifact list request, so the UI could only show everything in one list. When artifact count is getting larger, the page will start to break.

@Bobgy I don't have a list. We suggested AWS users to consider this solution for tracking purpose and built some tutorials in the workshops. I don't know how many users are really on it. I think if we think KFP UI is the replacement, it makes sense to remove the one from central dashboard.

[Bobgy]

Should we redirect them to KFP pipeline UI?

Yes. I checked and Executions and Artifacts not logged through KFP still show up there.

Right, in this case we can get rid of metadata UI

[Bobgy]

I will update k8s metadata manifest

[Bobgy]

@Jeffwan @jlewi I want to confirm, are there still dependents of metadata-server?
https://github.com/kubeflow/manifests/blob/729e3cda92b7c8656441480995da6fd0a9a18ce3/metadata/overlays/db/metadata-deployment.yaml#L4

KFP is not depending on it, KFP uses metadata-grpc-server directly

[Jeffwan]

@Bobgy I think it would be great to keep them. I think SDK use http endpoint, if we remove HTTP deployment, SDK should be deprecated as well.

User can still use SDK and check artifacts in pipeline manifest UI. We need to have more discussion on the roadmap. Before we have the conclusion, I would suggest to only delete UI components

[Bobgy]

I see, if we cannot remove http endpoint, then we cannot remove UI server yet.

[neuromage]

We would like to suggest we delete both SDK and UI here. Instead, we can consolidate this within KFP, where there is already a UI for metadata. Furthermore, we'd like to propose that we add SDK features directly into KFP SDK instead, hence allowing users to use one SDK to both construct pipelines, and query metadata from their notebooks.

We have outlined a proposal here, to be discussed during the upcoming community meeting:
https://docs.google.com/document/d/1YhnsMK_ktaeUKLS2XShocq4u2bcUCUcztT9rioRssOM/edit#

Would love your feedback!

[Jeffwan]

Great. If KFP already have a plan. We can deprecate UI server and SDK together along with frontend. @Bobgy Can you update the manifest PR?

[Bobgy]

OK, SGTM