[frontend] Listing all runs take a lot of time

[axel7083]

Description

When we navigate to the runs page, listing all the runs. The page take a lot of times to loads.

The frontend make a request to /pipeline/apis/v2beta1/runs then returns a list of run, containing each of them a run-id and a pipeline_version_id, for each run, the frontend will query the run details, and pipeline version details, to get their names.

On our cluster, it can take up to 4 seconds for 10 runs. Maybe we should includes in the /pipeline/apis/v2beta1/runs endpoint an optional query parameter to add directly the names to avoid making multiple requests.

---

Impacted by this bug? Give it a 👍.

[zijianjoy]

/cc @jlyaoyuli
/assign @zijianjoy

[zijianjoy]

@axel7083 This is a good point for improving the latency of rendering run list. There are multiple approaches to solve this problem:

1. Like you suggested, we might be able to extend /pipeline/apis/v2beta1/runs List call to include experiment name and run detail. However, it requires API changes and potentially database table-join action, it might not be ideal to include too many data in /pipeline/apis/v2beta1/runs List call, which is going to slow down this call because the amount of data transmit is too large for one call.
2. We can simplify by reducing repeated experiment query, by using useQuery() function and enable caching.
3. We can also adopt React Suspense, https://react.dev/reference/react/Suspense, so we can load the run list itself fairly quickly, while waiting for the rest of API calls to complete. Once all API calls return, run list page doesn't need to re-render.

@jlyaoyuli is doing some improvements to frontend API call performance currently.

I will open up the discussion and welcome more ideas to improve frontend page performance.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.