[JWT] Add more timeout for JWT validation

[johncalesp]

I think as of now, platform dev tries to verify the API key every hour??. It killed my process with this message when trying to delete endpoint.

Traceback (most recent call last):
  File "/home/john/Documents/centml/platform/centml-python-client/sandbox/endpoint.py", line 202, in <module>
    cclient.delete(deployment_id)
  File "/home/john/Documents/centml/platform/centml-python-client/centml/sdk/api.py", line 50, in delete
    self._update_status(id, DeploymentStatus.DELETED)
  File "/home/john/Documents/centml/platform/centml-python-client/centml/sdk/api.py", line 47, in _update_status
    self._api.update_deployment_status_deployments_status_deployment_id_put(id, status_req)
  File "/home/john/Documents/centml/platform/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 38, in wrapper_function
    return wrapper(*args, **kwargs)
  File "/home/john/Documents/centml/platform/venv/lib/python3.10/site-packages/pydantic/_internal/_validate_call.py", line 111, in __call__
    res = self.__pydantic_validator__.validate_python(pydantic_core.ArgsKwargs(args, kwargs))
  File "/home/john/Documents/centml/platform/venv/lib/python3.10/site-packages/platform_api_python_client/api/external_api.py", line 6378, in update_deployment_status_deployments_status_deployment_id_put
    return self.api_client.response_deserialize(
  File "/home/john/Documents/centml/platform/venv/lib/python3.10/site-packages/platform_api_python_client/api_client.py", line 322, in response_deserialize
    raise ApiException.from_response(
  File "/home/john/Documents/centml/platform/venv/lib/python3.10/site-packages/platform_api_python_client/exceptions.py", line 145, in from_response
    raise UnauthorizedException(http_resp=http_resp, body=body, data=data)
platform_api_python_client.exceptions.UnauthorizedException: (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'www-authenticate': 'Bearer realm="https://api.centml.org/deployments/status/3897", error="invalid_token"', 'content-length': '14', 'content-type': 'text/plain', 'date': 'Wed, 22 Jan 2025 22:37:06 GMT', 'server': 'istio-envoy', 'x-envoy-upstream-service-time': '6'})
HTTP response body: Jwt is expired

It will help a lot if we can do either of these two options:

• Somehow validate transparently the client's API key after certain timeout
• Extend the validation timeout to 90mins or more

I prefer the first option if it is possible.
Do you guys know if I make another call through the SDK the JWT token renews automatically? I was thinking of checking the status to reset the timeout

┆Issue is synchronized with this Notion page by Unito

[johncalesp]

@michaelshin can you check if you have seen this error before?

[michaelshin]

I don't think we can change the timeout ourselves as the JWT is issued by Firebase.
You may have to wrap the api calls to always get the JWT since we have the refresh token in the credentials.json

[johncalesp]

if I were to do this:

for rps_rate in rps_rates:
            for inp, out in uniform_distribution_pairs:
                config = generate_json_config_file(BASE_URL, MODEL, rps_rate, inp, out)
                run_benchmark(config)
                time.sleep(3)
                # check status of endpoint
                status = cclient.get_status(deployment_id)
                print(status.service_status)

when I call get_status() does that trigger a reset of the timeout?