Watch cache regression: changes behavior of "resource version too old" error #25151
Description
When attempting to fetch a resource version that is too old, the behavior of watch (for both HTTP and web sockets) changed when watch cache was enabled. The watch cache returns a 410 - without the watch cache, we return 200 and then write an "error" entry to the watch stream. This broke a client that dependent on the watch behavior, and moreover is inconsistent with other errors we set in the registry. Also, for watch over web sockets, most javascript clients will be unable to get at a 410 error from the connection, because browsers don't get access to the data.
We should restore the watch cache behavior to be consistent with the previous behavior as this is a regression in API semantics.
Resource without watch cache, watching too old:
$ curl ... "https:///oapi/v1/namespaces/clayton-dev/builds?watch=1&resourceVersion=1"
> GET /oapi/v1/namespaces/clayton-dev/builds?watch=1&resourceVersion=1 HTTP/1.1
> Accept: application/json, */*
> User-Agent: oc/v1.3.0 (darwin/amd64) kubernetes/2787678
>
< HTTP/1.1 200 OK
< Cache-Control: no-store
< Transfer-Encoding: chunked
< Date: Wed, 04 May 2016 18:10:26 GMT
< Content-Type: text/plain; charset=utf-8
< Transfer-Encoding: chunked
<
{"type":"ERROR","object":{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"401: The event in requested index is outdated and cleared (the requested history has been cleared [4046756/2]) [4047755]","reason":"Expired","code":410}}
Resource with watch cache, watching too old:
$ curl ... "https:///apis/extensions/v1beta1/namespaces/clayton-dev/horizontalpodautoscalers?watch=1&resourceVersion=1"
> GET /apis/extensions/v1beta1/namespaces/clayton-dev/horizontalpodautoscalers?watch=1&resourceVersion=1 HTTP/1.1
> Accept: application/json, */*
> User-Agent: oc/v1.3.0 (darwin/amd64) kubernetes/2787678
>
< HTTP/1.1 410 Gone
< Cache-Control: no-store
< Content-Type: application/json
< Date: Wed, 04 May 2016 18:08:25 GMT
< Content-Length: 146
<
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"too old resource version: 1 (4034675)","reason":"Gone","code":410}