Expiration of pagination tokens on `/hierarchy` not specified

[turt2live]

Link to problem area: https://spec.matrix.org/v1.2/client-server-api/#get_matrixclientv1roomsroomidhierarchy

Issue

MSC2946 (MD) states:

When the current response page is full, the current state should be persisted and a pagination token should be generated (if there is more data to return). To prevent resource exhaustion, the server may expire persisted data that it deems to be stale.

however, this note about expiration is not included in the spec for clients to rely on.

Expected behaviour

The spec should clearly state the expiration requirements to avoid confusion such as matrix-org/synapse#12697 / element-hq/element-web#22138

Historical proof

This was implemented in matrix-org/synapse#10574 as part of maintenance work for matrix-org/synapse#10495 which appeared shortly after MSC2946 was updated.

— matrix-org/synapse#12697 (comment)

[anoadragon453]

Sounds like all that needs to happen here is for the spec to mandate that homeservers return M_UNKNOWN_TOKEN. Clients can then retry the pagination from the first page, i.e. omitting the unknown from token.

I don't think the spec should say how long homeserver implementations should keep tokens for. That should be up to implementation.