Serializable state restoration (for restoring search results on Back navigation)

[ole]

This is a feature request for a public API for:

1. Vending Pagefind’s internal state in a serializable form, e.g. JSON
2. Initializing Pagefind with a copy of the serialized state to restore the previous state

Use case: I'd like to be able to restore the search page (e.g. /search) incl. any search results, scroll position etc. when the user navigates back to the search results using the browser's Back button.

Scenario:

1. The user performs a search and clicks on one of the search results.
2. That page wasn't what the user was looking for, so they click the browser's Back button to get back to the search page to try another search result. Ideally, the search results page would be restored in its previous form without having to re-run the search.

I think this is a very common scenario and it would be great if Pagefind supported it.

As far as I can tell, currently 2 things can happen when the user navigates back to a dynamically generated search results page:

1. If the user is lucky, the search results page is in the browser’s bfcache, i.e. the browser has frozen the entire state of the page, incl. the JavaScript state, and can restore it fully.

   This is the ideal outcome, but web developers can't rely on this happening. There are tons of reasons why a page is not eligible for bfcache inclusion, and some of these are not under the website's control, e.g. a browser extension that listens to the unload event. And even if the search page initially lands in the bfcache, browsers tend to evict pages pretty aggressively because the cache is expensive. I want to provide a fallback for this situation that tries to restore the search page to its previous state.

2. If the search page is not in the bfcache, all the page's JS code executes again, i.e. Pagefind is initialized to its initial state again.

   As the site's developer, I can add some JS code that:

   1. Saves the most recent search term and the scrol position in sessionStorage when the user navigates away from the search page
   2. When the user navigates back to the search results page, load the previous search term from sessionStorage and tell Pagefind to re-run the search. Then, after the search has completed and the search results have been populated, scroll back to the previous scroll position.

   This works, but it's not ideal because (a) running the search again is wasteful and takes at least a few hundred milliseconds, resulting in noticeable "flickering" to the user, and (b) restoring the scroll position after the search has completed is tricky to get right and causes more "flickering".

What Pagefind state restoration would enable

I'd like item (2) above to work as follows, but this would require Pagefind to provide new state restoration APIs:

1. In a pagehide event handler (when the user navigates away from the page), check if the current search term is non-empty. If yes, ask Pagefind for its current internal state and save that to session storage. Pseudocode:

   addEventListener("pagehide", (_e) => {
     if (!searchTerm || searchTerm === "") {
       return
     }
     // Generate a unique ID for the current page
     const pageStateID = crypto.randomUUID()
     const pagefindState = pagefind.serializableState()
     const pageState = {
       scrollPosition: window.scrollY,
       pagefindState,
     }
     // Store state in sessionStorage
     sessionStorage.setItem(pageStateID, JSON.stringify(searchState))
     // Associate the saved state with the current history item
     history.replaceState(pageStateID, "")
   })

2. In a pageshow event handler (when the user navigates back to the search page), first check if the page has been restored from the bfcache (i.e. event.persisted === true). If not, look into history.state if we have saved state for this history entry. If yes, retrieve the saved state from sessionStorage and initialize the Pagefind object with the stored state. E.g. const pagefind = new Pagefind(savedState).

Current workarounds

Since full state restoration isn't currently available in Pagefind, I experimented with storing the search results you get from Pagefind's search API or the onresults callback in the modular API. But the returned object is impossible to serialize to session storage because each result object contains a function data(). As far as I can tell, serializing the search results with the current public API would only work once the details data for each result has been fetched. And even then Pagefind doesn't provide an API to re-inject those results into a new instance on restoration.

I also experimented with storing the actual DOM of the search results (i.e. the generated HTML) in session storage and restoring that when the user navigates back. This is a hack, but it actually works quite well in terms of user experience (no waiting or flickering), albeit only for the results whose details have already been fetched, the non-fetched results can only be represented as placeholders in the HTML. Pagefind of course doesn't know anything about this HTML, so you'd have to tell it to re-run the search to get both states in sync. As I said, it's a hack.

On the granularity of Pagefind's internal state

I haven't looked at Pagefind's implementation to see how difficult it would be to produce a serializable version of its internal state.

I realize that it's probably very tricky to produce a serializable (and restorable) internal state at each point in time, e.g. while a search is running. But maybe that's not necessary. In the common case, the search has completed and Pagefind has an array of N search results, some of which have been fully loaded and some are placeholders that are still waiting to be loaded.

If Pagefind could vend this state as serializable JSON and was able to initialize a new instance with it, that would be extremely useful.

I also realize that Pagefind's internal state may depend on the version of the search index, so a stored state may become invalid once the search index gets regenerated. I think this is totally fine. I also think it would be completely acceptable if the format of Pagefind's internal state was not guaranteed to be stable between Pagefind versions. State restoration would still be very useful, even if it failed every now and then.

Lastly, Pagefind is great. Thanks for making it!