Get JavaScript working (table.js, plugins and more)

[simonw]

Relates to plugins challenge:

• Get one or more plugins working #5

The table.js script used by the table page doesn't load at the moment, which means no cog icons on the columns:

[simonw]

The SQL editor JavaScript on this page doesn't load either (well, none of the JavaScript loads): https://simonw.github.io/datasette-lite/#/fixtures?sql=select+sqlite_version%28%29

[simonw]

A few ideas in https://stackoverflow.com/questions/13390588/script-tag-create-with-innerhtml-of-a-div-doesnt-work

I'm going to try scanning the inserted HTML for <script src> elements, extracting the src=, fetching that using a different message to the worker and executing it when it returns.

[simonw]

Potential timing bug here - what if I fire off a message asking for a script, then the user navigates to another page, then I execute the JavaScript that they asked for on that new page?

I can avoid that by attaching some kind of ID attribute to the script element and checking for it in the DOM before running eval().

[simonw]

There's another way I could approach this: the code that runs in the web worker could parse the HTML before sending it back to the client and - if it finds any script elements - could fetch that JavaScript and send it in a script: key.

The index.html page could then execute that JavaScript after inserting the .innerHTML.

This is a kind of filthy hack and I like it. Let's see if it works!

[simonw]

There's another way I could approach this: the code that runs in the web worker could parse the HTML before sending it back to the client and - if it finds any script elements - could fetch that JavaScript and send it in a script: key.

Sadly "Error: DOMParser is not defined" - DOMParser isn't available inside web workers.

[simonw]

I think I need to scan through each <script> and if it has contents eval() that, but if it has a src= attribute fetch that and then eval() it.

A couple of things that worry me:

• Some scripts may attempt to load more scripts by URL, which may break
• Scripts that set global variables may end up polluting each other

Maybe I should render these things in an iframe purely to give them a fresh JavaScript context on each page load?

[simonw]

Here's how to run an HTML parser against the content returned by the Datasette server in the web worker:

   } else if (/^text\/html/.exec(event.data.contentType)) {
+    // Check for any script tags
+    let parser = new DOMParser();
+    let dom = parser.parseFromString(event.data.text, 'text/html');
+    console.log(dom);
     html = event.data.text;

[simonw]

Tip from @Jonty: https://twitter.com/jonty/status/1521947838300762112

Did you consider running pydiode in the webworker, but keeping the serviceworker just for request intercept and passing off to the webworker? It would be a lot cleaner.

I didn't know service workers could talk to web workers - maybe this could help me solve the asset loading challenge?

[simonw]

Another idea I just had: register a service worker for /-/static/ and /-/static-plugins/, then write some code in the web worker which, on startup, loops through ALL of the static files that have been provided by plugins and sends copies of those files to the service worker along with details of their paths.

That way the service worker doesn't have to run Pyodide and doesn't need to ask any questions itself - it just gets primed with the content it will need to serve when Datasette first starts running.