Update to 0.15.0; use ES Module Shims #275

JordanMartinez · 2022-05-19T03:13:52Z

Description of the change

Fixes #274 and fixes #263. Another take that succeeds #274.

Original opening thread

In #274, I experienced a few issues that made me rethink how I added initial support for 0.15.0. I can explain in a future comment because it's late right now.

Regardless, this PR is currently stuck due to this error:

Uncaught (in promise) DOMException: Permission denied to access property "_$s" on cross-origin object srcdoc:1

Clicking on srcdoc:1 leads to this message:

Error loading this URI: Unknown source

I'm baffled on how to proceed. I've been using Firefox to test this out, so it may be related to https://bugzilla.mozilla.org/show_bug.cgi?id=1721891.

When I tried it on Chromium, the error I get is this:

Uncaught (in promise) DOMException: Blocked a frame with origin "null" from accessing a cross-origin frame.
at about:srcdoc:1:163

Clicking on srcdoc:1:163 displays No resource with given URL found.

Changes made:

We're using [email protected] instead of 1.5.1 doesn't load the modules correctly.
Since the output of the compiler is ../Module.Path/index.js, we have to remap the import paths to Config.url <> "/Module.Path/index.js"
We use window.runCount to ensure main is only called once. Otherwise, it will be called an infinite number of times

Checklist:

Added the change to the changelog's "Unreleased" section with a reference to this PR (e.g. "- Made a change (#0 by @)")
Linked any existing issues or proposals that this pull request should close
Updated or added relevant documentation
Added a test for the contribution (if applicable)

JordanMartinez · 2022-05-19T05:00:44Z

Turns out the above error was a false alarm. We use index.html to create an iframe using frame.html as the source. es-module-shims appears to create an iframe within the head of frame.html. That's the source of the above error. I'm not sure why it does that, but it distracted me.

The actual reason why the code wasn't working is because there wasn't a wrapper calling main(); in the script's text. Adding that makes the example appear.

JordanMartinez · 2022-05-19T05:18:19Z

And I think the above error was fixed in a more recent version of es-module-shims. I was using 1.5.1, but it no longer appears in 1.5.5.

This reverts commit 37cdfc8.

mikesol · 2022-05-19T09:27:35Z

Going through this now. I was able to get files served by running npx http-server --cors (we should probably update the readme, as it currently recommends npx http-server).

After that, I'm getting:

VM60 about:srcdoc:1 Uncaught (in promise) DOMException: Blocked a frame with origin "null" from accessing a cross-origin frame.
    at about:srcdoc:1:163

Is that where you're at as well?

JordanMartinez · 2022-05-19T13:06:07Z

Going through this now. I was able to get files served by running npx http-server --cors (we should probably update the readme, as it currently recommends npx http-server).

After that, I'm getting:
VM60 about:srcdoc:1 Uncaught (in promise) DOMException: Blocked a frame with origin "null" from accessing a cross-origin frame.
    at about:srcdoc:1:163
Is that where you're at as well?

See the package.json file. If you run npm i && npm run serve:dev, it'll run the code properly. The DOMException is what I get, too, but it's something to ignore. On my side, the code does get run if I'm not using the shims.

AFAICT, the shims don't currently work because they aren't imported in the code we're using. Put differently, this should work:

import React from 'react';
..

but this doesn't work

// main import script
import * as React from "../React/index.js";
...

where React/index.js is:

import React from 'react';
...

I think the solution there are two possible solutions:

add another import map for all packages in the package set to frame.html
use esbuild to bundle the outputted JS and return it in the /compile API endpoint

For now, I'm going with the esbuild option since that seems to be what #264 is for.

mikesol · 2022-05-19T17:11:31Z

Would it be the case, then, that esbuild bundles and sends the whole application with each call to compile?

natefaubion · 2022-05-19T17:15:39Z

AFAICT, the shims don't currently work because they aren't imported in the code we're using. Put differently, this should work:

What convinced you that this is the case? It's not clear to me whether the imports are transitive should matter. Is this documented?

JordanMartinez · 2022-05-19T17:17:37Z

Code now works, but the main function gets called infinitely.

natefaubion · 2022-05-19T17:17:51Z

Doing a full bundle on every compile is extremely expensive, both on the server and for the user to pull down. I personally think it's a non-starter.

natefaubion · 2022-05-19T17:19:25Z

I'm pretty sure #264 is for only bundling the UI, not in general for all code the user inputs.

natefaubion · 2022-05-19T17:22:17Z

add another import map for all packages in the package set to frame.html

I don't think you need to add an import map for all packages in the package set? You only need it for specific packages that import specific foreign modules. This seems fine to me, and more preferable than using esbuild.

JordanMartinez · 2022-05-19T18:24:08Z

AFAICT, the shims don't currently work because they aren't imported in the code we're using. Put differently, this should work:

What convinced you that this is the case? It's not clear to me whether the imports are transitive should matter. Is this documented?

Let's say I want to run the following code that uses the decimal.js shim:

module Main where

import Prelude

import Data.Foldable (fold)
import Effect (Effect)
import Effect.Console (log)
import TryPureScript as TP
import Data.Decimal as D

main :: Effect Unit
main = TP.render =<< TP.withConsole do
  log $ show $ D.fromString "2424.24245238523752384034"

The unbundled JS returned is:

import * as Control_Bind from "../Control.Bind/index.js";
import * as Data_Decimal from "../Data.Decimal/index.js";
import * as Data_Maybe from "../Data.Maybe/index.js";
import * as Data_Show from "../Data.Show/index.js";
import * as Effect from "../Effect/index.js";
import * as Effect_Console from "../Effect.Console/index.js";
import * as TryPureScript from "../TryPureScript/index.js";
var main = /* #__PURE__ */ Control_Bind.bindFlipped(Effect.bindEffect)(TryPureScript.render)(/* #__PURE__ */ TryPureScript.withConsole(/* #__PURE__ */ Effect_Console.log(/* #__PURE__ */ Data_Show.show(/* #__PURE__ */ Data_Maybe.showMaybe(Data_Decimal.showDecimal))(/* #__PURE__ */ Data_Decimal.fromString("2424.24245238523752384034")))));
export {
    main
};

// I add this part so the code actually runs.
main();

If I remap those imports from ../Module.Path/index.js to ./js/output/Module.Path/index.js, and then update frame.html to the following:

<!DOCTYPE HTML>
<html>
<head>
  <title>Try PureScript!</title>
  <meta content="text/html;charset=utf-8" http-equiv="Content-Type">
  <meta content="utf-8" http-equiv="encoding">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <script>
    window.esmsInitOptions = {
      // -- Hooks --
      // Module load error
      onerror: (e) => {
        console.log("Error while loading module: ");
        console.log(e);
        throw e;
      },
    };
  </script>
  <!--
    JSPM Generator Import Map
    Edit URL: https://generator.jspm.io/#Y2NnYGCzD80rySzJSU1hSMpM183MK0lNTy1yMNQz0zM1ZEhJTc7MTczRyyp2MDTQM9YzZChKTUwu0U3Jz3UwNNMzxCqiX5xaVJZaBJGAKystzUxxsACaYQQAoBlP83cA
  -->
  <script type="importmap">
  {
    "imports": {
      "big-integer": "https://ga.jspm.io/npm:[email protected]/BigInteger.js",
      "decimal.js": "https://ga.jspm.io/npm:[email protected]/decimal.js",
      "react": "https://ga.jspm.io/npm:[email protected]/index.js",
      "react-dom": "https://ga.jspm.io/npm:[email protected]/index.js",
      "react-dom/server": "https://ga.jspm.io/npm:[email protected]/server.browser.js",
      "uuid": "https://ga.jspm.io/npm:[email protected]/dist/esm-browser/index.js"
    },
    "scopes": {
      "https://ga.jspm.io/": {
        "object-assign": "https://ga.jspm.io/npm:[email protected]/index.js",
        "react": "https://ga.jspm.io/npm:[email protected]/index.js",
        "scheduler": "https://ga.jspm.io/npm:[email protected]/index.js"
      }
    }
  }
  </script>

  <!-- ES Module Shims: Import maps polyfill for modules browsers without import maps support (all except Chrome 89+) -->
  <script async src="https://ga.jspm.io/npm:[email protected]/dist/es-module-shims.js" integrity="sha384-ZGtUNdKMtA2sSrO9dN9d2TOeiSlXlDSrOrasgO9YoKR4LsDV7RLdUvvX1M0gCkl2" crossorigin="anonymous"></script>

  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/BigInteger.js" integrity="sha384-Lxn1K4ox8//F5O+uihYrTIJvbfM+UJaRL2SdFA2s/z0v6i5TrWg9B8RzfjM60R9y"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/decimal.js" integrity="sha384-camBDJB1KoavtyRdQfdzzR4mj+uAKnNlvseJIwqKYOaHWnjZmnXR+1vyBP1WlZd0"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/index.js" integrity="sha384-iQp1zoaqIhfUYyYkz3UNk1QeFfmBGgt1Ojq0kZD5Prql1g7fgJVzVgsjDoR65lv8"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/cjs/react-dom-server.browser.production.min.js" integrity="sha384-XL5bjsaJ5UPjTIPsyzUfscq41OFbVXsLojl53DCgFfdzKNNUL2MOSd0WM/upjd53"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/cjs/react-dom.production.min.js" integrity="sha384-HDQjgsF+F2j5XuiiDtCiuLA1vgyHr9ONiNrBhKmJEVa43KqIZdAB7VN2+QjUGwvq"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/index.js" integrity="sha384-zAu0L7n/xnRAG7+D/qd1K4E1/2tYK4hL3puH/5YEliHNnO3OOt5jUhnnemFTHmIG"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/server.browser.js" integrity="sha384-dsW+4a3rjWELxqtR1rVEIeLNHGyMjRFwdBVz6OG30oBLs8tO2/dYUIxEvc515PSZ"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/index.js" integrity="sha384-T0mNrKFgmKBye+m7XrpShrh+7kfix2zQd9qrR8xytnppcxESpgzrb4IJXvE+uJcl"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/cjs/react.production.min.js" integrity="sha384-pTMZhybzHZ+1G029kWUmoGvTrBp1C+2oJAkZV48BBq7+e6Hk3bGuXtvxT2vQfBqj"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/index.js" integrity="sha384-jVagjV+2YtlseazU2byX6gMLPHaA5Ps2c6HhvsGDlWjn45YCCoU1q+QtQTOb1MjT"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/index.js" integrity="sha384-HIW1B3OQdGPkUEgh29MiYoUKQp+mFTjw50hRVvGhZgUaNYrUFYN07a+3CqC0/I7L"/>
  <link rel="modulepreload" href="https://ga.jspm.io/npm:[email protected]/dist/esm-browser/index.js" integrity="sha384-y7UmGnwe+/V/S4AD5T6MLU9dHjA3PLvuVGNhrqZApiNbq9Lf8s3aNRrcnzOa+qtR"/>
-  <script src="js/frame.js"></script>
</head>
<body>
  <main id="main"></main>
+  <script>
+    import * as Control_Bind from "./js/outpControl.Bind/index.js";
+    import * as Data_Decimal from "./js/outpData.Decimal/index.js";
+    import * as Data_Maybe from "./js/outpData.Maybe/index.js";
+    import * as Data_Show from "./js/outpData.Show/index.js";
+    import * as Effect from "./js/outpEffect/index.js";
+    import * as Effect_Console from "./js/outpEffect.Console/index.js";
+    import * as TryPureScript from "./js/outpTryPureScript/index.js";
+    var main = /* #__PURE__ */ Control_Bind.bindFlipped(Effect.bindEffect)(TryPureScript.render)(/* #__PURE__ */ TryPureScript.withConsole(/* #__PURE__ */ Effect_Console.log(/* #__PURE__ */ Data_Show.show(/* #__PURE__ */ Data_Maybe.showMaybe(Data_Decimal.showDecimal))(/* #__PURE__ */ Data_Decimal.fromString("2424.24245238523752384034")))));
+    export {
+        main
+    };
+    main();
  </script>
</body>
</html>

I get the following error in the console:

Uncaught SyntaxError: import declarations may only appear at top level of a module

and the position of the error is the first import:

import * as Control_Bind from "./js/outpControl.Bind/index.js";

You only need it for specific packages that import specific foreign modules.

If only a few need to be in the import map, that would be preferable than creating an import map for the entire package set.

natefaubion · 2022-05-19T18:28:33Z

For modules you need <script type="module">. I wonder if we can avoid rewriting paths by using a <base> tag.

JordanMartinez · 2022-05-19T19:22:58Z

For modules you need <script type="module">.

🤦 Wow... That change fixed it. And the import maps didn't need to be updated.

EDIT: actually, setting the script's type to module was what I was already doing in the frame.js code. Hm...

This reverts commit 76a5345.

JordanMartinez · 2022-05-19T23:36:27Z

This is ready for a final review. Code builds and ES modules work.

natefaubion · 2022-05-19T23:39:57Z

Not quite. Since this is used in both dev (./js/output/ignored) and production settings , hard-coding it for dev breaks it for production.

Sorry can you clarify? dev and production use different path schemes? Is that necessary? If so, is there a way to configure it at build time, or app launch time (not necessarily client-side)? If there is a reasonable way to avoid a search and replace, that would be nice.

JordanMartinez · 2022-05-20T12:43:11Z

Sorry can you clarify? dev and production use different path schemes? Is that necessary? If so, is there a way to configure it at build time, or app launch time (not necessarily client-side)? If there is a reasonable way to avoid a search and replace, that would be nice.

We sort of already configure it at build time via the Config.purs file:

If we wanted to do it without a search and replace approach and use the base element, then we'd need to overwrite that value. Otherwise, it would require duplicating the frame.html file.

I've pushed a commit that uses an NPM script to overwrite it to the correct values.

client/public/js/frame.js

client/public/frame.html

natefaubion · 2022-05-22T15:52:07Z

client/setBase.mjs

+  .map((line) => line.replace(/^( *<base href=")[^"]*(".+)$/, `$1${baseHref}$2`))
+  .join("\n");
+
+fs.writeFileSync(filePath, newHtml);


If we have to rewrite the file in place, I'm not sure this is the best way to do this. Will this dirty the source tree?

I'm also curious how this works with the actual prod deployment of the UI. I'm not fully aware of all that goes into that.

Yeah, it would dirty the source tree.

The client is built in the ci.yml file by using NPM scripts before packaging up the results. When we deploy the code, the server downloads the client.tar.gz, unpacks it, and then runs it.

How should this issue be resolved? Do we create 2 versions of the frame.html file, one with base set to one and another with base set to something else?

I added a check in CI to fail the build if the base element does not refer to the production path. One can use npm run base:dev to test out their code in development and then run npm run base:production before committing the final results. This ensures the following:

local dev still works without much friction in workflow

the source tree is clean when the production build is made

frame.html isn't duplicated, so there's only one place to update the import maps

I believe that resolves your final issue with this PR.

client/public/js/frame.js

mikesol

Amazing work! One tiny comment, otherwise 🚀

server/Main.hs

JordanMartinez · 2022-06-04T20:05:52Z

Can this PR get an approval? I believe I've addressed all feedback.

MonoidMusician

I was able to build and run locally, and I verified that the linked examples plus a Halogen demo of my own still work.

JordanMartinez added 13 commits March 31, 2022 11:25

Update changelog for last release

00ed3da

Update purescript deps for 0.15.0; drop cst dep

098fb42

Upgrade to latest pkg set; install entire set

905d54d

Get server code to compile

3880074

Migrated FFI to ES modules via 'lebab'

37cf79a

Removed '"use strict";' in FFI files

db55540

Remove redundant import

479619b

Update package set; install entire set

8474470

Update PureScript to 0.15.2

91f18d1

Use ES Modules Shims for iframe shims

84f9908

Add text to call the main function

f436199

Update es-module-shims to 1.5.5 (latest)

37cdfc8

Move import path update into client

8fa99e8

Revert "Update es-module-shims to 1.5.5 (latest)"

b15c61b

This reverts commit 37cdfc8.

Update readme

7366787

Bundle code via esbuild after remapping imports

76a5345

Revert "Bundle code via esbuild after remapping imports"

29c602b

This reverts commit 76a5345.

JordanMartinez added 4 commits May 19, 2022 18:17

Add serve:production script

e4b49d2

Add instructions for adding shims to import map

40f0d95

Doc how to calc integrity hash

f671cbc

Fix rendering of bullet list

6b81543

JordanMartinez marked this pull request as ready for review May 19, 2022 23:35

JordanMartinez mentioned this pull request May 19, 2022

Update to 0.15.0 PureScript #274

Closed

4 tasks

Remap import paths via base elem set at build time

27eb703

natefaubion reviewed May 22, 2022

View reviewed changes

client/public/js/frame.js Outdated Show resolved Hide resolved

natefaubion reviewed May 22, 2022

View reviewed changes

client/public/js/frame.js Outdated Show resolved Hide resolved

natefaubion reviewed May 22, 2022

View reviewed changes

client/public/frame.html Outdated Show resolved Hide resolved

natefaubion reviewed May 22, 2022

View reviewed changes

client/public/js/frame.js Outdated Show resolved Hide resolved

JordanMartinez added 2 commits May 23, 2022 08:03

Drop preloaders of shims

f9c30d6

Cleanup code appending script to body

c059482

mikesol approved these changes May 24, 2022

View reviewed changes

server/Main.hs Show resolved Hide resolved

JordanMartinez added 4 commits May 25, 2022 20:14

Stop infinite main invocation when DOM modified

a993bc6

Update package set to latest; install all packages

9a97914

Drop workaround; modify main el instead

0fb4edc

Update base to production; fail CI if not that

0584223

JordanMartinez added 2 commits June 8, 2022 12:16

Update readme to mention base script

0369198

Update to latest package set; install entire set

1cc3451

MonoidMusician approved these changes Jun 8, 2022

View reviewed changes

JordanMartinez merged commit 0758325 into purescript:master Jun 8, 2022

JordanMartinez deleted the up-to-15-with-esm branch June 8, 2022 18:58

JordanMartinez mentioned this pull request Jun 8, 2022

Refer to ESM modules, not CJS ones #277

Merged

4 tasks

thomashoneyman mentioned this pull request Jul 15, 2022

Refresh with Halogen, Tailwind, Webpack #192

Closed

Update to 0.15.0; use ES Module Shims #275

Update to 0.15.0; use ES Module Shims #275

Uh oh!

Conversation

JordanMartinez commented May 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

mikesol commented May 19, 2022

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

mikesol commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022

Uh oh!

JordanMartinez commented May 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JordanMartinez commented May 19, 2022

Uh oh!

natefaubion commented May 19, 2022

Uh oh!

JordanMartinez commented May 20, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

natefaubion May 22, 2022

Choose a reason for hiding this comment

Uh oh!

natefaubion May 22, 2022

Choose a reason for hiding this comment

Uh oh!

JordanMartinez May 23, 2022

Choose a reason for hiding this comment

Uh oh!

JordanMartinez May 26, 2022

Choose a reason for hiding this comment

Uh oh!

JordanMartinez May 28, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mikesol left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JordanMartinez commented Jun 4, 2022

Uh oh!

MonoidMusician left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

JordanMartinez commented May 19, 2022 •

edited

Loading

natefaubion commented May 19, 2022 •

edited

Loading

natefaubion commented May 19, 2022 •

edited

Loading

JordanMartinez commented May 19, 2022 •

edited

Loading