feat: parallel execution of DuckDB Queries using iFrame (#88)

* WIP Parallel exec

* Communication Ref

* IG manager

* WIP

* WIP: file register

* WIP: parallel memory manager

* WIP

* Get messages working

* Little more fix

* It works idk how

* fix: duplicate query execution

* Late night work

* some: cleanup

* Added comment

* update: cache logic

* Feat/parallel exec nik (#83)

* Dynamic list

* Clear memory and db

* Type fix

* Clean up and add parsing for JSON

* Fix E2E

* Remove unwanted file

* Remove hardcoding of origin (#85)

* Remove hardcoding

* Remove unwanted stuff

* Remove log

* test(parallel-dbm): add tests for parallel dbm (#84)

* add: tests

* update: path

* update: mock

* SharedArrayBuffer via Single domain (#86)

* Shared Array Paralle ecec

* Remove migrations.json

* Add logs for debugging

* fix(parallel-dbm): fix target and app id (#87)

* Shared Array Paralle ecec

* Remove migrations.json

* Add logs for debugging

* fix: app id

* update: index html

---------

Co-authored-by: itsTalwar <[email protected]>

* feat(parallel-dbm): convert json data to shared array (#90)

* handle: json shared array

* update: method

* feat: Clean types and integrate events with runners (#91)

* Adding more comments and events

* remove as much as possible

* Fix types

* Reduce the size of iframe and remove unwanted file (#92)

* handle: file loader buffer type (#94)

* chore(dbm): cleanup parallel dbm  (#95)

* some cleanup

* remove: unwanted change

* fix: types

* fix: type

* fix: test env

* fix: test fix attempt 1

* update: test

* update: package

* fix: build

* fix: lint

* Use node 18

* Do npm ci instead of install

* Fix types

* fix: address comments

* fix: types

* update: ref

* update: gc

* address comments

---------

Co-authored-by: balasaravanan <[email protected]>

Author: itsTalwar

.github/workflows/build-test.yml

@@ -12,7 +12,11 @@ jobs:
         - uses: actions/checkout@v2
           with:
             lfs: true
-        - run: npm install
+        - name: Use Node.js
+          uses: actions/setup-node@v2
+          with:
+            node-version: '18'
+        - run: npm ci
         - run: npx nx run-many --target=build --all --parallel
         - run: npx nx run-many --target=test --all --parallel
         - run: npx nx run-many --target=lint --all --parallel

.vscode/settings.json

@@ -1,32 +1,36 @@
 {
-    "[javascript]": {
-      "editor.formatOnSave": true,
-      "editor.defaultFormatter": "esbenp.prettier-vscode"
-    },
-    "[javascriptreact]": {
-      "editor.formatOnSave": true,
-      "editor.defaultFormatter": "esbenp.prettier-vscode"
-    },
-    "[typescript]": {
-      "editor.formatOnSave": true,
-      "editor.defaultFormatter": "esbenp.prettier-vscode"
-    },
-    "[typescriptreact]": {
-      "editor.formatOnSave": true,
-      "editor.defaultFormatter": "esbenp.prettier-vscode"
-    },
-    "[mdx]": {
-      "editor.wordWrap": "bounded",
-      "editor.wordWrapColumn": 120
-    },
-    "[markdown]": {
-      "editor.wordWrap": "bounded",
-      "editor.wordWrapColumn": 120
-    },
-    "editor.codeActionsOnSave": {
-      "source.fixAll.eslint": true,
-      "source.organizeImports": true
-    },
-    "eslint.validate": ["javascript", "javascriptreact", "typescript", "typescriptreact"],
-  }
-  
+  "[javascript]": {
+    "editor.formatOnSave": true,
+    "editor.defaultFormatter": "esbenp.prettier-vscode"
+  },
+  "[javascriptreact]": {
+    "editor.formatOnSave": true,
+    "editor.defaultFormatter": "esbenp.prettier-vscode"
+  },
+  "[typescript]": {
+    "editor.formatOnSave": true,
+    "editor.defaultFormatter": "esbenp.prettier-vscode"
+  },
+  "[typescriptreact]": {
+    "editor.formatOnSave": true,
+    "editor.defaultFormatter": "esbenp.prettier-vscode"
+  },
+  "[mdx]": {
+    "editor.wordWrap": "bounded",
+    "editor.wordWrapColumn": 120
+  },
+  "[markdown]": {
+    "editor.wordWrap": "bounded",
+    "editor.wordWrapColumn": 120
+  },
+  "editor.codeActionsOnSave": {
+    "source.fixAll.eslint": "explicit",
+    "source.organizeImports": "explicit"
+  },
+  "eslint.validate": [
+    "javascript",
+    "javascriptreact",
+    "typescript",
+    "typescriptreact"
+  ]
+}

benchmarking/benchmarking-app/.eslintrc.json renamed to benchmarking/.eslintrc.json

@@ -1,5 +1,5 @@
 {
-  "extends": ["plugin:@nx/react", "../../.eslintrc.json"],
+  "extends": ["plugin:@nx/react", "../.eslintrc.json"],
   "ignorePatterns": ["!**/*"],
   "overrides": [
     {

benchmarking/benchmarking-app/.babelrc

@@ -0,0 +1,16 @@
+<!DOCTYPE html>
+<html lang="en">
+  <head>
+    <meta charset="utf-8" />
+    <title>BenchmarkingApp</title>
+    <base href="/" />
+
+    <meta name="viewport" content="width=device-width, initial-scale=1" />
+    <link rel="icon" type="image/x-icon" href="/favicon.ico" />
+    <link rel="stylesheet" href="/src/styles.css" />
+  </head>
+  <body>
+    <div id="root"></div>
+    <script type="module" src="/src/main.tsx"></script>
+  </body>
+</html>