Web Feature Parity

[nadr0]

1. Background

We share the same codebase for deskop and web but there are a few locations that there is no feature parity between web and desktop. To improve the experience for web users we should implement more feature parity between desktop and web.

2. Goals

2.1 Home page support

We should be able to route to /home

• How can we use a settings application project directory path within indexedDB?
• projects are folders under the application project directory path
• use the same Project and FileEntry interface within @src/lib/project when reading from the indexedDB
• use SystemIOMachineWeb.ts to generate this data since the application uses this to find the projects.

2.2 Choose a web backend for local file storage

Localstorage and indexeddb at the two possible choices. Localstorage stores less and is sync which could lead to performance problems if we are writing a large file in the browser. Indexddb allows for more storage and async operations which the async read/write could lead to better performance when using the application.

I am leaning towards a TS supported indexdeddb library for the implementation of a web based file system. This could mimic more than the project/files structure. It could even store a settings.toml file even though we have that in localstorage.

Technical considerations for window.localStorage and IndexedDB API

window.localStorage
Benefits

• If storing kcl files as strings, a simpler API

Limitations

• Local storage has less data storage (most likely not a problem since we are dealing with txt files)
• Can only store data has strings
• sync read/writing
• Can be cleared from browser settings
• non async blocking API
• overhead for JSON stringify and parse
• cannot be used in web workers
• versioning would be encoded in deeply nested object

IndexedDB API
Benefits

• IndexedDB has significant amounts of structured data, including files/blobs
• Allows for larger storage
• can be used in web workers
• async read/write patterns (we write to files a lot and quickly)
• versioning of data will be out of the box
• can store binary data

Limitations

• lower level API (will most likely require a library in TS to make usage easier)
• Can be cleared from browser settings

2.3 Command bar project actions

We should be able to support multiple project command actions within the command bar.

2.4 Project files pane

We should be able to support the projects file pane

2.5 Text-To-CAD modify/create multiple files

Text-To-CAD operations should work with the multiple files.

3. Providers

3.1 Implement a SystemIOWebMachine provider

The SystemIOWebMachine can implement the web based way of accessing files and interacting with the project data. This could point to a indexeddb TS library. Many of the project actions are pointing to the SystemIOMachine, if one is hooked up to the web it should be able to be populated.

4. Open Questions

4.1 Do we mimic the entire FS system from desktop into the browser with our indexeddb solution?

settings.toml exists in the localstorage. Would it make sense to make that a file within the indexeddb and read/write it like we do the desktop or use localstorage?

• Would this help the mental model and code paths?
• Is it overkill to move some configuration from localstorage for settings to a file path structure to make it look the same when there isn't much of any benefit except the mental model pattern?

e.g. Oh you need to write that value to settings.toml! If you are on desktop you would know you have a file on disk. If you are on web it is actually an object in localstorage. Would it be better to know that it is a file in web? How should we think about these abstractions.

5. Security Considerations

6. Gotchas

6.1 Any parts of the system that are hard coded for desktop workflows

For example, clicking a project card has code that only runs on desktop because the router path is encoded for a desktop URL not a browser URL. There maybe more of these scenarios to update.

6.2 Unit testing / Playwright tests

They are probably directly interfacing with localStorage.getItem('persistCode') or setItem.

6.3 Other desktop files are in localStorage, do those need to be migrated?

settings.toml or a project.toml exist on disk on the desktop application. Those are implemented through localStorage in the web. In the future those could be "files" through indexeddb but that workflow could stay the same for now.

The proposal for indexeddb would be for project folders and files for the initial bodies of work. localStorage can make sense for certain pieces of data and I don't want to do busy work to migrate everything to the same pattern initially if there isn't much benefit.

Moving kcl files from localStorage to indexeddb has more pros. Moving a settings.toml to indexeddb` doesn't seem to have much of a benefit?

6.4 How would we implement a project.toml in web?

I'll have to see how this is implemented. If this is similar to a singular settings.toml then project.toml may get lumped into the indexeddb body of work so every project can have a project settings file.

6.5 How would this interact with temporary workspaces?

Temporary workspaces were made to someone could share a link and not overwrite their previous code to take a look at the link. We could allow share urls to make new projects so we do not need temporary workspaces anymore?

7. Diagrams

External References

1. https://developer.mozilla.org/en-US/docs/Web/API/Window/localStorage
2. https://developer.mozilla.org/en-US/docs/Web/API/IndexedDB_API
3. https://rxdb.info/articles/localstorage.html
4. https://rxdb.info/articles/localstorage-indexeddb-cookies-opfs-sqlite-wasm.html

[Irev-Dev]

IndexDB seems to be the better choice, just on the storage size alone.
Not something we need to worry about in the first pass, but we probably want to give users a way to migrate from the web-app to the desktop app, be that export, something more cloud like we build out with the infra team.

HOWEVER, on the export topic and index db, why don't we just use the user's file system directly for the web app too?
Seems like a silly question because this is the whole problem, but if we can do it, than the parity problem gets even smaller.
You can give the browser permission to read&write a folder using the File System Access API. This would still be a PITA for our users if they had to give permission to the right folder everytime they opened the web app, but you CAN persist the permision

https://developer.chrome.com/blog/persistent-permissions-for-the-file-system-access-api
But this persisted permision is chrome (chrome-y, MSedge etc) specific.

Anyway just wanted to make sure we consided this path as well.

How would this interact with temporary workspaces?

The move here might be to remove temporary workspaces. I thought a large part of the motivation of the temporary workspace was interwoven with the limitations of the web app, that there one only one file and one project. We can make the links for opening a sample, just create a new project, no temprary workspace needed?

[franknoirot]

• Local storage has less data storage (most likely not a problem since we are dealing with txt files)

I also agree with the IndexDB or File System API approaches, because they lead to the possibility of allowing other CAD file types more easily (like imported STEP files). So I don't think the desktop app as it exists today is dealing with txt files only.

[franknoirot]

They are probably directly interfacing with localStorage.getItem('persistCode') or setItem.

Indeed they are. I might have test Stockholm syndrome, but I was relieved to see [it's only used in 14 test files](https://github.com/search?q=repo:KittyCAD/modeling-app%20setItem('persistCode'&type=code)

[pierremtb]

Great write up @nadr0, thanks! Would love to talk more about these building blocks and how they could enable something like #7607, as I still have trouble seeing the value provided by the web app working only in a specific browser's local storage (or indexdb), beyond just a demo?

[pierremtb]

To be fair there is one example I can think of for a web app working directly in local storage: https://excalidraw.com/

IMO this is still pretty orthogonal to how most of the web apps work though, with data following your account wherever you sign in