Skip to content

fix(opfs): add auto OPFS file registration & update dropFiles method #1945

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from

Conversation

e1arikawa
Copy link
Contributor

Summary

  • Auto File Registration:
    When AsyncDuckDB#runQuery or startPendingQuery includes an opfs:// file path, the function automatically calls AsyncDuckDB#registerFileHandler.

    • Enabled by setting autoFileRegistration: true in the DuckDBConfig passed to AsyncDuckDB#open.
  • dropFiles Method Updates:

    • No arguments: Drops all registered files.
    • With arguments: Drops only the specified file(s).

Rationale

  • This feature simplifies the usage of OPFS by removing the need to manually register file handlers in most cases.
  • Updating the dropFiles method improves flexibility and convenience for file management.

Testing

  • Verified that autoFileRegistration: true triggers automatic file registration correctly.
  • Confirmed that dropFiles() without arguments removes all registered files, and dropFiles("someFile") removes only the specified file.

Impact

  • Users who do not enable autoFileRegistration will not see any change in behavior regarding file registration.
  • Since dropFiles originally did not accept arguments, existing calls using no arguments remain unaffected.
    However, if any code previously passed arguments (intentionally or unintentionally), the behavior might now differ because those arguments will be processed instead of ignored.
  • If your code relies on the old behavior, please review these changes and update as needed.

update opfs tests
change dropFiles args
@carlopi
Copy link
Collaborator

carlopi commented Jan 26, 2025

Problem I see with that is that's not obvious which files might/will be read for a given query / needs preparing.

Example:

COPY (SELECT 32) TO 'opfs://file.csv';
--- opfs://file.csv is written

COPY (SELECT 42) TO 'opfs://file.csv';
---  opfs://tmp_file.csv AND opfs://file.csv are written

I will write down later what seems a better solution (to me)

@e1arikawa e1arikawa force-pushed the feature/from_opfs_path branch 2 times, most recently from 095e848 to 69f4375 Compare February 19, 2025 23:27
@e1arikawa e1arikawa force-pushed the feature/from_opfs_path branch from 57ef254 to e532a2a Compare February 20, 2025 10:54
@e1arikawa e1arikawa changed the title Add auto OPFS file registration & update dropFiles method fix(opfs): add auto OPFS file registration & update dropFiles method Feb 28, 2025
@e1arikawa e1arikawa closed this Mar 3, 2025
@e1arikawa e1arikawa deleted the feature/from_opfs_path branch March 3, 2025 05:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants