Skip to content

sqlite: add tagged template #58748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

sqlite: add tagged template #58748

wants to merge 1 commit into from

Conversation

0hmX
Copy link
Contributor

@0hmX 0hmX commented Jun 18, 2025

Closes #57570

The current implementation is experimental and intended to gather early feedback on the proposed direction.

This initial approach introduces two new functions:

  • processSqlTemplate(): [the actual string interpolation logic.]
  • createSqlTag(): [a higher order function that stores the native db and returns another processSqlTemplate .]

Note: The API design is not final and is expected to evolve based on feedback.

Questions for Maintainers

  1. Is this direction a suitable foundation for solving the issue? Are there any alternative approaches I should consider?

As this is my first feature development for the project, any feedback on the code, approach, or contribution process would be greatly appreciated.

Here is my current plane for the usage

const {createSqlTag, DatabaseSync} = require("node:sqlite")

const db = new DatabaseSync(":memeory:")

const template = createSqlTag(db)

template`sql statement` // return the prepared statement

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem. labels Jun 18, 2025
@0hmX
Copy link
Contributor Author

0hmX commented Jun 18, 2025

looking forward to your answers cc @nodejs/sqlite

@0hmX 0hmX changed the title sqlite: add taged template to sqlite sqlite: add tagged template Jun 18, 2025
@RafaelGSS
Copy link
Member

cc: @geeksilva97 @miguelmarcondesf

@jasnell
Copy link
Member

jasnell commented Jun 21, 2025

Since the API requires passing in the database instance, I'd prefer something like...

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

As opposed to a standalone top-level function.

@geeksilva97
Copy link
Contributor

geeksilva97 commented Jun 21, 2025

Hello @0hmX . I wonder how this will fit the current API interface. I like @jasnell's suggestion. Do you think we could make it like the following?

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

template.run`...`;
template.get`...`
template.all`...`

EDIT: I'm not sure if my question makes sense. I asked about it in the issue to get more information.

@geeksilva97
Copy link
Contributor

geeksilva97 commented Jun 21, 2025

From a technical point of view, @0hmX . This PR needs tests and documentation.

The cache is important in the implementation since statements are meant to be reused. With template tags, we miss this. In such a situation, the cache is important, as in Matteo's implementation.

I also wonder if we could have this implementation on C++ side, as all the implementations of node:sqlite module.

@0hmX
Copy link
Contributor Author

0hmX commented Jun 29, 2025

@geeksilva97 and @jasnell, thank you for the feedback.

One of the key features that tag templates could bring is SQL syntax highlighting. However, this will only work if we have a pattern like this:

const willSyntaxHighlight = sql`SELECT * FROM users WHERE age = ${age}` // will get syntax highlighted with extensions
const willNotSyntaxHighlight = `SELECT * FROM users WHERE age = ${age}`

This is the pattern I see in all the extensions that allow you to highlight a string template inside ts or js for SQL in VS Code.

We need to have a template function called sql in front. Therefore, we should export a top-level function called SQL that returns an object with the raw SQL string and parameters. I have implemented this in C++ and added it! This means we will need to add support for this custom object in database.exec (we will convert the custom object into a raw string) and database.prepare.

const db = new DatabaseSync(":memory:");
const template = db.createSqlTag();

I think the name should be changed to something like db.createCache(). as this is creating a cache, storing the SQL string, and checking if a prepared statement is already made for it and calling the appropriate method on it and returning the outputs.

the final Api should look something like this

const { sql, DatabaseSync } = require("node:sql")

const db = new DatabaseSync(":memory:")

// adding sql template function in front for syntax highlight, else it's the same as using a multiline string
db.exec(sql`CREATE TABLE contacts (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    phone TEXT NOT NULL UNIQUE,
    email TEXT NOT NULL UNIQUE
)`)

// same as using a multiline string
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Alice"}, ${"111-222-3333"}, ${"[email protected]"})`)
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Bob"}, ${"444-555-6666"}, ${"[email protected]"})`)
db.exec(sql`INSERT INTO contacts (name, phone, email) VALUES (${"Charlie"}, ${"777-888-9999"}, ${"[email protected]"})`)

const cache = db.createCache()

const emailDomain = "%@example.com"
cache.all(sql`SELECT * FROM contacts WHERE email LIKE ${emailDomain}`)

const contactId = 2
cache.get(sql`SELECT * FROM contacts WHERE id = ${contactId}`)

const namePrefix = "C%"
cache.iterate(sql`SELECT * FROM contacts WHERE name LIKE ${namePrefix} ORDER BY name`)

@bakkot
Copy link
Contributor

bakkot commented Jun 30, 2025

One of the key features that tag templates could bring is SQL syntax highlighting. However, this will only work if we have a pattern like this: [...] We need to have a template function called sql in front.

VSCode supports more complex patterns than just literally an identifier, I'm almost certain. It can be made to handle db.prepare or anything.sql just as well as a bare sql. See this repo for an example.

// same as using a multiline string

It should very much not be the same as using a multiline string. The whole point of tagged templates - literally the only reason they are in the language - is that unlike strings they allow using a representation which is immune to problems like sql injection. From your implementation it looks like you're correctly avoiding sql injections, but this is something which should be emphasized in the docs and tested extensively. Users need to be aware that omitting the tag is not safe; it's not just a convenience for getting syntax highlighting.

On that note, you should also be doing parsing of the string up front: users should get an error as soon as they write sql`invalid`; rather than needing to actually pass it somewhere to get an error. Note that the first argument to the template tag is always the same value on subsequent calls so that it can be used as a key in a cache instead of needing to check every time the function is called. Probably the easiest thing is to prepare the statement right away, and store the prepared statement in a cache.

@0hmX 0hmX force-pushed the 57570 branch 6 times, most recently from f5396ab to b42e601 Compare July 7, 2025 18:20
@0hmX 0hmX marked this pull request as ready for review July 7, 2025 18:31
@0hmX
Copy link
Contributor Author

0hmX commented Jul 7, 2025

@bakkot, thanks for clarifying. After spending some time, I have come to understand that support for syntax highlighting and error checking of SQL strings is purely dependent on the IDE itself.


Till now (7/8/25)!

  • Implemented LRU cache as suggested in the issue. It's a standard implementation. Next need to work on testing for the LRU. Would love to know from the maintainers how you test C++ code? Especially any good advice for doing C++ testing!

  • I have also updated the names of the method to createTagStore, and it accepts a number representing the size of the store.

  • What version will I say when I start to make the docs?

what i think the final look will be and look like

'use strict';
require('../common');
const assert = require('assert');
const { DatabaseSync } = require('node:sqlite');
const { test } = require('node:test');

const db = new DatabaseSync(':memory:');
db.exec('CREATE TABLE IF NOT EXISTS foo (id INTEGER PRIMARY KEY, text TEXT)');
const sql = db.createTagStore(10); // Return an SqlTagStore Object

test('sqlite template tag', () => {
  assert.strictEqual(sql.run`INSERT INTO foo (text) VALUES (${'bob'})`.changes, 1);
  assert.strictEqual(sql.run`INSERT INTO foo (text) VALUES (${'mac'})`.changes, 1);
  assert.strictEqual(sql.run`INSERT INTO foo (text) VALUES (${'alice'})`.changes, 1);

  const first = sql.get`SELECT * FROM foo ORDER BY id ASC`;
  assert.ok(first);
  assert.strictEqual(first.text, 'bob');
  assert.strictEqual(first.id, 1);
  assert.strictEqual(Object.getPrototypeOf(first), null);

  const all = sql.all`SELECT * FROM foo ORDER BY id ASC`;
  assert.strictEqual(Array.isArray(all), true);
  assert.strictEqual(all.length, 3);
  for (const row of all) {
    assert.strictEqual(Object.getPrototypeOf(row), null);
  }
  assert.deepStrictEqual(all.map((r) => r.text), ['bob', 'mac', 'alice']);

  const iter = sql.iterate`SELECT * FROM foo ORDER BY id ASC`;
  const iterRows = [];
  for (const row of iter) {
    assert.ok(row);
    assert.strictEqual(Object.getPrototypeOf(row), null);
    iterRows.push(row.text);
  }
  assert.deepStrictEqual(iterRows, ['bob', 'mac', 'alice']);

  const none = sql.get`SELECT * FROM foo WHERE text = ${'notfound'}`;
  assert.strictEqual(none, undefined);

  const empty = sql.all`SELECT * FROM foo WHERE text = ${'notfound'}`;
  assert.deepStrictEqual(empty, []);

  let count = 0;
  // eslint-disable-next-line no-unused-vars
  for (const _ of sql.iterate`SELECT * FROM foo WHERE text = ${'notfound'}`) {
    count++;
  }
  assert.strictEqual(count, 0);
});

@0hmX 0hmX marked this pull request as draft July 8, 2025 14:21
Adding tagged template and LRU cache for prepared
statements in SQLite.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-ci PRs that need a full CI run. sqlite Issues and PRs related to the SQLite subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add template tags support to node:sqlite
6 participants