[BUG]: cloudflareDevProxyVitePlugin: Issue in DEV with Remix/Vite/Drizzle/Cloudflare Pages/Supabase/Postgres

[l4j3b]

Reproduction

Hi,

I am experiencing with my setup in dev mode (it works if I build and serve my project) with my Remix app.

I am using Remix/Vite/Drizzle/Cloudflare Pages/Supabase.

The issue seems to be cause by the await db.query.users.findMany() in my loader:

export async function loader({ context }: LoaderFunctionArgs) {
  const client = postgres(context.env.DATABASE_URL);
  const db = drizzle(client, { schema: { users } });

  const allUsers = await db.query.users.findMany();

  return json({ allUsers });
}

My vite config:

import { vitePlugin as remix, cloudflareDevProxyVitePlugin as remixCloudflareDevProxy } from "@remix-run/dev";
import { getLoadContext } from "./load-context";
import { defineConfig } from "vite";
import tsconfigPaths from "vite-tsconfig-paths";

export default defineConfig({
  plugins: [remixCloudflareDevProxy({ getLoadContext }), remix(), tsconfigPaths()],
});

The error I am getting:

Error: Only URLs with a scheme in: file, data are supported by the default ESM loader. Received protocol 'cloudflare:'
    at new NodeError (node:internal/errors:393:5)
    at throwIfUnsupportedURLScheme (node:internal/modules/esm/resolve:1026:11)
    at defaultResolve (node:internal/modules/esm/resolve:1106:3)
    at nextResolve (node:internal/modules/esm/loader:163:28)
    at ESMLoader.resolve (node:internal/modules/esm/loader:841:30)
    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:424:18)
    at ESMLoader.import (node:internal/modules/esm/loader:525:22)
    at importModuleDynamically (node:internal/modules/esm/translators:110:35)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:35:14)
    at EventEmitter.connect (file:///Users/jeb/test-cf/node_modules/postgres/cf/polyfills.js:150:27)
    at Timeout.connect [as _onTimeout] (file:///Users/jeb/test-cf/node_modules/postgres/cf/src/connection.js:347:12) {
  code: 'ERR_UNSUPPORTED_ESM_URL_SCHEME'

My scripts:

"scripts": {
    "build": "remix vite:build",
    "deploy": "wrangler pages deploy ./build/client",
    "dev": "remix vite:dev",
    "lint": "eslint --ignore-path .gitignore --cache --cache-location ./node_modules/.cache/eslint .",
    "start": "wrangler pages dev ./build/client",
    "typecheck": "tsc",
    "typegen": "wrangler types",
  },

As stated above, if I do npm run build && npm run start, it works. The issue happens with npm run dev.

I have tried quite a few things:

• different NodeJS version: 19, 20, 21.
• tried with neon (it works).
• tried with a local postgres db/not supabase (same error).

I think it may be an issue with remix/vite/postgres? I spent quite a bit of time looking into it but have not been able to able to debug it. I have limited time and was wondering if someone else ran into that issue.

I tried with drizzle-orm/node-postgres. It works locally but not in prod as pg is using Node.

✘ [ERROR] Build failed with 12 errors:

  ../node_modules/pg-connection-string/index.js:76:77: ERROR: Could not resolve "fs"
  ../node_modules/pg/lib/connection-parameters.js:3:18: ERROR: Could not resolve "dns"
  ../node_modules/pg/lib/connection.js:3:18: ERROR: Could not resolve "net"
  ../node_modules/pg/lib/crypto/utils-legacy.js:5:27: ERROR: Could not resolve "crypto"
  ../node_modules/pg/lib/crypto/utils-webcrypto.js:1:27: ERROR: Could not resolve "crypto"
  ...

So I guess I can use pg in "dev" and "postgres" for the build for now but I would prefer not to.

System Info

System:
    OS: macOS 14.4.1
    CPU: (10) arm64 Apple M1 Pro
    Memory: 72.64 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 21.7.3 - ~/.nvm/versions/node/v21.7.3/bin/node
    npm: 10.5.0 - ~/.nvm/versions/node/v21.7.3/bin/npm
    pnpm: 7.17.0 - ~/Library/pnpm/pnpm
  Browsers:
    Brave Browser: 123.1.64.122
    Chrome: 123.0.6312.124
    Chrome Canary: 119.0.6045.7
    Edge: 123.0.2420.97
    Safari: 17.4.1
  npmPackages:
    @remix-run/cloudflare: ^2.8.0 => 2.8.1 
    @remix-run/cloudflare-pages: ^2.8.0 => 2.8.1 
    @remix-run/dev: ^2.8.0 => 2.8.1 
    @remix-run/react: ^2.8.0 => 2.8.1 
    vite: ^5.1.0 => 5.2.8

Used Package Manager

npm

Expected Behavior

It should work locally.

Actual Behavior

It is not working locally and throwing an error.

[melvin2016]

Have this same issue with ‘postgres’

[oristian]

yeah at present my dev workflow is restarting npm run preview ad nauseum. It doesn't not make for a good DX - if anyone has done the local cloudflare build with the vite watch/hmr, please share here. following this issue to learn more ... cloudflare/workers-sdk#5315

remix/vite/drizzle/postgres/cloudflare-pages

[bmaciass]

Ran into the same bug and got it solved by replacing drizzle-orm/postgres-js for drizzle-orm/node-postgres. So far, only tested on my dev env, hope it also works in the edge.

[calumpeak]

Ran into the same bug and got it solved by replacing drizzle-orm/postgres-js for drizzle-orm/node-postgres. So far, only tested on my dev env, hope it also works in the edge.

Are you able to provide an example?

[0xNegative]

I've found the root cause of the issue and a short-term workaround. I'm not certain where the fix should be applied, so I'll leave some context and allow the subject matter experts to figure out the rest. 🙂

The issue arises from how postgres.js applies the Cloudflare polyfills using package export conditionals. The runtime is being detected as a "worker," which is suitable for deployment to CF but problematic when developing locally with Remix since we're not actually running within the workerd runtime unless you use workers pages dev.

The workaround I used was to alias the import in development mode to use the default entry point instead of the polyfilled one. Might not be the best, but works for now.

// vite.config.ts
export default defineConfig(({ mode }) => ({
  resolve: {
    alias: {
      ...(mode === 'development' && { 'postgres': path.resolve(__dirname, 'node_modules/postgres/src/index.js') })
    }
  },
  // ...
}));

[calumpeak]

@ryanflorence is there any chance this could be looked at or patched?

[superjose]

I've found the root cause of the issue and a short-term workaround. I'm not certain where the fix should be applied, so I'll leave some context and allow the subject matter experts to figure out the rest. 🙂

The issue arises from how postgres.js applies the Cloudflare polyfills using package export conditionals. The runtime is being detected as a "worker," which is suitable for deployment to CF but problematic when developing locally with Remix since we're not actually running within the workerd runtime unless you use workers pages dev.

The workaround I used was to alias the import in development mode to use the default entry point instead of the polyfilled one. Might not be the best, but works for now.

// vite.config.ts
export default defineConfig(({ mode }) => ({
  resolve: {
    alias: {
      ...(mode === 'development' && { 'postgres': path.resolve(__dirname, 'node_modules/postgres/src/index.js') })
    }
  },
  // ...
}));

This is crazy, but it works!

I wanted to add my 2 cents.
I have a monorepo with turborepo, and I had to find the location of postgres outside of the remix app.

import {
	vitePlugin as remix,
	cloudflareDevProxyVitePlugin as remixCloudflareDevProxy,
} from '@remix-run/dev';
import { defineConfig } from 'vite';
import tsconfigPaths from 'vite-tsconfig-paths';
import { getLoadContext } from './load-context';
import path from 'path';

export default defineConfig(({ mode }) => ({
	plugins: [
		remixCloudflareDevProxy({ getLoadContext }),
		remix(),
		tsconfigPaths(),
	],
	resolve: {
		alias: {
			...(mode === 'development' && {
				postgres: path.resolve(
					__dirname,
					'../../libs/db/node_modules/postgres/src/index.js',
				),
			}),
		},
	},
}));

[wataruoguchi]

I've found the root cause of the issue and a short-term workaround. I'm not certain where the fix should be applied, so I'll leave some context and allow the subject matter experts to figure out the rest. 🙂

The issue arises from how postgres.js applies the Cloudflare polyfills using package export conditionals. The runtime is being detected as a "worker," which is suitable for deployment to CF but problematic when developing locally with Remix since we're not actually running within the workerd runtime unless you use workers pages dev.

The workaround I used was to alias the import in development mode to use the default entry point instead of the polyfilled one. Might not be the best, but works for now.

// vite.config.ts
export default defineConfig(({ mode }) => ({
  resolve: {
    alias: {
      ...(mode === 'development' && { 'postgres': path.resolve(__dirname, 'node_modules/postgres/src/index.js') })
    }
  },
  // ...
}));

Amazing! It works great in the meantime.

FYI: I had to add ssr to enable the alias.

    ssr: {
      ...(mode === "development" && {
        noExternal: ["postgres"],
      }),
    },
    resolve: {
      alias: {
        ...(mode === "development" && {
          postgres: path.resolve(
            __dirname,
            "node_modules/postgres/src/index.js"
          ),
        }),
      },
    },