Clarify the optional role of glob* config in workbox-webpack-plugin docs

[johnnyreilly]

Library Affected:
workbox-webpack-plugin@3.0.0-alpha.2

Browser & Platform:
webpack 3

Issue or Feature Request Description:
Hi!

I'm in the process of writing a PWA guide for the webpack documentation site: webpack/webpack.js.org#1737

I realised that Workbox 3.0 was in alpha now and so it was probably better to use that rather than 2.0. However, I'm getting puzzling errors using this config with Workbox 3:

        new WorkboxPlugin({
            // we want our service worker to cache the dist directory
            globDirectory: 'dist',
            // these are the sorts of files we want to cache
            globPatterns: ['**/*.{html,js,css,png,svg,jpg,gif,json}'],
            // this is where we want our ServiceWorker to be created
            filename: path.resolve('dist', 'sw.js'),
            // these options encourage the ServiceWorkers to get in there fast 
            // and not allow any straggling "old" SWs to hang around
            clientsClaim: true,
            skipWaiting: true,
        }),

It errors out with "swDest" is not a supported parameter.

(node:720) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): ValidationError: "swDest" is not a supported parameter.
(node:720) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate
the Node.js process with a non-zero exit code.
john@john-dev:/mnt/c/Source/webpack-follow-along$

However the prerelease docs suggest it is: https://developers.google.com/web/tools/workbox/reference-docs/prerelease/module-workbox-webpack-plugin-WorkboxWebpackPlugin

If I read this correctly, it says "All the options as passed to module:workbox-build.generateSWString.".

And that includes swDest. Am I missing something?

https://developers.google.com/web/tools/workbox/reference-docs/prerelease/module-workbox-build#.generateSWString

[johnnyreilly]

Further to that, if you drop the swDest parameter you get this error:

(node:882) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: One of the glob patterns doesn't match any files. Please
remove or fix the following:{
  "globDirectory": "dist",
  "globPattern": "**/*.{html,js,css,png,svg,jpg,gif,json}",
  "globIgnores": [
    "node_modules/**/*"
  ]
}

This seems to be talking about globPattern but the docs speak only of globPatterns

Again somewhat confusing but I infer that globPattern is being constructed from globPatterns? Either way, globbing no longer seems to work. Unless I'm doing it wrong?

[johnnyreilly]

So my conclusion is that the mechanism for passing config is either broken (unlikely) or has changed and I'm just using the plugin wrong. Would you be able to clarify how config of the plugin should take place now please? It's probably worth updating the docs as well.

Thanks again for all the work you're doing; I really appreciated it.

[jeffposnick]

Hello @johnnyreilly—Yes, the build configuration docs are still a work in progress. Apologies, and thanks for bearing with us as we get things in order for the official v3 release.

As of #1085, which was part of the alpha.2 release, config options are passed through from the WorkboxWebpackPlugin() constructor to the underlying workbox-build.generateSWString() method. The extra wrinkle is that the webpack plugin will also generate its own manifest that contains the assets from the webpack build.

So you can use options like globPatterns, etc., but they're only relevant if you need to pick up additional URLs that aren't included in the compilation assets, for whatever reason. You could also pass through options like clientsClaim now.

swDest isn't a valid parameter in workbox-build.generateSWString(), which is why it was being rejected. (This will be clarified in the docs.)

Your error about a glob pattern not matching a file should be a legitimate error, due to that particular pattern not matching any files. Can you confirm that? (The logging tells you which specific pattern didn't match anything, which is why the log message is in the singular, rather than the plural.) But at the same time, you don't need to use globPatterns, etc., if you're fine with precaching only the compilation assets. In that case, the following should suffice:

new WorkboxPlugin({clientsClaim: true, skipWaiting: true});

[johnnyreilly]

Thanks for responding @jeffposnick - I appreciate you taking the time.

I think I've worked out what's available to me:

// from https://github.com/GoogleChrome/workbox/blob/v3.0.0-alpha.2/packages/workbox-build/src/entry-points/options/common-generate-schema.js

// Add some constraints that apply to both generateSW and generateSWString.
module.exports = baseSchema.keys({
  cacheId: joi.string(),
  clientsClaim: joi.boolean().default(defaults.clientsClaim),
  directoryIndex: joi.string(),
  ignoreUrlParametersMatching: joi.array().items(joi.object().type(RegExp)),
  navigateFallback: joi.string().default(defaults.navigateFallback),
  navigateFallbackBlacklist: joi.array().items(joi.object().type(RegExp)),
  navigateFallbackWhitelist: joi.array().items(joi.object().type(RegExp)),
  runtimeCaching: joi.array().items(joi.object().keys({
    urlPattern: [joi.object().type(RegExp), joi.string()],
    handler: [joi.func(), joi.string().valid(
      'cacheFirst',
      'cacheOnly',
      'networkFirst',
      'networkOnly',
      'staleWhileRevalidate'
    )],
    options: joi.object().keys({
      cacheName: joi.string(),
      plugins: joi.array().items(joi.object()),
      cacheExpiration: joi.object().keys({
        maxEntries: joi.number().min(1),
        maxAgeSeconds: joi.number().min(1),
      }).or('maxEntries', 'maxAgeSeconds'),
      cacheableResponse: joi.object().keys({
        statuses: joi.array().items(joi.number().min(0).max(599)),
        headers: joi.object(),
      }).or('statuses', 'headers'),
    }),
  }).requiredKeys('urlPattern', 'handler')),
  skipWaiting: joi.boolean().default(defaults.skipWaiting),
})

// https://github.com/GoogleChrome/workbox/blob/v3.0.0-alpha.2/packages/workbox-build/src/entry-points/options/generate-sw-string-schema.js

// Define some additional constraints.
module.exports = commonGenerateSchema.keys({
  globDirectory: joi.string(),
  globPatterns: joi.array().items(joi.string()).default(
    defaults.generateSWStringGlobPatterns),
  importScripts: joi.array().items(joi.string()).required(),
});

Also globIgnores no longer appears to be available. Is that intentional? How does one ignore with v3?

if you're fine with precaching only the compilation assets. In that case, the following should suffice:
new WorkboxPlugin({clientsClaim: true, skipWaiting: true});

Yes that does work. So maybe there is a problem? Here is a minimal repo if that helps: https://github.com/johnnyreilly/workbox-3-globPatterns-bug-repro

[johnnyreilly]

To be clear, (my comment above may not have been) I think there is an issue. With globDirectory / globPatterns - my minimal repro illustrates the issue. Running locally it fails with:

clean-webpack-plugin: /mnt/c/Source/workbox-3-globPatterns-bug-repro/dist has been removed.
(node:1499) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: One of the glob patterns doesn't match any files. Please remove or fix the following:{
  "globDirectory": "dist",
  "globPattern": "**/*.{html,js,css,png,svg,jpg,gif,json}",
  "globIgnores": [
    "node_modules/**/*"
  ]
}
(node:1499) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Interestingly the error message implies that globIgnores still exists. Yet it doesn't show up in the source code I found above. I'm assuming I'm looking at the wrong souce there...

[jeffposnick]

Hello @johnnyreilly—I cloned https://github.com/johnnyreilly/workbox-3-globPatterns-bug-repro and followed your build steps.

The issue that you're running into is that you're opting-in to using globPatterns and globDirectory with the webpack plugin, but you're referring to a directory (dist/) that doesn't yet exist at the time the webpack plugin is run. dist/ isn't created until after the first webpack compilation is completed, so the glob matches against it fail.

This is great feedback about informing the workbox-webpack-plugin v3 documentation (to be created in a follow-up PR to google/WebFundamentals#5498). In v3 we do support setting globPatterns and globDirectory, but it's optional, and is only intended when you need to pick up additional assets to precache that exist outside of the webpack compilation. If you're precaching assets that are part of the compilation (as you appear to be in this case) then there's no need to set globPatterns or globDirectory.

I'm going to leave this issue open as a reminder to clearly explain this in the forthcoming documentation.

[johnnyreilly]

The issue that you're running into is that you're opting-in to using globPatterns and globDirectory with the webpack plugin, but you're referring to a directory (dist/) that doesn't yet exist at the time the webpack plugin is run. dist/ isn't created until after the first webpack compilation is completed, so the glob matches against it fail.

Yes that makes sense. However, I'm guessing this is always going to be the case when the webpack / Workbox build is run in a continuous integration setup.

So does that mean a change need to be made to the plugin to delay running until the first webpack compilation has been completed? Otherwise it's presumably not going to be possible to use the globPatterns and globDirectory options in the CI use case (probably the main use case for me).

[jeffposnick]

We could theoretically make this either a warning rather than a fatal error, or just ignore it entirely (though I'd rather not do that, since misconfigured glob patterns is a common source of frustration).

But what I'd like to know more about is what you need Workbox to precache which

1. Don't exist in their final location until after the webpack compilation is complete the first time.
2. Isn't created as part of the webpack compilation.

[johnnyreilly]

But what I'd like to know more about is what you need Workbox to precache which

1. Don't exist in their final location until after the webpack compilation is complete the first time.
2. Isn't created as part of the webpack compilation.

For 2. : I don't want to cache anything that isn't dropped in dist as part of the webpack compilation. (Too many double negatives but hopefully you follow me.) dist is my output - the only thing that ends up on a users computer should have been dropped into dist during compilation. Likely, that will be done by webpack; it's possible that webpack may have been compiling source code to produce js output, or simply copying files in the case of images etc.

For 1 : I want to pre-cache artifacts of the webpack build process. So, if my build dumps out a bunch of images to the dist folder I'd likely want Workbox to precache those. If it dumps out some json files perhaps I'll want to cache those. Likewise js / css etc

Essentially if webpack builds it, I want Workbox to be able to precache it. Does that make sense?

[jeffposnick]

Yup, it makes sense.

So the takeaway there is that you shouldn't be using globPatterns and globDirectory at all. The default behavior of the webpack plugin, without those options configured, will give you the precaching behavior you expect.

I've attempted to clarify this at https://github.com/google/WebFundamentals/pull/5521/files#diff-a38ccf8d6592f317c963a81e44622233R36

Let's move any feedback about how this is explained to that PR, and I'll close this for the time behing.