Check what packages will be dropped from package sets

[thomashoneyman]

The current package set contains packages that will not be included in the new registry, and therefore won't be included in the package sets produced by the registry (since the package sets only include packages from the registry).

An example: purescript-parsing will be dropped because of #249, unless its LICENSE file gets fixed. It would be good to understand what other packages will be dropped from the package sets unless we fix something about them -- especially packages within the core, contrib, web, and node organizations (like purescript-parsing).

To handle this, we would need to:

1. Get a list of packages and package versions from the latest current package set
2. Run the legacy import tool to produce a bower-exclusions.json file
3. Cross-reference the bower-exclusions against the package set to find package versions found in both places

The list of package versions that exist in both the package set and the bower exclusions represents packages currently provided by the set but which will no longer be provided going forward. This is also the list of packages that we really ought to fix, if we're going to manually fix any packages, to provide a smooth upgrade experience for users.

[maxdeviant]

Still cleaning up the implementation a bit, but here's the initial list of packages that will currently be dropped from the package set:

aff-bus v5.0.0 will be dropped from the package set due to: manifestError
errors v4.1.0 will be dropped from the package set due to: manifestError
foreign-generic v11.0.0 will be dropped from the package set due to: manifestError
free v6.0.1 will be dropped from the package set due to: manifestError
geometry-plane v1.0.1 will be dropped from the package set due to: manifestError
gl-matrix v2.0.1 will be dropped from the package set due to: manifestError
grid-reactors v0.0.1 will be dropped from the package set due to: manifestError
homogeneous v0.3.0 will be dropped from the package set due to: manifestError
node-he v0.2.0 will be dropped from the package set due to: manifestError
node-sqlite3 v6.0.0 will be dropped from the package set due to: manifestError
parsing v7.0.0 will be dropped from the package set due to: manifestError
parsing-dataview v2.0.1 will be dropped from the package set due to: manifestError
parsing-repetition v0.0.6 will be dropped from the package set due to: manifestError
prettier v0.3.0 will be dropped from the package set due to: manifestError
ps-cst v1.2.0 will be dropped from the package set due to: manifestError
supply v0.1.0 will be dropped from the package set due to: manifestError
typelevel-lists v2.0.1 will be dropped from the package set due to: manifestError
unicode v5.0.0 will be dropped from the package set due to: manifestError

(I'll expand the error messages to make it clearer what error would result in them being dropped)

[maxdeviant]

Cleaned up the script (purescript/registry#266) and here is the more detailed output:

aff-bus v5.0.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

errors v4.1.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

foreign-generic v11.0.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

free v6.1.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

geometry-plane v1.0.1 will be dropped from the package set due to:
	missing license

gl-matrix v2.0.1 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

grid-reactors v2.1.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: canvas-action
		Failed bounds: artemisSystem/purescript-canvas-action#43de19ee369d1ff9fe7eff1e583b828809fd9e36

homogeneous v0.3.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

node-he v0.2.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: test-unit
		Failed bounds: justinwoo/purescript-test-unit#compiler/0.12

node-sqlite3 v6.0.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: test-unit
		Failed bounds: justinwoo/purescript-test-unit#compiler/0.12
		Dependency: node-fs-aff
		Failed bounds: justinwoo/purescript-node-fs-aff#compiler/0.12
		Dependency: simple-json
		Failed bounds: #compiler/0.12

parsing v7.0.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

parsing-dataview v2.0.1 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

parsing-repetition v0.0.6 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: parsing-expect
		Failed bounds: markfarrell/purescript-parsing-expect

prettier v0.3.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: spec
		Failed bounds: justinwoo/purescript-spec#compiler/0.12

ps-cst v1.2.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

supply v0.1.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

typelevel-lists v2.0.1 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

unicode v5.0.0 will be dropped from the package set due to:
	bad license:
		Invalid SPDX identifier: NOASSERTION
		Did you mean BSD-Protection?

[maxdeviant]

I think there's some additional noise in the output due to #267.

I put a temporary fix in place and was able to narrow down the list further:

geometry-plane v1.0.1 will be dropped from the package set due to:
	missing license

grid-reactors v2.1.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: canvas-action
		Failed bounds: artemisSystem/purescript-canvas-action#43de19ee369d1ff9fe7eff1e583b828809fd9e36

node-he v0.2.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: test-unit
		Failed bounds: justinwoo/purescript-test-unit#compiler/0.12

node-sqlite3 v6.0.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: test-unit
		Failed bounds: justinwoo/purescript-test-unit#compiler/0.12
		Dependency: node-fs-aff
		Failed bounds: justinwoo/purescript-node-fs-aff#compiler/0.12
		Dependency: simple-json
		Failed bounds: #compiler/0.12

parsing-repetition v0.0.6 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: parsing-expect
		Failed bounds: markfarrell/purescript-parsing-expect

prettier v0.3.0 will be dropped from the package set due to:
	bad dependency versions:
		Dependency: spec
		Failed bounds: justinwoo/purescript-spec#compiler/0.12

[maxdeviant]

Nevermind, I see now that we're treating the NOASSERTION issue as valid, so it would appear my original (longer) list is correct 😅

[thomashoneyman]

Only 18 packages? That's pretty good! We should fix anything from core, contrib, web, and node (ie. free, aff-bus, parsing, unicode) and release patch versions with the fixed licenses. That way some packages may be dropped out of the current package set, but only a handful and there's a clear path forward to fix them. We could open issues on all of them if we want to, but it's not our responsibility to fix them all.

[thomashoneyman]

I'm a little confused by the errors license failure; we should be rewriting Apache-2 to Apache-2.0 as part of the legacy import tool, and the LICENSE file itself looks about right to me.

[maxdeviant]

I'm a little confused by the errors license failure; we should be rewriting Apache-2 to Apache-2.0 as part of the legacy import tool, and the LICENSE file itself looks about right to me.

The issue AFAICT is that the presence of a single bad license flags the package as excluded due to a bad license.

So in the case of errors, the package.json file contains a license value of Apache-2, which is being detected as a NOASSERTION by licensee. This NOASSERTION is then present in the list along with the other valid versions of the license, but still results in the package being flagged.