Migrate vulnerability database to it own repo

[vdeturckheim]

As many suggested, we should move the vulnerability database (core + ecosystem) to its own repo.
Security Advisories repository: https://github.com/nodejs/security-advisories

Repository Structure

Suggested repo structure would be

package.json
README.md
core
    README.md
    ...json
npm
    README.md
    ...json

Open Questions

1. should we also build and push an npm package with the vuln db?
2. should we change the current identifier (the nswg-eco / nswg-core) that is based on the running ids? if so, to what?

Action Items

• transform the existing format to the new structure
• announce the change via Twitter and official channels (we can contact Zibby Keaton)
• announce the change via the official Node.js website by PRing an announcement there
• an on/off switch for the vuln db in the current sec wg repo to be able to revert changes as needed (we can change the name of the directory or have a commit that removes the vuln_db directory, and later revert the commit if needed to restore immediately)
• make sure we sync data between this repo and the vuln repo during the announce phase so consumers can already start playing around with it.

[reedloden]

👍 from me for moving it.

As for directory structure, I would recommend you add a second layer under npm for the package name. Otherwise, you'll have a giant folder of IDs.

Looks how https://github.com/rubysec/ruby-advisory-db/ does it (under the gems directory).

[lirantal]

@vdeturckheim we probably can't rush it such as doing it after next wg session as we should be following the same process as my proposal regarding vuln db updates (#351) to avoid interruptions to db consumers.

Also, if we have the opportunity of making some changes in term of the db location then I think it's worth thinking about how to structure the vuln db (we had some discussions before in terms of directory structure, etc), and maybe bring up the npm package again for discussion too?

[MarcinHoppe]

👍 for moving, too. I am curious what drives the urgency?

[vdeturckheim]

@lirantal fair enough!

@MarcinHoppe I'm building a new integration for this db: push everything to Algolia to offer a free to use API/search API ^^

[lirantal]

can't wait for that API already ;-)

[pxlpnk]

I am also up for separating this. I like the directory structure that bundle audit has and it seems to work well there, we should look into adapting it.
We probably should have some tooling that will:

• transform the existing format to the new structure
• make sure we sync data between this repo and the vuln repo during the announce phase so consumers can already start playing around with it.

Not sure if packaging it as an npm module makes so much sense, but I have no strong opinions on it. I am looking forward to the API already.

I am happy to support the efforts with a dedicated repo.

[lirantal]

Sounds lovely Andreas!
Let's bring up all the points and summarize them in the first issue comment.

[vdeturckheim]

should we also build and push an npm package with the vuln db

Probably pretty heavy and I don't see any big value if we have the Algolia API

should we change the current identifier (the nswg-eco / nswg-core) that is based on the running ids? if so, to what?

I'd be keeping numbering system for core.
For ecosystem, as suggested, we could move to multiple directory structure. The current system would not make much sense indeed.

[bufferoverflow]

Great idea to create a dedicated repo, publish as npm whould be nice. However, publish the advisories on a website such as https://security.nodejs.org/advisories/ so that clients can fetch the vulnerability infos easily via http whould be more important from my perspective. This would allow to gather them easily for multiple purposes, e.g.

• gather all advisories for offline mode, as it was done by nsp: https://github.com/nodesecurity/nsp/blob/master/commands/gather.js
• integrate within registry middleware, e.g. https://github.com/verdaccio/verdaccio-audit
• use within yarn audit or other node package managers, see [feat] yarn audit yarnpkg/yarn#5808 (comment)

[mhdawson]

+1 on a separate repo, but as mentioned we should follow the process we are working on to let people know in advance before changing where they need to get the data from.