Central packaging registry

[terah]

Hi,

I just read https://blog.gopheracademy.com/advent-2016/saga-go-dependency-management/ and thought I'd add some ideas.

Sorry if this has been discussed before or is outside the scope of this project.

Private packaging registry

It is common to use a mix of publicly accessible deps and deps that come from privately hosted repos. I work in an organisation that has many private packages and it's preferable that we wouldn't have to list those packages on a public packaging registry in order to get the full functionality of the tool. So, I'm flagging the idea of allowing for configuration of a private packaging registry. PHP has more-or-less settled on this tool to solve that problem: https://getcomposer.org/doc/articles/handling-private-packages-with-satis.md#private-packagist

[sdboyer]

Ah, I didn't see you'd posted this when I was responding to #176 :)

A registry is one of those topics that's been through the bikeshedding mill quite a bit. Some of my thoughts on it:

1. We should probably have one eventually
2. We do not NEED it now, we can slot it in later, and it would be an error to block any current progress on having it (which I know you're not suggesting - just being clear)
3. It should be immutable and append-only
4. It should be easy to set up private instances (yep, satis is great)
5. Yanking is a great idea (per Security service #176); a registry is the place through which we'd want to distribute, and possibly also coordinate, security information
6. Once we create the hosting platform, there should be a central, public one, and dep (or its successor) should be extended to include tools for managing releases within it (a la cargo, clojars, hex/mix, npm, pub, etc...)
7. There are lots of potholes and gotchas that other systems have encountered in building these, and we need to make sure we learn from them
8. We'll need to define a new import path pattern to deal with projects sourced from registries
9. Given the existing ecosystem's reliance on direct VCS interaction, it may be worth using a/the registry as a sort of proxy - can help with things like big projects have a tendency to be rate limited by github #115

That's the tip of the iceberg.

Item 2 is very important: IMO, we have other work that matters more right now. But this is FLOSS; if there's someone who's really inspired to work on this, then I'd love to work with them to hammer out a plan. I designed gps' SourceManager with the express idea that a subset of its methods provide the information that a registry would need to provide, so that this very transition would be reasonably smooth, if/when it came.

[nathany]

I do wonder if a central registry in the typical sense is necessary. That is, a place that stores the actual code in case it goes away.

Some sort of central index that references GitHub, etc. may be sufficient as a place to distribute security information. Right now godoc.org is the closest thing we have to serving this purpose.

[sdboyer]

Sure, and having a registry doesn't necessarily entail storing actual code with it. There are lot of useful properties registries might provide without that - for example: #174 (comment)

[nathany]

❯ gem install bundler
Fetching: bundler-1.14.4.gem (100%)
Did you know that maintaining and improving Bundler and RubyGems.org costs more than $25,000 USD every month? Help us keep the gem ecosystem free for everyone by joining the hundreds of companies and individuals who help cover these costs: https://ruby.to/support-bundler

There is that.

[rmg]

@nathany

I do wonder if a central registry in the typical sense is necessary. That is, a place that stores the actual code in case it goes away.

@sdboyer

Sure, and having a registry doesn't necessarily entail storing actual code with it.

It would probably be useful to make a conscious effort to distinguish between registry (index/metadata) and repository (source/binaries/content) here.

[sdboyer]

@rmg I agree that it's a worthwhile distinction, but in our current context, repository would be trading one ambiguity for another, as here it's generally be assumed to refer to a version control repository.

[AlekSi]

You may want to take a look at http://gonuts.io, back from Go 1.0 days.

[sdboyer]

@AlekSi yeah, i've looked at gonuts.io before, and now have gone through the code again at your suggestion - is there something in particular you think we should take from it?

[AlekSi]

I totally forgot why I mentioned this. 🤦‍♂️