Adopt a process for handling early features

[ddbeck]

This is mostly an issue to track the work represented by issues and PRs labeled with early features.

But there's an overarching need here as well. We ought to have an easily-triggered process that:

• Collects information about a possible early feature
• Makes a judgment about whether the possible early feature is sufficiently well-developed that it represents a genuine feature that is likely to be of interest to developers
• Enters it into web-features quickly (if it's a genuine feature) or else end or delay that feature

[ddbeck]

I have written a proposal which does a few things toward this issue. Here's the summary version:

• Defines an “early feature” as one that is not yet shipping (ungated) in a stable release of a browser.
• Identifies some likely goals for an early feature process:
  • Assigning IDs before features ship.
  • Avoiding giving developers the appearance of “missing” features.
  • Avoiding adding too many “failed” features (that is, features that never survive to at least “developers can try it”), if we can help it.
• Identifies some other risks that we might want to avoid, which aren’t exactly goals, but would be good outcomes for this process.
• Described some signals of likely good early features, some weaker signals which still might be dispositive, and some false signals that we ought to avoid taking into consideration.
• Proposes a process for handling these features by:
  • Detecting likely features not already associated with an ID (that is, like #3051 for Chrome)
  • Filing issues using a template specifically for early features
  • Scoring those submissions according to some criteria that biases us toward action for features that have strong signals (i.e., so we don’t block engineering or specification processes that wish to rely on a feature ID).
• Providing an outlet for features with weaker signals, by shunting them into the WebDX community group for shared confirmation or rejection.

My proposal has a weakness in that it couples "getting an ID" to "web-features takes responsibility for describing that feature." Based on the discussion about #3079 on the WebDX call today, I think it might be interesting to explore the idea of creating a pool of freely claimable but arbitrary IDs (purple-monkey-dishwasher), then move my proposed to the gate of getting a legible ID (drone-control) and web-features taking a more active responsibility in naming, describing, and maintaining that feature.

[jgraham]

My take on this is that in the early stage, being able to name web features is specifically of value to people who are actively working on standards. As a feature stabilises and the level of support grows, it then becomes more relevant to web developers, and finally once a feature has consensus and is shipping, developers (via MDN + etc.) are the main consumers of ids. Since we have mostly been dealing with features in later stages, we have correctly focused on web developer needs. But for early stage features that is the wrong approach because the use cases and audience are different.

At the earliest stages, I as a browser vendor, would like to use web-features to link together various pieces of data about where a specific proposal for the platform is in terms of progress through the standards and implementation process. For example I'd like to know:

• What's the relevant technical documentation for this proposal (draft standard / PR / explainer / etc.)
• What standards-position requests have been sought for this proposal, and what was their outcome
• What Intent documents are there for this proposal
• Who is shipping this proposal behind a flag/origin trial/etc.
• What is the relevant use counter data (or similar) data to look at for this proposal.

I would also like to capture some history. For example if a proposal is rejected and replaced with a new proposal that addresses some concerns I'd like to be able to see that evolution rather updating the old proposal in place, so it becomes unclear whether existing data refers to the old or new versions of proposals. This history will also provide insights into where the standards process is working well, and where improvements are required. For example a proposal undergoing frequent incompatible changes might be evidence that the original authors failed to consult the broader community sufficiently early in the process (e.g. before releasing a trial implementation, or formally asking for feedback).

Later in the process, as proposals firm up, more web developers start to get involved, for example by providing input in State-of surveys. Being able to associate this feedback with specific feature proposals is also useful (although of course one must be mindful that there's a difference between endorsement of a specific idea, and a specific concrete proposal).

Finally, once there is consensus on a specific design in standards bodies and non-experimental implementations are being shipped, web developers are the main target audience for the data, as today.

Because this process is a gradual transition from standards bodies and implementors being the main focus to developers being the main focus, it makes sense to reuse web-features at all stages, even though the primary use cases change along the way.

For that reason I would prefer that the early-stage web-feature ids look similar to the final form, but with something that distinguishes them as early stage features which are likely to undergo revision.

Developing a new feature already has a lot of requirements e.g. writing a specification, writing an implementation, formally requesting review (in working groups and via standards positions), getting approval to ship an experimental implementation etc. Therefore it seems reasonable to ask that minting a web-feature for the proposal is part of this process, without requiring us to optimise that too hard. For example a process where someone has to file an issue using a template requesting an id, provides a minimal amount of information e.g. a human-readable name and a link to the relevant technical document, whereupon a human checks that this is a real proposal, approves, and a script checks technical constraints such as that the id is not taken, and creates the relevant YAML file, would be ideal. This would be easy to incorporate into feature development processes, and could easily be required (e.g. by rejecting standards-position requests without an associated feature id).

By contrast, I think the idea of having a pool of free-to-take "nonsense" ids is optimising for the wrong constraints. I also think the oversight from WebDX in these early stages can be minimal; basically just checking that requests aren't spam, that the provided links are accurate, and that the requested id matches conventions and makes sense.

I also think that software package registries (npm, pypi, crates.io, etc.) are a good point of comparison here. They have similar constraints around uniqueness of names, and operate at a much higher volume, but generally have minimal manual curation. Although there are certainly occasional concerns around typosquatting etc. they generally work very well without needing to resort to opaque identifiers for packages. We have a much much lower volume, meaning some manual oversight is possible. Therefore I don't think we should be looking to take more drastic steps to protect our id space than those projects.

[jgraham]

To be clear: I'm not saying that the web-feature itself should contain the information I want as a browser vendor. I'm saying I want to use the id as a primary key to link together multiple different sources of information.

[ddbeck]

Thanks for writing that up, @jgraham. That's quite helpful to understand where you're coming and what you want from the process and how and when a feature ID might be useful.

I think I'm going to continue the bulk of this discussion on #3079, since that seems to speak most clearly to the non-developer audience you've articulated here.

In this issue, I'd like to narrow the scope to handling the situation we're already in with these other issues, where we have a good idea that something is soon to ship in a browser, but we're failing to do anything with them for lack of a process. I know of a few cases already where we failed to process those issues and instead simply added to both the backlog of unassigned BCD keys and backlog of unresolved issues. Not great, but I think it doesn't necessarily need a comprehensive model of proposals to resolve. Maybe we can work incrementally back from the moment of a BCD key appearing.

So to rephrase my inciting concern here: how do we avoid missing features about to ship? What's the next-last moment to mint a feature, if we wish to mint them before BCD keys appear? Does my proposal address that? Or does it go too early?