Three Branches of Governance - Proposal

[countspongebob]

The long term health of any open source project depends on the people that contribute. Just as importantly, those people need a framework for working together effectively. Kubernetes continues to enjoy rapid growth and adoption and has been transitioning rapidly from an exciting new project to one with large numbers of productions systems, many at scale.

One of the very important balances between forces our project must navigate is the balance between speed of innovation and feature evolution and the needs of the production users for boring, dependable infrastructure.

As the project has grown in usage, it has also grown in contributors. The more informal practices that suited our project in the early days are proving unsuitable, in part because those informal practices have been strongly based on personal relationships between the early contributors. Those practices and norms are opaque to new contributors and had become a barrier to attracting new contributors and ensuring they can be productive members of the community.

As production deployments of Kubernetes have expanded, the need for longer range planning by those users has become more urgent. As Kubernetes becomes more central to the infrastructure of any company, the need for roadmaps, goals, and planning transparency have also become more central.

This has led to one very significant (currently in place) change to the Kubernetes Way (PM group), and another major one is proposed - the Council of Elders. I propose that a third is needed.

A significant event in the Kubernetes community was the formation of the Product Management Group. The value of this group to aid in future planning, roadmaps, and in the feature-level planning of the project is undeniable.

Teams that tilt the power structures towards engineering teams tend to build great architectures, but struggle with user experience and to build features users really want. Teams that tilt the power structures towards product management are often feature-rich with stove-piped architectures that struggle to scale and meet their SLOs. One of the biggest challenges leaders have is balancing these views. The Kubernetes project has this same challenge.

Introduction of the Council of Elders is not just a positive step for the project but an utter necessity, and I support creation immediately. Critically, though, this Council must have wide enough representation and diversity, and sufficiently engaged members to directly guide and influence the main technical and architectural issues of the project. I echo the sentiments of many in the community: #28

Today, the Product Management Group has also become the de facto process management and project policy owner for the project. Just as with features vs architecture tradeoffs, project policies have to balance the needs and desires of the product management view of the project with the developer view, and manage the tradeoffs between predictability, reporting, agility, and productivity. The speed of change of processes can have a profound effect on project productivity - both positive and negative - and are not best managed from the product management view alone.

Project policies play a key role in how these tradeoffs are discussed, and how decisions are made and communicated. These policies are not meant to make those tradeoffs, but to ensure that balanced, judged, and rational decisions are made in a transparent way, that the policies of the project are documented, communicated and followed.

I propose model that has a track record of success....Three groups with clear separation of empowerment and responsibilities:

• Product Management Group (Features, Roadmaps, Releases, Documentation)
• Council of Elders (Architecture, Design, Owner of Technical Standards)
• Project Policy Board (Processes, SIGs, Transparency, Traceability, and Legal/Licensing)

For the separation-of-powers model to work well it is especially important that these three groups have non-overlapping members.

Project policies should be treated with equal respect and care to features and architecture, especially in a large open source project, where coordinating large groups of people from many companies is such a challenge.

This third empowered group in combination will help Kubernetes succeed over the long run.

I ask for your support in the community for this proposal.

-Bob Wise

[timothysc]

I like it. I think the @kubernetes/kubernetes-pm & @kubernetes/sig-contributor-experience-misc can/should propose items but they should not act on autonomy without the buy in from the (Project Policy Board), whose ultimate goal is to ensure the consistency of the project and overall satisfaction of those contributing to the project. I think the distinction is important.

[countspongebob]

I entirely agree. Guidance and input from the sig contributor experience is critical. Would a required representative from that SIG on the PPB work?

[zehicle]

+1

[philips]

I am on the fence on this after the discussion in the community hangout.

We already have a number of processes in place that are poorly functioning because not enough people care about them. Another approach would be creating a tighter "this existing process isn't working" to "lets try this experiment" feedback loop. Two examples top of mind for me:

1. client-go is the first thing to be pulled out of kubernetes.git and I would love updates on if that experiment is working on the ground (effort invested, problems solved, etc), I think many people assume the decision is made and everything is going great.
2. ensuring our API is well audited and reviewed is important but making even minor changes can be confusing between code reviewers, approvers, the SIG, etc. We have reached a point where there are so many reviewers and stakeholders that it is hard to know who makes the final call. So, we are stable there through stagnancy to some degree.

I will keep thinking on it though.

[derekwaynecarr]

I am confused by some of this. It was my understanding that individual sig groups and their technical leadership defined roadmaps, implemented features, and authored much of the documentation. The PM group in my experience just ensured the process is followed across the project and ensured status is rolled up from various sig groups. I can't tell if you are saying that a PM group identifies a feature and directs a SIG on a to implement. I disagree if that is the sentiment proposed. I prefer to keep roadmap decisions local to the SIG.

[countspongebob]

Derek:

In general, I'm not proposing anything about the PM group as regards feature management, roadmaps, etc. that is any different than what they are doing today. I'm suggesting that the role of the PM group be carefully narrowed to be just that, and to exclude project wide process and governance changes. The scope of the Council of Elders should also be similarly clear.

I am saying that project governance should be officially outside the scope of the PM group and have its own home. Examples would be...

• SIGS are expected to record and publish their meetings
• PRs are the mechanism for publishing content for review and comment
• 2-factor authentication is used for github accounts

These are the common and expected ways of working across the entire community.

There was an interesting example of this that came up in the PM meeting today, where there was a suggestion that a SIG lead had to approve something. This would have represented a new responsibility and power for SIG leads that would really need to be vetted fully in the community, and this is where we need to take a lot of care.

[bgrant0607]

@countspongebob @timothysc

With respect to project processes, another possibility is that SIG contributor experience could own that and more people could get involved to help. There's tons of work to do, such as writing up our design-proposal policy. We don't have a shortage of ideas for improvements. We're lacking people to drive, implement, communicate, and roll out improvements (and test, measure, rollback, etc.).

[bgrant0607]

Another example: I have yet to see a volunteer to update and organize our contributor docs.

[bgrant0607]

@philips I think new APIs and API changes should be covered by a proposal process, which I agree needs to be drafted. We'll need to make the process and stakeholders clear.

[bgrant0607]

@philips I'd be happy with a clause about evaluating new processes N weeks after they are introduced. In at least some cases, we'll need data to do so. I'd like data to determine how well the OWNERS changes are working, for example.

[bgrant0607]

There was an earlier suggestion by @shtatfeld for contributor experience to improve processes:

#28 (comment)

Comment I also made on the elders proposal:

The project is a meritocracy. People are promoted in authority based on the scope, quality, quantity, and duration of their contributions. Anyone who wants to eventually own an area needs to step up and contribute to that area first.

[bgrant0607]

Proposed refinement/change to the proposal:

We have an established pattern of reviewer/approver (and Champion/Sponsor) in the project: #286

We could clarify which processes were owned by:

• The PM group
• Contributor experience
• Release team

And then add project-wide PROCESS REVIEWERS and PROCESS APPROVERS.

As with API REVIEWERS and API APPROVERS, those groups should be comprised of people who have designed, driven, implemented, and rolled out new processes.

[thockin]

This has focused a lot on process definition, which is surely needed, but I have another concern. There will inevitably be a challenge to authority, on some topic. Before that occurs, I'd like to get the authority structure specified. My thinking i that the "Elders Council", if it exists, is a final arbiter and occasional direction-setter, but not a regularly convening architectural body. Brian is busy on the contrib ladder ( #286) so I would propose that the elders either be elected from the LEADS or *be* the LEADS. The PM group and this proposed project policy board are OK with me, though I am not sure I see the need for 2 groups. What is still missing is regional authority. Who is the federation TL? It's someone the federation SIG has approved. Can we formalize this? Every SIG should have not only a lead (to run the SIG) but a TL who is responsible for guiding the SIG's technical efforts and being the ultimate approver. This person must come from the area's OWNERS. I have very little energy for process when it is not helpful, but I think that formalizing the ownership of areas is actually important, and as much as I love consensus, it's not realistic to expect it in all cases. So I move that we write our little constitution, in broad strokes, and flesh out the details of each role in docs like Sarah's "elders" doc. This is in parallel to the contrib ladder, which I see as a vehicle for on-boarding people and comprehending the structure of the human org.

…

On Thu, Jan 26, 2017 at 7:36 PM, Brian Grant ***@***.***> wrote: Proposed refinement/change to the proposal: We have an established pattern of reviewer/approver (and Champion/Sponsor) in the project: #286 <#286> We could clarify which processes were owned by: - The PM group - Contributor experience - Release team And then add project-wide PROCESS REVIEWERS and PROCESS APPROVERS. As with API REVIEWERS and API APPROVERS, those groups should be comprised of people who have designed, driven, implemented, and rolled out new processes. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#295 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFVgVBS2sH-dPWTB4Sn68zAoON4OqFMgks5rWWY3gaJpZM4LuXXG> .