Documentation Refactor & Roadmap with MKDocs

[RoseWrightdev]

Introduction

I've incorporated the community's feedback from my last proposal to rework the documentation in Next.js, simplifying the process to better align with the needs of our audience and maintainers as networking engineers. While we are still using MkDocs, it's best to think of this as a migration due to the substantial changes and scope. This plan is long-term and high-level, and by no means must we follow it exactly. It's important to focus on the goal of improving the user experience through tried and true best practices laid out by The Grand Unified Theory of Documentation and subsequently SIG Docs.

Documentation Outline

outline:
    # "Welcome" serves as the high-level landing page and "front door" to the documentation.
    # Its goal is to orient the user and direct them to the section that best fits their needs.
    # This is NOT the website's landing page.
    # Example: https://kubernetes.io/docs/home/
    - "Welcome"

    # Tutorials are learning-focused, designed to take a user through a complete, hands-on learning exercise.
    # The tone should be encouraging and guide the user step-by-step.
    # https://diataxis.fr/tutorials/
    - "Tutorials"
        # The Quick Start is for users who want to see value immediately.
        # It should provide the fastest possible path to a working example, ideally with a docker image and a frontend visualizing whats happening.
        # https://react.dev/learn
        - "Quick Start"
        # Hello World is for users who want to understand the fundamental resources and how they connect.
        # It's a more deliberate, foundational walkthrough than the Quick Start.
        - "Hello World: Your First Gateway"

    # How-to Guides are goal-oriented recipes for users who already know what they want to accomplish.
    # They should be concise, focused on a single task, and easy to follow.
    # https://diataxis.fr/how-to-guides/
    - "How-to Guides":
        - "Configure a Simple Gateway"       # The most basic, foundational guide.
        - "HTTP Routing"                     # Covers common path/header/query param routing.
        - "gRPC Routing"                     # Specific guidance for a key L7 protocol.
        - "TCP Routing"                      # For L4 use cases.
        - "TLS Configuration"                # Covers certificates, termination, and other security aspects.
        - "Traffic Splitting"                # For canary releases and blue/green deployments.
        - "HTTP Header Modification"         # A common, practical need for manipulating requests/responses.
        - "HTTP Redirect & Rewrite"          # Covers URL rewrites and permanent/temporary redirects.
        - "HTTP Request Mirroring"           # For traffic shadowing and non-disruptive testing.
        - "Backend Protocol"                 # How to specify protocols like HTTP/2 or gRPC to the backend.
        - "Migrating from Ingress"           # A critical guide for a huge portion of the user base. Should address common patterns. Frictions. Etc.
        - "Implementers Guide"               # A key guide for the vendor/provider ecosystem building controllers.

    # Explanation provides understanding and context. This section builds a user's mental model of the API.
    # It answers "why" questions and explains the concepts behind the API's design.
    # https://diataxis.fr/explanation/
    - "Explanation":
        - "Introduction"                         # What is Gateway API and what core problems does it solve?
        - "Why Gateway API? (A Comparison to Ingress)" # A persuasive but balanced argument for adoption, targeting experienced k8s users.
        - "Core Concepts":                        # The "big ideas" that make Gateway API unique.
            - "The Role-Oriented Model (Ian, Ana, & Chihiro)" # Explains the *who* (personas), the most foundational concept.
            - "Design Principles (Portable, Expressive, Extensible)" # Explains the *how* and *why* of the API's structure.
            - "The Security Model"               # Explains the *safety* and delegation model that results from the role-oriented design.
            - "Policy Attachment"
            - "Service Mesh (GAMMA)"
            - "API Development Methodology"        # Explains how we think about updates to gateway api
        - "Use Cases":                            # Connects concepts to real-world scenarios.
            - "Ingress (Managing North-South Traffic)"
            - "Service Mesh (Managing East-West Traffic)"
        - "Best Practices": # For users moving from "it works" to "it works well in production."
            - "BackendTLS" # Explicity tell users to not misuse these features -> do's and dont's
            - "Policy Attachment" # ^
            - "CRD Management"
            - "Versioning Strategy"
        - "Ecosystem & Tooling":                  # Helps users understand the community and available tools.
            - "Implementations"                  # list of which proxies/controllers implement the API.
            - "Conformance"                      # Explains what it means to be a conformant implementation.
            - "Comparisons":
                - "1.3"
                - "1.2"
                - "1.1"
                - "1.0"

        - "Talks & Presentations":                # For deeper dives and community context.
            - "Future Events"
            - "KubeCon 2024"
            - "KubeCon 2023"
            - "KubeCon 2022"
        - "Glossary"                              # Here we want to have links to all of the jargon we use spefic to Gateway API

    # Reference is for technical, factual information. It should be accurate, comprehensive, and example-driven.
    # The audience is users and implementers who need to look up specific details.
    # https://diataxis.fr/reference/
    - "API Reference":
        - "Full Specification":                   # The canonical, formal API spec. The ultimate source of truth, aimed at implementers.
        - "Standard"
        - "Experimental"
        - "API Types":                            # Practical, example-rich reference pages for day-to-day use.
            - "Gateway"
            - "GatewayClass"
            - "Routes":
                - "(HTTP, gRPC, TCP, etc.)"
            - "Policies":
                - "(BackendTLS, BackendTraffic, etc.)"
            - "ReferenceGrant"
        - "Upgrade Guides":                       # Critical operational documents for users running in production.
            - "Upgrading to v1.1"                # Should include breaking changes, deprecations, and migration steps.
            - "Upgrading to v1.2"
            - "Upgrading to v1.3"

    # Contributing is for people who want to work on the project itself.
    - "Contributing":
        - "Getting Started":                        # The main on-ramp for new contributors. Should be very welcoming.
            - "Project Organization & Working Groups" # Explains who's who and the project structure.
            - "Finding an Issue to Work On"        # Actionable steps to making a first contribution.
        - "Contributor Guides":                     # The "handbook" for active and aspiring contributors.
            - "Contributor Ladder"                # The "career path" - explains roles and expectations at each level.
            - "Developer Guide (Setup & Testing)" # The primary "how-to" guide for setting up the development environment.
            - "Navigating the Codebase"           # A map of the repository.
            - "Release Cycle"                     # Explains the process and cadence of new releases.
            - "Style Guide"                       # Code and documentation style requirements.
        - "Enhancement Proposals (GEPs)":           # The "governance" section for proposing significant changes.
            - "Process Overview"
            - "Standard":
                - "GEP-XXXX: Name"
            - "Experimental":
                - "GEP-XXXX Name"
            - "Provisional":
                - "GEP-XXXX Name"

mkdocs.yml

We explicitly declare the navigation so we can control the order of the navigation elements.

  site_name: GatewayAPI
  theme:
    name: readthedocs
  repo_url: https://github.com/kubernetes-sigs/gateway-api
  plugins:
    - search
    - macros
    - redirects
  nav:
    - "Welcome": "welcome.md"
    - "Tutorials":
        - "Quick Start": "tutorials/quick-start.md"
        - "Hello World: Your First Gateway": "tutorials/hello-world.md"
    - "How-to Guides":
        - "Configure a Simple Gateway": "how-to-guides/simple-gateway.md"
        - "HTTP Routing": "how-to-guides/http-routing.md"
        - "gRPC Routing": "how-to-guides/grpc-routing.md"
        - "TCP Routing": "how-to-guides/tcp-routing.md"
        - "TLS Configuration": "how-to-guides/tls-configuration.md"
        - "Traffic Splitting": "how-to-guides/traffic-splitting.md"
        - "HTTP Header Modification": "how-to-guides/http-header-modification.md"
        - "HTTP Redirect & Rewrite": "how-to-guides/http-redirect-rewrite.md"
        - "HTTP Request Mirroring": "how-to-guides/http-request-mirroring.md"
        - "Backend Protocol": "how-to-guides/backend-protocol.md"
        - "Migrating from Ingress": "how-to-guides/migrating-from-ingress.md"
        - "Implementers Guide": "how-to-guides/implementers-guide.md"
    - "Explanation":
        - "Introduction": "explanation/introduction.md"
        - "Why Gateway API? (A Comparison to Ingress)": "explanation/why-gateway-api.md"
        - "Core Concepts":
            - "The Role-Oriented Model (Ian, Ana, & Chihiro)": "explanation/core-concepts/role-oriented-model.md"
            - "Design Principles (Portable, Expressive, Extensible)": "explanation/core-concepts/design-principles.md"
            - "The Security Model": "explanation/core-concepts/security-model.md"
            - "Policy Attachment": "explanation/core-concepts/policy-attachment.md"
            - "Service Mesh (GAMMA)": "explanation/core-concepts/service-mesh-gamma.md"
            - "API Development Methodolgy": "explanation/core-concepts/api-development-methodology.md"
        - "Use Cases":
            - "Ingress (Managing North-South Traffic)": "explanation/use-cases/ingress.md"
            - "Service Mesh (Managing East-West Traffic)": "explanation/use-cases/service-mesh.md"
        - "Best Practices":
            - "BackendTLS": "explanation/best-practices/backendtls.md"
            - "Policy Attachment": "explanation/best-practices/policy-attachment.md"
            - "CRD Management": "explanation/best-practices/crd-management.md"
            - "Versioning Strategy": "explanation/best-practices/versioning-strategy.md"
        - "Ecosystem & Tooling":
            - "Implementations": "explanation/ecosystem/implementations.md"
            - "Conformance":
              - "Comparisons":
                - "1.3": "explanation/ecosystem/conformance/1-3.md"
                - "1.2": "explanation/ecosystem/conformance/1-2.md"
                - "1.1": "explanation/ecosystem/conformance/1-1.md"
                - "1.0": "explanation/ecosystem/conformance/1-0.md"
        - "Talks & Presentations":
            - "Future Events": "talks/future-events.md"
            - "KubeCon 2024": "talks/kubecon-2024.md"
            - "KubeCon 2023": "talks/kubecon-2023.md"
            - "KubeCon 2022": "talks/kubecon-2022.md"
        - "Glossary": "explanation/glossary.md"
    - "API Reference":
        - "Full Specification":
          - "Standard": "api-reference/spec/standard.md"
          - "Experimental": "api-reference/spec/experimental.md"
        - "API Types":
            - "Gateway": "api-reference/types/gateway.md"
            - "GatewayClass": "api-reference/types/gateway-class.md"
            - "Routes (HTTP, gRPC, TCP, etc.)": "api-reference/types/routes.md"
            - "Policies (BackendTLS, BackendTraffic, etc.)": "api-reference/types/policies.md"
            - "ReferenceGrant": "api-reference/types/reference-grant.md"
        - "Upgrade Guides":
            - "Upgrading to v1.1": "api-reference/upgrades/v1-1.md"
            - "Upgrading to v1.2": "api-reference/upgrades/v1-2.md"
            - "Upgrading to v1.3": "api-reference/upgrades/v1-3.md"
    - "Contributing":
        - "Getting Started":
            - "Project Organization & Working Groups": "contributing/getting-started/project-organization.md"
            - "Finding an Issue to Work On": "contributing/getting-started/finding-an-issue.md"
        - "Contributor Guides":
            - "Contributor Ladder": "contributing/guides/contributor-ladder.md"
            - "Developer Guide (Setup & Testing)": "contributing/guides/developer-guide.md"
            - "Navigating the Codebase": "contributing/guides/navigating-codebase.md"
            - "Release Cycle": "contributing/guides/release-cycle.md"
            - "Style Guide": "contributing/guides/style-guide.md"
        - "Enhancement Proposals (GEPs)":
            - "Process Overview": "contributing/geps/process.md"
            - "GEP Statuses (Standard, Experimental, etc.)": "contributing/geps/statuses.md"
  hooks:
    - "redirect.py"

Streamlining the Refactor with a Mkdocs Python Hook

The minimum we need is a way to handle changing the file tree without breaking both internal and external links. Doing this by hand is functionally impossible due to time constraints. Therefore, we will leverage an MKDocs Python hook.

The Workflow

The core of this migration hinges on a MkDocs Python hook to automate the tedious parts of the reorganization. The goal is twofold:

1. Generate 301 Redirects: To ensure external links and user bookmarks don't break.
2. Fix Internal Links: To ensure the documentation site itself remains consistent and navigable after the file structure changes.

The script will manage this process by assigning a unique, persistent ID to every page.

The process is as follows:

1. Phase 1: Preparation (Before the move)
   • Assign Unique IDs: A script will traverse the current docs directory. For each .md file, it will generate a unique, human-readable ID (e.g., based on its path, like how-to-http-routing). This ID will be injected into the YAML frontmatter of each file.
   • Create "Before" State Map: The script will then create a map of the current structure: unique_id -> old_path. This map is saved to a file (e.g., redirect_map.json).
2. Phase 2: Manual Reorganization
   • With the IDs in place, we can now manually move files and directories to match the new information architecture outlined in this proposal. The content of the files is not changed, only their location.
3. Phase 3: Generating Redirects & Fixing Links (Using an on_config hook)

   • Create "After" State Map: On mkdocs build, the hook script runs. It walks the new file structure and creates an "after" map: unique_id -> new_path.

   • Compare and Generate Redirects: The script loads the "before" map and compares it with the "after" map. For any unique_id where the path has changed, it programmatically injects a redirect rule into the MkDocs configuration. For example:

     # This is generated by the script
     - 'old/guides/http.md': 'how-to-guides/http-routing.md'
     

   • Fix Internal Links: To make future refactors easier, the script can also find all relative markdown links (e.g., [link](../topics/concepts.md)) and replace them with a robust macro that uses the unique ID: [link]({{ internal_link('concepts-id') }}). This makes the links independent of the file structure. At build time, the script will replace [link]({{ internal_link('concepts-id') }}) with [link](./some-relative-path) to comply with mkdocs recommendations.

4. Phase 4: Build
   • MkDocs proceeds with the build. The mkdocs-redirects plugin uses the generated rules to create proper 301 redirects, and the mkdocs-macros plugin ensures all internal links point to the correct new locations.

This approach ensures that all links, both internal and external, remain functional throughout this significant refactor with minimal manual intervention.

Additional Consideration: Resilient Internal Linking

To improve maintainability moving forward, we should also implement a simple macro with mkdocs-macros to handle internal linking. This prevents future reorganizations from breaking links.

Instead of writing a fragile relative link:

My Link

We would use the page's unique ID:

{{ internal_link('another-page-id') }}

The macro would look up the page's current URL from a manifest and render the correct link at build time. This makes content robust against file moves.

Dependencies

• mkdocs-redirects: To handle the 301 redirects from old URLs to new ones.
• mkdocs-macros: To enable the custom internal_link macro for resilient linking.

Easy, but Impactful Improvements

readthedocs Theme

We should consider changing from our current theme to “readthedocs” due to it’s linear structure, more similar to the main Kubernetes documentation. This would require no additional dependencies. It is a one-line change.

Building a Landing Page

We should follow the standard and build a simple page similar to kubernetes.io so that the main domain doesn’t immediately direct to the documentation because it is jarring to users. This would be basic HTML & CSS.

Phased Rollout Plan

This is a large undertaking. We can break it down into manageable phases.

Phase 1: Foundational Reorganization

• Task 1: Implement the redirects.py hook script.
• Task 2: Physically move all existing .md files into the new directory structure outlined above.
• Task 3: Run the build. The script should generate all necessary redirects. Verify that old URLs correctly redirect to the new locations. (mkdocs dose this by default)
• Task 4: Update the nav section in mkdocs.yml to reflect the new structure.

Phase 2: Content Alignment & Rewriting

• Task 5: Audit every page. Does its content match its new home? A "How-to Guide" should be a recipe, an "Explanation" page should explain a concept, etc.
• Task 6: Create new placeholder pages for documentation that is missing from the outline.
• Task 7: Rewrite/refactor content to fit the Diátaxis model AND the Kubernetes Documentation Style Guide. This is the most significant content-focused part of the project.
• Task 8: Build a simple landing page.
• Task 9: Update the theme to readthedocs.

Call for Participation

This is a significant effort that will greatly improve the experience for all our users and contributors. I'm looking for feedback on this plan and volunteers to help tackle the phases outlined above, but I understand the project is already stretched thin.

As some of you may know, I'm a college student at the University of Minnesota. I've spoken with several of my classmates who are eager to contribute to this project. To make this migration a success, however, we need the support and expertise of the existing community.

Specifically, help would be appreciated in:

• Reviewing the proposed architecture.
• Auditing and rewriting content once the new structure is in place.

Please leave your comments, suggestions, and feedback below!

[candita]

Thanks, this will be great. One suggestion: https://kubernetes.io/docs/contribute/style/style-guide/ is a preferred resource for style.

[RoseWrightdev]

We're currently using MKDocs, and it lacks the functionality we need because it's just a simple static site generator. As it stands in the proposal, the next.config.js networking APIs do most of the work for us. Whatever framework we choose, we need a similar set of APIs to do this work because rewriting this functionality from scratch would massively expand the project. Right now, we are essentially just piping JSON data to the API; without it, this project becomes entirely impractical. We also need MDX and React at a minimum to implement something like the <InternalLink/> component. Another point is the wealth of templates and off-the-shelf components available in the Next.js/React ecosystem, like shadcn. I don't want to reinvent the wheel.

Edit:
I was gravely underestimating mkdocs! As @blake pointed out, much of this is possible within mkdocs without porting to Next.js. We could also polish these existing features with a Python hook or, as Blake said, a Git pre-commit hook.

[shaneutt]

I appreciate this well thought out proposal for an overhaul, our website is notoriously difficult to navigate. We discussed this on the call today, and there were some questions such as whether we can achieve any of the goals while sticking with the mkdocs, and how to maintain this sort of thing in the long run.

Do you have an example repository containing a site that uses this same tech stack? Ideally something small and illustrative?

[RoseWrightdev]

I appreciate this well thought out proposal for an overhaul, our website is notoriously difficult to navigate. We discussed this on the call today, and there were some questions such as whether we can achieve any of the goals while sticking with the mkdocs, and how to maintain this sort of thing in the long run.

Do you have an example repository containing a site that uses this same tech stack? Ideally something small and illustrative?

Tech stack as in Next.js with the Aria docs template? I'm not sure what you mean.

[snorwin]

In my view, the current content structure feels disorganized and fragmented, which makes navigation quite difficult. That said, I’m less concerned about whether links are hardcoded or whether the site is optimized for SEO.

What matters most to me is that this is technical documentation authored by developers and engineers. Because of that, I believe we should continue using a simple format like Markdown or AsciiDoc as the content source. The Gateway API site isn’t intended to be a flashy marketing website, and it should prioritize clarity, accuracy, and ease of contribution from technical folks.

If there are technical limitations or issues with the current MkDocs setup, I think it would be more valuable to raise them within the MkDocs project itself. Rather than switching frameworks, I’d prefer to invest our effort in restructuring and rewriting the actual content to improve its quality and usability.

[blake]

It feels like the primary motivation of this proposal is to avoid broken URLs. If that's the case, I think there's an opportunity to solve for most of these issues with the current docs system.

1. Absolute routing

MkDocs already avoids some of the identified edge cases by building URLs based off of the underlying directory structure. (MkDocs: Writing your Docs - File layout). It's not possible for pages or assets to have the same URL because two distinct resources cannot exist at the same file path.

Reserved Paths: The script should maintain a list of reserved paths (e.g., /api,/public) and fail the build if a seoPath conflicts with them.

I think this can already be accomplished by using the exclude_docs configuration parameter. This won't cause the build to fail, but it should achieve the intended result by excluding any content that may be introduced under these reserved paths.

# This prevents in the following directories under site-src/ from being included when the site is built.
exclude_docs: |
  /api-types/
  /geps/

2. 301 Redirect Generation

Renaming URLs is currently a bit of a pain because doing it correctly involves a few steps as you identified.

1. Move the file to a new location
2. Add a redirect under the redirects map in mkdocs.yml
3. Optionally update all files that reference the new file location. (This could be automated with a Git pre-commit hook.)

3. Dynamic and Resilient Internal Linking
   …
   Guaranteed Link Integrity: The build must fail if an component references a contentId that does not exist, eliminating broken internal links from the deployed site.

It's possible to enable this in MkDocs by setting a number of validation rules to warn and enabling the strict build mode.

[Seo-yul]

I used MkDocs to work on the localization #3882
Through this issue, I was able to understand how the MkDocs structure can cause certain path-related issues. (I think this is the topic now,)

However, I fully agree with @snorwin 's opinion. Although I am a Go engineer, I had no problem contributing using MkDocs. (Since I’m not a Next.js user, my perspective might have been off focus.)

While I believe that documentation maintenance is the responsibility of contributors and is definitely manageable, this experience also made me realize how beneficial it would be if technical maintenance could be supported.

[candita]

@RoseWrightdev here's a list of concrete suggestions. It may not align with Diataxis, but I think that model represents an ideal that doesn't fit all situations. For example, it doesn't tell you how to approach a case where you have very different audiences, e.g. we have both users and implementers sharing these docs so that guides and tutorials are going to converge in some cases.

In the Documentation Outline, consider putting the Explanations first in the menu, I suggest it be called “Concepts”. We are missing a few use cases from the Use Cases.

Under Best Practices - Versioning Strategy should probably go in Core Concepts, with API Development Methodology, if it points to https://gateway-api.sigs.k8s.io/concepts/versioning/

We don’t have tutorials yet, but we can provide some examples from various documentation sets (I have some, and implementers like Envoy Gateway do too, etc.). Talks and presentations probably belong under the tutorials section to some extent.

How-to Guides are missing a few of the guides in https://gateway-api.sigs.k8s.io/guides/ - cross-namespace routing, backend protocol selection, infrastructure attributes.

The “Ecosystem & Tooling” category seems misnamed - we want these links for both implementors and end users. Maybe a better name would be “Implementations and Conformance”.

We’re missing a FAQ.

The list of GEPs, which contain material about the state of a feature request, should not be under Contributing. Imo it deserves its own section, as it currently has. Also, we have a few other categories than Standard, Experimental, and Provisional.

Upgrade Guides would belong under How-to Guides, not API Reference, if we had any. It is really something that should be covered by each implementation’s documentation, not the API documentation so it may be safely omitted.

In the Phase 2 of the rollout plan when each page is audited, if it needs changes, you could just add a banner: “under construction”, or something similar. After we find out how many changes are needed, we could then prioritize them, or delete them, or combine content as needed.

[RoseWrightdev]

@candita I think all of these suggestions are great! I plan to implement all of them. Thanks for the feedback.