diff --git a/docs/README.md b/docs/README.md index 58fd7e41f..2fffb8b72 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,5 +1,19 @@ # Official Registry Documentation +## Project Documentation + +[`design_principles.md`](./design_principles.md) - Core constraints and principles guiding the registry design + +[`faq.md`](./faq.md) - Frequently asked questions about the MCP Registry + +[`roadmap.md`](./roadmap.md) - High-level roadmap for the MCP Registry development + +[`MCP Developers Summit 2025 - Registry Talk Slides.pdf`](./MCP%20Developers%20Summit%202025%20-%20Registry%20Talk%20Slides.pdf) - Slides from a talk given at the MCP Developers Summit on May 23, 2025, with an up-to-date vision of how we are thinking about the official registry. + +## API & Technical Specifications + [`openapi.yaml`](./openapi.yaml) - OpenAPI specification for the official registry API + [`api_examples.md`](./api_examples.md) - Examples of what data will actually look like coming from the official registry API -[`MCP Developers Summit 2025 - Registry Talk Slides.pdf`](./MCP%20Developers%20Summit%202025%20-%20Registry%20Talk%20Slides.pdf) - Slides from a talk given at the MCP Developers Summit on May 23, 2025, with an up-to-date vision of how we are thinking about the official registry. \ No newline at end of file + +[`architecture.md`](./architecture.md) - Technical architecture, deployment strategies, and data flows \ No newline at end of file diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 000000000..bd55b0a0c --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,207 @@ +# MCP Registry Architecture + +This document describes the technical architecture of the MCP Registry, including system components, deployment strategies, and data flows. + +## System Overview + +The MCP Registry is designed as a lightweight metadata service that bridges MCP server creators with consumers (MCP clients and aggregators). + +```mermaid +graph TB + subgraph "Server Maintainers" + CLI[CLI Tool] + end + + subgraph "MCP Registry" + API[REST API
Go] + DB[(MongoDB or PostgreSQL)] + CDN[CDN Cache] + end + + subgraph "Intermediaries" + MKT[Marketplaces] + AGG[Aggregators] + end + + subgraph "End Consumers" + MC[MCP Client Host Apps
e.g. Claude Desktop] + end + + subgraph "External Services" + NPM[npm Registry] + PYPI[PyPI Registry] + DOCKER[Docker Hub] + DNS[DNS Services] + GH[GitHub OAuth] + end + + CLI --> |Publish| API + API --> DB + API --> CDN + CDN --> |Daily ETL| MKT + CDN --> |Daily ETL| AGG + MKT --> MC + AGG --> MC + API -.-> |Auth| GH + API -.-> |Verify| DNS + API -.-> |Reference| NPM + API -.-> |Reference| PYPI + API -.-> |Reference| DOCKER +``` + +## Core Components + +### REST API (Go) + +The main application server implemented in Go, providing: +- Public read endpoints for server discovery +- Authenticated write endpoints for server publication +- GitHub OAuth integration (extensible to other providers) +- DNS verification system (optional for custom namespaces) + +### Database (MongoDB or PostgreSQL) + +Primary data store for: +- Versioned server metadata (server.json contents) +- User authentication state +- DNS verification records + +### CDN Layer + +Critical for scalability: +- Caches all public read endpoints +- Reduces load on origin servers +- Enables global distribution +- Designed for daily consumer polling patterns + +### CLI Tool + +Developer interface for: +- Server publication workflow +- GitHub OAuth flow +- DNS verification + +## Deployment Architecture + +### Kubernetes Deployment (Helm) + +The registry is designed to run on Kubernetes using Helm charts: + +```mermaid +graph TB + subgraph "Kubernetes Cluster" + subgraph "Namespace: mcp-registry" + subgraph "Registry Service" + LB[Load Balancer
:80] + RS[Registry Service
:8080] + RP1[Registry Pod 1] + RP2[Registry Pod 2] + RP3[Registry Pod N] + end + + subgraph "Database Service" + DBS[DB Service
:27017] + SS[StatefulSet] + PV[Persistent Volume] + end + + subgraph "Secrets" + GHS[GitHub OAuth Secret] + end + end + end + + LB --> RS + RS --> RP1 + RS --> RP2 + RS --> RP3 + RP1 --> DBS + RP2 --> DBS + RP3 --> DBS + DBS --> SS + SS --> PV + RP1 -.-> GHS + RP2 -.-> GHS + RP3 -.-> GHS +``` + +## Data Flow Patterns + +### 1. Server Publication Flow + +```mermaid +sequenceDiagram + participant Dev as Developer + participant CLI as CLI Tool + participant API as Registry API + participant DB as Database + participant GH as GitHub + participant DNS as DNS Provider + + Dev->>CLI: mcp publish server.json + CLI->>CLI: Validate server.json + CLI->>GH: OAuth flow + GH-->>CLI: Access token + CLI->>API: POST /servers + API->>GH: Verify token + API->>DNS: Verify domain (if applicable) + API->>DB: Store metadata + API-->>CLI: Success + CLI-->>Dev: Published! +``` + +### 2. Consumer Discovery Flow + +```mermaid +sequenceDiagram + participant Client as MCP Client Host App + participant INT as Intermediary
(Marketplace/Aggregator) + participant CDN as CDN Cache + participant API as Registry API + participant DB as Database + + Note over INT,CDN: Daily ETL Process + INT->>CDN: GET /servers + alt Cache Hit + CDN-->>INT: Cached response + else Cache Miss + CDN->>API: GET /servers + API->>DB: Query servers + DB-->>API: Server list + API-->>CDN: Response + cache headers + CDN-->>INT: Response + end + INT->>INT: Process & enhance data + INT->>INT: Store in local cache + + Note over Client,INT: Real-time Client Access + Client->>INT: Request server list + INT-->>Client: Curated/enhanced data +``` + +### 3. DNS Verification Flow + +```mermaid +sequenceDiagram + participant User as User + participant CLI as CLI Tool + participant API as Registry API + participant DNS as DNS Provider + participant DB as Database + + User->>CLI: mcp verify-domain example.com + CLI->>API: POST /verify-domain + API->>API: Generate verification token + API->>DB: Store pending verification + API-->>CLI: TXT record: mcp-verify=abc123 + CLI-->>User: Add TXT record to DNS + User->>DNS: Configure TXT record + User->>CLI: Confirm added + CLI->>API: POST /verify-domain/check + API->>DNS: Query TXT records + DNS-->>API: TXT records + API->>API: Validate token + API->>DB: Store verification + API-->>CLI: Domain verified + CLI-->>User: Success! +``` \ No newline at end of file diff --git a/docs/design_principles.md b/docs/design_principles.md new file mode 100644 index 000000000..ef400c2cd --- /dev/null +++ b/docs/design_principles.md @@ -0,0 +1,39 @@ +# MCP Registry Design Principles + +These are the core constraints that guide the design of the MCP Registry. They are not exhaustive, but they are the most important principles that we will use to evaluate design decisions. + +## 1. Single Source of Truth + +The registry serves as the authoritative metadata repository for publicly-available MCP servers, both locally-run and remote, open source and closed source. Server creators publish once, and all consumers (MCP clients, aggregators, etc.) reference the same canonical data. + +## 2. Minimal Operational Burden + +- Design for low maintenance and operational overhead +- Delegate complexity to existing services where possible (GitHub for auth, npm/PyPI for packages) +- Avoid features that require constant human intervention or moderation +- Build for reasonable downtime tolerance (24h acceptable) by having consumers cache data for their end-users + +## 3. Vendor Neutrality + +- No preferential treatment for specific servers or organizations +- No built-in ranking, curation, or quality judgments +- Let consumers (MCP clients, aggregators) make their own curation decisions + +## 4. Meets Industry Security Standards + +- Leverage existing package registries (npm, PyPI, Docker Hub, etc.) for source code distribution, obviating the need to reinvent source code security +- Use mechanisms like DNS verification, OAuth to provide base layer of authentication and trust +- Implement rate limiting, field validation, and blacklisting to prevent abuse + +## 6. Reusable, Extensible Shapes; Not Infrastructure + +- API shapes (OpenAPI, server.json) designed for reuse +- Enable private/internal registries using same formats +- Don't mandate infrastructure reuse - focus on interface compatibility + +## 7. Progressive Enhancement + +- Start with MVP that provides immediate value +- Build foundation that supports future features +- Don't over-engineer for hypothetical needs +- Each milestone should be independently valuable diff --git a/docs/faq.md b/docs/faq.md new file mode 100644 index 000000000..a8a19e70b --- /dev/null +++ b/docs/faq.md @@ -0,0 +1,179 @@ +# MCP Registry FAQ + +These questions come up often in discussions about the MCP Registry. If you have a question that isn't answered here, please start a discussion on the [MCP Registry Discussions page](https://github.com/modelcontextprotocol/registry/discussions). + +## General Questions + +### What is the MCP Registry? + +The MCP Registry is the official centralized metadata repository for publicly-accessible MCP servers. It provides: + +- A single place for server creators to publish metadata about their servers +- A REST API for MCP clients and aggregators to discover available servers +- Standardized installation and configuration information +- Namespace management through DNS verification + +### Is the MCP Registry a package registry? + +No. The MCP Registry stores metadata about MCP servers and references to where they're hosted (npm, PyPI, Docker Hub, etc.), but does not host the actual source code or packages. + +### Who should use the MCP Registry directly? + +The registry is designed for programmatic consumption by: + +- MCP client applications (Claude Desktop, Cline, etc.) +- Server aggregators (Smithery, PulseMCP, etc.) +- NOT individual end-users (they should use MCP clients or aggregator UIs) + +### Will there be a UI for browsing servers? + +A UI is planned as a future milestone after the initial API launch, but is not part of the MVP. + +## Publishing Servers + +### How do I publish my MCP server? + +Servers are published by submitting a `server.json` file through our CLI tool. The process requires: + +1. GitHub authentication +2. A public GitHub repository (even for closed-source servers - just for the metadata) +3. Your server package published to a supported registry (npm, PyPI, Docker Hub, etc.) +4. Optional: DNS verification for custom namespacing + +### What namespaces are available? + +- **With DNS verification**: `com.yourcompany/server-name` (reverse DNS notation) +- **Without DNS verification**: `io.github.yourusername/server-name` +- DNS verification is done via TXT records and enables authoritative namespacing + +### Is open source required? + +No. While open source code is encouraged, it is not required for either locally or remotely run servers. + +### What package registries are supported? + +- npm (Node.js packages) +- PyPI (Python packages) +- GitHub Container Registry (GHCR) + +More can be added as the community desires; feel free to open an issue if you are interested in building support for another registry. + +### Can I publish multiple versions? + +Yes, versioning is supported: + +- Each version gets its own immutable metadata +- Version bumps are required for updates +- Old versions remain accessible for compatibility +- The registry tracks which version is "latest" + +### How do I update my server metadata? + +Submit a new `server.json` with an incremented version number. Once published, version metadata is immutable (similar to npm). + +### Can I delete/unpublish my server? + +A reverse-publication flow is planned to allow quick deletion of accidentally published data. + +## Security & Trust + +### How do I know a server is from the claimed organization? + +DNS verification ensures namespace ownership. For example: + +- `com.microsoft/server` requires DNS verification of microsoft.com +- `io.github.username/server` is tied to a GitHub account or GitHub organization + +### What about typosquatting? + +The registry implements (or is slated to soon implement): + +- Automatic blocking of names within a certain edit distance of existing servers +- Community reporting mechanisms + +### Is there security scanning? + +The MVP delegates security to the underlying package registries. Future iterations may include: + +- Vulnerability scanning +- Dependency analysis + +### How is spam prevented? + +- GitHub authentication requirement +- Rate limiting (e.g., 10 new servers per user per day) +- Character limits and regex validation on free-form fields +- Potential AI-based spam detection +- Community reporting and admin blacklisting capabilities + +## API & Integration + +### How often should I poll the registry? + +Recommended polling frequency: + +- `/servers` endpoint: once per hour +- `/servers/:id` endpoint: once per version (results are immutable) +- Design assumes CDN caching between registry and consumers + +### Will there be webhooks? + +Not in the initial MVP, but the architecture supports adding webhooks for update notifications in the future. + +### Can I run my own registry instance? + +While the API shapes and data formats are designed for reuse, the registry implementation itself is not designed for self-hosting. Organizations needing private registries should: + +- Implement the same API shape +- Use the same `server.json` format +- Potentially mirror/filter the official registry data + +## Operations & Maintenance + +### What's the expected reliability? + +- 24-hour+ downtime tolerance is acceptable, so you shouldn't rely on the registry being always available. +- No direct end-user impact (consumers cache data) +- Relies on CDN for actual availability +- Volunteer maintainer model (no formal on-call) + +### How are DNS records verified? + +- Initial verification via TXT records +- Daily re-verification to catch domain transfers +- Publishing blocked if verification fails +- Historical packages remain available with warnings + +### What about GitHub repository transfers? + +The registry tracks GitHub repository IDs (not just URLs) to detect transfers. Daily checks update metadata if repositories move. + +### How is namespace transfer handled? + +Namespace transfers (e.g., when someone leaves a company) are handled through the DNS verification system and GitHub organization membership. + +## Future Considerations + +### Will download counts be tracked? + +Download count tracking is being considered as a quality signal, but must be designed carefully to not impact CDN caching. + +### What about tags or categories? + +Categorization and curation are intentionally left to consumers (MCP clients and aggregators) to avoid maintenance burden and subjective decisions. + +### Will there be quality metrics? + +Quality assessment is explicitly out of scope for the official registry. This is delegated to: + +- MCP clients (for their specific use cases) +- Third-party aggregators +- Community reviews on external platforms + +### What about internationalization? + +Internationalization is a future consideration but not part of the MVP. + +### Will private registries be supported? + +The registry design (API shapes, data formats) is intended to be reusable for private deployments, but the official registry will only host public servers. diff --git a/docs/roadmap.md b/docs/roadmap.md new file mode 100644 index 000000000..c3bcebda0 --- /dev/null +++ b/docs/roadmap.md @@ -0,0 +1,40 @@ +# MCP Registry Roadmap + +This is a high-level roadmap for the MCP Registry. It is subject to change and not exhaustive, but it outlines the general thinking of the sequencing and scope of our work in this repository. + +This roadmap may occasionally drift out of date. Please review [Issues](https://github.com/modelcontextprotocol/registry/issues) (and corresponding Labels) for the most current work in progress. + +## Current Status + +The initial version of the MCP Registry is actively being developed. The initial focus is on delivering a REST API to which server creators can publish, and aggregator/marketplace consumers can ETL. + +## Phase 1: MVP/Go-Live (Current Focus) + +- [ ] Specification: OpenAPI definition for the official REST API +- [ ] Specification: Extensible OpenAPI definition for marketplace/aggregator consumers to repurpose +- [ ] REST API: Publish workflow +- [ ] REST API: Read workflow +- [ ] CLI: Publish tool +- [ ] CLI/REST API: DNS verification system for namespacing + +## Backlog (Future Work) + +- [ ] UI implementation +- [ ] Store and surface other data besides servers (e.g. [clients](https://modelcontextprotocol.io/clients), resources) +- [ ] Additional IdP support (beyond GitHub) +- [ ] Download count tracking + +## Out of Scope (Not Planned) + +- **Source code hosting**: The registry will never host actual server code +- **Quality rankings**: No built-in server quality assessments or rankings +- **Curation**: No editorial decisions about which servers are "better" +- **Unified runtime**: Not solving how servers are executed +- **Server hosting**: The registry does not provide hosting for servers +- **Search engine**: The registry will not provide a commercial grade search engine for servers +- **Server tagging**: The registry will not support server tagging +- **Server rankings**: The registry will not rank servers by subjective measures of quality + +## Contributing + +See [CONTRIBUTING.md](../CONTRIBUTING.md) for details on how to contribute to the MCP Registry project.