Skip to content

Commit d460060

Browse files
authored
Update README.md (#4547)
1 parent 2dae477 commit d460060

1 file changed

Lines changed: 148 additions & 99 deletions

File tree

README.md

Lines changed: 148 additions & 99 deletions
Original file line numberDiff line numberDiff line change
@@ -9,160 +9,209 @@
99
[![Homebrew](https://img.shields.io/homebrew/v/buf)](https://github.com/bufbuild/homebrew-buf)
1010
[![Slack](https://img.shields.io/badge/slack-buf-%23e01563)][badges_slack]
1111

12-
<a name="features"></a>
13-
The [`buf`][buf] CLI is the best tool for working with [Protocol Buffers][protobuf]. It provides:
14-
15-
- A [linter][lint_usage] that enforces good API design choices and structure.
16-
- A [breaking change detector][breaking_tutorial] that enforces compatibility at the source code or wire level.
17-
- A [generator][generate_usage] that invokes your plugins based on [configuration files][templates].
18-
- A [formatter][format_usage] that formats your Protobuf files in accordance with industry standards.
19-
- Integration with the [Buf Schema Registry][bsr], including full dependency management.
12+
Buf is the modern toolchain for [Protobuf][protobuf]. It replaces day-to-day `protoc` use with a fast compiler, module-aware workspaces, formatting, linting, breaking-change detection, code generation, dependency management, API calls, and a client for the [Buf Schema Registry][bsr].
2013

21-
## Installation
14+
If you are still driving Protobuf with shell scripts around `protoc -I ...`, Buf is the upgrade you want: the same schema language, the same generated-code plugin model, fewer moving parts, and a direct path from local `.proto` files to governed, versioned APIs.
2215

23-
### Homebrew
16+
## Start
2417

25-
You can install `buf` using [Homebrew][brew] (macOS or Linux):
18+
Install `buf` with [Homebrew][brew]:
2619

2720
```sh
2821
brew install bufbuild/buf/buf
2922
```
3023

31-
This installs:
24+
Initialize a workspace and run the checks you should expect every Protobuf repository to pass:
25+
26+
```sh
27+
buf config init
28+
buf build
29+
buf format -w
30+
buf lint
31+
buf breaking --against '.git#branch=main'
32+
```
3233

33-
- The `buf`, [`protoc-gen-buf-breaking`][breaking], and [`protoc-gen-buf-lint`][lint] binaries
34-
- Shell completion scripts for [Bash], [Fish], [Powershell], and [zsh]
34+
Generate code from a checked-in `buf.gen.yaml` instead of a hand-maintained shell command:
3535

36-
### Other methods
36+
```sh
37+
buf generate
38+
```
3739

38-
For other installation methods, see our [official documentation][install], which covers:
40+
For a guided walkthrough from an empty workspace to a working Connect service, run the [Buf CLI quickstart][cli_quickstart].
3941

40-
- Installing `buf` via [npm]
41-
- Installing `buf` on [Windows]
42-
- Using `buf` as a [Docker image][docker]
43-
- Installing as a [binary], from a [tarball], and from [source] through [GitHub Releases][releases]
44-
- [Verifying] releases using a [minisign] public key
42+
## Why Buf wins
4543

44+
<a name="features"></a>
4645

47-
## Usage
46+
| Protobuf work | With `protoc` and scripts | With Buf |
47+
| --- | --- | --- |
48+
| Finding files | Maintain `-I` paths and hope import order does not change behavior. | Declare modules once in `buf.yaml`; Buf discovers files and rejects ambiguous imports. |
49+
| Compiling | Manage a local `protoc` install and parse changing stderr output. | Use Buf's internal compiler, tested against `protoc` descriptor output and built for deterministic parallel compilation. |
50+
| Style | Rely on review comments or separate tooling. | Run `buf lint` locally, in editors, in CI, and on the BSR with 40+ built-in rules plus custom plugins. |
51+
| Compatibility | Find breakage after generated code fails, clients fail, or serialized data becomes unreadable. | Run `buf breaking` against Git, a BSR module, a tarball, a zip file, or a Buf image before merge. |
52+
| Code generation | Keep plugin binaries installed on every machine and encode behavior in long commands. | Put plugins, outputs, options, inputs, and managed-mode settings in `buf.gen.yaml`; use local or remote plugins. |
53+
| Dependencies | Copy `.proto` files between repositories or vendor them by hand. | Declare BSR module dependencies in `buf.yaml` and pin them in `buf.lock`. |
54+
| API consumers | Send people your schemas and generation instructions. | Publish to the BSR and let consumers install generated SDKs with `go get`, `npm install`, Maven, Gradle, `pip install`, NuGet, Cargo, SwiftPM, CMake, or an archive. |
55+
| Governance | Reimplement checks in every repository and hope every team keeps them enabled. | Enforce breaking-change, uniqueness, and custom policies at the BSR layer. |
56+
57+
Core CLI features work without a BSR account. Signing in to the registry adds distribution, remote plugins, generated SDKs, hosted docs, dependency resolution for private modules, and server-side checks when you need them.
58+
59+
## Core workflow
60+
61+
Buf treats a directory tree of `.proto` files as a module, and a project as a workspace. A small `buf.yaml` is enough to make build, lint, breaking-change detection, generation, dependency resolution, and publishing agree on the same input.
62+
63+
```yaml
64+
version: v2
65+
modules:
66+
- path: proto
67+
lint:
68+
use:
69+
- STANDARD
70+
breaking:
71+
use:
72+
- FILE
73+
```
4874
49-
Buf's help interface provides summaries for commands and flags:
75+
From there, the useful commands are deliberately boring:
5076
5177
```sh
52-
buf --help
78+
buf build
79+
buf format -w
80+
buf lint
81+
buf breaking --against '.git#branch=main'
82+
buf generate
83+
buf push
5384
```
5485

55-
For more comprehensive usage information, consult Buf's [documentation][docs], especially these guides:
86+
`buf build` compiles the workspace. `buf lint` catches API-shape problems while the author is still editing. `buf breaking` compares the current schema against a previous version and flags source, JSON, or wire-format incompatibilities. `buf generate` runs `protoc` plugins from a checked-in template. `buf push` publishes named modules to the BSR.
87+
88+
## Code generation
89+
90+
`buf generate` is compatible with the normal `protoc` plugin model, but it moves generation into versioned configuration. This example generates Go Protobuf types and ConnectRPC handlers from `proto/` using remote plugins hosted on the BSR:
91+
92+
```yaml
93+
version: v2
94+
clean: true
95+
managed:
96+
enabled: true
97+
override:
98+
- file_option: go_package_prefix
99+
value: github.com/acme/weather/gen/go
100+
plugins:
101+
- remote: buf.build/protocolbuffers/go
102+
out: gen/go
103+
opt: paths=source_relative
104+
- remote: buf.build/connectrpc/gosimple
105+
out: gen/go
106+
opt:
107+
- paths=source_relative
108+
- simple
109+
inputs:
110+
- directory: proto
111+
```
56112
57-
* [`buf breaking`][breaking_tutorial]
58-
* [`buf build`][build_usage]
59-
* [`buf generate`][generate_usage]
60-
* [`buf lint`][lint_usage]
61-
* [`buf format`][format_usage]
62-
* [`buf registry`][bsr_quickstart] (for using the [BSR])
113+
Remote plugins remove the need to install and maintain generator binaries on every developer machine or CI runner. Managed mode lets API producers keep language-specific file options out of `.proto` files while consumers still get correct generated package names for their target language.
63114

64-
## CLI breaking change policy
115+
Local plugins work too. If a plugin speaks the standard Protobuf plugin protocol, Buf can run it.
65116

66-
We will never make breaking changes within a given major version of the CLI. After `buf` reached v1.0, you can expect no breaking changes until v2.0. But as we have no plans to ever release a v2.0, we will likely never break the `buf` CLI.
117+
## Breaking changes
67118

68-
> This breaking change policy does _not_ apply to commands behind the `buf beta` gate, and you should expect breaking changes to commands like `buf beta registry`. The policy does go into effect, however, when those commands or flags are elevated out of beta.
119+
Protobuf compatibility is not one thing. Renaming a field can break generated source code while preserving the binary wire format; changing a field from `int32` to `string` breaks every existing serialized message. `buf breaking` makes that distinction explicit with rule categories for `FILE`, `PACKAGE`, `WIRE_JSON`, and `WIRE` compatibility.
69120

70-
## Our goals for Protobuf
121+
```sh
122+
buf breaking --against '.git#branch=main'
123+
```
71124

72-
[Buf]'s goal is to replace the current paradigm of API development, centered around REST/JSON, with a **schema-driven** paradigm. Defining APIs using an [IDL] provides numerous benefits over REST/JSON, and [Protobuf] is by far the most stable and widely adopted IDL in the industry. We've chosen to build on this widely trusted foundation rather than creating a new IDL from scratch.
125+
`--against` accepts a Git branch, a BSR module, a tarball, a zip file, a local directory, or a prebuilt Buf image. That matters in real repositories: the same command works on a laptop, in CI, and in release automation.
73126

74-
But despite its technical merits, actually _using_ Protobuf has long been more challenging than it needs to be. The Buf CLI and the [BSR](#the-buf-schema-registry) are the cornerstones of our effort to change that for good and to make Protobuf reliable and easy to use for service owners and clients alike—in other words, to create a **modern Protobuf ecosystem**.
127+
## Buf Schema Registry
75128

76-
While we intend to incrementally improve on the `buf` CLI and the [BSR](#the-buf-schema-registry), we're confident that the basic groundwork for such an ecosystem is _already_ in place.
129+
<a name="the-buf-schema-registry"></a>
77130

78-
## The Buf Schema Registry
131+
[Buf Schema Registry][bsr] is a Protobuf-aware registry. It stores modules, verifies they compile, renders documentation, resolves dependencies, hosts remote plugins, produces generated SDKs, and can enforce schema checks before a breaking change reaches consumers.
79132

80-
The [Buf Schema Registry][bsr] (BSR) is a SaaS platform for managing your Protobuf APIs. It provides a centralized registry and a single source of truth for all of your Protobuf assets, including not just your `.proto` files but also [remote plugins][bsr_plugins]. Although the BSR provides an intuitive browser UI, `buf` enables you to perform most BSR-related tasks from the command line, such as [pushing] Protobuf sources to the registry and managing [users] and [repositories].
133+
```sh
134+
buf push
135+
```
81136

82-
> The BSR is not required to use `buf`. We've made the core [features] of the `buf` CLI available to _all_ Protobuf users.
137+
Pushing a module to the BSR gives your organization a source of truth for Protobuf APIs. Consumers can depend on the schema as a BSR module, install generated SDKs from their normal package manager, or use the BSR docs to inspect services, messages, fields, enums, references, and historical commits.
83138

84-
## More advanced CLI features
139+
## Related projects
85140

86-
While `buf`'s [core features][features] should cover most use cases, we've included some more advanced features to cover edge cases:
141+
Buf is most useful when schemas drive more than code generation. [ConnectRPC][connectrpc] uses Protobuf schemas to build simple HTTP APIs that support Connect, gRPC, and gRPC-Web without separate service definitions. [Protobuf-ES][protobuf_es] gives JavaScript and TypeScript users a modern Protobuf runtime and generator. [Protovalidate][protovalidate] puts validation rules in the schema and runs them consistently across languages.
87142

88-
* **Automatic file discovery**. Buf walks your file tree and builds your `.proto` files in accordance with your supplied [build configuration][build_config], which means that you no longer need to manually specify `--proto_paths`. You can still, however, specify `.proto` files manually through CLI flags in cases where file discovery needs to be disabled.
89-
* **Fine-grained rule configuration** for [linting][lint_rules] and [breaking changes][breaking_rules]. While we do have recommended defaults, you can always select the exact set of rules that your use case requires, with [40 lint rules][lint_rules] and [53 breaking change rules][breaking_rules] available.
90-
* **Configurable error formats** for CLI output. `buf` outputs information in `file:line:column:message` form by default for each lint error and breaking change it encounters, but you can also select JSON, MSVS, JUnit, and Github Actions output.
91-
* **Editor integration** driven by `buf`'s granular error output. We currently provide linting integrations for both [Vim and Visual Studio Code][ide] and [JetBrains IDEs][jetbrains] like IntelliJ and GoLand, but we plan to support other editors such as Emacs in the future.
92-
* **Universal Input targeting**. Buf enables you to perform actions like linting and breaking change detection not just against local `.proto` files but also against a broad range of other [Inputs], such as tarballs and ZIP files, remote Git repositories, and pre-built [image][images] files.
93-
* **Speed**. Buf's internal Protobuf [compiler] compiles your Protobuf sources using all available cores without compromising deterministic output, which is considerably faster than `protoc`. This allows for near-instantaneous feedback, which is of special importance for features like [editor integration][ide].
143+
One contract should drive the whole workflow: compile, lint, compatibility checks, generated clients and servers, validation, API calls, package publishing, and governed changes.
94144

95-
## Next steps
145+
## Installation
96146

97-
Once you've installed `buf`, we recommend completing the [CLI tutorial][cli-tutorial], which provides a broad but hands-on overview of the core functionality of the CLI. The tour takes about 10 minutes to complete.
147+
Homebrew installs the `buf`, [`protoc-gen-buf-breaking`][breaking_plugin], and [`protoc-gen-buf-lint`][lint_plugin] binaries, plus shell completion scripts for [Bash], [Fish], [PowerShell], and [zsh].
98148

99-
After completing the tour, check out the remainder of the [docs] for your specific areas of interest.
149+
```sh
150+
brew install bufbuild/buf/buf
151+
```
100152

101-
## Builds
153+
Other supported installation methods include [npm], [Windows], [Docker], [binary downloads], [tarballs], [source builds], and [minisign verification][verifying]. See the [installation docs][install] for the full list.
102154

103-
The following is a breakdown of the binaries by CPU architecture and operating system available through our [releases]:
155+
## CLI stability
104156

105-
| | Linux | MacOS | Windows | OpenBSD | FreeBSD |
106-
| --- | --- | --- | --- | --- | --- |
107-
| x86 (64-bit) ||||||
108-
| ARM (64-bit) ||||||
109-
| ARMv7 (32-bit) ||||||
110-
| RISC-V (64-bit) ||||||
111-
| ppc64le ||||||
112-
| s390x ||||||
157+
Buf CLI releases do not make breaking changes within a major version. Since `buf` reached v1.0, you can expect no breaking changes until v2.0. We have no plans to release v2.0.
113158

114-
## Community
159+
This policy does not apply to commands behind the `buf beta` gate. Expect breaking changes for beta commands until they are promoted.
160+
161+
## Documentation
115162

116-
For help and discussion around Protobuf, best practices, and more, join us on [Slack][badges_slack].
163+
- [Buf CLI][cli]
164+
- [CLI quickstart][cli_quickstart]
165+
- [Modules and workspaces][modules_workspaces]
166+
- [Code generation][generate]
167+
- [Linting][lint]
168+
- [Breaking-change detection][breaking]
169+
- [Formatting][format]
170+
- [Calling APIs with `buf curl`][curl]
171+
- [Buf Schema Registry][bsr]
172+
- [Generated SDKs][generated_sdks]
173+
- [Remote plugins][remote_plugins]
174+
- [Schema checks][schema_checks]
175+
- [Migrating from `protoc`][migrate_from_protoc]
176+
177+
## Community
117178

118-
For updates on the Buf CLI, [follow this repo on GitHub][repo].
179+
For help and discussion around Protobuf, best practices, and Buf, join us on [Slack][badges_slack].
119180

120-
For feature requests, bugs, or technical questions, email us at [dev@buf.build][email_dev]. For general inquiries or inclusion in our upcoming feature betas, email us at [info@buf.build][email_info].
181+
For bugs, feature requests, and technical questions, open an issue in this repository or email [dev@buf.build][email_dev]. For general inquiries, email [info@buf.build][email_info].
121182

122183
[badges_slack]: https://buf.build/links/slack
123184
[bash]: https://www.gnu.org/software/bash
124-
[binary]: https://buf.build/docs/cli/installation/#source
125-
[breaking]: https://buf.build/docs/breaking/overview/
126-
[breaking_rules]: https://buf.build/docs/breaking/rules/
127-
[breaking_tutorial]: https://buf.build/docs/breaking/tutorial/
185+
[binary downloads]: https://buf.build/docs/cli/installation/#github
186+
[breaking]: https://buf.build/docs/breaking/
187+
[breaking_plugin]: https://buf.build/docs/breaking/
128188
[brew]: https://brew.sh
129189
[bsr]: https://buf.build/docs/bsr/
130-
[bsr_plugins]: https://buf.build/plugins
131-
[bsr_quickstart]: https://buf.build/docs/bsr/quickstart/
132-
[buf]: https://buf.build
133-
[build_config]: https://buf.build/docs/build/usage/#key-concepts
134-
[build_usage]: https://buf.build/docs/build/usage
135-
[cli-tutorial]: https://buf.build/docs/cli/quickstart/
136-
[compiler]: https://buf.build/docs/reference/internal-compiler/
190+
[cli]: https://buf.build/docs/cli/
191+
[cli_quickstart]: https://buf.build/docs/cli/quickstart/
192+
[connectrpc]: https://connectrpc.com
193+
[curl]: https://buf.build/docs/curl/
137194
[docker]: https://buf.build/docs/cli/installation/#docker
138-
[docs]: https://buf.build/docs
139195
[email_dev]: mailto:dev@buf.build
140196
[email_info]: mailto:info@buf.build
141-
[features]: #features
142197
[fish]: https://fishshell.com
143-
[format_usage]: https://buf.build/docs/format/style/
144-
[generate_usage]: https://buf.build/docs/generate/tutorial/
145-
[ide]: https://buf.build/docs/cli/editor-integration/
146-
[idl]: https://en.wikipedia.org/wiki/Interface_description_language
147-
[images]: https://buf.build/docs/reference/images/
148-
[inputs]: https://buf.build/docs/reference/inputs/
198+
[format]: https://buf.build/docs/format/
199+
[generate]: https://buf.build/docs/generate/
200+
[generated_sdks]: https://buf.build/docs/bsr/generated-sdks/
149201
[install]: https://buf.build/docs/cli/installation/
150-
[jetbrains]: https://buf.build/docs/cli/editor-integration/#jetbrains-ides
151-
[lint]: https://buf.build/docs/lint/overview/
152-
[lint_rules]: https://buf.build/docs/lint/rules/
153-
[lint_usage]: https://buf.build/docs/lint/tutorial/
202+
[lint]: https://buf.build/docs/lint/
203+
[lint_plugin]: https://buf.build/docs/lint/
204+
[migrate_from_protoc]: https://buf.build/docs/migration-guides/migrate-from-protoc/
205+
[modules_workspaces]: https://buf.build/docs/cli/modules-workspaces/
154206
[npm]: https://buf.build/docs/cli/installation/#npm
155-
[minisign]: https://github.com/jedisct1/minisign
156-
[powershell]: https://docs.microsoft.com/en-us/powershell
207+
[powershell]: https://learn.microsoft.com/en-us/powershell/
157208
[protobuf]: https://protobuf.dev
158-
[pushing]: https://buf.build/docs/bsr/module/publish/
159-
[releases]: https://buf.build/docs/cli/installation/#github
160-
[repo]: https://github.com/bufbuild/buf/
161-
[repositories]: https://buf.build/docs/concepts/repositories/
162-
[source]: https://buf.build/docs/cli/installation/#source
163-
[tarball]: https://buf.build/docs/cli/installation/#github
164-
[templates]: https://buf.build/docs/configuration/v2/buf-gen-yaml/
165-
[users]: https://buf.build/docs/admin/manage-members/
209+
[protobuf_es]: https://github.com/bufbuild/protobuf-es
210+
[protovalidate]: https://protovalidate.com
211+
[remote_plugins]: https://buf.build/docs/bsr/remote-plugins/
212+
[schema_checks]: https://buf.build/docs/bsr/checks/
213+
[source builds]: https://buf.build/docs/cli/installation/#source
214+
[tarballs]: https://buf.build/docs/cli/installation/#github
166215
[verifying]: https://buf.build/docs/cli/installation/#github
167216
[windows]: https://buf.build/docs/cli/installation/#windows
168217
[zsh]: https://zsh.org

0 commit comments

Comments
 (0)