Support Protobuf 33.4 w/ prebuilt protoc toolchain (#1803)

Replaces the `rules_scala` prebuilt `protoc` toolchain everywhere except
in a few `test_dependency_versions.sh` test cases. Updates `README.md`
to recommend using the `protobuf` prebuilt `protoc` toolchain, and
refactors and adds test cases to `test_dependency_versions.sh`.

Note that legacy `WORKSPACE` builds continue to use the `rules_scala`
prebuilt `protoc` toolchain, since the `protobuf` implementation only
supports Bzlmod.

After removing `protobuf.patch` files, updating `MODULE.bazel{,.lock}`
files, and updating Maven dependencies, the most significant changes for
supporting `protobuf` v33.4 are:

- `protoc/BUILD`, `protoc/private/protoc_toolchains.bzl`:
  Moves `proto_lang_toolchain` from `@rules_scala_protoc_toolchains` to
  `//protoc:scala_protoc_toolchain`. This can break users using
  `rules_proto` < 7.0.0 and `protobuf` < v28.2, but `README.md` contains
  a patch for such users.

- `protoc/private/toolchain_impl.bzl`:
  Applies changes from the final state of protocolbuffers/protobuf#19679
  and updates the prebuilt `protoc` toolchain resolution check and error
  message.

- `test_dependency_versions.sh`, `deps/test/MODULE.bazel.template`,
  `deps/test/protoc_toolchains.template`: Adds Bazel 9 and `protobuf`
  prebuilt `protoc` toolchain test cases. Updates `do_build_and_test` to
  support both toolchains. Extracts the `protoc_toolchains.template` to
  include it only in test cases using the `rules_scala` toolchain.

Other key components of this change include:

- `protoc/BUILD`:
  Aliases `@protobuf//bazel/toolchains:prefer_prebuilt_protoc` as
  `@rules_scala//protoc:prefer_prebuilt_protoc`.

- `.bazelrc`, `tools/bazel.rc.buildkite`:
  Adds `common --@rules_scala//protoc:prefer_prebuilt_protoc`.

- `.bazelci/presubmit.yml`, `.bcr/presubmit.yml`:
  Adds `9.x` to the `bcr_bazel` matrix . Also applies whitespace and
  quoting format changes to `.bazelci/presubmit.yml`.

Also contains these small, but important changes to fix continuous
integration breakages and `last_green` Bazel compatibility:

- `.github/workflows/release_prep.sh`:
  Excludes `.bazelversion` symlinks, since per
  bazelbuild/bazel-worker-api#21, these break Bazel as of
  bazelbuild/bazel@f942a70.

- `MODULE.bazel`, `scala/private/macros/workspace_compat.bzl`:
  Bumps `bazelbuild/bazel-worker-api` to v0.0.10 to fix `last_green`
  builds per bazelbuild/bazel-worker-api#22.

- `test/shell/test_runner.sh`, `test/shell/test_bzlmod_macros.sh`:
  Replaces `relative_path_to_parent` with `convert_msys2_path` for
  bazelbuild/continuous-integration#2445, bazelbuild/bazel#28406.
  It turns out @WojciechMazur patched this in #1794, but this change
  supersedes the change from that pull request.

Other minor changes include:

- `MODULE.bazel`:
  Removes the `version` parameter from the `module` declaration. The
  Publish to BCR GitHub Actions workflow patches this parameter anyway.
  See:
  - https://bazel.build/external/faq#module-versioning-best-practices
  - https://github.com/bazel-contrib/publish-to-bcr/blob/v1.1.0/src/domain/create-entry.ts#L176-L216
  - https://bazelbuild.slack.com/archives/C014RARENH0/p1767892286188219

- Replaces `precompiled` with `prebuilt` in every reference to the
  prebuilt `protoc` toolchain.

- `scala/toolchains_repo.bzl`:
  Replaces `@@{rules_scala_repo}` references with plain `@rules_scala`
  references, allowing repo mapping to do its thing.

- `test_dependency_version.sh`:
  Skips non-prebuilt `protoc` prebuilt toolchain test cases from for
  macOS on continuous integration. This test suite was taking a
  disproportionately long time to run before adding the new test cases,
  which Linux builds will continue to cover.

A summary of these changes excluding `README.md`, `protobuf.patch`,
`MODULE.bazel{,.lock}` files, and Maven dependency updates:

```txt
$ git diff --stat HEAD^ ':!README.md' ':!**MODULE.bazel*' \
  ':!**protobuf.patch' ':!third_party/repositories/scala_*'

.bazelci/presubmit.yml                                   |  66 ++++++...
.bazelrc                                                 |   9 ++++--
.bcr/presubmit.yml                                       |   2 +-
.github/workflows/release_prep.sh                        |   7 ++++-
WORKSPACE                                                |  15 ++++++...
deps/test/protoc_toolchains.template                     |  12 ++++++++
protoc/BUILD                                             |  20 ++++++...
protoc/private/protoc_integrity.bzl                      |  10 +++++++
protoc/private/protoc_toolchains.bzl                     |  19 +++---...
protoc/private/toolchain_impl.bzl                        |  30 ++++++...
protoc/toolchains.bzl                                    |   2 +-
scala/extensions/protoc.bzl                              |   8 +++---
scala/latest_deps.bzl                                    |   8 ++----
scala/private/macros/test/MODULE.bzlmod_test             |   8 +++++-
scala/private/macros/test/MODULE.bzlmod_test_root_module |   8 +++++-
scala/private/macros/workspace_compat.bzl                |   6 ++--
scala/toolchains.bzl                                     |   5 +++-
scala/toolchains_repo.bzl                                |  26 ++++++...
scala_proto/scala_proto_toolchain.bzl                    |   2 ++
scripts/create_repository.py                             |   2 +-
scripts/update_protoc_integrity.py                       |   6 ++--
test/shell/test_bzlmod_macros.sh                         |  23 ++++--...
test/shell/test_runner.sh                                |  34 ++++++...
test_dependency_versions.sh                              | 151 ++++++...
test_version.sh                                          |   2 --
tools/bazel.rc.buildkite                                 |   9 ++++--
26 files changed, 317 insertions(+), 173 deletions(-)
```

---

In the following Bazel Slack thread, @rafikk pointed out the apparent
incompatibility with the new Bazel 9.0.0 release:

- https://bazelbuild.slack.com/archives/CDCKJ2KFZ/p1768383285232129
- https://github.com/bazelbuild/bazel/releases/tag/9.0.0

This happened because Bazel 9 now sets
`--incompatible_enable_proto_toolchain_resolution` by default:

- https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution

Since the `scala_proto` implementation depends on this flag, this broke
projects that didn't configure the `rules_scala` prebuilt `protoc`
toolchain. This change fixes that problem while ensuring seamless
compatibility with the `protobuf` v33.4 prebuilt protoc toolchain.

---

Also, Bazel 10.0.0-pre.20251208.3 selects the prebuilt `protoc`
toolchain, and 10.0.0-pre.20251217.3 doesn't. See:

- protocolbuffers/protobuf#25561

Author: mbland

.bazelci/presubmit.yml

@@ -3,62 +3,62 @@ validate_config: 1
 
 matrix:
   bcr_platform: ["debian10", "macos", "ubuntu2004", "windows"]
-  bcr_bazel: [7.x, 8.x, rolling, last_green]
+  bcr_bazel: [7.x, 8.x, 9.x, rolling, last_green]
 
 tasks:
   ubuntu2004:
     name: "bazel test //test/..."
     platform: ubuntu2004
     shell_commands:
-    - mv tools/bazel.rc.buildkite tools/bazel.rc
-    - echo "import %workspace%/tools/bazel.rc" > .bazelrc
+      - mv tools/bazel.rc.buildkite tools/bazel.rc
+      - echo "import %workspace%/tools/bazel.rc" > .bazelrc
     build_targets:
-    - "//test/..."
+      - "//test/..."
     test_targets:
-    - "//test/..."
+      - "//test/..."
   macos:
     name: "bazel test //test/..."
     platform: macos
     shell_commands:
-    # Disable local disk caching on CI.
-    - mv tools/bazel.rc.buildkite tools/bazel.rc
-    - echo "import %workspace%/tools/bazel.rc" > .bazelrc
+      # Disable local disk caching on CI.
+      - mv tools/bazel.rc.buildkite tools/bazel.rc
+      - echo "import %workspace%/tools/bazel.rc" > .bazelrc
     build_targets:
-    - "//test/..."
+      - "//test/..."
     test_targets:
-    - "//test/..."
+      - "//test/..."
   test_rules_scala_linux:
     name: "./test_rules_scala"
     platform: ubuntu2004
     shell_commands:
-    # Install xmllint
-    - sudo apt update && sudo apt install --reinstall libxml2-utils -y
-    - "./test_rules_scala.sh"
+      # Install xmllint
+      - sudo apt update && sudo apt install --reinstall libxml2-utils -y
+      - "./test_rules_scala.sh"
   test_rules_scala_linux_last_green:
     name: "./test_rules_scala (last_green Bazel)"
     platform: ubuntu2004
     bazel: last_green
     shell_commands:
-    # Install xmllint
-    - sudo apt update && sudo apt install --reinstall libxml2-utils -y
-    - echo "common --lockfile_mode=update" >>.bazelrc
-    - "./test_rules_scala.sh"
+      # Install xmllint
+      - sudo apt update && sudo apt install --reinstall libxml2-utils -y
+      - echo "common --lockfile_mode=update" >>.bazelrc
+      - "./test_rules_scala.sh"
     soft_fail:
       - exit_status: "*"
   test_rules_scala_macos:
     name: "./test_rules_scala"
     platform: macos
     shell_commands:
-    - "./test_rules_scala.sh"
+      - "./test_rules_scala.sh"
   test_rules_scala_win:
     name: "./test_rules_scala"
     platform: windows
     environment:
       MSYS2_ARG_CONV_EXCL: "*"
     batch_commands:
-    - "set PATH=/usr/bin;%PATH%" #Make sure bash uses msys commands over windows commands. (i.e. find).
-    - "bash -lc \"pacman --noconfirm --needed -S libxml2\"" #tests require xmllint
-    - "bash test_rules_scala.sh"
+      - "set PATH=/usr/bin;%PATH%" #Make sure bash uses msys commands over windows commands. (i.e. find).
+      - 'bash -lc "pacman --noconfirm --needed -S libxml2"' #tests require xmllint
+      - "bash test_rules_scala.sh"
   test_coverage_linux:
     name: "./test_coverage"
     platform: ubuntu2004
@@ -73,27 +73,27 @@ tasks:
     name: "./test_reproducibility.sh"
     platform: ubuntu1804
     shell_commands:
-    - "./test_reproducibility.sh"
+      - "./test_reproducibility.sh"
   test_reproducibility_macos:
     name: "./test_reproducibility.sh"
     platform: macos
     shell_commands:
-    - "./test_reproducibility.sh"
+      - "./test_reproducibility.sh"
   versions_linux:
     name: "./test_version.sh"
     platform: ubuntu2004
     shell_commands:
-    - "./test_version.sh"
+      - "./test_version.sh"
   versions_macos:
     name: "./test_version.sh"
     platform: macos
     shell_commands:
-    - "./test_version.sh"
+      - "./test_version.sh"
   thirdparty_version_linux:
     name: "./test_thirdparty_version.sh"
     platform: ubuntu2204_java17
     shell_commands:
-    - "./test_thirdparty_version.sh"
+      - "./test_thirdparty_version.sh"
   examples_linux:
     name: "./test_examples"
     platform: ubuntu2004
@@ -108,7 +108,7 @@ tasks:
     name: "./test_lint.sh"
     platform: ubuntu2004
     shell_commands:
-    -  "./test_lint.sh"
+      - "./test_lint.sh"
   test_rules_scala_jdk21:
     name: "./test_rules_scala with jdk21"
     platform: ubuntu2004
@@ -125,26 +125,26 @@ tasks:
     name: "dt_patches/dt_patch_test"
     platform: ubuntu2004
     shell_commands:
-    - "dt_patches/dt_patch_test.sh"
+      - "dt_patches/dt_patch_test.sh"
   dependency_versions_linux:
     name: "./test_dependency_versions"
     platform: ubuntu2004
     shell_commands:
-    - "./test_dependency_versions.sh"
+      - "./test_dependency_versions.sh"
   dependency_versions_macos:
     name: "./test_dependency_versions"
     platform: macos
     shell_commands:
-    - "./test_dependency_versions.sh"
+      - "./test_dependency_versions.sh"
   dependency_versions_windows:
     name: "./test_dependency_versions"
     platform: windows
     environment:
       MSYS2_ARG_CONV_EXCL: "*"
     batch_commands:
-    - "set PATH=/usr/bin;%PATH%" #Make sure bash uses msys commands over windows commands. (i.e. find).
-    - "bash -lc \"pacman --noconfirm --needed -S libxml2\"" #tests require xmllint
-    - "bash ./test_dependency_versions.sh" # script removes ./ from BASH_SOURCE
+      - "set PATH=/usr/bin;%PATH%" #Make sure bash uses msys commands over windows commands. (i.e. find).
+      - 'bash -lc "pacman --noconfirm --needed -S libxml2"' #tests require xmllint
+      - "bash ./test_dependency_versions.sh" # script removes ./ from BASH_SOURCE
   bcr_presubmit:
     # Keep in sync with .bcr/presubmit.yml.
     name: "BCR {bcr_bazel}"

.bazelrc

@@ -15,11 +15,16 @@ common --lockfile_mode=error
 # https://github.com/bazelbuild/rules_java/releases/tag/8.12.0
 #common --repositories_without_autoloads=bazel_features_version,bazel_features_globals
 
-# Remove if protocol compiler toolchainization ever becomes the default.
+# Remove once Bazel 9 becomes the default.
 # - https://bazel.build/reference/command-line-reference#flag--incompatible_enable_proto_toolchain_resolution
-# - https://docs.google.com/document/d/1CE6wJHNfKbUPBr7-mmk_0Yo3a4TaqcTPE0OWNuQkhPs/edit
+# - https://github.com/bazelbuild/bazel/releases/tag/9.0.0
+# - https://blog.bazel.build/2026/01/20/bazel-9.html#prebuilt-protobuf-compiler
 common --incompatible_enable_proto_toolchain_resolution
 
+# Remove once protobuf 34 becomes the default.
+# - https://protobuf.dev/news/2026-01-16/
+common --@rules_scala//protoc:prefer_prebuilt_protoc
+
 build --enable_platform_specific_config
 
 # Use Java 17+ required for Scala 3.8+

.bcr/presubmit.yml

@@ -3,7 +3,7 @@ bcr_test_module:
   module_path: "examples/crossbuild"
   matrix:
     platform: ["debian10", "macos", "ubuntu2004", "windows"]
-    bazel: [7.x, 8.x, rolling, last_green]
+    bazel: [7.x, 8.x, 9.x, rolling, last_green]
   tasks:
     run_tests:
       name: "Build and test the example module"

.github/workflows/release_prep.sh

@@ -7,7 +7,12 @@ TAG="$1"
 VERSION="${TAG#v}"
 PREFIX="rules_scala-${VERSION}"
 ARCHIVE="rules_scala-$TAG.tar.gz"
-git archive --format=tar --prefix=${PREFIX}/ ${TAG} | gzip > $ARCHIVE
+
+# Don't archive .bazelversion symlinks, as they break Bazel as of
+# bazelbuild/bazel@f942a706a39f9803e5611b1973172ece361197ac.
+# See: https://github.com/bazelbuild/bazel-worker-api/issues/21
+git archive --format=tar --prefix=${PREFIX}/ ${TAG} ':!:**.bazelversion' |
+  gzip > $ARCHIVE
 SHA=$(shasum -a 256 $ARCHIVE | awk '{print $1}')
 
 cat << EOF

MODULE.bazel

@@ -2,7 +2,6 @@
 
 module(
     name = "rules_scala",
-    version = "7.1.6",
     bazel_compatibility = [">=7.1.0"],
     compatibility_level = 7,
 )
@@ -46,33 +45,22 @@ single_version_override(
     version = "9.3.0",
 )
 
-# TODO: remove this dep, the repo is deprecated and archived.
-# All public API symbols are aliased to @protobuf
+# TODO: Remove this dep once protobuf 33.4 becomes the minimum supported
+# version. The repo is deprecated and archived and all public API symbols are
+# aliased to @protobuf.
 bazel_dep(name = "rules_proto", version = "7.1.0")
 bazel_dep(
     name = "protobuf",
     version = "28.2",
     repo_name = "com_google_protobuf",
 )
-
-# Temporarily required for `protoc` toolchainization until
-# protocolbuffers/protobuf#19679 appears in a published release.
 single_version_override(
     module_name = "protobuf",
-    patch_strip = 1,
-    patches = ["//protoc:0001-protobuf-19679-rm-protoc-dep.patch"],
-    version = "33.1",
+    version = "33.4",
 )
 
 # See //src/java/io/bazel/rulesscala/worker:worker_protocol_java_proto.
-bazel_dep(name = "bazel_worker_api", version = "0.0.6")
-
-scala_protoc = use_extension(
-    "//scala/extensions:protoc.bzl",
-    "scala_protoc",
-    dev_dependency = True,
-)
-use_repo(scala_protoc, "rules_scala_protoc_toolchains")
+bazel_dep(name = "bazel_worker_api", version = "0.0.10")
 
 scala_config = use_extension(
     "//scala/extensions:config.bzl",
@@ -100,18 +88,16 @@ use_repo(
 
 # Register some of our testing toolchains first when building our repo.
 register_toolchains(
-    # This is optional, but still safe to include even when not using
-    # `--incompatible_enable_proto_toolchain_resolution`. Requires invoking the
-    # `scala_protoc_toolchains` repo rule. Register this toolchain before any
-    # others.
-    "@rules_scala_protoc_toolchains//...:all",
     "//scala:unused_dependency_checker_error_toolchain",
     "//test/proto:scalapb_toolchain",
     "//test/toolchains:java21_toolchain_definition",
     dev_dependency = True,
 )
 
-register_toolchains("@rules_scala_toolchains//...:all")
+register_toolchains(
+    "@rules_scala_toolchains//...:all",
+    "//protoc:all",  # //protoc:scala_protoc_toolchain creates a cycle
+)
 
 # Dev dependencies