[CI] Migrate to runtimes build

[boomanaiden154]

This patch migrates the premerge pipeline to use LLVM_ENABLE_RUNTIMES to
build libc and compiler-rt.

[llvmbot]

@llvm/pr-subscribers-github-workflow

Author: Aiden Grossman (boomanaiden154)

Changes

This patch migrates the premerge pipeline to use LLVM_ENABLE_RUNTIMES to
build libc and compiler-rt.

---

Full diff: https://github.com/llvm/llvm-project/pull/142696.diff

4 Files Affected:

• (modified) .ci/compute_projects.py (+80-41)
• (modified) .ci/compute_projects_test.py (+39-4)
• (modified) .ci/monolithic-linux.sh (+10-3)
• (modified) .github/workflows/premerge.yaml (+2-1)

diff --git a/.ci/compute_projects.py b/.ci/compute_projects.py
index 40dd0507a9eaf..b12b729eadd3f 100644
--- a/.ci/compute_projects.py
+++ b/.ci/compute_projects.py
@@ -49,8 +49,7 @@
     },
     "lld": {"bolt", "cross-project-tests"},
     # TODO(issues/132795): LLDB should be enabled on clang changes.
-    "clang": {"clang-tools-extra", "compiler-rt", "cross-project-tests"},
-    "clang-tools-extra": {"libc"},
+    "clang": {"clang-tools-extra", "cross-project-tests"},
     "mlir": {"flang"},
     # Test everything if ci scripts are changed.
     # FIXME: Figure out what is missing and add here.
@@ -64,7 +63,14 @@
 
 # This mapping describes runtimes that should be tested when the key project is
 # touched.
-DEPENDENT_RUNTIMES_TO_TEST = {"clang": {"libcxx", "libcxxabi", "libunwind"}}
+DEPENDENT_RUNTIMES_TO_TEST = {
+    "clang": {"compiler-rt"},
+}
+DEPENDENT_RUNTIMES_TO_TEST_MULTICONFIG = {
+    "llvm": {"libcxx", "libcxxabi", "libunwind"},
+    "clang": {"libcxx", "libcxxabi", "libunwind"},
+    ".ci": {"libcxx", "libcxxabi", "libunwind"},
+}
 
 EXCLUDE_LINUX = {
     "cross-project-tests",  # TODO(issues/132796): Tests are failing.
@@ -93,9 +99,6 @@
     "cross-project-tests",
     "flang",
     "libc",
-    "libcxx",
-    "libcxxabi",
-    "libunwind",
     "lldb",
     "openmp",
     "polly",
@@ -122,10 +125,10 @@
     "polly": "check-polly",
 }
 
-RUNTIMES = {"libcxx", "libcxxabi", "libunwind"}
+RUNTIMES = {"libcxx", "libcxxabi", "libunwind", "compiler-rt", "libc"}
 
 
-def _add_dependencies(projects: Set[str]) -> Set[str]:
+def _add_dependencies(projects: Set[str], runtimes: Set[str]) -> Set[str]:
     projects_with_dependents = set(projects)
     current_projects_count = 0
     while current_projects_count != len(projects_with_dependents):
@@ -134,9 +137,32 @@ def _add_dependencies(projects: Set[str]) -> Set[str]:
             if project not in PROJECT_DEPENDENCIES:
                 continue
             projects_with_dependents.update(PROJECT_DEPENDENCIES[project])
+    for runtime in runtimes:
+        if runtime not in PROJECT_DEPENDENCIES:
+            continue
+        projects_with_dependents.update(PROJECT_DEPENDENCIES[runtime])
     return projects_with_dependents
 
 
+def _exclude_projects(current_projects: Set[str], platform: str) -> Set[str]:
+    new_project_set = set(current_projects)
+    if platform == "Linux":
+        for to_exclude in EXCLUDE_LINUX:
+            if to_exclude in new_project_set:
+                new_project_set.remove(to_exclude)
+    elif platform == "Windows":
+        for to_exclude in EXCLUDE_WINDOWS:
+            if to_exclude in new_project_set:
+                new_project_set.remove(to_exclude)
+    elif platform == "Darwin":
+        for to_exclude in EXCLUDE_MAC:
+            if to_exclude in new_project_set:
+                new_project_set.remove(to_exclude)
+    else:
+        raise ValueError("Unexpected platform.")
+    return new_project_set
+
+
 def _compute_projects_to_test(modified_projects: Set[str], platform: str) -> Set[str]:
     projects_to_test = set()
     for modified_project in modified_projects:
@@ -154,25 +180,14 @@ def _compute_projects_to_test(modified_projects: Set[str], platform: str) -> Set
             ):
                 continue
             projects_to_test.add(dependent_project)
-    if platform == "Linux":
-        for to_exclude in EXCLUDE_LINUX:
-            if to_exclude in projects_to_test:
-                projects_to_test.remove(to_exclude)
-    elif platform == "Windows":
-        for to_exclude in EXCLUDE_WINDOWS:
-            if to_exclude in projects_to_test:
-                projects_to_test.remove(to_exclude)
-    elif platform == "Darwin":
-        for to_exclude in EXCLUDE_MAC:
-            if to_exclude in projects_to_test:
-                projects_to_test.remove(to_exclude)
-    else:
-        raise ValueError("Unexpected platform.")
+    projects_to_test = _exclude_projects(projects_to_test, platform)
     return projects_to_test
 
 
-def _compute_projects_to_build(projects_to_test: Set[str]) -> Set[str]:
-    return _add_dependencies(projects_to_test)
+def _compute_projects_to_build(
+    projects_to_test: Set[str], runtimes: Set[str]
+) -> Set[str]:
+    return _add_dependencies(projects_to_test, runtimes)
 
 
 def _compute_project_check_targets(projects_to_test: Set[str]) -> Set[str]:
@@ -184,24 +199,36 @@ def _compute_project_check_targets(projects_to_test: Set[str]) -> Set[str]:
     return check_targets
 
 
-def _compute_runtimes_to_test(projects_to_test: Set[str]) -> Set[str]:
+def _compute_runtimes_to_test(modified_projects: Set[str], platform: str) -> Set[str]:
     runtimes_to_test = set()
-    for project_to_test in projects_to_test:
-        if project_to_test in DEPENDENT_RUNTIMES_TO_TEST:
-            runtimes_to_test.update(DEPENDENT_RUNTIMES_TO_TEST[project_to_test])
-        if project_to_test in DEPENDENT_RUNTIMES_TO_BUILD:
-            runtimes_to_test.update(DEPENDENT_RUNTIMES_TO_BUILD[project_to_test])
-    return runtimes_to_test
+    for modified_project in modified_projects:
+        if modified_project not in DEPENDENT_RUNTIMES_TO_TEST:
+            continue
+        runtimes_to_test.update(DEPENDENT_RUNTIMES_TO_TEST[modified_project])
+    return _exclude_projects(runtimes_to_test, platform)
 
 
-def _compute_runtime_check_targets(projects_to_test: Set[str]) -> Set[str]:
-    check_targets = set()
-    for project_to_test in projects_to_test:
-        if project_to_test not in DEPENDENT_RUNTIMES_TO_TEST:
+def _compute_runtimes_to_test_multiconfig(
+    modified_projects: Set[str], platform: str
+) -> Set[str]:
+    runtimes_to_test = set()
+    for modified_project in modified_projects:
+        if modified_project not in DEPENDENT_RUNTIMES_TO_TEST_MULTICONFIG:
             continue
-        for runtime_to_test in DEPENDENT_RUNTIMES_TO_TEST[project_to_test]:
-            check_targets.add(PROJECT_CHECK_TARGETS[runtime_to_test])
-    return check_targets
+        runtimes_to_test.update(
+            DEPENDENT_RUNTIMES_TO_TEST_MULTICONFIG[modified_project]
+        )
+    return _exclude_projects(runtimes_to_test, platform)
+
+
+def _compute_runtimes_to_build(
+    runtimes_to_test: Set[str], modified_projects: Set[str], platform: str
+) -> Set[str]:
+    runtimes_to_build = set(runtimes_to_test)
+    for modified_project in modified_projects:
+        if modified_project in DEPENDENT_RUNTIMES_TO_BUILD:
+            runtimes_to_build.update(DEPENDENT_RUNTIMES_TO_BUILD[modified_project])
+    return _exclude_projects(runtimes_to_build, platform)
 
 
 def _get_modified_projects(modified_files: list[str]) -> Set[str]:
@@ -225,10 +252,19 @@ def _get_modified_projects(modified_files: list[str]) -> Set[str]:
 def get_env_variables(modified_files: list[str], platform: str) -> Set[str]:
     modified_projects = _get_modified_projects(modified_files)
     projects_to_test = _compute_projects_to_test(modified_projects, platform)
-    projects_to_build = _compute_projects_to_build(projects_to_test)
+    runtimes_to_test = _compute_runtimes_to_test(modified_projects, platform)
+    runtimes_to_test_multiconfig = _compute_runtimes_to_test_multiconfig(
+        modified_projects, platform
+    )
+    runtimes_to_build = _compute_runtimes_to_build(
+        runtimes_to_test | runtimes_to_test_multiconfig, modified_projects, platform
+    )
+    projects_to_build = _compute_projects_to_build(projects_to_test, runtimes_to_build)
     projects_check_targets = _compute_project_check_targets(projects_to_test)
-    runtimes_to_build = _compute_runtimes_to_test(projects_to_test)
-    runtimes_check_targets = _compute_runtime_check_targets(projects_to_test)
+    runtimes_check_targets = _compute_project_check_targets(runtimes_to_test)
+    runtimes_check_targets_multiconfig = _compute_project_check_targets(
+        runtimes_to_test_multiconfig
+    )
     # We use a semicolon to separate the projects/runtimes as they get passed
     # to the CMake invocation and thus we need to use the CMake list separator
     # (;). We use spaces to separate the check targets as they end up getting
@@ -238,6 +274,9 @@ def get_env_variables(modified_files: list[str], platform: str) -> Set[str]:
         "project_check_targets": " ".join(sorted(projects_check_targets)),
         "runtimes_to_build": ";".join(sorted(runtimes_to_build)),
         "runtimes_check_targets": " ".join(sorted(runtimes_check_targets)),
+        "runtimes_check_targets_multiconfig": " ".join(
+            sorted(runtimes_check_targets_multiconfig)
+        ),
     }
 
 
diff --git a/.ci/compute_projects_test.py b/.ci/compute_projects_test.py
index ae376ea6a43cd..dc57a1a1f3fbc 100644
--- a/.ci/compute_projects_test.py
+++ b/.ci/compute_projects_test.py
@@ -26,6 +26,10 @@ def test_llvm(self):
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -46,6 +50,10 @@ def test_llvm_windows(self):
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -66,6 +74,10 @@ def test_llvm_mac(self):
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -75,17 +87,21 @@ def test_clang(self):
         )
         self.assertEqual(
             env_variables["projects_to_build"],
-            "clang;clang-tools-extra;compiler-rt;lld;llvm",
+            "clang;clang-tools-extra;lld;llvm",
         )
         self.assertEqual(
             env_variables["project_check_targets"],
-            "check-clang check-clang-tools check-compiler-rt",
+            "check-clang check-clang-tools",
         )
         self.assertEqual(
-            env_variables["runtimes_to_build"], "libcxx;libcxxabi;libunwind"
+            env_variables["runtimes_to_build"], "compiler-rt;libcxx;libcxxabi;libunwind"
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "check-compiler-rt",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -104,6 +120,10 @@ def test_clang_windows(self):
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -115,6 +135,7 @@ def test_bolt(self):
         self.assertEqual(env_variables["project_check_targets"], "check-bolt")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_lldb(self):
         env_variables = compute_projects.get_env_variables(
@@ -124,6 +145,7 @@ def test_lldb(self):
         self.assertEqual(env_variables["project_check_targets"], "check-lldb")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_mlir(self):
         env_variables = compute_projects.get_env_variables(
@@ -135,6 +157,7 @@ def test_mlir(self):
         )
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_flang(self):
         env_variables = compute_projects.get_env_variables(
@@ -144,6 +167,7 @@ def test_flang(self):
         self.assertEqual(env_variables["project_check_targets"], "check-flang")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_invalid_subproject(self):
         env_variables = compute_projects.get_env_variables(
@@ -153,6 +177,7 @@ def test_invalid_subproject(self):
         self.assertEqual(env_variables["project_check_targets"], "")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_top_level_file(self):
         env_variables = compute_projects.get_env_variables(["README.md"], "Linux")
@@ -160,6 +185,7 @@ def test_top_level_file(self):
         self.assertEqual(env_variables["project_check_targets"], "")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_exclude_runtiems_in_projects(self):
         env_variables = compute_projects.get_env_variables(
@@ -169,6 +195,7 @@ def test_exclude_runtiems_in_projects(self):
         self.assertEqual(env_variables["project_check_targets"], "")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_exclude_docs(self):
         env_variables = compute_projects.get_env_variables(
@@ -178,6 +205,7 @@ def test_exclude_docs(self):
         self.assertEqual(env_variables["project_check_targets"], "")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_exclude_gn(self):
         env_variables = compute_projects.get_env_variables(
@@ -187,6 +215,7 @@ def test_exclude_gn(self):
         self.assertEqual(env_variables["project_check_targets"], "")
         self.assertEqual(env_variables["runtimes_to_build"], "")
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
     def test_ci(self):
         env_variables = compute_projects.get_env_variables(
@@ -198,10 +227,15 @@ def test_ci(self):
             "check-clang check-lld check-lldb check-llvm",
         )
         self.assertEqual(
-            env_variables["runtimes_to_build"], "libcxx;libcxxabi;libunwind"
+            env_variables["runtimes_to_build"],
+            "libcxx;libcxxabi;libunwind",
         )
         self.assertEqual(
             env_variables["runtimes_check_targets"],
+            "",
+        )
+        self.assertEqual(
+            env_variables["runtimes_check_targets_multiconfig"],
             "check-cxx check-cxxabi check-unwind",
         )
 
@@ -215,6 +249,7 @@ def test_lldb(self):
             env_variables["runtimes_to_build"], "libcxx;libcxxabi;libunwind"
         )
         self.assertEqual(env_variables["runtimes_check_targets"], "")
+        self.assertEqual(env_variables["runtimes_check_targets_multiconfig"], "")
 
 
 if __name__ == "__main__":
diff --git a/.ci/monolithic-linux.sh b/.ci/monolithic-linux.sh
index 7503ea4e6a992..aa558017ae76a 100755
--- a/.ci/monolithic-linux.sh
+++ b/.ci/monolithic-linux.sh
@@ -57,6 +57,7 @@ projects="${1}"
 targets="${2}"
 runtimes="${3}"
 runtime_targets="${4}"
+runtime_targets_multiconfig="${5}"
 
 lit_args="-v --xunit-xml-output ${BUILD_DIR}/test-results.xml --use-unique-output-file-name --timeout=1200 --time-tests"
 
@@ -93,9 +94,15 @@ echo "--- ninja"
 # Targets are not escaped as they are passed as separate arguments.
 ninja -C "${BUILD_DIR}" -k 0 ${targets}
 
+if [[ "${runtime_targets}" != "" ]]; then
+  echo "--- ninja runtimes"
+
+  ninja -C "${BUILD_DIR}" ${runtime_targets}
+fi
+
 # Compiling runtimes with just-built Clang and running their tests
 # as an additional testing for Clang.
-if [[ "${runtimes_targets}" != "" ]]; then
+if [[ "${runtime_targets_multiconfig}" != "" ]]; then
   echo "--- cmake runtimes C++26"
 
   cmake \
@@ -105,7 +112,7 @@ if [[ "${runtimes_targets}" != "" ]]; then
 
   echo "--- ninja runtimes C++26"
 
-  ninja -C "${BUILD_DIR}" ${runtime_targets}
+  ninja -C "${BUILD_DIR}" ${runtime_targets_multiconfig}
 
   echo "--- cmake runtimes clang modules"
 
@@ -116,5 +123,5 @@ if [[ "${runtimes_targets}" != "" ]]; then
 
   echo "--- ninja runtimes clang modules"
 
-  ninja -C "${BUILD_DIR}" ${runtime_targets}
+  ninja -C "${BUILD_DIR}" ${runtime_targets_multiconfig}
 fi
diff --git a/.github/workflows/premerge.yaml b/.github/workflows/premerge.yaml
index 709b6d03d94c3..8bf8d35abf0cf 100644
--- a/.github/workflows/premerge.yaml
+++ b/.github/workflows/premerge.yaml
@@ -56,11 +56,12 @@ jobs:
           echo "Running project checks targets: ${project_check_targets}"
           echo "Building runtimes: ${runtimes_to_build}"
           echo "Running runtimes checks targets: ${runtimes_check_targets}"
+          echo "Running multiconfig runtimes checks targets: ${runtimes_check_targets_multiconfig}"
 
           export CC=/opt/llvm/bin/clang
           export CXX=/opt/llvm/bin/clang++
 
-          ./.ci/monolithic-linux.sh "${projects_to_build}" "${project_check_targets}" "${runtimes_to_build}" "${runtimes_check_targets}"
+          ./.ci/monolithic-linux.sh "${projects_to_build}" "${project_check_targets}" "${runtimes_to_build}" "${runtimes_check_targets}" "${runtimes_check_targets_multiconfig}"
       - name: Upload Artifacts
         uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
         with:

[boomanaiden154]

This/dependent patches are intended to land after we deprecate/shutdown Buildkite. No effort was made here to keep the Buildkite pipeline working in the same manner as the Github workflow.

[DavidSpickett]

What does multiconfig actually mean in this context?

It doesn't relate to multilib, I understand that, but does it mean we're going to test more than one runtime or that we'll test the same runtime multiple ways?

[boomanaiden154]

It doesn't relate to multilib, I understand that, but does it mean we're going to test more than one runtime or that we'll test the same runtime multiple ways?

It's runtimes that we test in multiple ways (-std=c++26 and enable_modules=clang currently). I felt multiconfig covered that and couldn't really think of a better name. If anyone else has better ideas I'd be happy to change it up.

[Endilll]

It doesn't relate to multilib, I understand that, but does it mean we're going to test more than one runtime or that we'll test the same runtime multiple ways?

It's runtimes that we test in multiple ways (-std=c++26 and enable_modules=clang currently). I felt multiconfig covered that and couldn't really think of a better name. If anyone else has better ideas I'd be happy to change it up.

Multiconfig in this context has some strong associations with CMake's Ninja Multi-Config generator for me. My suggestion is needs_reconfig.

[DavidSpickett]

Agree with needs_reconfig.

I do see your logic with multiconfig but reconfig is a bit better for a casual reader.

[DavidSpickett]

Some of the if/else are awkward but you said you'd clean that up later and that's sensible.

With the multi/reconfig thing fixed, this LGTM.

[boomanaiden154]

Multiconfig in this context has some strong associations with CMake's Ninja Multi-Config generator for me. My suggestion is needs_reconfig.

Agree with needs_reconfig.

Updated. Thanks for the suggestion!