Use sysroot 2.17 on CUDA < 11.8.

[bdice]

Description

Nightly CI was failing on CUDA 11.4 due to #741. This fixes it.
Old versions of nvcc for CUDA 11 do not support sysroot 2.28 until CUDA 11.8.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.
• The cmake-format.json is up to date with these changes.
• I have added new files under rapids-cmake/
  • I have added include guards (include_guard(GLOBAL))
  • I have added the associated docs/ rst file and update the api.rst

[vyasr]

/merge

[jakirkham]

Thanks Bradley! 🙏

Think CUDA 11 ARM still needs 2.28. Have added a suggestion below

As the CI failure was only on x86_64, think this would be ok

[jakirkham]

ARM is not compatible with GLIBC 2.17

$ strings /opt/conda/envs/cuda11/lib/libcurand.so | grep GLIBC
GLIBC_2.17
GLIBC_2.27
GLIBC_2.17

So think we should stick to GLIBC 2.28 here

Suggested change

	- matrix:
	arch: aarch64
	cuda: "11.[2456]"
	packages:
	- sysroot_linux-aarch64==2.17
	- matrix:
	arch: aarch64
	cuda: "11.[2456]"
	packages:
	- sysroot_linux-aarch64==2.28

[bdice]

Good call. We can equivalently just erase this and let it use the matrix entry below.

[jakirkham]

Agree that looks cleaner. Let's do it 👍

[bdice]

Oops: this is not correct. https://anaconda.org/conda-forge/nvcc_linux-aarch64/files?version=11.4

The aarch64 packages for nvcc 11.4 (up to 11.7) require sysroot 2.17. The current state of the PR (using sysroot 2.17) is correct for all nvcc versions < 11.8, and is the same on x86 and ARM.

[bdice]

I agree that CUDA libraries require glibc versions newer than 2.17 -- but the nvcc compiler metapackage for CUDA 11 is only installable with sysroot 2.17 until we get to recent builds of nvcc 11.8 that permit newer sysroots.

[bdice]

@jakirkham I am going to merge this as-is so that we can unblock nightly CI.

I think there's nothing else we should do here, but if you want to lift the restrictions for old CUDA compilers, a follow-up PR would be welcome.

[jakirkham]

So I had looked into this recently. Here is what I observed:

• The overly strict sysroot constraint was flagged as a bug
  • BUG: nvcc_linux-aarch64 11.8 cannot be installed with sysroot_linux-aarch64 2.28 conda-forge/nvcc-feedstock#107
• A fix was made to relax it
  • Clean up; drop CUDA <11.8, add stdlib; avoid libgcc-ng in ignore_run_exports conda-forge/nvcc-feedstock#106
• The old versions don't have this fix, but could benefit from the same treatment

Typically that is enough to justify a repodata patch and it makes sense

Additionally noticed that nvcc_{{ target_platform }}'s pinnings could be handled more systematically, which is now fixed

• Use pin_compatible for cudatoolkit & sysroot conda-forge/nvcc-feedstock#115

Following this submitted a repodata patch

• Relax nvcc's sysroot upper bound conda-forge/conda-forge-repodata-patches-feedstock#950

The repodata patch landed before last night's nightly run, which passed in CUDA 11.4 (the affected job)

Note there is a CUDA 12.5 failure in that nightly, but that is due to actual test failures

The following tests FAILED:
	570 - install_relocatable-with-gtest_discover_tests-ninja_configure (Failed)
	571 - install_relocatable-with-gtest_discover_tests-ninja (Not Run) ninja test
Errors while running CTest

So AFAICT this issue is already fixed

[bdice]

Wonderful. I apologize, I didn't see those PRs, just the end of the Slack conversation, and thought this might have been the best we could do. I appreciate you taking the extra steps, which now allow us to simplify things.

I filed a follow-up to simplify the dependencies.yaml and use sysroot 2.28 everywhere in #749.

[jakirkham]

I'm sorry as well. I meant to update this thread with those details and just hadn't had time to do so. The CUDA 12.8 rollout started very soon after

If you are feeling stuck, please feel free to DM me. Am here to help

Merging to unblock nightly CI, a follow-up may be created later.