[NVPTX] atomicrmw on <4 x float> relies on __atomic_compare_exchange_16

[Artem-B]

NVPTX currently lowers atomixrmw on <4 x float> as a call to __atomic_compare_exchange_16 which does not exist on the GPU:
https://godbolt.org/z/ovf4cqKK5

Newer GPUs do have support for vectorized atomic ops on some data types, but on the older GPUs they must be lowered without relying on runtime.

[Artem-B]

@gonzalobg - just a FYI as you've done most of the recent NVPTX backend changes related to atomics.

[Artem-B]

The root cause is that NVPTX's TLI->getMaxAtomicSizeInBitsSupported() currently returns 64:

llvm-project/llvm/lib/CodeGen/AtomicExpandPass.cpp

Lines 231 to 238 in e55164a

	template <typename Inst>
	static bool atomicSizeSupported(const TargetLowering *TLI, Inst *I) {
	unsigned Size = getAtomicOpSize(I);
	Align Alignment = I->getAlign();
	return Alignment >= Size &&
	Size <= TLI->getMaxAtomicSizeInBitsSupported() / 8;
	}

which leads to atomicSizeSupported(TLI, RMWI) returning false, which results in LLVM falling back on expanding atomic via libcall:

llvm-project/llvm/lib/CodeGen/AtomicExpandPass.cpp

Line 277 in e55164a

if (!atomicSizeSupported(TLI, RMWI)) {

https://llvm.org/docs/Atomics.html#libcalls-atomic

The only problem is that we do not have that libcall on the GPU, and therefore atomics on 128-bit large types end up trying to call non-existing function.

PTX did introduce .b128 atomic ops on sm_90+ in PTX 8.3, so we could add native support for them.
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#parallel-synchronization-and-communication-instructions-atom

LLVM does not seem to provide any target-specific way to customize this behavior. It does not even check for the libcall availability on any given target, assuming it's always there.

We could set getMaxAtomicSizeInBitsSupported() to return 128, and then tell LLVM to expand large atomics during lowering, but that leads us to another libcall generated by legalizer:

llvm-project/llvm/lib/CodeGen/SelectionDAG/LegalizeIntegerTypes.cpp

Lines 3129 to 3135 in 6767555

	if (TLI.getLibcallName(LC)) {
	Ops.append(Node->op_begin() + 2, Node->op_end());
	Ops.push_back(Node->getOperand(1));
	} else {
	LC = RTLIB::getSYNC(Opc, VT);
	assert(LC != RTLIB::UNKNOWN_LIBCALL &&
	"Unexpected atomic op or value type!");

It also does not give us any options -- it's libcall or assert.

Looks like adding .b128 support for newer GPUs and leaving large atomics unsupported on the older GPUs is the best we can do short term.

[Artem-B]

@AlexMaclean WDYT?