Skip to content

Commit 8e45bd4

Browse files
[OMPIRBuilder] - Make offloading input data persist for deferred target tasks.
When we offload to the target, the pointers to data used by the kernel are passed in arrays created by OMPIRBuilder. These arrays of pointers are allocated on the stack on the host. This is fine for the most part because absent the `nowait` clause, the default behavior is that target tasks are included tasks. That is, the host waits for the target task (in other words, the target kernel) to complete before proceeding. In turn, this means that the host's stack frame is intact and accessing the array of pointers when offloading is safe. However, when nowait is used on the `!$ omp target` instance, then the target task is a deferred task meaning, the generating task on the host does not have to wait for the target kernel to finish. In such cases, it is very likely that the stack frame of the function invoking the target call is wound up thereby leading to memory access errors as shown below. AMDGPU error: Error in hsa_amd_memory_pool_allocate: HSA_STATUS_ERROR_INVALID_ALLOCATION: The requested allocation is not valid. AMDGPU error: Error in hsa_amd_memory_pool_allocate: HSA_STATUS_ERROR_INVALID_ALLOCATION: The requested allocation is not valid. "PluginInterface" error: Failure to allocate device memory: Failed to allocate from memory manager fort.cod.out: /llvm/llvm-project/offload/plugins-nextgen/common/src/PluginInterface.cpp:1434: Error llvm::omp::target::plugin::PinnedAllocationMapTy::lockMappedHostBuffer(void *, size_t): Assertion `HstPtr && "Invalid pointer"' failed. Aborted (core dumped) This PR implements support in OMPIRBuilder to store these arrays of pointers in the task structure that is passed to the target task thereby ensuring it is available to the target task when the target task is eventually scheduled.
1 parent 8e2a42a commit 8e45bd4

File tree

4 files changed

+272
-100
lines changed

4 files changed

+272
-100
lines changed

llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2496,7 +2496,7 @@ class OpenMPIRBuilder {
24962496
TargetTaskBodyCallbackTy TaskBodyCB, Value *DeviceID, Value *RTLoc,
24972497
OpenMPIRBuilder::InsertPointTy AllocaIP,
24982498
const SmallVector<llvm::OpenMPIRBuilder::DependData> &Dependencies,
2499-
bool HasNoWait);
2499+
TargetDataRTArgs &RTArgs, bool HasNoWait);
25002500

25012501
/// Emit the arguments to be passed to the runtime library based on the
25022502
/// arrays of base pointers, pointers, sizes, map types, and mappers. If

0 commit comments

Comments
 (0)