Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions docs_input/basics/memory.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
.. _memory:

Memory
======

MatX allows tensors to be allocated in several different spaces either corresponding to physical or logical
allocations. The space where the memory is allocated dictates where a user is allowed to access the memory.
For example, allocating device memory on some system is not accessible by the CPU and will result in a SEGFAULT
when trying to access.

The type of allocation may behave differently across systems. For example, on a Grace-Hopper system (GH200)
standard host memory from malloc is accessible from the GPU, but may not be on other platforms. Some types
may not be available in certain environments. On WSL2 CUDA Unified Memory (UM) or managed memory is not fully
supported and may result in slow code or other issues.

The memory type is typically chosen when creating a tensor with `make_tensor`. The memory *may* be allocated
immediately, but it is not guaranteed. The memory is guaranteed to be available before it used used, however.

.. doxygenenum:: matxMemorySpace_t
12 changes: 6 additions & 6 deletions include/matx/core/allocator.h
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@ namespace matx {
*
*/
enum matxMemorySpace_t {
MATX_MANAGED_MEMORY,
MATX_HOST_MEMORY,
MATX_HOST_MALLOC_MEMORY,
MATX_DEVICE_MEMORY,
MATX_ASYNC_DEVICE_MEMORY,
MATX_INVALID_MEMORY
MATX_MANAGED_MEMORY, ///< CUDA managed memory or CUDA Unified Memory (UM) from cudaMallocManaged
MATX_HOST_MEMORY, ///< CUDA host-pinned memory from cudaHostAlloc
MATX_HOST_MALLOC_MEMORY, ///< Host-alloced memory (pageable) from malloc
MATX_DEVICE_MEMORY, ///< CUDA device memory from cudaMalloc
MATX_ASYNC_DEVICE_MEMORY, ///< CUDA asynchronous device memory corresponding to a stream from cudaMallocAsync
MATX_INVALID_MEMORY ///< Sentinel value
};

namespace detail {
Expand Down