verbs: RDMA support for DMA handle #1623
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 7 commits
July 7, 2025 14:35
To commit: da3711074f52 ("RDMA/core: Add support to optional-counters
binding configuration").
Signed-off-by: Yishai Hadas <[email protected]>
Introduce verbs for DMA handle allocation and deallocation. This addition allows applications to allocate an opaque DMA handle that can be used for optimizing DMA and RDMA operations associated with a registered memory region. An upcoming patch in the series will enhance the ibv_reg_mr() verb to support usage of the allocated DMA handle. Signed-off-by: Yishai Hadas <[email protected]>
Add ibv_cmd_alloc/free commands for DMA handle to be used by drivers. Signed-off-by: Yishai Hadas <[email protected]>
After validating the input as supported, the corresponding ibv_cmd_alloc_dmah() or ibv_cmd_free_dmah() function is invoked to perform the operation. Signed-off-by: Yishai Hadas <[email protected]>
This verb can be used for all current ibv_reg_mr() variants (e.g. dmabuf, iova, etc.) and opens the door for applications to use the DMA handle functionality. Signed-off-by: Yishai Hadas <[email protected]>
Add ibv_cmd_reg_mr_ex() to be used by drivers which implement the ibv_reg_mr_ex() verb. Signed-off-by: Yishai Hadas <[email protected]>
Add support for the ibv_reg_mr_ex() verb. Signed-off-by: Yishai Hadas <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch series introduces a new DMA Handle (DMAH) object, along with its corresponding verbs for allocation and deallocation. (i.e. ibv_alloc_dmah(), ibv_dealloc_dmah())
The DMAH object encapsulates attributes relevant for DMA transactions.
While initially intended to support TLP Processing Hints (TPH) [1], the design is extensible to accommodate future features such as PCI multipath for DMA, PCI UIO configurations, traffic class selection, and more.
This addition enables applications to allocate an opaque DMA handle that can be used to optimize DMA and RDMA operations associated with a registered memory region.
Additionally, we introduce a new verb named ibv_reg_mr_ex() with its corresponding ioctl (i.e. UVERBS_METHOD_REG_MR) on the MR object.
This verb consolidates multiple reg_mr variants under a single user-space ioctl interface, supporting: ibv_reg_mr(), ibv_reg_mr_iova(), ibv_reg_mr_iova2() and ibv_reg_dmabuf_mr().
It also enables passing a DMA handle as part of the registration process to associate it with the registered MR.
The series includes the implementation of the new functionality in the mlx5 driver.
Detailed man pages exist to describe the expected usage.
The matching kernel series was sent to rdma-next.
[1] Background, from PCIe specification 6.2.
TLP Processing Hints (TPH)
TLP Processing Hints is an optional feature that provides hints in Request TLP headers to facilitate optimized processing of Requests that target Memory Space. These Processing Hints enable the system hardware (e.g., the Root Complex and/ or Endpoints) to optimize platform resources such as system and memory interconnect on a per TLP basis.