8:08 p.m.
I've done some initial review on your patches. Your NVMe changes
look fine to me - but I'm not the one to review that.
Thanks a lot for the prompt review Darius!
I have some comments regarding your memory management refactor. I
posted all of them on a single patch -
https://review.gerrithub.io/c/397038/
I have put some replies on Gerrit. I think your comments are good and I will include them
in v2.
I'm thinking the g_phys_regions could be a global vtophys
override map. We could expose public
spdk_vtophys_override(vaddr, len, paddr) API -> then we wouldn't need to modify
spdk_mem_register.
I am not opposed to this but I would like others to chime in on this approach. It does
involve two function calls rather than one which complicates life for the user. Does
anyone else have opinion on this? I did like the way "improved"
spdk_mem_register to accomodate this new class of memory region rather than add a separate
function.
I am going to give it a few more days for others to review and comment on this and other
aspects of the patch series and then I will resubmit.
Is there any chance a core maintainer can pull these into the CI environment so I can see
if there are any other issues I need to be aware off before the respin?
Cheers
Stephen
On 2018-01-30, 1:45 AM, "SPDK on behalf of Stojaczyk, DariuszX"
<spdk-bounces(a)lists.01.org on behalf of dariuszx.stojaczyk(a)intel.com> wrote:
Hi Stephen,
I've done some initial review on your patches. Your NVMe changes look fine to me -
but I'm not the one to review that.
I have some comments regarding your memory management refactor. I posted all of them
on a single patch - https://review.gerrithub.io/c/397038/
Basically:
```
I'm thinking the g_phys_regions could be a global vtophys override map.
We could expose public spdk_vtophys_override(vaddr, len, paddr) API -> then we
wouldn't need to modify spdk_mem_register.
NVMe could do:
spdk_vtophys_override(cmb_vaddr, len, cmb_paddr);
spdk_mem_register(cmb_vaddr, len);
```
Regards,
D.
-----Original Message-----
From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of Stephen Bates
Sent: Tuesday, January 30, 2018 4:35 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] SPDK support for NVMe CMBs and PMRs with WDS/RDS
Hi All
I just uploaded a patchset to Gerrit that adds support for NVMe controllers with
CMBs/PMRs that support WDS and RDS (my full branch of the code is at [1]).
This allows NVMe controllers to move/copy data from a namespace on one
controller to a namespace on a different controller without requiring a system
memory buffer. There are lots of interesting use cases for such data-
movement.
The biggest issue with using a CMB or PMR for data copies is getting a vtophys
translation. This series adds a new vtophys method for physical memory
regions that fail via the existing methods (e.g. a PCIe BAR). This new method
using a linked list of physical regions that can be added/deleted via
spdk_reg_memory calls. We can then allocate/free memory from these
regions and the current maps can handle both the reference counting and store
the vtophys translations.
The hello_world example is updated to utilize the CMB WDS/RDS capability if
the associated controller supports it. It addition a new example application
called cmb_copy is included that performs the aforementioned offloaded copy
when a CMB is available.
We have confirmed both cmb_copy and the new hello_world work as expected
on both hardware from Eiditicom and Everspin. We plan to do more testing as
more drives with WDS/RDS capable CMBs become available. We used PCIe
packet counters in the Microsemi PCIe switch to confirm traffic is moving
directly between the two NVMe SSDs and not being routed to the root complex
on the CPU.
Feedback on the patches is gratefully received!
Cheers
Stephen
[1] https://github.com/Eideticom/spdk/tree/cmb-copy-v3
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk
_______________________________________________
SPDK mailing list
SPDK(a)lists.01.org
https://lists.01.org/mailman/listinfo/spdk