No worries! You are correct - using the same shared memory ID is the correct approach for
what you are trying to do.
On 8/28/18, 12:24 PM, "SPDK on behalf of Yue Zhu" <spdk-bounces(a)lists.01.org
on behalf of yuezhu1103(a)gmail.com> wrote:
Sorry for sending out another emails. I think my problem can be solved by
simply assigned the same shared memory ID for the program on the same node
to let multiple processes access the same SSD.
Please feel free to correct me if anything is incorrect.
On Tue, Aug 28, 2018 at 2:09 PM Yue Zhu <yuezhu1103(a)gmail.com> wrote:
I wonder if there is anyway that allows a regular SPDK program (e.g.,
perf) and NVMe-oF function properly on the same node.
I have tested running NVMe-oF and perf on the same node, and got some
The error message I got for running perf and NVMe-oF on the same node
(configuration file attached after error info):
./perf -q 1 -o 65536 -w randread -t 10 -r 'trtype:PCIe
traddr:0000:04:00.0' -s 128
Starting SPDK v18.10-pre / DPDK 18.05.0 initialization...
[ DPDK EAL parameters: perf --no-shconf -c 0x1 -m 128 --legacy-mem
EAL: Detected 40 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Multi-process socket /run/user/1003/dpdk/spdk_pid120483/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
Initializing NVMe Controllers
EAL: PCI device 0000:04:00.0 on NUMA socket 0
EAL: probe driver: 8086:953 spdk_nvme
Attaching to NVMe Controller at 0000:04:00.0
Cannot create lock on device /tmp/spdk_pci_lock_0000:04:00.0, probably
process 120478 has claimed it
nvme_pcie.c: 793:nvme_pcie_ctrlr_construct: *ERROR*: could not claim
nvme.c: 384:nvme_ctrlr_probe: *ERROR*: Failed to construct NVMe controller
for SSD: 0000:04:00.0
EAL: Driver cannot attach the device (0000:04:00.0)
No valid NVMe controllers or AIO devices found
The NVMe-oF configuration file I used:
TransportID "trtype:PCIe traddr:0000:04:00.0" Nvme0
Listen RDMA 10.10.16.11:4420
Namespace Nvme0n1 1
I also tried to remove the [NVMe] section in the configuration to let perf
only detect local SSD (it works), but the perf still cannot detect remote
SSD if NVMe-oFs are running on the local and remote nodes.
I did a quick debugging. It appears that nvme_ctrlr_identify() in
nvme_ctrlr_start() cannot initialize ctrlr->cdata.nn to 1. So the program
fails to create active_ns_list. I wonder if this can be solved in any ways.
nvme_ctrlr.c: 824:nvme_ctrlr_identify_active_ns: *ERROR*: Failed to
nvme.c: 424:nvme_init_controllers: *ERROR*: Failed to initialize SSD:
spdk_nvme_probe() failed for transport address '10.10.16.9'
Any comments or suggestions are appreciated.
Thank you in advance.
SPDK mailing list