New subject: 回复： NVMeF Target Error NVMe controller

Tuesday, 28 February 2017

Hi Thomas,

No I am not having any nvme ssd drive in my system, I was using 'nullb0'
device installed through "modprobe null_blk nr_devices=1" for my NVMe over
Fabric bench-marking.

Few Question:
1)  Since it is nullb0 device it can't be there in lspci, does this mean
this is an issue?
2)  Also

 # Direct controller
[Subsystem1]
  NQN nqn.2016-06.io.spdk:cnode1
  Core 0
  Mode Direct
  Listen RDMA 192.168.25.3:4420
  Host nqn.2016-06.io.spdk:init
  NVMe 0000:06:00.0

In the string "NVMe 0000:06:00.0" above, Do  these numbers(
'domain:bus:func.dev') belong to the NVMe SSD drive attached via pci ??

Thanks in advance

Regards
Ankur

On Tue, Feb 28, 2017 at 8:30 PM, thomas.zhang.8288 <
thomas.zhang.8288(a)aliyun.com&gt; wrote:

...
 Hi Ankur,

 Do you have any NVME SSD drive(s) in the system? You can assign one of
 nvme ssd bus info in the 'subsystem1',  also you can dump you nvme ssd
 drive info and extract 'domain:bus:func.dev' info in your system.
     'lspci | grep Non-volatile'

 Mode 'direct' means front nvmf directly control backend nvme ssd drive.

 Thomas

 ------------------------------------------------------------------
 发件人：Ankur Srivastava <asrivastava014(a)gmail.com&gt;
 发送时间：2017年2月28日(星期二) 22:38
 收件人：Storage Performance Development Kit <spdk(a)lists.01.org&gt;;
 thomas.zhang.8288 <thomas.zhang.8288(a)aliyun.com&gt;
 主 题：Re: [SPDK] 回复： NVMeF Target Error NVMe controller

 Thanks Thomas for the work around I have made the subsystem2 as Virtual
 Mode, but still the error persists.
 Error I am getting is related to the subsystem1.

 Modified conf.in
 -------------------------------
 # Direct controller
 [Subsystem1]
   NQN nqn.2016-06.io.spdk:cnode1
   Core 0
   Mode Direct
   Listen RDMA 192.168.25.3:4420
   Host nqn.2016-06.io.spdk:init
   NVMe 0000:06:00.0

 # Multiple subsystems are allowed.
 # Virtual controller
 [Subsystem2]
   NQN nqn.2016-06.io.spdk:cnode2
   Core 0
   Mode Virtual
   Listen RDMA 192.168.25.31:4420
   Host nqn.2016-06.io.spdk:init
   SN SPDK00000000000001
   Namespace Malloc0
   Namespace Malloc1

 Error:
 --------------------------
 [root@localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/
 nvmf.conf.in
 Starting Intel(R) DPDK initialization ...
 [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0
 --file-prefix=rte0 --proc-type=auto ]
 EAL: Detected 12 lcore(s)
 EAL: Auto-detected process type: PRIMARY
 EAL: Probing VFIO support...
 done.
 Occupied cpu core mask is 0x1
 Occupied cpu socket mask is 0x1
 Ioat Copy Engine Offload Enabled
 Total cores available: 1
 Reactor started on core 0 on socket 0
 *** RDMA Transport Init ***
 *** RDMA Transport Init ***
 allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on
 socket 0
 allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
 *** NVMf Target Listening on 192.168.25.3 port 4420 ***
 conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe
 controller for Subsystem1
 nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf()
 failed

 Can you please tell me more about "domain:bus:func:dev", from where I can
 extract this data ?

 Thanks and Regards
 Ankur

 On Tue, Feb 28, 2017 at 7:56 PM, thomas.zhang.8288 <
 thomas.zhang.8288(a)aliyun.com&gt; wrote:
 Hi Ankur,

 You configuration missing nvme 'domain:bus:func:dev' info in the second
 nvmf subsystem 'subsystem2'. If you choose mode 'Direct' , you have to
 specific 'nvme' bus info in the  nvmf subsystem, otherwise you can choose
 'virtual mode'.

 Best Reagrds

 Zhihao

 ------------------------------------------------------------------
 发件人：Ankur Srivastava <asrivastava014(a)gmail.com&gt;
 发送时间：2017年2月28日(星期二) 21:57
 收件人：spdk <spdk(a)lists.01.org&gt;
 主 题：[SPDK] NVMeF Target Error NVMe controller

 Hi All,

 Here are some notes that I took on my system when starting the nvmf target and faced an
issue with NVMe Controller discovery

 1. Mellanox NICs detected successfully.

 2. Installed neccesary modules

    a. modprobe nvme

    b. modprobe nvmet

    c. modprobe nvme-rdma

    d. modprobe nvmet-rdma

 3. Assign NICs ip addresses

    a. ip addr add 192.168.25.3/24 dev nic0
    b. ip addr add 192.168.25.31/24 dev nic1

 4. My configuration file has the following Subsystems

 # Direct controller
 [Subsystem1]
   NQN nqn.2016-06.io.spdk:cnode1
   Core 0
   Mode Direct
   Listen RDMA 192.168.25.3:4420
   Host nqn.2016-06.io.spdk:init
   NVMe 0000:06:00.0

 # Multiple subsystems are allowed.
 [Subsystem2]
   NQN nqn.2016-06.io.spdk:cnode2
   Core 0
   Mode Direct
   Listen RDMA 192.168.25.31:4420
   Host nqn.2016-06.io.spdk:init
   SN SPDK00000000000001
   Namespace Malloc0
   Namespace Malloc1

 5. Start the nvmf target

 cd ~/spdk/app/nvmf_tgt
 sudo ./nvmf_tgt -c nvmf.conf.in

 Getting the following error

 [root@localhost spdk-master]# app/nvmf_tgt/nvmf_tgt -c etc/spdk/nvmf.conf.in
 Starting Intel(R) DPDK initialization ...
 [ DPDK EAL parameters: nvmf -c 1 -n 4 -m 2048 --master-lcore=0 --file-prefix=rte0
--proc-type=auto ]
 EAL: Detected 12 lcore(s)
 EAL: Auto-detected process type: PRIMARY
 EAL: Probing VFIO support...
 done.
 Occupied cpu core mask is 0x1
 Occupied cpu socket mask is 0x1
 Ioat Copy Engine Offload Enabled
 Total cores available: 1
 Reactor started on core 0 on socket 0
 *** RDMA Transport Init ***
 *** RDMA Transport Init ***
 allocated subsystem nqn.2014-08.org.nvmexpress.discovery on lcore 0 on socket 0
 allocated subsystem nqn.2016-06.io.spdk:cnode1 on lcore 0 on socket 0
 *** NVMf Target Listening on 192.168.25.3 port 4420 ***
 conf.c: 565:spdk_nvmf_parse_subsystem: ***ERROR*** Could not find NVMe controller for
Subsystem1
 nvmf_tgt.c: 336:spdk_nvmf_startup: ***ERROR*** spdk_nvmf_parse_conf() failed

 Best Reagrds

 Ankur

 _______________________________________________
 SPDK mailing list
 SPDK(a)lists.01.org
 https://lists.01.org/mailman/listinfo/spdk

2022

2021

2020

2019

2018

2017

2016

Re: [SPDK] 回复： NVMeF Target Error NVMe controller