./rpc.py bdev_nvme_attach_controller -b Nvme0 -t PCIe -a 0000:3b:00.0 ./rpc.py bdev_aio_create /dev/sdk1 Aio0 512 ./rpc.py bdev_ocf_create Cache0 wb Nvme0n1 Aio0 --cache-line-size 64 Inserting cache Cache0 Cache0: Metadata initialized Cache0: Successfully added Cache0: Cache mode : wb Cache0: Attaching cache device failed vbdev_ocf.c:1016:start_cache_cmpl: *ERROR*: Error -1000013 during start cache Cache0, starting rollback Cache0: Cannot flush cache - cache device is detached Cache0: Cache Cache0 successfully stopped Looks like the cache/core device is not attached, what else should I do to make it work? P.S. I have run Open-CAS-Linux successfully on this machine, so the cas_cache and cas_cache ko are currently loaded. casctl and casadm tools exist as well. _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org

What kind of workloads from FIO?

And what's the capacity of the NVMe SSD as Cache device for the Core device (HDD).

Are you using the OCF bdev for the FIO?

If there is detailed configuration of FIO and OCF Bdev, that will be helpful. Thanks, Gang -----Original Message----- From: Bob Chen <a175818323(a)gmail.com&gt; Sent: Friday, August 28, 2020 15:20 To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: [SPDK] Re: Unable to run OCF bdev, bdev not attached Hi, folks I solved this issue by myself. It's because the latest SPDK needs 2MB hugepages while the spdk_tgt app applies 1GB by default. Then I began to run some fio commands on the core device (/dev/sdk1), but surprisingly found that there was NO performance improvement at all. Using SPDK + OCF to accelerate a HDD by an NVMe SSD, is this a correct scenario? <a175818323(a)gmail.com&gt; 于2020年8月27日周四 下午5:58写道： > ./rpc.py bdev_nvme_attach_controller -b Nvme0 -t PCIe -a 0000:3b:00.0 > ./rpc.py bdev_aio_create /dev/sdk1 Aio0 512 ./rpc.py bdev_ocf_create > Cache0 wb Nvme0n1 Aio0 --cache-line-size 64 > > > Inserting cache Cache0 > Cache0: Metadata initialized > Cache0: Successfully added > Cache0: Cache mode : wb > Cache0: Attaching cache device failed > vbdev_ocf.c:1016:start_cache_cmpl: *ERROR*: Error -1000013 during > start cache Cache0, starting rollback > Cache0: Cannot flush cache - cache device is detached > Cache0: Cache Cache0 successfully stopped > > > Looks like the cache/core device is not attached, what else should I > do to make it work? > > P.S. I have run Open-CAS-Linux successfully on this machine, so the > cas_cache and cas_cache ko are currently loaded. casctl and casadm > tools exist as well. > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to > spdk-leave(a)lists.01.org > _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org

> > Are you using the OCF bdev for the FIO? > No I'm not, the FIO command was directly run on /dev/sdk1. I suppose it's a aio bdev so I can read/write it directly with the OS? [Gang]: Looks like this is the problem. You will need to configure the SPDK OCF bdev for the FIO and the FIO is based on SPDK bdev layer. Please follow below link: https://mp.weixin.qq.com/s?__biz=MzI3NDA4ODY4MA==&mid=2653335424&... For the SPDK code, you still could use you current code, this is more about reference of the OCF and FIO configuration. -----Original Message----- From: Bob Chen <a175818323(a)gmail.com&gt; Sent: Friday, August 28, 2020 15:38 To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: [SPDK] Re: Unable to run OCF bdev, bdev not attached > > > What kind of workloads from FIO? 4KB sequential write or randwrite > And what's the capacity of the NVMe SSD as Cache device for the Core > device (HDD). > 7TB SSD, 3.7TB HDD, 64KB OCF cache line > > Are you using the OCF bdev for the FIO? > No I'm not, the FIO command was directly run on /dev/sdk1. I suppose it's a aio bdev so I can read/write it directly with the OS? > > If there is detailed configuration of FIO and OCF Bdev, that will be > helpful. > > Thanks, > Gang > > -----Original Message----- > From: Bob Chen <a175818323(a)gmail.com&gt; > Sent: Friday, August 28, 2020 15:20 > To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; > Subject: [SPDK] Re: Unable to run OCF bdev, bdev not attached > > Hi, folks > > I solved this issue by myself. It's because the latest SPDK needs 2MB > hugepages while the spdk_tgt app applies 1GB by default. > > Then I began to run some fio commands on the core device (/dev/sdk1), > but surprisingly found that there was NO performance improvement at all. > > Using SPDK + OCF to accelerate a HDD by an NVMe SSD, is this a correct > scenario? > > <a175818323(a)gmail.com&gt; 于2020年8月27日周四 下午5:58写道： > > > ./rpc.py bdev_nvme_attach_controller -b Nvme0 -t PCIe -a > > 0000:3b:00.0 ./rpc.py bdev_aio_create /dev/sdk1 Aio0 512 ./rpc.py > > bdev_ocf_create > > Cache0 wb Nvme0n1 Aio0 --cache-line-size 64 > > > > > > Inserting cache Cache0 > > Cache0: Metadata initialized > > Cache0: Successfully added > > Cache0: Cache mode : wb > > Cache0: Attaching cache device failed > > vbdev_ocf.c:1016:start_cache_cmpl: *ERROR*: Error -1000013 during > > start cache Cache0, starting rollback > > Cache0: Cannot flush cache - cache device is detached > > Cache0: Cache Cache0 successfully stopped > > > > > > Looks like the cache/core device is not attached, what else should I > > do to make it work? > > > > P.S. I have run Open-CAS-Linux successfully on this machine, so the > > cas_cache and cas_cache ko are currently loaded. casctl and casadm > > tools exist as well. > > _______________________________________________ > > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email > > to spdk-leave(a)lists.01.org > > > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to > spdk-leave(a)lists.01.org > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to > spdk-leave(a)lists.01.org > _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org

Yes I've read this article. I'm just curious that is there any possibility I can directly use the filesystem(ext4) above OS, while entire filesystem (on HDD) is accelerated by SPDK + OCF transparently. I know SPDK has its BlobFS, however by far it only supports append-only write. It seems to me that using the OS filesystem is a more practical way. Cao, Gang <gang.cao(a)intel.com&gt; 于2020年8月28日周五 下午3:43写道： > > > > Are you using the OCF bdev for the FIO? > > > > No I'm not, the FIO command was directly run on /dev/sdk1. I suppose > it's a aio bdev so I can read/write it directly with the OS? > > [Gang]: Looks like this is the problem. You will need to configure > the SPDK OCF bdev for the FIO and the FIO is based on SPDK bdev > layer. Please follow below link: > > https://mp.weixin.qq.com/s?__biz=MzI3NDA4ODY4MA==&mid=2653335424&... > 1&sn=0f4dfc6c884cea4780c016902dab59e8&chksm=f0cb5807c7bcd111a815ded1b > ab1dd8260167c59eab71ba729e27ec1f76b266e23b5f1a08bf2&token=1477511457& > lang=zh_CN#rd > > For the SPDK code, you still could use you current code, this is more > about reference of the OCF and FIO configuration. > > -----Original Message----- > From: Bob Chen <a175818323(a)gmail.com&gt; > Sent: Friday, August 28, 2020 15:38 > To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; > Subject: [SPDK] Re: Unable to run OCF bdev, bdev not attached > > > > > > > What kind of workloads from FIO? > > > 4KB sequential write or randwrite > > > > And what's the capacity of the NVMe SSD as Cache device for the > > Core device (HDD). > > > > 7TB SSD, 3.7TB HDD, 64KB OCF cache line > > > > > > Are you using the OCF bdev for the FIO? > > > > No I'm not, the FIO command was directly run on /dev/sdk1. I suppose > it's a aio bdev so I can read/write it directly with the OS? > > > > > > If there is detailed configuration of FIO and OCF Bdev, that will > > be helpful. > > > > Thanks, > > Gang > > > > -----Original Message----- > > From: Bob Chen <a175818323(a)gmail.com&gt; > > Sent: Friday, August 28, 2020 15:20 > > To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; > > Subject: [SPDK] Re: Unable to run OCF bdev, bdev not attached > > > > Hi, folks > > > > I solved this issue by myself. It's because the latest SPDK needs > > 2MB hugepages while the spdk_tgt app applies 1GB by default. > > > > Then I began to run some fio commands on the core device > > (/dev/sdk1), but surprisingly found that there was NO performance improvement at all. > > > > Using SPDK + OCF to accelerate a HDD by an NVMe SSD, is this a > > correct scenario? > > > > <a175818323(a)gmail.com&gt; 于2020年8月27日周四 下午5:58写道： > > > > > ./rpc.py bdev_nvme_attach_controller -b Nvme0 -t PCIe -a > > > 0000:3b:00.0 ./rpc.py bdev_aio_create /dev/sdk1 Aio0 512 ./rpc.py > > > bdev_ocf_create > > > Cache0 wb Nvme0n1 Aio0 --cache-line-size 64 > > > > > > > > > Inserting cache Cache0 > > > Cache0: Metadata initialized > > > Cache0: Successfully added > > > Cache0: Cache mode : wb > > > Cache0: Attaching cache device failed > > > vbdev_ocf.c:1016:start_cache_cmpl: *ERROR*: Error -1000013 during > > > start cache Cache0, starting rollback > > > Cache0: Cannot flush cache - cache device is detached > > > Cache0: Cache Cache0 successfully stopped > > > > > > > > > Looks like the cache/core device is not attached, what else > > > should I do to make it work? > > > > > > P.S. I have run Open-CAS-Linux successfully on this machine, so > > > the cas_cache and cas_cache ko are currently loaded. casctl and > > > casadm tools exist as well. > > > _______________________________________________ > > > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an > > > email to spdk-leave(a)lists.01.org > > > > > _______________________________________________ > > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email > > to spdk-leave(a)lists.01.org > > _______________________________________________ > > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email > > to spdk-leave(a)lists.01.org > > > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email > to spdk-leave(a)lists.01.org > _______________________________________________ > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email > to spdk-leave(a)lists.01.org >