November 2020 - SPDK - Ml01.01.Org

SPDK vhost mechanism only can use one CPU core for polling even specify multiple cores

by dingbao.chen＠outlook.com

Background: The storage target side is Lightbits' NVMe/TCP storage solution, which provide a NVMe/TCP storage pool, user can create volumes from this pool to any spcecific application servers. Using the vhost is to present the nvme/tcp volume to KVM based virutal machine efficiently. Test steps: 1. Assign 16 cores to vhost build/bin/vhost -S /var/tmp -m 0xFFFF 2. Create a bdev controller with 8 cores, the Nvme0n17 is the NVMe/TCP volume with SPDK initiator. scripts/rpc.py vhost_create_blk_controller --cpumask 0xFF vhost.0 Nvme0n17 3. virtual machine device virtio-blk-pci num-queues=8 4. Running the FIO inside of the VM, the performance result is not as good as expected, did some investigation and suspect the bottleneck is on the vhost controller, which only use one CPU cores even I assigned 8 to it. Checked with spdk top and our LightOS lb_top. The result shows only one core is occupied in vhost, And by inforamtion from Lightbits target side, only one active nvme-tcp connection between HOST Server and Lightbits storage server I checked the vhost bdev module document, it has such word: “All the I/O polling will be pinned to the least occupied CPU core within given cpumask” I suspect this behavior is by design, but still want to get confirmation, and is there any plan to improve?

6 months, 2 weeks

2
1
0 / 0

[RFC]Windows Support

by Nick Connolly

I've added a patchset (https://review.spdk.io/gerrit/c/spdk/spdk/+/5301) that gives an indication of the changes that would be required for an initial cut at adding Windows support to SPDK. These are just an indication of what some of the changes might look like. It is still very much WIP, so please don't focus too much on the details - it's just intended as a courtesy 'heads up'. If anyone would like to play with the code, the repo at https://github.com/wpdk/wpdk contains build instructions and further details on the current state. Regards, Nick

6 months, 2 weeks

1
0
0 / 0

[RFC]Windows Support

by Nick Connolly

I've added a patchset (https://review.spdk.io/gerrit/c/spdk/spdk/+/5301) that gives an indication of the changes that would be required for an initial cut at adding Windows support to SPDK. These are just an indication of what some of the changes might look like. It is still very much WIP, so please don't focus too much on the details - it's just intended as a courtesy 'heads up'. If anyone would like to play with the code, the repo at https://github.com/wpdk/wpdk contains build instructions and further details on the current state. Regards, Nick

6 months, 2 weeks

1
0
0 / 0

Rocksdb over SPDK/Blobfs

by Dimitra Giantsidi

Hi everyone, I am running the rocksdb::RandomTransaction benchmark with *1 thread* with the following specifications: RocksDB: version 5.14 Date: Wed Nov 25 13:05:48 2020 CPU: 16 * Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz CPUCache: 16384 KB Keys: 16 bytes each Values: 1024 bytes each (1024 bytes after compression) Entries: 5000000 Prefix: 0 bytes Keys per prefix: 0 RawSize: 4959.1 MB (estimated) FileSize: 4959.1 MB (estimated) Write rate: 0 bytes/second Read rate: 0 ops/second Compression: NoCompression Memtablerep: skip_list Perf Level: 1 ----------------------------------------------------------------------------------------------------------------------- When using the kernel NVMe driver I have better performance (x 5 times). In particular the db_bench output is: NVMe driver: randomtransaction : *16.727 micros/op 59782 ops/sec;* 1.5 MB/s (transactions:5000000 aborts:0) SPDK: randomtransaction : *71.226 micros/op 14039 ops/sec;* 0.3 MB/s (transactions:5000000 aborts:0) I am using SPDK v18.01.1 and DPDK v18.02. The SPDK cache is 4GB and apparently I cannot increase it in my platform. I am not sure how to explain this behavior. I would appreciate any help/comment on the result. Thanks, Dimitra -- Giantsidi Dimitra e-mail:dimitra.giantsidi@gmail.com

6 months, 3 weeks

2
4
0 / 0

Cancelled: SPDK Asia-Friendly Thursday, November 26, 2020, 12:00 PM GMT+8 Bug Scrub meeting Community Meeting

by Liu, Changpeng

Cancel the next bug scrub meeting due to the US. holidays.

6 months, 3 weeks

1
0
0 / 0

Gerrit vulnerability - please change GitHub credentials

by Harris, James R

Hi, An upstream Gerrit vulnerability was recently found and fixed. The SPDK Gerrit configuration has been updated to mitigate the vulnerability for now. We are in the process of upgrading our Gerrit installation to a version that includes the full fix and expect that to be done by tomorrow morning. Since we are using the GitHub oauth plugin, it is recommended that anyone with an account on the SPDK Gerrit site update their GitHub credentials. More information on the Gerrit vulnerability can be found in the Gerrit 3.2.5 release notes: https://www.gerritcodereview.com/3.2.html#325. Thanks, Jim

6 months, 3 weeks

1
0
0 / 0

AIO unmap support

by khristenko.vladimir1986＠gmail.com

Environment: * SPDK v20.10 * OS Centos8.2 * SSD Intel DC S4600 https://ark.intel.com/content/www/us/en/ark/products/120522/intel-ssd-dc-... My SSD support trim, but when I add it as AIO bdev I see that unmup not supported: #scripts/rpc.py bdev_aio_create /dev/sdf AIO0 # scripts/rpc.py bdev_get_bdevs [ { "name": "AIO0", "aliases": [], "product_name": "AIO disk", "block_size": 512, "num_blocks": 937703088, "uuid": "78b0d637-87ca-49ad-80ec-95c614a66685", ... }, "claimed": false, "zoned": false, "supported_io_types": { "read": true, "write": true, "unmap": false, "write_zeroes": true, "flush": true, "reset": true, "nvme_admin": false, "nvme_io": false }, "driver_specific": { "aio": { "filename": "/dev/sdf" } } } ] Is there any opportunity to enable the unmap support for AIO bdev? If not, is it possible to implement the unmap support for AIO in the near future? Thank you.

6 months, 3 weeks

2
1
0 / 0

Possible to use several hugepage files for NVMe's DMA destination?

by Jing Liu

Hi, In general, I'm wondering how to provide the isolation of the hugepage across several processes using SPDK in one computer In my understanding, SPDK's hugepage is initialized by the primary process, and all will use *shmid* to be able to find that by several processes. I'm wondering is that possible that the SPDK is initialized with several *shmid*, such that several processes can be isolated? PS: I'm using the SPDK 18.04.01, does some new version help? Thanks, Jing

6 months, 4 weeks

2
3
0 / 0

Change path of file lock in spdk_pci_device_claim

by swf504＠163.com

Hi SPDK team, I notice that the posix lock file locate in '/tmp' directory which no safe in running linux system, user maybe delete files in /tmp directory or remout the tmpfs, and that cause the posix lock inactive. This is a serious fatal because the application can't detect the file change, After a new application start and the spdk_pci_device_claim function still return true, that means multi application maybe probe same controller, Unexpect behavior or controller init hung maybe happen during controller colliding. Is there special reason the need locate in '/tmp', if not, I suggest locate in '/var/run' directory. Need your comment! Br, Weifeng Su

7 months

3
2
0 / 0

Reporting intermittent test failures

by Harris, James R

Hi all, I’ve seen a lot of cases recently where -1 votes from the test pool have been removed from a patch due to a failure unrelated to the patch, but then nothing was filed in GitHub for that failure. The filing in GitHub could be a new issue, or a comment on an existing issue. Please make those GitHub updates a priority. It’s the only way the project can understand the frequency of those intermittent failures and gather to get them fixed. If you’re not sure if a failure has been seen before, search GitHub issues with the “Intermittent Failure” label, or ask on Slack if anyone else has seen the issue. There is no harm in filing a new issue that may be a duplicate – we can always clean these up later during the next bug scrub meeting. The important thing is that we get the failure tracked. Thanks, -Jim

7 months, 1 week

3
2
0 / 0

2021

2020

2019

2018

2017

2016

SPDK November 2020