Reporting intermittent test failures
by Harris, James R
Hi all,
I’ve seen a lot of cases recently where -1 votes from the test pool have been removed from a patch due to a failure unrelated to the patch, but then nothing was filed in GitHub for that failure. The filing in GitHub could be a new issue, or a comment on an existing issue.
Please make those GitHub updates a priority. It’s the only way the project can understand the frequency of those intermittent failures and gather to get them fixed. If you’re not sure if a failure has been seen before, search GitHub issues with the “Intermittent Failure” label, or ask on Slack if anyone else has seen the issue. There is no harm in filing a new issue that may be a duplicate – we can always clean these up later during the next bug scrub meeting. The important thing is that we get the failure tracked.
Thanks,
-Jim
5 months
NVMe hotplug for RDMA and TCP transports
by Andrey Kuzmin
Hi team,
is NVMe hotplug functionality as implemented limited to PCIe transport or
does it also work for other transports? If it's currently PCIe only, are
there any plans to extend the support to RDMA/TCP?
Thanks,
Andrey
9 months
SPDK socket abstraction layer
by Zawadzki, Tomasz
Hello everyone,
Summary:
With this message I wanted to update SPDK community on state of VPP socket abstraction as of SPDK 19.07 release.
At this time there does not seem to be a clear efficiency improvements with VPP. There is no further work planned on SPDK and VPP integration.
Details:
As some of you may remember, SPDK 18.04 release introduced support for alternative socket types. Along with that release, Vector Packet Processing (VPP)<https://wiki.fd.io/view/VPP> 18.01 was integrated with SPDK, by expanding socket abstraction to use VPP Communications Library (VCL). TCP/IP stack in VPP<https://wiki.fd.io/view/VPP/HostStack> was in early stages back then and has seen improvements throughout the last year.
To better use VPP capabilities, following fruitful collaboration with VPP team, in SPDK 19.07, this implementation was changed from VCL to VPP Session API from VPP 19.04.2.
VPP socket abstraction has met some challenges due to inherent design of both projects, in particular related to running separate processes and memory copies.
Seeing improvements from original implementation was encouraging, yet measuring against posix socket abstraction (taking into consideration entire system, i.e. both processes), results are comparable. In other words, at this time there does not seem to be a clear benefit of either socket abstraction from standpoint of CPU efficiency or IOPS.
With this message I just wanted to update SPDK community on state of socket abstraction layers as of SPDK 19.07 release. Each SPDK release always brings improvements to the abstraction and its implementations, with exciting work on more efficient use of kernel TCP stack - changes in SPDK 19.10 and SPDK 20.01.
However there is no active involvement at this point around VPP implementation of socket abstraction in SPDK. Contributions in this area are always welcome. In case you're interested in implementing further enhancements of VPP and SPDK integration feel free to reply, or to use one of the many SPDK community communications channels<https://spdk.io/community/>.
Thanks,
Tomek
10 months, 1 week
May 27th: Code walkthrough of the SPDK NVMe hello_world application
by Harris, James R
Hi,
I will be doing a code walkthrough of the SPDK nvme hello_world application next Wednesday (May 27th) from 8-9am US PDT time. This will be a great opportunity if you have heard about SPDK and its userspace NVMe driver, but want to know more about how its APIs work. We will use the same conference call link as we do for our weekly community meetings (https://spdk.io/community).
We are considering monthly walkthroughs for other SPDK libraries as well. If you have a request for a walkthrough of a specific library, please let us know during next Wednesday’s meeting or by replying here on the mailing list.
Thanks,
Jim Harris
10 months, 2 weeks
Intel SPDK Jenkins CI Shutdown - May 29th to June 1st
by Latecki, Karol
SPDK Jenkins CI system will be shut down
Why?
Scheduled planned electrical works on campus.
When?
Shutdown is planned for May 29th 3:00 PM GMT to June 1st 8:00AM GMT.
How does that affect us?
CI will be unable to pick up changes and perform tests during this time.
Thanks,
Karol
10 months, 2 weeks
Re: Question about MDTS and MAXH2CDATA for NVMe/TCP target
by Harris, James R
Hi Wenhua,
On 5/20/20, 6:14 PM, "Wenhua Liu" <liuw(a)vmware.com> wrote:
Hi,
I see in the current implementation, for NVMe/TCP target, both controller MDTS and NVMe/TCP have the same value.
spdk_nvmf_ctrlr_identify_ctrlr:
cdata->mdts = spdk_u32log2(transport->opts.max_io_size / 4096);
nvmf_tcp_icreq_handle:
ic_resp->maxh2cdata = ttransport->transport.opts.max_io_size;
On host side, upper layer (which is NVMe/TCP independent) uses MDTS to determine the maximum IO size. But with the current implementation, there can only be H2CData PDU transferred, the R2T mechanism is not fully utilized. Because of this, if the max_io_size is small, the upper layer has to split large IO coming from application to multiple smaller chunks and send to NVMe/TCP initiator driver. If the max_io_size is large, it may take long time for one PDU to be transferred (I’m not expertized on Network and don’t know what problem can happen).
[Jim] I don't really think splitting the large I/O is a problem. Both the Linux kernel and SPDK drivers automatically do the splitting on behalf of the upper layer or application. The overhead of the splitting is also minimal, since we are only doing one split per 64KiB (or more).
[Jim] Regarding the PDU size - splitting into payload into multiple PDUs is possible but we haven't seen a need to implement that yet with SPDK. One PDU may take a long time to be transferred, but no longer than if it is broken into multiple PDUs. In either case, the PDU will be split into multiple network packets based on MTU.
I found this when I ran fio with blocksize setting to 512KB. I wanted to test a fix I made for MAXH2CDATA impacts the IO behavior but only saw one H2CData PDU for any IOs.
In my understanding, the MDTS and the MAXH2CDATA don’t have to be same, making them initialized from different config parameters will give us more flexibility.
[Jim] It may give more flexibility, but it would also make the code more complex. But if you can share performance data that shows a clear advantage to making those values different, it could justify that extra complexity.
Regards,
Jim
10 months, 3 weeks
SPDK Thread and Nvmf QPair migration
by Hari Kadayam
We are trying to add SPDK to our existing async based framework and would like to do at spdk_thread level instead of reactor. I think we can run multiple spdk_threads in a single system thread by polling each thread in a round robin fashion. However, how do we move the spdk_thread into a different system thread, should we pause and pass the object to other thread and resume? What happens to the NVMf poll group which is associated in that thread. Are they movable to a different system thread? My understanding is that nvmf q-pair once associated needs to stick that poll group. Is that the same for system threads as well (meaning does it use any thread local variables) which stops them from moving to a different thread.
Thanks
10 months, 3 weeks
