On Fri, 2019-08-30 at 16:11 -0600, Michael Haeuptle wrote:
I have a couple of observations/questions regarding vendor specific
commands over NVMf and I was hoping to get your take on this.
- I can get vendor specific *IO *commands to work in my custom bdev by
supporting SPDK_BDEV_IO_TYPE_NVME_IO. However, doing the same for
fails. Is there a reason why we are blocking the vendor specific range of
C0h-FFh for Admin commands (see spdk_nvmf_ctrlr_process_admin_cmd).
I'm a bit torn on what to do about vendor specific commands.In the NVMe-oF
target, each bdev is treated as a namespace in a virtualized controller. For
physical NVMe devices, each namespace on the device becomes a bdev. The issues
all arise when a vendor specific command is performing some operation that
targets the whole controller instead of just a namespace. For example:
1) It's entirely possible that a device with two namespaces places each of those
namespaces in an entirely different subsystem on the NVMe-oF side. The hosts
then think these namespaces aren't related to one another. If a vendor specific
command that changes some global state on the SSD is allowed through, this can
cause serious problems to the other unwitting subsystem.
2) The NVMe-oF target allows multiple hosts to connect simultaneously, emulating
a separate NVMe controller for each. Even in the case where the physical SSD has
a single namespace, surfaced as a bdev and added to a single subsystem, allowing
a vendor specific command through may have an impact on both controllers
presented to the two hosts, but there's no way for the NVMe-oF code to know
I think the reality is that all vendor-specific commands will likely need to be
handled in the NVMe-oF software emulation instead of simply passing through to
the backing device blindly. The cleanest way to do this is to probably add a
public API function to the nvmf library that allows users to register a handler
for a vendor specific command, and then in spdk_nvmf_ctrlr_process_admin_cmd it
looks up if a handler is present and calls it rather than just failing
everything in the vendor-specific range.
- The current bdev API to complete a NVMe specific request is
spdk_bdev_io_complete_nvme_status(struct spdk_bdev_io *bdev_io, int sct,
int sc) but it only takes in the 2 status code that are written to the
completion queue status field. I would also like to set the CDW0 of the
completion queue. Are there any plans to support this or do we want to keep
the bdev API as front-end protocol agnostic as possible?
I guess one way to support it would be to add an additional field to
bdev_io's structure like there is for the NVMe/SCSI specific status code
handling, but again, do we want to add more protocol specifics to bdev_io?
Are the any other options to set CDW0?
Thanks for your help.
SPDK mailing list