Hi Edwin,
Another possibility would be to use spdk_bdev_nvme_io_passthru and avoid trying to create
a KV abstraction at all in the bdev layer. Then the bdev nvme module can just pass those
operations straight through to a KV-capable NVMe SSD. Or a bdev module can implement a
handler to emulate NVMe KV opcodes.
-Jim
On 3/9/21, 6:31 AM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I've been implementing the NVMeOf KV protocol and have a question about
implementing the KV list operation. Currently, I've implemented it as returning the
same array of packed keys expected by the KV protocol, but it strikes me as tying bdev too
closely to the NVME protocol.
A simple solution is to return one key for each list operation, then iteratively
increment the key and pass it back to bdev to search for the next key. Downside is that
the bdev implementation could be expensive to search for a key and that would have to be
done for each and every key.
Another solution is to return a list or array which the controller iterates over to
fill in the NVME list of keys, however this requires the controller to estimate how many
keys will be needed to fill in the results buffer. Since keys are variable length and
packed together that will pose a challenge.
And another solution would be to create bdev operations to iterate over the keys in
the bdev, but that requires at least 3 operations - open, next and close and requires
maintaining state somewhere, probably in the bdev_io itself.
What would be in keeping with the design philosphy of bdev? Keep it as is, returning
an nvme structure of packed keys or something more abstract?
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
Show replies by thread
We're actually implementing a fabric attached KV store using a bdev for the storage.
Eventually, we'd like use a KV SSD on the back end and envision using a pass through
nvme bdev to access it. But for now, we have a couple of types of bdevs using disk based
kv stores. It would have been cool to use the blobstore enabled rocksdb used by SPDK but
I haven't yet figured out how to configure one bdev while in the midst of configuring
a different bdev. And it also introduces a circular dependency between rocksdb and
spdk.
________________________________
From: Harris, James R <james.r.harris(a)intel.com>
Sent: Tuesday, March 9, 2021 9:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: bdev architecture
NetApp Security WARNING: This is an external email. Do not click links or open attachments
unless you recognize the sender and know the content is safe.
Hi Edwin,
Another possibility would be to use spdk_bdev_nvme_io_passthru and avoid trying to create
a KV abstraction at all in the bdev layer. Then the bdev nvme module can just pass those
operations straight through to a KV-capable NVMe SSD. Or a bdev module can implement a
handler to emulate NVMe KV opcodes.
-Jim
On 3/9/21, 6:31 AM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I've been implementing the NVMeOf KV protocol and have a question about
implementing the KV list operation. Currently, I've implemented it as returning the
same array of packed keys expected by the KV protocol, but it strikes me as tying bdev too
closely to the NVME protocol.
A simple solution is to return one key for each list operation, then iteratively
increment the key and pass it back to bdev to search for the next key. Downside is that
the bdev implementation could be expensive to search for a key and that would have to be
done for each and every key.
Another solution is to return a list or array which the controller iterates over to
fill in the NVME list of keys, however this requires the controller to estimate how many
keys will be needed to fill in the results buffer. Since keys are variable length and
packed together that will pose a challenge.
And another solution would be to create bdev operations to iterate over the keys in
the bdev, but that requires at least 3 operations - open, next and close and requires
maintaining state somewhere, probably in the bdev_io itself.
What would be in keeping with the design philosphy of bdev? Keep it as is, returning
an nvme structure of packed keys or something more abstract?
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
Maybe another approach is for the caller to pass a function pointer to a callback function
that gets called for each element in the list by the bdev. That might be the most
flexible approach from the bdev perspective, though it requires adding more state to
bdev_io.
________________________________
From: Harris, James R <james.r.harris(a)intel.com>
Sent: Tuesday, March 9, 2021 9:24 AM
To: Storage Performance Development Kit <spdk(a)lists.01.org>
Subject: [SPDK] Re: bdev architecture
NetApp Security WARNING: This is an external email. Do not click links or open attachments
unless you recognize the sender and know the content is safe.
Hi Edwin,
Another possibility would be to use spdk_bdev_nvme_io_passthru and avoid trying to create
a KV abstraction at all in the bdev layer. Then the bdev nvme module can just pass those
operations straight through to a KV-capable NVMe SSD. Or a bdev module can implement a
handler to emulate NVMe KV opcodes.
-Jim
On 3/9/21, 6:31 AM, "Rodriguez, Edwin" <Ed.Rodriguez(a)netapp.com> wrote:
I've been implementing the NVMeOf KV protocol and have a question about
implementing the KV list operation. Currently, I've implemented it as returning the
same array of packed keys expected by the KV protocol, but it strikes me as tying bdev too
closely to the NVME protocol.
A simple solution is to return one key for each list operation, then iteratively
increment the key and pass it back to bdev to search for the next key. Downside is that
the bdev implementation could be expensive to search for a key and that would have to be
done for each and every key.
Another solution is to return a list or array which the controller iterates over to
fill in the NVME list of keys, however this requires the controller to estimate how many
keys will be needed to fill in the results buffer. Since keys are variable length and
packed together that will pose a challenge.
And another solution would be to create bdev operations to iterate over the keys in
the bdev, but that requires at least 3 operations - open, next and close and requires
maintaining state somewhere, probably in the bdev_io itself.
What would be in keeping with the design philosphy of bdev? Keep it as is, returning
an nvme structure of packed keys or something more abstract?
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org