Thanks James! The unit test files have been very helpful, and thanks for sending over those functions to review. To follow up, when using malloc to create a buffer and then calling the spdk_bs_io_write function, is the data actually getting written to the NVMe drive that is connected to the machine? Or, after writing to the blob, do I have to then use the spdk_dma_zmalloc to allocate space on the NVME drive and then actually write the drive using spdk_nvme_ns_cmd_write function? Thanks again for the help! Best, Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Tuesday, July 11, 2017 4:00:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Welcome to SPDK! A Blobstore “Hello World” example is a fantastic idea – definitely something to put on our Trello backlog (which I’ve done just now). You can see our SPDK Trello boards at trello.com/spdk – your suggestion is now in the Low Hanging Fruit column on the Things to Do board. Until that example is ready, I can point you to some of our unit test code that could help in meantime. test/unit/lib/blob/blob.c/blob_ut.c has a number of tests that show some very basic operations with the SPDK Blobstore. All of these tests use a simple malloc buffer to simulate the underlying block device – this code is in test/unit/lib/blob/bs_dev_common.c. The key functions to look at are: Initialize new blobstore => spdk_bs_init Load existing blobstore => spdk_bs_load Create blob => spdk_bs_md_create_blob Open blob => spdk_bs_md_open_blob Read from blob => spdk_bs_io_read_blob Write to blob => spdk_bs_io_write_blob Close blob => spdk_bs_md_close_blob Unload blobstore => spdk_bs_unload One last note as an FYI – a “logical volume store” or lvolstore is under development for SPDK. Lvolstore will provide the ability to dynamically partition a block device into smaller block devices using Blobstore for managing the on-disk metadata. You can see some additional details on the corresponding Trello board. Thanks, -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Tuesday, July 11, 2017 at 7:20 AM To: &quot;spdk(a)lists.01.org&quot; <spdk(a)lists.01.org&gt; Subject: [SPDK] SPDK Blob Store Hello, I am pretty new to the SPDK but am currently working on trying to implement a basic blobstore application for an NVMe Drive. It looks like all of the relevant code has already been written, but i do not see any example code for implementation. Are there any recommended resources or places to start? I am basically, just for the purpose of understanding, attempting something very similar to the hello_world example. However, I am trying to write/read/list/delete blobs instead of just a basic string. Is it really as easy as creating a spdk_blob type and then calling the corresponding functions from blobstore? Create blob -> spdk_bs_md_create_blob Read blob -> spdk_bs_md_open_blob Write blob -> _spdk_blob_persist delete blob -> spdk_bs_md_delete_blob Thanks, Neil

Awesome, Thanks James! In terms of the spdk_bs_io_read/write functions. can you clarify the use of the offset and length parameters? When I call this function, am I essentially specifying the actual offset within the cluster and the length of the blob? or is this an offset to where the data exists within the blob? Thanks, Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Tuesday, July 11, 2017 5:46:38 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, For the SPDK Blobstore, read and write operations to the underlying block device are abstracted through the spdk_bs_dev interface. For the unit tests, this spdk_bs_dev interface is just implemented with 4KB memcopies to/from a malloc buffer – no NVMe device. For real applications, the typical usage model would be implementing spdk_bs_dev with an SPDK block device (bdev). We have a blob_bdev helper library for this in lib/blob/bdev. With blob_bdev, any SPDK bdev can be plugged in – an NVMe bdev (either locally attached or remote using NVMe-oF), libaio, Ceph RBD, etc. But of course locally attached NVMe is the most popular choice here. To write data to a blob, first allocate your data buffer. spdk_dma_zmalloc is the best choice, to ensure the buffer is DMA safe (but you’ll see for unit tests that buffers are either allocated using malloc or just off the stack, since spdk_bs_dev is not implemented with a device using DMA). Then call spdk_bs_io_read/write_blob. Blobstore will then convert your blob + page index into the block device LBA and issue a read/write call on the spdk_bs_dev interface. Then the spdk_bs_dev implementation handles the rest. If you choose to use the blob_bdev library, it will convert the read/write operation to the appropriate SPDK bdev API call. Alternatively, depending on your usage model, you may want to implement the spdk_bs_dev interface using the SPDK NVMe driver directly (and avoiding the SPDK bdev layer). In this case, you would translate the spdk_bs_dev write call to use spdk_nvme_ns_cmd_write. -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Tuesday, July 11, 2017 at 2:32 PM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] SPDK Blob Store Thanks James! The unit test files have been very helpful, and thanks for sending over those functions to review. To follow up, when using malloc to create a buffer and then calling the spdk_bs_io_write function, is the data actually getting written to the NVMe drive that is connected to the machine? Or, after writing to the blob, do I have to then use the spdk_dma_zmalloc to allocate space on the NVME drive and then actually write the drive using spdk_nvme_ns_cmd_write function? Thanks again for the help! Best, Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Tuesday, July 11, 2017 4:00:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Welcome to SPDK! A Blobstore “Hello World” example is a fantastic idea – definitely something to put on our Trello backlog (which I’ve done just now). You can see our SPDK Trello boards at trello.com/spdk – your suggestion is now in the Low Hanging Fruit column on the Things to Do board. Until that example is ready, I can point you to some of our unit test code that could help in meantime. test/unit/lib/blob/blob.c/blob_ut.c has a number of tests that show some very basic operations with the SPDK Blobstore. All of these tests use a simple malloc buffer to simulate the underlying block device – this code is in test/unit/lib/blob/bs_dev_common.c. The key functions to look at are: Initialize new blobstore => spdk_bs_init Load existing blobstore => spdk_bs_load Create blob => spdk_bs_md_create_blob Open blob => spdk_bs_md_open_blob Read from blob => spdk_bs_io_read_blob Write to blob => spdk_bs_io_write_blob Close blob => spdk_bs_md_close_blob Unload blobstore => spdk_bs_unload One last note as an FYI – a “logical volume store” or lvolstore is under development for SPDK. Lvolstore will provide the ability to dynamically partition a block device into smaller block devices using Blobstore for managing the on-disk metadata. You can see some additional details on the corresponding Trello board. Thanks, -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Tuesday, July 11, 2017 at 7:20 AM To: &quot;spdk(a)lists.01.org&quot; <spdk(a)lists.01.org&gt; Subject: [SPDK] SPDK Blob Store Hello, I am pretty new to the SPDK but am currently working on trying to implement a basic blobstore application for an NVMe Drive. It looks like all of the relevant code has already been written, but i do not see any example code for implementation. Are there any recommended resources or places to start? I am basically, just for the purpose of understanding, attempting something very similar to the hello_world example. However, I am trying to write/read/list/delete blobs instead of just a basic string. Is it really as easy as creating a spdk_blob type and then calling the corresponding functions from blobstore? Create blob -> spdk_bs_md_create_blob Read blob -> spdk_bs_md_open_blob Write blob -> _spdk_blob_persist delete blob -> spdk_bs_md_delete_blob Thanks, Neil

Beautiful thanks! So the idea then is that multiple datasets (payloads) can potentially be stored in one blob? to follow up on that, If our use case involves datasets that range from 10-100MB, its probably best to allocate each dataset to its own unique blob? And finally, The documentation for the overall spdk is pretty awesome. I love that there is a description for every function and most types are very well defined. However, the blob.h documentation is really lacking. It would be awesome to have descriptions for the data structures being used as well as the functions that are being called (ie spdk_bs_dev, init_dev) Thanks again! -Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Wednesday, July 12, 2017 10:03:12 AM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Offset is the offset into the blob. Length is the length of the data being read/written starting at that offset. -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Wednesday, July 12, 2017 at 6:59 AM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] SPDK Blob Store Awesome, Thanks James! In terms of the spdk_bs_io_read/write functions. can you clarify the use of the offset and length parameters? When I call this function, am I essentially specifying the actual offset within the cluster and the length of the blob? or is this an offset to where the data exists within the blob? Thanks, Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Tuesday, July 11, 2017 5:46:38 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, For the SPDK Blobstore, read and write operations to the underlying block device are abstracted through the spdk_bs_dev interface. For the unit tests, this spdk_bs_dev interface is just implemented with 4KB memcopies to/from a malloc buffer – no NVMe device. For real applications, the typical usage model would be implementing spdk_bs_dev with an SPDK block device (bdev). We have a blob_bdev helper library for this in lib/blob/bdev. With blob_bdev, any SPDK bdev can be plugged in – an NVMe bdev (either locally attached or remote using NVMe-oF), libaio, Ceph RBD, etc. But of course locally attached NVMe is the most popular choice here. To write data to a blob, first allocate your data buffer. spdk_dma_zmalloc is the best choice, to ensure the buffer is DMA safe (but you’ll see for unit tests that buffers are either allocated using malloc or just off the stack, since spdk_bs_dev is not implemented with a device using DMA). Then call spdk_bs_io_read/write_blob. Blobstore will then convert your blob + page index into the block device LBA and issue a read/write call on the spdk_bs_dev interface. Then the spdk_bs_dev implementation handles the rest. If you choose to use the blob_bdev library, it will convert the read/write operation to the appropriate SPDK bdev API call. Alternatively, depending on your usage model, you may want to implement the spdk_bs_dev interface using the SPDK NVMe driver directly (and avoiding the SPDK bdev layer). In this case, you would translate the spdk_bs_dev write call to use spdk_nvme_ns_cmd_write. -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Tuesday, July 11, 2017 at 2:32 PM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] SPDK Blob Store Thanks James! The unit test files have been very helpful, and thanks for sending over those functions to review. To follow up, when using malloc to create a buffer and then calling the spdk_bs_io_write function, is the data actually getting written to the NVMe drive that is connected to the machine? Or, after writing to the blob, do I have to then use the spdk_dma_zmalloc to allocate space on the NVME drive and then actually write the drive using spdk_nvme_ns_cmd_write function? Thanks again for the help! Best, Neil ________________________________ From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Harris, James R <james.r.harris(a)intel.com&gt; Sent: Tuesday, July 11, 2017 4:00:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Welcome to SPDK! A Blobstore “Hello World” example is a fantastic idea – definitely something to put on our Trello backlog (which I’ve done just now). You can see our SPDK Trello boards at trello.com/spdk – your suggestion is now in the Low Hanging Fruit column on the Things to Do board. Until that example is ready, I can point you to some of our unit test code that could help in meantime. test/unit/lib/blob/blob.c/blob_ut.c has a number of tests that show some very basic operations with the SPDK Blobstore. All of these tests use a simple malloc buffer to simulate the underlying block device – this code is in test/unit/lib/blob/bs_dev_common.c. The key functions to look at are: Initialize new blobstore => spdk_bs_init Load existing blobstore => spdk_bs_load Create blob => spdk_bs_md_create_blob Open blob => spdk_bs_md_open_blob Read from blob => spdk_bs_io_read_blob Write to blob => spdk_bs_io_write_blob Close blob => spdk_bs_md_close_blob Unload blobstore => spdk_bs_unload One last note as an FYI – a “logical volume store” or lvolstore is under development for SPDK. Lvolstore will provide the ability to dynamically partition a block device into smaller block devices using Blobstore for managing the on-disk metadata. You can see some additional details on the corresponding Trello board. Thanks, -Jim From: SPDK <spdk-bounces(a)lists.01.org&gt; on behalf of Neil Kumar <nkumar(a)ocient.com&gt; Reply-To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Date: Tuesday, July 11, 2017 at 7:20 AM To: &quot;spdk(a)lists.01.org&quot; <spdk(a)lists.01.org&gt; Subject: [SPDK] SPDK Blob Store Hello, I am pretty new to the SPDK but am currently working on trying to implement a basic blobstore application for an NVMe Drive. It looks like all of the relevant code has already been written, but i do not see any example code for implementation. Are there any recommended resources or places to start? I am basically, just for the purpose of understanding, attempting something very similar to the hello_world example. However, I am trying to write/read/list/delete blobs instead of just a basic string. Is it really as easy as creating a spdk_blob type and then calling the corresponding functions from blobstore? Create blob -> spdk_bs_md_create_blob Read blob -> spdk_bs_md_open_blob Write blob -> _spdk_blob_persist delete blob -> spdk_bs_md_delete_blob Thanks, Neil

Hi Neil, I’m going to start working on a ‘hello world’ type example for blobstor here soon, I believe I saw you on IRC not too long ago. I’m in there daily so feel free to jump in at any time with more questions. I’ve not done anything with this component before either so I’ll be asking lots of questions there also which may benefit you as well. I’ll work on beefing up the docs as I go as well… Thx Paul PS: If that wasn’t use on IRC, we’re on Freenode channel #spdk From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of Harris, James R Sent: July 12, 2017 8:53 AM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] SPDK Blob Store Hi Neil, It sounds like for your use case, you would want to create a separate blob for each dataset. We hear your request for more documentation in blob.h. There is always room for more and better documentation. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Wednesday, July 12, 2017 at 8:39 AM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Beautiful thanks! So the idea then is that multiple datasets (payloads) can potentially be stored in one blob? to follow up on that, If our use case involves datasets that range from 10-100MB, its probably best to allocate each dataset to its own unique blob? And finally, The documentation for the overall spdk is pretty awesome. I love that there is a description for every function and most types are very well defined. However, the blob.h documentation is really lacking. It would be awesome to have descriptions for the data structures being used as well as the functions that are being called (ie spdk_bs_dev, init_dev) Thanks again! -Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Wednesday, July 12, 2017 10:03:12 AM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Offset is the offset into the blob. Length is the length of the data being read/written starting at that offset. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Wednesday, July 12, 2017 at 6:59 AM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Awesome, Thanks James! In terms of the spdk_bs_io_read/write functions. can you clarify the use of the offset and length parameters? When I call this function, am I essentially specifying the actual offset within the cluster and the length of the blob? or is this an offset to where the data exists within the blob? Thanks, Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Tuesday, July 11, 2017 5:46:38 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, For the SPDK Blobstore, read and write operations to the underlying block device are abstracted through the spdk_bs_dev interface. For the unit tests, this spdk_bs_dev interface is just implemented with 4KB memcopies to/from a malloc buffer – no NVMe device. For real applications, the typical usage model would be implementing spdk_bs_dev with an SPDK block device (bdev). We have a blob_bdev helper library for this in lib/blob/bdev. With blob_bdev, any SPDK bdev can be plugged in – an NVMe bdev (either locally attached or remote using NVMe-oF), libaio, Ceph RBD, etc. But of course locally attached NVMe is the most popular choice here. To write data to a blob, first allocate your data buffer. spdk_dma_zmalloc is the best choice, to ensure the buffer is DMA safe (but you’ll see for unit tests that buffers are either allocated using malloc or just off the stack, since spdk_bs_dev is not implemented with a device using DMA). Then call spdk_bs_io_read/write_blob. Blobstore will then convert your blob + page index into the block device LBA and issue a read/write call on the spdk_bs_dev interface. Then the spdk_bs_dev implementation handles the rest. If you choose to use the blob_bdev library, it will convert the read/write operation to the appropriate SPDK bdev API call. Alternatively, depending on your usage model, you may want to implement the spdk_bs_dev interface using the SPDK NVMe driver directly (and avoiding the SPDK bdev layer). In this case, you would translate the spdk_bs_dev write call to use spdk_nvme_ns_cmd_write. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Tuesday, July 11, 2017 at 2:32 PM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Thanks James! The unit test files have been very helpful, and thanks for sending over those functions to review. To follow up, when using malloc to create a buffer and then calling the spdk_bs_io_write function, is the data actually getting written to the NVMe drive that is connected to the machine? Or, after writing to the blob, do I have to then use the spdk_dma_zmalloc to allocate space on the NVME drive and then actually write the drive using spdk_nvme_ns_cmd_write function? Thanks again for the help! Best, Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Tuesday, July 11, 2017 4:00:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Welcome to SPDK! A Blobstore “Hello World” example is a fantastic idea – definitely something to put on our Trello backlog (which I’ve done just now). You can see our SPDK Trello boards at trello.com/spdk – your suggestion is now in the Low Hanging Fruit column on the Things to Do board. Until that example is ready, I can point you to some of our unit test code that could help in meantime. test/unit/lib/blob/blob.c/blob_ut.c has a number of tests that show some very basic operations with the SPDK Blobstore. All of these tests use a simple malloc buffer to simulate the underlying block device – this code is in test/unit/lib/blob/bs_dev_common.c. The key functions to look at are: Initialize new blobstore => spdk_bs_init Load existing blobstore => spdk_bs_load Create blob => spdk_bs_md_create_blob Open blob => spdk_bs_md_open_blob Read from blob => spdk_bs_io_read_blob Write to blob => spdk_bs_io_write_blob Close blob => spdk_bs_md_close_blob Unload blobstore => spdk_bs_unload One last note as an FYI – a “logical volume store” or lvolstore is under development for SPDK. Lvolstore will provide the ability to dynamically partition a block device into smaller block devices using Blobstore for managing the on-disk metadata. You can see some additional details on the corresponding Trello board. Thanks, -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Tuesday, July 11, 2017 at 7:20 AM To: "spdk@lists.01.org<mailto:spdk@lists.01.org>" <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: [SPDK] SPDK Blob Store Hello, I am pretty new to the SPDK but am currently working on trying to implement a basic blobstore application for an NVMe Drive. It looks like all of the relevant code has already been written, but i do not see any example code for implementation. Are there any recommended resources or places to start? I am basically, just for the purpose of understanding, attempting something very similar to the hello_world example. However, I am trying to write/read/list/delete blobs instead of just a basic string. Is it really as easy as creating a spdk_blob type and then calling the corresponding functions from blobstore? Create blob -> spdk_bs_md_create_blob Read blob -> spdk_bs_md_open_blob Write blob -> _spdk_blob_persist delete blob -> spdk_bs_md_delete_blob Thanks, Neil

Will do, on my way out of town for a few days vacation and plan to get started when I get back. -Paul From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of Neil Kumar Sent: July 17, 2017 6:46 AM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] SPDK Blob Store Thanks Paul! Please keep me posted, would love to see what you come up with! Related question (just posted in IRC as well) - Is it possible to assign the blobid when a blob is created? I know its possible to create an xattr such as name, and associate it to a blob. However, it would be great to actually assign the blob ID ourselves. Thanks! -Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Luse, Paul E <paul.e.luse@intel.com<mailto:paul.e.luse@intel.com>> Sent: Friday, July 14, 2017 9:08:59 AM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, I'm going to start working on a 'hello world' type example for blobstor here soon, I believe I saw you on IRC not too long ago. I'm in there daily so feel free to jump in at any time with more questions. I've not done anything with this component before either so I'll be asking lots of questions there also which may benefit you as well. I'll work on beefing up the docs as I go as well... Thx Paul PS: If that wasn't use on IRC, we're on Freenode channel #spdk From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of Harris, James R Sent: July 12, 2017 8:53 AM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Hi Neil, It sounds like for your use case, you would want to create a separate blob for each dataset. We hear your request for more documentation in blob.h. There is always room for more and better documentation. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Wednesday, July 12, 2017 at 8:39 AM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Beautiful thanks! So the idea then is that multiple datasets (payloads) can potentially be stored in one blob? to follow up on that, If our use case involves datasets that range from 10-100MB, its probably best to allocate each dataset to its own unique blob? And finally, The documentation for the overall spdk is pretty awesome. I love that there is a description for every function and most types are very well defined. However, the blob.h documentation is really lacking. It would be awesome to have descriptions for the data structures being used as well as the functions that are being called (ie spdk_bs_dev, init_dev) Thanks again! -Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Wednesday, July 12, 2017 10:03:12 AM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Offset is the offset into the blob. Length is the length of the data being read/written starting at that offset. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Wednesday, July 12, 2017 at 6:59 AM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Awesome, Thanks James! In terms of the spdk_bs_io_read/write functions. can you clarify the use of the offset and length parameters? When I call this function, am I essentially specifying the actual offset within the cluster and the length of the blob? or is this an offset to where the data exists within the blob? Thanks, Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Tuesday, July 11, 2017 5:46:38 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, For the SPDK Blobstore, read and write operations to the underlying block device are abstracted through the spdk_bs_dev interface. For the unit tests, this spdk_bs_dev interface is just implemented with 4KB memcopies to/from a malloc buffer - no NVMe device. For real applications, the typical usage model would be implementing spdk_bs_dev with an SPDK block device (bdev). We have a blob_bdev helper library for this in lib/blob/bdev. With blob_bdev, any SPDK bdev can be plugged in - an NVMe bdev (either locally attached or remote using NVMe-oF), libaio, Ceph RBD, etc. But of course locally attached NVMe is the most popular choice here. To write data to a blob, first allocate your data buffer. spdk_dma_zmalloc is the best choice, to ensure the buffer is DMA safe (but you'll see for unit tests that buffers are either allocated using malloc or just off the stack, since spdk_bs_dev is not implemented with a device using DMA). Then call spdk_bs_io_read/write_blob. Blobstore will then convert your blob + page index into the block device LBA and issue a read/write call on the spdk_bs_dev interface. Then the spdk_bs_dev implementation handles the rest. If you choose to use the blob_bdev library, it will convert the read/write operation to the appropriate SPDK bdev API call. Alternatively, depending on your usage model, you may want to implement the spdk_bs_dev interface using the SPDK NVMe driver directly (and avoiding the SPDK bdev layer). In this case, you would translate the spdk_bs_dev write call to use spdk_nvme_ns_cmd_write. -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Tuesday, July 11, 2017 at 2:32 PM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] SPDK Blob Store Thanks James! The unit test files have been very helpful, and thanks for sending over those functions to review. To follow up, when using malloc to create a buffer and then calling the spdk_bs_io_write function, is the data actually getting written to the NVMe drive that is connected to the machine? Or, after writing to the blob, do I have to then use the spdk_dma_zmalloc to allocate space on the NVME drive and then actually write the drive using spdk_nvme_ns_cmd_write function? Thanks again for the help! Best, Neil ________________________________ From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Harris, James R <james.r.harris@intel.com<mailto:james.r.harris@intel.com>> Sent: Tuesday, July 11, 2017 4:00:33 PM To: Storage Performance Development Kit Subject: Re: [SPDK] SPDK Blob Store Hi Neil, Welcome to SPDK! A Blobstore "Hello World" example is a fantastic idea - definitely something to put on our Trello backlog (which I've done just now). You can see our SPDK Trello boards at trello.com/spdk - your suggestion is now in the Low Hanging Fruit column on the Things to Do board. Until that example is ready, I can point you to some of our unit test code that could help in meantime. test/unit/lib/blob/blob.c/blob_ut.c has a number of tests that show some very basic operations with the SPDK Blobstore. All of these tests use a simple malloc buffer to simulate the underlying block device - this code is in test/unit/lib/blob/bs_dev_common.c. The key functions to look at are: Initialize new blobstore => spdk_bs_init Load existing blobstore => spdk_bs_load Create blob => spdk_bs_md_create_blob Open blob => spdk_bs_md_open_blob Read from blob => spdk_bs_io_read_blob Write to blob => spdk_bs_io_write_blob Close blob => spdk_bs_md_close_blob Unload blobstore => spdk_bs_unload One last note as an FYI - a "logical volume store" or lvolstore is under development for SPDK. Lvolstore will provide the ability to dynamically partition a block device into smaller block devices using Blobstore for managing the on-disk metadata. You can see some additional details on the corresponding Trello board. Thanks, -Jim From: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> on behalf of Neil Kumar <nkumar@ocient.com<mailto:nkumar@ocient.com>> Reply-To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Date: Tuesday, July 11, 2017 at 7:20 AM To: "spdk@lists.01.org<mailto:spdk@lists.01.org>" <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: [SPDK] SPDK Blob Store Hello, I am pretty new to the SPDK but am currently working on trying to implement a basic blobstore application for an NVMe Drive. It looks like all of the relevant code has already been written, but i do not see any example code for implementation. Are there any recommended resources or places to start? I am basically, just for the purpose of understanding, attempting something very similar to the hello_world example. However, I am trying to write/read/list/delete blobs instead of just a basic string. Is it really as easy as creating a spdk_blob type and then calling the corresponding functions from blobstore? Create blob -> spdk_bs_md_create_blob Read blob -> spdk_bs_md_open_blob Write blob -> _spdk_blob_persist delete blob -> spdk_bs_md_delete_blob Thanks, Neil