3:06 a.m.
hello, sorry if this is not the correct place to send such an email. I'm
writing a distributed storage system, which uses SPDK as the data
plane. My purpose is using it as a persistent volume for k8s or a
backend driver for openstack cinder. It is a personal open source
project. I hope to find someone who is interested in it too.
My idea is creating a virtual disk array, using SPDK as the disk
array's controller and disk. The SPDK on the disk exports the physical
disk to the controller. The SPDK on the controller aggregates multiple
disks to a single volume and exports to a host. We could set raid level
or encryption on the controller.
Now I have a workable version, and my next step is writing a CSI
plugin for it, let it work on k8s. Hope to hear some feedback about
it.
Here is the home page of the project:
https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
Best regards.
3:25 a.m.
Hello Peng Yu,
Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1], looks similar to
your requirement.
FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
[1] https://github.com/spdk/spdk-csi
[2] https://youtu.be/9I2b1z016Ns
Yibo
On 9/24/20 3:06 PM, peng yu wrote:
> hello, sorry if this is not the correct place to send such an email. I'm
> writing a distributed storage system, which uses SPDK as the data
> plane. My purpose is using it as a persistent volume for k8s or a
> backend driver for openstack cinder. It is a personal open source
> project. I hope to find someone who is interested in it too.
>
> My idea is creating a virtual disk array, using SPDK as the disk
> array's controller and disk. The SPDK on the disk exports the physical
> disk to the controller. The SPDK on the controller aggregates multiple
> disks to a single volume and exports to a host. We could set raid level
> or encryption on the controller.
>
> Now I have a workable version, and my next step is writing a CSI
> plugin for it, let it work on k8s. Hope to hear some feedback about
> it.
>
> Here is the home page of the project:
> https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
>
> Best regards.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>
3:29 a.m.
Hi, Peng Yu
It's a very interesting project.
For your planning CSI, you may reference https://github.com/spdk/spdk-csi created by Yibo.
Maybe it can help you reduce some workload.
There is also one distributed NVMf innovation "Add Adaptive Distributed NVMe-oF
Namespaces" https://review.spdk.io/gerrit/c/spdk/spdk/+/4325 by Peterson inside SPDK
gerrit, I'm sure you will be interested at it. Seems Peteson also has a presentation
about it at SDC20.
--Thanks
From Xiaodong
-----Original Message-----
From: Yibo Cai <yibo.cai(a)arm.com>
Sent: Thursday, September 24, 2020 3:25 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>; peng yu
<yupeng0921(a)gmail.com>
Subject: [SPDK] Re: a distributed storage system base on SPDK
Hello Peng Yu,
Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1], looks similar to
your requirement.
FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
[1] https://github.com/spdk/spdk-csi
[2] https://youtu.be/9I2b1z016Ns
Yibo
On 9/24/20 3:06 PM, peng yu wrote:
hello, sorry if this is not the correct place to send such an email.
I'm writing a distributed storage system, which uses SPDK as the data
plane. My purpose is using it as a persistent volume for k8s or a
backend driver for openstack cinder. It is a personal open source
project. I hope to find someone who is interested in it too.
My idea is creating a virtual disk array, using SPDK as the disk
array's controller and disk. The SPDK on the disk exports the physical
disk to the controller. The SPDK on the controller aggregates multiple
disks to a single volume and exports to a host. We could set raid
level or encryption on the controller.
Now I have a workable version, and my next step is writing a CSI
plugin for it, let it work on k8s. Hope to hear some feedback about
it.
Here is the home page of the project:
https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
Best regards.
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
12:32 p.m.
The spdk-csi is a very good reference for me. I'm looking Yibo's
youtube video about it.
About ADNN, I only find a description here:
https://www.snia.org/events/storage-developer/2020/abstracts
Besides reading the code, can I find any introduction about it?
Best regards.
On Thu, Sep 24, 2020 at 12:29 AM Liu, Xiaodong <xiaodong.liu(a)intel.com> wrote:
>
> Hi, Peng Yu
>
> It's a very interesting project.
> For your planning CSI, you may reference https://github.com/spdk/spdk-csi created by
Yibo. Maybe it can help you reduce some workload.
>
> There is also one distributed NVMf innovation "Add Adaptive Distributed NVMe-oF
Namespaces" https://review.spdk.io/gerrit/c/spdk/spdk/+/4325 by Peterson inside SPDK
gerrit, I'm sure you will be interested at it. Seems Peteson also has a presentation
about it at SDC20.
>
>
> --Thanks
> From Xiaodong
>
> -----Original Message-----
> From: Yibo Cai <yibo.cai(a)arm.com>
> Sent: Thursday, September 24, 2020 3:25 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>; peng yu
<yupeng0921(a)gmail.com>
> Subject: [SPDK] Re: a distributed storage system base on SPDK
>
> Hello Peng Yu,
>
> Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1], looks
similar to your requirement.
> FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
>
> [1] https://github.com/spdk/spdk-csi
> [2] https://youtu.be/9I2b1z016Ns
>
> Yibo
>
> On 9/24/20 3:06 PM, peng yu wrote:
> > hello, sorry if this is not the correct place to send such an email.
> > I'm writing a distributed storage system, which uses SPDK as the data
> > plane. My purpose is using it as a persistent volume for k8s or a
> > backend driver for openstack cinder. It is a personal open source
> > project. I hope to find someone who is interested in it too.
> >
> > My idea is creating a virtual disk array, using SPDK as the disk
> > array's controller and disk. The SPDK on the disk exports the physical
> > disk to the controller. The SPDK on the controller aggregates multiple
> > disks to a single volume and exports to a host. We could set raid
> > level or encryption on the controller.
> >
> > Now I have a workable version, and my next step is writing a CSI
> > plugin for it, let it work on k8s. Hope to hear some feedback about
> > it.
> >
> > Here is the home page of the project:
> > https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
> >
> > Best regards.
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> > spdk-leave(a)lists.01.org
> >
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
12:39 p.m.
Try @Scott Peterson on slack... (not sure he monitors this email very often)
On 9/24/20, 9:33 AM, "peng yu" <yupeng0921(a)gmail.com> wrote:
The spdk-csi is a very good reference for me. I'm looking Yibo's
youtube video about it.
About ADNN, I only find a description here:
https://www.snia.org/events/storage-developer/2020/abstracts
Besides reading the code, can I find any introduction about it?
Best regards.
On Thu, Sep 24, 2020 at 12:29 AM Liu, Xiaodong <xiaodong.liu(a)intel.com> wrote:
Hi, Peng Yu
It's a very interesting project.
For your planning CSI, you may reference https://github.com/spdk/spdk-csi created by
Yibo. Maybe it can help you reduce some workload.
There is also one distributed NVMf innovation "Add Adaptive Distributed NVMe-oF
Namespaces" https://review.spdk.io/gerrit/c/spdk/spdk/+/4325 by Peterson inside SPDK
gerrit, I'm sure you will be interested at it. Seems Peteson also has a presentation
about it at SDC20.
--Thanks
From Xiaodong
-----Original Message-----
From: Yibo Cai <yibo.cai(a)arm.com>
Sent: Thursday, September 24, 2020 3:25 PM
To: Storage Performance Development Kit <spdk(a)lists.01.org>; peng yu
<yupeng0921(a)gmail.com>
Subject: [SPDK] Re: a distributed storage system base on SPDK
Hello Peng Yu,
Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1], looks similar to
your requirement.
FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
[1] https://github.com/spdk/spdk-csi
[2] https://youtu.be/9I2b1z016Ns
Yibo
On 9/24/20 3:06 PM, peng yu wrote:
> hello, sorry if this is not the correct place to send such an email.
> I'm writing a distributed storage system, which uses SPDK as the data
> plane. My purpose is using it as a persistent volume for k8s or a
> backend driver for openstack cinder. It is a personal open source
> project. I hope to find someone who is interested in it too.
>
> My idea is creating a virtual disk array, using SPDK as the disk
> array's controller and disk. The SPDK on the disk exports the physical
> disk to the controller. The SPDK on the controller aggregates multiple
> disks to a single volume and exports to a host. We could set raid
> level or encryption on the controller.
>
> Now I have a workable version, and my next step is writing a CSI
> plugin for it, let it work on k8s. Hope to hear some feedback about
> it.
>
> Here is the home page of the project:
> https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
>
> Best regards.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> spdk-leave(a)lists.01.org
>
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
12:54 p.m.
If you’re an SDC attendee you should be able to see the presentation video and slides,
which provide an introduction. If not, I can send you a copy.
This should all be in the README, but I pushed the patch without that finished so the code
would be visible before SDC. I’ll update the patch when that’s finished.
--- sdp
On Sep 24, 2020, at 9:32 AM, peng yu <yupeng0921(a)gmail.com>
wrote:
The spdk-csi is a very good reference for me. I'm looking Yibo's
youtube video about it.
About ADNN, I only find a description here:
https://www.snia.org/events/storage-developer/2020/abstracts
Besides reading the code, can I find any introduction about it?
Best regards.
> On Thu, Sep 24, 2020 at 12:29 AM Liu, Xiaodong <xiaodong.liu(a)intel.com>
wrote:
>
> Hi, Peng Yu
>
> It's a very interesting project.
> For your planning CSI, you may reference https://github.com/spdk/spdk-csi created by
Yibo. Maybe it can help you reduce some workload.
>
> There is also one distributed NVMf innovation "Add Adaptive Distributed NVMe-oF
Namespaces" https://review.spdk.io/gerrit/c/spdk/spdk/+/4325 by Peterson inside SPDK
gerrit, I'm sure you will be interested at it. Seems Peteson also has a presentation
about it at SDC20.
>
>
> --Thanks
> From Xiaodong
>
> -----Original Message-----
> From: Yibo Cai <yibo.cai(a)arm.com>
> Sent: Thursday, September 24, 2020 3:25 PM
> To: Storage Performance Development Kit <spdk(a)lists.01.org>; peng yu
<yupeng0921(a)gmail.com>
> Subject: [SPDK] Re: a distributed storage system base on SPDK
>
> Hello Peng Yu,
>
> Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1], looks
similar to your requirement.
> FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
>
> [1] https://github.com/spdk/spdk-csi
> [2] https://youtu.be/9I2b1z016Ns
>
> Yibo
>
>> On 9/24/20 3:06 PM, peng yu wrote:
>> hello, sorry if this is not the correct place to send such an email.
>> I'm writing a distributed storage system, which uses SPDK as the data
>> plane. My purpose is using it as a persistent volume for k8s or a
>> backend driver for openstack cinder. It is a personal open source
>> project. I hope to find someone who is interested in it too.
>>
>> My idea is creating a virtual disk array, using SPDK as the disk
>> array's controller and disk. The SPDK on the disk exports the physical
>> disk to the controller. The SPDK on the controller aggregates multiple
>> disks to a single volume and exports to a host. We could set raid
>> level or encryption on the controller.
>>
>> Now I have a workable version, and my next step is writing a CSI
>> plugin for it, let it work on k8s. Hope to hear some feedback about
>> it.
>>
>> Here is the home page of the project:
>> https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
>>
>> Best regards.
>> _______________________________________________
>> SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
>> spdk-leave(a)lists.01.org
>>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
4:16 p.m.
I'm not an SCD attendee. it is appreciated if you can send me a copy.
Best regards.
On Thu, Sep 24, 2020 at 9:54 AM Peterson, Scott D
<scott.d.peterson(a)intel.com> wrote:
>
> If you’re an SDC attendee you should be able to see the presentation video and
slides, which provide an introduction. If not, I can send you a copy.
>
> This should all be in the README, but I pushed the patch without that finished so the
code would be visible before SDC. I’ll update the patch when that’s finished.
>
> --- sdp
>
>
> > On Sep 24, 2020, at 9:32 AM, peng yu <yupeng0921(a)gmail.com> wrote:
> >
> > The spdk-csi is a very good reference for me. I'm looking Yibo's
> > youtube video about it.
> > About ADNN, I only find a description here:
> > https://www.snia.org/events/storage-developer/2020/abstracts
> > Besides reading the code, can I find any introduction about it?
> >
> > Best regards.
> >
> >> On Thu, Sep 24, 2020 at 12:29 AM Liu, Xiaodong
<xiaodong.liu(a)intel.com> wrote:
> >>
> >> Hi, Peng Yu
> >>
> >> It's a very interesting project.
> >> For your planning CSI, you may reference https://github.com/spdk/spdk-csi
created by Yibo. Maybe it can help you reduce some workload.
> >>
> >> There is also one distributed NVMf innovation "Add Adaptive Distributed
NVMe-oF Namespaces" https://review.spdk.io/gerrit/c/spdk/spdk/+/4325 by Peterson
inside SPDK gerrit, I'm sure you will be interested at it. Seems Peteson also has a
presentation about it at SDC20.
> >>
> >>
> >> --Thanks
> >> From Xiaodong
> >>
> >> -----Original Message-----
> >> From: Yibo Cai <yibo.cai(a)arm.com>
> >> Sent: Thursday, September 24, 2020 3:25 PM
> >> To: Storage Performance Development Kit <spdk(a)lists.01.org>; peng yu
<yupeng0921(a)gmail.com>
> >> Subject: [SPDK] Re: a distributed storage system base on SPDK
> >>
> >> Hello Peng Yu,
> >>
> >> Regarding SPDK CSI plugin for K8s, we have created SPDK-CSI project [1],
looks similar to your requirement.
> >> FWIW, I just gave a talk about this project at SDC20 [2]. Hope it helps.
> >>
> >> [1] https://github.com/spdk/spdk-csi
> >> [2] https://youtu.be/9I2b1z016Ns
> >>
> >> Yibo
> >>
> >>> On 9/24/20 3:06 PM, peng yu wrote:
> >>> hello, sorry if this is not the correct place to send such an email.
> >>> I'm writing a distributed storage system, which uses SPDK as the
data
> >>> plane. My purpose is using it as a persistent volume for k8s or a
> >>> backend driver for openstack cinder. It is a personal open source
> >>> project. I hope to find someone who is interested in it too.
> >>>
> >>> My idea is creating a virtual disk array, using SPDK as the disk
> >>> array's controller and disk. The SPDK on the disk exports the
physical
> >>> disk to the controller. The SPDK on the controller aggregates multiple
> >>> disks to a single volume and exports to a host. We could set raid
> >>> level or encryption on the controller.
> >>>
> >>> Now I have a workable version, and my next step is writing a CSI
> >>> plugin for it, let it work on k8s. Hope to hear some feedback about
> >>> it.
> >>>
> >>> Here is the home page of the project:
> >>> https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
> >>>
> >>> Best regards.
> >>> _______________________________________________
> >>> SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to
> >>> spdk-leave(a)lists.01.org
> >>>
> >> _______________________________________________
> >> SPDK mailing list -- spdk(a)lists.01.org
> >> To unsubscribe send an email to spdk-leave(a)lists.01.org
> > _______________________________________________
> > SPDK mailing list -- spdk(a)lists.01.org
> > To unsubscribe send an email to spdk-leave(a)lists.01.org
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
8:05 p.m.
This should all be in the README, but I pushed the patch without that
finished so the code
would be visible before SDC. I’ll update the patch when that’s finished.
The ADNN patch now includes a README:
https://review.spdk.io/gerrit/c/spdk/spdk/+/4325/2/module/bdev/redirector...
--- sdp
11:39 a.m.
Hi Ping,
This is indeed the right place to send an email ( I applaud you for such a complex and
large endeavor! I'm sure you saw the other replies as well, as you work on your
project please feel free to send your SPDK questions and comments here (or in Slack) and,
of course, if you have any relevant SPDK contributions as you make forward progress
we'd love to see those as well.
Thanks!
Paul
On 9/24/20, 12:07 AM, "peng yu" <yupeng0921(a)gmail.com> wrote:
hello, sorry if this is not the correct place to send such an email. I'm
writing a distributed storage system, which uses SPDK as the data
plane. My purpose is using it as a persistent volume for k8s or a
backend driver for openstack cinder. It is a personal open source
project. I hope to find someone who is interested in it too.
My idea is creating a virtual disk array, using SPDK as the disk
array's controller and disk. The SPDK on the disk exports the physical
disk to the controller. The SPDK on the controller aggregates multiple
disks to a single volume and exports to a host. We could set raid level
or encryption on the controller.
Now I have a workable version, and my next step is writing a CSI
plugin for it, let it work on k8s. Hope to hear some feedback about
it.
Here is the home page of the project:
https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
Best regards.
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
12:42 p.m.
I submitted patches about SPDK "auto examine", those are for letting
my project work correctly. I have a plan to submit additional patches
to SPDK, one is supporting the concat (or linear) logical volume,
another is raid1. Both of them are for this project. We have discussed
them in other email threads about 1 year ago. I would work on them in
the future.
Best regards.
On Thu, Sep 24, 2020 at 8:39 AM Luse, Paul E <paul.e.luse(a)intel.com> wrote:
>
> Hi Ping,
>
> This is indeed the right place to send an email ( I applaud you for such a complex
and large endeavor! I'm sure you saw the other replies as well, as you work on your
project please feel free to send your SPDK questions and comments here (or in Slack) and,
of course, if you have any relevant SPDK contributions as you make forward progress
we'd love to see those as well.
>
> Thanks!
> Paul
>
> On 9/24/20, 12:07 AM, "peng yu" <yupeng0921(a)gmail.com> wrote:
>
> hello, sorry if this is not the correct place to send such an email. I'm
> writing a distributed storage system, which uses SPDK as the data
> plane. My purpose is using it as a persistent volume for k8s or a
> backend driver for openstack cinder. It is a personal open source
> project. I hope to find someone who is interested in it too.
>
> My idea is creating a virtual disk array, using SPDK as the disk
> array's controller and disk. The SPDK on the disk exports the physical
> disk to the controller. The SPDK on the controller aggregates multiple
> disks to a single volume and exports to a host. We could set raid level
> or encryption on the controller.
>
> Now I have a workable version, and my next step is writing a CSI
> plugin for it, let it work on k8s. Hope to hear some feedback about
> it.
>
> Here is the home page of the project:
> https://virtual-disk-array.readthedocs.io/en/latest/Introduction.html
>
> Best regards.
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
>
> _______________________________________________
> SPDK mailing list -- spdk(a)lists.01.org
> To unsubscribe send an email to spdk-leave(a)lists.01.org
162
days inactive
163
days old
9 comments
6 participants
participants (6)
-
Liu, Xiaodong -
Luse, Paul E -
peng yu -
Peterson, Scott D -
scott.d.peterson＠intel.com
-
Yibo Cai