Hi Li, The acceptor is running on core 0 on default, but when the connection is in ffp phase, it will be transferred to other reactors for connection handling, see the code in spdk_iscsi_conn_migration in lib/iscsi/iscsi.c. According to the code, you can see it is also related with the portal group info (cpu mask). Thanks. Best Regards Ziye Yang From: SPDK [mailto:spdk-bounces@lists.01.org] On Behalf Of 松本周平 / MATSUMOTO，SHUUHEI Sent: Tuesday, July 31, 2018 2:06 PM To: Storage Performance Development Kit <spdk(a)lists.01.org&gt; Subject: Re: [SPDK] How to utilize multiple threads? Hi Li, But the acceptor is only running on the core 0, right? Yes. But if what you see is in acceptor.c, it may be the acceptor dedicated to check start of communication by initiator. Regardless of this, you are correct and each iSCSI connection is handled by single CPU core because SPDK is highly efficient and lockless. By increasing number of iSCSI connections, all CPU cores will be used. The following by Ziye Yang is not maintained now but it will be helpful if you read. https://review.gerrithub.io/#/c/spdk/spdk/+/392733/18/doc/iscsi_pg.md Thanks, Shuhei ________________________________ 差出人: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> が Li Feng <fengli@smartx.com<mailto:fengli@smartx.com>> の代理で送信 送信日�: 2018年7月31日 14:32 宛先: Storage Performance Development Kit 件名: [!][SPDK] How to utilize multiple threads? Hi experts, When I'm digging into SPDK iscsi source codes, I have some doubt about the threads/reactors. Look at this example. Start iscsi target on core 0-7: ``` app/iscsi_tgt/iscsi_tgt -L all -d 1 -m 0xff ``` I can see 8 threads which are polling. But the acceptor is only running on the core 0, right? This means all IOs are processed on core 0, no matter how many bdevs are connecting ? I can see sending NOP IN is a poller running on all reactors. But this is not about read/write. So how to utilize the other threads to maximum the iops and throughput? Thanks in advance. The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

Hi Li, As long as I know, we don't have any activity in SCSI persistent reservation yet. Adding the item into https://trello.com/b/P5xBO7UR/things-to-do may be a good small start. [https://s3.amazonaws.com/trello/images/og/trello-icon.png?v=2013-08-15]&l... Things To Do - Trello<https://trello.com/b/P5xBO7UR/things-to-do> trello.com Trello is the visual collaboration platform that gives teams perspective on projects. Use Trello to collaborate, communicate and coordinate on all of your projects. Thanks, Shuhei ________________________________ 差出人: SPDK <spdk-bounces(a)lists.01.org&gt; が Li Feng <fengli(a)smartx.com&gt; の代理で送信 送信日�: 2018年7月31日 16:22:24 宛先: Storage Performance Development Kit 件名: [!]Re: [SPDK] How to utilize multiple threads? Thank you, shuuhei and Ziye. Your response are so fast. The spdk_iscsi_conn_migration will schedule a reactor to handle the new connection. I have understood now. As I know, the openiscsi initiator only uses one session/connection when connecting to one target, even though it has multiple luns. So it will process IOs on the same core when they belong to the same target. Another question, would you have a plan to support SCSI PR(persistent reservation)? Thanks, Feng Li [https://docs.google.com/a/smartx.com/uc?id=0B6v_9z-4gTszRGFCWVFMSGlsMVE&a...] 手机：13401157876 Web：http://www.smartx.com/<https://clicktime.symantec.com/a/1/NIaW0nHC... 地址：北京市海淀区科学院南路2号融科资讯中心C座北楼712 2018-07-31 14:50 GMT+08:00 Yang, Ziye <ziye.yang@intel.com<mailto:ziye.yang@intel.com>>: Hi Li, The acceptor is running on core 0 on default, but when the connection is in ffp phase, it will be transferred to other reactors for connection handling, see the code in spdk_iscsi_conn_migration in lib/iscsi/iscsi.c. According to the code, you can see it is also related with the portal group info (cpu mask). Thanks. Best Regards Ziye Yang From: SPDK [mailto:spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>] On Behalf Of 松本周平 / MATSUMOTO，SHUUHEI Sent: Tuesday, July 31, 2018 2:06 PM To: Storage Performance Development Kit <spdk@lists.01.org<mailto:spdk@lists.01.org>> Subject: Re: [SPDK] How to utilize multiple threads? Hi Li, But the acceptor is only running on the core 0, right? Yes. But if what you see is in acceptor.c, it may be the acceptor dedicated to check start of communication by initiator. Regardless of this, you are correct and each iSCSI connection is handled by single CPU core because SPDK is highly efficient and lockless. By increasing number of iSCSI connections, all CPU cores will be used. The following by Ziye Yang is not maintained now but it will be helpful if you read. https://review.gerrithub.io/#/c/spdk/spdk/+/392733/18/doc/iscsi_pg.md<... Thanks, Shuhei ________________________________ 差出人: SPDK <spdk-bounces@lists.01.org<mailto:spdk-bounces@lists.01.org>> が Li Feng <fengli@smartx.com<mailto:fengli@smartx.com>> の代理で送信 送信日�: 2018年7月31日 14:32 宛先: Storage Performance Development Kit 件名: [!][SPDK] How to utilize multiple threads? Hi experts, When I'm digging into SPDK iscsi source codes, I have some doubt about the threads/reactors. Look at this example. Start iscsi target on core 0-7: ``` app/iscsi_tgt/iscsi_tgt -L all -d 1 -m 0xff ``` I can see 8 threads which are polling. But the acceptor is only running on the core 0, right? This means all IOs are processed on core 0, no matter how many bdevs are connecting ? I can see sending NOP IN is a poller running on all reactors. But this is not about read/write. So how to utilize the other threads to maximum the iops and throughput? Thanks in advance. The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权. _______________________________________________ SPDK mailing list SPDK@lists.01.org<mailto:SPDK@lists.01.org> https://lists.01.org/mailman/listinfo/spdk<https://clicktime.symantec.... The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.

Hi Li, Sorry, some help may be needed not by me. I’ll add it this time. Thanks Shuhei 2018年7月31日(火) 19:34 Li Feng <fengli(a)smartx.com&gt;:

Hi Jim, Thank you. This is my Trello ID: vonnyfly Thanks, Feng Li 手机：13401157876 Web：http://www.smartx.com/ 地址：北京市海淀区科学院南路2号融科资讯中心C座北楼712 2018-07-31 22:55 GMT+08:00 Harris, James R <james.r.harris(a)intel.com&gt;: -- The SmartX email address is only for business purpose. Any sent message that is not related to the business is not authorized or permitted by SmartX. 本邮箱为北京志凌海纳科技有限公司（SmartX）工作邮箱. 如本邮箱发出的邮件与工作无关,该邮件未得到本公司任何的明示或默示的授权.