I see, I'll reorg the code a bit. Thanks for your help!
On 2020/2/28, 12:02 AM, "Harris, James R" <james.r.harris(a)intel.com>
wrote:
Hi Niu,
Do you have the ability to call spdk_env_init() before setting the CPU affinity?
If not, you'll likely need to save the CPU affinity before calling
spdk_env_init(), and then set it back again after spdk_env_init() returns.
-Jim
On 2/27/20, 8:54 AM, "Niu, Yawei" <yawei.niu(a)intel.com> wrote:
Hi, Jim
Our threading model looks like: On server start, the service process creates all
service threads one by one, each thread function will set CPU affinity before doing
anything, and only the first created thread will call spdk_env_init().
So in my case, will the following created threads inherit the affinity set by
spdk_env_init() from the first thread? And I suppose calling spdk_unaffinitize_thread()
after spdk_env_init() could break our CPU affinity as well, so we need re-set affinity
after the spdk_unaffinitize_thread() call?
Thanks
-Niu
On 2020/2/27, 11:11 PM, "Harris, James R"
<james.r.harris(a)intel.com> wrote:
Hi Niu,
The core mask is used to specify which cores will be used for polling threads
managed by the SPDK/DPDK application framework. There really isn't a way to disable
the core mask, but there are some ways to minimize its effects.
I'm curious - in your environment, when do you create your DAOS threads?
And you create them from the same thread that is calling spdk_env_init()?
Note that the thread calling spdk_env_init() will be pinned to that single
core you've specified in your core mask. And threads created after spdk_env_init()
will inherit that affinity. You can call spdk_unaffinitize_thread() immediately after
spdk_env_init() returns to eliminate that affinity. Could you try that and see if it
helps when specifying core_mask 0x1 in your environment?
-Jim
On 2/27/20, 2:02 AM, "Niu, Yawei" <yawei.niu(a)intel.com>
wrote:
Hi,
We (DAOS) used to call spdk_env_init() with default ‘core_mask’ “0x1” to
initialize DPDK EAL, during recent tests, we realized that performance can be boosted a
lot by changing the ‘core_mask’ to some other value like ‘0x8’, so I’m wondering if it
because some sort of DPDK service thread was stealing CPU cycles from DAOS service thread,
and when the “core_mask” is changed, the DPDK thread is happen to be created on some
unused core? But I didn’t observe any suspicious thread by ‘ps’ command (with core_mask
‘0x1’ or ‘0x8’).
So my question is what this ‘core_mask’ is used for? I suppose it’s not
useful for our server (we have our own polling thread and user level scheduler), is there
any way to disable it? What’s the best practice of this ‘core_mask’ configuration for SPDK
apps?
Thanks
-Niu
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org
_______________________________________________
SPDK mailing list -- spdk(a)lists.01.org
To unsubscribe send an email to spdk-leave(a)lists.01.org