On Tue, 2016-12-06 at 16:58 +0000, Walker, Benjamin wrote:
On Tue, 2016-12-06 at 16:35 +0000, Harris, James R wrote:
> > On Dec 6, 2016, at 5:49 AM, Kumaraparameshwaran Rathnavel <krath@
> > cloudsimple
> > .com> wrote:
> > The setup is the default setup of NVMf target with Direct
> > attached
> > Subsystem. The FIO plugin was at the target side and the IOPS
> > from SPDK was
> > found to be around 2.5K and the IOPS from kernel was found to be
> > 100k.
> Hi Param,
> Can you check the AcceptorPollRate setting in your configuration
> file? There
> was a recent fix to the default value for this parameter. You
> should set it
> to at least 10000 (10ms).
Just to reinforce here - this is the solution. We saw this internally
weeks ago. Increasing the AcceptorPollRate to at least 10ms (you can
go as much
as 1 second if you want, but the performance benefit decreases the
make it), or moving the accceptor to a different core using the
option will fix the problem.
To follow up on this - we've now root caused exactly why our acceptor
polling function was slowing down the system so much. Before, I had
suggested polling less often or doing the polling on a separate core.
After these two commits
that is no longer necessary. Any remotely reasonable value for AcceptorPollRate
and placing the Acceptor on any core will now yield excellent performance
results with SPDK.