Hi Jiahao, I don't know off the top of my head but I can try and repro with one of the SPDK tools and see what I can see. In the meantime you can try "/scripts/setup.sh reset" followed by setup.sh again to see if that gives you a temp workaround Thx Paul On 7/14/20, 7:49 PM, &quot;lijh2015(a)mail.ustc.edu.cn&quot; <lijh2015(a)mail.ustc.edu.cn&gt; wrote: Hi SPDK team, I'm using SPDK (v20-01-pre) to evaluate I/OAT performance. There are some bugs in my benchmark program, but after I use ctrl+c to quit my program and restart it again, some error messages appeared: ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:00:04.0 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:00:04.1 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:00:04.2 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:00:04.3 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:40:04.0 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 ioat.c: 547:ioat_enum_cb: *ERROR*: ioat_attach() failed EAL: Requested device 0000:40:04.1 cannot be used ioat.c: 480:ioat_channel_start: *ERROR*: could not start channel: status = 0x3 error = 0 I checked the code in lib/ioat/ioat.c, it seems is_ioat_idle always returns negative results. What should I do to deal with this problem without rebooting my server? Or, at least, how to prevent it? Thanks, Jiahao Li _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org

Hi Jiahao, Well that’s good news. I did try ctrl-c exit from both ioat_perf and the new accel_perf that I'm working on now and they both worked fine. BTW, accel_perf is being developed as the perf tool for the new (renamed and expanded) acceleration framework that you might be interested in. It's not quite feature complete yet but for the upcoming release in just a few weeks the accel framework will include support for IOAT, DSA (next gen IOAT not yet available in Si but once it is it will be a huge performance boost, plus new features) as well as sw implementations for all the features so that the framework API can be used in any environment. Let me know if you'd like more info on any of that. Now, wrt your IOAT issue, are you saying that following simply using a bogus address in a command that you're in a state where powering down the machine and powering it that some channels are still bad? Please confirm, that sounds "odd" for sure. Also please confirm that you power cycled, not just rebooted. Thx Paul On 7/16/20, 4:23 AM, &quot;lijh2015(a)mail.ustc.edu.cn&quot; <lijh2015(a)mail.ustc.edu.cn&gt; wrote: Hi Paul, Thanks for your advice. But that didn't seem to work. The output of /scripts/setup.sh is same as normal In fact, I found out how such error happens. It's because I accidentally launch ioat copy on system malloc-ed memory space, I've already fixed that in my code. But there are still some errors around. After reboot, and run scripts/setup.sh again, there are still 4 channels (1 channel from each process) cannot be attached: E0716 19:02:24.496325 189532 ioat.c:480] [SPDK/ioat_channel_start] could not start channel: status = 0x3 error = 0 E0716 19:02:24.496567 189532 ioat.c:547] [SPDK/ioat_enum_cb] ioat_attach() failed EAL: Requested device 0000:00:04.4 cannot be used E0716 19:02:24.527472 189532 ioat.c:480] [SPDK/ioat_channel_start] could not start channel: status = 0x3 error = 0 E0716 19:02:24.527948 189532 ioat.c:547] [SPDK/ioat_enum_cb] ioat_attach() failed EAL: Requested device 0000:40:04.4 cannot be used E0716 19:02:24.558609 189532 ioat.c:480] [SPDK/ioat_channel_start] could not start channel: status = 0x3 error = 0 E0716 19:02:24.559036 189532 ioat.c:547] [SPDK/ioat_enum_cb] ioat_attach() failed EAL: Requested device 0000:80:04.4 cannot be used E0716 19:02:24.589241 189532 ioat.c:480] [SPDK/ioat_channel_start] could not start channel: status = 0x3 error = 0 E0716 19:02:24.589682 189532 ioat.c:547] [SPDK/ioat_enum_cb] ioat_attach() failed EAL: Requested device 0000:c0:04.4 cannot be used The other channels are working fine now, but these errors are strange. Thanks, Jiahao Li _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org

You bet! Here's the main announcement https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator and there are several links at the bottom, including the spec. Right now I've done all the development using a simulator that is not publicly available but as soon as this launches the support will be there in SPDK. Here's the bottom of the current patch chain that finishes off all planned capabilities to add to SPDK https://review.spdk.io/gerrit/c/spdk/spdk/+/3319/1 and currently there is basic support for copy and already introduced last release. This release adds quite a bit of functionality. Next release we'll start working on using the accel_fw within various SPDK modules where it makes sense (CRC32C, compare, fill, copy, etc) -Paul On 7/16/20, 11:20 AM, &quot;lijh2015(a)mail.ustc.edu.cn&quot; <lijh2015(a)mail.ustc.edu.cn&gt; wrote: Hi Paul, A power cycle solved my problem. Thanks again for your help! Being new to IOAT technology, I am interested to learn how powerful the next gen IOAT technology will be, and what performance gain does the new accel framework have. Please tell me more about them. Thanks, Jiahao Li _______________________________________________ SPDK mailing list -- spdk(a)lists.01.org To unsubscribe send an email to spdk-leave(a)lists.01.org