[sched/fair] 3c29e651e1: hackbench.throughput -15.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -15.2% regression of hackbench.throughput due to commit:
commit: 3c29e651e16dd3b3179cfb2d055ee9538e37515c ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: hackbench
on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:
nr_threads: 100%
mode: threads
ipc: pipe
cpufreq_governor: performance
ucode: 0xca
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc...
In addition to that, the commit also has significant impact on the following tests:
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -11.3% regression |
| test machine | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | iterations=18 |
| | mode=threads |
| | nr_threads=1600% |
| | ucode=0x43 |
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -10.6% regression |
| test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | mode=process |
| | nr_threads=50% |
| | ucode=0xb000038 |
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -6.8% regression |
| test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | mode=threads |
| | nr_threads=100% |
| | ucode=0xb8 |
+------------------+-----------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-20191114.cgz/lkp-cfl-e1/hackbench/0xca
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
%stddev %change %stddev
\ | \
69492 ± 3% -15.2% 58899 hackbench.throughput
7.727e+08 ± 7% +38.3% 1.069e+09 ± 2% hackbench.time.involuntary_context_switches
131708 ± 4% -15.4% 111382 hackbench.time.minor_page_faults
684.98 ± 2% -2.5% 667.72 hackbench.time.user_time
1.755e+09 ± 2% +11.2% 1.953e+09 hackbench.time.voluntary_context_switches
4.32e+08 ± 4% -15.6% 3.648e+08 hackbench.workload
186.50 ± 12% -16.6% 155.50 ± 11% interrupts.TLB:TLB_shootdowns
4147892 ± 5% +19.6% 4961643 vmstat.system.cs
0.72 ± 10% -0.1 0.58 ± 17% mpstat.cpu.all.idle%
0.00 ±156% +0.0 0.01 ± 62% mpstat.cpu.all.soft%
1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.active_objs
1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.num_objs
40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_hit
40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_local
40724029 ± 10% -34.5% 26661546 proc-vmstat.pgalloc_normal
763906 ± 2% -7.5% 706781 proc-vmstat.pgfault
40701242 ± 10% -34.6% 26637947 proc-vmstat.pgfree
1636058 ± 11% -45.0% 899629 ± 11% turbostat.C1
0.10 ± 7% -0.0 0.07 ± 7% turbostat.C1%
15201 ± 25% +113.3% 32422 ± 8% turbostat.C1E
0.01 ± 57% +0.0 0.03 ± 13% turbostat.C1E%
9093 ± 11% +217.2% 28841 ± 11% turbostat.C3
0.00 ±173% +0.0 0.03 ± 13% turbostat.C3%
16977 ± 2% -92.8% 1222 ± 98% turbostat.C8
0.17 ± 4% -0.2 0.01 ±110% turbostat.C8%
0.13 -100.0% 0.00 turbostat.CPU%c7
9623106 ± 5% -35.1% 6241651 ± 8% cpuidle.C1.time
1636668 ± 11% -45.0% 899926 ± 11% cpuidle.C1.usage
899494 ± 33% +272.6% 3351412 ± 5% cpuidle.C1E.time
15327 ± 25% +112.6% 32592 ± 8% cpuidle.C1E.usage
435674 ± 20% +643.5% 3239351 ± 14% cpuidle.C3.time
9179 ± 11% +215.2% 28933 ± 11% cpuidle.C3.usage
16850924 ± 2% -94.2% 979516 ± 86% cpuidle.C8.time
17150 ± 2% -92.0% 1378 ± 88% cpuidle.C8.usage
6022328 ± 18% -57.1% 2584773 ± 7% cpuidle.POLL.time
5339703 ± 19% -60.1% 2130089 ± 7% cpuidle.POLL.usage
14.06 ± 25% +64.0% 23.07 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.min
25.10 ± 5% -15.8% 21.12 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.stddev
13339 ± 24% +45.7% 19433 ± 7% sched_debug.cfs_rq:/.runnable_weight.min
166.51 ± 24% +48.9% 247.86 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.min
253.56 ± 2% -12.3% 222.30 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev
159602 ± 23% +55.6% 248411 ± 6% sched_debug.cpu.avg_idle.max
51567 ± 24% +49.1% 76905 ± 11% sched_debug.cpu.avg_idle.stddev
11576 ± 5% -9.5% 10478 sched_debug.cpu.curr->pid.avg
9976 ± 8% -16.8% 8305 ± 9% sched_debug.cpu.curr->pid.min
4.81 ± 15% +43.3% 6.89 ± 8% sched_debug.cpu.nr_running.min
6.68 ± 4% -17.0% 5.54 ± 8% sched_debug.cpu.nr_running.stddev
76003928 ± 2% +21.0% 91975904 sched_debug.cpu.nr_switches.avg
78115404 ± 2% +23.4% 96397551 sched_debug.cpu.nr_switches.max
74275755 ± 2% +19.0% 88359903 sched_debug.cpu.nr_switches.min
1043054 ± 15% +83.9% 1917752 ± 10% sched_debug.cpu.nr_switches.stddev
-685.26 +64.8% -1129 sched_debug.cpu.nr_uninterruptible.min
15470 ± 9% -31.0% 10674 ± 9% softirqs.CPU0.SCHED
13756 ± 10% -40.8% 8140 ± 4% softirqs.CPU1.SCHED
13645 ± 11% -39.7% 8227 ± 3% softirqs.CPU10.SCHED
14065 ± 12% -38.8% 8609 ± 2% softirqs.CPU11.SCHED
13948 ± 11% -39.9% 8381 ± 6% softirqs.CPU12.SCHED
13823 ± 12% -39.7% 8332 ± 9% softirqs.CPU13.SCHED
73323 +13.4% 83165 ± 11% softirqs.CPU14.RCU
13872 ± 12% -36.5% 8809 ± 7% softirqs.CPU14.SCHED
13726 ± 12% -40.4% 8175 ± 4% softirqs.CPU15.SCHED
13780 ± 11% -34.8% 8987 ± 12% softirqs.CPU2.SCHED
13895 ± 10% -40.5% 8266 ± 6% softirqs.CPU3.SCHED
221879 +13.4% 251589 ± 14% softirqs.CPU3.TIMER
13873 ± 10% -40.4% 8262 ± 6% softirqs.CPU4.SCHED
72625 +12.0% 81355 ± 10% softirqs.CPU5.RCU
13876 ± 13% -41.4% 8137 ± 3% softirqs.CPU5.SCHED
14118 ± 12% -40.8% 8354 ± 3% softirqs.CPU6.SCHED
13503 ± 12% -40.0% 8099 softirqs.CPU7.SCHED
13549 ± 11% -38.2% 8374 ± 8% softirqs.CPU8.SCHED
14273 ± 10% -40.1% 8548 ± 6% softirqs.CPU9.SCHED
223184 ± 11% -38.9% 136383 ± 5% softirqs.SCHED
55.80 +9.9% 61.34 perf-stat.i.MPKI
1.32 +0.1 1.39 perf-stat.i.branch-miss-rate%
1.066e+08 +3.7% 1.105e+08 perf-stat.i.branch-misses
0.15 ± 8% -0.0 0.11 ± 3% perf-stat.i.cache-miss-rate%
2521584 ± 9% -26.5% 1854353 ± 3% perf-stat.i.cache-misses
2.27e+09 +8.0% 2.451e+09 perf-stat.i.cache-references
4162783 ± 4% +19.7% 4981708 perf-stat.i.context-switches
1.52 +2.0% 1.55 perf-stat.i.cpi
227633 ± 14% +75.1% 398694 ± 3% perf-stat.i.cpu-migrations
33931 ± 12% +42.7% 48428 ± 2% perf-stat.i.cycles-between-cache-misses
0.01 -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate%
1408442 ± 2% -16.4% 1177727 ± 4% perf-stat.i.dTLB-load-misses
1.229e+10 -2.1% 1.203e+10 perf-stat.i.dTLB-loads
0.00 ± 2% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate%
35757 ± 2% -16.0% 30032 ± 3% perf-stat.i.dTLB-store-misses
7.449e+09 -2.3% 7.279e+09 perf-stat.i.dTLB-stores
39246970 -5.5% 37079604 perf-stat.i.iTLB-load-misses
76813 ± 8% -26.2% 56724 ± 6% perf-stat.i.iTLB-loads
4.096e+10 -1.9% 4.02e+10 perf-stat.i.instructions
1062 +4.1% 1105 perf-stat.i.instructions-per-iTLB-miss
0.66 -1.9% 0.65 perf-stat.i.ipc
1210 -7.1% 1124 perf-stat.i.minor-faults
105889 ± 10% -24.8% 79596 ± 5% perf-stat.i.node-loads
174508 ± 7% -24.4% 131915 perf-stat.i.node-stores
1210 -7.1% 1124 perf-stat.i.page-faults
55.42 ± 2% +10.0% 60.98 perf-stat.overall.MPKI
1.31 +0.1 1.38 perf-stat.overall.branch-miss-rate%
0.11 ± 11% -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate%
1.51 +1.9% 1.54 perf-stat.overall.cpi
24740 ± 9% +34.9% 33377 ± 3% perf-stat.overall.cycles-between-cache-misses
0.01 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate%
1043 +3.9% 1084 perf-stat.overall.instructions-per-iTLB-miss
0.66 -1.9% 0.65 perf-stat.overall.ipc
57826 ± 3% +15.6% 66874 perf-stat.overall.path-length
1.064e+08 +3.7% 1.104e+08 perf-stat.ps.branch-misses
2517524 ± 9% -26.5% 1851335 ± 3% perf-stat.ps.cache-misses
2.266e+09 +8.0% 2.447e+09 perf-stat.ps.cache-references
4155898 ± 4% +19.7% 4973458 perf-stat.ps.context-switches
227258 ± 14% +75.1% 398041 ± 3% perf-stat.ps.cpu-migrations
1406119 ± 2% -16.4% 1175777 ± 4% perf-stat.ps.dTLB-load-misses
1.227e+10 -2.1% 1.201e+10 perf-stat.ps.dTLB-loads
35699 ± 2% -16.0% 29983 ± 3% perf-stat.ps.dTLB-store-misses
7.437e+09 -2.3% 7.267e+09 perf-stat.ps.dTLB-stores
39182154 -5.5% 37018206 perf-stat.ps.iTLB-load-misses
76695 ± 8% -26.2% 56638 ± 6% perf-stat.ps.iTLB-loads
4.09e+10 -1.9% 4.013e+10 perf-stat.ps.instructions
1208 -7.1% 1122 perf-stat.ps.minor-faults
105716 ± 10% -24.8% 79465 ± 5% perf-stat.ps.node-loads
174234 ± 7% -24.4% 131703 perf-stat.ps.node-stores
1208 -7.1% 1122 perf-stat.ps.page-faults
29.39 ±104% -29.4 0.00 perf-profile.calltrace.cycles-pp.start_thread
15.86 ±105% -15.9 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread
14.88 ±105% -14.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
14.75 ±105% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
13.53 ±104% -13.5 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
12.51 ±104% -12.5 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread
12.45 ±105% -12.4 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
11.21 ±105% -11.2 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
11.08 ±105% -11.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
10.47 ±105% -10.5 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
9.73 ±105% -9.7 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
3.09 ± 12% -0.6 2.50 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
2.45 ± 14% -0.5 1.95 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
1.15 ± 13% -0.2 0.95 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
0.99 ± 13% -0.2 0.83 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.92 ± 15% +0.2 1.10 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.84 ± 23% +0.3 1.10 ± 2% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.99 ± 19% +0.3 1.26 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.97 ± 23% +0.3 1.26 perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.29 ±100% +0.3 0.60 ± 5% perf-profile.calltrace.cycles-pp.reschedule_interrupt.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write
0.55 ± 62% +0.3 0.89 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.28 ±100% +0.4 0.63 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop
0.33 ±100% +0.4 0.69 ± 2% perf-profile.calltrace.cycles-pp.prepare_to_wait.pipe_wait.pipe_read.new_sync_read.vfs_read
0.29 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
1.00 ± 27% +0.4 1.37 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.57 ± 63% +0.4 0.95 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.46 ± 31% +0.4 1.86 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.72 ± 60% +0.4 1.14 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.29 ±100% +0.4 0.70 ± 6% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.34 ±100% +0.4 0.77 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task
1.26 ± 26% +0.5 1.72 perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.13 ± 26% +0.5 1.59 perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.35 ±100% +0.5 0.83 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
1.28 ± 28% +0.5 1.77 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.40 ± 26% +0.5 1.89 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.44 ±100% +0.5 0.95 ± 7% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
2.33 ± 17% +0.5 2.88 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
1.33 ± 28% +0.6 1.89 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.40 ± 24% +0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
0.83 ± 66% +0.6 1.40 ± 5% perf-profile.calltrace.cycles-pp.native_write_msr
5.01 ± 8% +0.8 5.81 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.75 ± 39% +0.9 2.61 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
1.33 ± 67% +1.0 2.28 ± 6% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.56 ± 12% +1.2 7.78 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
8.96 ± 9% +1.6 10.56 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
9.23 ± 10% +1.6 10.87 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
9.17 ± 10% +1.6 10.81 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
3.72 ± 31% +1.7 5.44 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.37 ± 8% +1.9 5.32 ± 4% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.37 ± 10% +2.3 6.65 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
15.27 ± 10% +2.3 17.59 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
15.59 ± 10% +2.4 17.96 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
3.00 ± 71% +2.6 5.60 ± 6% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.10 ± 71% +2.7 5.77 ± 6% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
18.05 ± 10% +2.7 20.73 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
28.21 ± 5% +3.7 31.87 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
24.29 ± 8% +4.7 28.98 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
23.38 ± 8% +4.7 28.08 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
23.72 ± 8% +4.7 28.43 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
31.18 ± 41% +12.7 43.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
33.44 ± 40% +13.2 46.61 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
64.81 ± 43% +27.5 92.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
65.30 ± 43% +27.7 92.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
29.39 ±104% -29.4 0.00 perf-profile.children.cycles-pp.start_thread
16.09 ±105% -16.1 0.00 perf-profile.children.cycles-pp.__GI___libc_write
12.73 ±104% -12.7 0.00 perf-profile.children.cycles-pp.__GI___libc_read
5.80 ± 10% -1.2 4.65 ± 3% perf-profile.children.cycles-pp.security_file_permission
4.10 ± 16% -1.0 3.10 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
3.21 ± 9% -0.7 2.55 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter
1.45 ± 32% -0.6 0.83 perf-profile.children.cycles-pp.entry_SYSCALL_64
3.04 ± 8% -0.6 2.44 ± 3% perf-profile.children.cycles-pp.selinux_file_permission
2.56 ± 11% -0.6 1.99 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter
2.80 ± 9% -0.5 2.25 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
3.38 ± 7% -0.5 2.87 ± 2% perf-profile.children.cycles-pp.mutex_lock
1.62 ± 12% -0.4 1.23 ± 3% perf-profile.children.cycles-pp.___might_sleep
1.82 ± 9% -0.4 1.43 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
2.09 ± 12% -0.4 1.72 ± 6% perf-profile.children.cycles-pp.file_has_perm
0.90 ± 20% -0.3 0.57 ± 9% perf-profile.children.cycles-pp.__mutex_lock
1.14 ± 12% -0.3 0.82 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate
1.26 ± 10% -0.3 0.97 ± 4% perf-profile.children.cycles-pp.copyin
1.70 ± 8% -0.3 1.42 ± 3% perf-profile.children.cycles-pp.copyout
1.25 ± 11% -0.3 0.98 ± 3% perf-profile.children.cycles-pp.fsnotify
1.04 ± 9% -0.3 0.79 ± 3% perf-profile.children.cycles-pp._cond_resched
1.72 ± 7% -0.2 1.48 ± 5% perf-profile.children.cycles-pp.fput_many
1.16 ± 10% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.__might_sleep
1.09 ± 8% -0.2 0.85 ± 3% perf-profile.children.cycles-pp.__fsnotify_parent
0.83 ± 13% -0.2 0.63 ± 3% perf-profile.children.cycles-pp.__might_fault
0.69 ± 11% -0.1 0.56 perf-profile.children.cycles-pp.current_time
0.99 ± 4% -0.1 0.89 ± 4% perf-profile.children.cycles-pp.touch_atime
0.65 ± 6% -0.1 0.55 ± 6% perf-profile.children.cycles-pp.atime_needs_update
0.39 ± 4% -0.1 0.29 perf-profile.children.cycles-pp.rcu_all_qs
0.13 ± 39% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.34 ± 11% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common
0.49 ± 5% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.wake_up_q
0.24 ± 2% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.30 ± 11% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.inode_has_perm
0.23 ± 8% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.__sb_end_write
0.21 ± 9% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x64_sys_read
0.29 ± 8% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.timespec64_trunc
0.21 ± 8% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.__x64_sys_write
0.22 ± 7% -0.0 0.19 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.08 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.bpf_fd_pass
0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.perf_exclude_event
0.08 ± 24% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.rcu_note_context_switch
0.17 ± 19% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.23 ± 19% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.resched_curr
0.61 ± 8% +0.1 0.72 ± 6% perf-profile.children.cycles-pp.sched_clock
0.67 ± 8% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu
0.50 ± 9% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.account_entity_enqueue
0.61 ± 10% +0.2 0.79 perf-profile.children.cycles-pp.account_entity_dequeue
0.43 ± 19% +0.2 0.62 ± 7% perf-profile.children.cycles-pp.clear_buddies
1.42 ± 14% +0.2 1.66 ± 4% perf-profile.children.cycles-pp.native_write_msr
1.17 ± 13% +0.3 1.42 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup
0.85 ± 23% +0.3 1.12 ± 7% perf-profile.children.cycles-pp.put_prev_entity
0.94 ± 14% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.___perf_sw_event
1.45 ± 11% +0.3 1.73 perf-profile.children.cycles-pp.check_preempt_curr
1.45 ± 12% +0.3 1.76 perf-profile.children.cycles-pp.__enqueue_entity
1.60 ± 10% +0.3 1.91 perf-profile.children.cycles-pp.ttwu_do_wakeup
1.12 ± 9% +0.3 1.43 ± 3% perf-profile.children.cycles-pp.__update_load_avg_se
1.64 ± 8% +0.4 2.00 ± 4% perf-profile.children.cycles-pp.available_idle_cpu
0.96 ± 19% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.pick_next_entity
1.85 ± 16% +0.4 2.28 ± 6% perf-profile.children.cycles-pp.switch_fpu_return
1.90 ± 9% +0.6 2.50 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
3.22 ± 13% +0.6 3.87 perf-profile.children.cycles-pp.update_load_avg
4.03 ± 10% +0.6 4.68 ± 2% perf-profile.children.cycles-pp.update_curr
3.01 ± 14% +0.7 3.72 ± 2% perf-profile.children.cycles-pp.reweight_entity
5.10 ± 8% +0.8 5.87 ± 2% perf-profile.children.cycles-pp.enqueue_entity
4.71 ± 15% +1.1 5.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair
6.73 ± 11% +1.1 7.87 perf-profile.children.cycles-pp.dequeue_task_fair
9.07 ± 9% +1.5 10.61 perf-profile.children.cycles-pp.enqueue_task_fair
9.32 ± 9% +1.6 10.89 perf-profile.children.cycles-pp.ttwu_do_activate
91.46 +1.6 93.03 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
9.27 ± 9% +1.6 10.84 perf-profile.children.cycles-pp.activate_task
4.16 ± 26% +1.6 5.78 ± 6% perf-profile.children.cycles-pp.exit_to_usermode_loop
90.79 +1.7 92.47 perf-profile.children.cycles-pp.do_syscall_64
3.48 ± 8% +1.9 5.36 ± 4% perf-profile.children.cycles-pp.select_idle_sibling
4.44 ± 10% +2.2 6.67 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair
18.46 ± 9% +2.3 20.81 perf-profile.children.cycles-pp.pipe_wait
19.93 ± 11% +3.5 23.46 ± 2% perf-profile.children.cycles-pp.__schedule
28.82 ± 5% +3.6 32.45 perf-profile.children.cycles-pp.__wake_up_common_lock
19.89 ± 12% +3.7 23.62 ± 2% perf-profile.children.cycles-pp.schedule
23.96 ± 8% +4.5 28.45 perf-profile.children.cycles-pp.try_to_wake_up
24.53 ± 8% +4.5 29.06 perf-profile.children.cycles-pp.__wake_up_common
23.91 ± 8% +4.6 28.46 perf-profile.children.cycles-pp.autoremove_wake_function
2.76 ± 9% -0.5 2.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.46 ± 18% -0.4 1.06 ± 4% perf-profile.self.cycles-pp.pipe_write
1.81 ± 9% -0.4 1.43 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
1.56 ± 12% -0.4 1.19 ± 3% perf-profile.self.cycles-pp.___might_sleep
2.09 ± 7% -0.3 1.81 ± 2% perf-profile.self.cycles-pp.mutex_lock
1.86 ± 8% -0.3 1.59 ± 3% perf-profile.self.cycles-pp.selinux_file_permission
1.21 ± 11% -0.3 0.96 ± 2% perf-profile.self.cycles-pp.fsnotify
1.70 ± 7% -0.2 1.46 ± 5% perf-profile.self.cycles-pp.fput_many
1.04 ± 11% -0.2 0.82 ± 3% perf-profile.self.cycles-pp.__might_sleep
1.01 ± 8% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.__fsnotify_parent
1.04 ± 10% -0.2 0.83 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.67 ± 10% -0.2 0.51 ± 5% perf-profile.self.cycles-pp.new_sync_write
0.64 ± 12% -0.2 0.48 ± 5% perf-profile.self.cycles-pp.vfs_write
1.67 -0.2 1.51 ± 2% perf-profile.self.cycles-pp.pipe_read
0.58 ± 17% -0.1 0.44 ± 7% perf-profile.self.cycles-pp.file_has_perm
0.44 ± 18% -0.1 0.31 ± 7% perf-profile.self.cycles-pp.__mutex_lock
0.49 ± 13% -0.1 0.35 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter
0.64 ± 8% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.67 ± 7% -0.1 0.55 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.30 ± 5% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ksys_write
0.40 ± 12% -0.1 0.30 ± 7% perf-profile.self.cycles-pp.security_file_permission
0.13 ± 38% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.36 ± 11% -0.1 0.28 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate
0.76 ± 5% -0.1 0.68 ± 5% perf-profile.self.cycles-pp.do_syscall_64
0.21 ± 2% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.26 ± 12% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.inode_has_perm
0.29 ± 11% -0.1 0.23 ± 6% perf-profile.self.cycles-pp.current_time
0.28 ± 4% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.rcu_all_qs
0.22 ± 7% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.__sb_end_write
0.31 -0.1 0.26 ± 7% perf-profile.self.cycles-pp.ksys_read
0.20 ± 11% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_read
0.26 ± 8% -0.0 0.22 perf-profile.self.cycles-pp.timespec64_trunc
0.20 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_write
0.24 ± 9% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.__might_fault
0.09 ± 9% -0.0 0.06 perf-profile.self.cycles-pp.copyin
0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.07 ± 10% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.sched_clock
0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.20 ± 12% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__list_add_valid
0.10 ± 26% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.15 ± 16% +0.1 0.21 ± 11% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.35 ± 7% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.prepare_to_wait
0.55 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__fget_light
0.22 ± 18% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.resched_curr
0.48 ± 11% +0.1 0.58 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup
0.39 ± 11% +0.1 0.50 perf-profile.self.cycles-pp.account_entity_enqueue
0.85 ± 7% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.enqueue_entity
0.43 ± 16% +0.1 0.55 ± 7% perf-profile.self.cycles-pp.pick_next_entity
0.86 ± 10% +0.1 0.99 perf-profile.self.cycles-pp.dequeue_task_fair
0.37 ± 21% +0.2 0.55 ± 6% perf-profile.self.cycles-pp.clear_buddies
0.45 ± 14% +0.2 0.65 perf-profile.self.cycles-pp.account_entity_dequeue
1.41 ± 14% +0.2 1.65 ± 4% perf-profile.self.cycles-pp.native_write_msr
0.80 ± 16% +0.3 1.06 ± 4% perf-profile.self.cycles-pp.___perf_sw_event
1.09 ± 9% +0.3 1.39 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
0.84 ± 16% +0.3 1.14 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair
1.45 ± 12% +0.3 1.75 perf-profile.self.cycles-pp.__enqueue_entity
1.62 ± 8% +0.4 1.99 ± 4% perf-profile.self.cycles-pp.available_idle_cpu
1.83 ± 16% +0.4 2.25 ± 6% perf-profile.self.cycles-pp.switch_fpu_return
2.15 ± 11% +0.5 2.60 ± 5% perf-profile.self.cycles-pp.update_curr
1.88 ± 9% +0.6 2.45 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
1.21 ± 9% +0.7 1.92 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.80 ± 11% +1.4 2.17 ± 3% perf-profile.self.cycles-pp.select_idle_sibling
hackbench.throughput
74000 +-+-----------------------------------------------------------------+
72000 +-+.+ .+ |
|.+ : .+. .+. + |
70000 +-+ : + + +. : |
68000 +-+ + + : |
| :: |
66000 +-+ + |
64000 +-+ |
62000 +-+ |
| |
60000 +-+ O O OO O O O O O O O OO |
58000 +-+ O O O O O O O O O O O
| O O OO O O |
56000 O-+ O O O O O O |
54000 +-O-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/18/x86_64-rhel-7.6/threads/1600%/debian-x86_64-2019-11-14.cgz/lkp-hsw-ep4/hackbench/0x43
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
176075 ± 2% -11.3% 156152 hackbench.throughput
891.79 +12.3% 1001 hackbench.time.elapsed_time
891.79 +12.3% 1001 hackbench.time.elapsed_time.max
1.889e+09 ± 9% +33.1% 2.514e+09 ± 3% hackbench.time.involuntary_context_switches
42939 ± 3% +17.0% 50220 hackbench.time.system_time
18978 +3.2% 19592 hackbench.time.user_time
2.558e+09 ± 8% +33.5% 3.415e+09 ± 2% hackbench.time.voluntary_context_switches
43133704 ± 3% +39.2% 60035429 ± 33% cpuidle.C1.time
48661 +17.7% 57274 ± 5% meminfo.Shmem
19043 ± 2% -14.0% 16377 meminfo.max_used_kB
2502 +1.3% 2535 turbostat.Avg_MHz
3.368e+08 ± 5% +30.6% 4.398e+08 turbostat.IRQ
66.25 +4.5% 69.25 vmstat.cpu.sy
29.25 ± 2% -8.5% 26.75 vmstat.cpu.us
5000443 ± 7% +18.6% 5931772 vmstat.system.cs
370173 ± 3% +17.1% 433616 vmstat.system.in
98786923 ± 7% +74.5% 1.724e+08 numa-numastat.node0.local_node
98791606 ± 7% +74.5% 1.724e+08 numa-numastat.node0.numa_hit
4686 ± 99% +297.8% 18642 ± 24% numa-numastat.node0.other_node
1.818e+08 -51.4% 88447082 ± 3% numa-numastat.node1.local_node
1.819e+08 -51.4% 88451833 ± 3% numa-numastat.node1.numa_hit
18694 ± 25% -74.6% 4753 ± 97% numa-numastat.node1.other_node
155316 +1.5% 157632 proc-vmstat.nr_active_anon
99.50 +5.3% 104.75 ± 3% proc-vmstat.nr_dirtied
4440 +4.9% 4656 proc-vmstat.nr_inactive_anon
6283 +1.9% 6399 proc-vmstat.nr_mapped
12248 ± 2% +16.6% 14276 ± 5% proc-vmstat.nr_shmem
98.75 +5.3% 104.00 proc-vmstat.nr_written
155316 +1.5% 157632 proc-vmstat.nr_zone_active_anon
4440 +4.9% 4656 proc-vmstat.nr_zone_inactive_anon
2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_hit
2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_local
2.83e+08 -7.0% 2.632e+08 proc-vmstat.pgalloc_normal
3212744 +2.3% 3286078 proc-vmstat.pgfault
2.828e+08 -7.0% 2.631e+08 proc-vmstat.pgfree
39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_active_anon
31864 ± 44% +211.0% 99111 ± 4% numa-vmstat.node0.nr_anon_pages
11183 ± 3% +4212.8% 482321 ± 2% numa-vmstat.node0.nr_kernel_stack
499.50 ± 23% +66.9% 833.75 ± 8% numa-vmstat.node0.nr_page_table_pages
11417 -63.5% 4168 ± 5% numa-vmstat.node0.nr_shmem
15905 ± 4% -12.5% 13919 ± 2% numa-vmstat.node0.nr_slab_reclaimable
17996 ± 3% +587.4% 123708 ± 2% numa-vmstat.node0.nr_slab_unreclaimable
39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_zone_active_anon
52471566 ± 6% +73.3% 90915199 numa-vmstat.node0.numa_hit
52466577 ± 6% +73.2% 90896366 numa-vmstat.node0.numa_local
4990 ± 92% +277.5% 18836 ± 24% numa-vmstat.node0.numa_other
114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_active_anon
114545 ± 12% -58.2% 47874 ± 9% numa-vmstat.node1.nr_anon_pages
462075 -96.6% 15717 ± 67% numa-vmstat.node1.nr_kernel_stack
871.50 ± 13% -37.9% 541.00 ± 13% numa-vmstat.node1.nr_page_table_pages
691.25 ± 22% +1359.2% 10086 ± 8% numa-vmstat.node1.nr_shmem
11357 ± 5% +18.1% 13414 ± 4% numa-vmstat.node1.nr_slab_reclaimable
118594 -85.5% 17243 ± 16% numa-vmstat.node1.nr_slab_unreclaimable
114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_zone_active_anon
93308245 ± 2% -49.7% 46921467 ± 4% numa-vmstat.node1.numa_hit
93128224 ± 2% -49.8% 46755145 ± 4% numa-vmstat.node1.numa_local
157762 ± 35% +154.4% 401333 ± 4% numa-meminfo.node0.Active
157721 ± 35% +154.4% 401249 ± 4% numa-meminfo.node0.Active(anon)
128232 ± 44% +212.2% 400337 ± 4% numa-meminfo.node0.AnonPages
63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.KReclaimable
11691 ± 3% +4087.3% 489553 numa-meminfo.node0.KernelStack
1170596 ± 4% +104.2% 2390602 numa-meminfo.node0.MemUsed
2003 ± 23% +67.3% 3351 ± 8% numa-meminfo.node0.PageTables
63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.SReclaimable
72301 ± 3% +591.9% 500285 numa-meminfo.node0.SUnreclaim
45749 -63.5% 16679 ± 5% numa-meminfo.node0.Shmem
135789 ± 3% +309.6% 556136 numa-meminfo.node0.Slab
460025 ± 11% -50.2% 228882 ± 7% numa-meminfo.node1.Active
459902 ± 11% -50.2% 228802 ± 7% numa-meminfo.node1.Active(anon)
459237 ± 11% -58.3% 191354 ± 9% numa-meminfo.node1.AnonPages
45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.KReclaimable
468955 -96.7% 15461 ± 67% numa-meminfo.node1.KernelStack
2241450 -51.9% 1078199 ± 3% numa-meminfo.node1.MemUsed
3493 ± 13% -38.0% 2167 ± 13% numa-meminfo.node1.PageTables
45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.SReclaimable
479010 -85.6% 68750 ± 16% numa-meminfo.node1.SUnreclaim
2779 ± 23% +1354.1% 40419 ± 8% numa-meminfo.node1.Shmem
524580 -76.7% 122408 ± 8% numa-meminfo.node1.Slab
250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.avg
1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev
20766 ± 9% -25.3% 15519 ± 10% sched_debug.cfs_rq:/.load.avg
43820 ± 19% -40.6% 26044 ± 36% sched_debug.cfs_rq:/.load.stddev
250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.max_vruntime.avg
1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev
10990567 ± 9% +22.9% 13509735 ± 7% sched_debug.cfs_rq:/.min_vruntime.stddev
102.40 -33.3% 68.27 ± 57% sched_debug.cfs_rq:/.removed.load_avg.max
4711 -33.4% 3136 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.max
19930 ± 10% -25.1% 14917 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg
-5713498 -349.0% 14226386 ± 8% sched_debug.cfs_rq:/.spread0.avg
8106715 ± 13% +283.1% 31057182 ± 6% sched_debug.cfs_rq:/.spread0.max
-18025645 -99.7% -45393 sched_debug.cfs_rq:/.spread0.min
11027552 ± 9% +21.5% 13398318 ± 7% sched_debug.cfs_rq:/.spread0.stddev
570.46 ± 5% +109.2% 1193 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg
2898 ± 11% +90.0% 5507 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.max
563.27 ± 13% +149.3% 1404 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev
155578 ± 22% +84.1% 286457 ± 28% sched_debug.cpu.avg_idle.min
1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock.stddev
1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock_task.stddev
50207 -12.3% 44017 ± 7% sched_debug.cpu.curr->pid.avg
82673 -19.5% 66514 ± 5% sched_debug.cpu.curr->pid.max
746.05 ± 11% +554.3% 4881 ± 8% sched_debug.cpu.curr->pid.min
30728 ± 2% -40.8% 18187 ± 10% sched_debug.cpu.curr->pid.stddev
0.00 ± 43% +106.0% 0.00 ± 31% sched_debug.cpu.next_balance.stddev
20.49 ± 6% +149.0% 51.03 ± 23% sched_debug.cpu.nr_running.avg
157.83 ± 9% +77.0% 279.37 ± 8% sched_debug.cpu.nr_running.max
29.95 ± 11% +137.1% 70.99 ± 16% sched_debug.cpu.nr_running.stddev
29803727 ± 9% +29.6% 38623870 ± 3% sched_debug.cpu.nr_switches.avg
51549476 ± 10% +22.5% 63171598 ± 3% sched_debug.cpu.nr_switches.max
12792570 ± 3% +43.5% 18362678 ± 4% sched_debug.cpu.nr_switches.min
2.03 ± 80% +430.9% 10.76 ± 38% sched_debug.cpu.nr_uninterruptible.avg
6.99 ± 4% +48.7% 10.39 ± 5% perf-stat.i.MPKI
7.49 ± 5% -1.2 6.29 ± 5% perf-stat.i.cache-miss-rate%
27202880 ± 2% +46.9% 39966416 ± 2% perf-stat.i.cache-misses
4.662e+08 ± 5% +71.5% 7.998e+08 ± 5% perf-stat.i.cache-references
5989366 ± 4% +12.9% 6760958 ± 5% perf-stat.i.context-switches
197319 ± 12% -19.2% 159504 ± 3% perf-stat.i.cpu-migrations
7415 ± 2% -23.3% 5686 ± 3% perf-stat.i.cycles-between-cache-misses
0.61 ± 35% -0.3 0.35 ± 4% perf-stat.i.dTLB-load-miss-rate%
1.506e+08 ± 38% -44.9% 82967539 ± 5% perf-stat.i.dTLB-load-misses
0.22 ± 4% -0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate%
35974976 ± 5% -13.4% 31146077 ± 6% perf-stat.i.dTLB-store-misses
1.627e+10 -3.7% 1.566e+10 perf-stat.i.dTLB-stores
46665620 ± 3% -14.6% 39838962 ± 5% perf-stat.i.iTLB-load-misses
2191 ± 5% +21.4% 2660 ± 7% perf-stat.i.instructions-per-iTLB-miss
0.47 -2.5% 0.46 perf-stat.i.ipc
6660 ± 8% -19.1% 5391 ± 8% perf-stat.i.minor-faults
62.93 ± 2% +11.1 74.06 perf-stat.i.node-load-miss-rate%
12537290 ± 4% +73.5% 21750457 ± 4% perf-stat.i.node-load-misses
5459308 ± 3% -7.4% 5057404 ± 3% perf-stat.i.node-loads
29.39 ± 2% -4.1 25.29 ± 2% perf-stat.i.node-store-miss-rate%
2863317 +13.6% 3252632 ± 2% perf-stat.i.node-store-misses
6381925 ± 2% +53.5% 9796918 ± 4% perf-stat.i.node-stores
6638 ± 8% -19.4% 5348 ± 8% perf-stat.i.page-faults
4.89 ± 4% +78.4% 8.73 perf-stat.overall.MPKI
1.29 +0.0 1.31 perf-stat.overall.branch-miss-rate%
6.13 ± 4% -1.3 4.85 ± 3% perf-stat.overall.cache-miss-rate%
2.15 +2.0% 2.19 perf-stat.overall.cpi
7187 -27.9% 5182 ± 3% perf-stat.overall.cycles-between-cache-misses
0.63 ± 39% -0.3 0.36 ± 5% perf-stat.overall.dTLB-load-miss-rate%
0.23 ± 5% -0.0 0.21 ± 3% perf-stat.overall.dTLB-store-miss-rate%
75.65 -1.1 74.59 perf-stat.overall.iTLB-load-miss-rate%
1604 ± 3% +15.9% 1859 perf-stat.overall.instructions-per-iTLB-miss
0.47 -2.0% 0.46 perf-stat.overall.ipc
63.14 ± 2% +13.6 76.70 perf-stat.overall.node-load-miss-rate%
30.74 ± 2% -5.1 25.62 ± 2% perf-stat.overall.node-store-miss-rate%
47754 ± 2% +11.7% 53321 perf-stat.overall.path-length
24893507 +40.6% 34993733 ± 2% perf-stat.ps.cache-misses
4.072e+08 ± 5% +77.1% 7.212e+08 perf-stat.ps.cache-references
4968881 ± 7% +19.0% 5911450 perf-stat.ps.context-switches
1.789e+11 +1.3% 1.812e+11 perf-stat.ps.cpu-cycles
1.59e+08 ± 39% -44.1% 88801351 ± 6% perf-stat.ps.dTLB-load-misses
2.502e+10 -2.0% 2.453e+10 perf-stat.ps.dTLB-loads
39138578 ± 5% -12.6% 34190129 ± 3% perf-stat.ps.dTLB-store-misses
1.675e+10 -3.7% 1.613e+10 perf-stat.ps.dTLB-stores
51901139 ± 2% -14.3% 44461197 perf-stat.ps.iTLB-load-misses
16711264 ± 2% -9.3% 15153636 ± 2% perf-stat.ps.iTLB-loads
3573 -9.1% 3247 perf-stat.ps.minor-faults
10427704 ± 3% +73.2% 18055799 ± 3% perf-stat.ps.node-load-misses
6085553 ± 4% -10.0% 5479559 ± 2% perf-stat.ps.node-loads
2566511 +13.4% 2911367 ± 2% perf-stat.ps.node-store-misses
5784179 ± 2% +46.2% 8454210 ± 3% perf-stat.ps.node-stores
3573 -9.1% 3247 perf-stat.ps.page-faults
7.427e+13 ± 2% +11.7% 8.293e+13 perf-stat.total.instructions
127394 ± 5% +19.1% 151665 softirqs.CPU0.RCU
321761 ± 3% +12.8% 363107 ± 3% softirqs.CPU0.TIMER
124526 ± 5% +20.2% 149697 ± 3% softirqs.CPU1.RCU
123768 ± 4% +24.7% 154332 ± 2% softirqs.CPU10.RCU
130315 ± 3% +22.1% 159111 softirqs.CPU11.RCU
126684 ± 4% +26.1% 159755 ± 2% softirqs.CPU12.RCU
124649 ± 9% +27.8% 159298 ± 3% softirqs.CPU13.RCU
124528 ± 4% +23.2% 153372 ± 2% softirqs.CPU14.RCU
326150 ± 2% +9.8% 357991 ± 4% softirqs.CPU14.TIMER
121923 ± 6% +17.4% 143119 softirqs.CPU15.RCU
122114 ± 5% +16.7% 142560 softirqs.CPU16.RCU
121425 ± 6% +14.4% 138861 ± 3% softirqs.CPU17.RCU
330948 ± 2% +22.2% 404570 ± 16% softirqs.CPU17.TIMER
312317 ± 4% +18.2% 369091 ± 2% softirqs.CPU18.TIMER
9626 ± 2% +15.0% 11067 ± 3% softirqs.CPU19.SCHED
322308 ± 3% +21.1% 390454 ± 2% softirqs.CPU19.TIMER
124281 ± 5% +22.8% 152670 ± 2% softirqs.CPU2.RCU
9452 ± 2% +15.6% 10927 ± 3% softirqs.CPU20.SCHED
318575 ± 6% +22.3% 389741 ± 3% softirqs.CPU20.TIMER
9327 ± 3% +16.7% 10886 ± 3% softirqs.CPU21.SCHED
306628 ± 4% +23.3% 377962 ± 3% softirqs.CPU21.TIMER
9424 ± 2% +17.6% 11084 ± 3% softirqs.CPU22.SCHED
302796 ± 4% +24.2% 376126 ± 2% softirqs.CPU22.TIMER
9798 ± 2% +11.4% 10918 ± 4% softirqs.CPU23.SCHED
312009 ± 4% +23.6% 385626 ± 2% softirqs.CPU23.TIMER
9559 ± 3% +13.2% 10822 softirqs.CPU24.SCHED
311772 ± 4% +23.0% 383504 ± 2% softirqs.CPU24.TIMER
9514 ± 4% +13.2% 10766 ± 3% softirqs.CPU25.SCHED
307640 ± 4% +22.4% 376479 ± 2% softirqs.CPU25.TIMER
9573 ± 3% +13.8% 10897 ± 3% softirqs.CPU26.SCHED
314607 ± 9% +21.1% 380918 ± 2% softirqs.CPU26.TIMER
9385 +13.7% 10668 ± 2% softirqs.CPU27.SCHED
301949 ± 4% +25.2% 378140 ± 3% softirqs.CPU27.TIMER
9483 ± 2% +14.7% 10874 ± 2% softirqs.CPU28.SCHED
299462 ± 3% +24.9% 373896 ± 2% softirqs.CPU28.TIMER
9505 ± 3% +13.3% 10769 ± 3% softirqs.CPU29.SCHED
307087 ± 4% +23.8% 380066 ± 3% softirqs.CPU29.TIMER
124992 ± 4% +19.4% 149253 ± 3% softirqs.CPU3.RCU
9630 ± 4% +13.6% 10942 ± 3% softirqs.CPU30.SCHED
306382 ± 4% +25.4% 384355 ± 2% softirqs.CPU30.TIMER
9496 ± 6% +12.6% 10689 ± 2% softirqs.CPU31.SCHED
308689 ± 4% +24.5% 384419 ± 2% softirqs.CPU31.TIMER
9714 +12.6% 10937 ± 4% softirqs.CPU32.SCHED
304137 ± 3% +35.9% 413240 ± 13% softirqs.CPU32.TIMER
9598 ± 2% +13.4% 10882 ± 5% softirqs.CPU33.SCHED
9622 +13.8% 10949 ± 3% softirqs.CPU34.SCHED
305882 ± 4% +27.3% 389315 ± 4% softirqs.CPU34.TIMER
9409 ± 4% +15.3% 10853 ± 4% softirqs.CPU35.SCHED
306215 ± 4% +23.8% 379222 ± 2% softirqs.CPU35.TIMER
321617 ± 4% +12.9% 362966 ± 3% softirqs.CPU36.TIMER
134843 ± 6% +19.4% 161053 ± 2% softirqs.CPU37.RCU
130954 ± 6% +20.4% 157716 softirqs.CPU38.RCU
129884 ± 5% +17.8% 152978 softirqs.CPU39.RCU
124744 ± 4% +20.1% 149790 ± 3% softirqs.CPU4.RCU
130593 ± 5% +17.2% 153092 ± 2% softirqs.CPU40.RCU
134185 ± 5% +19.9% 160871 softirqs.CPU41.RCU
133336 ± 5% +18.1% 157409 softirqs.CPU42.RCU
134486 ± 4% +18.7% 159613 softirqs.CPU43.RCU
129857 ± 6% +18.5% 153887 softirqs.CPU44.RCU
131816 ± 5% +20.0% 158144 softirqs.CPU45.RCU
132691 ± 5% +22.4% 162412 softirqs.CPU46.RCU
138736 ± 3% +18.7% 164668 softirqs.CPU47.RCU
132971 ± 6% +24.2% 165142 softirqs.CPU48.RCU
136142 ± 3% +20.8% 164493 softirqs.CPU49.RCU
125011 ± 4% +23.6% 154462 ± 2% softirqs.CPU5.RCU
133660 ± 4% +19.5% 159753 ± 2% softirqs.CPU50.RCU
140405 ± 4% +19.5% 167772 softirqs.CPU51.RCU
140523 ± 4% +18.5% 166576 softirqs.CPU52.RCU
138849 ± 5% +14.6% 159157 ± 4% softirqs.CPU53.RCU
332171 ± 2% +21.4% 403260 ± 15% softirqs.CPU53.TIMER
133037 ± 4% +14.0% 151688 ± 2% softirqs.CPU54.RCU
9584 +14.3% 10959 ± 2% softirqs.CPU54.SCHED
310603 ± 4% +18.0% 366472 ± 3% softirqs.CPU54.TIMER
9565 ± 2% +13.8% 10889 ± 5% softirqs.CPU55.SCHED
321231 ± 3% +21.3% 389753 ± 2% softirqs.CPU55.TIMER
136633 ± 4% +12.6% 153854 softirqs.CPU56.RCU
9430 ± 2% +15.6% 10898 ± 5% softirqs.CPU56.SCHED
319477 ± 6% +22.1% 390107 ± 2% softirqs.CPU56.TIMER
128672 ± 4% +14.1% 146791 ± 3% softirqs.CPU57.RCU
9493 ± 2% +13.1% 10740 ± 3% softirqs.CPU57.SCHED
305670 ± 4% +23.5% 377597 ± 2% softirqs.CPU57.TIMER
127129 ± 4% +15.6% 146993 ± 2% softirqs.CPU58.RCU
9529 ± 3% +13.6% 10824 ± 4% softirqs.CPU58.SCHED
302317 ± 4% +23.8% 374371 ± 3% softirqs.CPU58.TIMER
9427 ± 2% +14.0% 10743 ± 2% softirqs.CPU59.SCHED
310985 ± 4% +23.5% 384119 ± 3% softirqs.CPU59.TIMER
124937 ± 4% +21.6% 151962 ± 3% softirqs.CPU6.RCU
9435 ± 2% +14.1% 10762 ± 3% softirqs.CPU60.SCHED
311646 ± 4% +22.8% 382731 ± 3% softirqs.CPU60.TIMER
9585 +13.5% 10877 ± 2% softirqs.CPU61.SCHED
306857 ± 4% +22.8% 376944 ± 2% softirqs.CPU61.TIMER
9483 ± 2% +12.5% 10671 ± 5% softirqs.CPU62.SCHED
9620 ± 4% +9.7% 10556 ± 4% softirqs.CPU63.SCHED
301085 ± 3% +24.7% 375397 ± 3% softirqs.CPU63.TIMER
300443 ± 4% +23.7% 371620 ± 2% softirqs.CPU64.TIMER
9667 +11.2% 10751 ± 4% softirqs.CPU65.SCHED
307194 ± 4% +23.4% 379020 ± 3% softirqs.CPU65.TIMER
9493 ± 3% +14.9% 10909 ± 3% softirqs.CPU66.SCHED
306226 ± 4% +25.1% 383140 ± 3% softirqs.CPU66.TIMER
9674 ± 3% +13.8% 11010 ± 3% softirqs.CPU67.SCHED
307870 ± 4% +25.0% 384793 ± 2% softirqs.CPU67.TIMER
9451 ± 2% +14.2% 10792 ± 4% softirqs.CPU68.SCHED
301873 ± 3% +26.1% 380748 softirqs.CPU68.TIMER
9542 +13.8% 10863 ± 3% softirqs.CPU69.SCHED
316206 ± 8% +20.5% 380893 ± 2% softirqs.CPU69.TIMER
128158 ± 4% +20.4% 154294 ± 2% softirqs.CPU7.RCU
9456 ± 3% +13.5% 10734 ± 3% softirqs.CPU70.SCHED
305361 ± 4% +36.4% 416546 ± 15% softirqs.CPU70.TIMER
9520 ± 3% +15.4% 10985 ± 4% softirqs.CPU71.SCHED
305605 ± 4% +24.6% 380653 ± 2% softirqs.CPU71.TIMER
122831 ± 4% +21.6% 149424 ± 2% softirqs.CPU8.RCU
123870 ± 4% +23.0% 152396 ± 2% softirqs.CPU9.RCU
9357957 ± 5% +15.4% 10794953 softirqs.RCU
23209193 ± 3% +15.5% 26806870 ± 3% softirqs.TIMER
4.79 ± 4% -3.1 1.71 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.85 ± 4% -2.4 1.41 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.39 ± 4% -2.0 1.35 ± 16% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
2.93 ± 5% -1.9 0.99 ± 18% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
3.25 ± 4% -1.9 1.31 ± 10% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
2.14 ± 4% -1.2 0.91 ± 14% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
2.06 ± 4% -1.2 0.85 ± 13% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
1.52 ± 3% -1.0 0.49 ± 59% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
1.46 ± 3% -1.0 0.46 ± 59% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.53 ± 8% -0.8 0.68 ± 12% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read
1.65 ± 4% -0.8 0.81 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.45 ± 5% -0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.35 ± 5% -0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
1.28 ± 8% -0.7 0.62 ± 12% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write
1.29 ± 3% -0.6 0.67 ± 9% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.64 ± 11% +0.3 0.96 ± 7% perf-profile.calltrace.cycles-pp.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
0.00 +0.5 0.55 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.00 +0.6 0.57 ± 6% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.6 0.61 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.7 0.69 ± 8% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule
0.00 +0.7 0.72 ± 7% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.7 0.72 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.8 0.77 ± 5% perf-profile.calltrace.cycles-pp.__switch_to
0.00 +0.8 0.77 ± 6% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +0.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.find_next_bit.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up
0.00 +0.9 0.91 ± 4% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +0.9 0.91 ± 6% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr
0.00 +1.1 1.05 ± 5% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.1 1.08 ± 8% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
0.00 +1.2 1.20 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
0.00 +1.3 1.30 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +1.3 1.34 ± 6% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +1.6 1.62 ± 4% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.9 1.88 ± 5% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
0.00 +1.9 1.93 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.13 ±173% +2.0 2.11 ± 5% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
0.00 +2.1 2.06 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +2.4 2.38 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.00 +2.7 2.65 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.35 ±103% +4.3 4.67 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.22 ±173% +4.6 4.81 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.23 ±173% +4.7 4.97 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.72 ± 23% +5.0 5.72 ± 5% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
0.69 ± 17% +5.6 6.25 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.74 ± 17% +5.8 6.50 ± 4% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.74 ± 18% +5.8 6.55 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
15.99 ± 54% +7.5 23.51 ± 6% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.13 +7.7 20.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.49 +8.7 20.19 ± 3% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
1.31 ± 27% +8.9 10.21 ± 7% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
1.81 ± 22% +11.2 12.98 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
1.89 ± 22% +11.4 13.28 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
2.18 ± 21% +13.1 15.28 ± 4% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
2.01 ± 25% +19.1 21.10 ± 7% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
2.38 ± 23% +20.0 22.40 ± 7% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
19.01 ± 3% +23.4 42.38 ± 4% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
21.13 ± 53% +23.8 44.92 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
17.85 ± 4% +23.9 41.76 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
19.16 ± 53% +24.6 43.74 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.79 ± 22% +30.7 34.45 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
3.84 ± 23% +30.9 34.69 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
4.13 ± 21% +31.0 35.10 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
5.77 ± 14% +31.0 36.76 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
9.20 ± 6% -6.0 3.16 ± 12% perf-profile.children.cycles-pp.syscall_return_via_sysret
8.88 ± 3% -5.8 3.03 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64
8.72 ± 4% -5.3 3.39 ± 11% perf-profile.children.cycles-pp.security_file_permission
4.27 ± 3% -2.4 1.89 ± 9% perf-profile.children.cycles-pp.selinux_file_permission
3.15 ± 5% -2.1 1.07 ± 12% perf-profile.children.cycles-pp.file_has_perm
3.48 ± 3% -2.0 1.46 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter
3.80 ± 3% -2.0 1.79 ± 6% perf-profile.children.cycles-pp.mutex_unlock
3.02 ± 5% -1.9 1.11 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter
3.31 ± 4% -1.6 1.75 ± 6% perf-profile.children.cycles-pp.__fdget_pos
2.55 ± 3% -1.5 1.02 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
3.17 ± 4% -1.5 1.70 ± 6% perf-profile.children.cycles-pp.__fget_light
2.10 ± 7% -1.4 0.71 ± 12% perf-profile.children.cycles-pp.fsnotify
2.52 ± 6% -1.3 1.23 ± 8% perf-profile.children.cycles-pp.__fget
1.77 ± 6% -1.1 0.66 ± 9% perf-profile.children.cycles-pp.avc_has_perm
1.89 ± 5% -1.1 0.78 ± 10% perf-profile.children.cycles-pp.___might_sleep
1.44 ± 4% -1.0 0.48 ± 12% perf-profile.children.cycles-pp.__inode_security_revalidate
1.33 ± 10% -0.9 0.40 ± 16% perf-profile.children.cycles-pp.current_time
1.56 ± 3% -0.9 0.66 ± 9% perf-profile.children.cycles-pp.copyout
1.56 ± 8% -0.8 0.73 ± 7% perf-profile.children.cycles-pp.touch_atime
1.33 ± 4% -0.8 0.53 ± 10% perf-profile.children.cycles-pp.__might_sleep
1.15 ± 9% -0.8 0.36 ± 12% perf-profile.children.cycles-pp.atime_needs_update
1.17 ± 4% -0.7 0.43 ± 13% perf-profile.children.cycles-pp.copyin
1.23 ± 5% -0.7 0.54 ± 9% perf-profile.children.cycles-pp.__fsnotify_parent
1.32 ± 7% -0.6 0.68 ± 6% perf-profile.children.cycles-pp.file_update_time
1.02 ± 3% -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__might_fault
2.54 ± 2% -0.5 2.03 perf-profile.children.cycles-pp.mutex_lock
0.78 ± 4% -0.4 0.36 ± 10% perf-profile.children.cycles-pp.fput_many
0.53 ± 4% -0.4 0.15 ± 21% perf-profile.children.cycles-pp.inode_has_perm
0.44 ± 12% -0.3 0.11 ± 22% perf-profile.children.cycles-pp.iov_iter_init
0.76 ± 4% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.53 ± 3% -0.3 0.21 ± 6% perf-profile.children.cycles-pp.rcu_all_qs
0.41 ± 9% -0.3 0.12 ± 15% perf-profile.children.cycles-pp.timespec64_trunc
0.97 -0.3 0.71 ± 3% perf-profile.children.cycles-pp._cond_resched
0.47 ± 7% -0.3 0.21 ± 11% perf-profile.children.cycles-pp.__x64_sys_read
0.45 ± 7% -0.3 0.20 ± 10% perf-profile.children.cycles-pp.__x64_sys_write
0.37 ± 4% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.44 ± 12% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.__sb_start_write
0.31 ± 6% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__sb_end_write
0.36 ± 7% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.rw_verify_area
0.26 ± 14% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.29 ± 8% -0.2 0.14 ± 16% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.22 ± 39% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.__vfs_read
0.21 ± 19% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_write
0.12 ± 17% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.wake_up_q
0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.kill_fasync
0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__mnt_want_write
0.07 ± 25% +0.1 0.12 ± 27% perf-profile.children.cycles-pp.__softirqentry_text_start
0.08 ± 30% +0.1 0.14 ± 26% perf-profile.children.cycles-pp.irq_exit
0.06 ± 6% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__mark_inode_dirty
0.00 +0.1 0.06 perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.is_cpu_allowed
0.03 ±100% +0.1 0.10 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.01 ±173% +0.1 0.09 ± 32% perf-profile.children.cycles-pp.migrate_task_rq_fair
0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.05 ± 61% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.native_load_tls
0.03 ±102% +0.1 0.12 ± 30% perf-profile.children.cycles-pp.set_task_cpu
0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.rb_next
0.10 ± 10% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.generic_update_time
0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.__x64_sys_exit
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.do_exit
0.00 +0.1 0.11 ± 8% perf-profile.children.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_add_valid
0.00 +0.1 0.12 ± 7% perf-profile.children.cycles-pp.resched_curr
0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.13 ± 10% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid
0.00 +0.1 0.15 ± 4% perf-profile.children.cycles-pp.set_next_buddy
0.00 +0.2 0.16 ± 5% perf-profile.children.cycles-pp.deactivate_task
0.05 ± 9% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.rb_erase
0.06 ± 13% +0.2 0.23 ± 9% perf-profile.children.cycles-pp.finish_wait
0.17 ± 12% +0.2 0.35 ± 2% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.01 ±173% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.cpumask_next
0.07 ± 22% +0.3 0.40 ± 2% perf-profile.children.cycles-pp.prepare_to_wait
0.03 ±100% +0.4 0.39 ± 5% perf-profile.children.cycles-pp.update_min_vruntime
0.03 ±100% +0.4 0.39 ± 12% perf-profile.children.cycles-pp.cpus_share_cache
0.06 ± 14% +0.4 0.43 perf-profile.children.cycles-pp.clear_buddies
0.11 ± 15% +0.4 0.50 ± 3% perf-profile.children.cycles-pp.cpuacct_charge
0.10 ± 26% +0.4 0.49 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.32 ± 31% +0.4 0.75 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending
0.11 ± 26% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.sched_clock
0.34 ± 32% +0.5 0.80 ± 15% perf-profile.children.cycles-pp.scheduler_ipi
0.08 ± 12% +0.5 0.56 ± 7% perf-profile.children.cycles-pp.account_entity_enqueue
0.13 ± 26% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.14 ± 26% +0.6 0.77 ± 4% perf-profile.children.cycles-pp.__calc_delta
0.08 ± 16% +0.7 0.76 ± 3% perf-profile.children.cycles-pp.account_entity_dequeue
0.84 ± 5% +0.7 1.54 ± 5% perf-profile.children.cycles-pp.__lock_text_start
1.61 ± 16% +0.7 2.31 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.42 ± 31% +0.7 1.12 ± 13% perf-profile.children.cycles-pp.reschedule_interrupt
0.24 ± 28% +0.9 1.11 perf-profile.children.cycles-pp.native_write_msr
0.14 ± 22% +0.9 1.01 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup
0.24 ± 32% +0.9 1.12 ± 7% perf-profile.children.cycles-pp.finish_task_switch
0.22 ± 22% +0.9 1.10 ± 3% perf-profile.children.cycles-pp.find_next_bit
0.22 ± 23% +0.9 1.12 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.21 ± 32% +0.9 1.14 ± 3% perf-profile.children.cycles-pp.update_cfs_group
0.16 ± 23% +1.0 1.15 ± 4% perf-profile.children.cycles-pp.check_preempt_curr
0.21 ± 30% +1.0 1.21 ± 2% perf-profile.children.cycles-pp.__switch_to
0.23 ± 21% +1.0 1.24 ± 2% perf-profile.children.cycles-pp.__switch_to_asm
0.16 ± 38% +1.0 1.20 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
0.15 ± 26% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.pick_next_entity
0.15 ± 20% +1.1 1.21 ± 7% perf-profile.children.cycles-pp.put_prev_entity
0.17 ± 22% +1.1 1.23 ± 4% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.19 ± 10% +1.3 1.52 ± 10% perf-profile.children.cycles-pp.__enqueue_entity
0.24 ± 23% +1.3 1.59 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se
0.28 ± 23% +1.4 1.64 ± 3% perf-profile.children.cycles-pp.switch_fpu_return
0.31 ± 29% +1.6 1.88 ± 5% perf-profile.children.cycles-pp.update_rq_clock
0.44 ± 21% +1.8 2.24 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap
0.33 ± 20% +1.8 2.15 ± 4% perf-profile.children.cycles-pp.dequeue_entity
23.28 ± 2% +2.0 25.27 ± 2% perf-profile.children.cycles-pp.ksys_read
0.46 ± 25% +2.1 2.57 ± 5% perf-profile.children.cycles-pp.update_load_avg
0.20 ± 21% +2.1 2.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.91 ± 29% +2.2 3.12 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.38 ± 23% +2.4 2.80 ± 4% perf-profile.children.cycles-pp.reweight_entity
0.52 ± 18% +2.4 2.96 ± 3% perf-profile.children.cycles-pp.enqueue_entity
0.49 ± 23% +2.6 3.10 ± 4% perf-profile.children.cycles-pp.update_curr
20.92 ± 2% +3.2 24.12 ± 2% perf-profile.children.cycles-pp.vfs_read
0.21 ± 23% +3.4 3.65 ± 7% perf-profile.children.cycles-pp._raw_spin_lock
0.76 ± 24% +3.9 4.67 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair
0.66 ± 34% +4.4 5.06 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.81 ± 22% +5.0 5.83 ± 4% perf-profile.children.cycles-pp.dequeue_task_fair
1.01 ± 21% +5.8 6.86 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair
1.06 ± 21% +6.0 7.10 ± 4% perf-profile.children.cycles-pp.activate_task
1.07 ± 21% +6.1 7.15 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate
13.19 +7.8 20.94 ± 3% perf-profile.children.cycles-pp.new_sync_read
11.61 +8.7 20.29 ± 3% perf-profile.children.cycles-pp.pipe_read
1.45 ± 25% +8.9 10.35 ± 7% perf-profile.children.cycles-pp.available_idle_cpu
78.85 ± 2% +9.8 88.66 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
77.90 ± 2% +10.1 88.00 perf-profile.children.cycles-pp.do_syscall_64
2.42 ± 21% +12.9 15.36 ± 4% perf-profile.children.cycles-pp.pipe_wait
2.71 ± 26% +15.4 18.12 ± 4% perf-profile.children.cycles-pp.__schedule
2.76 ± 25% +15.6 18.37 ± 4% perf-profile.children.cycles-pp.schedule
27.59 ± 2% +18.6 46.17 ± 3% perf-profile.children.cycles-pp.ksys_write
2.26 ± 23% +19.0 21.30 ± 7% perf-profile.children.cycles-pp.select_idle_sibling
24.98 ± 2% +19.9 44.88 ± 3% perf-profile.children.cycles-pp.vfs_write
2.48 ± 23% +20.0 22.46 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair
19.06 ± 3% +23.3 42.41 ± 4% perf-profile.children.cycles-pp.new_sync_write
17.98 ± 4% +23.9 41.93 ± 4% perf-profile.children.cycles-pp.pipe_write
4.03 ± 21% +30.6 34.60 ± 6% perf-profile.children.cycles-pp.try_to_wake_up
3.97 ± 22% +30.8 34.74 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function
4.30 ± 20% +30.9 35.19 ± 6% perf-profile.children.cycles-pp.__wake_up_common
6.06 ± 14% +31.2 37.21 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock
24.41 ± 6% -15.5 8.96 ± 10% perf-profile.self.cycles-pp.do_syscall_64
9.18 ± 6% -6.0 3.14 ± 12% perf-profile.self.cycles-pp.syscall_return_via_sysret
8.63 ± 5% -5.6 3.00 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64
3.71 ± 3% -2.0 1.73 ± 6% perf-profile.self.cycles-pp.mutex_unlock
2.47 ± 3% -1.5 0.96 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
2.76 ± 3% -1.4 1.35 ± 8% perf-profile.self.cycles-pp.selinux_file_permission
2.02 ± 7% -1.3 0.68 ± 12% perf-profile.self.cycles-pp.fsnotify
2.46 ± 6% -1.3 1.20 ± 8% perf-profile.self.cycles-pp.__fget
1.75 ± 6% -1.1 0.64 ± 10% perf-profile.self.cycles-pp.avc_has_perm
1.83 ± 5% -1.1 0.75 ± 10% perf-profile.self.cycles-pp.___might_sleep
1.75 -1.0 0.77 ± 7% perf-profile.self.cycles-pp.pipe_write
1.20 ± 4% -0.7 0.46 ± 10% perf-profile.self.cycles-pp.__might_sleep
1.29 ± 11% -0.7 0.58 ± 10% perf-profile.self.cycles-pp.new_sync_read
1.63 ± 4% -0.7 0.97 ± 5% perf-profile.self.cycles-pp.pipe_read
1.15 ± 5% -0.7 0.49 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent
0.89 ± 8% -0.6 0.27 ± 15% perf-profile.self.cycles-pp.copy_page_from_iter
0.89 ± 4% -0.6 0.29 ± 14% perf-profile.self.cycles-pp.security_file_permission
0.83 ± 5% -0.6 0.24 ± 21% perf-profile.self.cycles-pp.file_has_perm
0.90 ± 2% -0.5 0.41 ± 6% perf-profile.self.cycles-pp.new_sync_write
0.67 ± 10% -0.5 0.20 ± 15% perf-profile.self.cycles-pp.current_time
0.83 ± 4% -0.4 0.41 ± 10% perf-profile.self.cycles-pp.copy_page_to_iter
0.72 ± 7% -0.4 0.31 ± 9% perf-profile.self.cycles-pp.vfs_write
0.75 ± 4% -0.4 0.35 ± 9% perf-profile.self.cycles-pp.fput_many
1.01 ± 4% -0.3 0.69 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.45 ± 4% -0.3 0.14 ± 19% perf-profile.self.cycles-pp.inode_has_perm
0.54 ± 8% -0.3 0.23 ± 9% perf-profile.self.cycles-pp.file_update_time
0.41 ± 10% -0.3 0.11 ± 20% perf-profile.self.cycles-pp.iov_iter_init
0.49 ± 7% -0.3 0.19 ± 8% perf-profile.self.cycles-pp.atime_needs_update
0.44 ± 7% -0.3 0.15 ± 14% perf-profile.self.cycles-pp.ksys_read
0.44 ± 7% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.__inode_security_revalidate
0.41 ± 2% -0.3 0.15 ± 8% perf-profile.self.cycles-pp.rcu_all_qs
0.43 ± 8% -0.3 0.17 ± 11% perf-profile.self.cycles-pp.ksys_write
0.37 ± 9% -0.3 0.11 ± 17% perf-profile.self.cycles-pp.timespec64_trunc
0.42 ± 14% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__sb_start_write
0.39 ± 7% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.__x64_sys_write
0.40 ± 8% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.__x64_sys_read
0.41 ± 7% -0.2 0.20 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock
0.60 ± 4% -0.2 0.39 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
1.25 -0.2 1.04 ± 2% perf-profile.self.cycles-pp.mutex_lock
0.30 ± 6% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.__sb_end_write
0.35 ± 5% -0.2 0.18 ± 3% perf-profile.self.cycles-pp.rw_verify_area
0.34 ± 7% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.touch_atime
0.32 ± 4% -0.2 0.15 ± 14% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.23 ± 16% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.74 ± 3% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.vfs_read
0.60 ± 3% -0.2 0.45 ± 4% perf-profile.self.cycles-pp.__fget_light
0.27 ± 8% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.18 ± 45% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__vfs_read
0.18 ± 5% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.__fdget_pos
0.25 ± 5% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.__might_fault
0.07 ± 7% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.kill_fasync
0.01 ±173% +0.1 0.06 ± 17% perf-profile.self.cycles-pp.sched_ttwu_pending
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.scheduler_ipi
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rcu_note_context_switch
0.01 ±173% +0.1 0.07 ± 12% perf-profile.self.cycles-pp.generic_update_time
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.cpumask_next
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ttwu_do_activate
0.00 +0.1 0.06 perf-profile.self.cycles-pp.__mnt_want_write
0.06 ± 14% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mark_inode_dirty
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.is_cpu_allowed
0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.sched_clock_cpu
0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.sched_clock
0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.00 +0.1 0.08 perf-profile.self.cycles-pp.native_load_tls
0.05 ± 60% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret
0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.rb_next
0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.__list_add_valid
0.00 +0.1 0.10 ± 10% perf-profile.self.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.resched_curr
0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.32 ± 7% +0.1 0.45 ± 5% perf-profile.self.cycles-pp.__wake_up_common
0.13 ± 10% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid
0.00 +0.1 0.13 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function
0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.check_preempt_curr
0.04 ± 58% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.finish_wait
0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.set_next_buddy
0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.put_prev_entity
0.00 +0.2 0.15 ± 5% perf-profile.self.cycles-pp.deactivate_task
0.05 ± 9% +0.2 0.21 ± 5% perf-profile.self.cycles-pp.rb_erase
0.00 +0.2 0.15 ± 10% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.00 +0.2 0.18 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.15 ± 7% +0.2 0.34 ± 2% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.03 ±100% +0.2 0.21 ± 8% perf-profile.self.cycles-pp.pipe_wait
0.03 ±102% +0.2 0.24 ± 3% perf-profile.self.cycles-pp.activate_task
0.01 ±173% +0.2 0.23 ± 3% perf-profile.self.cycles-pp.prepare_to_wait
0.03 ±100% +0.2 0.25 ± 4% perf-profile.self.cycles-pp.set_next_entity
0.02 ±173% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.04 ±100% +0.3 0.29 ± 5% perf-profile.self.cycles-pp.enqueue_entity
0.08 ± 31% +0.3 0.34 ± 2% perf-profile.self.cycles-pp.schedule
0.78 ± 3% +0.3 1.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.09 ± 24% +0.3 0.42 ± 7% perf-profile.self.cycles-pp.finish_task_switch
0.03 ±100% +0.3 0.37 ± 6% perf-profile.self.cycles-pp.update_min_vruntime
0.03 ±102% +0.4 0.38 perf-profile.self.cycles-pp.clear_buddies
0.03 ±100% +0.4 0.39 ± 12% perf-profile.self.cycles-pp.cpus_share_cache
0.07 ± 22% +0.4 0.44 ± 7% perf-profile.self.cycles-pp.check_preempt_wakeup
0.09 ± 27% +0.4 0.47 ± 4% perf-profile.self.cycles-pp.native_sched_clock
0.07 ± 26% +0.4 0.45 ± 4% perf-profile.self.cycles-pp.pick_next_entity
0.11 ± 15% +0.4 0.50 ± 3% perf-profile.self.cycles-pp.cpuacct_charge
0.07 ± 14% +0.5 0.53 ± 8% perf-profile.self.cycles-pp.account_entity_enqueue
0.10 ± 21% +0.5 0.56 ± 8% perf-profile.self.cycles-pp.try_to_wake_up
0.10 ± 18% +0.5 0.62 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.14 ± 26% +0.6 0.75 ± 4% perf-profile.self.cycles-pp.__calc_delta
0.06 ± 13% +0.6 0.68 ± 4% perf-profile.self.cycles-pp.account_entity_dequeue
0.15 ± 27% +0.6 0.79 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair
0.17 ± 30% +0.7 0.89 ± 4% perf-profile.self.cycles-pp.update_load_avg
0.15 ± 20% +0.7 0.88 ± 4% perf-profile.self.cycles-pp.reweight_entity
0.15 ± 23% +0.8 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.24 ± 30% +0.9 1.11 perf-profile.self.cycles-pp.native_write_msr
0.20 ± 23% +0.9 1.07 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair
0.20 ± 22% +0.9 1.07 ± 3% perf-profile.self.cycles-pp.find_next_bit
0.20 ± 31% +0.9 1.11 ± 3% perf-profile.self.cycles-pp.update_cfs_group
0.20 ± 28% +0.9 1.14 ± 2% perf-profile.self.cycles-pp.__switch_to
0.14 ± 37% +1.0 1.13 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
0.21 ± 24% +1.0 1.22 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
0.24 ± 23% +1.1 1.31 ± 5% perf-profile.self.cycles-pp.cpumask_next_wrap
0.23 ± 32% +1.3 1.49 ± 5% perf-profile.self.cycles-pp.update_rq_clock
0.19 ± 28% +1.3 1.45 ± 6% perf-profile.self.cycles-pp.update_curr
0.23 ± 23% +1.3 1.55 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.19 ± 10% +1.3 1.51 ± 10% perf-profile.self.cycles-pp.__enqueue_entity
0.28 ± 22% +1.3 1.63 ± 3% perf-profile.self.cycles-pp.switch_fpu_return
0.11 ± 19% +1.6 1.72 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.42 ± 26% +1.7 2.12 ± 2% perf-profile.self.cycles-pp.__schedule
0.19 ± 20% +2.0 2.20 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.91 ± 29% +2.2 3.11 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.32 ± 19% +7.7 8.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling
1.44 ± 25% +8.8 10.23 ± 7% perf-profile.self.cycles-pp.available_idle_cpu
439.00 +11.7% 490.50 interrupts.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
436.25 +13.1% 493.25 interrupts.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
436.25 +12.4% 490.50 interrupts.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
436.25 +12.4% 490.50 interrupts.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
509.75 ± 3% +545.1% 3288 ±120% interrupts.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
474.00 ± 3% +49.7% 709.75 ± 24% interrupts.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
588.00 ± 35% +196.0% 1740 ± 50% interrupts.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
436.75 ± 2% +12.8% 492.50 interrupts.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
438.00 +39.5% 611.00 ± 33% interrupts.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
436.25 +13.8% 496.25 interrupts.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
436.25 +14.3% 498.50 interrupts.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
437.00 +12.2% 490.50 interrupts.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
436.25 +12.4% 490.50 interrupts.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
436.25 +12.4% 490.50 interrupts.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
441.00 +11.8% 493.00 interrupts.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
437.25 +13.3% 495.25 interrupts.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
436.25 +12.4% 490.50 interrupts.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
436.25 +12.4% 490.50 interrupts.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
436.25 +12.4% 490.50 interrupts.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
436.25 +12.4% 490.50 interrupts.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
437.25 ± 2% +12.2% 490.50 interrupts.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
436.25 +12.6% 491.25 interrupts.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
440.75 +12.4% 495.25 ± 2% interrupts.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
436.25 +12.6% 491.00 interrupts.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
436.75 +13.1% 493.75 interrupts.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
436.25 +12.8% 492.25 interrupts.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
436.25 +12.8% 492.25 interrupts.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
436.25 +12.4% 490.50 interrupts.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
436.25 +12.4% 490.50 interrupts.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
436.25 +12.4% 490.50 interrupts.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
436.25 +14.6% 499.75 ± 2% interrupts.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
436.25 +12.4% 490.50 interrupts.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
436.50 +12.4% 490.50 interrupts.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
436.25 +12.4% 490.50 interrupts.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
436.25 +12.4% 490.50 interrupts.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
436.25 +12.6% 491.00 interrupts.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
441.25 ± 3% +14.1% 503.25 ± 3% interrupts.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
436.25 +12.4% 490.50 interrupts.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
436.25 +14.4% 499.25 ± 3% interrupts.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
439.25 ± 2% +11.7% 490.50 interrupts.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
437.75 +12.1% 490.50 interrupts.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
436.50 +12.4% 490.50 interrupts.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55
436.25 +12.4% 490.50 interrupts.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
439.00 ± 2% +11.7% 490.50 interrupts.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
442.25 ± 3% +10.9% 490.50 interrupts.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
1989 +12.1% 2230 interrupts.9:IR-IO-APIC.9-fasteoi.acpi
6596036 ± 10% -21.1% 5206443 ± 6% interrupts.CAL:Function_call_interrupts
86097 ± 13% -18.4% 70229 ± 7% interrupts.CPU0.CAL:Function_call_interrupts
1785369 +12.4% 2006294 interrupts.CPU0.LOC:Local_timer_interrupts
102823 ± 12% -18.3% 84031 ± 5% interrupts.CPU0.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496548 ± 4% interrupts.CPU0.TRM:Thermal_event_interrupts
1989 +12.1% 2230 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi
90370 ± 8% -21.4% 70995 ± 6% interrupts.CPU1.CAL:Function_call_interrupts
1784834 +12.4% 2006046 interrupts.CPU1.LOC:Local_timer_interrupts
3088391 ± 15% -20.9% 2441882 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts
105391 ± 8% -18.6% 85793 ± 6% interrupts.CPU1.TLB:TLB_shootdowns
388373 ± 12% +27.9% 496558 ± 4% interrupts.CPU1.TRM:Thermal_event_interrupts
88236 ± 11% -19.5% 71001 ± 7% interrupts.CPU10.CAL:Function_call_interrupts
1785471 +12.3% 2004740 interrupts.CPU10.LOC:Local_timer_interrupts
3040774 ± 4% -21.1% 2398396 ± 8% interrupts.CPU10.RES:Rescheduling_interrupts
104483 ± 10% -18.6% 85025 ± 5% interrupts.CPU10.TLB:TLB_shootdowns
388417 ± 12% +27.8% 496560 ± 4% interrupts.CPU10.TRM:Thermal_event_interrupts
1785039 +12.3% 2005274 interrupts.CPU11.LOC:Local_timer_interrupts
925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.NMI:Non-maskable_interrupts
925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.PMI:Performance_monitoring_interrupts
3094193 ± 14% -21.5% 2430228 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts
102398 ± 10% -16.1% 85884 ± 5% interrupts.CPU11.TLB:TLB_shootdowns
388413 ± 12% +27.8% 496474 ± 4% interrupts.CPU11.TRM:Thermal_event_interrupts
588.00 ± 35% +196.0% 1740 ± 50% interrupts.CPU12.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
87438 ± 9% -18.1% 71580 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
1785376 +12.3% 2005068 interrupts.CPU12.LOC:Local_timer_interrupts
103057 ± 10% -16.4% 86186 ± 6% interrupts.CPU12.TLB:TLB_shootdowns
388416 ± 12% +27.8% 496490 ± 4% interrupts.CPU12.TRM:Thermal_event_interrupts
88735 ± 9% -19.4% 71496 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1785460 +12.3% 2005445 interrupts.CPU13.LOC:Local_timer_interrupts
104230 ± 9% -17.5% 86038 ± 6% interrupts.CPU13.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU13.TRM:Thermal_event_interrupts
88327 ± 12% -19.0% 71507 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
1785094 +12.3% 2005441 interrupts.CPU14.LOC:Local_timer_interrupts
103317 ± 11% -18.1% 84581 ± 6% interrupts.CPU14.TLB:TLB_shootdowns
388412 ± 12% +27.8% 496564 ± 4% interrupts.CPU14.TRM:Thermal_event_interrupts
86972 ± 10% -17.3% 71898 ± 7% interrupts.CPU15.CAL:Function_call_interrupts
1784668 +12.4% 2005375 interrupts.CPU15.LOC:Local_timer_interrupts
3039799 ± 9% -21.1% 2397505 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts
103367 ± 12% -16.9% 85892 ± 6% interrupts.CPU15.TLB:TLB_shootdowns
388342 ± 12% +27.9% 496559 ± 4% interrupts.CPU15.TRM:Thermal_event_interrupts
436.75 ± 2% +12.8% 492.50 interrupts.CPU16.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
1784750 +12.4% 2005330 interrupts.CPU16.LOC:Local_timer_interrupts
55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.NMI:Non-maskable_interrupts
55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.PMI:Performance_monitoring_interrupts
2969996 ± 10% -18.7% 2413141 ± 9% interrupts.CPU16.RES:Rescheduling_interrupts
102367 ± 11% -13.7% 88305 ± 3% interrupts.CPU16.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496563 ± 4% interrupts.CPU16.TRM:Thermal_event_interrupts
1785363 +12.3% 2005849 interrupts.CPU17.LOC:Local_timer_interrupts
388418 ± 12% +27.8% 496560 ± 4% interrupts.CPU17.TRM:Thermal_event_interrupts
438.00 +39.5% 611.00 ± 33% interrupts.CPU18.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
88992 ± 12% -22.3% 69145 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1785398 +12.1% 2001316 interrupts.CPU18.LOC:Local_timer_interrupts
2199105 ± 7% +115.2% 4731679 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
108465 ± 8% -23.0% 83467 ± 5% interrupts.CPU18.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380131 ± 2% interrupts.CPU18.TRM:Thermal_event_interrupts
436.25 +13.8% 496.25 interrupts.CPU19.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
92059 ± 6% -24.2% 69776 ± 5% interrupts.CPU19.CAL:Function_call_interrupts
1784665 +12.2% 2001579 interrupts.CPU19.LOC:Local_timer_interrupts
1284622 ± 7% +225.8% 4185208 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts
108492 ± 7% -23.4% 83059 ± 4% interrupts.CPU19.TLB:TLB_shootdowns
329166 ± 3% +15.5% 380141 ± 2% interrupts.CPU19.TRM:Thermal_event_interrupts
1784512 +12.4% 2005572 interrupts.CPU2.LOC:Local_timer_interrupts
3091132 ± 8% -22.3% 2401342 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts
102562 ± 9% -16.5% 85643 ± 5% interrupts.CPU2.TLB:TLB_shootdowns
388380 ± 12% +27.8% 496544 ± 4% interrupts.CPU2.TRM:Thermal_event_interrupts
436.25 +14.3% 498.50 interrupts.CPU20.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
91785 ± 9% -23.9% 69853 ± 4% interrupts.CPU20.CAL:Function_call_interrupts
1785284 +12.1% 2001952 interrupts.CPU20.LOC:Local_timer_interrupts
1281621 ± 12% +203.6% 3890516 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts
106297 ± 9% -19.9% 85180 ± 3% interrupts.CPU20.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU20.TRM:Thermal_event_interrupts
437.00 +12.2% 490.50 interrupts.CPU21.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
92635 ± 7% -23.9% 70512 ± 6% interrupts.CPU21.CAL:Function_call_interrupts
1785480 +12.1% 2001723 interrupts.CPU21.LOC:Local_timer_interrupts
1595447 ± 6% +187.8% 4591004 interrupts.CPU21.RES:Rescheduling_interrupts
107575 ± 7% -21.6% 84296 ± 5% interrupts.CPU21.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380086 ± 2% interrupts.CPU21.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU22.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
92674 ± 10% -24.4% 70047 ± 6% interrupts.CPU22.CAL:Function_call_interrupts
1784881 +12.1% 2001482 interrupts.CPU22.LOC:Local_timer_interrupts
1605143 ± 6% +179.5% 4486945 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts
108749 ± 10% -23.3% 83409 ± 5% interrupts.CPU22.TLB:TLB_shootdowns
329295 ± 3% +15.4% 380136 ± 2% interrupts.CPU22.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU23.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
90975 ± 11% -23.6% 69492 ± 5% interrupts.CPU23.CAL:Function_call_interrupts
1785236 +12.1% 2001811 interrupts.CPU23.LOC:Local_timer_interrupts
1600988 ± 2% +180.2% 4486667 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts
106819 ± 10% -22.3% 83007 ± 4% interrupts.CPU23.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380146 ± 2% interrupts.CPU23.TRM:Thermal_event_interrupts
441.00 +11.8% 493.00 interrupts.CPU24.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
91919 ± 11% -23.1% 70689 ± 5% interrupts.CPU24.CAL:Function_call_interrupts
1784996 +12.1% 2000922 interrupts.CPU24.LOC:Local_timer_interrupts
1612237 ± 5% +188.4% 4649702 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts
108187 ± 8% -22.5% 83831 ± 3% interrupts.CPU24.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380122 ± 2% interrupts.CPU24.TRM:Thermal_event_interrupts
437.25 +13.3% 495.25 interrupts.CPU25.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
88417 ± 12% -20.8% 70029 ± 4% interrupts.CPU25.CAL:Function_call_interrupts
1785404 +12.0% 1999622 interrupts.CPU25.LOC:Local_timer_interrupts
1612074 ± 2% +182.4% 4551699 interrupts.CPU25.RES:Rescheduling_interrupts
105324 ± 10% -20.1% 84201 ± 4% interrupts.CPU25.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU25.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU26.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
92632 ± 12% -23.5% 70907 ± 5% interrupts.CPU26.CAL:Function_call_interrupts
1784730 +12.1% 2000264 interrupts.CPU26.LOC:Local_timer_interrupts
1639148 ± 6% +188.0% 4720149 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts
108099 ± 11% -21.6% 84727 ± 4% interrupts.CPU26.TLB:TLB_shootdowns
329271 ± 3% +15.4% 379984 ± 2% interrupts.CPU26.TRM:Thermal_event_interrupts
91312 ± 12% -23.4% 69963 ± 6% interrupts.CPU27.CAL:Function_call_interrupts
1785171 +12.1% 2001743 interrupts.CPU27.LOC:Local_timer_interrupts
1704847 ± 11% +179.4% 4764019 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts
106515 ± 11% -21.6% 83491 ± 5% interrupts.CPU27.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380080 ± 2% interrupts.CPU27.TRM:Thermal_event_interrupts
89178 ± 8% -22.0% 69565 ± 6% interrupts.CPU28.CAL:Function_call_interrupts
1785194 +12.1% 2001836 interrupts.CPU28.LOC:Local_timer_interrupts
1593025 ± 8% +185.7% 4551782 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts
104408 ± 10% -19.2% 84365 ± 6% interrupts.CPU28.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380142 ± 2% interrupts.CPU28.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU29.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
90850 ± 12% -21.9% 70993 ± 6% interrupts.CPU29.CAL:Function_call_interrupts
1785398 +12.1% 2000545 interrupts.CPU29.LOC:Local_timer_interrupts
1657508 ± 5% +179.4% 4631109 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts
106383 ± 13% -20.8% 84284 ± 4% interrupts.CPU29.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380133 ± 2% interrupts.CPU29.TRM:Thermal_event_interrupts
509.75 ± 3% +545.1% 3288 ±120% interrupts.CPU3.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
87008 ± 11% -19.9% 69730 ± 5% interrupts.CPU3.CAL:Function_call_interrupts
1784446 +12.4% 2005368 interrupts.CPU3.LOC:Local_timer_interrupts
3169674 ± 9% -20.4% 2521963 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts
103703 ± 12% -17.1% 85958 ± 7% interrupts.CPU3.TLB:TLB_shootdowns
388356 ± 12% +27.9% 496551 ± 4% interrupts.CPU3.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU30.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
92043 ± 12% -24.2% 69765 ± 5% interrupts.CPU30.CAL:Function_call_interrupts
1785175 +12.1% 2001248 interrupts.CPU30.LOC:Local_timer_interrupts
1637465 ± 6% +191.3% 4769701 ± 3% interrupts.CPU30.RES:Rescheduling_interrupts
106836 ± 11% -22.5% 82810 ± 5% interrupts.CPU30.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU30.TRM:Thermal_event_interrupts
1785301 +12.0% 1999270 interrupts.CPU31.LOC:Local_timer_interrupts
62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.NMI:Non-maskable_interrupts
62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.PMI:Performance_monitoring_interrupts
1589500 ± 5% +184.1% 4515325 interrupts.CPU31.RES:Rescheduling_interrupts
106576 ± 11% -21.5% 83698 ± 5% interrupts.CPU31.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU31.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU32.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
93348 ± 12% -24.4% 70595 ± 8% interrupts.CPU32.CAL:Function_call_interrupts
1785245 +12.1% 2001145 interrupts.CPU32.LOC:Local_timer_interrupts
1626883 ± 6% +176.3% 4494456 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts
107151 ± 12% -21.4% 84236 ± 5% interrupts.CPU32.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380131 ± 2% interrupts.CPU32.TRM:Thermal_event_interrupts
437.25 ± 2% +12.2% 490.50 interrupts.CPU33.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
91487 ± 12% -22.7% 70758 ± 3% interrupts.CPU33.CAL:Function_call_interrupts
1785371 +12.0% 1999987 interrupts.CPU33.LOC:Local_timer_interrupts
1578761 ± 6% +201.2% 4754762 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts
106071 ± 12% -21.3% 83430 ± 2% interrupts.CPU33.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU33.TRM:Thermal_event_interrupts
436.25 +12.6% 491.25 interrupts.CPU34.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
91116 ± 11% -24.3% 68943 ± 7% interrupts.CPU34.CAL:Function_call_interrupts
1785353 +12.1% 2001525 interrupts.CPU34.LOC:Local_timer_interrupts
1553828 ± 4% +186.0% 4444035 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts
105575 ± 10% -20.7% 83762 ± 4% interrupts.CPU34.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380139 ± 2% interrupts.CPU34.TRM:Thermal_event_interrupts
440.75 +12.4% 495.25 ± 2% interrupts.CPU35.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
91396 ± 11% -25.6% 67986 ± 11% interrupts.CPU35.CAL:Function_call_interrupts
1785237 +12.0% 2000288 interrupts.CPU35.LOC:Local_timer_interrupts
1602616 ± 10% +189.7% 4642115 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts
107669 ± 12% -21.3% 84736 ± 4% interrupts.CPU35.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380145 ± 2% interrupts.CPU35.TRM:Thermal_event_interrupts
436.25 +12.6% 491.00 interrupts.CPU36.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
93553 ± 8% -19.8% 75044 ± 6% interrupts.CPU36.CAL:Function_call_interrupts
1784636 +12.4% 2005389 interrupts.CPU36.LOC:Local_timer_interrupts
2588984 ± 6% +25.8% 3256348 ± 6% interrupts.CPU36.RES:Rescheduling_interrupts
107053 ± 7% -19.0% 86754 ± 5% interrupts.CPU36.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496560 ± 4% interrupts.CPU36.TRM:Thermal_event_interrupts
436.75 +13.1% 493.75 interrupts.CPU37.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
89518 ± 10% -18.9% 72619 ± 5% interrupts.CPU37.CAL:Function_call_interrupts
1784648 +12.4% 2005651 interrupts.CPU37.LOC:Local_timer_interrupts
3112476 ± 10% -20.8% 2464103 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts
102576 ± 10% -14.4% 87794 ± 6% interrupts.CPU37.TLB:TLB_shootdowns
388412 ± 12% +27.8% 496563 ± 4% interrupts.CPU37.TRM:Thermal_event_interrupts
436.25 +12.8% 492.25 interrupts.CPU38.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
92566 ± 10% -20.6% 73466 ± 6% interrupts.CPU38.CAL:Function_call_interrupts
1785132 +12.4% 2005883 interrupts.CPU38.LOC:Local_timer_interrupts
3163520 ± 10% -26.0% 2340820 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts
105823 ± 10% -18.0% 86785 ± 4% interrupts.CPU38.TLB:TLB_shootdowns
388401 ± 12% +27.8% 496554 ± 4% interrupts.CPU38.TRM:Thermal_event_interrupts
436.25 +12.8% 492.25 interrupts.CPU39.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
93197 ± 11% -21.8% 72854 ± 7% interrupts.CPU39.CAL:Function_call_interrupts
1784624 +12.4% 2005934 interrupts.CPU39.LOC:Local_timer_interrupts
105953 ± 11% -20.8% 83925 ± 6% interrupts.CPU39.TLB:TLB_shootdowns
388418 ± 12% +27.8% 496548 ± 4% interrupts.CPU39.TRM:Thermal_event_interrupts
86883 ± 13% -17.8% 71449 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
1784433 +12.4% 2005462 interrupts.CPU4.LOC:Local_timer_interrupts
3119739 ± 9% -21.1% 2461192 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts
104829 ± 11% -19.1% 84802 ± 5% interrupts.CPU4.TLB:TLB_shootdowns
388417 ± 12% +27.8% 496563 ± 4% interrupts.CPU4.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU40.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
91881 ± 12% -21.4% 72188 ± 8% interrupts.CPU40.CAL:Function_call_interrupts
1784831 +12.4% 2005991 interrupts.CPU40.LOC:Local_timer_interrupts
3101288 ± 10% -17.2% 2569221 ± 7% interrupts.CPU40.RES:Rescheduling_interrupts
105884 ± 10% -19.0% 85772 ± 5% interrupts.CPU40.TLB:TLB_shootdowns
388415 ± 12% +27.8% 496564 ± 4% interrupts.CPU40.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU41.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
91739 ± 12% -19.9% 73475 ± 7% interrupts.CPU41.CAL:Function_call_interrupts
1785231 +12.3% 2005398 interrupts.CPU41.LOC:Local_timer_interrupts
3223405 ± 11% -27.9% 2325498 ± 5% interrupts.CPU41.RES:Rescheduling_interrupts
104410 ± 11% -18.7% 84846 ± 6% interrupts.CPU41.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU41.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU42.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
1785354 +12.4% 2006034 interrupts.CPU42.LOC:Local_timer_interrupts
3068304 ± 9% -22.1% 2390347 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts
107102 ± 9% -20.7% 84890 ± 4% interrupts.CPU42.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496564 ± 4% interrupts.CPU42.TRM:Thermal_event_interrupts
436.25 +14.6% 499.75 ± 2% interrupts.CPU43.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
92178 ± 9% -18.3% 75306 ± 7% interrupts.CPU43.CAL:Function_call_interrupts
1784759 +12.4% 2005986 interrupts.CPU43.LOC:Local_timer_interrupts
6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.NMI:Non-maskable_interrupts
6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.PMI:Performance_monitoring_interrupts
104773 ± 9% -17.6% 86369 ± 6% interrupts.CPU43.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496556 ± 4% interrupts.CPU43.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU44.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
92387 ± 10% -21.9% 72174 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
1784603 +12.4% 2006064 interrupts.CPU44.LOC:Local_timer_interrupts
106179 ± 9% -18.6% 86461 ± 6% interrupts.CPU44.TLB:TLB_shootdowns
388426 ± 12% +27.8% 496562 ± 4% interrupts.CPU44.TRM:Thermal_event_interrupts
436.50 +12.4% 490.50 interrupts.CPU45.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
93641 ± 7% -21.9% 73155 ± 6% interrupts.CPU45.CAL:Function_call_interrupts
1784811 +12.4% 2005654 interrupts.CPU45.LOC:Local_timer_interrupts
3093195 ± 11% -15.6% 2609354 ± 4% interrupts.CPU45.RES:Rescheduling_interrupts
106463 ± 7% -19.6% 85593 ± 5% interrupts.CPU45.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496564 ± 4% interrupts.CPU45.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU46.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
91739 ± 10% -19.0% 74279 ± 8% interrupts.CPU46.CAL:Function_call_interrupts
1784609 +12.4% 2005522 interrupts.CPU46.LOC:Local_timer_interrupts
2970471 ± 9% -21.0% 2345439 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts
108158 ± 10% -19.8% 86743 ± 4% interrupts.CPU46.TLB:TLB_shootdowns
388368 ± 12% +27.8% 496469 ± 4% interrupts.CPU46.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU47.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
92856 ± 9% -20.0% 74327 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1784835 +12.4% 2005700 interrupts.CPU47.LOC:Local_timer_interrupts
3116242 ± 10% -20.4% 2480398 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts
104927 ± 9% -18.6% 85393 ± 6% interrupts.CPU47.TLB:TLB_shootdowns
388403 ± 12% +27.8% 496554 ± 4% interrupts.CPU47.TRM:Thermal_event_interrupts
436.25 +12.6% 491.00 interrupts.CPU48.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
92889 ± 11% -21.5% 72890 ± 8% interrupts.CPU48.CAL:Function_call_interrupts
1784772 +12.4% 2006205 interrupts.CPU48.LOC:Local_timer_interrupts
3175210 ± 12% -19.7% 2548339 ± 13% interrupts.CPU48.RES:Rescheduling_interrupts
105183 ± 11% -18.6% 85618 ± 6% interrupts.CPU48.TLB:TLB_shootdowns
388426 ± 12% +27.8% 496563 ± 4% interrupts.CPU48.TRM:Thermal_event_interrupts
441.25 ± 3% +14.1% 503.25 ± 3% interrupts.CPU49.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
90134 ± 10% -18.3% 73603 ± 6% interrupts.CPU49.CAL:Function_call_interrupts
1785145 +12.4% 2005932 interrupts.CPU49.LOC:Local_timer_interrupts
3185609 ± 11% -17.6% 2626502 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts
102737 ± 10% -17.3% 84932 ± 5% interrupts.CPU49.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496560 ± 4% interrupts.CPU49.TRM:Thermal_event_interrupts
92114 ± 9% -23.2% 70721 ± 6% interrupts.CPU5.CAL:Function_call_interrupts
1785135 +12.3% 2005424 interrupts.CPU5.LOC:Local_timer_interrupts
3210646 ± 12% -24.8% 2415935 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts
107349 ± 9% -18.9% 87034 ± 6% interrupts.CPU5.TLB:TLB_shootdowns
388418 ± 12% +27.8% 496564 ± 4% interrupts.CPU5.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU50.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
92406 ± 10% -20.2% 73771 ± 8% interrupts.CPU50.CAL:Function_call_interrupts
1785200 +12.4% 2006488 interrupts.CPU50.LOC:Local_timer_interrupts
106635 ± 9% -19.9% 85431 ± 6% interrupts.CPU50.TLB:TLB_shootdowns
388419 ± 12% +27.8% 496564 ± 4% interrupts.CPU50.TRM:Thermal_event_interrupts
436.25 +14.4% 499.25 ± 3% interrupts.CPU51.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
91491 ± 9% -20.8% 72473 ± 6% interrupts.CPU51.CAL:Function_call_interrupts
1784683 +12.4% 2005527 interrupts.CPU51.LOC:Local_timer_interrupts
3051494 ± 8% -24.6% 2300032 ± 10% interrupts.CPU51.RES:Rescheduling_interrupts
104954 ± 10% -20.2% 83775 ± 5% interrupts.CPU51.TLB:TLB_shootdowns
388264 ± 12% +27.9% 496462 ± 4% interrupts.CPU51.TRM:Thermal_event_interrupts
92938 ± 11% -19.7% 74594 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
1785451 +12.4% 2006171 interrupts.CPU52.LOC:Local_timer_interrupts
3025681 ± 11% -17.1% 2506800 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts
107100 ± 11% -19.6% 86156 ± 6% interrupts.CPU52.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU52.TRM:Thermal_event_interrupts
439.25 ± 2% +11.7% 490.50 interrupts.CPU53.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
93000 ± 12% -20.9% 73549 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
1785290 +12.3% 2005441 interrupts.CPU53.LOC:Local_timer_interrupts
3212623 ± 7% -18.2% 2629478 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts
107303 ± 11% -17.5% 88567 ± 10% interrupts.CPU53.TLB:TLB_shootdowns
388411 ± 12% +27.8% 496557 ± 4% interrupts.CPU53.TRM:Thermal_event_interrupts
437.75 +12.1% 490.50 interrupts.CPU54.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
1785170 +12.0% 2000090 interrupts.CPU54.LOC:Local_timer_interrupts
5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.NMI:Non-maskable_interrupts
5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.PMI:Performance_monitoring_interrupts
2066106 ± 9% +92.7% 3982033 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts
108150 ± 12% -21.2% 85173 ± 5% interrupts.CPU54.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380146 ± 2% interrupts.CPU54.TRM:Thermal_event_interrupts
436.50 +12.4% 490.50 interrupts.CPU55.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55
91164 ± 12% -19.9% 73014 ± 6% interrupts.CPU55.CAL:Function_call_interrupts
1785472 +12.0% 1999115 interrupts.CPU55.LOC:Local_timer_interrupts
1286654 ± 6% +228.2% 4223034 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts
104943 ± 11% -19.5% 84466 ± 5% interrupts.CPU55.TLB:TLB_shootdowns
329292 ± 3% +15.4% 379918 ± 2% interrupts.CPU55.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU56.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
96934 ± 13% -23.4% 74215 ± 6% interrupts.CPU56.CAL:Function_call_interrupts
1785010 +12.1% 2000169 interrupts.CPU56.LOC:Local_timer_interrupts
1312758 ± 17% +194.5% 3866522 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts
109307 ± 12% -22.1% 85139 ± 4% interrupts.CPU56.TLB:TLB_shootdowns
329273 ± 3% +15.4% 380131 ± 2% interrupts.CPU56.TRM:Thermal_event_interrupts
439.00 ± 2% +11.7% 490.50 interrupts.CPU57.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
94960 ± 11% -22.9% 73246 ± 5% interrupts.CPU57.CAL:Function_call_interrupts
1785297 +12.0% 2000019 interrupts.CPU57.LOC:Local_timer_interrupts
1666475 ± 10% +169.6% 4492937 interrupts.CPU57.RES:Rescheduling_interrupts
107534 ± 10% -22.0% 83871 ± 5% interrupts.CPU57.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380136 ± 2% interrupts.CPU57.TRM:Thermal_event_interrupts
442.25 ± 3% +10.9% 490.50 interrupts.CPU58.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
92818 ± 12% -21.4% 72925 ± 5% interrupts.CPU58.CAL:Function_call_interrupts
1785429 +12.0% 1999528 interrupts.CPU58.LOC:Local_timer_interrupts
1606064 ± 5% +176.8% 4446179 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts
103696 ± 11% -19.5% 83458 ± 4% interrupts.CPU58.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380022 ± 2% interrupts.CPU58.TRM:Thermal_event_interrupts
439.00 +11.7% 490.50 interrupts.CPU59.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
93876 ± 13% -20.5% 74669 ± 6% interrupts.CPU59.CAL:Function_call_interrupts
1785085 +12.0% 1999992 interrupts.CPU59.LOC:Local_timer_interrupts
1634812 ± 5% +174.5% 4487501 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts
106929 ± 13% -21.0% 84522 ± 5% interrupts.CPU59.TLB:TLB_shootdowns
329291 ± 3% +15.4% 380111 ± 2% interrupts.CPU59.TRM:Thermal_event_interrupts
89930 ± 9% -20.5% 71520 ± 7% interrupts.CPU6.CAL:Function_call_interrupts
1785220 +12.3% 2005468 interrupts.CPU6.LOC:Local_timer_interrupts
106307 ± 7% -19.9% 85162 ± 6% interrupts.CPU6.TLB:TLB_shootdowns
388421 ± 12% +27.8% 496564 ± 4% interrupts.CPU6.TRM:Thermal_event_interrupts
436.25 +13.1% 493.25 interrupts.CPU60.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
98595 ± 11% -25.6% 73379 ± 5% interrupts.CPU60.CAL:Function_call_interrupts
1785116 +12.1% 2000324 interrupts.CPU60.LOC:Local_timer_interrupts
1604314 ± 8% +186.8% 4601614 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts
110442 ± 10% -24.0% 83932 ± 5% interrupts.CPU60.TLB:TLB_shootdowns
329256 ± 3% +15.5% 380140 ± 2% interrupts.CPU60.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU61.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
93035 ± 11% -21.4% 73124 ± 5% interrupts.CPU61.CAL:Function_call_interrupts
1785176 +12.1% 2000960 interrupts.CPU61.LOC:Local_timer_interrupts
4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.NMI:Non-maskable_interrupts
4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.PMI:Performance_monitoring_interrupts
1575186 ± 6% +194.9% 4644760 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts
103665 ± 11% -18.7% 84259 ± 5% interrupts.CPU61.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380127 ± 2% interrupts.CPU61.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU62.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
95188 ± 10% -21.9% 74376 ± 6% interrupts.CPU62.CAL:Function_call_interrupts
1785028 +12.0% 1999262 interrupts.CPU62.LOC:Local_timer_interrupts
1613834 ± 8% +184.7% 4594499 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts
106856 ± 10% -21.4% 83987 ± 5% interrupts.CPU62.TLB:TLB_shootdowns
329270 ± 3% +15.4% 380120 ± 2% interrupts.CPU62.TRM:Thermal_event_interrupts
94411 ± 11% -21.3% 74269 ± 5% interrupts.CPU63.CAL:Function_call_interrupts
1785322 +12.0% 1999237 interrupts.CPU63.LOC:Local_timer_interrupts
1669218 ± 6% +187.2% 4794684 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts
105402 ± 10% -18.3% 86076 ± 8% interrupts.CPU63.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU63.TRM:Thermal_event_interrupts
96060 ± 11% -22.6% 74364 ± 6% interrupts.CPU64.CAL:Function_call_interrupts
1785304 +12.0% 1999571 interrupts.CPU64.LOC:Local_timer_interrupts
1681501 ± 7% +164.0% 4438647 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts
108534 ± 10% -22.8% 83813 ± 5% interrupts.CPU64.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU64.TRM:Thermal_event_interrupts
94178 ± 9% -22.7% 72758 ± 8% interrupts.CPU65.CAL:Function_call_interrupts
1785338 +12.0% 1999408 interrupts.CPU65.LOC:Local_timer_interrupts
1674108 ± 4% +182.2% 4724262 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts
107519 ± 10% -22.6% 83259 ± 5% interrupts.CPU65.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380128 ± 2% interrupts.CPU65.TRM:Thermal_event_interrupts
95135 ± 8% -22.4% 73795 ± 5% interrupts.CPU66.CAL:Function_call_interrupts
1785413 +12.0% 2000352 interrupts.CPU66.LOC:Local_timer_interrupts
616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.NMI:Non-maskable_interrupts
616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts
1646166 ± 6% +169.3% 4433267 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts
107190 ± 7% -22.5% 83034 ± 4% interrupts.CPU66.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380118 ± 2% interrupts.CPU66.TRM:Thermal_event_interrupts
95627 ± 10% -22.4% 74249 ± 5% interrupts.CPU67.CAL:Function_call_interrupts
1785314 +12.1% 2001128 interrupts.CPU67.LOC:Local_timer_interrupts
1611146 ± 4% +183.8% 4572204 ± 7% interrupts.CPU67.RES:Rescheduling_interrupts
109283 ± 11% -22.5% 84676 ± 4% interrupts.CPU67.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU67.TRM:Thermal_event_interrupts
95479 ± 11% -20.6% 75791 ± 5% interrupts.CPU68.CAL:Function_call_interrupts
1785192 +12.0% 1999411 interrupts.CPU68.LOC:Local_timer_interrupts
1575394 ± 7% +189.9% 4567277 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts
106972 ± 12% -18.6% 87039 ± 5% interrupts.CPU68.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380129 ± 2% interrupts.CPU68.TRM:Thermal_event_interrupts
93067 ± 14% -19.9% 74538 ± 4% interrupts.CPU69.CAL:Function_call_interrupts
1785257 +12.1% 2000636 interrupts.CPU69.LOC:Local_timer_interrupts
1578663 ± 9% +192.9% 4624518 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts
104271 ± 12% -18.5% 84989 ± 2% interrupts.CPU69.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380066 ± 2% interrupts.CPU69.TRM:Thermal_event_interrupts
474.00 ± 3% +49.7% 709.75 ± 24% interrupts.CPU7.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
92062 ± 9% -19.7% 73884 ± 5% interrupts.CPU7.CAL:Function_call_interrupts
1784266 +12.4% 2004980 interrupts.CPU7.LOC:Local_timer_interrupts
2505703 ± 11% -19.0% 2029187 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts
107698 ± 8% -18.4% 87903 ± 6% interrupts.CPU7.TLB:TLB_shootdowns
388281 ± 12% +27.8% 496347 ± 4% interrupts.CPU7.TRM:Thermal_event_interrupts
92337 ± 12% -17.5% 76148 ± 2% interrupts.CPU70.CAL:Function_call_interrupts
1785161 +12.0% 1999757 interrupts.CPU70.LOC:Local_timer_interrupts
1562360 ± 4% +170.8% 4231208 ± 9% interrupts.CPU70.RES:Rescheduling_interrupts
103862 ± 10% -18.8% 84371 ± 3% interrupts.CPU70.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU70.TRM:Thermal_event_interrupts
95829 ± 12% -21.2% 75509 ± 5% interrupts.CPU71.CAL:Function_call_interrupts
1784894 +12.1% 2001287 interrupts.CPU71.LOC:Local_timer_interrupts
1533567 ± 7% +196.9% 4553440 interrupts.CPU71.RES:Rescheduling_interrupts
105883 ± 11% -20.4% 84248 ± 6% interrupts.CPU71.TLB:TLB_shootdowns
329247 ± 3% +15.5% 380146 ± 2% interrupts.CPU71.TRM:Thermal_event_interrupts
88372 ± 11% -20.3% 70389 ± 7% interrupts.CPU8.CAL:Function_call_interrupts
1784452 +12.4% 2005305 interrupts.CPU8.LOC:Local_timer_interrupts
103992 ± 10% -17.4% 85887 ± 5% interrupts.CPU8.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496557 ± 4% interrupts.CPU8.TRM:Thermal_event_interrupts
89290 ± 10% -19.1% 72221 ± 6% interrupts.CPU9.CAL:Function_call_interrupts
1784716 +12.4% 2005710 interrupts.CPU9.LOC:Local_timer_interrupts
104340 ± 10% -16.0% 87647 ± 3% interrupts.CPU9.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU9.TRM:Thermal_event_interrupts
1.285e+08 +12.2% 1.442e+08 interrupts.LOC:Local_timer_interrupts
144.00 +50.0% 216.00 interrupts.MCP:Machine_check_polls
1.677e+08 ± 8% +50.4% 2.522e+08 ± 2% interrupts.RES:Rescheduling_interrupts
7624145 ± 10% -19.7% 6125835 ± 5% interrupts.TLB:TLB_shootdowns
25836833 ± 8% +22.2% 31559779 ± 2% interrupts.TRM:Thermal_event_interrupts
***************************************************************************************************
lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-11-14.cgz/lkp-bdw-ep6/hackbench/0xb000038
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
134091 -10.6% 119937 hackbench.throughput
3.352e+09 ± 2% +16.1% 3.892e+09 ± 5% hackbench.time.involuntary_context_switches
535571 ± 2% +10.6% 592420 ± 4% numa-meminfo.node0.FilePages
12614 ± 18% -12.2% 11078 ± 18% numa-meminfo.node1.Mapped
3472 ± 6% +18.0% 4096 slabinfo.skbuff_head_cache.active_objs
3568 ± 6% +15.7% 4128 slabinfo.skbuff_head_cache.num_objs
730128 ± 5% -52.2% 349073 ± 25% turbostat.C3
14.77 +5.9% 15.64 turbostat.RAMWatt
24613783 ± 5% -32.6% 16595193 ± 23% numa-numastat.node0.local_node
24632941 ± 5% -32.6% 16605181 ± 23% numa-numastat.node0.numa_hit
25827961 -29.9% 18097150 ± 19% numa-numastat.node1.local_node
25836948 -29.9% 18115676 ± 19% numa-numastat.node1.numa_hit
50467170 ± 2% -31.2% 34733456 ± 10% proc-vmstat.numa_hit
50439016 ± 2% -31.2% 34704933 ± 10% proc-vmstat.numa_local
50622650 ± 2% -31.1% 34896854 ± 10% proc-vmstat.pgalloc_normal
50548381 ± 2% -31.1% 34834391 ± 10% proc-vmstat.pgfree
79.00 +2.2% 80.75 vmstat.cpu.sy
19.00 -7.9% 17.50 ± 2% vmstat.cpu.us
13450755 +4.6% 14066265 vmstat.system.cs
1122772 -12.1% 986681 vmstat.system.in
133891 ± 2% +10.6% 148112 ± 4% numa-vmstat.node0.nr_file_pages
12301995 ± 4% -30.2% 8590153 ± 22% numa-vmstat.node0.numa_hit
12282910 ± 4% -30.1% 8580139 ± 22% numa-vmstat.node0.numa_local
12964354 ± 5% -31.2% 8914408 ± 18% numa-vmstat.node1.numa_hit
12806275 ± 5% -31.7% 8746928 ± 19% numa-vmstat.node1.numa_local
33816695 ± 10% +17.8% 39824104 ± 7% cpuidle.C1.time
21015951 ± 3% +19.6% 25127666 ± 7% cpuidle.C1E.time
730991 ± 5% -52.1% 350128 ± 25% cpuidle.C3.usage
1.497e+08 ± 28% +41.6% 2.119e+08 ± 15% cpuidle.C6.time
6771631 ± 4% -45.2% 3713804 ± 15% cpuidle.POLL.time
5209842 ± 4% -44.1% 2913245 ± 15% cpuidle.POLL.usage
26965506 +5.5% 28444381 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg
28257967 +6.6% 30126596 ± 4% sched_debug.cfs_rq:/.min_vruntime.max
582203 ± 9% +37.0% 797353 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev
4.65 ± 8% -10.7% 4.16 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev
4620 ± 6% -10.7% 4125 ± 4% sched_debug.cfs_rq:/.runnable_weight.stddev
582831 ± 9% +36.6% 796226 ± 7% sched_debug.cfs_rq:/.spread0.stddev
154.73 ± 2% -14.7% 132.00 ± 3% sched_debug.cfs_rq:/.util_avg.stddev
120.93 ± 25% +43.5% 173.50 ± 19% sched_debug.cfs_rq:/.util_est_enqueued.min
184.93 ± 2% -11.0% 164.54 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev
182135 ± 18% +57.1% 286098 ± 6% sched_debug.cpu.avg_idle.avg
185926 ± 19% +49.0% 276991 ± 9% sched_debug.cpu.avg_idle.stddev
3.32 ± 4% -10.8% 2.96 ± 3% sched_debug.cpu.nr_running.stddev
46710201 +9.0% 50934866 ± 5% sched_debug.cpu.nr_switches.avg
50053735 +16.9% 58491253 ± 7% sched_debug.cpu.nr_switches.max
43751407 +6.7% 46698664 ± 4% sched_debug.cpu.nr_switches.min
1344975 ± 10% +78.9% 2406504 ± 16% sched_debug.cpu.nr_switches.stddev
1.63 ± 3% -32.8% 1.09 ± 11% sched_debug.cpu.nr_uninterruptible.avg
1017 ± 27% -59.5% 411.75 ± 16% interrupts.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6
6547829 -14.4% 5603910 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts
1017 ± 27% -59.5% 411.75 ± 16% interrupts.CPU19.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6
6801598 -14.7% 5801847 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts
6609756 -13.7% 5703084 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts
6446878 ± 2% -11.8% 5689145 ± 7% interrupts.CPU21.RES:Rescheduling_interrupts
6634825 -9.4% 6008203 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts
6801182 -9.8% 6137528 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts
6687252 -12.9% 5821967 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts
6583387 -10.9% 5868659 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts
6585631 -16.6% 5490726 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts
6759973 -13.2% 5867800 ± 7% interrupts.CPU57.RES:Rescheduling_interrupts
6459616 ± 2% -10.5% 5783047 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts
6580818 ± 2% -16.9% 5468945 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts
6723178 ± 2% -12.4% 5887778 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts
6480129 -16.2% 5427168 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts
6439558 ± 2% -11.6% 5691160 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts
6662822 -12.0% 5866390 ± 7% interrupts.CPU68.RES:Rescheduling_interrupts
6602178 -12.0% 5809760 ± 7% interrupts.CPU7.RES:Rescheduling_interrupts
6616172 -14.1% 5685889 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts
6591477 -13.8% 5682655 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts
13.97 +15.8% 16.18 perf-stat.i.MPKI
1.70 -0.1 1.65 perf-stat.i.branch-miss-rate%
3.607e+08 -1.6% 3.549e+08 perf-stat.i.branch-misses
1.22 ± 6% +0.8 2.01 ± 18% perf-stat.i.cache-miss-rate%
17650711 ± 6% +98.0% 34953649 ± 18% perf-stat.i.cache-misses
1.496e+09 +17.3% 1.755e+09 perf-stat.i.cache-references
13503062 +4.6% 14122231 perf-stat.i.context-switches
2946654 ± 2% +18.5% 3491269 ± 3% perf-stat.i.cpu-migrations
29365 ± 8% -30.6% 20387 ± 31% perf-stat.i.cycles-between-cache-misses
0.35 ± 2% +0.0 0.36 ± 2% perf-stat.i.dTLB-store-miss-rate%
2.048e+10 -2.1% 2.006e+10 perf-stat.i.dTLB-stores
55.17 -1.4 53.74 perf-stat.i.iTLB-load-miss-rate%
1.019e+08 +5.5% 1.074e+08 perf-stat.i.iTLB-loads
3068 -3.0% 2975 perf-stat.i.minor-faults
9195204 ± 6% +113.0% 19589023 ± 18% perf-stat.i.node-load-misses
28132 ± 6% +125.3% 63396 ± 13% perf-stat.i.node-loads
58.92 -9.8 49.16 ± 4% perf-stat.i.node-store-miss-rate%
3588196 ± 6% +60.5% 5760265 ± 19% perf-stat.i.node-store-misses
2415953 ± 5% +145.2% 5924042 ± 18% perf-stat.i.node-stores
3068 -3.0% 2975 perf-stat.i.page-faults
13.96 +16.0% 16.19 perf-stat.overall.MPKI
1.70 -0.1 1.65 perf-stat.overall.branch-miss-rate%
1.18 ± 6% +0.8 1.99 ± 18% perf-stat.overall.cache-miss-rate%
13823 ± 6% -47.8% 7221 ± 20% perf-stat.overall.cycles-between-cache-misses
0.35 ± 2% +0.0 0.36 ± 2% perf-stat.overall.dTLB-store-miss-rate%
55.11 -1.4 53.70 perf-stat.overall.iTLB-load-miss-rate%
59.75 -10.5 49.23 perf-stat.overall.node-store-miss-rate%
78192 +13.0% 88370 perf-stat.overall.path-length
3.601e+08 -1.6% 3.543e+08 perf-stat.ps.branch-misses
17622620 ± 6% +98.0% 34899005 ± 18% perf-stat.ps.cache-misses
1.494e+09 +17.3% 1.752e+09 perf-stat.ps.cache-references
13480657 +4.6% 14099389 perf-stat.ps.context-switches
2941749 ± 2% +18.5% 3485631 ± 3% perf-stat.ps.cpu-migrations
2.045e+10 -2.1% 2.003e+10 perf-stat.ps.dTLB-stores
1.017e+08 +5.5% 1.072e+08 perf-stat.ps.iTLB-loads
3064 -3.0% 2972 perf-stat.ps.minor-faults
9180406 ± 6% +113.0% 19558101 ± 18% perf-stat.ps.node-load-misses
28101 ± 6% +125.3% 63314 ± 13% perf-stat.ps.node-loads
3582467 ± 6% +60.5% 5751138 ± 19% perf-stat.ps.node-store-misses
2412086 ± 5% +145.2% 5914636 ± 18% perf-stat.ps.node-stores
3064 -3.0% 2972 perf-stat.ps.page-faults
223485 +9.1% 243918 ± 5% softirqs.CPU0.TIMER
11887 ± 11% -18.2% 9723 ± 6% softirqs.CPU1.SCHED
215411 +9.2% 235267 ± 6% softirqs.CPU1.TIMER
11293 ± 3% -14.3% 9678 ± 10% softirqs.CPU10.SCHED
213916 +9.8% 234908 ± 5% softirqs.CPU10.TIMER
212758 +7.7% 229141 ± 5% softirqs.CPU11.TIMER
11098 ± 2% -15.9% 9337 ± 10% softirqs.CPU12.SCHED
213397 +9.0% 232706 ± 6% softirqs.CPU12.TIMER
212419 +8.3% 230115 ± 6% softirqs.CPU13.TIMER
10777 ± 4% -12.5% 9426 ± 6% softirqs.CPU14.SCHED
11241 ± 3% -11.5% 9950 ± 6% softirqs.CPU15.SCHED
214604 +10.0% 236116 ± 8% softirqs.CPU15.TIMER
11147 -14.9% 9482 ± 8% softirqs.CPU16.SCHED
216343 +8.4% 234558 ± 6% softirqs.CPU16.TIMER
214680 +7.0% 229748 ± 6% softirqs.CPU17.TIMER
212791 +8.6% 230988 ± 6% softirqs.CPU19.TIMER
11211 ± 4% -10.4% 10047 ± 6% softirqs.CPU2.SCHED
219918 +9.8% 241513 ± 6% softirqs.CPU2.TIMER
10825 ± 4% -14.3% 9273 ± 7% softirqs.CPU20.SCHED
215209 +8.8% 234149 ± 6% softirqs.CPU20.TIMER
10869 ± 4% -11.4% 9626 ± 4% softirqs.CPU21.SCHED
216225 +8.7% 235105 ± 5% softirqs.CPU21.TIMER
13013 ± 7% -14.0% 11192 ± 3% softirqs.CPU22.SCHED
220406 +8.9% 239975 ± 4% softirqs.CPU22.TIMER
11925 -14.9% 10145 ± 3% softirqs.CPU23.SCHED
220231 +7.6% 237056 ± 5% softirqs.CPU23.TIMER
11140 ± 2% -11.5% 9861 ± 7% softirqs.CPU24.SCHED
216091 +8.9% 235338 ± 5% softirqs.CPU24.TIMER
11438 ± 3% -16.3% 9578 ± 2% softirqs.CPU25.SCHED
214880 +9.4% 235143 ± 5% softirqs.CPU25.TIMER
214888 +8.8% 233874 ± 5% softirqs.CPU26.TIMER
11410 ± 2% -13.7% 9844 ± 10% softirqs.CPU27.SCHED
215012 +8.7% 233790 ± 4% softirqs.CPU27.TIMER
220618 ± 2% +7.8% 237788 ± 5% softirqs.CPU28.TIMER
11096 ± 5% -14.3% 9513 ± 6% softirqs.CPU29.SCHED
217194 +8.2% 235108 ± 5% softirqs.CPU29.TIMER
11445 ± 2% -16.7% 9536 ± 10% softirqs.CPU3.SCHED
214692 +9.1% 234221 ± 5% softirqs.CPU3.TIMER
11127 ± 5% -10.5% 9954 ± 6% softirqs.CPU30.SCHED
214861 +8.2% 232532 ± 5% softirqs.CPU30.TIMER
217582 +7.8% 234493 ± 4% softirqs.CPU31.TIMER
214606 +8.8% 233473 ± 4% softirqs.CPU32.TIMER
11299 ± 2% -11.7% 9975 ± 6% softirqs.CPU33.SCHED
11503 ± 2% -16.0% 9658 ± 10% softirqs.CPU34.SCHED
213233 +10.9% 236580 ± 6% softirqs.CPU34.TIMER
11162 ± 2% -11.6% 9863 ± 7% softirqs.CPU35.SCHED
11241 -13.9% 9678 ± 6% softirqs.CPU36.SCHED
215681 +9.2% 235446 ± 4% softirqs.CPU36.TIMER
11358 -15.2% 9632 ± 6% softirqs.CPU37.SCHED
219691 ± 2% +18.5% 260407 ± 13% softirqs.CPU37.TIMER
11570 ± 2% -13.1% 10049 ± 6% softirqs.CPU38.SCHED
215559 +8.2% 233189 ± 4% softirqs.CPU38.TIMER
11295 ± 4% -13.1% 9819 ± 10% softirqs.CPU39.SCHED
215781 ± 2% +7.6% 232108 ± 5% softirqs.CPU39.TIMER
11506 ± 7% -13.0% 10006 ± 6% softirqs.CPU4.SCHED
214973 +8.9% 234140 ± 6% softirqs.CPU4.TIMER
11304 ± 2% -13.7% 9759 ± 10% softirqs.CPU40.SCHED
213827 +10.1% 235341 ± 4% softirqs.CPU40.TIMER
11462 ± 4% -13.5% 9913 ± 8% softirqs.CPU41.SCHED
215879 ± 2% +7.2% 231499 ± 5% softirqs.CPU41.TIMER
11418 ± 3% -14.0% 9819 ± 8% softirqs.CPU43.SCHED
214864 +8.3% 232698 ± 4% softirqs.CPU43.TIMER
10706 ± 5% -13.9% 9223 ± 11% softirqs.CPU44.SCHED
217386 +9.4% 237916 ± 5% softirqs.CPU44.TIMER
215191 +9.7% 236017 ± 5% softirqs.CPU45.TIMER
214214 +9.9% 235466 ± 6% softirqs.CPU46.TIMER
214471 +9.9% 235675 ± 5% softirqs.CPU47.TIMER
11251 ± 2% -13.0% 9785 ± 8% softirqs.CPU48.SCHED
212500 +9.9% 233451 ± 6% softirqs.CPU48.TIMER
213609 +9.9% 234657 ± 5% softirqs.CPU49.TIMER
216836 +8.5% 235225 ± 5% softirqs.CPU5.TIMER
215854 +9.0% 235383 ± 5% softirqs.CPU50.TIMER
11468 -14.4% 9820 ± 9% softirqs.CPU51.SCHED
215402 +8.8% 234360 ± 5% softirqs.CPU51.TIMER
214152 +8.8% 233038 ± 5% softirqs.CPU52.TIMER
212843 +8.4% 230716 ± 5% softirqs.CPU53.TIMER
11373 ± 2% -14.9% 9679 ± 7% softirqs.CPU54.SCHED
214008 +9.5% 234380 ± 5% softirqs.CPU54.TIMER
212619 +7.5% 228539 ± 5% softirqs.CPU55.TIMER
11136 ± 3% -16.4% 9312 ± 10% softirqs.CPU56.SCHED
213404 +9.0% 232542 ± 6% softirqs.CPU56.TIMER
11171 ± 4% -11.9% 9837 ± 7% softirqs.CPU57.SCHED
212283 +8.6% 230474 ± 6% softirqs.CPU57.TIMER
11137 ± 4% -14.0% 9578 ± 6% softirqs.CPU58.SCHED
11226 ± 4% -17.2% 9289 ± 4% softirqs.CPU59.SCHED
214193 +23.1% 263668 ± 22% softirqs.CPU59.TIMER
217195 +8.2% 234975 ± 5% softirqs.CPU6.TIMER
11051 ± 4% -14.4% 9460 ± 9% softirqs.CPU60.SCHED
217131 +7.9% 234363 ± 5% softirqs.CPU60.TIMER
11206 ± 3% -12.6% 9793 ± 9% softirqs.CPU61.SCHED
211791 +9.2% 231302 ± 5% softirqs.CPU61.TIMER
214956 ± 2% +8.8% 233770 ± 5% softirqs.CPU62.TIMER
11180 ± 3% -14.8% 9526 ± 8% softirqs.CPU63.SCHED
213413 +8.7% 232008 ± 5% softirqs.CPU63.TIMER
215674 +8.1% 233086 ± 6% softirqs.CPU64.TIMER
11032 ± 3% -15.7% 9302 ± 7% softirqs.CPU65.SCHED
215870 +9.0% 235386 ± 5% softirqs.CPU65.TIMER
11403 ± 3% -14.2% 9786 ± 6% softirqs.CPU66.SCHED
215968 +9.4% 236175 ± 5% softirqs.CPU66.TIMER
11098 ± 3% -7.4% 10281 ± 4% softirqs.CPU67.SCHED
217846 +8.3% 236023 ± 4% softirqs.CPU67.TIMER
11094 -12.2% 9740 ± 7% softirqs.CPU68.SCHED
215512 +9.4% 235796 ± 5% softirqs.CPU68.TIMER
11321 ± 3% -14.1% 9724 ± 3% softirqs.CPU69.SCHED
213564 +9.0% 232867 ± 5% softirqs.CPU69.TIMER
218822 +8.8% 238080 ± 6% softirqs.CPU7.TIMER
11507 ± 2% -12.9% 10026 ± 4% softirqs.CPU70.SCHED
213928 +9.4% 233986 ± 4% softirqs.CPU70.TIMER
213528 +8.5% 231730 ± 4% softirqs.CPU71.TIMER
11262 ± 2% -11.1% 10013 ± 5% softirqs.CPU72.SCHED
11145 ± 3% -14.5% 9533 ± 8% softirqs.CPU73.SCHED
216467 +9.2% 236409 ± 5% softirqs.CPU73.TIMER
11447 ± 4% -13.2% 9935 ± 4% softirqs.CPU74.SCHED
212578 +9.1% 232022 ± 5% softirqs.CPU74.TIMER
215256 +7.8% 232072 ± 5% softirqs.CPU75.TIMER
11324 ± 4% -11.8% 9985 ± 3% softirqs.CPU76.SCHED
214170 +8.6% 232516 ± 5% softirqs.CPU76.TIMER
11646 ± 5% -15.7% 9824 ± 7% softirqs.CPU77.SCHED
213706 +8.0% 230880 ± 5% softirqs.CPU77.TIMER
11397 ± 6% -13.9% 9810 ± 10% softirqs.CPU78.SCHED
11179 ± 3% -11.5% 9894 ± 5% softirqs.CPU79.SCHED
213215 +8.4% 231081 ± 5% softirqs.CPU79.TIMER
11165 ± 4% -16.2% 9352 ± 9% softirqs.CPU8.SCHED
216508 ± 2% +7.9% 233692 ± 6% softirqs.CPU8.TIMER
11203 ± 2% -13.9% 9643 ± 4% softirqs.CPU80.SCHED
216446 +8.3% 234469 ± 5% softirqs.CPU80.TIMER
11151 ± 2% -12.8% 9726 ± 6% softirqs.CPU81.SCHED
11373 ± 3% -13.0% 9891 ± 6% softirqs.CPU82.SCHED
213597 +8.9% 232608 ± 5% softirqs.CPU82.TIMER
214308 +7.9% 231148 ± 5% softirqs.CPU83.TIMER
214384 +20.9% 259149 ± 15% softirqs.CPU84.TIMER
11458 -13.7% 9887 ± 7% softirqs.CPU86.SCHED
215459 +8.2% 233058 ± 5% softirqs.CPU86.TIMER
11303 ± 4% -12.4% 9904 ± 8% softirqs.CPU87.SCHED
214185 +7.9% 231116 ± 5% softirqs.CPU87.TIMER
11030 ± 2% -13.0% 9593 ± 8% softirqs.CPU9.SCHED
213229 +8.7% 231788 ± 6% softirqs.CPU9.TIMER
995635 -12.9% 867688 ± 5% softirqs.SCHED
19052052 +8.6% 20690944 ± 5% softirqs.TIMER
31.32 -1.6 29.74 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.24 -1.5 28.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.06 -0.9 25.17 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.35 ± 5% -0.9 1.48 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
25.36 -0.8 24.54 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
2.75 ± 4% -0.7 2.08 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
3.84 ± 2% -0.6 3.23 ± 2% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
7.83 -0.5 7.30 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
1.68 ± 6% -0.5 1.17 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
8.02 -0.5 7.53 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
8.08 -0.5 7.60 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.63 ± 4% -0.4 1.19 ± 6% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
1.71 ± 6% -0.4 1.28 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
2.60 -0.4 2.19 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.03 ± 2% -0.4 1.64 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.47 -0.4 3.08 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.95 ± 2% -0.3 1.61 ± 3% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
2.67 -0.3 2.35 ± 3% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
0.56 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.22 ± 4% -0.3 0.94 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
1.25 ± 2% -0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
2.64 -0.2 2.41 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.53 -0.2 1.34 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.59 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.34 -0.2 1.16 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.07 ± 2% -0.2 0.90 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
1.05 -0.2 0.88 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.56 ± 2% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.17 ± 4% -0.2 1.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
0.98 ± 2% -0.2 0.82 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.66 ± 2% -0.2 1.50 ± 5% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
0.73 ± 3% -0.2 0.57 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.96 ± 2% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.89 ± 2% -0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.66 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
2.42 -0.1 2.32 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read
0.89 ± 4% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
0.74 ± 2% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.84 -0.1 0.79 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.56 -0.0 0.54 ± 2% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.77 ± 2% +0.1 0.82 perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.90 ± 3% +0.1 0.97 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr
0.69 ± 3% +0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.80 ± 4% +0.1 0.89 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
1.10 +0.1 1.20 perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.73 ± 3% +0.1 0.83 ± 2% perf-profile.calltrace.cycles-pp.__switch_to
1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.63 ± 4% +0.1 0.75 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.99 ± 5% +0.1 1.12 ± 5% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.07 ± 3% +0.1 1.21 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
0.64 ± 6% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
1.41 ± 3% +0.2 1.59 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.77 ± 5% +0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop
1.73 ± 3% +0.3 2.06 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.14 ±173% +0.5 0.63 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read
0.13 ±173% +0.5 0.64 ± 5% perf-profile.calltrace.cycles-pp.migrate_task_rq_fair.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.85 ± 5% +0.5 2.38 ± 3% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.00 +0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.15 ±173% +1.0 1.14 ± 12% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
87.66 +1.1 88.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
86.87 +1.1 88.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.59 ± 4% +1.4 6.96 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.72 ± 4% +1.4 7.12 ± 3% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.87 ± 4% +1.4 7.28 ± 3% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.70 ± 30% +3.3 4.98 ± 13% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
35.54 +3.3 38.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
34.28 +3.5 37.82 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.12 +4.1 35.22 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.49 +4.1 34.63 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
22.54 ± 2% +6.0 28.53 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
20.58 ± 2% +6.2 26.76 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
19.97 ± 2% +6.2 26.16 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
19.73 ± 3% +6.2 25.92 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
3.39 ± 20% +7.0 10.39 ± 12% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.89 ± 15% +7.1 11.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
31.34 -1.6 29.75 perf-profile.children.cycles-pp.ksys_read
30.28 -1.5 28.79 perf-profile.children.cycles-pp.vfs_read
4.14 ± 5% -1.3 2.85 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
3.51 ± 6% -1.0 2.54 ± 8% perf-profile.children.cycles-pp.update_cfs_group
26.10 -0.9 25.20 perf-profile.children.cycles-pp.new_sync_read
25.42 -0.8 24.59 perf-profile.children.cycles-pp.pipe_read
4.67 -0.8 3.86 ± 4% perf-profile.children.cycles-pp.security_file_permission
4.33 ± 2% -0.7 3.65 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64
2.18 ± 4% -0.5 1.64 ± 6% perf-profile.children.cycles-pp.mutex_unlock
7.92 -0.5 7.40 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
8.10 -0.5 7.61 ± 3% perf-profile.children.cycles-pp.activate_task
8.16 -0.5 7.67 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
4.12 ± 4% -0.5 3.64 ± 5% perf-profile.children.cycles-pp._raw_spin_lock
2.83 -0.4 2.38 ± 6% perf-profile.children.cycles-pp.selinux_file_permission
3.53 -0.4 3.13 ± 4% perf-profile.children.cycles-pp.enqueue_entity
1.99 ± 2% -0.4 1.64 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter
3.34 -0.4 2.99 ± 4% perf-profile.children.cycles-pp.update_load_avg
2.70 -0.3 2.39 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter
2.66 -0.3 2.35 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.93 ± 3% -0.3 0.64 ± 7% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
1.41 ± 4% -0.3 1.13 perf-profile.children.cycles-pp.file_has_perm
2.75 -0.3 2.49 ± 2% perf-profile.children.cycles-pp.dequeue_entity
2.29 ± 2% -0.3 2.04 perf-profile.children.cycles-pp.mutex_lock
0.85 ± 3% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate
0.59 ± 3% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.__mutex_lock
1.79 ± 2% -0.2 1.59 ± 4% perf-profile.children.cycles-pp.__fdget_pos
1.70 ± 2% -0.2 1.52 ± 3% perf-profile.children.cycles-pp.__fget_light
0.93 ± 3% -0.2 0.75 perf-profile.children.cycles-pp.avc_has_perm
1.09 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.copyin
2.33 ± 2% -0.2 2.16 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.96 ± 2% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.___might_sleep
1.67 ± 2% -0.2 1.52 ± 5% perf-profile.children.cycles-pp.copyout
0.83 ± 4% -0.1 0.68 ± 2% perf-profile.children.cycles-pp._cond_resched
0.33 ± 5% -0.1 0.19 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common
0.42 ± 2% -0.1 0.28 ± 8% perf-profile.children.cycles-pp.wake_up_q
0.70 ± 3% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.__might_sleep
0.65 ± 5% -0.1 0.54 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent
0.52 ± 2% -0.1 0.43 perf-profile.children.cycles-pp.__might_fault
0.53 ± 5% -0.1 0.45 perf-profile.children.cycles-pp.current_time
0.84 ± 3% -0.1 0.77 perf-profile.children.cycles-pp.fsnotify
0.16 ± 7% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.15 ± 8% -0.1 0.08 ± 15% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.22 ± 3% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.wake_q_add
0.49 -0.1 0.44 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.26 ± 4% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs
0.08 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_read
0.25 ± 4% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.finish_wait
0.38 -0.0 0.34 ± 2% perf-profile.children.cycles-pp.__x64_sys_read
0.43 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.prepare_to_wait
0.14 ± 20% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_tick_fair
0.16 ± 19% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.scheduler_tick
0.10 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.18 ± 8% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.inode_has_perm
0.21 ± 2% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.deactivate_task
0.20 ± 2% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.13 ± 3% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__sb_end_write
0.14 ± 9% -0.0 0.12 perf-profile.children.cycles-pp.timespec64_trunc
0.14 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret
0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.17 ± 4% -0.0 0.15 perf-profile.children.cycles-pp.iov_iter_init
0.11 ± 4% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.15 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.rb_insert_color
0.18 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy
0.08 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.interrupt_entry
0.23 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.cpumask_next
0.67 +0.0 0.69 perf-profile.children.cycles-pp.__calc_delta
0.15 ± 3% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.17 ± 6% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.remove_entity_load_avg
0.14 ± 6% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg
0.80 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.check_preempt_wakeup
0.37 ± 3% +0.1 0.45 ± 4% perf-profile.children.cycles-pp.account_entity_enqueue
0.79 ± 4% +0.1 0.88 ± 5% perf-profile.children.cycles-pp.finish_task_switch
0.80 ± 2% +0.1 0.89 ± 2% perf-profile.children.cycles-pp.put_prev_entity
0.54 ± 2% +0.1 0.65 perf-profile.children.cycles-pp.account_entity_dequeue
1.01 ± 5% +0.1 1.13 ± 5% perf-profile.children.cycles-pp.set_task_cpu
0.49 ± 7% +0.2 0.65 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair
1.23 ± 4% +0.2 1.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm
1.15 ± 2% +0.2 1.33 ± 2% perf-profile.children.cycles-pp.__switch_to
1.42 ± 3% +0.2 1.59 ± 2% perf-profile.children.cycles-pp.switch_fpu_return
0.76 ± 3% +0.2 0.96 ± 3% perf-profile.children.cycles-pp.pick_next_entity
2.21 +0.2 2.42 perf-profile.children.cycles-pp.reweight_entity
2.20 ± 2% +0.2 2.43 perf-profile.children.cycles-pp.load_new_mm_cr3
1.24 ± 5% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.update_rq_clock
1.43 ± 5% +0.3 1.73 ± 2% perf-profile.children.cycles-pp.___perf_sw_event
0.38 ± 15% +0.3 0.68 ± 9% perf-profile.children.cycles-pp.find_next_bit
3.96 +0.3 4.28 perf-profile.children.cycles-pp.pick_next_task_fair
4.43 ± 2% +0.4 4.82 perf-profile.children.cycles-pp.switch_mm_irqs_off
1.11 +0.7 1.83 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.46 ± 26% +0.7 1.20 ± 12% perf-profile.children.cycles-pp.cpumask_next_wrap
87.76 +1.1 88.86 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
86.98 +1.1 88.10 perf-profile.children.cycles-pp.do_syscall_64
21.71 +1.3 23.02 perf-profile.children.cycles-pp.__schedule
5.98 ± 4% +1.4 7.34 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
21.59 +1.4 22.99 perf-profile.children.cycles-pp.schedule
1.73 ± 29% +3.3 5.03 ± 13% perf-profile.children.cycles-pp.available_idle_cpu
35.55 +3.3 38.88 perf-profile.children.cycles-pp.ksys_write
34.31 +3.5 37.84 perf-profile.children.cycles-pp.vfs_write
31.14 +4.1 35.23 ± 2% perf-profile.children.cycles-pp.new_sync_write
30.56 +4.2 34.74 ± 2% perf-profile.children.cycles-pp.pipe_write
22.96 ± 2% +6.0 28.93 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock
20.11 ± 2% +6.1 26.19 ± 3% perf-profile.children.cycles-pp.try_to_wake_up
20.66 ± 2% +6.2 26.82 ± 3% perf-profile.children.cycles-pp.__wake_up_common
20.01 ± 2% +6.2 26.19 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function
3.46 ± 20% +7.0 10.48 ± 12% perf-profile.children.cycles-pp.select_idle_sibling
4.92 ± 15% +7.1 12.01 ± 10% perf-profile.children.cycles-pp.select_task_rq_fair
11.85 ± 3% -2.1 9.72 ± 2% perf-profile.self.cycles-pp.do_syscall_64
4.13 ± 5% -1.3 2.85 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
3.49 ± 6% -1.0 2.52 ± 9% perf-profile.self.cycles-pp.update_cfs_group
4.33 ± 2% -0.7 3.64 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64
2.12 ± 4% -0.5 1.59 ± 6% perf-profile.self.cycles-pp.mutex_unlock
2.61 -0.3 2.31 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.60 ± 2% -0.3 1.30 ± 5% perf-profile.self.cycles-pp.update_load_avg
1.48 ± 3% -0.2 1.25 ± 2% perf-profile.self.cycles-pp.mutex_lock
1.95 -0.2 1.75 ± 7% perf-profile.self.cycles-pp.selinux_file_permission
1.06 ± 5% -0.2 0.86 ± 5% perf-profile.self.cycles-pp.pipe_write
1.46 ± 2% -0.2 1.28 ± 3% perf-profile.self.cycles-pp.pipe_read
1.65 ± 2% -0.2 1.49 ± 4% perf-profile.self.cycles-pp.__fget_light
0.91 ± 3% -0.2 0.74 perf-profile.self.cycles-pp.avc_has_perm
0.94 ± 2% -0.2 0.79 ± 2% perf-profile.self.cycles-pp.___might_sleep
1.84 -0.1 1.72 ± 2% perf-profile.self.cycles-pp.update_curr
1.05 -0.1 0.93 perf-profile.self.cycles-pp.enqueue_task_fair
0.63 ± 3% -0.1 0.52 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.60 ± 6% -0.1 0.51 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent
0.29 ± 3% -0.1 0.19 ± 5% perf-profile.self.cycles-pp.__mutex_lock
0.38 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.vfs_write
0.51 -0.1 0.43 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.49 -0.1 0.41 ± 2% perf-profile.self.cycles-pp.new_sync_write
0.35 ± 4% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter
0.81 ± 3% -0.1 0.75 perf-profile.self.cycles-pp.fsnotify
0.26 -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate
0.20 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.__mutex_unlock_slowpath
0.29 ± 5% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.file_has_perm
0.22 -0.1 0.16 ± 8% perf-profile.self.cycles-pp.wake_q_add
0.57 ± 2% -0.1 0.52 perf-profile.self.cycles-pp.new_sync_read
0.34 ± 2% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.dequeue_entity
0.26 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ksys_read
0.28 ± 2% -0.0 0.23 perf-profile.self.cycles-pp.security_file_permission
0.22 ± 3% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.finish_wait
0.25 -0.0 0.21 ± 4% perf-profile.self.cycles-pp.ksys_write
0.88 -0.0 0.83 perf-profile.self.cycles-pp.__lock_text_start
0.66 -0.0 0.61 perf-profile.self.cycles-pp.vfs_read
1.37 -0.0 1.32 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.29 ± 4% -0.0 0.25 perf-profile.self.cycles-pp.current_time
0.20 ± 3% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.rcu_all_qs
0.10 ± 8% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.42 -0.0 0.39 ± 2% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.34 -0.0 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_read
0.09 ± 5% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.wake_up_q
0.15 ± 8% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.inode_has_perm
0.14 ± 5% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.rb_next
0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.deactivate_task
0.11 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__fdget_pos
0.24 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.prepare_to_wait
0.14 ± 3% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret
0.13 ± 7% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.timespec64_trunc
0.06 -0.0 0.04 ± 57% perf-profile.self.cycles-pp.copyout
0.11 ± 4% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.17 ± 4% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.18 ± 2% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__might_fault
0.17 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy
0.15 ± 2% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.rb_insert_color
0.12 ± 3% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__sb_end_write
0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.06 -0.0 0.05 perf-profile.self.cycles-pp.kill_fasync
0.14 ± 3% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.66 +0.0 0.68 perf-profile.self.cycles-pp.__calc_delta
0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_load_tls
0.14 ± 3% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.67 +0.0 0.70 perf-profile.self.cycles-pp.dequeue_task_fair
0.23 ± 3% +0.0 0.27 ± 3% perf-profile.self.cycles-pp._cond_resched
0.17 ± 4% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.activate_task
0.35 ± 3% +0.0 0.40 perf-profile.self.cycles-pp.pick_next_entity
1.20 +0.1 1.25 perf-profile.self.cycles-pp.select_task_rq_fair
0.13 ± 6% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.attach_entity_load_avg
0.87 ± 3% +0.1 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.27 ± 6% +0.1 0.36 ± 7% perf-profile.self.cycles-pp.migrate_task_rq_fair
0.32 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.account_entity_enqueue
0.42 +0.1 0.56 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue
2.24 ± 3% +0.1 2.39 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.09 ± 2% +0.2 1.26 ± 2% perf-profile.self.cycles-pp.__switch_to
1.20 ± 4% +0.2 1.37 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
1.40 ± 3% +0.2 1.58 ± 2% perf-profile.self.cycles-pp.switch_fpu_return
0.89 ± 6% +0.2 1.10 ± 5% perf-profile.self.cycles-pp.update_rq_clock
2.19 ± 2% +0.2 2.42 perf-profile.self.cycles-pp.load_new_mm_cr3
1.33 ± 4% +0.3 1.60 ± 3% perf-profile.self.cycles-pp.___perf_sw_event
0.34 ± 14% +0.3 0.66 ± 9% perf-profile.self.cycles-pp.find_next_bit
0.28 ± 23% +0.4 0.72 ± 12% perf-profile.self.cycles-pp.cpumask_next_wrap
0.89 +0.7 1.58 ± 4% perf-profile.self.cycles-pp._raw_spin_lock
1.04 +0.7 1.76 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.65 ± 12% +3.0 3.61 ± 13% perf-profile.self.cycles-pp.select_idle_sibling
1.71 ± 29% +3.3 4.97 ± 13% perf-profile.self.cycles-pp.available_idle_cpu
***************************************************************************************************
lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2018-04-03.cgz/lkp-cfl-e1/hackbench/0xb8
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:2 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
1:2 -50% :4 dmesg.WARNING:stack_recursion
%stddev %change %stddev
\ | \
49845 -6.8% 46466 hackbench.throughput
8.776e+08 +13.0% 9.915e+08 hackbench.time.involuntary_context_switches
93623 -6.3% 87707 hackbench.time.minor_page_faults
740.08 -1.4% 729.54 hackbench.time.user_time
1.569e+09 +3.5% 1.623e+09 hackbench.time.voluntary_context_switches
3.072e+08 -6.2% 2.88e+08 hackbench.workload
21.45 -1.7% 21.09 boot-time.boot
1747212 ± 3% -31.0% 1205072 ± 14% cpuidle.POLL.time
1289000 ± 4% -36.6% 817407 ± 13% cpuidle.POLL.usage
4038486 +6.3% 4293331 vmstat.system.cs
394711 -10.2% 354353 ± 2% vmstat.system.in
10059 -15.4% 8506 ± 5% softirqs.CPU0.SCHED
309881 -27.3% 225159 ± 4% softirqs.CPU2.TIMER
220105 +20.8% 265807 ± 16% softirqs.CPU6.TIMER
120937 ± 3% -17.6% 99667 ± 2% softirqs.SCHED
18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.active_objs
18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.num_objs
8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.active_objs
8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.num_objs
5280 -8.8% 4814 slabinfo.task_delay_info.active_objs
5280 -8.8% 4814 slabinfo.task_delay_info.num_objs
65295 -1.9% 64034 proc-vmstat.nr_active_anon
60605 -1.5% 59671 proc-vmstat.nr_anon_pages
12935 -0.9% 12820 proc-vmstat.nr_slab_unreclaimable
65295 -1.9% 64034 proc-vmstat.nr_zone_active_anon
24774010 -16.6% 20673098 proc-vmstat.numa_hit
24774010 -16.6% 20673098 proc-vmstat.numa_local
24839243 -16.5% 20735391 proc-vmstat.pgalloc_normal
673024 -2.4% 657142 proc-vmstat.pgfault
24822112 -16.5% 20717243 proc-vmstat.pgfree
247.95 ± 14% -19.1% 200.59 ± 4% sched_debug.cfs_rq:/.load_avg.max
43.55 ± 3% -19.9% 34.86 ± 6% sched_debug.cfs_rq:/.load_avg.min
55.51 ± 6% -13.2% 48.19 ± 7% sched_debug.cfs_rq:/.load_avg.stddev
20116 ± 8% -12.8% 17531 ± 5% sched_debug.cfs_rq:/.runnable_weight.min
1288 +7.5% 1384 ± 3% sched_debug.cfs_rq:/.util_avg.max
54763 ± 9% +130.8% 126390 ± 25% sched_debug.cpu.avg_idle.avg
301502 ± 18% +84.9% 557406 ± 14% sched_debug.cpu.avg_idle.max
82233 ± 22% +122.7% 183153 ± 17% sched_debug.cpu.avg_idle.stddev
0.00 ± 12% -12.0% 0.00 ± 8% sched_debug.cpu.next_balance.stddev
16.92 ± 4% -7.5% 15.66 ± 3% sched_debug.cpu.nr_running.avg
2.92 ± 10% -20.9% 2.31 ± 16% sched_debug.cpu.nr_uninterruptible.avg
2977 ± 50% -73.0% 803.75 ± 43% interrupts.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1
546.00 ± 23% +119.5% 1198 ± 16% interrupts.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3
13831842 -11.5% 12245495 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts
45259 ± 5% +68.9% 76440 ± 15% interrupts.CPU0.TRM:Thermal_event_interrupts
2977 ± 50% -73.0% 803.75 ± 43% interrupts.CPU1.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1
45283 ± 5% +68.9% 76474 ± 15% interrupts.CPU1.TRM:Thermal_event_interrupts
13947887 -11.0% 12413074 interrupts.CPU10.RES:Rescheduling_interrupts
45275 ± 5% +68.9% 76468 ± 15% interrupts.CPU10.TRM:Thermal_event_interrupts
45286 ± 5% +68.9% 76468 ± 15% interrupts.CPU11.TRM:Thermal_event_interrupts
13579741 -11.8% 11974820 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU12.TRM:Thermal_event_interrupts
13668073 -11.3% 12119346 interrupts.CPU13.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76474 ± 15% interrupts.CPU13.TRM:Thermal_event_interrupts
13578574 -7.8% 12518854 interrupts.CPU14.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76467 ± 15% interrupts.CPU14.TRM:Thermal_event_interrupts
13689117 -10.1% 12310290 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76473 ± 15% interrupts.CPU15.TRM:Thermal_event_interrupts
13924470 -10.5% 12461613 interrupts.CPU2.RES:Rescheduling_interrupts
45264 ± 5% +68.9% 76467 ± 15% interrupts.CPU2.TRM:Thermal_event_interrupts
546.00 ± 23% +119.5% 1198 ± 16% interrupts.CPU3.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3
13951884 -10.5% 12480739 interrupts.CPU3.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76469 ± 15% interrupts.CPU3.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76471 ± 15% interrupts.CPU4.TRM:Thermal_event_interrupts
14030690 -10.7% 12530992 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU5.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76474 ± 15% interrupts.CPU6.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76472 ± 15% interrupts.CPU7.TRM:Thermal_event_interrupts
13319036 -10.2% 11966842 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts
45280 ± 5% +68.7% 76405 ± 15% interrupts.CPU8.TRM:Thermal_event_interrupts
13401328 -11.8% 11816711 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts
45279 ± 5% +68.9% 76467 ± 15% interrupts.CPU9.TRM:Thermal_event_interrupts
2.19e+08 -9.8% 1.974e+08 ± 2% interrupts.RES:Rescheduling_interrupts
724498 ± 5% +68.9% 1223435 ± 15% interrupts.TRM:Thermal_event_interrupts
53.44 +4.2% 55.68 perf-stat.i.MPKI
0.12 -0.0 0.10 ± 4% perf-stat.i.cache-miss-rate%
1764490 -13.1% 1533362 ± 3% perf-stat.i.cache-misses
1.767e+09 +3.6% 1.832e+09 perf-stat.i.cache-references
4055785 +6.2% 4306694 perf-stat.i.context-switches
336154 +21.5% 408363 perf-stat.i.cpu-migrations
34528 ± 5% +10.3% 38089 ± 3% perf-stat.i.cycles-between-cache-misses
0.01 ± 6% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate%
899577 ± 6% -15.2% 762553 ± 5% perf-stat.i.dTLB-load-misses
0.00 ± 7% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate%
22816 ± 7% -13.2% 19813 ± 4% perf-stat.i.dTLB-store-misses
6.044e+09 -1.2% 5.969e+09 perf-stat.i.dTLB-stores
31331199 -3.4% 30272777 perf-stat.i.iTLB-load-misses
1061 +3.1% 1094 perf-stat.i.instructions-per-iTLB-miss
1077 -2.9% 1046 perf-stat.i.minor-faults
73018 -10.9% 65028 ± 2% perf-stat.i.node-loads
122632 ± 2% -9.3% 111181 ± 4% perf-stat.i.node-stores
1077 -2.9% 1046 perf-stat.i.page-faults
53.29 +4.2% 55.53 perf-stat.overall.MPKI
0.10 -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate%
25264 +15.3% 29120 ± 3% perf-stat.overall.cycles-between-cache-misses
0.01 ± 6% -0.0 0.01 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 7% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate%
1058 +2.9% 1089 perf-stat.overall.instructions-per-iTLB-miss
65158 +6.6% 69487 perf-stat.overall.path-length
1761814 -13.1% 1530883 ± 3% perf-stat.ps.cache-misses
1.765e+09 +3.7% 1.829e+09 perf-stat.ps.cache-references
4048770 +6.2% 4299578 perf-stat.ps.context-switches
335565 +21.5% 407697 perf-stat.ps.cpu-migrations
898100 ± 6% -15.2% 761293 ± 5% perf-stat.ps.dTLB-load-misses
22777 ± 7% -13.2% 19781 ± 4% perf-stat.ps.dTLB-store-misses
6.034e+09 -1.2% 5.96e+09 perf-stat.ps.dTLB-stores
31277565 -3.4% 30222833 perf-stat.ps.iTLB-load-misses
1075 -2.9% 1044 perf-stat.ps.minor-faults
72896 -10.9% 64922 ± 2% perf-stat.ps.node-loads
122424 ± 2% -9.3% 111005 ± 4% perf-stat.ps.node-stores
1075 -2.9% 1044 perf-stat.ps.page-faults
39.37 ±100% -36.4 2.98 ±173% perf-profile.calltrace.cycles-pp.start_thread
21.68 ±100% -20.0 1.64 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread
20.14 ±100% -18.6 1.52 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
20.00 ±100% -18.5 1.51 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
18.05 ±100% -16.7 1.39 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
16.75 ±100% -15.5 1.22 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread
16.70 ±100% -15.4 1.26 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
14.98 ±100% -13.9 1.09 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
14.82 ±100% -13.7 1.08 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
13.95 ±100% -12.9 1.03 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
12.97 ±100% -12.0 0.95 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
3.69 ± 6% -1.1 2.58 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.25 ± 6% -1.0 3.29 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.54 ± 10% -0.5 1.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
1.07 ± 16% -0.5 0.60 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
1.14 ± 2% -0.3 0.80 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait
0.55 ± 4% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read
1.90 ± 9% -0.2 1.67 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.71 ± 7% -0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
0.68 ± 19% +0.2 0.86 perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.67 ± 16% +0.2 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
0.29 ±100% +0.4 0.64 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
0.31 ±100% +0.4 0.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.32 ±100% +0.4 0.68 ± 2% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.29 ±100% +0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.35 ±100% +0.4 0.78 perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
1.23 ± 19% +0.4 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.29 ± 18% +0.5 1.76 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule
0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule
8.82 +0.6 9.46 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.47 ±100% +0.7 1.14 ± 6% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
2.21 ± 10% +0.7 2.88 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
9.10 +0.7 9.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
9.04 +0.7 9.74 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.71 ±100% +0.9 1.61 ± 3% perf-profile.calltrace.cycles-pp.__switch_to
0.68 ± 99% +0.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr
0.86 ±100% +0.9 1.79 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
6.54 +1.0 7.53 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
26.96 +1.0 27.95 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
1.42 ± 63% +1.0 2.47 ± 3% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.54 ± 8% +1.2 5.79 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
14.99 +1.3 16.27 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
15.35 +1.3 16.66 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
27.37 +1.3 28.70 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
17.81 +1.6 19.38 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
3.95 ± 11% +1.8 5.70 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
23.38 +2.0 25.33 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
24.30 +2.0 26.27 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
23.73 +2.0 25.71 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
5.24 ± 10% +2.0 7.25 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
3.23 ± 57% +2.7 5.98 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.34 ± 57% +2.8 6.18 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.37 ±100% -36.4 2.98 ±173% perf-profile.children.cycles-pp.start_thread
21.68 ±100% -20.0 1.67 ±173% perf-profile.children.cycles-pp.__GI___libc_write
16.99 ±100% -15.7 1.24 ±173% perf-profile.children.cycles-pp.__GI___libc_read
6.84 ± 10% -2.1 4.71 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
45.35 -1.0 44.33 perf-profile.children.cycles-pp.ksys_write
6.70 ± 5% -1.0 5.70 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
5.90 ± 4% -0.6 5.34 ± 6% perf-profile.children.cycles-pp.security_file_permission
3.35 ± 9% -0.6 2.80 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
42.14 -0.5 41.63 perf-profile.children.cycles-pp.vfs_write
3.41 -0.4 3.01 perf-profile.children.cycles-pp.__fdget_pos
3.30 -0.4 2.91 perf-profile.children.cycles-pp.__fget_light
2.71 ± 2% -0.4 2.32 perf-profile.children.cycles-pp.__fget
3.23 -0.3 2.93 ± 3% perf-profile.children.cycles-pp.mutex_lock
2.76 -0.3 2.47 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
2.21 ± 3% -0.3 1.93 ± 10% perf-profile.children.cycles-pp.file_has_perm
3.14 -0.2 2.90 ± 4% perf-profile.children.cycles-pp.copy_page_to_iter
3.03 ± 5% -0.2 2.79 ± 5% perf-profile.children.cycles-pp.selinux_file_permission
2.10 ± 5% -0.2 1.86 ± 2% perf-profile.children.cycles-pp.mutex_unlock
1.50 ± 5% -0.2 1.27 ± 2% perf-profile.children.cycles-pp.fput_many
0.68 ± 17% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.__mutex_lock
1.66 ± 2% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.copyout
1.73 -0.2 1.57 ± 2% perf-profile.children.cycles-pp.update_rq_clock
1.30 -0.1 1.17 ± 6% perf-profile.children.cycles-pp.copyin
0.27 ± 11% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common
1.25 ± 3% -0.1 1.15 ± 3% perf-profile.children.cycles-pp.fsnotify
0.79 -0.1 0.69 ± 7% perf-profile.children.cycles-pp.reschedule_interrupt
0.91 -0.1 0.82 ± 5% perf-profile.children.cycles-pp._cond_resched
0.80 ± 2% -0.1 0.71 ± 3% perf-profile.children.cycles-pp.file_update_time
0.29 ± 10% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.rb_insert_color
0.26 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.07 ± 14% -0.0 0.03 ±100% perf-profile.children.cycles-pp.scheduler_ipi
0.22 ± 6% -0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_read
0.25 ± 4% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.27 -0.0 0.23 ± 9% perf-profile.children.cycles-pp.timespec64_trunc
0.23 ± 2% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.22 -0.0 0.20 ± 5% perf-profile.children.cycles-pp.__x64_sys_write
0.24 -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__list_add_valid
0.14 -0.0 0.12 ± 10% perf-profile.children.cycles-pp.kill_fasync
0.11 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_exclude_event
0.10 +0.0 0.12 ± 8% perf-profile.children.cycles-pp.is_cpu_allowed
0.09 ± 11% +0.0 0.12 ± 7% perf-profile.children.cycles-pp.rcu_note_context_switch
0.07 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.pkg_thermal_notify
0.17 ± 3% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.deactivate_task
0.16 +0.0 0.21 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.08 ± 6% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.intel_thermal_interrupt
0.08 ± 6% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.smp_thermal_interrupt
0.08 +0.1 0.13 ± 9% perf-profile.children.cycles-pp.thermal_interrupt
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.default_wake_function
0.19 ± 5% +0.1 0.25 ± 5% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.19 ± 5% +0.1 0.27 ± 4% perf-profile.children.cycles-pp.finish_wait
0.89 ± 8% +0.1 0.98 perf-profile.children.cycles-pp.__calc_delta
0.63 ± 4% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.33 ± 4% +0.1 0.44 ± 5% perf-profile.children.cycles-pp.set_next_buddy
1.23 ± 3% +0.1 1.34 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup
0.69 ± 4% +0.1 0.81 ± 3% perf-profile.children.cycles-pp.sched_clock
1.52 ± 3% +0.1 1.65 ± 2% perf-profile.children.cycles-pp.set_next_entity
0.73 ± 4% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.57 ± 4% +0.2 0.74 ± 4% perf-profile.children.cycles-pp.account_entity_dequeue
1.21 ± 4% +0.2 1.39 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se
1.99 ± 3% +0.2 2.20 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.64 ± 6% +0.2 1.85 perf-profile.children.cycles-pp.__switch_to_asm
1.60 ± 4% +0.2 1.85 ± 3% perf-profile.children.cycles-pp.update_cfs_group
3.91 +0.2 4.15 perf-profile.children.cycles-pp.update_curr
1.04 ± 5% +0.3 1.30 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
1.05 ± 9% +0.3 1.33 ± 2% perf-profile.children.cycles-pp.pick_next_entity
1.57 ± 3% +0.3 1.89 perf-profile.children.cycles-pp.native_write_msr
1.01 ± 9% +0.3 1.33 ± 5% perf-profile.children.cycles-pp.put_prev_entity
2.17 ± 7% +0.3 2.51 ± 2% perf-profile.children.cycles-pp.switch_fpu_return
1.93 ± 7% +0.4 2.36 perf-profile.children.cycles-pp.__switch_to
2.99 ± 4% +0.6 3.57 ± 2% perf-profile.children.cycles-pp.reweight_entity
8.89 +0.6 9.50 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
9.16 +0.7 9.82 perf-profile.children.cycles-pp.ttwu_do_activate
9.09 +0.7 9.76 ± 2% perf-profile.children.cycles-pp.activate_task
5.18 ± 6% +0.9 6.04 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
6.71 +0.9 7.63 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
28.01 +1.4 29.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock
4.91 ± 7% +1.4 6.28 ± 4% perf-profile.children.cycles-pp.exit_to_usermode_loop
18.19 +1.4 19.56 perf-profile.children.cycles-pp.pipe_wait
4.03 ± 11% +1.7 5.75 ± 6% perf-profile.children.cycles-pp.select_idle_sibling
23.86 ± 2% +1.8 25.71 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
24.44 +1.9 26.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common
23.84 +1.9 25.77 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
5.29 ± 10% +2.0 7.29 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair
20.27 ± 2% +2.3 22.53 ± 2% perf-profile.children.cycles-pp.__schedule
20.36 ± 2% +2.5 22.82 ± 2% perf-profile.children.cycles-pp.schedule
6.82 ± 10% -2.1 4.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
2.67 ± 2% -0.4 2.29 perf-profile.self.cycles-pp.__fget
2.72 -0.3 2.44 ± 5% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.22 -0.2 0.98 ± 2% perf-profile.self.cycles-pp.update_rq_clock
1.36 ± 3% -0.2 1.13 ± 6% perf-profile.self.cycles-pp.pipe_write
1.48 ± 5% -0.2 1.24 ± 3% perf-profile.self.cycles-pp.fput_many
2.04 ± 5% -0.2 1.81 ± 2% perf-profile.self.cycles-pp.mutex_unlock
1.96 -0.2 1.73 ± 2% perf-profile.self.cycles-pp.mutex_lock
0.66 ± 12% -0.1 0.53 ± 10% perf-profile.self.cycles-pp.file_has_perm
0.38 ± 10% -0.1 0.28 ± 9% perf-profile.self.cycles-pp.ksys_read
0.34 ± 7% -0.1 0.25 ± 7% perf-profile.self.cycles-pp.__mutex_lock
1.21 ± 3% -0.1 1.11 ± 2% perf-profile.self.cycles-pp.fsnotify
1.00 -0.1 0.94 ± 5% perf-profile.self.cycles-pp.__might_sleep
0.60 ± 4% -0.1 0.55 ± 8% perf-profile.self.cycles-pp.vfs_write
0.29 ± 10% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.rb_insert_color
0.32 ± 6% -0.0 0.28 ± 6% perf-profile.self.cycles-pp.ksys_write
0.26 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.scheduler_ipi
0.36 ± 5% -0.0 0.32 ± 8% perf-profile.self.cycles-pp.__sb_start_write
0.29 -0.0 0.24 ± 4% perf-profile.self.cycles-pp.check_preempt_curr
0.22 ± 6% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.__x64_sys_read
0.25 -0.0 0.21 ± 8% perf-profile.self.cycles-pp.timespec64_trunc
0.22 ± 4% -0.0 0.19 ± 11% perf-profile.self.cycles-pp.wake_q_add
0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__x64_sys_write
0.21 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__list_add_valid
0.11 ± 4% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.kill_fasync
0.06 ± 9% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.rcu_note_context_switch
0.15 +0.0 0.18 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function
0.16 +0.0 0.19 ± 9% perf-profile.self.cycles-pp.deactivate_task
0.26 +0.0 0.29 ± 5% perf-profile.self.cycles-pp.rcu_all_qs
0.33 ± 3% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.set_next_entity
0.18 ± 5% +0.0 0.22 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.03 ±100% +0.0 0.07 perf-profile.self.cycles-pp.check_cfs_rq_runtime
0.16 ± 6% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.16 +0.0 0.20 ± 12% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.20 ± 2% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.activate_task
0.41 ± 6% +0.1 0.46 perf-profile.self.cycles-pp.schedule
1.96 +0.1 2.02 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.15 ± 6% +0.1 0.22 perf-profile.self.cycles-pp.finish_wait
0.41 ± 4% +0.1 0.48 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue
0.61 ± 4% +0.1 0.71 ± 3% perf-profile.self.cycles-pp.native_sched_clock
0.83 ± 2% +0.1 0.93 perf-profile.self.cycles-pp.dequeue_task_fair
0.29 ± 3% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.set_next_buddy
1.02 +0.1 1.15 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.46 ± 12% +0.2 0.61 ± 3% perf-profile.self.cycles-pp.pick_next_entity
1.17 ± 4% +0.2 1.35 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
0.42 ± 8% +0.2 0.61 ± 5% perf-profile.self.cycles-pp.account_entity_dequeue
1.64 ± 6% +0.2 1.85 perf-profile.self.cycles-pp.__switch_to_asm
1.23 ± 4% +0.2 1.46 ± 3% perf-profile.self.cycles-pp.reweight_entity
1.07 ± 5% +0.2 1.31 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.90 ± 5% +0.2 1.14 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
1.57 ± 4% +0.2 1.82 ± 2% perf-profile.self.cycles-pp.update_cfs_group
1.17 ± 3% +0.3 1.42 ± 7% perf-profile.self.cycles-pp.update_load_avg
1.56 ± 4% +0.3 1.88 perf-profile.self.cycles-pp.native_write_msr
2.16 ± 7% +0.3 2.50 ± 2% perf-profile.self.cycles-pp.switch_fpu_return
1.84 ± 8% +0.4 2.25 ± 2% perf-profile.self.cycles-pp.__switch_to
1.22 +0.5 1.73 ± 4% perf-profile.self.cycles-pp._raw_spin_lock
0.92 ± 11% +1.3 2.27 ± 6% perf-profile.self.cycles-pp.select_idle_sibling
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
4 months, 4 weeks
[btrfs] c75e839414: aim7.jobs-per-min -9.1% regression
by kernel test robot
Greeting,
FYI, we noticed a -9.1% regression of aim7.jobs-per-min due to commit:
commit: c75e839414d3610e6487ae3145199c500d55f7f7 ("btrfs: kill the subvol_srcu")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: aim7
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
disk: 4BRD_12G
md: RAID0
fs: btrfs
test: disk_wrt
load: 1500
cpufreq_governor: performance
ucode: 0x500002c
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/4BRD_12G/btrfs/x86_64-rhel-7.6/1500/RAID0/debian-x86_64-20191114.cgz/lkp-csl-2ap2/disk_wrt/aim7/0x500002c
commit:
efc3453494 ("btrfs: make btrfs_cleanup_fs_roots use the radix tree lock")
c75e839414 ("btrfs: kill the subvol_srcu")
efc3453494af7818 c75e839414d3610e6487ae31451
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
3:9 -33% :8 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
%stddev %change %stddev
\ | \
29509 ± 2% -9.1% 26837 ± 2% aim7.jobs-per-min
305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time
305.28 ± 2% +10.0% 335.72 ± 2% aim7.time.elapsed_time.max
4883135 ± 10% +37.9% 6735464 ± 7% aim7.time.involuntary_context_switches
56288 ± 2% +10.5% 62202 ± 2% aim7.time.system_time
2344783 +6.5% 2497364 ± 2% aim7.time.voluntary_context_switches
62337721 ± 2% +9.8% 68456490 ± 2% turbostat.IRQ
431.56 ± 6% +22.3% 527.88 ± 4% vmstat.procs.r
27340 ± 2% +11.2% 30397 ± 2% vmstat.system.cs
226804 ± 6% +21.7% 276057 ± 4% meminfo.Active(file)
221309 ± 6% +22.3% 270668 ± 4% meminfo.Dirty
720.89 ±111% +49.3% 1076 ± 73% meminfo.Mlocked
14278 ± 2% -8.3% 13094 ± 2% meminfo.max_used_kB
57228 ± 6% +22.7% 70195 ± 5% numa-meminfo.node0.Active(file)
55433 ± 6% +21.6% 67431 ± 4% numa-meminfo.node0.Dirty
56152 ± 6% +21.4% 68180 ± 5% numa-meminfo.node1.Active(file)
55001 ± 6% +22.5% 67397 ± 4% numa-meminfo.node1.Dirty
56373 ± 6% +21.7% 68594 ± 4% numa-meminfo.node2.Active(file)
55222 ± 7% +22.6% 67726 ± 4% numa-meminfo.node2.Dirty
56671 ± 6% +20.5% 68317 ± 3% numa-meminfo.node3.Active(file)
55285 ± 6% +21.8% 67355 ± 4% numa-meminfo.node3.Dirty
56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_active_file
55342 ± 6% +22.3% 67662 ± 4% proc-vmstat.nr_dirty
402316 +2.1% 410951 proc-vmstat.nr_file_pages
180.22 ±111% +49.4% 269.25 ± 73% proc-vmstat.nr_mlock
56694 ± 6% +21.7% 69019 ± 4% proc-vmstat.nr_zone_active_file
54680 ± 6% +22.8% 67168 ± 4% proc-vmstat.nr_zone_write_pending
3144381 ± 2% +6.1% 3335275 proc-vmstat.pgactivate
1387558 ± 2% +7.9% 1496754 ± 2% proc-vmstat.pgfault
983.33 ± 4% +5.4% 1036 proc-vmstat.unevictable_pgs_culled
14331 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_active_file
13884 ± 6% +21.6% 16884 ± 4% numa-vmstat.node0.nr_dirty
14330 ± 6% +22.6% 17566 ± 5% numa-vmstat.node0.nr_zone_active_file
13714 ± 6% +22.2% 16755 ± 4% numa-vmstat.node0.nr_zone_write_pending
14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_active_file
13763 ± 6% +22.3% 16838 ± 4% numa-vmstat.node1.nr_dirty
14047 ± 6% +21.3% 17043 ± 4% numa-vmstat.node1.nr_zone_active_file
13599 ± 6% +23.0% 16726 ± 4% numa-vmstat.node1.nr_zone_write_pending
14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_active_file
13786 ± 6% +22.6% 16906 ± 4% numa-vmstat.node2.nr_dirty
14074 ± 5% +21.7% 17130 ± 4% numa-vmstat.node2.nr_zone_active_file
13617 ± 7% +23.3% 16789 ± 4% numa-vmstat.node2.nr_zone_write_pending
14151 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_active_file
13790 ± 6% +22.1% 16832 ± 4% numa-vmstat.node3.nr_dirty
14152 ± 6% +20.7% 17074 ± 3% numa-vmstat.node3.nr_zone_active_file
13627 ± 6% +22.5% 16696 ± 4% numa-vmstat.node3.nr_zone_write_pending
828631 ± 16% +49.9% 1241934 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev
1.33 ± 30% +96.9% 2.61 ± 14% sched_debug.cfs_rq:/.nr_spread_over.avg
831152 ± 16% +49.7% 1244586 ± 8% sched_debug.cfs_rq:/.spread0.stddev
1.92 ± 14% +28.4% 2.47 ± 8% sched_debug.cpu.nr_running.avg
5.00 ± 9% +29.7% 6.48 ± 9% sched_debug.cpu.nr_running.max
0.90 ± 9% +16.0% 1.05 ± 3% sched_debug.cpu.nr_running.stddev
21790 ± 11% +16.7% 25440 ± 2% sched_debug.cpu.nr_switches.avg
19761 ± 11% +17.9% 23289 ± 2% sched_debug.cpu.nr_switches.min
20403 ± 12% +18.1% 24101 ± 2% sched_debug.cpu.sched_count.avg
27825 ± 10% +15.8% 32227 ± 4% sched_debug.cpu.sched_count.max
18995 ± 13% +18.3% 22463 ± 2% sched_debug.cpu.sched_count.min
1143 ± 11% +15.6% 1322 ± 4% sched_debug.cpu.sched_count.stddev
1404 ± 7% -31.0% 969.66 ± 11% sched_debug.cpu.sched_goidle.avg
1186 ± 8% -36.0% 759.25 ± 14% sched_debug.cpu.sched_goidle.min
16821 ± 14% +20.6% 20284 ± 6% sched_debug.cpu.ttwu_count.max
2790 ± 11% +21.7% 3395 ± 7% sched_debug.cpu.ttwu_count.stddev
398.19 ± 16% +28.3% 510.73 ± 10% sched_debug.cpu.ttwu_local.stddev
43238516 ± 5% +19.0% 51459028 ± 4% perf-stat.i.cache-misses
83870573 ± 4% +13.8% 95436555 ± 4% perf-stat.i.cache-references
27181 ± 2% +11.1% 30198 ± 2% perf-stat.i.context-switches
3008 ± 2% -7.1% 2794 perf-stat.i.cpu-migrations
12612 ± 5% -15.6% 10641 ± 3% perf-stat.i.cycles-between-cache-misses
1.1e+09 ± 2% -7.3% 1.019e+09 ± 2% perf-stat.i.dTLB-stores
10684629 ± 2% +10.8% 11842781 ± 2% perf-stat.i.node-load-misses
641429 ± 2% +24.4% 798190 ± 2% perf-stat.i.node-loads
5915955 ± 2% -6.3% 5543368 ± 2% perf-stat.i.node-store-misses
380971 -6.7% 355629 ± 2% perf-stat.i.node-stores
1.66 ± 4% +14.4% 1.90 ± 3% perf-stat.overall.MPKI
13064 ± 5% -15.8% 10999 ± 4% perf-stat.overall.cycles-between-cache-misses
43484558 ± 5% +19.2% 51837476 ± 4% perf-stat.ps.cache-misses
83278267 ± 4% +14.4% 95253129 ± 3% perf-stat.ps.cache-references
27414 ± 2% +11.2% 30486 ± 2% perf-stat.ps.context-switches
3028 ± 2% -7.0% 2817 perf-stat.ps.cpu-migrations
1.104e+09 ± 2% -7.3% 1.023e+09 ± 2% perf-stat.ps.dTLB-stores
4394 -1.8% 4315 perf-stat.ps.minor-faults
10766873 ± 2% +11.0% 11947268 ± 2% perf-stat.ps.node-load-misses
645940 ± 2% +24.7% 805392 ± 2% perf-stat.ps.node-loads
5965618 ± 2% -6.2% 5595562 ± 2% perf-stat.ps.node-store-misses
383092 -6.5% 358124 ± 2% perf-stat.ps.node-stores
4395 -1.8% 4315 perf-stat.ps.page-faults
1.538e+13 ± 2% +10.0% 1.692e+13 ± 2% perf-stat.total.instructions
27.39 -0.1 27.28 perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
35.13 +0.1 35.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write
35.23 +0.1 35.34 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
35.48 +0.1 35.60 perf-profile.calltrace.cycles-pp.btrfs_reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
35.08 +0.1 35.22 perf-profile.calltrace.cycles-pp._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter
34.97 +0.1 35.12 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write
35.57 +0.1 35.72 perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
35.10 +0.2 35.26 perf-profile.calltrace.cycles-pp.btrfs_block_rsv_release.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write
35.11 +0.2 35.27 perf-profile.calltrace.cycles-pp.btrfs_inode_rsv_release.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write
98.71 +0.2 98.88 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
98.69 +0.2 98.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
98.74 +0.2 98.91 perf-profile.calltrace.cycles-pp.write
98.71 +0.2 98.89 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
98.68 +0.2 98.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
98.64 +0.2 98.82 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
98.63 +0.2 98.81 perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
98.58 +0.2 98.77 perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.new_sync_write.vfs_write.ksys_write
27.39 -0.1 27.28 perf-profile.children.cycles-pp.btrfs_dirty_pages
0.56 ± 5% -0.1 0.46 ± 9% perf-profile.children.cycles-pp.btrfs_get_extent
0.51 ± 5% -0.1 0.41 ± 9% perf-profile.children.cycles-pp.btrfs_lookup_file_extent
0.52 ± 5% -0.1 0.43 ± 9% perf-profile.children.cycles-pp.btrfs_search_slot
0.44 ± 6% -0.1 0.35 ± 11% perf-profile.children.cycles-pp.btrfs_read_lock_root_node
0.43 ± 6% -0.1 0.34 ± 11% perf-profile.children.cycles-pp.btrfs_tree_read_lock
0.19 ± 15% -0.1 0.11 ± 27% perf-profile.children.cycles-pp.start_secondary
0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter
0.19 ± 16% -0.1 0.10 ± 27% perf-profile.children.cycles-pp.cpuidle_enter_state
0.41 ± 7% -0.1 0.33 ± 8% perf-profile.children.cycles-pp.rwsem_optimistic_spin
0.46 ± 7% -0.1 0.38 ± 8% perf-profile.children.cycles-pp.rwsem_down_write_slowpath
0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.secondary_startup_64
0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.cpu_startup_entry
0.19 ± 15% -0.1 0.11 ± 25% perf-profile.children.cycles-pp.do_idle
0.18 ± 15% -0.1 0.10 ± 24% perf-profile.children.cycles-pp.intel_idle
0.29 ± 9% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.osq_lock
0.19 ± 6% -0.0 0.15 ± 16% perf-profile.children.cycles-pp.finish_wait
0.39 ± 5% -0.0 0.35 ± 6% perf-profile.children.cycles-pp.do_unlinkat
0.39 ± 6% -0.0 0.35 ± 7% perf-profile.children.cycles-pp.unlink
0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_open
0.26 ± 5% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_sys_openat2
0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.do_filp_open
0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.path_openat
0.26 ± 4% -0.0 0.22 ± 6% perf-profile.children.cycles-pp.creat
0.00 +0.0 0.05 perf-profile.children.cycles-pp.btrfs_calculate_inode_block_rsv_size
99.65 +0.1 99.73 perf-profile.children.cycles-pp.do_syscall_64
99.66 +0.1 99.74 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
35.62 +0.1 35.74 perf-profile.children.cycles-pp.btrfs_reserve_metadata_bytes
35.57 +0.1 35.72 perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
97.55 +0.2 97.72 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
98.74 +0.2 98.92 perf-profile.children.cycles-pp.write
98.70 +0.2 98.88 perf-profile.children.cycles-pp.ksys_write
98.69 +0.2 98.87 perf-profile.children.cycles-pp.vfs_write
98.65 +0.2 98.83 perf-profile.children.cycles-pp.new_sync_write
98.63 +0.2 98.81 perf-profile.children.cycles-pp.btrfs_file_write_iter
98.59 +0.2 98.78 perf-profile.children.cycles-pp.btrfs_buffered_write
97.60 +0.2 97.79 perf-profile.children.cycles-pp._raw_spin_lock
0.18 ± 15% -0.1 0.10 ± 24% perf-profile.self.cycles-pp.intel_idle
0.28 ± 9% -0.1 0.21 ± 12% perf-profile.self.cycles-pp.osq_lock
0.45 ± 2% -0.0 0.41 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
135788 ± 2% +13.1% 153639 ± 3% softirqs.CPU0.RCU
115547 ± 3% +8.0% 124768 ± 4% softirqs.CPU0.TIMER
135243 ± 3% +14.0% 154188 ± 2% softirqs.CPU1.RCU
136056 ± 3% +12.4% 152960 ± 3% softirqs.CPU10.RCU
134253 ± 3% +14.2% 153260 ± 3% softirqs.CPU100.RCU
134668 ± 4% +13.7% 153157 ± 2% softirqs.CPU101.RCU
109995 ± 3% +7.9% 118637 ± 3% softirqs.CPU101.TIMER
134989 ± 4% +13.9% 153700 ± 2% softirqs.CPU102.RCU
109876 ± 3% +8.9% 119608 ± 2% softirqs.CPU102.TIMER
134868 ± 4% +13.8% 153427 ± 3% softirqs.CPU103.RCU
109933 ± 3% +8.2% 118946 ± 3% softirqs.CPU103.TIMER
134454 ± 3% +14.0% 153285 ± 3% softirqs.CPU104.RCU
109731 ± 2% +8.4% 118965 ± 3% softirqs.CPU104.TIMER
134878 ± 3% +13.6% 153283 ± 3% softirqs.CPU105.RCU
134096 ± 3% +14.1% 153022 ± 3% softirqs.CPU106.RCU
109821 ± 2% +8.2% 118845 ± 3% softirqs.CPU106.TIMER
133905 ± 3% +15.0% 154004 ± 3% softirqs.CPU107.RCU
134117 ± 3% +14.3% 153232 ± 3% softirqs.CPU108.RCU
109545 ± 2% +8.8% 119182 ± 3% softirqs.CPU108.TIMER
134574 ± 3% +13.8% 153202 ± 3% softirqs.CPU109.RCU
109804 ± 3% +8.2% 118760 ± 3% softirqs.CPU109.TIMER
135106 ± 3% +14.8% 155124 ± 3% softirqs.CPU11.RCU
111746 ± 4% +7.8% 120460 ± 3% softirqs.CPU11.TIMER
134482 ± 4% +14.3% 153687 ± 3% softirqs.CPU110.RCU
110314 ± 3% +8.1% 119241 ± 3% softirqs.CPU110.TIMER
134464 ± 3% +14.3% 153634 ± 3% softirqs.CPU111.RCU
110626 ± 3% +8.3% 119815 ± 3% softirqs.CPU111.TIMER
134895 ± 3% +14.4% 154257 ± 3% softirqs.CPU112.RCU
110040 ± 3% +14.6% 126058 ± 16% softirqs.CPU112.TIMER
134644 ± 3% +13.6% 152993 ± 3% softirqs.CPU113.RCU
109693 ± 3% +8.1% 118569 ± 3% softirqs.CPU113.TIMER
134313 ± 3% +13.9% 152993 ± 3% softirqs.CPU114.RCU
109711 ± 3% +8.6% 119143 ± 3% softirqs.CPU114.TIMER
134559 ± 3% +13.8% 153115 ± 3% softirqs.CPU115.RCU
110226 ± 3% +8.2% 119221 ± 3% softirqs.CPU115.TIMER
134087 ± 3% +14.2% 153077 ± 3% softirqs.CPU116.RCU
110225 ± 3% +9.0% 120184 ± 3% softirqs.CPU116.TIMER
134271 ± 3% +14.1% 153220 ± 3% softirqs.CPU117.RCU
109179 ± 2% +8.3% 118274 ± 3% softirqs.CPU117.TIMER
134281 ± 3% +14.1% 153230 ± 3% softirqs.CPU118.RCU
134415 ± 3% +14.0% 153215 ± 3% softirqs.CPU119.RCU
109514 ± 2% +8.3% 118643 ± 3% softirqs.CPU119.TIMER
135560 ± 3% +13.5% 153874 ± 3% softirqs.CPU12.RCU
110316 ± 2% +8.5% 119734 ± 3% softirqs.CPU12.TIMER
134177 ± 3% +14.6% 153815 ± 3% softirqs.CPU120.RCU
134367 ± 3% +15.0% 154493 ± 3% softirqs.CPU121.RCU
133972 ± 3% +14.3% 153173 ± 3% softirqs.CPU122.RCU
134801 ± 4% +13.6% 153136 ± 3% softirqs.CPU123.RCU
134709 ± 3% +13.9% 153440 ± 3% softirqs.CPU124.RCU
135185 ± 3% +13.1% 152925 ± 3% softirqs.CPU125.RCU
134977 ± 3% +13.8% 153608 ± 3% softirqs.CPU126.RCU
133862 ± 4% +14.7% 153558 ± 3% softirqs.CPU127.RCU
134554 ± 3% +13.9% 153237 ± 3% softirqs.CPU128.RCU
134254 ± 3% +14.1% 153130 ± 3% softirqs.CPU129.RCU
135133 ± 3% +14.0% 154103 ± 3% softirqs.CPU13.RCU
110389 ± 2% +8.1% 119382 ± 3% softirqs.CPU13.TIMER
134319 ± 3% +13.8% 152909 ± 3% softirqs.CPU130.RCU
133590 ± 3% +14.5% 152925 ± 3% softirqs.CPU131.RCU
134416 ± 4% +14.0% 153223 ± 3% softirqs.CPU132.RCU
133931 ± 3% +14.5% 153359 ± 3% softirqs.CPU133.RCU
134031 ± 3% +14.5% 153527 ± 3% softirqs.CPU134.RCU
133851 ± 4% +14.3% 153052 ± 3% softirqs.CPU135.RCU
134235 ± 4% +13.9% 152959 ± 3% softirqs.CPU136.RCU
134003 ± 3% +14.2% 153037 ± 3% softirqs.CPU137.RCU
133453 ± 3% +14.4% 152625 ± 3% softirqs.CPU138.RCU
133514 ± 3% +14.8% 153289 ± 3% softirqs.CPU139.RCU
134952 ± 3% +13.6% 153372 ± 3% softirqs.CPU14.RCU
134045 ± 3% +13.7% 152361 ± 3% softirqs.CPU140.RCU
133662 ± 3% +14.1% 152528 ± 3% softirqs.CPU141.RCU
133643 ± 3% +14.7% 153283 ± 2% softirqs.CPU142.RCU
133313 ± 3% +15.0% 153254 ± 3% softirqs.CPU143.RCU
133585 ± 3% +14.4% 152837 ± 3% softirqs.CPU144.RCU
108590 ± 3% +12.7% 122339 ± 5% softirqs.CPU144.TIMER
133471 ± 3% +14.5% 152883 ± 3% softirqs.CPU145.RCU
108095 ± 3% +14.5% 123726 ± 6% softirqs.CPU145.TIMER
133588 ± 3% +15.4% 154173 ± 3% softirqs.CPU146.RCU
108268 ± 3% +19.8% 129684 ± 16% softirqs.CPU146.TIMER
134060 ± 3% +14.3% 153270 ± 3% softirqs.CPU147.RCU
108731 ± 3% +13.0% 122902 ± 5% softirqs.CPU147.TIMER
133778 ± 3% +14.1% 152681 ± 3% softirqs.CPU148.RCU
108208 ± 3% +12.6% 121875 ± 5% softirqs.CPU148.TIMER
133514 ± 3% +14.2% 152460 ± 3% softirqs.CPU149.RCU
108054 ± 3% +15.6% 124938 ± 9% softirqs.CPU149.TIMER
134677 ± 3% +14.5% 154138 ± 2% softirqs.CPU15.RCU
133245 ± 3% +14.4% 152373 ± 3% softirqs.CPU150.RCU
107788 ± 3% +13.1% 121957 ± 5% softirqs.CPU150.TIMER
133747 ± 3% +14.0% 152473 ± 3% softirqs.CPU151.RCU
108258 ± 4% +12.4% 121662 ± 5% softirqs.CPU151.TIMER
133819 ± 3% +14.1% 152726 ± 3% softirqs.CPU152.RCU
108163 ± 3% +12.7% 121940 ± 5% softirqs.CPU152.TIMER
134057 ± 3% +14.0% 152844 ± 3% softirqs.CPU153.RCU
107680 ± 3% +13.1% 121787 ± 5% softirqs.CPU153.TIMER
133660 ± 3% +14.2% 152675 ± 3% softirqs.CPU154.RCU
107911 ± 3% +15.4% 124485 ± 9% softirqs.CPU154.TIMER
133349 ± 3% +14.6% 152811 ± 3% softirqs.CPU155.RCU
107513 ± 3% +15.1% 123737 ± 6% softirqs.CPU155.TIMER
133901 ± 3% +14.0% 152615 ± 3% softirqs.CPU156.RCU
107853 ± 3% +13.0% 121904 ± 5% softirqs.CPU156.TIMER
133621 ± 3% +14.2% 152615 ± 3% softirqs.CPU157.RCU
108040 ± 3% +12.6% 121684 ± 5% softirqs.CPU157.TIMER
133891 ± 3% +14.2% 152862 ± 3% softirqs.CPU158.RCU
108534 ± 3% +12.7% 122300 ± 5% softirqs.CPU158.TIMER
133925 ± 3% +14.5% 153347 ± 3% softirqs.CPU159.RCU
108099 ± 3% +12.9% 122015 ± 5% softirqs.CPU159.TIMER
134893 ± 3% +14.5% 154389 ± 3% softirqs.CPU16.RCU
134169 ± 3% +13.9% 152812 ± 3% softirqs.CPU160.RCU
108300 ± 3% +12.9% 122318 ± 5% softirqs.CPU160.TIMER
133818 ± 3% +15.1% 154048 ± 4% softirqs.CPU161.RCU
107919 ± 3% +13.6% 122606 ± 6% softirqs.CPU161.TIMER
133867 ± 3% +14.2% 152856 ± 3% softirqs.CPU162.RCU
107991 ± 3% +13.2% 122209 ± 5% softirqs.CPU162.TIMER
133579 ± 3% +14.3% 152654 ± 3% softirqs.CPU163.RCU
108317 ± 3% +12.9% 122271 ± 5% softirqs.CPU163.TIMER
133914 ± 3% +14.4% 153158 ± 3% softirqs.CPU164.RCU
133684 ± 3% +14.4% 152895 ± 3% softirqs.CPU165.RCU
107435 ± 3% +12.5% 120877 ± 5% softirqs.CPU165.TIMER
133983 ± 3% +13.9% 152582 ± 3% softirqs.CPU166.RCU
107432 ± 3% +13.2% 121641 ± 5% softirqs.CPU166.TIMER
133947 ± 3% +13.6% 152137 ± 3% softirqs.CPU167.RCU
134005 ± 3% +14.5% 153385 ± 4% softirqs.CPU168.RCU
115752 ± 4% +12.0% 129634 ± 4% softirqs.CPU168.TIMER
133557 ± 3% +15.2% 153865 ± 4% softirqs.CPU169.RCU
115884 ± 3% +11.7% 129481 ± 4% softirqs.CPU169.TIMER
135348 ± 3% +13.8% 153977 ± 3% softirqs.CPU17.RCU
110589 ± 3% +7.9% 119298 ± 3% softirqs.CPU17.TIMER
134463 ± 3% +14.0% 153229 ± 3% softirqs.CPU170.RCU
133740 ± 3% +14.1% 152630 ± 3% softirqs.CPU171.RCU
116524 ± 4% +10.9% 129237 ± 3% softirqs.CPU171.TIMER
133716 ± 3% +13.8% 152196 ± 3% softirqs.CPU172.RCU
116393 ± 4% +10.9% 129087 ± 4% softirqs.CPU172.TIMER
133564 ± 3% +13.9% 152165 ± 3% softirqs.CPU173.RCU
116237 ± 3% +11.4% 129460 ± 3% softirqs.CPU173.TIMER
133411 ± 3% +14.5% 152764 ± 3% softirqs.CPU174.RCU
116018 ± 3% +11.2% 129006 ± 4% softirqs.CPU174.TIMER
133147 ± 3% +15.0% 153104 ± 2% softirqs.CPU175.RCU
116052 ± 4% +11.7% 129664 ± 3% softirqs.CPU175.TIMER
133989 ± 3% +13.8% 152504 ± 3% softirqs.CPU176.RCU
116071 ± 4% +10.9% 128756 ± 3% softirqs.CPU176.TIMER
133902 ± 3% +14.1% 152811 ± 3% softirqs.CPU177.RCU
115994 ± 3% +11.1% 128849 ± 4% softirqs.CPU177.TIMER
133679 ± 3% +14.4% 152887 ± 3% softirqs.CPU178.RCU
115942 ± 4% +13.4% 131439 ± 5% softirqs.CPU178.TIMER
133985 ± 3% +14.0% 152762 ± 3% softirqs.CPU179.RCU
116347 ± 4% +11.2% 129422 ± 4% softirqs.CPU179.TIMER
134982 ± 3% +14.0% 153838 ± 3% softirqs.CPU18.RCU
110310 ± 3% +8.6% 119851 ± 3% softirqs.CPU18.TIMER
133236 ± 3% +14.7% 152870 ± 3% softirqs.CPU180.RCU
115901 ± 4% +11.3% 129054 ± 4% softirqs.CPU180.TIMER
133615 ± 3% +14.4% 152876 ± 3% softirqs.CPU181.RCU
115505 ± 4% +11.3% 128560 ± 3% softirqs.CPU181.TIMER
133171 ± 3% +14.7% 152810 ± 3% softirqs.CPU182.RCU
115955 ± 4% +11.0% 128687 ± 3% softirqs.CPU182.TIMER
133413 ± 3% +14.4% 152671 ± 3% softirqs.CPU183.RCU
116313 ± 4% +11.0% 129140 ± 3% softirqs.CPU183.TIMER
133598 ± 3% +14.8% 153388 ± 3% softirqs.CPU184.RCU
115462 ± 3% +13.0% 130430 ± 6% softirqs.CPU184.TIMER
133884 ± 3% +13.9% 152525 ± 3% softirqs.CPU185.RCU
115598 ± 4% +11.1% 128397 ± 4% softirqs.CPU185.TIMER
133262 ± 3% +14.5% 152646 ± 3% softirqs.CPU186.RCU
115745 ± 3% +11.1% 128612 ± 4% softirqs.CPU186.TIMER
133430 ± 3% +14.5% 152767 ± 3% softirqs.CPU187.RCU
115769 ± 4% +11.0% 128464 ± 3% softirqs.CPU187.TIMER
133259 ± 3% +14.8% 153036 ± 3% softirqs.CPU188.RCU
116108 ± 4% +11.3% 129232 ± 3% softirqs.CPU188.TIMER
133609 ± 3% +14.1% 152436 ± 3% softirqs.CPU189.RCU
115984 ± 4% +11.0% 128710 ± 3% softirqs.CPU189.TIMER
135090 ± 3% +13.9% 153802 ± 3% softirqs.CPU19.RCU
110865 ± 2% +8.2% 119936 ± 3% softirqs.CPU19.TIMER
133685 ± 3% +14.2% 152631 ± 3% softirqs.CPU190.RCU
115642 ± 3% +11.2% 128581 ± 4% softirqs.CPU190.TIMER
131031 ± 3% +14.6% 150126 ± 3% softirqs.CPU191.RCU
114275 ± 4% +11.4% 127249 ± 4% softirqs.CPU191.TIMER
134928 ± 3% +14.3% 154165 ± 3% softirqs.CPU2.RCU
135126 ± 3% +14.0% 154083 ± 3% softirqs.CPU20.RCU
111013 ± 3% +8.6% 120549 ± 3% softirqs.CPU20.TIMER
134663 ± 3% +14.5% 154251 ± 3% softirqs.CPU21.RCU
109671 ± 2% +8.2% 118708 ± 3% softirqs.CPU21.TIMER
135047 ± 3% +13.9% 153766 ± 3% softirqs.CPU22.RCU
134437 ± 3% +14.2% 153492 ± 3% softirqs.CPU23.RCU
109675 ± 2% +8.3% 118824 ± 3% softirqs.CPU23.TIMER
136114 ± 3% +13.3% 154277 ± 3% softirqs.CPU24.RCU
134825 ± 3% +14.2% 154007 ± 3% softirqs.CPU25.RCU
139234 ± 4% +13.8% 158515 ± 3% softirqs.CPU26.RCU
135056 ± 3% +14.1% 154062 ± 3% softirqs.CPU27.RCU
135307 ± 3% +13.7% 153897 ± 3% softirqs.CPU28.RCU
134686 ± 3% +14.7% 154497 ± 3% softirqs.CPU29.RCU
134688 ± 3% +14.4% 154115 ± 2% softirqs.CPU3.RCU
134642 ± 3% +14.5% 154226 ± 3% softirqs.CPU30.RCU
134773 ± 3% +14.2% 153887 ± 3% softirqs.CPU31.RCU
134523 ± 3% +14.3% 153733 ± 3% softirqs.CPU32.RCU
134488 ± 3% +14.4% 153801 ± 3% softirqs.CPU33.RCU
134483 ± 3% +14.1% 153380 ± 3% softirqs.CPU34.RCU
134886 ± 3% +13.6% 153180 ± 3% softirqs.CPU35.RCU
135021 ± 3% +13.6% 153340 ± 3% softirqs.CPU36.RCU
134064 ± 3% +15.1% 154277 ± 3% softirqs.CPU37.RCU
134166 ± 3% +14.8% 153967 ± 3% softirqs.CPU38.RCU
134488 ± 3% +14.4% 153871 ± 3% softirqs.CPU39.RCU
135294 ± 2% +14.6% 155025 ± 3% softirqs.CPU4.RCU
134598 ± 4% +14.3% 153848 ± 3% softirqs.CPU40.RCU
134255 ± 3% +14.8% 154099 ± 3% softirqs.CPU41.RCU
134334 ± 3% +14.3% 153491 ± 3% softirqs.CPU42.RCU
134201 ± 3% +14.5% 153654 ± 3% softirqs.CPU43.RCU
134260 ± 3% +14.2% 153359 ± 3% softirqs.CPU44.RCU
134005 ± 3% +14.4% 153367 ± 3% softirqs.CPU45.RCU
134236 ± 3% +14.3% 153419 ± 3% softirqs.CPU46.RCU
134552 ± 3% +13.8% 153056 ± 3% softirqs.CPU47.RCU
134623 ± 3% +14.4% 154028 ± 3% softirqs.CPU48.RCU
134348 ± 3% +14.7% 154080 ± 3% softirqs.CPU49.RCU
109020 ± 3% +18.6% 129279 ± 14% softirqs.CPU49.TIMER
134334 ± 3% +15.0% 154455 ± 3% softirqs.CPU5.RCU
110260 ± 2% +8.8% 119951 ± 4% softirqs.CPU5.TIMER
134377 ± 3% +13.8% 152954 ± 3% softirqs.CPU50.RCU
109169 ± 3% +15.0% 125548 ± 9% softirqs.CPU50.TIMER
134780 ± 3% +14.2% 153967 ± 3% softirqs.CPU51.RCU
109951 ± 3% +12.5% 123641 ± 5% softirqs.CPU51.TIMER
133929 ± 3% +15.0% 154012 ± 3% softirqs.CPU52.RCU
108665 ± 3% +12.8% 122613 ± 5% softirqs.CPU52.TIMER
133761 ± 3% +14.9% 153675 ± 3% softirqs.CPU53.RCU
108606 ± 3% +19.6% 129861 ± 16% softirqs.CPU53.TIMER
134369 ± 3% +14.1% 153255 ± 3% softirqs.CPU54.RCU
108917 ± 3% +12.5% 122585 ± 5% softirqs.CPU54.TIMER
133810 ± 3% +14.6% 153282 ± 3% softirqs.CPU55.RCU
108875 ± 3% +12.5% 122468 ± 5% softirqs.CPU55.TIMER
134038 ± 3% +14.3% 153248 ± 3% softirqs.CPU56.RCU
108569 ± 3% +12.8% 122502 ± 5% softirqs.CPU56.TIMER
133993 ± 4% +14.5% 153426 ± 3% softirqs.CPU57.RCU
108465 ± 3% +12.8% 122399 ± 5% softirqs.CPU57.TIMER
134049 ± 3% +14.7% 153708 ± 3% softirqs.CPU58.RCU
108634 ± 4% +18.8% 129107 ± 16% softirqs.CPU58.TIMER
134025 ± 3% +14.9% 154015 ± 3% softirqs.CPU59.RCU
108501 ± 3% +18.6% 128721 ± 13% softirqs.CPU59.TIMER
135247 ± 3% +14.5% 154804 ± 3% softirqs.CPU6.RCU
133745 ± 4% +14.7% 153452 ± 3% softirqs.CPU60.RCU
108526 ± 3% +13.0% 122688 ± 5% softirqs.CPU60.TIMER
133931 ± 3% +14.6% 153549 ± 3% softirqs.CPU61.RCU
108764 ± 3% +12.9% 122787 ± 5% softirqs.CPU61.TIMER
134682 ± 3% +13.8% 153266 ± 3% softirqs.CPU62.RCU
109188 ± 3% +12.5% 122806 ± 5% softirqs.CPU62.TIMER
134190 ± 3% +14.6% 153761 ± 2% softirqs.CPU63.RCU
108489 ± 3% +12.8% 122426 ± 5% softirqs.CPU63.TIMER
134280 ± 3% +14.3% 153538 ± 3% softirqs.CPU64.RCU
108493 ± 3% +13.3% 122953 ± 5% softirqs.CPU64.TIMER
134074 ± 3% +14.7% 153765 ± 3% softirqs.CPU65.RCU
108331 ± 3% +13.0% 122440 ± 5% softirqs.CPU65.TIMER
134424 ± 4% +14.3% 153692 ± 3% softirqs.CPU66.RCU
108874 ± 3% +12.5% 122490 ± 5% softirqs.CPU66.TIMER
133801 ± 3% +14.9% 153746 ± 3% softirqs.CPU67.RCU
108903 ± 3% +13.2% 123240 ± 5% softirqs.CPU67.TIMER
134085 ± 3% +14.4% 153371 ± 3% softirqs.CPU68.RCU
109310 ± 3% +12.3% 122810 ± 5% softirqs.CPU68.TIMER
134091 ± 3% +14.5% 153515 ± 3% softirqs.CPU69.RCU
108193 ± 3% +12.5% 121716 ± 5% softirqs.CPU69.TIMER
134883 ± 3% +13.6% 153181 ± 3% softirqs.CPU7.RCU
134392 ± 3% +14.8% 154240 ± 3% softirqs.CPU70.RCU
108056 ± 3% +13.4% 122578 ± 5% softirqs.CPU70.TIMER
133901 ± 3% +14.2% 152873 ± 3% softirqs.CPU71.RCU
134273 ± 3% +14.4% 153630 ± 3% softirqs.CPU72.RCU
116189 ± 3% +11.5% 129495 ± 3% softirqs.CPU72.TIMER
134231 ± 3% +14.5% 153682 ± 3% softirqs.CPU73.RCU
116348 ± 3% +11.3% 129453 ± 3% softirqs.CPU73.TIMER
134235 ± 3% +14.1% 153158 ± 3% softirqs.CPU74.RCU
133896 ± 3% +14.6% 153477 ± 3% softirqs.CPU75.RCU
116933 ± 4% +11.2% 129980 ± 3% softirqs.CPU75.TIMER
134014 ± 3% +14.3% 153198 ± 3% softirqs.CPU76.RCU
116889 ± 3% +11.0% 129768 ± 3% softirqs.CPU76.TIMER
134608 ± 3% +13.7% 152994 ± 3% softirqs.CPU77.RCU
117342 ± 3% +11.0% 130274 ± 3% softirqs.CPU77.TIMER
134551 ± 3% +13.9% 153253 ± 3% softirqs.CPU78.RCU
117229 ± 4% +10.5% 129549 ± 4% softirqs.CPU78.TIMER
134118 ± 3% +14.3% 153325 ± 3% softirqs.CPU79.RCU
116982 ± 3% +11.0% 129802 ± 3% softirqs.CPU79.TIMER
135303 ± 3% +13.9% 154129 ± 3% softirqs.CPU8.RCU
134699 ± 3% +13.6% 153047 ± 3% softirqs.CPU80.RCU
117058 ± 4% +10.6% 129470 ± 3% softirqs.CPU80.TIMER
134088 ± 3% +15.6% 155063 ± 3% softirqs.CPU81.RCU
116539 ± 4% +12.3% 130886 ± 5% softirqs.CPU81.TIMER
133989 ± 3% +14.6% 153526 ± 3% softirqs.CPU82.RCU
116614 ± 4% +16.8% 136238 ± 12% softirqs.CPU82.TIMER
133966 ± 3% +14.3% 153142 ± 3% softirqs.CPU83.RCU
117140 ± 3% +11.2% 130205 ± 4% softirqs.CPU83.TIMER
133561 ± 3% +15.0% 153661 ± 3% softirqs.CPU84.RCU
116648 ± 4% +11.3% 129844 ± 4% softirqs.CPU84.TIMER
134217 ± 3% +14.2% 153330 ± 3% softirqs.CPU85.RCU
116247 ± 4% +11.3% 129336 ± 3% softirqs.CPU85.TIMER
133956 ± 3% +14.5% 153338 ± 3% softirqs.CPU86.RCU
116654 ± 4% +11.1% 129560 ± 3% softirqs.CPU86.TIMER
133834 ± 3% +15.1% 154049 ± 2% softirqs.CPU87.RCU
116898 ± 4% +12.0% 130912 ± 3% softirqs.CPU87.TIMER
133951 ± 3% +15.1% 154211 ± 3% softirqs.CPU88.RCU
116139 ± 3% +16.0% 134676 ± 13% softirqs.CPU88.TIMER
134288 ± 3% +14.3% 153433 ± 2% softirqs.CPU89.RCU
116227 ± 3% +11.0% 129064 ± 3% softirqs.CPU89.TIMER
135289 ± 3% +13.6% 153744 ± 3% softirqs.CPU9.RCU
133598 ± 3% +14.5% 153019 ± 3% softirqs.CPU90.RCU
116395 ± 3% +10.9% 129084 ± 4% softirqs.CPU90.TIMER
133854 ± 4% +14.9% 153799 ± 3% softirqs.CPU91.RCU
116562 ± 4% +11.1% 129452 ± 3% softirqs.CPU91.TIMER
134107 ± 3% +14.1% 152958 ± 3% softirqs.CPU92.RCU
117107 ± 4% +10.6% 129559 ± 3% softirqs.CPU92.TIMER
133800 ± 3% +14.7% 153465 ± 3% softirqs.CPU93.RCU
116415 ± 3% +11.5% 129823 ± 4% softirqs.CPU93.TIMER
134092 ± 3% +14.0% 152835 ± 3% softirqs.CPU94.RCU
116063 ± 3% +11.4% 129252 ± 4% softirqs.CPU94.TIMER
134284 ± 3% +14.1% 153230 ± 4% softirqs.CPU95.RCU
117908 ± 3% +11.0% 130904 ± 4% softirqs.CPU95.TIMER
134264 ± 3% +14.3% 153455 ± 3% softirqs.CPU96.RCU
109819 ± 3% +8.6% 119258 ± 3% softirqs.CPU96.TIMER
135000 ± 3% +13.7% 153529 ± 3% softirqs.CPU97.RCU
134163 ± 3% +14.2% 153253 ± 3% softirqs.CPU98.RCU
109990 ± 3% +8.1% 118896 ± 3% softirqs.CPU98.TIMER
134966 ± 3% +13.5% 153245 ± 3% softirqs.CPU99.RCU
25776297 ± 3% +14.3% 29452390 ± 3% softirqs.RCU
21445010 ± 3% +9.5% 23488096 ± 3% softirqs.TIMER
614.00 ± 2% +9.9% 674.50 ± 2% interrupts.9:IO-APIC.9-fasteoi.acpi
611854 ± 2% +10.1% 673945 ± 2% interrupts.CPU0.LOC:Local_timer_interrupts
614.00 ± 2% +9.9% 674.50 ± 2% interrupts.CPU1.9:IO-APIC.9-fasteoi.acpi
611835 ± 2% +10.1% 673883 ± 2% interrupts.CPU1.LOC:Local_timer_interrupts
611859 ± 2% +10.2% 673975 ± 2% interrupts.CPU10.LOC:Local_timer_interrupts
7508 ± 5% +15.5% 8675 ± 4% interrupts.CPU10.RES:Rescheduling_interrupts
611802 ± 2% +10.2% 673904 ± 2% interrupts.CPU100.LOC:Local_timer_interrupts
7323 ± 5% +16.1% 8503 ± 3% interrupts.CPU100.RES:Rescheduling_interrupts
611804 ± 2% +10.2% 673917 ± 2% interrupts.CPU101.LOC:Local_timer_interrupts
7342 ± 3% +13.8% 8352 ± 3% interrupts.CPU101.RES:Rescheduling_interrupts
611780 ± 2% +10.2% 673877 ± 2% interrupts.CPU102.LOC:Local_timer_interrupts
7506 ± 5% +12.2% 8424 ± 4% interrupts.CPU102.RES:Rescheduling_interrupts
611734 ± 2% +10.2% 673851 ± 2% interrupts.CPU103.LOC:Local_timer_interrupts
7436 ± 3% +15.8% 8607 ± 4% interrupts.CPU103.RES:Rescheduling_interrupts
611800 ± 2% +10.2% 673908 ± 2% interrupts.CPU104.LOC:Local_timer_interrupts
7519 ± 5% +13.1% 8502 ± 2% interrupts.CPU104.RES:Rescheduling_interrupts
611790 ± 2% +10.1% 673864 ± 2% interrupts.CPU105.LOC:Local_timer_interrupts
7346 ± 5% +16.4% 8553 ± 2% interrupts.CPU105.RES:Rescheduling_interrupts
611801 ± 2% +10.1% 673859 ± 2% interrupts.CPU106.LOC:Local_timer_interrupts
7352 ± 5% +16.5% 8567 ± 2% interrupts.CPU106.RES:Rescheduling_interrupts
611797 ± 2% +10.1% 673890 ± 2% interrupts.CPU107.LOC:Local_timer_interrupts
7238 ± 6% +15.3% 8344 ± 4% interrupts.CPU107.RES:Rescheduling_interrupts
611778 ± 2% +10.2% 673882 ± 2% interrupts.CPU108.LOC:Local_timer_interrupts
7310 ± 5% +16.3% 8503 ± 5% interrupts.CPU108.RES:Rescheduling_interrupts
611788 ± 2% +10.1% 673881 ± 2% interrupts.CPU109.LOC:Local_timer_interrupts
7324 ± 6% +13.4% 8307 interrupts.CPU109.RES:Rescheduling_interrupts
611827 ± 2% +10.2% 673960 ± 2% interrupts.CPU11.LOC:Local_timer_interrupts
7598 ± 6% +12.6% 8557 ± 5% interrupts.CPU11.RES:Rescheduling_interrupts
611796 ± 2% +10.1% 673865 ± 2% interrupts.CPU110.LOC:Local_timer_interrupts
7326 ± 4% +16.2% 8513 ± 4% interrupts.CPU110.RES:Rescheduling_interrupts
611804 ± 2% +10.1% 673890 ± 2% interrupts.CPU111.LOC:Local_timer_interrupts
7312 ± 3% +14.4% 8365 ± 3% interrupts.CPU111.RES:Rescheduling_interrupts
611801 ± 2% +10.1% 673890 ± 2% interrupts.CPU112.LOC:Local_timer_interrupts
7277 ± 2% +14.6% 8337 ± 5% interrupts.CPU112.RES:Rescheduling_interrupts
611789 ± 2% +10.2% 673896 ± 2% interrupts.CPU113.LOC:Local_timer_interrupts
7346 ± 4% +15.2% 8461 ± 4% interrupts.CPU113.RES:Rescheduling_interrupts
611759 ± 2% +10.2% 673861 ± 2% interrupts.CPU114.LOC:Local_timer_interrupts
7290 ± 4% +17.9% 8594 ± 4% interrupts.CPU114.RES:Rescheduling_interrupts
611783 ± 2% +10.2% 673883 ± 2% interrupts.CPU115.LOC:Local_timer_interrupts
7470 ± 4% +13.6% 8489 ± 5% interrupts.CPU115.RES:Rescheduling_interrupts
611807 ± 2% +10.2% 673925 ± 2% interrupts.CPU116.LOC:Local_timer_interrupts
7336 ± 4% +15.3% 8455 ± 2% interrupts.CPU116.RES:Rescheduling_interrupts
611778 ± 2% +10.2% 673890 ± 2% interrupts.CPU117.LOC:Local_timer_interrupts
7326 ± 5% +15.9% 8493 ± 3% interrupts.CPU117.RES:Rescheduling_interrupts
611782 ± 2% +10.1% 673849 ± 2% interrupts.CPU118.LOC:Local_timer_interrupts
7239 ± 4% +18.4% 8574 ± 5% interrupts.CPU118.RES:Rescheduling_interrupts
611778 ± 2% +10.2% 673881 ± 2% interrupts.CPU119.LOC:Local_timer_interrupts
7327 ± 4% +13.7% 8335 ± 6% interrupts.CPU119.RES:Rescheduling_interrupts
611853 ± 2% +10.1% 673944 ± 2% interrupts.CPU12.LOC:Local_timer_interrupts
7516 ± 4% +17.0% 8798 ± 6% interrupts.CPU12.RES:Rescheduling_interrupts
611417 ± 2% +10.2% 673606 ± 2% interrupts.CPU120.LOC:Local_timer_interrupts
7392 ± 4% +14.2% 8444 ± 5% interrupts.CPU120.RES:Rescheduling_interrupts
611457 ± 2% +10.2% 673608 ± 2% interrupts.CPU121.LOC:Local_timer_interrupts
7376 ± 3% +15.6% 8525 ± 4% interrupts.CPU121.RES:Rescheduling_interrupts
611426 ± 2% +10.2% 673610 ± 2% interrupts.CPU122.LOC:Local_timer_interrupts
7466 ± 5% +11.7% 8336 ± 4% interrupts.CPU122.RES:Rescheduling_interrupts
611384 ± 2% +10.2% 673577 ± 2% interrupts.CPU123.LOC:Local_timer_interrupts
7240 ± 5% +18.2% 8559 ± 3% interrupts.CPU123.RES:Rescheduling_interrupts
611388 ± 2% +10.2% 673597 ± 2% interrupts.CPU124.LOC:Local_timer_interrupts
7245 ± 5% +18.0% 8548 ± 5% interrupts.CPU124.RES:Rescheduling_interrupts
611391 ± 2% +10.2% 673567 ± 2% interrupts.CPU125.LOC:Local_timer_interrupts
7318 ± 3% +15.3% 8436 ± 5% interrupts.CPU125.RES:Rescheduling_interrupts
611421 ± 2% +10.2% 673570 ± 2% interrupts.CPU126.LOC:Local_timer_interrupts
7258 ± 5% +16.4% 8448 ± 4% interrupts.CPU126.RES:Rescheduling_interrupts
611468 ± 2% +10.2% 673614 ± 2% interrupts.CPU127.LOC:Local_timer_interrupts
7276 ± 5% +16.9% 8508 ± 5% interrupts.CPU127.RES:Rescheduling_interrupts
611422 ± 2% +10.2% 673591 ± 2% interrupts.CPU128.LOC:Local_timer_interrupts
7278 ± 4% +17.0% 8518 ± 3% interrupts.CPU128.RES:Rescheduling_interrupts
611407 ± 2% +10.2% 673562 ± 2% interrupts.CPU129.LOC:Local_timer_interrupts
7284 ± 3% +17.0% 8519 ± 5% interrupts.CPU129.RES:Rescheduling_interrupts
611825 ± 2% +10.1% 673920 ± 2% interrupts.CPU13.LOC:Local_timer_interrupts
7550 ± 4% +16.2% 8771 ± 3% interrupts.CPU13.RES:Rescheduling_interrupts
611428 ± 2% +10.2% 673571 ± 2% interrupts.CPU130.LOC:Local_timer_interrupts
7337 ± 5% +16.4% 8541 ± 5% interrupts.CPU130.RES:Rescheduling_interrupts
611406 ± 2% +10.2% 673582 ± 2% interrupts.CPU131.LOC:Local_timer_interrupts
7205 ± 5% +17.1% 8440 ± 6% interrupts.CPU131.RES:Rescheduling_interrupts
611449 ± 2% +10.2% 673586 ± 2% interrupts.CPU132.LOC:Local_timer_interrupts
7431 ± 4% +12.7% 8372 ± 3% interrupts.CPU132.RES:Rescheduling_interrupts
611464 ± 2% +10.2% 673588 ± 2% interrupts.CPU133.LOC:Local_timer_interrupts
7289 ± 4% +15.9% 8446 ± 3% interrupts.CPU133.RES:Rescheduling_interrupts
611437 ± 2% +10.2% 673583 ± 2% interrupts.CPU134.LOC:Local_timer_interrupts
7321 ± 4% +17.7% 8616 ± 4% interrupts.CPU134.RES:Rescheduling_interrupts
611401 ± 2% +10.2% 673595 ± 2% interrupts.CPU135.LOC:Local_timer_interrupts
7300 ± 4% +15.7% 8446 ± 4% interrupts.CPU135.RES:Rescheduling_interrupts
611462 ± 2% +10.2% 673602 ± 2% interrupts.CPU136.LOC:Local_timer_interrupts
7389 ± 4% +13.9% 8417 ± 5% interrupts.CPU136.RES:Rescheduling_interrupts
611424 ± 2% +10.2% 673588 ± 2% interrupts.CPU137.LOC:Local_timer_interrupts
7126 ± 4% +20.0% 8550 ± 5% interrupts.CPU137.RES:Rescheduling_interrupts
611383 ± 2% +10.2% 673556 ± 2% interrupts.CPU138.LOC:Local_timer_interrupts
7297 ± 5% +16.1% 8475 ± 5% interrupts.CPU138.RES:Rescheduling_interrupts
611422 ± 2% +10.2% 673603 ± 2% interrupts.CPU139.LOC:Local_timer_interrupts
7160 ± 4% +17.6% 8423 ± 4% interrupts.CPU139.RES:Rescheduling_interrupts
611815 ± 2% +10.1% 673901 ± 2% interrupts.CPU14.LOC:Local_timer_interrupts
7576 ± 5% +16.3% 8810 ± 3% interrupts.CPU14.RES:Rescheduling_interrupts
611388 ± 2% +10.2% 673577 ± 2% interrupts.CPU140.LOC:Local_timer_interrupts
7342 ± 6% +12.9% 8293 ± 4% interrupts.CPU140.RES:Rescheduling_interrupts
611394 ± 2% +10.2% 673596 ± 2% interrupts.CPU141.LOC:Local_timer_interrupts
7257 ± 4% +16.1% 8423 ± 5% interrupts.CPU141.RES:Rescheduling_interrupts
611411 ± 2% +10.2% 673590 ± 2% interrupts.CPU142.LOC:Local_timer_interrupts
7300 ± 6% +16.1% 8475 ± 3% interrupts.CPU142.RES:Rescheduling_interrupts
611396 ± 2% +10.2% 673622 ± 2% interrupts.CPU143.LOC:Local_timer_interrupts
7399 ± 5% +14.0% 8435 ± 4% interrupts.CPU143.RES:Rescheduling_interrupts
610929 ± 2% +10.3% 673686 ± 2% interrupts.CPU144.LOC:Local_timer_interrupts
7281 ± 5% +13.4% 8257 ± 5% interrupts.CPU144.RES:Rescheduling_interrupts
610918 ± 2% +10.3% 673755 ± 2% interrupts.CPU145.LOC:Local_timer_interrupts
7343 ± 5% +14.5% 8408 ± 4% interrupts.CPU145.RES:Rescheduling_interrupts
610910 ± 2% +10.3% 673705 ± 2% interrupts.CPU146.LOC:Local_timer_interrupts
7524 ± 4% +13.6% 8550 ± 6% interrupts.CPU146.RES:Rescheduling_interrupts
610858 ± 2% +10.3% 673769 ± 2% interrupts.CPU147.LOC:Local_timer_interrupts
7433 ± 4% +10.8% 8237 ± 3% interrupts.CPU147.RES:Rescheduling_interrupts
610872 ± 2% +10.3% 673727 ± 2% interrupts.CPU148.LOC:Local_timer_interrupts
7354 ± 3% +13.1% 8320 ± 4% interrupts.CPU148.RES:Rescheduling_interrupts
610909 ± 2% +10.3% 673750 ± 2% interrupts.CPU149.LOC:Local_timer_interrupts
7365 ± 3% +16.5% 8582 ± 3% interrupts.CPU149.RES:Rescheduling_interrupts
611818 ± 2% +10.2% 673937 ± 2% interrupts.CPU15.LOC:Local_timer_interrupts
7632 ± 4% +12.5% 8587 ± 4% interrupts.CPU15.RES:Rescheduling_interrupts
610898 ± 2% +10.3% 673770 ± 2% interrupts.CPU150.LOC:Local_timer_interrupts
7369 ± 4% +12.4% 8282 ± 3% interrupts.CPU150.RES:Rescheduling_interrupts
610887 ± 2% +10.3% 673754 ± 2% interrupts.CPU151.LOC:Local_timer_interrupts
7322 ± 4% +15.3% 8445 ± 3% interrupts.CPU151.RES:Rescheduling_interrupts
610849 ± 2% +10.3% 673762 ± 2% interrupts.CPU152.LOC:Local_timer_interrupts
7305 ± 4% +15.5% 8439 ± 5% interrupts.CPU152.RES:Rescheduling_interrupts
610909 ± 2% +10.3% 673759 ± 2% interrupts.CPU153.LOC:Local_timer_interrupts
7363 ± 6% +14.5% 8432 ± 2% interrupts.CPU153.RES:Rescheduling_interrupts
610878 ± 2% +10.3% 673730 ± 2% interrupts.CPU154.LOC:Local_timer_interrupts
7373 ± 4% +14.1% 8414 ± 4% interrupts.CPU154.RES:Rescheduling_interrupts
610891 ± 2% +10.3% 673720 ± 2% interrupts.CPU155.LOC:Local_timer_interrupts
7385 ± 5% +11.8% 8260 ± 4% interrupts.CPU155.RES:Rescheduling_interrupts
610909 ± 2% +10.3% 673753 ± 2% interrupts.CPU156.LOC:Local_timer_interrupts
7377 ± 5% +14.0% 8410 ± 5% interrupts.CPU156.RES:Rescheduling_interrupts
610886 ± 2% +10.3% 673754 ± 2% interrupts.CPU157.LOC:Local_timer_interrupts
7399 ± 5% +13.1% 8370 ± 4% interrupts.CPU157.RES:Rescheduling_interrupts
610854 ± 2% +10.3% 673699 ± 2% interrupts.CPU158.LOC:Local_timer_interrupts
7489 ± 4% +10.5% 8278 ± 2% interrupts.CPU158.RES:Rescheduling_interrupts
610921 ± 2% +10.3% 673726 ± 2% interrupts.CPU159.LOC:Local_timer_interrupts
7281 ± 5% +15.4% 8402 ± 4% interrupts.CPU159.RES:Rescheduling_interrupts
611836 ± 2% +10.1% 673889 ± 2% interrupts.CPU16.LOC:Local_timer_interrupts
7548 ± 4% +15.6% 8723 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts
610870 ± 2% +10.3% 673742 ± 2% interrupts.CPU160.LOC:Local_timer_interrupts
610856 ± 2% +10.3% 673722 ± 2% interrupts.CPU161.LOC:Local_timer_interrupts
7472 ± 5% +13.4% 8475 ± 5% interrupts.CPU161.RES:Rescheduling_interrupts
610891 ± 2% +10.3% 673708 ± 2% interrupts.CPU162.LOC:Local_timer_interrupts
7177 ± 5% +14.8% 8239 ± 5% interrupts.CPU162.RES:Rescheduling_interrupts
610864 ± 2% +10.3% 673696 ± 2% interrupts.CPU163.LOC:Local_timer_interrupts
7337 ± 5% +12.7% 8266 ± 4% interrupts.CPU163.RES:Rescheduling_interrupts
610858 ± 2% +10.3% 673724 ± 2% interrupts.CPU164.LOC:Local_timer_interrupts
7269 ± 5% +15.2% 8376 ± 4% interrupts.CPU164.RES:Rescheduling_interrupts
610861 ± 2% +10.3% 673711 ± 2% interrupts.CPU165.LOC:Local_timer_interrupts
7431 ± 4% +9.4% 8131 ± 4% interrupts.CPU165.RES:Rescheduling_interrupts
610867 ± 2% +10.3% 673684 ± 2% interrupts.CPU166.LOC:Local_timer_interrupts
7339 ± 2% +14.1% 8374 ± 4% interrupts.CPU166.RES:Rescheduling_interrupts
610913 ± 2% +10.3% 673730 ± 2% interrupts.CPU167.LOC:Local_timer_interrupts
7174 ± 3% +17.5% 8427 ± 4% interrupts.CPU167.RES:Rescheduling_interrupts
611289 ± 2% +10.2% 673888 ± 2% interrupts.CPU168.LOC:Local_timer_interrupts
7311 ± 5% +14.7% 8382 ± 3% interrupts.CPU168.RES:Rescheduling_interrupts
611333 ± 2% +10.2% 673887 ± 2% interrupts.CPU169.LOC:Local_timer_interrupts
7232 ± 2% +17.8% 8517 ± 5% interrupts.CPU169.RES:Rescheduling_interrupts
611838 ± 2% +10.1% 673879 ± 2% interrupts.CPU17.LOC:Local_timer_interrupts
7539 ± 2% +17.7% 8873 ± 10% interrupts.CPU17.RES:Rescheduling_interrupts
611349 ± 2% +10.2% 673889 ± 2% interrupts.CPU170.LOC:Local_timer_interrupts
7374 ± 2% +14.2% 8424 ± 5% interrupts.CPU170.RES:Rescheduling_interrupts
611297 ± 2% +10.2% 673873 ± 2% interrupts.CPU171.LOC:Local_timer_interrupts
7438 ± 4% +13.9% 8471 ± 5% interrupts.CPU171.RES:Rescheduling_interrupts
611304 ± 2% +10.2% 673845 ± 2% interrupts.CPU172.LOC:Local_timer_interrupts
7317 ± 3% +16.6% 8529 ± 5% interrupts.CPU172.RES:Rescheduling_interrupts
611297 ± 2% +10.2% 673887 ± 2% interrupts.CPU173.LOC:Local_timer_interrupts
611309 ± 2% +10.2% 673858 ± 2% interrupts.CPU174.LOC:Local_timer_interrupts
7369 ± 5% +15.8% 8531 ± 5% interrupts.CPU174.RES:Rescheduling_interrupts
611311 ± 2% +10.2% 673872 ± 2% interrupts.CPU175.LOC:Local_timer_interrupts
7352 ± 4% +15.1% 8465 ± 4% interrupts.CPU175.RES:Rescheduling_interrupts
611309 ± 2% +10.2% 673886 ± 2% interrupts.CPU176.LOC:Local_timer_interrupts
7395 ± 2% +14.0% 8434 ± 5% interrupts.CPU176.RES:Rescheduling_interrupts
611322 ± 2% +10.2% 673868 ± 2% interrupts.CPU177.LOC:Local_timer_interrupts
611248 ± 2% +10.2% 673805 ± 2% interrupts.CPU178.LOC:Local_timer_interrupts
7355 ± 5% +14.1% 8393 ± 4% interrupts.CPU178.RES:Rescheduling_interrupts
611319 ± 2% +10.2% 673906 ± 2% interrupts.CPU179.LOC:Local_timer_interrupts
7269 ± 4% +15.2% 8374 ± 6% interrupts.CPU179.RES:Rescheduling_interrupts
611799 ± 2% +10.1% 673864 ± 2% interrupts.CPU18.LOC:Local_timer_interrupts
7585 ± 4% +14.5% 8683 ± 4% interrupts.CPU18.RES:Rescheduling_interrupts
611330 ± 2% +10.2% 673869 ± 2% interrupts.CPU180.LOC:Local_timer_interrupts
7404 ± 4% +14.0% 8442 ± 5% interrupts.CPU180.RES:Rescheduling_interrupts
611324 ± 2% +10.2% 673889 ± 2% interrupts.CPU181.LOC:Local_timer_interrupts
7435 ± 2% +13.0% 8399 ± 3% interrupts.CPU181.RES:Rescheduling_interrupts
611322 ± 2% +10.2% 673874 ± 2% interrupts.CPU182.LOC:Local_timer_interrupts
7250 ± 4% +14.6% 8305 ± 4% interrupts.CPU182.RES:Rescheduling_interrupts
611307 ± 2% +10.2% 673852 ± 2% interrupts.CPU183.LOC:Local_timer_interrupts
7320 ± 4% +13.2% 8290 ± 5% interrupts.CPU183.RES:Rescheduling_interrupts
611329 ± 2% +10.2% 673877 ± 2% interrupts.CPU184.LOC:Local_timer_interrupts
7377 ± 4% +12.5% 8299 ± 5% interrupts.CPU184.RES:Rescheduling_interrupts
611312 ± 2% +10.2% 673866 ± 2% interrupts.CPU185.LOC:Local_timer_interrupts
7333 ± 4% +13.9% 8351 ± 5% interrupts.CPU185.RES:Rescheduling_interrupts
611290 ± 2% +10.2% 673868 ± 2% interrupts.CPU186.LOC:Local_timer_interrupts
7426 ± 4% +14.1% 8471 ± 3% interrupts.CPU186.RES:Rescheduling_interrupts
611304 ± 2% +10.2% 673853 ± 2% interrupts.CPU187.LOC:Local_timer_interrupts
7380 ± 3% +12.5% 8304 ± 5% interrupts.CPU187.RES:Rescheduling_interrupts
611307 ± 2% +10.2% 673881 ± 2% interrupts.CPU188.LOC:Local_timer_interrupts
7277 ± 5% +16.3% 8467 ± 6% interrupts.CPU188.RES:Rescheduling_interrupts
611299 ± 2% +10.2% 673879 ± 2% interrupts.CPU189.LOC:Local_timer_interrupts
7436 ± 3% +14.5% 8514 ± 6% interrupts.CPU189.RES:Rescheduling_interrupts
611761 ± 2% +10.2% 673928 ± 2% interrupts.CPU19.LOC:Local_timer_interrupts
7515 ± 6% +12.7% 8467 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts
611321 ± 2% +10.2% 673856 ± 2% interrupts.CPU190.LOC:Local_timer_interrupts
7188 ± 3% +16.4% 8366 ± 5% interrupts.CPU190.RES:Rescheduling_interrupts
611544 ± 2% +10.2% 674120 ± 2% interrupts.CPU191.LOC:Local_timer_interrupts
7207 ± 3% +15.3% 8312 ± 3% interrupts.CPU191.RES:Rescheduling_interrupts
611805 ± 2% +10.2% 673920 ± 2% interrupts.CPU2.LOC:Local_timer_interrupts
611781 ± 2% +10.2% 673897 ± 2% interrupts.CPU20.LOC:Local_timer_interrupts
7531 ± 5% +16.4% 8768 ± 3% interrupts.CPU20.RES:Rescheduling_interrupts
611715 ± 2% +10.2% 673897 ± 2% interrupts.CPU21.LOC:Local_timer_interrupts
7467 ± 3% +14.9% 8577 ± 4% interrupts.CPU21.RES:Rescheduling_interrupts
611834 ± 2% +10.1% 673900 ± 2% interrupts.CPU22.LOC:Local_timer_interrupts
7427 ± 6% +17.9% 8754 ± 4% interrupts.CPU22.RES:Rescheduling_interrupts
611788 ± 2% +10.1% 673869 ± 2% interrupts.CPU23.LOC:Local_timer_interrupts
7592 ± 4% +12.2% 8518 ± 4% interrupts.CPU23.RES:Rescheduling_interrupts
611426 ± 2% +10.2% 673619 ± 2% interrupts.CPU24.LOC:Local_timer_interrupts
7457 ± 4% +14.7% 8551 ± 3% interrupts.CPU24.RES:Rescheduling_interrupts
611439 ± 2% +10.2% 673636 ± 2% interrupts.CPU25.LOC:Local_timer_interrupts
611402 ± 2% +10.2% 673620 ± 2% interrupts.CPU26.LOC:Local_timer_interrupts
7463 ± 3% +18.4% 8838 ± 7% interrupts.CPU26.RES:Rescheduling_interrupts
611413 ± 2% +10.2% 673615 ± 2% interrupts.CPU27.LOC:Local_timer_interrupts
7412 ± 4% +16.1% 8607 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts
611409 ± 2% +10.2% 673625 ± 2% interrupts.CPU28.LOC:Local_timer_interrupts
7478 ± 5% +15.6% 8648 ± 3% interrupts.CPU28.RES:Rescheduling_interrupts
611415 ± 2% +10.2% 673620 ± 2% interrupts.CPU29.LOC:Local_timer_interrupts
7431 ± 2% +15.5% 8585 ± 2% interrupts.CPU29.RES:Rescheduling_interrupts
611807 ± 2% +10.1% 673896 ± 2% interrupts.CPU3.LOC:Local_timer_interrupts
7659 ± 5% +20.9% 9264 ± 10% interrupts.CPU3.RES:Rescheduling_interrupts
611433 ± 2% +10.2% 673520 ± 2% interrupts.CPU30.LOC:Local_timer_interrupts
7432 ± 3% +16.5% 8659 ± 4% interrupts.CPU30.RES:Rescheduling_interrupts
611450 ± 2% +10.2% 673604 ± 2% interrupts.CPU31.LOC:Local_timer_interrupts
7490 ± 4% +14.6% 8585 ± 5% interrupts.CPU31.RES:Rescheduling_interrupts
611444 ± 2% +10.2% 673588 ± 2% interrupts.CPU32.LOC:Local_timer_interrupts
7393 ± 4% +19.0% 8799 ± 3% interrupts.CPU32.RES:Rescheduling_interrupts
611407 ± 2% +10.2% 673606 ± 2% interrupts.CPU33.LOC:Local_timer_interrupts
7398 ± 4% +16.8% 8644 ± 4% interrupts.CPU33.RES:Rescheduling_interrupts
611452 ± 2% +10.2% 673609 ± 2% interrupts.CPU34.LOC:Local_timer_interrupts
7414 ± 5% +18.8% 8806 ± 4% interrupts.CPU34.RES:Rescheduling_interrupts
611433 ± 2% +10.2% 673588 ± 2% interrupts.CPU35.LOC:Local_timer_interrupts
7486 ± 5% +18.6% 8878 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts
611450 ± 2% +10.2% 673603 ± 2% interrupts.CPU36.LOC:Local_timer_interrupts
7503 ± 3% +15.0% 8627 ± 3% interrupts.CPU36.RES:Rescheduling_interrupts
611465 ± 2% +10.2% 673611 ± 2% interrupts.CPU37.LOC:Local_timer_interrupts
7416 ± 3% +17.6% 8719 ± 2% interrupts.CPU37.RES:Rescheduling_interrupts
611445 ± 2% +10.2% 673575 ± 2% interrupts.CPU38.LOC:Local_timer_interrupts
7477 ± 5% +14.2% 8540 ± 3% interrupts.CPU38.RES:Rescheduling_interrupts
611420 ± 2% +10.2% 673600 ± 2% interrupts.CPU39.LOC:Local_timer_interrupts
7562 ± 5% +14.0% 8620 ± 4% interrupts.CPU39.RES:Rescheduling_interrupts
611823 ± 2% +10.2% 673949 ± 2% interrupts.CPU4.LOC:Local_timer_interrupts
7579 ± 4% +14.9% 8708 ± 3% interrupts.CPU4.RES:Rescheduling_interrupts
611390 ± 2% +10.2% 673618 ± 2% interrupts.CPU40.LOC:Local_timer_interrupts
7444 ± 4% +15.9% 8626 ± 3% interrupts.CPU40.RES:Rescheduling_interrupts
611429 ± 2% +10.2% 673612 ± 2% interrupts.CPU41.LOC:Local_timer_interrupts
7464 ± 5% +15.3% 8605 ± 2% interrupts.CPU41.RES:Rescheduling_interrupts
611418 ± 2% +10.2% 673597 ± 2% interrupts.CPU42.LOC:Local_timer_interrupts
7394 ± 4% +18.6% 8772 ± 5% interrupts.CPU42.RES:Rescheduling_interrupts
611364 ± 2% +10.2% 673601 ± 2% interrupts.CPU43.LOC:Local_timer_interrupts
7483 ± 5% +15.3% 8630 ± 3% interrupts.CPU43.RES:Rescheduling_interrupts
611421 ± 2% +10.2% 673593 ± 2% interrupts.CPU44.LOC:Local_timer_interrupts
7463 ± 2% +16.0% 8656 ± 4% interrupts.CPU44.RES:Rescheduling_interrupts
611423 ± 2% +10.2% 673593 ± 2% interrupts.CPU45.LOC:Local_timer_interrupts
7455 ± 4% +14.8% 8558 ± 3% interrupts.CPU45.RES:Rescheduling_interrupts
611333 ± 2% +10.2% 673616 ± 2% interrupts.CPU46.LOC:Local_timer_interrupts
7406 ± 3% +17.4% 8696 ± 3% interrupts.CPU46.RES:Rescheduling_interrupts
611421 ± 2% +10.2% 673605 ± 2% interrupts.CPU47.LOC:Local_timer_interrupts
7482 ± 5% +15.7% 8660 ± 2% interrupts.CPU47.RES:Rescheduling_interrupts
610910 ± 2% +10.3% 673737 ± 2% interrupts.CPU48.LOC:Local_timer_interrupts
7521 ± 6% +15.1% 8655 ± 4% interrupts.CPU48.RES:Rescheduling_interrupts
610921 ± 2% +10.3% 673770 ± 2% interrupts.CPU49.LOC:Local_timer_interrupts
7594 ± 5% +12.7% 8558 ± 3% interrupts.CPU49.RES:Rescheduling_interrupts
611796 ± 2% +10.1% 673831 ± 2% interrupts.CPU5.LOC:Local_timer_interrupts
610930 ± 2% +10.3% 673671 ± 2% interrupts.CPU50.LOC:Local_timer_interrupts
7504 ± 2% +17.6% 8824 ± 4% interrupts.CPU50.RES:Rescheduling_interrupts
610882 ± 2% +10.3% 673738 ± 2% interrupts.CPU51.LOC:Local_timer_interrupts
610884 ± 2% +10.3% 673731 ± 2% interrupts.CPU52.LOC:Local_timer_interrupts
7436 ± 4% +18.1% 8783 ± 3% interrupts.CPU52.RES:Rescheduling_interrupts
610885 ± 2% +10.3% 673774 ± 2% interrupts.CPU53.LOC:Local_timer_interrupts
7544 ± 4% +15.3% 8700 ± 4% interrupts.CPU53.RES:Rescheduling_interrupts
610902 ± 2% +10.3% 673763 ± 2% interrupts.CPU54.LOC:Local_timer_interrupts
7534 ± 4% +15.6% 8709 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts
610927 ± 2% +10.3% 673739 ± 2% interrupts.CPU55.LOC:Local_timer_interrupts
610883 ± 2% +10.3% 673719 ± 2% interrupts.CPU56.LOC:Local_timer_interrupts
610921 ± 2% +10.3% 673689 ± 2% interrupts.CPU57.LOC:Local_timer_interrupts
7473 ± 5% +14.8% 8577 ± 4% interrupts.CPU57.RES:Rescheduling_interrupts
610893 ± 2% +10.3% 673781 ± 2% interrupts.CPU58.LOC:Local_timer_interrupts
7407 ± 5% +16.6% 8635 ± 3% interrupts.CPU58.RES:Rescheduling_interrupts
610898 ± 2% +10.3% 673763 ± 2% interrupts.CPU59.LOC:Local_timer_interrupts
7642 ± 5% +13.1% 8643 ± 3% interrupts.CPU59.RES:Rescheduling_interrupts
611823 ± 2% +10.1% 673914 ± 2% interrupts.CPU6.LOC:Local_timer_interrupts
7542 ± 3% +17.3% 8845 ± 3% interrupts.CPU6.RES:Rescheduling_interrupts
610869 ± 2% +10.3% 673784 ± 2% interrupts.CPU60.LOC:Local_timer_interrupts
7535 ± 4% +14.8% 8647 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts
610906 ± 2% +10.3% 673701 ± 2% interrupts.CPU61.LOC:Local_timer_interrupts
7631 ± 4% +13.5% 8660 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts
610863 ± 2% +10.3% 673705 ± 2% interrupts.CPU62.LOC:Local_timer_interrupts
7634 ± 4% +11.8% 8532 ± 4% interrupts.CPU62.RES:Rescheduling_interrupts
610920 ± 2% +10.3% 673730 ± 2% interrupts.CPU63.LOC:Local_timer_interrupts
7549 ± 5% +14.8% 8663 ± 5% interrupts.CPU63.RES:Rescheduling_interrupts
610884 ± 2% +10.3% 673727 ± 2% interrupts.CPU64.LOC:Local_timer_interrupts
7644 ± 3% +13.4% 8672 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts
610853 ± 2% +10.3% 673708 ± 2% interrupts.CPU65.LOC:Local_timer_interrupts
7555 ± 5% +14.8% 8672 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts
610950 ± 2% +10.3% 673804 ± 2% interrupts.CPU66.LOC:Local_timer_interrupts
7457 ± 5% +16.8% 8710 ± 4% interrupts.CPU66.RES:Rescheduling_interrupts
610888 ± 2% +10.3% 673735 ± 2% interrupts.CPU67.LOC:Local_timer_interrupts
7550 ± 4% +11.5% 8415 ± 5% interrupts.CPU67.RES:Rescheduling_interrupts
610891 ± 2% +10.3% 673727 ± 2% interrupts.CPU68.LOC:Local_timer_interrupts
7612 ± 4% +12.1% 8534 ± 3% interrupts.CPU68.RES:Rescheduling_interrupts
610901 ± 2% +10.3% 673659 ± 2% interrupts.CPU69.LOC:Local_timer_interrupts
7493 ± 4% +14.8% 8601 ± 4% interrupts.CPU69.RES:Rescheduling_interrupts
611808 ± 2% +10.1% 673844 ± 2% interrupts.CPU7.LOC:Local_timer_interrupts
7673 ± 5% +15.0% 8821 ± 4% interrupts.CPU7.RES:Rescheduling_interrupts
610906 ± 2% +10.3% 673733 ± 2% interrupts.CPU70.LOC:Local_timer_interrupts
7412 ± 3% +16.9% 8664 ± 4% interrupts.CPU70.RES:Rescheduling_interrupts
610871 ± 2% +10.3% 673671 ± 2% interrupts.CPU71.LOC:Local_timer_interrupts
7519 ± 4% +13.8% 8557 ± 5% interrupts.CPU71.RES:Rescheduling_interrupts
611325 ± 2% +10.2% 673906 ± 2% interrupts.CPU72.LOC:Local_timer_interrupts
7720 ± 4% +9.6% 8459 ± 4% interrupts.CPU72.RES:Rescheduling_interrupts
611328 ± 2% +10.2% 673908 ± 2% interrupts.CPU73.LOC:Local_timer_interrupts
7535 ± 4% +14.6% 8634 ± 4% interrupts.CPU73.RES:Rescheduling_interrupts
611311 ± 2% +10.2% 673883 ± 2% interrupts.CPU74.LOC:Local_timer_interrupts
7526 ± 5% +14.7% 8635 ± 6% interrupts.CPU74.RES:Rescheduling_interrupts
611315 ± 2% +10.2% 673868 ± 2% interrupts.CPU75.LOC:Local_timer_interrupts
7712 ± 5% +9.8% 8469 ± 4% interrupts.CPU75.RES:Rescheduling_interrupts
611304 ± 2% +10.2% 673860 ± 2% interrupts.CPU76.LOC:Local_timer_interrupts
7476 ± 5% +14.2% 8537 ± 5% interrupts.CPU76.RES:Rescheduling_interrupts
611322 ± 2% +10.2% 673834 ± 2% interrupts.CPU77.LOC:Local_timer_interrupts
7670 ± 3% +12.5% 8630 ± 5% interrupts.CPU77.RES:Rescheduling_interrupts
611326 ± 2% +10.2% 673851 ± 2% interrupts.CPU78.LOC:Local_timer_interrupts
7417 ± 3% +15.6% 8578 ± 4% interrupts.CPU78.RES:Rescheduling_interrupts
611328 ± 2% +10.2% 673820 ± 2% interrupts.CPU79.LOC:Local_timer_interrupts
7545 ± 4% +13.9% 8590 ± 4% interrupts.CPU79.RES:Rescheduling_interrupts
611842 ± 2% +10.1% 673928 ± 2% interrupts.CPU8.LOC:Local_timer_interrupts
7673 ± 3% +14.9% 8818 ± 4% interrupts.CPU8.RES:Rescheduling_interrupts
611366 ± 2% +10.2% 673868 ± 2% interrupts.CPU80.LOC:Local_timer_interrupts
7513 ± 3% +16.1% 8723 ± 5% interrupts.CPU80.RES:Rescheduling_interrupts
611322 ± 2% +10.2% 673860 ± 2% interrupts.CPU81.LOC:Local_timer_interrupts
611299 ± 2% +10.2% 673868 ± 2% interrupts.CPU82.LOC:Local_timer_interrupts
7523 ± 4% +13.8% 8562 ± 4% interrupts.CPU82.RES:Rescheduling_interrupts
611320 ± 2% +10.2% 673727 ± 2% interrupts.CPU83.LOC:Local_timer_interrupts
7518 ± 5% +12.4% 8454 ± 5% interrupts.CPU83.RES:Rescheduling_interrupts
611326 ± 2% +10.2% 673899 ± 2% interrupts.CPU84.LOC:Local_timer_interrupts
7452 ± 6% +12.3% 8365 ± 5% interrupts.CPU84.RES:Rescheduling_interrupts
611329 ± 2% +10.2% 673885 ± 2% interrupts.CPU85.LOC:Local_timer_interrupts
7399 ± 5% +11.1% 8218 ± 5% interrupts.CPU85.RES:Rescheduling_interrupts
611278 ± 2% +10.2% 673875 ± 2% interrupts.CPU86.LOC:Local_timer_interrupts
7376 ± 4% +16.3% 8581 ± 3% interrupts.CPU86.RES:Rescheduling_interrupts
611313 ± 2% +10.2% 673874 ± 2% interrupts.CPU87.LOC:Local_timer_interrupts
7544 ± 4% +12.8% 8508 ± 4% interrupts.CPU87.RES:Rescheduling_interrupts
611351 ± 2% +10.2% 673928 ± 2% interrupts.CPU88.LOC:Local_timer_interrupts
7418 ± 6% +13.3% 8406 ± 4% interrupts.CPU88.RES:Rescheduling_interrupts
611324 ± 2% +10.2% 673795 ± 2% interrupts.CPU89.LOC:Local_timer_interrupts
7426 ± 4% +13.6% 8439 ± 5% interrupts.CPU89.RES:Rescheduling_interrupts
611832 ± 2% +10.2% 673968 ± 2% interrupts.CPU9.LOC:Local_timer_interrupts
7538 ± 3% +15.6% 8715 ± 4% interrupts.CPU9.RES:Rescheduling_interrupts
611298 ± 2% +10.2% 673878 ± 2% interrupts.CPU90.LOC:Local_timer_interrupts
7421 ± 3% +12.7% 8363 ± 4% interrupts.CPU90.RES:Rescheduling_interrupts
611305 ± 2% +10.2% 673874 ± 2% interrupts.CPU91.LOC:Local_timer_interrupts
7377 ± 5% +15.6% 8528 ± 5% interrupts.CPU91.RES:Rescheduling_interrupts
611288 ± 2% +10.2% 673876 ± 2% interrupts.CPU92.LOC:Local_timer_interrupts
7349 ± 4% +12.0% 8234 ± 4% interrupts.CPU92.RES:Rescheduling_interrupts
611329 ± 2% +10.2% 673857 ± 2% interrupts.CPU93.LOC:Local_timer_interrupts
7438 ± 3% +14.7% 8530 ± 4% interrupts.CPU93.RES:Rescheduling_interrupts
611276 ± 2% +10.2% 673866 ± 2% interrupts.CPU94.LOC:Local_timer_interrupts
7363 ± 4% +12.7% 8300 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts
611381 ± 2% +10.2% 673922 ± 2% interrupts.CPU95.LOC:Local_timer_interrupts
10549 ± 3% +14.9% 12117 ± 4% interrupts.CPU95.RES:Rescheduling_interrupts
611824 ± 2% +10.1% 673847 ± 2% interrupts.CPU96.LOC:Local_timer_interrupts
7416 ± 3% +12.9% 8375 ± 5% interrupts.CPU96.RES:Rescheduling_interrupts
611751 ± 2% +10.2% 673870 ± 2% interrupts.CPU97.LOC:Local_timer_interrupts
7437 ± 4% +14.0% 8477 ± 4% interrupts.CPU97.RES:Rescheduling_interrupts
611787 ± 2% +10.1% 673877 ± 2% interrupts.CPU98.LOC:Local_timer_interrupts
7555 ± 4% +13.0% 8534 ± 3% interrupts.CPU98.RES:Rescheduling_interrupts
611783 ± 2% +10.1% 673872 ± 2% interrupts.CPU99.LOC:Local_timer_interrupts
7452 ± 3% +12.8% 8404 ± 5% interrupts.CPU99.RES:Rescheduling_interrupts
1.174e+08 ± 2% +10.2% 1.294e+08 ± 2% interrupts.LOC:Local_timer_interrupts
1433875 ± 3% +14.6% 1643307 ± 3% interrupts.RES:Rescheduling_interrupts
aim7.jobs-per-min
30500 +-------------------------------------------------------------------+
| ..+.... .|
30000 |-+ ...... ... ... |
29500 |................ ...... .. ... |
| +.. ... .. |
29000 |-+ ... ... |
28500 |-+ . ... |
| +. |
28000 |-+ O O |
27500 |-+ |
| |
27000 |-+ |
26500 |-+ |
| O |
26000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
5 months, 2 weeks
[sched] c3a340f7e7: invalid_opcode:#[##]
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-4.9):
commit: c3a340f7e7eadac7662ab104ceb16432e5a4c6b2 ("sched: Have sched_class_highest define by vmlinux.lds.h")
https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git sched/core
in testcase: trinity
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 16G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------+------------+------------+
| | 590d697963 | c3a340f7e7 |
+------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 4 |
| invalid_opcode:#[##] | 0 | 4 |
| EIP:sched_init | 0 | 4 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 4 |
| kernel_BUG_at_kernel/sched/core.c | 0 | 4 |
+------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 1.813939] ** If you see this message and you are not debugging **
[ 1.814781] ** the kernel, report this immediately to your vendor! **
[ 1.815661] ** **
[ 1.816509] ** NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE NOTICE **
[ 1.817362] **********************************************************
[ 1.820914] invalid opcode: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 1.821634] CPU: 0 PID: 0 Comm: swapper Not tainted 5.8.0-rc1-00024-gc3a340f7e7ead #2
[ 1.822630] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 1.823710] EIP: sched_init+0x3d/0x2fb
[ 1.824191] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1
[ 1.826607] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001
[ 1.827434] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78
[ 1.828241] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002
[ 1.829112] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690
[ 1.829919] Call Trace:
[ 1.830240] ? start_kernel+0x43f/0x677
[ 1.830738] i386_start_kernel+0x48/0x4a
[ 1.831261] startup_32_smp+0x15f/0x164
[ 1.831757] Modules linked in:
[ 1.832165] ---[ end trace db9d28bea02c9e7f ]---
[ 1.832761] EIP: sched_init+0x3d/0x2fb
[ 1.833243] Code: 20 a6 e5 c1 3d 40 a6 e5 c1 0f 95 c0 08 c2 75 1a b8 a0 a6 e5 c1 3d c0 a6 e5 c1 75 0e b8 20 a7 e5 c1 3d 40 a7 e5 c1 74 04 0f 0b <0f> 0b 55 89 e5 57 56 53 83 ec 08 e8 67 04 00 00 8b 3d 80 ad ec c1
[ 1.835656] EAX: c1e5a601 EBX: c1dac484 ECX: 00000000 EDX: 00000001
[ 1.836463] ESI: 00000000 EDI: 000003f8 EBP: c1e93f9c ESP: c1e93f78
[ 1.838603] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0068 EFLAGS: 00210002
[ 1.839496] CR0: 80050033 CR2: ffc70000 CR3: 021ce000 CR4: 00000690
[ 1.840300] Kernel panic - not syncing: Fatal exception
[ 1.840969] ------------[ cut here ]------------
[ 1.840970] kernel BUG at kernel/sched/core.c:6652!
Elapsed time: 60
qemu-img create -f qcow2 disk-vm-snb-94-0 256G
qemu-img create -f qcow2 disk-vm-snb-94-1 256G
To reproduce:
# build kernel
cd linux
cp config-5.8.0-rc1-00024-gc3a340f7e7ead .config
make HOSTCC=gcc-4.9 CC=gcc-4.9 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
5 months, 3 weeks
[hugetlbfs] c0d0381ade: vm-scalability.throughput -33.4% regression
by kernel test robot
Greeting,
FYI, we noticed a -33.4% regression of vm-scalability.throughput due to commit:
commit: c0d0381ade79885c04a04c303284b040616b116e ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: vm-scalability
on test machine: 288 threads Intel(R) Xeon Phi(TM) CPU 7295 @ 1.50GHz with 80G memory
with following parameters:
runtime: 300s
size: 8T
test: anon-cow-seq-hugetlb
cpufreq_governor: performance
ucode: 0x11
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-knm01/anon-cow-seq-hugetlb/vm-scalability/0x11
commit:
49aef7175c ("mm/memblock.c: remove redundant assignment to variable max_addr")
c0d0381ade ("hugetlbfs: use i_mmap_rwsem for more pmd sharing synchronization")
49aef7175cc6eb70 c0d0381ade79885c04a04c30328
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:4 25% 1:4 dmesg.WARNING:at_ip__mutex_lock/0x
:4 25% 1:4 dmesg.WARNING:at_ip_perf_event_mmap_output/0x
%stddev %change %stddev
\ | \
37643 -29.5% 26549 ± 4% vm-scalability.median
9.18 ± 16% +9.3 18.48 ± 21% vm-scalability.median_stddev%
12666048 -33.4% 8429873 ± 4% vm-scalability.throughput
1701516 -19.5% 1369683 ± 5% vm-scalability.time.minor_page_faults
13447 ± 3% -19.3% 10848 vm-scalability.time.percent_of_cpu_this_job_got
26620 ± 4% -10.5% 23830 vm-scalability.time.system_time
14297 ± 3% -35.3% 9255 ± 4% vm-scalability.time.user_time
125377 ± 2% +621.3% 904348 ± 4% vm-scalability.time.voluntary_context_switches
2.468e+09 -19.6% 1.984e+09 ± 5% vm-scalability.workload
3.807e+10 ± 4% +20.9% 4.605e+10 ± 2% cpuidle.C1.time
9524 ± 10% -17.3% 7873 ± 4% numa-vmstat.node0.nr_mapped
2494252 -12.4% 2184278 ± 3% numa-numastat.node0.local_node
2494194 -12.4% 2184223 ± 3% numa-numastat.node0.numa_hit
49.26 ± 3% +9.4 58.62 mpstat.cpu.all.idle%
32.89 ± 4% -3.2 29.70 mpstat.cpu.all.sys%
17.79 ± 3% -6.2 11.64 ± 4% mpstat.cpu.all.usr%
4856 +47.6% 7169 numa-meminfo.node0.HugePages_Surp
4856 +47.6% 7169 numa-meminfo.node0.HugePages_Total
37725 ± 10% -17.7% 31054 ± 4% numa-meminfo.node0.Mapped
12274212 +38.3% 16981169 numa-meminfo.node0.MemUsed
49.00 ± 3% +18.9% 58.25 vmstat.cpu.id
17.00 ± 4% -33.8% 11.25 ± 3% vmstat.cpu.us
138.25 ± 2% -19.5% 111.25 ± 2% vmstat.procs.r
5836 ± 2% +88.0% 10969 ± 3% vmstat.system.cs
4855 +48.0% 7186 meminfo.HugePages_Surp
4856 +48.0% 7186 meminfo.HugePages_Total
9946017 +48.0% 14718559 meminfo.Hugetlb
48593 ± 8% -13.9% 41859 ± 3% meminfo.Mapped
12877361 +36.8% 17619364 meminfo.Memused
81607 ± 2% +29.1% 105324 meminfo.max_used_kB
2890 ±100% +285.4% 11139 ± 3% perf-stat.i.context-switches
298.31 ±100% +243.7% 1025 ± 2% perf-stat.i.cpu-migrations
0.11 ±100% +119.3% 0.24 perf-stat.i.ipc
0.04 ±100% +129.7% 0.10 ± 4% perf-stat.i.metric.K/sec
0.11 ±100% +122.1% 0.24 perf-stat.overall.ipc
3107 ±100% +137.3% 7372 ± 3% perf-stat.overall.path-length
2836 ±100% +284.4% 10902 ± 3% perf-stat.ps.context-switches
275.01 ±100% +257.9% 984.31 ± 2% perf-stat.ps.cpu-migrations
111986 -11.7% 98924 ± 2% slabinfo.anon_vma.active_objs
2434 -11.7% 2150 ± 2% slabinfo.anon_vma.active_slabs
111986 -11.7% 98924 ± 2% slabinfo.anon_vma.num_objs
2434 -11.7% 2150 ± 2% slabinfo.anon_vma.num_slabs
258615 -9.1% 235038 ± 2% slabinfo.anon_vma_chain.active_objs
4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.active_slabs
258670 -9.1% 235049 ± 2% slabinfo.anon_vma_chain.num_objs
4041 -9.1% 3672 ± 2% slabinfo.anon_vma_chain.num_slabs
1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.active_objs
1500 -17.8% 1232 ± 5% slabinfo.hugetlbfs_inode_cache.num_objs
682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.active_objs
682.00 ± 6% +15.9% 790.50 ± 6% slabinfo.numa_policy.num_objs
1082230 -19.6% 869890 ± 5% proc-vmstat.htlb_buddy_alloc_success
1721553 -6.8% 1604364 proc-vmstat.nr_dirty_background_threshold
3447317 -6.8% 3212652 proc-vmstat.nr_dirty_threshold
17371789 -6.8% 16197198 proc-vmstat.nr_free_pages
351.00 +3.8% 364.25 proc-vmstat.nr_inactive_file
39705 +1.4% 40269 proc-vmstat.nr_kernel_stack
12252 ± 7% -13.7% 10570 ± 3% proc-vmstat.nr_mapped
2632 +8.8% 2863 proc-vmstat.nr_page_table_pages
98880 -1.6% 97341 proc-vmstat.nr_slab_unreclaimable
351.00 +3.8% 364.25 proc-vmstat.nr_zone_inactive_file
6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults
6285 ± 50% -90.8% 577.50 ± 47% proc-vmstat.numa_hint_faults_local
2514799 -12.3% 2205532 ± 3% proc-vmstat.numa_hit
2514798 -12.3% 2205531 ± 3% proc-vmstat.numa_local
5.557e+08 -19.6% 4.469e+08 ± 5% proc-vmstat.pgalloc_normal
2510883 -12.9% 2187221 ± 3% proc-vmstat.pgfault
5.553e+08 -19.5% 4.468e+08 ± 5% proc-vmstat.pgfree
60107 ± 2% -19.7% 48269 ± 17% sched_debug.cfs_rq:/.exec_clock.avg
71511 ± 2% -23.4% 54794 ± 16% sched_debug.cfs_rq:/.exec_clock.max
44464 ± 7% -16.9% 36954 ± 18% sched_debug.cfs_rq:/.exec_clock.min
4619 ± 4% -47.8% 2412 ± 11% sched_debug.cfs_rq:/.exec_clock.stddev
614.00 ± 10% +371.9% 2897 ±124% sched_debug.cfs_rq:/.load_avg.max
49.58 ± 11% +321.1% 208.79 ±123% sched_debug.cfs_rq:/.load_avg.stddev
11526482 ± 4% -37.6% 7189313 ± 17% sched_debug.cfs_rq:/.min_vruntime.avg
13399855 ± 4% -40.0% 8039878 ± 15% sched_debug.cfs_rq:/.min_vruntime.max
8425626 ± 8% -37.2% 5294770 ± 18% sched_debug.cfs_rq:/.min_vruntime.min
788640 ± 4% -56.4% 343805 ± 11% sched_debug.cfs_rq:/.min_vruntime.stddev
0.37 ± 8% -40.0% 0.22 ± 10% sched_debug.cfs_rq:/.nr_running.avg
21.17 ± 5% +134.9% 49.72 ± 21% sched_debug.cfs_rq:/.nr_spread_over.avg
56.15 ± 13% +60.7% 90.25 ± 19% sched_debug.cfs_rq:/.nr_spread_over.max
6.95 ± 7% +289.4% 27.06 ± 23% sched_debug.cfs_rq:/.nr_spread_over.min
7.32 ± 5% +39.4% 10.20 ± 19% sched_debug.cfs_rq:/.nr_spread_over.stddev
413.38 ± 11% -27.5% 299.85 ± 14% sched_debug.cfs_rq:/.runnable_avg.avg
834.44 ± 53% -65.7% 286.48 ± 5% sched_debug.cfs_rq:/.runnable_avg.stddev
2625208 ± 11% -29.7% 1845657 ± 13% sched_debug.cfs_rq:/.spread0.avg
4483653 ± 6% -39.8% 2699533 ± 11% sched_debug.cfs_rq:/.spread0.max
-475138 -94.3% -26940 sched_debug.cfs_rq:/.spread0.min
801419 ± 4% -56.8% 345835 ± 11% sched_debug.cfs_rq:/.spread0.stddev
371.84 ± 8% -22.5% 288.10 ± 14% sched_debug.cfs_rq:/.util_avg.avg
382.64 ± 4% -32.5% 258.09 ± 4% sched_debug.cfs_rq:/.util_avg.stddev
330.75 ± 10% -54.8% 149.57 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.avg
358.21 ± 6% -27.6% 259.25 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
4047967 ± 4% +17.6% 4760305 ± 4% sched_debug.cpu.avg_idle.avg
561715 ± 52% +116.4% 1215479 ± 7% sched_debug.cpu.avg_idle.min
731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock.stddev
731.80 ± 6% -38.6% 449.23 ± 8% sched_debug.cpu.clock_task.stddev
4741 ± 13% -43.0% 2702 ± 18% sched_debug.cpu.curr->pid.avg
5519 ± 7% -15.8% 4650 ± 11% sched_debug.cpu.curr->pid.stddev
2119875 ± 5% +16.3% 2465873 ± 4% sched_debug.cpu.max_idle_balance_cost.avg
1107931 ± 2% +24.6% 1380330 ± 3% sched_debug.cpu.max_idle_balance_cost.min
581556 ± 5% +11.1% 646020 ± 5% sched_debug.cpu.max_idle_balance_cost.stddev
0.00 ± 4% -30.8% 0.00 ± 11% sched_debug.cpu.next_balance.stddev
0.31 ± 12% -32.6% 0.21 ± 10% sched_debug.cpu.nr_running.avg
5116 +46.1% 7477 ± 11% sched_debug.cpu.nr_switches.avg
2301 ± 5% +85.0% 4257 ± 22% sched_debug.cpu.nr_switches.min
0.09 ± 44% +300.2% 0.34 ± 17% sched_debug.cpu.nr_uninterruptible.avg
296.40 ± 3% +24.1% 367.95 ± 8% sched_debug.cpu.nr_uninterruptible.max
-50.40 +38.8% -69.98 sched_debug.cpu.nr_uninterruptible.min
23.04 ± 3% +53.3% 35.32 ± 12% sched_debug.cpu.nr_uninterruptible.stddev
2343 +98.2% 4646 ± 18% sched_debug.cpu.sched_count.avg
1110 ± 2% +193.0% 3252 ± 23% sched_debug.cpu.sched_count.min
913.60 ± 2% +125.6% 2060 ± 18% sched_debug.cpu.sched_goidle.avg
356.95 +296.7% 1416 ± 25% sched_debug.cpu.sched_goidle.min
1148 +102.6% 2326 ± 18% sched_debug.cpu.ttwu_count.avg
510.15 ± 5% +80.8% 922.26 ± 21% sched_debug.cpu.ttwu_count.min
148.25 ± 6% -11.3% 131.52 ± 6% sched_debug.cpu.ttwu_local.stddev
21.96 ± 6% -10.0 11.99 ± 11% perf-profile.calltrace.cycles-pp.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
19.05 ± 5% -8.1 10.96 ± 11% perf-profile.calltrace.cycles-pp.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault
18.72 ± 5% -8.0 10.76 ± 11% perf-profile.calltrace.cycles-pp.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault
17.82 ± 4% -7.6 10.17 ± 10% perf-profile.calltrace.cycles-pp.copy_page.copy_subpage.copy_user_huge_page.hugetlb_cow.hugetlb_fault
2.22 ± 44% -1.5 0.68 ± 23% perf-profile.calltrace.cycles-pp.alloc_huge_page.hugetlb_cow.hugetlb_fault.handle_mm_fault.do_page_fault
1.03 ± 22% -0.6 0.40 ±102% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.do_access
0.91 ± 23% -0.6 0.32 ±103% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.do_access
0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_finish_mmu.exit_mmap.mmput.do_exit.do_group_exit
0.81 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput.do_exit
0.80 ± 19% -0.2 0.60 ± 14% perf-profile.calltrace.cycles-pp.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap.mmput
0.79 ± 19% -0.2 0.59 ± 15% perf-profile.calltrace.cycles-pp.__free_huge_page.release_pages.tlb_flush_mmu.tlb_finish_mmu.exit_mmap
1.44 ± 22% +0.6 2.00 ± 28% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
0.71 ± 80% +1.5 2.16 ± 33% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock
1.03 ± 55% +1.6 2.67 ± 32% perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock
1.24 ± 57% +2.0 3.29 ± 26% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault
1.38 ± 55% +2.3 3.66 ± 25% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault
29.76 ± 21% +20.1 49.83 ± 13% perf-profile.calltrace.cycles-pp.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault.page_fault
15.71 ± 27% +23.2 38.88 ± 18% perf-profile.calltrace.cycles-pp.osq_lock.__mutex_lock.hugetlb_fault.handle_mm_fault.do_page_fault
21.96 ± 6% -10.0 12.00 ± 11% perf-profile.children.cycles-pp.hugetlb_cow
19.36 ± 5% -8.2 11.11 ± 11% perf-profile.children.cycles-pp.copy_user_huge_page
18.74 ± 5% -8.0 10.77 ± 11% perf-profile.children.cycles-pp.copy_subpage
18.31 ± 5% -7.8 10.54 ± 11% perf-profile.children.cycles-pp.copy_page
16.95 ± 20% -7.0 10.00 ± 53% perf-profile.children.cycles-pp.do_rw_once
2.46 ± 41% -1.7 0.80 ± 14% perf-profile.children.cycles-pp.alloc_huge_page
3.11 ± 36% -1.6 1.48 ± 4% perf-profile.children.cycles-pp._raw_spin_lock
2.64 ± 44% -1.5 1.10 ± 6% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
1.60 ± 38% -1.0 0.57 ± 14% perf-profile.children.cycles-pp.alloc_surplus_huge_page
1.81 ± 12% -0.5 1.34 ± 4% perf-profile.children.cycles-pp.do_syscall_64
1.81 ± 12% -0.5 1.35 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.76 ± 28% -0.4 0.38 ± 12% perf-profile.children.cycles-pp.alloc_fresh_huge_page
1.21 ± 10% -0.4 0.85 ± 11% perf-profile.children.cycles-pp.perf_mux_hrtimer_handler
0.40 ± 52% -0.3 0.09 ± 20% perf-profile.children.cycles-pp.prep_new_huge_page
0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_finish_mmu
0.81 ± 19% -0.2 0.60 ± 14% perf-profile.children.cycles-pp.tlb_flush_mmu
0.80 ± 18% -0.2 0.60 ± 15% perf-profile.children.cycles-pp.release_pages
0.79 ± 19% -0.2 0.59 ± 15% perf-profile.children.cycles-pp.__free_huge_page
0.44 ± 13% -0.2 0.26 ± 14% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 14% -0.1 0.06 ± 6% perf-profile.children.cycles-pp.update_and_free_page
0.32 ± 12% -0.1 0.21 ± 6% perf-profile.children.cycles-pp.ksys_read
0.29 ± 13% -0.1 0.18 ± 10% perf-profile.children.cycles-pp.read
0.31 ± 11% -0.1 0.20 ± 8% perf-profile.children.cycles-pp.vfs_read
0.18 ± 17% -0.1 0.09 ± 8% perf-profile.children.cycles-pp._cond_resched
0.16 ± 9% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.__libc_start_main
0.16 ± 11% -0.1 0.08 ± 17% perf-profile.children.cycles-pp.main
0.19 ± 14% -0.1 0.12 ± 17% perf-profile.children.cycles-pp.x86_pmu_disable
0.26 ± 8% -0.1 0.19 ± 6% perf-profile.children.cycles-pp.ksys_write
0.26 ± 9% -0.1 0.19 ± 8% perf-profile.children.cycles-pp.vfs_write
0.15 ± 12% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.new_sync_read
0.15 ± 11% -0.1 0.09 ± 17% perf-profile.children.cycles-pp.pipe_read
0.24 ± 9% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.new_sync_write
0.08 ± 6% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__perf_sw_event
0.11 ± 14% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.__wake_up_common
0.12 ± 3% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.schedule_timeout
0.12 ± 13% -0.0 0.07 ± 17% perf-profile.children.cycles-pp.__wake_up_common_lock
0.10 ± 15% -0.0 0.06 ± 15% perf-profile.children.cycles-pp.autoremove_wake_function
0.11 ± 29% -0.0 0.07 ± 10% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.26 ± 6% -0.0 0.22 ± 5% perf-profile.children.cycles-pp.tick_program_event
0.09 ± 31% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.09 ± 9% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.rcu_gp_kthread
0.08 ± 10% -0.0 0.06 ± 14% perf-profile.children.cycles-pp.cmd_stat
0.01 ±173% +0.0 0.06 ± 11% perf-profile.children.cycles-pp.select_task_rq_fair
0.11 ± 11% +0.0 0.16 ± 9% perf-profile.children.cycles-pp.finish_task_switch
0.00 +0.1 0.06 ± 14% perf-profile.children.cycles-pp.rwsem_wake
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.arch_stack_walk
0.00 +0.1 0.06 ± 13% perf-profile.children.cycles-pp.dequeue_task_fair
0.00 +0.1 0.07 ± 13% perf-profile.children.cycles-pp.stack_trace_save_tsk
0.13 ± 6% +0.1 0.21 ± 10% perf-profile.children.cycles-pp.schedule_idle
0.06 ± 11% +0.1 0.14 ± 10% perf-profile.children.cycles-pp.enqueue_entity
0.03 ±100% +0.1 0.11 ± 7% perf-profile.children.cycles-pp.update_load_avg
0.00 +0.1 0.09 ± 12% perf-profile.children.cycles-pp.__account_scheduler_latency
0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.activate_task
0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.ttwu_do_activate
0.07 ± 17% +0.1 0.16 ± 11% perf-profile.children.cycles-pp.enqueue_task_fair
0.00 +0.1 0.09 ± 8% perf-profile.children.cycles-pp.wake_up_q
0.14 ± 24% +0.1 0.27 ± 13% perf-profile.children.cycles-pp.schedule_preempt_disabled
0.00 +0.1 0.14 ± 15% perf-profile.children.cycles-pp.sched_ttwu_pending
1.17 ± 13% +0.4 1.56 ± 8% perf-profile.children.cycles-pp.load_balance
0.99 ± 11% +0.4 1.41 ± 9% perf-profile.children.cycles-pp.find_busiest_group
0.96 ± 11% +0.4 1.38 ± 9% perf-profile.children.cycles-pp.update_sd_lb_stats
0.66 ± 9% +0.5 1.20 ± 11% perf-profile.children.cycles-pp.newidle_balance
0.53 ± 7% +0.6 1.11 ± 12% perf-profile.children.cycles-pp.pick_next_task_fair
0.71 ± 8% +0.6 1.35 ± 10% perf-profile.children.cycles-pp.schedule
0.00 +0.7 0.68 ± 17% perf-profile.children.cycles-pp.rwsem_down_read_slowpath
0.86 ± 7% +0.7 1.58 ± 9% perf-profile.children.cycles-pp.__schedule
52.14 ± 14% +10.7 62.86 ± 9% perf-profile.children.cycles-pp.page_fault
52.09 ± 14% +10.7 62.83 ± 9% perf-profile.children.cycles-pp.do_page_fault
51.96 ± 14% +10.8 62.74 ± 9% perf-profile.children.cycles-pp.handle_mm_fault
51.86 ± 14% +10.8 62.67 ± 9% perf-profile.children.cycles-pp.hugetlb_fault
29.77 ± 21% +20.1 49.84 ± 13% perf-profile.children.cycles-pp.__mutex_lock
15.83 ± 27% +23.4 39.18 ± 18% perf-profile.children.cycles-pp.osq_lock
16.53 ± 4% -7.1 9.48 ± 9% perf-profile.self.cycles-pp.copy_page
15.27 ± 22% -6.4 8.90 ± 53% perf-profile.self.cycles-pp.do_rw_once
2.38 ± 42% -1.4 1.02 ± 7% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.70 ± 9% -0.2 0.48 ± 14% perf-profile.self.cycles-pp.perf_mux_hrtimer_handler
0.38 ± 2% -0.2 0.21 ± 11% perf-profile.self.cycles-pp.copy_subpage
0.33 ± 4% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.copy_user_huge_page
0.18 ± 18% -0.1 0.05 ± 8% perf-profile.self.cycles-pp.update_and_free_page
0.19 ± 14% -0.1 0.11 ± 18% perf-profile.self.cycles-pp.x86_pmu_disable
0.19 ± 6% -0.1 0.12 ± 10% perf-profile.self.cycles-pp.___might_sleep
0.35 ± 14% -0.1 0.28 ± 8% perf-profile.self.cycles-pp.run_timer_softirq
0.18 ± 10% -0.1 0.12 ± 6% perf-profile.self.cycles-pp.get_page_from_freelist
0.26 ± 6% -0.0 0.22 ± 5% perf-profile.self.cycles-pp.tick_program_event
0.07 -0.0 0.06 ± 7% perf-profile.self.cycles-pp.try_to_wake_up
0.00 +0.1 0.06 ± 9% perf-profile.self.cycles-pp.update_load_avg
0.73 ± 12% +0.3 1.06 ± 8% perf-profile.self.cycles-pp.update_sd_lb_stats
14.37 ± 25% +21.2 35.60 ± 18% perf-profile.self.cycles-pp.osq_lock
22773 ± 3% +11.6% 25420 ± 2% softirqs.CPU0.SCHED
15462 ± 6% +24.5% 19248 ± 5% softirqs.CPU1.SCHED
8331 ± 11% +27.1% 10590 ± 11% softirqs.CPU10.RCU
14783 ± 4% +16.0% 17145 ± 4% softirqs.CPU10.SCHED
14236 ± 3% +15.6% 16462 ± 2% softirqs.CPU100.SCHED
14117 ± 3% +18.2% 16683 ± 3% softirqs.CPU101.SCHED
13918 ± 4% +18.4% 16472 ± 5% softirqs.CPU102.SCHED
14183 ± 6% +18.0% 16742 ± 3% softirqs.CPU103.SCHED
13795 ± 5% +19.8% 16528 softirqs.CPU104.SCHED
14326 ± 6% +17.7% 16859 softirqs.CPU105.SCHED
13842 ± 5% +19.4% 16530 ± 2% softirqs.CPU106.SCHED
14155 ± 4% +20.0% 16983 ± 2% softirqs.CPU107.SCHED
13907 ± 3% +18.6% 16492 ± 2% softirqs.CPU108.SCHED
14411 ± 7% +16.1% 16726 ± 2% softirqs.CPU109.SCHED
14354 +19.0% 17085 ± 2% softirqs.CPU11.SCHED
13689 ± 6% +21.8% 16667 ± 3% softirqs.CPU110.SCHED
13461 ± 6% +25.6% 16907 ± 3% softirqs.CPU111.SCHED
13738 ± 9% +23.2% 16925 softirqs.CPU112.SCHED
14135 ± 3% +20.5% 17032 softirqs.CPU113.SCHED
14477 ± 4% +20.8% 17491 ± 4% softirqs.CPU114.SCHED
123003 ± 2% -7.8% 113426 ± 2% softirqs.CPU114.TIMER
14014 ± 4% +23.4% 17299 softirqs.CPU115.SCHED
13528 ± 6% +24.4% 16824 ± 2% softirqs.CPU116.SCHED
13530 ± 6% +24.5% 16840 ± 2% softirqs.CPU117.SCHED
14263 ± 3% +16.1% 16557 ± 2% softirqs.CPU119.SCHED
7902 ± 2% +22.1% 9648 ± 6% softirqs.CPU12.RCU
14332 ± 3% +19.5% 17130 ± 4% softirqs.CPU12.SCHED
13476 ± 5% +26.5% 17047 softirqs.CPU120.SCHED
14240 ± 6% +20.7% 17183 softirqs.CPU121.SCHED
13161 ± 5% +27.0% 16708 ± 2% softirqs.CPU122.SCHED
13037 ± 7% +28.8% 16786 softirqs.CPU123.SCHED
14229 ± 5% +16.5% 16577 ± 3% softirqs.CPU124.SCHED
14427 ± 4% +15.4% 16652 softirqs.CPU125.SCHED
14426 ± 4% +16.3% 16771 softirqs.CPU126.SCHED
6680 +39.3% 9305 ± 22% softirqs.CPU127.RCU
13659 ± 3% +23.7% 16890 ± 3% softirqs.CPU128.SCHED
14396 ± 5% +17.9% 16979 ± 3% softirqs.CPU129.SCHED
14183 ± 7% +20.2% 17048 ± 2% softirqs.CPU13.SCHED
6700 ± 2% +35.8% 9097 ± 17% softirqs.CPU130.RCU
13949 ± 4% +22.2% 17051 ± 4% softirqs.CPU130.SCHED
14271 ± 4% +21.8% 17379 ± 2% softirqs.CPU131.SCHED
129315 ± 3% -7.0% 120290 softirqs.CPU131.TIMER
14135 ± 2% +16.0% 16396 ± 2% softirqs.CPU132.SCHED
13567 ± 4% +22.5% 16617 ± 3% softirqs.CPU133.SCHED
13912 ± 5% +20.1% 16712 ± 2% softirqs.CPU134.SCHED
14309 ± 5% +18.9% 17016 ± 2% softirqs.CPU135.SCHED
13637 ± 4% +20.3% 16402 ± 5% softirqs.CPU136.SCHED
13843 ± 4% +19.3% 16518 ± 2% softirqs.CPU137.SCHED
14049 ± 5% +19.2% 16750 ± 2% softirqs.CPU138.SCHED
14244 ± 7% +18.1% 16816 softirqs.CPU139.SCHED
14139 +22.1% 17271 ± 2% softirqs.CPU14.SCHED
13460 ± 8% +22.9% 16546 softirqs.CPU140.SCHED
13645 ± 6% +21.7% 16601 ± 2% softirqs.CPU141.SCHED
13278 ± 6% +25.5% 16661 ± 3% softirqs.CPU142.SCHED
12487 ± 5% +29.2% 16136 ± 2% softirqs.CPU143.SCHED
13440 ± 9% +22.8% 16502 softirqs.CPU144.SCHED
126064 ± 2% -8.1% 115836 softirqs.CPU144.TIMER
13995 ± 6% +21.2% 16966 ± 3% softirqs.CPU145.SCHED
13834 ± 3% +14.8% 15883 ± 6% softirqs.CPU146.SCHED
14297 ± 9% +17.1% 16735 softirqs.CPU147.SCHED
14500 ± 3% +15.2% 16711 softirqs.CPU148.SCHED
14419 ± 6% +17.5% 16945 softirqs.CPU149.SCHED
8018 ± 6% +24.4% 9977 ± 7% softirqs.CPU15.RCU
13832 ± 5% +28.2% 17737 ± 5% softirqs.CPU15.SCHED
14538 ± 5% +15.0% 16723 ± 2% softirqs.CPU150.SCHED
15092 +9.5% 16526 ± 4% softirqs.CPU152.SCHED
14533 ± 2% +15.6% 16804 softirqs.CPU153.SCHED
14415 ± 2% +14.8% 16554 ± 2% softirqs.CPU154.SCHED
14583 ± 2% +15.4% 16833 ± 2% softirqs.CPU155.SCHED
14851 ± 2% +11.8% 16598 ± 2% softirqs.CPU156.SCHED
6747 +35.2% 9124 ± 19% softirqs.CPU158.RCU
13839 ± 4% +25.9% 17429 ± 4% softirqs.CPU158.SCHED
15020 ± 5% +11.8% 16785 ± 2% softirqs.CPU159.SCHED
13468 ± 5% +25.8% 16945 ± 2% softirqs.CPU16.SCHED
128305 ± 3% -8.1% 117855 softirqs.CPU160.TIMER
15111 ± 2% +13.7% 17184 softirqs.CPU162.SCHED
14788 ± 2% +12.8% 16685 ± 3% softirqs.CPU163.SCHED
13903 ± 5% +21.5% 16889 softirqs.CPU164.SCHED
14547 ± 5% +13.5% 16510 ± 2% softirqs.CPU165.SCHED
14440 ± 4% +13.3% 16354 ± 3% softirqs.CPU166.SCHED
14450 ± 5% +14.1% 16494 ± 2% softirqs.CPU167.SCHED
131454 ± 3% -6.9% 122396 ± 2% softirqs.CPU167.TIMER
13683 ± 6% +23.0% 16824 ± 3% softirqs.CPU168.SCHED
14468 ± 5% +18.1% 17085 ± 3% softirqs.CPU169.SCHED
7703 +22.5% 9437 ± 4% softirqs.CPU17.RCU
14392 +19.8% 17237 softirqs.CPU17.SCHED
14570 ± 6% +16.9% 17038 ± 3% softirqs.CPU170.SCHED
14172 ± 5% +19.3% 16905 softirqs.CPU171.SCHED
14176 ± 3% +17.9% 16709 softirqs.CPU172.SCHED
127159 ± 2% -8.4% 116497 ± 2% softirqs.CPU172.TIMER
15034 ± 2% +11.1% 16698 ± 2% softirqs.CPU174.SCHED
14923 ± 8% +17.0% 17458 ± 5% softirqs.CPU175.SCHED
14164 ± 5% +18.8% 16833 ± 2% softirqs.CPU176.SCHED
127027 ± 2% -8.4% 116378 ± 2% softirqs.CPU176.TIMER
14359 ± 3% +16.9% 16788 ± 4% softirqs.CPU177.SCHED
14466 ± 3% +18.7% 17169 softirqs.CPU178.SCHED
14632 ± 2% +15.1% 16837 softirqs.CPU179.SCHED
123562 ± 3% -8.2% 113392 ± 2% softirqs.CPU179.TIMER
7677 +27.6% 9794 ± 2% softirqs.CPU18.RCU
14177 ± 2% +21.6% 17233 softirqs.CPU18.SCHED
14429 ± 7% +15.8% 16707 softirqs.CPU180.SCHED
15194 ± 5% +12.4% 17084 softirqs.CPU181.SCHED
13917 ± 3% +20.2% 16723 ± 3% softirqs.CPU182.SCHED
14587 ± 2% +15.1% 16786 ± 3% softirqs.CPU183.SCHED
14266 ± 3% +18.0% 16827 ± 2% softirqs.CPU184.SCHED
14661 ± 3% +16.3% 17053 softirqs.CPU185.SCHED
14222 ± 5% +21.0% 17202 ± 3% softirqs.CPU186.SCHED
15059 ± 2% +13.0% 17022 ± 3% softirqs.CPU187.SCHED
125388 ± 2% -7.6% 115887 ± 2% softirqs.CPU187.TIMER
13435 ± 4% +25.0% 16801 ± 3% softirqs.CPU188.SCHED
13434 ± 2% +28.8% 17297 ± 2% softirqs.CPU189.SCHED
7719 +27.0% 9806 ± 9% softirqs.CPU19.RCU
14075 ± 2% +20.7% 16986 softirqs.CPU19.SCHED
13735 ± 3% +20.4% 16536 ± 2% softirqs.CPU190.SCHED
13772 ± 3% +19.8% 16494 softirqs.CPU192.SCHED
13911 ± 2% +17.4% 16327 ± 2% softirqs.CPU193.SCHED
6417 +34.3% 8616 ± 17% softirqs.CPU194.RCU
14008 +17.6% 16468 ± 8% softirqs.CPU194.SCHED
14094 ± 4% +19.9% 16905 softirqs.CPU195.SCHED
14387 ± 7% +16.1% 16698 ± 5% softirqs.CPU196.SCHED
13704 ± 3% +23.7% 16952 softirqs.CPU198.SCHED
8214 ± 10% +23.7% 10157 ± 7% softirqs.CPU2.RCU
14431 +17.4% 16946 ± 4% softirqs.CPU2.SCHED
7770 +30.4% 10135 ± 8% softirqs.CPU20.RCU
13996 ± 2% +23.7% 17320 softirqs.CPU20.SCHED
14611 ± 5% +12.7% 16463 ± 3% softirqs.CPU200.SCHED
14897 ± 3% +15.5% 17199 softirqs.CPU201.SCHED
14105 ± 6% +21.7% 17167 ± 2% softirqs.CPU202.SCHED
127698 ± 3% -7.1% 118685 softirqs.CPU202.TIMER
14369 ± 6% +18.6% 17048 ± 2% softirqs.CPU203.SCHED
126668 ± 3% -8.8% 115539 softirqs.CPU203.TIMER
14461 ± 4% +13.1% 16356 ± 3% softirqs.CPU205.SCHED
13139 ± 10% +26.6% 16634 ± 3% softirqs.CPU206.SCHED
14808 ± 2% +11.8% 16550 ± 2% softirqs.CPU207.SCHED
13744 ± 5% +21.0% 16624 ± 4% softirqs.CPU208.SCHED
14039 ± 5% +18.7% 16661 ± 2% softirqs.CPU209.SCHED
127488 ± 2% -8.2% 116998 softirqs.CPU209.TIMER
7755 ± 2% +30.5% 10122 ± 12% softirqs.CPU21.RCU
13731 ± 5% +25.2% 17185 ± 2% softirqs.CPU21.SCHED
14755 ± 6% +16.1% 17123 ± 4% softirqs.CPU211.SCHED
13920 ± 3% +18.5% 16501 ± 2% softirqs.CPU212.SCHED
14435 ± 4% +16.1% 16765 ± 3% softirqs.CPU213.SCHED
13857 ± 6% +21.6% 16844 ± 2% softirqs.CPU214.SCHED
13417 ± 8% +16.5% 15635 ± 2% softirqs.CPU215.SCHED
127636 ± 3% -7.4% 118224 softirqs.CPU215.TIMER
13715 ± 3% +21.7% 16686 ± 3% softirqs.CPU216.SCHED
126421 ± 2% -7.8% 116544 softirqs.CPU216.TIMER
13885 ± 5% +18.2% 16416 softirqs.CPU217.SCHED
14003 ± 8% +20.5% 16875 softirqs.CPU218.SCHED
13891 ± 4% +23.9% 17210 ± 4% softirqs.CPU22.SCHED
14901 ± 2% +12.0% 16694 ± 2% softirqs.CPU220.SCHED
14623 ± 6% +17.3% 17153 softirqs.CPU221.SCHED
14613 ± 8% +13.9% 16643 ± 2% softirqs.CPU224.SCHED
14745 ± 4% +12.9% 16643 softirqs.CPU225.SCHED
126232 -8.3% 115717 softirqs.CPU225.TIMER
14290 ± 2% +17.2% 16753 softirqs.CPU226.SCHED
15229 ± 3% +10.3% 16800 ± 3% softirqs.CPU228.SCHED
14282 ± 6% +15.4% 16479 ± 3% softirqs.CPU23.SCHED
13785 ± 9% +24.2% 17126 ± 4% softirqs.CPU230.SCHED
126791 ± 4% -9.4% 114889 ± 3% softirqs.CPU231.TIMER
14749 ± 3% +15.0% 16958 ± 3% softirqs.CPU232.SCHED
15099 ± 5% +11.9% 16893 ± 2% softirqs.CPU233.SCHED
14825 ± 6% +12.9% 16738 softirqs.CPU234.SCHED
14272 ± 5% +18.7% 16945 ± 2% softirqs.CPU236.SCHED
15187 ± 2% +10.9% 16846 softirqs.CPU237.SCHED
15267 ± 7% +11.1% 16955 ± 3% softirqs.CPU239.SCHED
14156 ± 4% +16.3% 16466 ± 3% softirqs.CPU24.SCHED
14671 ± 3% +13.9% 16711 ± 3% softirqs.CPU240.SCHED
127318 ± 2% -7.6% 117651 ± 2% softirqs.CPU240.TIMER
14218 ± 5% +20.2% 17097 softirqs.CPU241.SCHED
14554 ± 3% +15.2% 16771 softirqs.CPU242.SCHED
14003 +19.9% 16785 softirqs.CPU243.SCHED
13761 ± 6% +21.5% 16724 ± 2% softirqs.CPU244.SCHED
14418 ± 6% +16.4% 16781 softirqs.CPU245.SCHED
14760 ± 7% +14.8% 16940 ± 4% softirqs.CPU246.SCHED
14891 ± 3% +12.3% 16722 ± 4% softirqs.CPU249.SCHED
13823 ± 4% +24.1% 17148 softirqs.CPU25.SCHED
14201 ± 4% +18.1% 16774 ± 3% softirqs.CPU250.SCHED
14620 ± 10% +18.5% 17331 softirqs.CPU251.SCHED
14877 ± 3% +11.3% 16564 ± 2% softirqs.CPU252.SCHED
14517 ± 6% +16.3% 16877 ± 3% softirqs.CPU254.SCHED
14284 ± 7% +18.0% 16858 ± 4% softirqs.CPU255.SCHED
121538 ± 3% -7.5% 112473 softirqs.CPU255.TIMER
14582 +14.8% 16736 softirqs.CPU256.SCHED
14057 ± 3% +22.8% 17259 softirqs.CPU257.SCHED
15076 ± 3% +13.4% 17097 ± 3% softirqs.CPU258.SCHED
14356 ± 3% +20.2% 17251 ± 3% softirqs.CPU259.SCHED
13998 ± 3% +20.1% 16806 ± 2% softirqs.CPU26.SCHED
13899 ± 4% +19.0% 16534 ± 5% softirqs.CPU260.SCHED
128277 ± 4% -7.4% 118758 softirqs.CPU260.TIMER
14288 ± 5% +20.7% 17251 softirqs.CPU261.SCHED
14456 ± 5% +15.1% 16644 softirqs.CPU262.SCHED
14114 ± 2% +17.8% 16624 ± 4% softirqs.CPU263.SCHED
13328 ± 4% +25.0% 16657 softirqs.CPU264.SCHED
13328 ± 4% +25.1% 16676 ± 4% softirqs.CPU266.SCHED
14367 ± 3% +19.5% 17165 ± 2% softirqs.CPU267.SCHED
122663 ± 3% -7.6% 113293 ± 2% softirqs.CPU268.TIMER
13527 ± 6% +25.2% 16940 softirqs.CPU27.SCHED
14341 ± 5% +19.8% 17187 softirqs.CPU270.SCHED
14345 ± 5% +14.3% 16403 ± 5% softirqs.CPU271.SCHED
14990 ± 3% +14.4% 17152 ± 4% softirqs.CPU272.SCHED
15007 ± 2% +13.4% 17012 softirqs.CPU273.SCHED
14668 ± 8% +17.0% 17157 ± 4% softirqs.CPU274.SCHED
14287 ± 6% +17.2% 16749 ± 4% softirqs.CPU277.SCHED
6301 +41.8% 8938 ± 16% softirqs.CPU278.RCU
13980 ± 10% +24.6% 17424 ± 2% softirqs.CPU278.SCHED
14763 ± 7% +16.0% 17119 softirqs.CPU279.SCHED
13682 ± 5% +23.9% 16950 softirqs.CPU28.SCHED
14204 ± 3% +17.0% 16617 ± 3% softirqs.CPU280.SCHED
14202 ± 3% +14.4% 16240 ± 2% softirqs.CPU281.SCHED
14629 ± 7% +16.3% 17013 ± 3% softirqs.CPU283.SCHED
13692 ± 6% +22.7% 16795 softirqs.CPU284.SCHED
14194 ± 7% +17.5% 16679 ± 3% softirqs.CPU285.SCHED
123721 ± 3% -8.1% 113741 ± 3% softirqs.CPU285.TIMER
14757 ± 7% +15.5% 17050 softirqs.CPU286.SCHED
13560 ± 6% +17.2% 15894 softirqs.CPU287.SCHED
14201 ± 4% +19.9% 17020 ± 2% softirqs.CPU29.SCHED
14345 ± 3% +20.5% 17292 softirqs.CPU3.SCHED
13910 ± 4% +20.9% 16814 ± 4% softirqs.CPU30.SCHED
7587 +41.8% 10759 ± 23% softirqs.CPU31.RCU
13401 ± 4% +26.6% 16967 ± 2% softirqs.CPU31.SCHED
13593 +24.6% 16937 softirqs.CPU32.SCHED
13175 ± 2% +29.0% 16989 softirqs.CPU33.SCHED
13491 ± 5% +25.8% 16974 ± 3% softirqs.CPU34.SCHED
13520 ± 7% +25.9% 17018 ± 2% softirqs.CPU35.SCHED
13723 ± 6% +22.1% 16763 ± 2% softirqs.CPU36.SCHED
13449 ± 3% +27.1% 17091 softirqs.CPU37.SCHED
7405 +40.0% 10368 ± 17% softirqs.CPU38.RCU
14299 ± 6% +22.4% 17509 ± 3% softirqs.CPU38.SCHED
14269 ± 3% +18.6% 16918 softirqs.CPU39.SCHED
7977 ± 2% +23.6% 9862 ± 5% softirqs.CPU4.RCU
14055 ± 2% +23.4% 17349 ± 3% softirqs.CPU4.SCHED
14257 ± 2% +19.4% 17030 ± 2% softirqs.CPU40.SCHED
13774 ± 2% +24.1% 17090 softirqs.CPU41.SCHED
14425 ± 8% +17.7% 16976 softirqs.CPU42.SCHED
14314 ± 2% +21.2% 17345 ± 2% softirqs.CPU43.SCHED
13274 ± 4% +27.7% 16954 softirqs.CPU44.SCHED
135612 ± 10% -15.7% 114264 ± 3% softirqs.CPU44.TIMER
13919 ± 2% +20.9% 16829 softirqs.CPU45.SCHED
13156 +27.9% 16834 ± 2% softirqs.CPU46.SCHED
13617 ± 2% +22.0% 16614 softirqs.CPU47.SCHED
13565 ± 3% +25.7% 17054 ± 2% softirqs.CPU48.SCHED
13791 ± 4% +21.6% 16769 softirqs.CPU49.SCHED
14260 ± 3% +20.1% 17126 ± 4% softirqs.CPU5.SCHED
13510 ± 7% +25.3% 16934 softirqs.CPU50.SCHED
12790 ± 10% +30.3% 16669 softirqs.CPU51.SCHED
13989 +19.9% 16774 ± 3% softirqs.CPU52.SCHED
14279 ± 3% +17.1% 16722 ± 2% softirqs.CPU53.SCHED
7263 +36.9% 9940 ± 21% softirqs.CPU54.RCU
14088 ± 2% +20.1% 16918 softirqs.CPU54.SCHED
13508 ± 6% +23.1% 16629 softirqs.CPU55.SCHED
130302 ± 6% -9.4% 118014 ± 7% softirqs.CPU55.TIMER
13313 ± 3% +27.9% 17030 ± 2% softirqs.CPU56.SCHED
7360 ± 2% +33.2% 9803 ± 12% softirqs.CPU57.RCU
13818 ± 2% +21.3% 16768 ± 2% softirqs.CPU57.SCHED
7296 +27.1% 9274 ± 7% softirqs.CPU58.RCU
13922 ± 3% +23.4% 17181 ± 2% softirqs.CPU58.SCHED
7271 +30.3% 9476 ± 11% softirqs.CPU59.RCU
14169 ± 5% +19.7% 16956 ± 2% softirqs.CPU59.SCHED
8176 ± 7% +16.5% 9524 ± 3% softirqs.CPU6.RCU
13689 ± 3% +25.2% 17132 ± 5% softirqs.CPU6.SCHED
13645 ± 2% +23.3% 16825 softirqs.CPU60.SCHED
13441 ± 6% +24.9% 16786 ± 2% softirqs.CPU61.SCHED
13497 ± 5% +25.7% 16959 softirqs.CPU62.SCHED
13682 ± 4% +25.5% 17170 softirqs.CPU63.SCHED
13656 ± 6% +22.3% 16695 ± 3% softirqs.CPU64.SCHED
13704 ± 7% +22.7% 16819 softirqs.CPU65.SCHED
13683 ± 3% +23.0% 16826 softirqs.CPU66.SCHED
14076 ± 2% +19.3% 16797 ± 2% softirqs.CPU67.SCHED
13713 ± 4% +18.2% 16215 ± 2% softirqs.CPU68.SCHED
13396 ± 3% +25.1% 16754 ± 2% softirqs.CPU69.SCHED
8042 ± 3% +21.0% 9730 ± 7% softirqs.CPU7.RCU
13604 ± 3% +27.3% 17320 ± 5% softirqs.CPU7.SCHED
13023 ± 2% +29.2% 16828 ± 2% softirqs.CPU70.SCHED
12307 ± 3% +32.1% 16255 softirqs.CPU71.SCHED
7727 ± 22% +37.3% 10612 ± 22% softirqs.CPU72.RCU
13320 ± 3% +28.0% 17050 ± 2% softirqs.CPU72.SCHED
13838 ± 3% +22.0% 16883 ± 3% softirqs.CPU73.SCHED
14604 ± 2% +14.4% 16707 ± 4% softirqs.CPU74.SCHED
14184 ± 4% +19.1% 16892 ± 2% softirqs.CPU75.SCHED
14138 ± 4% +19.0% 16830 softirqs.CPU76.SCHED
7098 +28.1% 9090 ± 12% softirqs.CPU77.RCU
13082 ± 4% +28.0% 16741 softirqs.CPU77.SCHED
126355 ± 4% -10.8% 112765 ± 2% softirqs.CPU78.TIMER
14718 ± 5% +12.6% 16576 ± 3% softirqs.CPU79.SCHED
7959 ± 2% +27.1% 10119 ± 10% softirqs.CPU8.RCU
13775 ± 4% +25.1% 17232 ± 5% softirqs.CPU8.SCHED
14397 +13.6% 16355 ± 2% softirqs.CPU80.SCHED
13999 ± 6% +20.7% 16896 ± 2% softirqs.CPU81.SCHED
14129 ± 7% +18.2% 16706 softirqs.CPU82.SCHED
13931 +17.6% 16385 ± 3% softirqs.CPU83.SCHED
14455 ± 5% +16.1% 16776 ± 2% softirqs.CPU84.SCHED
14426 ± 5% +16.4% 16799 softirqs.CPU86.SCHED
7049 +28.2% 9036 ± 10% softirqs.CPU88.RCU
13901 ± 3% +20.1% 16697 ± 2% softirqs.CPU88.SCHED
127258 ± 2% -7.3% 118017 softirqs.CPU88.TIMER
14353 ± 5% +17.4% 16855 ± 2% softirqs.CPU89.SCHED
14823 ± 4% +14.7% 17004 softirqs.CPU9.SCHED
14278 ± 4% +18.4% 16908 softirqs.CPU90.SCHED
14225 ± 2% +18.8% 16894 ± 2% softirqs.CPU91.SCHED
7091 +26.7% 8987 ± 10% softirqs.CPU92.RCU
13726 ± 3% +24.0% 17014 ± 3% softirqs.CPU92.SCHED
14051 ± 5% +20.6% 16939 softirqs.CPU93.SCHED
14123 ± 4% +17.9% 16649 ± 3% softirqs.CPU94.SCHED
130109 ± 5% -8.2% 119400 softirqs.CPU95.TIMER
13638 ± 4% +23.5% 16847 ± 2% softirqs.CPU96.SCHED
13134 ± 2% +29.4% 17001 ± 3% softirqs.CPU97.SCHED
13357 ± 3% +25.7% 16791 ± 2% softirqs.CPU98.SCHED
14112 ± 6% +20.0% 16938 softirqs.CPU99.SCHED
1991578 +21.8% 2426642 ± 3% softirqs.RCU
4100803 +18.4% 4855612 softirqs.SCHED
454.00 ± 27% +675.8% 3522 ±139% interrupts.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0
384622 -1.6% 378485 interrupts.CAL:Function_call_interrupts
930.25 ± 8% -46.7% 496.00 ± 10% interrupts.CPU0.TLB:TLB_shootdowns
758.50 ± 9% -44.8% 418.75 ± 11% interrupts.CPU1.TLB:TLB_shootdowns
454.00 ± 27% +675.8% 3522 ±139% interrupts.CPU10.30:IR-PCI-MSI.2097153-edge.eth0-TxRx-0
886.75 ± 5% -39.4% 537.00 ± 3% interrupts.CPU10.TLB:TLB_shootdowns
169.50 ± 11% +67.0% 283.00 ± 9% interrupts.CPU100.TLB:TLB_shootdowns
178.50 ± 20% +44.0% 257.00 ± 9% interrupts.CPU101.TLB:TLB_shootdowns
169.00 ± 23% +62.7% 275.00 ± 20% interrupts.CPU104.TLB:TLB_shootdowns
1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.NMI:Non-maskable_interrupts
1463 ± 28% +88.1% 2753 ± 22% interrupts.CPU105.PMI:Performance_monitoring_interrupts
158.75 ± 21% +73.4% 275.25 ± 4% interrupts.CPU106.TLB:TLB_shootdowns
157.50 ± 24% +67.3% 263.50 ± 13% interrupts.CPU109.TLB:TLB_shootdowns
2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.NMI:Non-maskable_interrupts
2884 ± 27% -43.7% 1625 ± 6% interrupts.CPU11.PMI:Performance_monitoring_interrupts
713.50 ± 5% -38.8% 436.50 ± 13% interrupts.CPU11.TLB:TLB_shootdowns
146.00 ± 38% +79.3% 261.75 ± 14% interrupts.CPU110.TLB:TLB_shootdowns
162.00 ± 14% +82.6% 295.75 ± 12% interrupts.CPU112.TLB:TLB_shootdowns
445.75 ± 52% +170.1% 1204 ± 91% interrupts.CPU114.RES:Rescheduling_interrupts
160.25 ± 37% +70.8% 273.75 ± 14% interrupts.CPU114.TLB:TLB_shootdowns
1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.NMI:Non-maskable_interrupts
1417 ± 17% +66.0% 2352 ± 34% interrupts.CPU115.PMI:Performance_monitoring_interrupts
380.75 ± 9% +37.9% 525.00 ± 15% interrupts.CPU116.RES:Rescheduling_interrupts
183.00 ± 9% +48.2% 271.25 ± 14% interrupts.CPU116.TLB:TLB_shootdowns
170.50 ± 15% +74.5% 297.50 ± 5% interrupts.CPU118.TLB:TLB_shootdowns
454.25 ± 38% +251.3% 1596 ±103% interrupts.CPU12.RES:Rescheduling_interrupts
847.00 ± 12% -36.9% 534.25 ± 10% interrupts.CPU12.TLB:TLB_shootdowns
173.25 ± 20% +65.7% 287.00 ± 13% interrupts.CPU122.TLB:TLB_shootdowns
420.50 ± 32% +80.5% 759.00 ± 37% interrupts.CPU123.RES:Rescheduling_interrupts
247.75 ± 22% +75.9% 435.75 ± 14% interrupts.CPU124.RES:Rescheduling_interrupts
181.75 ± 33% +58.7% 288.50 ± 15% interrupts.CPU126.TLB:TLB_shootdowns
364.75 ± 21% +39.1% 507.50 ± 11% interrupts.CPU128.RES:Rescheduling_interrupts
337.75 ± 24% +248.5% 1177 ± 74% interrupts.CPU13.RES:Rescheduling_interrupts
783.25 ± 14% -47.8% 409.00 ± 3% interrupts.CPU13.TLB:TLB_shootdowns
277.00 ± 14% +84.9% 512.25 ± 10% interrupts.CPU131.RES:Rescheduling_interrupts
155.75 ± 24% +71.7% 267.50 ± 21% interrupts.CPU132.TLB:TLB_shootdowns
174.25 ± 24% +50.4% 262.00 ± 19% interrupts.CPU133.TLB:TLB_shootdowns
137.00 ± 7% +116.4% 296.50 ± 11% interrupts.CPU134.TLB:TLB_shootdowns
152.00 ± 24% +86.3% 283.25 ± 17% interrupts.CPU138.TLB:TLB_shootdowns
820.00 ± 10% -32.5% 553.75 ± 3% interrupts.CPU14.TLB:TLB_shootdowns
277.75 ± 19% +81.1% 503.00 ± 7% interrupts.CPU140.RES:Rescheduling_interrupts
298.25 ± 31% +59.3% 475.00 ± 15% interrupts.CPU142.RES:Rescheduling_interrupts
192.50 ± 22% +38.6% 266.75 ± 4% interrupts.CPU143.TLB:TLB_shootdowns
290.00 ± 38% +103.4% 590.00 ± 18% interrupts.CPU145.RES:Rescheduling_interrupts
104.50 ± 34% +90.4% 199.00 ± 16% interrupts.CPU145.TLB:TLB_shootdowns
226.50 ± 10% +96.9% 446.00 ± 15% interrupts.CPU146.RES:Rescheduling_interrupts
95.75 ± 8% +145.4% 235.00 ± 12% interrupts.CPU146.TLB:TLB_shootdowns
145.00 ± 22% +65.2% 239.50 ± 16% interrupts.CPU147.TLB:TLB_shootdowns
90.25 ± 23% +212.2% 281.75 ± 19% interrupts.CPU148.TLB:TLB_shootdowns
104.25 ± 22% +142.7% 253.00 ± 11% interrupts.CPU149.TLB:TLB_shootdowns
2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.NMI:Non-maskable_interrupts
2888 ± 16% -35.6% 1860 ± 30% interrupts.CPU15.PMI:Performance_monitoring_interrupts
472.25 ± 31% +275.2% 1771 ±106% interrupts.CPU15.RES:Rescheduling_interrupts
816.50 ± 16% -50.0% 408.50 ± 11% interrupts.CPU15.TLB:TLB_shootdowns
84.75 ± 24% +207.4% 260.50 ± 10% interrupts.CPU150.TLB:TLB_shootdowns
133.50 ± 16% +108.6% 278.50 ± 17% interrupts.CPU151.TLB:TLB_shootdowns
103.75 ± 25% +132.0% 240.75 ± 8% interrupts.CPU152.TLB:TLB_shootdowns
105.00 ± 60% +155.0% 267.75 ± 7% interrupts.CPU153.TLB:TLB_shootdowns
100.50 ± 28% +133.1% 234.25 ± 8% interrupts.CPU154.TLB:TLB_shootdowns
73.50 ± 36% +220.4% 235.50 ± 11% interrupts.CPU155.TLB:TLB_shootdowns
229.50 ± 18% +182.1% 647.50 ± 8% interrupts.CPU156.RES:Rescheduling_interrupts
114.50 ± 20% +110.9% 241.50 ± 29% interrupts.CPU156.TLB:TLB_shootdowns
125.75 ± 27% +79.7% 226.00 ± 8% interrupts.CPU157.TLB:TLB_shootdowns
97.50 ± 21% +143.1% 237.00 ± 16% interrupts.CPU158.TLB:TLB_shootdowns
94.75 ± 49% +136.4% 224.00 ± 21% interrupts.CPU159.TLB:TLB_shootdowns
452.00 ± 36% +89.2% 855.25 ± 39% interrupts.CPU16.RES:Rescheduling_interrupts
959.00 ± 16% -54.3% 438.25 ± 8% interrupts.CPU16.TLB:TLB_shootdowns
96.75 ± 9% +145.0% 237.00 ± 9% interrupts.CPU161.TLB:TLB_shootdowns
94.25 ± 24% +169.5% 254.00 ± 10% interrupts.CPU162.TLB:TLB_shootdowns
100.50 ± 8% +141.8% 243.00 ± 14% interrupts.CPU163.TLB:TLB_shootdowns
87.00 ± 42% +186.5% 249.25 ± 8% interrupts.CPU164.TLB:TLB_shootdowns
93.50 ± 29% +145.7% 229.75 ± 9% interrupts.CPU165.TLB:TLB_shootdowns
110.50 ± 33% +165.4% 293.25 ± 15% interrupts.CPU166.TLB:TLB_shootdowns
323.25 ± 37% +105.7% 665.00 ± 39% interrupts.CPU167.RES:Rescheduling_interrupts
96.75 ± 48% +140.3% 232.50 ± 14% interrupts.CPU167.TLB:TLB_shootdowns
224.00 ± 22% +140.8% 539.50 ± 40% interrupts.CPU168.RES:Rescheduling_interrupts
87.50 ± 26% +151.1% 219.75 ± 14% interrupts.CPU168.TLB:TLB_shootdowns
1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.NMI:Non-maskable_interrupts
1120 ± 47% +79.9% 2015 ± 27% interrupts.CPU169.PMI:Performance_monitoring_interrupts
100.25 ± 22% +117.5% 218.00 ± 9% interrupts.CPU169.TLB:TLB_shootdowns
667.75 ± 12% -30.3% 465.50 ± 14% interrupts.CPU17.TLB:TLB_shootdowns
75.00 ± 28% +252.7% 264.50 ± 22% interrupts.CPU170.TLB:TLB_shootdowns
94.25 ± 19% +126.0% 213.00 ± 13% interrupts.CPU171.TLB:TLB_shootdowns
318.25 ± 28% +53.3% 488.00 ± 18% interrupts.CPU172.RES:Rescheduling_interrupts
85.25 ± 33% +221.1% 273.75 ± 19% interrupts.CPU172.TLB:TLB_shootdowns
82.25 ± 18% +130.7% 189.75 ± 20% interrupts.CPU173.TLB:TLB_shootdowns
67.75 ± 55% +261.6% 245.00 ± 12% interrupts.CPU174.TLB:TLB_shootdowns
227.75 ± 41% +588.6% 1568 ± 95% interrupts.CPU175.RES:Rescheduling_interrupts
91.50 ± 10% +150.8% 229.50 ± 10% interrupts.CPU175.TLB:TLB_shootdowns
99.50 ± 17% +147.2% 246.00 ± 22% interrupts.CPU176.TLB:TLB_shootdowns
79.25 ± 18% +200.0% 237.75 ± 27% interrupts.CPU177.TLB:TLB_shootdowns
87.50 ± 36% +129.7% 201.00 ± 4% interrupts.CPU178.TLB:TLB_shootdowns
127.25 ± 44% +82.7% 232.50 ± 15% interrupts.CPU179.TLB:TLB_shootdowns
1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.NMI:Non-maskable_interrupts
1161 ± 15% +137.6% 2758 ± 27% interrupts.CPU18.PMI:Performance_monitoring_interrupts
425.75 ± 15% +149.5% 1062 ± 64% interrupts.CPU18.RES:Rescheduling_interrupts
985.50 ± 13% -50.1% 492.25 ± 13% interrupts.CPU18.TLB:TLB_shootdowns
112.25 ± 33% +120.7% 247.75 ± 8% interrupts.CPU180.TLB:TLB_shootdowns
126.25 ± 15% +96.0% 247.50 ± 9% interrupts.CPU181.TLB:TLB_shootdowns
229.25 ± 25% +74.6% 400.25 ± 7% interrupts.CPU182.RES:Rescheduling_interrupts
102.50 ± 41% +173.4% 280.25 ± 4% interrupts.CPU182.TLB:TLB_shootdowns
253.25 ± 13% +52.6% 386.50 ± 8% interrupts.CPU183.RES:Rescheduling_interrupts
97.00 ± 50% +153.4% 245.75 ± 4% interrupts.CPU183.TLB:TLB_shootdowns
99.00 ± 14% +143.7% 241.25 ± 18% interrupts.CPU184.TLB:TLB_shootdowns
287.50 ± 37% +57.0% 451.25 ± 32% interrupts.CPU185.RES:Rescheduling_interrupts
92.50 ± 27% +139.2% 221.25 ± 5% interrupts.CPU185.TLB:TLB_shootdowns
2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.NMI:Non-maskable_interrupts
2302 ± 26% +37.4% 3162 ± 5% interrupts.CPU186.PMI:Performance_monitoring_interrupts
87.75 ± 16% +204.0% 266.75 ± 20% interrupts.CPU186.TLB:TLB_shootdowns
97.00 ± 31% +157.0% 249.25 ± 10% interrupts.CPU187.TLB:TLB_shootdowns
261.50 ± 10% +54.2% 403.25 ± 15% interrupts.CPU188.RES:Rescheduling_interrupts
108.50 ± 42% +103.7% 221.00 ± 26% interrupts.CPU188.TLB:TLB_shootdowns
223.75 ± 21% +88.4% 421.50 ± 18% interrupts.CPU189.RES:Rescheduling_interrupts
119.25 ± 11% +108.4% 248.50 ± 25% interrupts.CPU189.TLB:TLB_shootdowns
400.25 ± 27% +117.4% 870.25 ± 52% interrupts.CPU19.RES:Rescheduling_interrupts
699.50 ± 16% -31.8% 476.75 ± 10% interrupts.CPU19.TLB:TLB_shootdowns
99.75 ± 43% +148.6% 248.00 ± 12% interrupts.CPU190.TLB:TLB_shootdowns
98.25 ± 10% +146.6% 242.25 ± 17% interrupts.CPU191.TLB:TLB_shootdowns
105.25 ± 29% +145.4% 258.25 ± 19% interrupts.CPU192.TLB:TLB_shootdowns
89.50 ± 23% +191.3% 260.75 ± 17% interrupts.CPU193.TLB:TLB_shootdowns
66.25 ± 25% +285.7% 255.50 ± 17% interrupts.CPU194.TLB:TLB_shootdowns
278.75 ± 10% +84.9% 515.50 ± 32% interrupts.CPU195.RES:Rescheduling_interrupts
80.00 ± 18% +247.8% 278.25 ± 14% interrupts.CPU195.TLB:TLB_shootdowns
70.25 ± 12% +246.3% 243.25 ± 11% interrupts.CPU196.TLB:TLB_shootdowns
104.75 ± 26% +147.0% 258.75 ± 23% interrupts.CPU197.TLB:TLB_shootdowns
86.75 ± 62% +188.2% 250.00 ± 20% interrupts.CPU198.TLB:TLB_shootdowns
94.25 ± 17% +150.1% 235.75 ± 19% interrupts.CPU199.TLB:TLB_shootdowns
3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.NMI:Non-maskable_interrupts
3084 ± 4% -52.9% 1453 ± 6% interrupts.CPU2.PMI:Performance_monitoring_interrupts
556.25 ± 46% +179.6% 1555 ± 83% interrupts.CPU2.RES:Rescheduling_interrupts
891.00 ± 15% -42.6% 511.50 ± 13% interrupts.CPU2.TLB:TLB_shootdowns
364.75 ± 16% +193.9% 1072 ± 40% interrupts.CPU20.RES:Rescheduling_interrupts
825.75 ± 13% -35.7% 530.75 ± 6% interrupts.CPU20.TLB:TLB_shootdowns
129.25 ± 17% +83.0% 236.50 ± 7% interrupts.CPU200.TLB:TLB_shootdowns
88.50 ± 58% +200.0% 265.50 ± 12% interrupts.CPU201.TLB:TLB_shootdowns
116.75 ± 24% +99.8% 233.25 ± 24% interrupts.CPU202.TLB:TLB_shootdowns
254.25 ± 35% +121.8% 564.00 ± 14% interrupts.CPU203.RES:Rescheduling_interrupts
83.25 ± 27% +203.0% 252.25 ± 18% interrupts.CPU203.TLB:TLB_shootdowns
110.25 ± 16% +195.2% 325.50 ± 35% interrupts.CPU204.TLB:TLB_shootdowns
332.00 ± 44% +60.5% 532.75 ± 15% interrupts.CPU205.RES:Rescheduling_interrupts
89.00 ± 32% +165.4% 236.25 ± 11% interrupts.CPU205.TLB:TLB_shootdowns
236.25 ± 23% +112.8% 502.75 ± 10% interrupts.CPU206.RES:Rescheduling_interrupts
82.75 ± 32% +195.2% 244.25 ± 13% interrupts.CPU206.TLB:TLB_shootdowns
121.25 ± 15% +86.6% 226.25 ± 18% interrupts.CPU207.TLB:TLB_shootdowns
83.75 ± 24% +170.7% 226.75 ± 5% interrupts.CPU208.TLB:TLB_shootdowns
330.25 ± 13% +131.5% 764.50 ± 44% interrupts.CPU209.RES:Rescheduling_interrupts
75.75 ± 20% +217.8% 240.75 ± 9% interrupts.CPU209.TLB:TLB_shootdowns
778.00 ± 16% -39.2% 473.00 ± 34% interrupts.CPU21.TLB:TLB_shootdowns
268.75 ± 27% +178.4% 748.25 ± 37% interrupts.CPU210.RES:Rescheduling_interrupts
114.75 ± 16% +100.0% 229.50 ± 13% interrupts.CPU210.TLB:TLB_shootdowns
112.00 ± 29% +101.1% 225.25 ± 15% interrupts.CPU211.TLB:TLB_shootdowns
97.00 ± 58% +159.3% 251.50 ± 21% interrupts.CPU212.TLB:TLB_shootdowns
263.75 ± 29% +80.3% 475.50 ± 15% interrupts.CPU213.RES:Rescheduling_interrupts
101.25 ± 25% +98.5% 201.00 ± 28% interrupts.CPU215.TLB:TLB_shootdowns
1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.NMI:Non-maskable_interrupts
1275 ± 12% +70.6% 2177 ± 30% interrupts.CPU216.PMI:Performance_monitoring_interrupts
278.75 ± 15% +48.7% 414.50 ± 14% interrupts.CPU216.RES:Rescheduling_interrupts
95.50 ± 26% +118.3% 208.50 ± 11% interrupts.CPU216.TLB:TLB_shootdowns
1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.NMI:Non-maskable_interrupts
1391 ± 30% +54.2% 2146 ± 33% interrupts.CPU217.PMI:Performance_monitoring_interrupts
114.50 ± 10% +84.9% 211.75 ± 16% interrupts.CPU217.TLB:TLB_shootdowns
77.00 ± 48% +308.4% 314.50 ± 55% interrupts.CPU218.TLB:TLB_shootdowns
70.75 ± 41% +259.0% 254.00 ± 21% interrupts.CPU219.TLB:TLB_shootdowns
884.75 ± 18% -36.6% 561.25 ± 12% interrupts.CPU22.TLB:TLB_shootdowns
58.25 ± 36% +315.9% 242.25 ± 13% interrupts.CPU220.TLB:TLB_shootdowns
230.50 ± 35% +103.9% 470.00 ± 18% interrupts.CPU221.RES:Rescheduling_interrupts
61.00 ± 40% +254.1% 216.00 ± 3% interrupts.CPU221.TLB:TLB_shootdowns
114.75 ± 47% +102.6% 232.50 ± 22% interrupts.CPU223.TLB:TLB_shootdowns
94.25 ± 19% +146.9% 232.75 ± 15% interrupts.CPU224.TLB:TLB_shootdowns
92.75 ± 47% +142.6% 225.00 ± 12% interrupts.CPU225.TLB:TLB_shootdowns
84.75 ± 19% +182.0% 239.00 ± 4% interrupts.CPU226.TLB:TLB_shootdowns
91.00 ± 34% +164.8% 241.00 ± 20% interrupts.CPU227.TLB:TLB_shootdowns
2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.NMI:Non-maskable_interrupts
2345 ± 32% -40.0% 1407 ± 17% interrupts.CPU228.PMI:Performance_monitoring_interrupts
61.75 ± 24% +232.0% 205.00 ± 10% interrupts.CPU228.TLB:TLB_shootdowns
75.00 ± 49% +188.7% 216.50 ± 14% interrupts.CPU229.TLB:TLB_shootdowns
706.25 ± 17% -38.5% 434.00 ± 19% interrupts.CPU23.TLB:TLB_shootdowns
78.50 ± 20% +208.6% 242.25 ± 11% interrupts.CPU230.TLB:TLB_shootdowns
67.50 ± 35% +214.4% 212.25 ± 20% interrupts.CPU231.TLB:TLB_shootdowns
204.25 ± 15% +141.9% 494.00 ± 11% interrupts.CPU232.RES:Rescheduling_interrupts
65.25 ± 19% +254.0% 231.00 ± 13% interrupts.CPU232.TLB:TLB_shootdowns
104.75 ± 28% +106.9% 216.75 ± 26% interrupts.CPU233.TLB:TLB_shootdowns
65.00 ± 44% +226.9% 212.50 ± 8% interrupts.CPU234.TLB:TLB_shootdowns
72.00 ± 53% +186.8% 206.50 ± 20% interrupts.CPU235.TLB:TLB_shootdowns
262.50 ± 38% +75.3% 460.25 ± 13% interrupts.CPU236.RES:Rescheduling_interrupts
72.75 ± 61% +195.5% 215.00 ± 7% interrupts.CPU236.TLB:TLB_shootdowns
78.25 ± 21% +187.2% 224.75 ± 32% interrupts.CPU237.TLB:TLB_shootdowns
92.25 ± 27% +166.9% 246.25 ± 17% interrupts.CPU238.TLB:TLB_shootdowns
120.50 ± 37% +118.5% 263.25 ± 19% interrupts.CPU239.TLB:TLB_shootdowns
871.75 ± 7% -42.3% 502.75 ± 12% interrupts.CPU24.TLB:TLB_shootdowns
63.75 ± 34% +271.0% 236.50 ± 6% interrupts.CPU240.TLB:TLB_shootdowns
77.25 ± 29% +207.4% 237.50 ± 15% interrupts.CPU241.TLB:TLB_shootdowns
49.75 ± 29% +310.6% 204.25 ± 13% interrupts.CPU242.TLB:TLB_shootdowns
76.00 ± 19% +194.7% 224.00 ± 17% interrupts.CPU243.TLB:TLB_shootdowns
80.75 ± 10% +183.6% 229.00 ± 17% interrupts.CPU244.TLB:TLB_shootdowns
86.75 ± 57% +171.8% 235.75 ± 8% interrupts.CPU245.TLB:TLB_shootdowns
56.50 ± 51% +344.2% 251.00 ± 13% interrupts.CPU246.TLB:TLB_shootdowns
270.50 ± 24% +153.1% 684.75 ± 34% interrupts.CPU247.RES:Rescheduling_interrupts
88.25 ± 24% +180.2% 247.25 ± 18% interrupts.CPU247.TLB:TLB_shootdowns
81.25 ± 37% +167.1% 217.00 ± 20% interrupts.CPU248.TLB:TLB_shootdowns
67.00 ± 26% +216.4% 212.00 ± 11% interrupts.CPU249.TLB:TLB_shootdowns
740.75 ± 11% -36.9% 467.25 ± 7% interrupts.CPU25.TLB:TLB_shootdowns
67.50 ± 29% +247.0% 234.25 ± 11% interrupts.CPU250.TLB:TLB_shootdowns
92.00 ± 63% +111.1% 194.25 ± 24% interrupts.CPU251.TLB:TLB_shootdowns
72.25 ± 45% +223.5% 233.75 ± 16% interrupts.CPU252.TLB:TLB_shootdowns
78.75 ± 29% +178.7% 219.50 ± 12% interrupts.CPU253.TLB:TLB_shootdowns
64.25 ± 58% +258.4% 230.25 ± 11% interrupts.CPU254.TLB:TLB_shootdowns
267.00 ± 17% +44.3% 385.25 ± 13% interrupts.CPU255.RES:Rescheduling_interrupts
114.25 ± 36% +106.1% 235.50 ± 12% interrupts.CPU255.TLB:TLB_shootdowns
74.00 ± 33% +213.9% 232.25 ± 13% interrupts.CPU256.TLB:TLB_shootdowns
86.00 ± 81% +158.4% 222.25 ± 19% interrupts.CPU257.TLB:TLB_shootdowns
1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.NMI:Non-maskable_interrupts
1565 ± 38% +54.4% 2417 ± 27% interrupts.CPU258.PMI:Performance_monitoring_interrupts
252.00 ± 25% +118.6% 550.75 ± 54% interrupts.CPU258.RES:Rescheduling_interrupts
81.75 ± 21% +192.7% 239.25 ± 9% interrupts.CPU258.TLB:TLB_shootdowns
106.75 ± 37% +109.1% 223.25 ± 20% interrupts.CPU259.TLB:TLB_shootdowns
899.75 ± 8% -42.9% 514.00 ± 6% interrupts.CPU26.TLB:TLB_shootdowns
67.25 ± 41% +291.8% 263.50 ± 22% interrupts.CPU260.TLB:TLB_shootdowns
59.25 ± 34% +289.0% 230.50 ± 8% interrupts.CPU261.TLB:TLB_shootdowns
292.75 ± 12% +61.0% 471.25 ± 16% interrupts.CPU262.RES:Rescheduling_interrupts
49.75 ± 45% +400.0% 248.75 ± 14% interrupts.CPU262.TLB:TLB_shootdowns
325.00 ± 17% +52.1% 494.25 ± 27% interrupts.CPU263.RES:Rescheduling_interrupts
87.00 ± 32% +143.1% 211.50 ± 11% interrupts.CPU263.TLB:TLB_shootdowns
218.25 ± 14% +71.7% 374.75 ± 5% interrupts.CPU264.RES:Rescheduling_interrupts
75.75 ± 28% +188.8% 218.75 ± 9% interrupts.CPU264.TLB:TLB_shootdowns
56.25 ± 55% +306.2% 228.50 ± 20% interrupts.CPU265.TLB:TLB_shootdowns
80.75 ± 53% +221.1% 259.25 ± 12% interrupts.CPU266.TLB:TLB_shootdowns
271.25 ± 27% +56.2% 423.75 ± 22% interrupts.CPU267.RES:Rescheduling_interrupts
60.25 ± 48% +352.7% 272.75 ± 16% interrupts.CPU267.TLB:TLB_shootdowns
56.00 ± 43% +295.5% 221.50 ± 20% interrupts.CPU268.TLB:TLB_shootdowns
97.25 ± 14% +151.2% 244.25 ± 17% interrupts.CPU269.TLB:TLB_shootdowns
688.00 ± 13% -32.2% 466.25 ± 11% interrupts.CPU27.TLB:TLB_shootdowns
208.00 ± 27% +93.6% 402.75 ± 8% interrupts.CPU270.RES:Rescheduling_interrupts
57.50 ± 30% +368.3% 269.25 ± 19% interrupts.CPU270.TLB:TLB_shootdowns
73.75 ± 34% +288.5% 286.50 ± 14% interrupts.CPU271.TLB:TLB_shootdowns
279.25 ± 32% +37.3% 383.50 ± 10% interrupts.CPU272.RES:Rescheduling_interrupts
79.25 ± 14% +189.6% 229.50 ± 28% interrupts.CPU272.TLB:TLB_shootdowns
216.50 ± 15% +201.8% 653.50 ± 66% interrupts.CPU273.RES:Rescheduling_interrupts
65.50 ± 22% +256.1% 233.25 ± 11% interrupts.CPU273.TLB:TLB_shootdowns
74.75 ± 68% +198.3% 223.00 ± 17% interrupts.CPU274.TLB:TLB_shootdowns
68.50 ± 42% +204.4% 208.50 ± 10% interrupts.CPU275.TLB:TLB_shootdowns
59.75 ± 35% +226.8% 195.25 ± 13% interrupts.CPU276.TLB:TLB_shootdowns
292.25 ± 35% +80.2% 526.75 ± 32% interrupts.CPU277.RES:Rescheduling_interrupts
73.25 ± 39% +233.8% 244.50 ± 14% interrupts.CPU277.TLB:TLB_shootdowns
61.50 ± 60% +248.0% 214.00 ± 9% interrupts.CPU278.TLB:TLB_shootdowns
163.25 ± 15% +188.1% 470.25 ± 18% interrupts.CPU279.RES:Rescheduling_interrupts
81.50 ± 10% +142.0% 197.25 ± 8% interrupts.CPU279.TLB:TLB_shootdowns
372.75 ± 20% +139.3% 892.00 ± 59% interrupts.CPU28.RES:Rescheduling_interrupts
891.00 ± 4% -41.6% 520.50 ± 10% interrupts.CPU28.TLB:TLB_shootdowns
75.75 ± 23% +223.8% 245.25 ± 12% interrupts.CPU280.TLB:TLB_shootdowns
66.25 ± 39% +234.0% 221.25 ± 12% interrupts.CPU281.TLB:TLB_shootdowns
239.25 ± 24% +90.9% 456.75 ± 15% interrupts.CPU282.RES:Rescheduling_interrupts
63.50 ± 36% +281.9% 242.50 ± 11% interrupts.CPU282.TLB:TLB_shootdowns
75.50 ± 39% +215.2% 238.00 ± 10% interrupts.CPU283.TLB:TLB_shootdowns
269.75 ± 41% +72.2% 464.50 ± 20% interrupts.CPU284.RES:Rescheduling_interrupts
66.25 ± 51% +251.7% 233.00 ± 22% interrupts.CPU284.TLB:TLB_shootdowns
103.75 ± 33% +117.3% 225.50 ± 6% interrupts.CPU285.TLB:TLB_shootdowns
274.75 ± 25% +42.6% 391.75 ± 9% interrupts.CPU286.RES:Rescheduling_interrupts
102.50 ± 22% +106.1% 211.25 ± 3% interrupts.CPU286.TLB:TLB_shootdowns
1604 ± 9% -17.7% 1320 interrupts.CPU287.CAL:Function_call_interrupts
1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.NMI:Non-maskable_interrupts
1860 ± 18% -26.0% 1376 ± 16% interrupts.CPU287.PMI:Performance_monitoring_interrupts
245.50 ± 19% +54.3% 378.75 ± 8% interrupts.CPU287.RES:Rescheduling_interrupts
685.00 ± 13% -35.5% 441.75 ± 4% interrupts.CPU29.TLB:TLB_shootdowns
689.75 ± 18% -32.4% 466.00 ± 6% interrupts.CPU3.TLB:TLB_shootdowns
399.75 ± 15% +149.8% 998.75 ± 69% interrupts.CPU30.RES:Rescheduling_interrupts
793.00 ± 16% -36.1% 506.75 ± 7% interrupts.CPU30.TLB:TLB_shootdowns
3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.NMI:Non-maskable_interrupts
3182 ± 24% -56.9% 1372 ± 6% interrupts.CPU31.PMI:Performance_monitoring_interrupts
283.00 ± 10% +134.5% 663.50 ± 31% interrupts.CPU31.RES:Rescheduling_interrupts
783.50 ± 20% -43.8% 440.25 ± 3% interrupts.CPU31.TLB:TLB_shootdowns
905.25 ± 8% -39.7% 546.25 ± 19% interrupts.CPU32.TLB:TLB_shootdowns
709.25 ± 4% -35.0% 461.00 ± 14% interrupts.CPU33.TLB:TLB_shootdowns
474.50 ± 31% +92.7% 914.50 ± 50% interrupts.CPU34.RES:Rescheduling_interrupts
879.50 ± 11% -43.0% 501.50 ± 9% interrupts.CPU34.TLB:TLB_shootdowns
750.00 ± 12% -42.5% 431.25 ± 10% interrupts.CPU35.TLB:TLB_shootdowns
889.00 ± 5% -45.4% 485.25 ± 6% interrupts.CPU36.TLB:TLB_shootdowns
415.25 ± 26% +34.3% 557.75 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts
818.50 ± 14% -45.1% 449.25 ± 14% interrupts.CPU37.TLB:TLB_shootdowns
453.50 ± 19% +23.7% 561.00 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts
899.00 ± 5% -37.8% 558.75 ± 7% interrupts.CPU38.TLB:TLB_shootdowns
752.25 ± 14% -41.3% 441.25 ± 11% interrupts.CPU39.TLB:TLB_shootdowns
1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.NMI:Non-maskable_interrupts
1502 ± 7% +48.7% 2234 ± 24% interrupts.CPU4.PMI:Performance_monitoring_interrupts
1012 ± 26% -47.3% 534.00 ± 6% interrupts.CPU4.TLB:TLB_shootdowns
882.25 ± 14% -41.4% 517.00 ± 17% interrupts.CPU40.TLB:TLB_shootdowns
762.50 ± 13% -35.8% 489.25 ± 4% interrupts.CPU41.TLB:TLB_shootdowns
995.75 ± 12% -49.2% 506.00 ± 8% interrupts.CPU42.TLB:TLB_shootdowns
652.50 ± 12% -35.4% 421.25 ± 11% interrupts.CPU43.TLB:TLB_shootdowns
421.25 ± 18% +102.1% 851.25 ± 47% interrupts.CPU44.RES:Rescheduling_interrupts
819.00 ± 10% -39.3% 497.25 ± 7% interrupts.CPU44.TLB:TLB_shootdowns
369.00 ± 21% +79.1% 660.75 ± 22% interrupts.CPU45.RES:Rescheduling_interrupts
801.00 ± 7% -41.6% 467.75 ± 8% interrupts.CPU45.TLB:TLB_shootdowns
786.75 ± 15% -38.5% 483.75 ± 12% interrupts.CPU46.TLB:TLB_shootdowns
716.75 ± 13% -35.4% 463.25 ± 11% interrupts.CPU47.TLB:TLB_shootdowns
813.25 ± 9% -34.6% 531.75 ± 6% interrupts.CPU48.TLB:TLB_shootdowns
658.50 ± 6% -26.2% 486.25 ± 6% interrupts.CPU49.TLB:TLB_shootdowns
735.25 ± 20% -36.6% 466.00 ± 18% interrupts.CPU5.TLB:TLB_shootdowns
852.50 ± 17% -43.0% 485.75 ± 9% interrupts.CPU50.TLB:TLB_shootdowns
815.75 ± 8% -40.3% 487.25 ± 12% interrupts.CPU51.TLB:TLB_shootdowns
899.50 ± 9% -42.5% 517.50 ± 10% interrupts.CPU52.TLB:TLB_shootdowns
654.50 ± 9% -33.8% 433.00 ± 19% interrupts.CPU53.TLB:TLB_shootdowns
920.00 ± 8% -40.9% 543.50 ± 6% interrupts.CPU54.TLB:TLB_shootdowns
687.25 ± 19% -31.5% 470.75 ± 16% interrupts.CPU55.TLB:TLB_shootdowns
2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.NMI:Non-maskable_interrupts
2467 ± 22% -41.5% 1442 ± 13% interrupts.CPU56.PMI:Performance_monitoring_interrupts
342.00 ± 7% +106.1% 705.00 ± 30% interrupts.CPU56.RES:Rescheduling_interrupts
884.75 ± 19% -45.2% 485.25 ± 8% interrupts.CPU56.TLB:TLB_shootdowns
772.00 ± 11% -43.0% 440.00 ± 11% interrupts.CPU57.TLB:TLB_shootdowns
942.50 ± 11% -46.3% 506.00 ± 10% interrupts.CPU58.TLB:TLB_shootdowns
754.25 ± 5% -38.0% 467.50 interrupts.CPU59.TLB:TLB_shootdowns
393.75 ± 9% +311.4% 1619 ±109% interrupts.CPU6.RES:Rescheduling_interrupts
860.25 ± 12% -46.6% 459.00 ± 7% interrupts.CPU6.TLB:TLB_shootdowns
894.50 ± 16% -43.4% 506.00 ± 13% interrupts.CPU60.TLB:TLB_shootdowns
1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.NMI:Non-maskable_interrupts
1684 ± 32% +78.1% 2999 ± 14% interrupts.CPU61.PMI:Performance_monitoring_interrupts
693.50 ± 17% -34.9% 451.75 ± 9% interrupts.CPU61.TLB:TLB_shootdowns
936.25 ± 16% -49.3% 474.75 ± 8% interrupts.CPU62.TLB:TLB_shootdowns
421.25 ± 44% +112.9% 897.00 ± 11% interrupts.CPU63.RES:Rescheduling_interrupts
731.25 ± 8% -41.1% 431.00 ± 7% interrupts.CPU63.TLB:TLB_shootdowns
883.00 ± 15% -43.7% 496.75 ± 7% interrupts.CPU64.TLB:TLB_shootdowns
736.00 ± 11% -36.8% 465.50 ± 5% interrupts.CPU65.TLB:TLB_shootdowns
880.00 ± 7% -42.6% 505.25 ± 8% interrupts.CPU66.TLB:TLB_shootdowns
943.50 ± 11% -40.5% 561.75 ± 10% interrupts.CPU68.TLB:TLB_shootdowns
393.25 ± 36% +100.5% 788.50 ± 26% interrupts.CPU69.RES:Rescheduling_interrupts
790.25 ± 9% -38.3% 487.50 ± 8% interrupts.CPU69.TLB:TLB_shootdowns
415.50 ± 24% +407.0% 2106 ±123% interrupts.CPU7.RES:Rescheduling_interrupts
679.00 ± 14% -36.8% 429.25 interrupts.CPU7.TLB:TLB_shootdowns
350.25 ± 13% +92.1% 672.75 ± 12% interrupts.CPU70.RES:Rescheduling_interrupts
778.75 ± 3% -41.8% 453.50 ± 11% interrupts.CPU70.TLB:TLB_shootdowns
673.50 ± 8% -39.8% 405.75 ± 11% interrupts.CPU71.TLB:TLB_shootdowns
460.75 ± 74% +95.7% 901.75 ± 43% interrupts.CPU74.RES:Rescheduling_interrupts
160.75 ± 33% +82.1% 292.75 ± 15% interrupts.CPU76.TLB:TLB_shootdowns
198.00 ± 22% +45.8% 288.75 ± 9% interrupts.CPU77.TLB:TLB_shootdowns
2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.NMI:Non-maskable_interrupts
2305 ± 26% -35.5% 1486 ± 12% interrupts.CPU79.PMI:Performance_monitoring_interrupts
155.25 ± 31% +72.9% 268.50 ± 8% interrupts.CPU79.TLB:TLB_shootdowns
887.25 ± 17% -45.4% 484.50 ± 10% interrupts.CPU8.TLB:TLB_shootdowns
192.00 ± 18% +32.9% 255.25 ± 12% interrupts.CPU80.TLB:TLB_shootdowns
158.75 ± 30% +95.1% 309.75 ± 12% interrupts.CPU82.TLB:TLB_shootdowns
265.75 ± 35% +120.8% 586.75 ± 30% interrupts.CPU84.RES:Rescheduling_interrupts
146.75 ± 29% +80.2% 264.50 ± 20% interrupts.CPU84.TLB:TLB_shootdowns
186.00 ± 31% +56.0% 290.25 ± 9% interrupts.CPU86.TLB:TLB_shootdowns
169.25 ± 15% +66.5% 281.75 ± 2% interrupts.CPU88.TLB:TLB_shootdowns
879.75 ± 10% -47.5% 462.00 ± 4% interrupts.CPU9.TLB:TLB_shootdowns
161.00 ± 20% +75.8% 283.00 ± 5% interrupts.CPU90.TLB:TLB_shootdowns
194.75 ± 12% +50.3% 292.75 ± 11% interrupts.CPU91.TLB:TLB_shootdowns
2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.NMI:Non-maskable_interrupts
2722 ± 34% -37.8% 1692 ± 6% interrupts.CPU92.PMI:Performance_monitoring_interrupts
190.75 ± 14% +36.3% 260.00 ± 22% interrupts.CPU92.TLB:TLB_shootdowns
188.00 ± 15% +57.0% 295.25 ± 10% interrupts.CPU94.TLB:TLB_shootdowns
200.25 ± 11% +59.6% 319.50 ± 16% interrupts.CPU95.TLB:TLB_shootdowns
316.25 ± 55% +105.1% 648.50 ± 19% interrupts.CPU96.RES:Rescheduling_interrupts
143.75 ± 42% +91.0% 274.50 ± 8% interrupts.CPU96.TLB:TLB_shootdowns
2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.NMI:Non-maskable_interrupts
2409 ± 41% -34.2% 1586 ± 46% interrupts.CPU97.PMI:Performance_monitoring_interrupts
186.00 ± 33% +56.0% 290.25 ± 16% interrupts.CPU98.TLB:TLB_shootdowns
vm-scalability.time.user_time
16000 +-------------------------------------------------------------------+
| + |
15000 |-+ :+ + + + |
14000 |-+ .++. +. + + +.+ :+ .++ ++. +.++ + + :+ + +|
|+.+++ ++ + + + ++.+++ +.+++.++.+ ++ + +.+ + :+ |
13000 |-+ + |
| |
12000 |-+ |
| |
11000 |-+ |
10000 |-+ |
| OO O O |
9000 |-+ OO O OO O OO O |
|O OOO O O OO |
8000 +-------------------------------------------------------------------+
vm-scalability.time.voluntary_context_switches
1e+06 +------------------------------------------------------------------+
| O O |
900000 |O+OOO OO O OOO OOO O O OO |
800000 |-+ |
| |
700000 |-+ |
600000 |-+ |
| |
500000 |-+ |
400000 |-+ |
| |
300000 |-+ |
200000 |-+ |
|+.+++.+++.+ + +.+++.++.+++.+++.+++.+++. .+ +. |
100000 +------------------------------------------------------------------+
vm-scalability.throughput
1.6e+07 +-----------------------------------------------------------------+
| |
1.5e+07 |-+ + + + |
1.4e+07 |+. + + +. + + +. + + |
| +++ +.+ +.+++ .++ +++ +.++ +++ +.+ +. |
1.3e+07 |-+ + +.+++ +++.++ +++.+++.+|
1.2e+07 |-+ |
| |
1.1e+07 |-+ |
1e+07 |-+ |
| |
9e+06 |-+ O O O O |
8e+06 |-+ O OO O OO O OO OO O |
|O O O O O |
7e+06 +-----------------------------------------------------------------+
vm-scalability.median
44000 +-------------------------------------------------------------------+
42000 |-+ + + + |
| : : : : :: |
40000 |+. : +. + ++.+ .++. : + .+++. : +. + |
38000 |-++++ + +.+ ++.+++ +++ + ++.+ + +. .+++. +. ++. |
| +++ + + +|
36000 |-+ |
34000 |-+ |
32000 |-+ |
| |
30000 |-+ |
28000 |-+ O O O |
| O O O O |
26000 |O+OOO O O OO OOO O O O |
24000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months
[fs] b6509f6a8c: will-it-scale.per_thread_ops -12.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -12.6% regression of will-it-scale.per_thread_ops due to commit:
commit: b6509f6a8c4313c068c69785c001451415969e44 ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
nr_task: 100%
mode: thread
test: eventfd1
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_process_ops -6.4% regression |
| test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | mode=process |
| | nr_task=100% |
| | test=unix1 |
| | ucode=0x5002f01 |
+------------------+---------------------------------------------------------------------------+
| testcase: change | will-it-scale: will-it-scale.per_thread_ops -2.3% regression |
| test machine | 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory |
| test parameters | cpufreq_governor=performance |
| | mode=thread |
| | nr_task=100% |
| | test=pipe1 |
| | ucode=0x5002f01 |
+------------------+---------------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/eventfd1/will-it-scale/0x5002f01
commit:
v5.8-rc3
b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"")
v5.8-rc3 b6509f6a8c4313c068c69785c00
---------------- ---------------------------
%stddev %change %stddev
\ | \
1652352 -12.6% 1444002 ± 2% will-it-scale.per_thread_ops
3.173e+08 -12.6% 2.772e+08 ± 2% will-it-scale.workload
23.95 -3.1 20.90 ± 2% mpstat.cpu.all.usr%
23.00 -12.0% 20.25 ± 4% vmstat.cpu.us
578849 ± 4% +8.6% 628428 ± 4% sched_debug.cpu.max_idle_balance_cost.max
0.00 ± 9% -14.4% 0.00 ± 13% sched_debug.cpu.next_balance.stddev
2207 ± 24% +34.7% 2973 ± 20% numa-vmstat.node0.nr_mapped
6949 ± 22% +43.7% 9984 ± 10% numa-vmstat.node0.nr_slab_reclaimable
9739 ± 13% -29.0% 6913 ± 12% numa-vmstat.node1.nr_slab_reclaimable
7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.active_objs
7247 ± 13% -28.4% 5190 ± 19% slabinfo.eventpoll_pwq.num_objs
960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.active_objs
960.00 ± 11% -40.0% 576.00 ± 26% slabinfo.kmalloc-rcl-128.num_objs
27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.KReclaimable
8718 ± 25% +35.0% 11773 ± 20% numa-meminfo.node0.Mapped
27801 ± 22% +43.7% 39940 ± 10% numa-meminfo.node0.SReclaimable
38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.KReclaimable
38959 ± 13% -29.0% 27656 ± 12% numa-meminfo.node1.SReclaimable
107406 ± 2% -13.6% 92844 ± 6% numa-meminfo.node1.Slab
0.00 +1.1 1.09 ± 16% perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.3 1.26 ± 18% perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.00 +1.8 1.76 ± 21% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.7 3.70 ± 20% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +3.0 3.02 ± 17% perf-profile.children.cycles-pp.fsnotify_parent
0.00 +5.5 5.55 ± 20% perf-profile.children.cycles-pp.fsnotify
0.00 +2.7 2.66 ± 17% perf-profile.self.cycles-pp.fsnotify_parent
0.00 +5.4 5.35 ± 20% perf-profile.self.cycles-pp.fsnotify
10890 ± 21% +53.7% 16738 ± 12% interrupts.CPU168.CAL:Function_call_interrupts
4602 ± 5% +22.7% 5649 ± 13% interrupts.CPU169.CAL:Function_call_interrupts
4920 ± 13% -21.3% 3874 ± 2% interrupts.CPU170.CAL:Function_call_interrupts
463.80 ± 11% -25.1% 347.50 ± 4% interrupts.CPU170.RES:Rescheduling_interrupts
3619 ± 2% +10.2% 3990 ± 9% interrupts.CPU36.CAL:Function_call_interrupts
5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.NMI:Non-maskable_interrupts
5768 ± 34% +43.3% 8267 ± 2% interrupts.CPU74.PMI:Performance_monitoring_interrupts
5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.NMI:Non-maskable_interrupts
5769 ± 34% +43.3% 8268 ± 2% interrupts.CPU75.PMI:Performance_monitoring_interrupts
5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.NMI:Non-maskable_interrupts
5769 ± 34% +43.3% 8267 ± 2% interrupts.CPU76.PMI:Performance_monitoring_interrupts
5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.NMI:Non-maskable_interrupts
5768 ± 34% +43.3% 8268 ± 2% interrupts.CPU77.PMI:Performance_monitoring_interrupts
3602 ± 2% +12.9% 4068 ± 13% interrupts.CPU81.CAL:Function_call_interrupts
0.05 ± 32% +121.5% 0.10 ± 39% perf-stat.i.MPKI
1.06 -0.1 0.94 perf-stat.i.branch-miss-rate%
1.107e+09 -13.3% 9.596e+08 ± 2% perf-stat.i.branch-misses
199835 -11.5% 176905 ± 2% perf-stat.i.dTLB-store-misses
9.558e+10 +9.4% 1.046e+11 ± 2% perf-stat.i.dTLB-stores
9.687e+08 -13.8% 8.345e+08 ± 2% perf-stat.i.iTLB-load-misses
535.53 +20.3% 644.34 perf-stat.i.instructions-per-iTLB-miss
1.10 ± 12% +79.6% 1.98 ± 2% perf-stat.i.metric.K/sec
1.05 -0.1 0.92 perf-stat.overall.branch-miss-rate%
0.00 ± 13% -0.0 0.00 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate%
528.82 +20.8% 638.59 perf-stat.overall.instructions-per-iTLB-miss
487060 +20.3% 586095 perf-stat.overall.path-length
1.103e+09 -13.3% 9.564e+08 ± 2% perf-stat.ps.branch-misses
199661 -11.5% 176727 ± 2% perf-stat.ps.dTLB-store-misses
9.525e+10 +9.5% 1.043e+11 ± 2% perf-stat.ps.dTLB-stores
9.67e+08 -13.9% 8.33e+08 ± 3% perf-stat.ps.iTLB-load-misses
will-it-scale.per_thread_ops
1.7e+06 +----------------------------------------------------------------+
| .+.. .+.+.. .+.. .+ |
1.65e+06 |.+ .+.+ + +.+. .+.+ +. .+.+. |
| +. +. + |
1.6e+06 |-+ |
| |
1.55e+06 |-+ |
| |
1.5e+06 |-+ |
| O O O O O O |
1.45e+06 |-O O O O O O O O O O O O O O O O O O O O |
| O |
1.4e+06 |-+ |
| |
1.35e+06 +----------------------------------------------------------------+
will-it-scale.workload
3.3e+08 +-----------------------------------------------------------------+
| |
3.2e+08 |.+ .+.+..+.+..+.+.+.. .+..+.+. .+.+ |
| +..+ +.+ +..+ |
3.1e+08 |-+ |
| |
3e+08 |-+ |
| |
2.9e+08 |-+ |
| |
2.8e+08 |-O O O O O O O O O O O O O O O O O O O O O O O O |
| O O O |
2.7e+08 |-+ |
| |
2.6e+08 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/unix1/will-it-scale/0x5002f01
commit:
v5.8-rc3
b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"")
v5.8-rc3 b6509f6a8c4313c068c69785c00
---------------- ---------------------------
%stddev %change %stddev
\ | \
720286 -6.4% 674507 will-it-scale.per_process_ops
1.383e+08 -6.4% 1.295e+08 will-it-scale.workload
97977 +1.4% 99342 proc-vmstat.nr_anon_pages
8.31 ± 41% +48.5% 12.33 ± 3% sched_debug.cpu.sched_goidle.min
1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.active_objs
1097 ± 8% +19.8% 1315 ± 9% slabinfo.task_group.num_objs
13194 ± 25% -29.6% 9295 ± 24% softirqs.CPU124.RCU
9178 ± 8% -7.8% 8467 ± 4% softirqs.CPU48.RCU
1229 ± 68% +156.3% 3151 ± 70% interrupts.CPU0.CAL:Function_call_interrupts
8259 -37.6% 5157 ± 34% interrupts.CPU163.NMI:Non-maskable_interrupts
8259 -37.6% 5157 ± 34% interrupts.CPU163.PMI:Performance_monitoring_interrupts
15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_active_anon
15187 ± 81% +147.5% 37594 ± 45% numa-vmstat.node2.nr_anon_pages
15337 ± 81% +146.4% 37788 ± 45% numa-vmstat.node2.nr_zone_active_anon
127540 ± 6% -8.0% 117311 numa-vmstat.node2.numa_other
62308 ± 81% +144.0% 152033 ± 45% numa-meminfo.node2.Active
61336 ± 81% +146.7% 151288 ± 45% numa-meminfo.node2.Active(anon)
15647 ±195% +497.8% 93536 ± 57% numa-meminfo.node2.AnonHugePages
60733 ± 82% +147.8% 150509 ± 45% numa-meminfo.node2.AnonPages
803985 ± 8% +10.5% 888495 ± 9% numa-meminfo.node2.MemUsed
1.14 -0.1 1.07 perf-stat.i.branch-miss-rate%
1.223e+09 -6.3% 1.146e+09 perf-stat.i.branch-misses
0.99 -1.5% 0.98 perf-stat.i.cpi
1.619e+11 +1.4% 1.641e+11 perf-stat.i.dTLB-loads
1e+11 +4.0% 1.04e+11 perf-stat.i.dTLB-stores
1.06e+09 -7.8% 9.779e+08 perf-stat.i.iTLB-load-misses
5.531e+11 +1.3% 5.603e+11 perf-stat.i.instructions
527.69 +9.7% 578.78 perf-stat.i.instructions-per-iTLB-miss
1.01 +1.4% 1.02 perf-stat.i.ipc
1931 +1.4% 1958 perf-stat.i.metric.M/sec
1.12 -0.1 1.06 perf-stat.overall.branch-miss-rate%
0.99 -1.4% 0.97 perf-stat.overall.cpi
0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate%
521.71 +9.8% 572.92 perf-stat.overall.instructions-per-iTLB-miss
1.01 +1.4% 1.03 perf-stat.overall.ipc
1206072 +8.1% 1303735 perf-stat.overall.path-length
1.218e+09 -6.3% 1.142e+09 perf-stat.ps.branch-misses
1.613e+11 +1.4% 1.635e+11 perf-stat.ps.dTLB-loads
9.97e+10 +4.0% 1.036e+11 perf-stat.ps.dTLB-stores
1.057e+09 -7.8% 9.747e+08 perf-stat.ps.iTLB-load-misses
5.512e+11 +1.3% 5.584e+11 perf-stat.ps.instructions
1.668e+14 +1.2% 1.688e+14 perf-stat.total.instructions
38.39 -2.2 36.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
36.80 -2.1 34.66 perf-profile.calltrace.cycles-pp.sock_write_iter.new_sync_write.vfs_write.ksys_write.do_syscall_64
34.46 -2.0 32.44 perf-profile.calltrace.cycles-pp.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write.ksys_write
33.29 -1.8 31.47 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.96 -1.8 29.19 perf-profile.calltrace.cycles-pp.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
31.72 -1.7 30.05 perf-profile.calltrace.cycles-pp.sock_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
25.95 -1.5 24.46 perf-profile.calltrace.cycles-pp.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
53.97 -1.5 52.49 perf-profile.calltrace.cycles-pp.write
24.74 -1.4 23.30 perf-profile.calltrace.cycles-pp.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read.vfs_read
47.12 -1.0 46.10 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write
46.34 -1.0 45.38 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
44.29 -0.9 43.43 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
42.43 -0.8 41.64 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
13.05 -0.8 12.28 perf-profile.calltrace.cycles-pp.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
11.31 -0.7 10.65 perf-profile.calltrace.cycles-pp.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
10.31 -0.6 9.71 perf-profile.calltrace.cycles-pp.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg
8.93 -0.5 8.39 perf-profile.calltrace.cycles-pp.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
8.64 -0.5 8.14 perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
8.31 -0.5 7.85 perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic.unix_stream_recvmsg
7.91 -0.5 7.45 perf-profile.calltrace.cycles-pp.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
6.55 -0.4 6.19 perf-profile.calltrace.cycles-pp.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.54 ± 2% -0.3 0.25 ±100% perf-profile.calltrace.cycles-pp.___might_sleep.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg
3.81 -0.2 3.58 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.write
3.98 -0.2 3.75 perf-profile.calltrace.cycles-pp.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
3.63 -0.2 3.41 perf-profile.calltrace.cycles-pp._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
3.88 -0.2 3.65 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.read
3.82 -0.2 3.60 perf-profile.calltrace.cycles-pp.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
2.87 -0.2 2.66 perf-profile.calltrace.cycles-pp.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write.vfs_write
3.71 -0.2 3.50 perf-profile.calltrace.cycles-pp.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic.unix_stream_recvmsg
3.62 -0.2 3.42 perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
3.91 -0.2 3.71 perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor.unix_stream_read_generic
2.95 -0.2 2.76 perf-profile.calltrace.cycles-pp.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read
3.50 -0.2 3.31 perf-profile.calltrace.cycles-pp.__check_object_size.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
3.33 -0.2 3.15 perf-profile.calltrace.cycles-pp.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
3.44 -0.2 3.25 perf-profile.calltrace.cycles-pp.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb.unix_stream_read_generic
3.09 -0.2 2.91 perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
3.65 -0.2 3.47 perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
2.38 -0.2 2.21 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.08 -0.1 1.94 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
2.63 -0.1 2.50 perf-profile.calltrace.cycles-pp.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all.consume_skb
3.44 -0.1 3.31 perf-profile.calltrace.cycles-pp.sock_recvmsg.sock_read_iter.new_sync_read.vfs_read.ksys_read
1.71 -0.1 1.59 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
1.69 -0.1 1.58 perf-profile.calltrace.cycles-pp.common_file_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
1.96 -0.1 1.85 perf-profile.calltrace.cycles-pp.aa_sk_perm.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read
1.44 -0.1 1.34 perf-profile.calltrace.cycles-pp.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
1.57 -0.1 1.48 perf-profile.calltrace.cycles-pp.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg
1.47 -0.1 1.39 perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.unix_stream_read_actor
1.20 -0.1 1.13 perf-profile.calltrace.cycles-pp.sock_def_readable.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.70 -0.1 0.63 ± 4% perf-profile.calltrace.cycles-pp.__might_sleep.kmem_cache_alloc_node.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb
1.10 ± 2% -0.1 1.03 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.98 -0.1 0.91 perf-profile.calltrace.cycles-pp.skb_set_owner_w.sock_alloc_send_pskb.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
1.01 ± 2% -0.1 0.94 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.01 -0.1 0.94 perf-profile.calltrace.cycles-pp.__might_fault._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg.sock_sendmsg
1.13 -0.1 1.07 perf-profile.calltrace.cycles-pp.unix_write_space.sock_wfree.unix_destruct_scm.skb_release_head_state.skb_release_all
1.05 -0.1 0.99 ± 3% perf-profile.calltrace.cycles-pp.__ksize.__alloc_skb.alloc_skb_with_frags.sock_alloc_send_pskb.unix_stream_sendmsg
0.96 -0.1 0.91 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.96 -0.1 0.90 perf-profile.calltrace.cycles-pp._raw_spin_unlock_irqrestore.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.98 -0.1 0.93 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.write
1.04 -0.1 0.98 perf-profile.calltrace.cycles-pp.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
1.00 -0.1 0.94 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.read
0.80 -0.1 0.75 perf-profile.calltrace.cycles-pp._raw_spin_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
1.02 -0.0 0.97 perf-profile.calltrace.cycles-pp.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter.new_sync_write
0.85 -0.0 0.80 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_unlink.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter
0.68 ± 2% -0.0 0.64 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.71 -0.0 0.67 perf-profile.calltrace.cycles-pp.___might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
0.70 ± 2% -0.0 0.66 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin._copy_from_iter.skb_copy_datagram_from_iter.unix_stream_sendmsg
0.85 -0.0 0.81 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.skb_queue_tail.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.56 -0.0 0.52 ± 2% perf-profile.calltrace.cycles-pp.apparmor_socket_recvmsg.security_socket_recvmsg.sock_recvmsg.sock_read_iter.new_sync_read
0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.___might_sleep.wait_for_unix_gc.unix_stream_sendmsg.sock_sendmsg.sock_write_iter
0.68 -0.0 0.64 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
0.57 -0.0 0.54 perf-profile.calltrace.cycles-pp.__might_sleep.__kmalloc_node_track_caller.__kmalloc_reserve.__alloc_skb.alloc_skb_with_frags
1.68 -0.0 1.64 perf-profile.calltrace.cycles-pp.mutex_lock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.65 -0.0 0.62 perf-profile.calltrace.cycles-pp.mutex_unlock.unix_stream_read_generic.unix_stream_recvmsg.sock_read_iter.new_sync_read
0.00 +0.6 0.57 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.6 0.62 perf-profile.calltrace.cycles-pp.fsnotify_parent.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.00 +0.6 0.64 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.50 +0.7 3.20 perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +0.9 0.93 perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
48.49 +1.3 49.77 perf-profile.calltrace.cycles-pp.read
41.56 +1.7 43.30 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
40.78 +1.8 42.56 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
38.81 +1.9 40.68 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
37.05 +1.9 38.94 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
0.00 +1.9 1.90 perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
38.49 -2.2 36.24 perf-profile.children.cycles-pp.new_sync_write
36.86 -2.1 34.71 perf-profile.children.cycles-pp.sock_write_iter
34.57 -2.0 32.53 perf-profile.children.cycles-pp.sock_sendmsg
33.36 -1.8 31.55 perf-profile.children.cycles-pp.new_sync_read
31.28 -1.8 29.50 perf-profile.children.cycles-pp.unix_stream_sendmsg
31.77 -1.7 30.10 perf-profile.children.cycles-pp.sock_read_iter
26.00 -1.5 24.50 perf-profile.children.cycles-pp.unix_stream_recvmsg
54.00 -1.5 52.53 perf-profile.children.cycles-pp.write
24.99 -1.5 23.53 perf-profile.children.cycles-pp.unix_stream_read_generic
44.34 -0.9 43.48 perf-profile.children.cycles-pp.ksys_write
13.14 -0.8 12.36 perf-profile.children.cycles-pp.sock_alloc_send_pskb
42.57 -0.7 41.85 perf-profile.children.cycles-pp.vfs_write
11.37 -0.7 10.71 perf-profile.children.cycles-pp.alloc_skb_with_frags
10.51 -0.6 9.90 perf-profile.children.cycles-pp.__alloc_skb
9.00 -0.5 8.46 perf-profile.children.cycles-pp.unix_stream_read_actor
8.72 -0.5 8.21 perf-profile.children.cycles-pp.skb_copy_datagram_iter
8.00 -0.5 7.53 perf-profile.children.cycles-pp.skb_copy_datagram_from_iter
8.38 -0.5 7.91 perf-profile.children.cycles-pp.__skb_datagram_iter
7.43 -0.4 7.04 perf-profile.children.cycles-pp.__check_object_size
6.57 -0.4 6.21 perf-profile.children.cycles-pp.consume_skb
5.14 -0.3 4.83 perf-profile.children.cycles-pp.entry_SYSCALL_64
4.60 -0.3 4.32 perf-profile.children.cycles-pp.syscall_return_via_sysret
4.16 -0.3 3.89 perf-profile.children.cycles-pp.aa_sk_perm
4.22 -0.3 3.96 perf-profile.children.cycles-pp.___might_sleep
3.50 -0.2 3.26 perf-profile.children.cycles-pp.common_file_perm
4.07 -0.2 3.83 perf-profile.children.cycles-pp.__kmalloc_reserve
3.71 -0.2 3.48 perf-profile.children.cycles-pp._copy_from_iter
3.85 -0.2 3.63 perf-profile.children.cycles-pp.skb_release_all
2.95 -0.2 2.73 perf-profile.children.cycles-pp.__might_sleep
2.91 -0.2 2.70 perf-profile.children.cycles-pp.security_socket_sendmsg
3.72 -0.2 3.51 perf-profile.children.cycles-pp._copy_to_iter
3.76 -0.2 3.55 perf-profile.children.cycles-pp.skb_release_head_state
3.01 -0.2 2.81 perf-profile.children.cycles-pp.security_socket_recvmsg
3.54 -0.2 3.33 perf-profile.children.cycles-pp.__kmalloc_node_track_caller
3.97 -0.2 3.77 perf-profile.children.cycles-pp.simple_copy_to_iter
3.28 -0.2 3.09 perf-profile.children.cycles-pp.kmem_cache_alloc_node
3.54 -0.2 3.35 perf-profile.children.cycles-pp.unix_destruct_scm
2.36 ± 2% -0.2 2.21 ± 2% perf-profile.children.cycles-pp.__fdget_pos
2.64 -0.1 2.50 perf-profile.children.cycles-pp.sock_wfree
3.50 -0.1 3.37 perf-profile.children.cycles-pp.sock_recvmsg
2.01 ± 2% -0.1 1.88 ± 3% perf-profile.children.cycles-pp.__fget_light
2.10 ± 2% -0.1 1.97 ± 2% perf-profile.children.cycles-pp.__might_fault
1.97 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
1.54 -0.1 1.44 perf-profile.children.cycles-pp.wait_for_unix_gc
1.95 -0.1 1.85 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.43 -0.1 1.34 perf-profile.children.cycles-pp._cond_resched
1.60 -0.1 1.51 perf-profile.children.cycles-pp.copyin
1.52 -0.1 1.43 perf-profile.children.cycles-pp.__prepare_exit_to_usermode
1.67 ± 3% -0.1 1.58 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
1.64 -0.1 1.55 perf-profile.children.cycles-pp.copy_user_generic_unrolled
1.50 -0.1 1.42 perf-profile.children.cycles-pp.copyout
1.21 -0.1 1.13 perf-profile.children.cycles-pp.sock_def_readable
0.98 -0.1 0.91 perf-profile.children.cycles-pp.skb_set_owner_w
0.95 -0.1 0.89 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
1.05 -0.1 0.99 ± 2% perf-profile.children.cycles-pp.__ksize
0.84 -0.1 0.78 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
1.13 -0.1 1.08 perf-profile.children.cycles-pp.unix_write_space
1.05 -0.1 0.99 perf-profile.children.cycles-pp.skb_unlink
0.74 ± 2% -0.1 0.69 perf-profile.children.cycles-pp.apparmor_file_permission
1.02 -0.1 0.97 perf-profile.children.cycles-pp.skb_queue_tail
1.75 -0.0 1.70 perf-profile.children.cycles-pp.mutex_lock
0.56 -0.0 0.52 ± 2% perf-profile.children.cycles-pp.apparmor_socket_recvmsg
0.71 -0.0 0.67 perf-profile.children.cycles-pp.rcu_all_qs
0.67 -0.0 0.63 perf-profile.children.cycles-pp.__syscall_return_slowpath
0.67 -0.0 0.64 perf-profile.children.cycles-pp.check_stack_object
0.44 -0.0 0.41 perf-profile.children.cycles-pp.apparmor_socket_sendmsg
0.67 -0.0 0.65 perf-profile.children.cycles-pp.mutex_unlock
0.30 -0.0 0.28 ± 2% perf-profile.children.cycles-pp.put_pid
0.23 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.skb_release_data
0.43 +0.0 0.46 perf-profile.children.cycles-pp.apparmor_socket_getpeersec_dgram
4.96 +0.6 5.57 perf-profile.children.cycles-pp.security_file_permission
88.79 +0.7 89.51 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
87.30 +0.8 88.11 perf-profile.children.cycles-pp.do_syscall_64
48.55 +1.3 49.85 perf-profile.children.cycles-pp.read
0.00 +1.9 1.87 perf-profile.children.cycles-pp.fsnotify_parent
38.85 +1.9 40.72 perf-profile.children.cycles-pp.ksys_read
37.17 +2.0 39.17 perf-profile.children.cycles-pp.vfs_read
0.00 +2.9 2.90 perf-profile.children.cycles-pp.fsnotify
4.56 -0.3 4.28 perf-profile.self.cycles-pp.syscall_return_via_sysret
4.10 -0.2 3.85 perf-profile.self.cycles-pp.___might_sleep
2.96 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_read_generic
2.65 -0.2 2.46 perf-profile.self.cycles-pp.__might_sleep
2.94 -0.2 2.75 perf-profile.self.cycles-pp.unix_stream_sendmsg
2.85 ± 2% -0.2 2.67 perf-profile.self.cycles-pp.common_file_perm
1.40 ± 4% -0.2 1.22 perf-profile.self.cycles-pp.new_sync_read
2.32 -0.2 2.17 perf-profile.self.cycles-pp.aa_sk_perm
2.56 -0.1 2.41 perf-profile.self.cycles-pp.entry_SYSCALL_64
1.90 ± 2% -0.1 1.77 ± 3% perf-profile.self.cycles-pp.__fget_light
2.13 -0.1 2.00 perf-profile.self.cycles-pp.__alloc_skb
2.29 -0.1 2.17 perf-profile.self.cycles-pp.sock_write_iter
1.93 -0.1 1.81 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
1.79 -0.1 1.67 perf-profile.self.cycles-pp.write
1.83 -0.1 1.72 perf-profile.self.cycles-pp.read
1.52 -0.1 1.42 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
1.71 -0.1 1.61 perf-profile.self.cycles-pp.__kmalloc_node_track_caller
1.93 -0.1 1.84 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.60 ± 3% -0.1 1.52 ± 2% perf-profile.self.cycles-pp._raw_spin_lock
1.81 -0.1 1.73 perf-profile.self.cycles-pp.kmem_cache_alloc_node
1.17 -0.1 1.09 perf-profile.self.cycles-pp.sock_def_readable
1.55 -0.1 1.47 perf-profile.self.cycles-pp.copy_user_generic_unrolled
1.25 ± 2% -0.1 1.17 ± 2% perf-profile.self.cycles-pp._copy_to_iter
0.61 -0.1 0.54 perf-profile.self.cycles-pp.ksys_write
1.44 -0.1 1.37 perf-profile.self.cycles-pp.sock_wfree
0.95 -0.1 0.88 perf-profile.self.cycles-pp.skb_set_owner_w
1.06 -0.1 0.99 ± 2% perf-profile.self.cycles-pp._copy_from_iter
0.58 ± 5% -0.1 0.52 ± 5% perf-profile.self.cycles-pp.aa_file_perm
1.04 -0.1 0.97 ± 2% perf-profile.self.cycles-pp.__ksize
1.10 -0.1 1.04 perf-profile.self.cycles-pp.unix_write_space
0.68 ± 2% -0.1 0.62 perf-profile.self.cycles-pp.__skb_datagram_iter
0.93 -0.1 0.87 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.52 -0.1 0.47 perf-profile.self.cycles-pp.ksys_read
0.68 ± 2% -0.1 0.63 ± 2% perf-profile.self.cycles-pp.apparmor_file_permission
0.83 -0.0 0.78 perf-profile.self.cycles-pp.unix_destruct_scm
0.87 -0.0 0.82 perf-profile.self.cycles-pp.consume_skb
0.83 -0.0 0.79 perf-profile.self.cycles-pp.sock_alloc_send_pskb
0.71 -0.0 0.67 perf-profile.self.cycles-pp._cond_resched
0.82 -0.0 0.78 perf-profile.self.cycles-pp.alloc_skb_with_frags
0.29 ± 3% -0.0 0.25 perf-profile.self.cycles-pp.unix_stream_read_actor
0.66 -0.0 0.62 perf-profile.self.cycles-pp.skb_copy_datagram_from_iter
0.53 -0.0 0.49 ± 2% perf-profile.self.cycles-pp.apparmor_socket_recvmsg
0.64 -0.0 0.60 perf-profile.self.cycles-pp.__syscall_return_slowpath
0.60 ± 2% -0.0 0.56 ± 4% perf-profile.self.cycles-pp.__might_fault
0.50 -0.0 0.46 perf-profile.self.cycles-pp.__kmalloc_reserve
0.37 -0.0 0.34 perf-profile.self.cycles-pp.security_socket_recvmsg
2.25 -0.0 2.21 perf-profile.self.cycles-pp.sock_read_iter
0.54 -0.0 0.51 perf-profile.self.cycles-pp.rcu_all_qs
0.37 ± 5% -0.0 0.34 ± 4% perf-profile.self.cycles-pp.sock_sendmsg
0.43 -0.0 0.40 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.98 -0.0 0.95 perf-profile.self.cycles-pp.unix_stream_recvmsg
0.35 -0.0 0.32 perf-profile.self.cycles-pp.skb_copy_datagram_iter
0.41 -0.0 0.38 perf-profile.self.cycles-pp.apparmor_socket_sendmsg
0.56 -0.0 0.54 perf-profile.self.cycles-pp.check_stack_object
0.44 -0.0 0.41 perf-profile.self.cycles-pp.wait_for_unix_gc
0.29 ± 3% -0.0 0.26 ± 2% perf-profile.self.cycles-pp.security_socket_sendmsg
0.42 -0.0 0.40 perf-profile.self.cycles-pp.security_socket_getpeersec_dgram
0.63 -0.0 0.61 perf-profile.self.cycles-pp.mutex_unlock
0.40 ± 2% -0.0 0.38 perf-profile.self.cycles-pp.__fdget_pos
0.31 -0.0 0.29 perf-profile.self.cycles-pp.copyin
0.23 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.skb_release_data
0.19 -0.0 0.17 ± 2% perf-profile.self.cycles-pp.skb_release_head_state
0.24 -0.0 0.23 perf-profile.self.cycles-pp.copyout
0.21 ± 2% -0.0 0.19 ± 2% perf-profile.self.cycles-pp.simple_copy_to_iter
0.26 -0.0 0.25 perf-profile.self.cycles-pp.kmalloc_slab
0.17 ± 2% +0.0 0.20 ± 2% perf-profile.self.cycles-pp.__vfs_read
0.40 +0.0 0.43 perf-profile.self.cycles-pp.apparmor_socket_getpeersec_dgram
0.73 +0.0 0.77 perf-profile.self.cycles-pp.mutex_lock
0.51 +0.1 0.57 perf-profile.self.cycles-pp.sock_recvmsg
1.33 +0.2 1.57 perf-profile.self.cycles-pp.vfs_write
0.75 ± 6% +0.4 1.10 ± 3% perf-profile.self.cycles-pp.security_file_permission
0.97 +0.5 1.49 ± 2% perf-profile.self.cycles-pp.vfs_read
0.00 +1.7 1.65 perf-profile.self.cycles-pp.fsnotify_parent
0.00 +2.8 2.80 perf-profile.self.cycles-pp.fsnotify
***************************************************************************************************
lkp-csl-2ap4: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-7.6/thread/100%/debian-x86_64-20191114.cgz/lkp-csl-2ap4/pipe1/will-it-scale/0x5002f01
commit:
v5.8-rc3
b6509f6a8c ("Revert "fs: Do not check if there is a fsnotify watcher on pseudo inodes"")
v5.8-rc3 b6509f6a8c4313c068c69785c00
---------------- ---------------------------
%stddev %change %stddev
\ | \
1045643 -2.3% 1021331 will-it-scale.per_thread_ops
2.008e+08 -2.3% 1.961e+08 will-it-scale.workload
29490176 ± 3% -9.8% 26607616 ± 3% meminfo.DirectMap2M
202721 ± 4% +11.5% 225943 ± 3% numa-numastat.node2.local_node
25892 ± 28% -75.5% 6341 ± 97% numa-numastat.node2.other_node
1843 +8.9% 2007 ± 5% numa-vmstat.node2.nr_mapped
130681 ± 5% -14.7% 111436 ± 5% numa-vmstat.node2.numa_other
9801 ± 5% +16.0% 11370 ± 7% sched_debug.cpu.ttwu_count.max
3052 ± 12% +23.6% 3771 ± 11% sched_debug.cpu.ttwu_local.max
136156 +1.4% 138002 proc-vmstat.nr_active_anon
109009 +1.1% 110229 proc-vmstat.nr_anon_pages
136156 +1.4% 138002 proc-vmstat.nr_zone_active_anon
1120468 +1.6% 1137854 proc-vmstat.pgalloc_normal
337.83 ± 61% +53.8% 519.75 ± 47% interrupts.39:PCI-MSI.524290-edge.eth0-TxRx-1
337.83 ± 61% +53.8% 519.75 ± 47% interrupts.CPU11.39:PCI-MSI.524290-edge.eth0-TxRx-1
303.33 +45.7% 442.00 ± 44% interrupts.CPU148.RES:Rescheduling_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU155.NMI:Non-maskable_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU155.PMI:Performance_monitoring_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU157.NMI:Non-maskable_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU157.PMI:Performance_monitoring_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU158.NMI:Non-maskable_interrupts
5509 ± 35% +50.1% 8271 interrupts.CPU158.PMI:Performance_monitoring_interrupts
15130 ± 14% -18.1% 12396 ± 8% interrupts.CPU168.CAL:Function_call_interrupts
4137 ± 15% +44.3% 5970 ± 26% interrupts.CPU48.CAL:Function_call_interrupts
336.00 ± 6% +60.0% 537.75 ± 31% interrupts.CPU48.RES:Rescheduling_interrupts
0.75 -0.0 0.73 perf-stat.i.branch-miss-rate%
8.192e+08 -2.2% 8.01e+08 perf-stat.i.branch-misses
0.98 -1.4% 0.97 perf-stat.i.cpi
1.7e+11 +1.5% 1.725e+11 perf-stat.i.dTLB-loads
1.055e+11 +2.7% 1.084e+11 perf-stat.i.dTLB-stores
7.304e+08 -2.5% 7.123e+08 perf-stat.i.iTLB-load-misses
5.6e+11 +1.6% 5.688e+11 perf-stat.i.instructions
775.04 +4.2% 807.76 perf-stat.i.instructions-per-iTLB-miss
1.02 +1.5% 1.04 perf-stat.i.ipc
2015 +1.5% 2046 perf-stat.i.metric.M/sec
0.73 -0.0 0.72 perf-stat.overall.branch-miss-rate%
0.98 -1.5% 0.96 perf-stat.overall.cpi
0.00 -0.0 0.00 perf-stat.overall.dTLB-store-miss-rate%
764.55 +4.1% 796.24 perf-stat.overall.instructions-per-iTLB-miss
1.02 +1.5% 1.04 perf-stat.overall.ipc
841233 +4.0% 874479 perf-stat.overall.path-length
8.164e+08 -2.2% 7.983e+08 perf-stat.ps.branch-misses
1.695e+11 +1.5% 1.72e+11 perf-stat.ps.dTLB-loads
1.052e+11 +2.7% 1.08e+11 perf-stat.ps.dTLB-stores
7.301e+08 -2.5% 7.119e+08 perf-stat.ps.iTLB-load-misses
5.582e+11 +1.6% 5.668e+11 perf-stat.ps.instructions
1.689e+14 +1.5% 1.715e+14 perf-stat.total.instructions
50.20 -1.4 48.82 perf-profile.calltrace.cycles-pp.__GI___libc_read
40.68 -1.2 39.52 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read
39.53 -1.1 38.39 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
36.75 -1.1 35.70 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
32.27 -0.9 31.36 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
24.69 -0.6 24.14 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
22.23 -0.5 21.72 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
21.04 -0.5 20.55 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
18.70 -0.4 18.27 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
6.80 -0.1 6.65 perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
5.39 -0.1 5.26 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_write
2.96 -0.1 2.84 ± 2% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.27 -0.1 2.15 perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
2.42 -0.1 2.31 ± 2% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
5.38 -0.1 5.28 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.__GI___libc_read
2.61 -0.1 2.52 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_read.new_sync_read.vfs_read.ksys_read
3.46 -0.1 3.38 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
2.62 -0.1 2.55 perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
2.08 -0.1 2.02 perf-profile.calltrace.cycles-pp.__pthread_disable_asynccancel
0.99 -0.1 0.94 perf-profile.calltrace.cycles-pp.fput_many.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
1.04 -0.0 1.00 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
0.97 -0.0 0.93 ± 2% perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyin.copy_page_from_iter.pipe_write.new_sync_write
1.23 -0.0 1.19 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
1.11 -0.0 1.08 perf-profile.calltrace.cycles-pp.copy_user_generic_unrolled.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.20 -0.0 1.17 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_write.new_sync_write.vfs_write.ksys_write
1.01 -0.0 0.98 perf-profile.calltrace.cycles-pp.__prepare_exit_to_usermode.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
0.79 -0.0 0.76 perf-profile.calltrace.cycles-pp.___might_sleep.mutex_lock.pipe_read.new_sync_read.vfs_read
1.50 -0.0 1.47 perf-profile.calltrace.cycles-pp.__might_fault.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
1.13 -0.0 1.11 perf-profile.calltrace.cycles-pp._raw_spin_lock_irq.pipe_read.new_sync_read.vfs_read.ksys_read
0.79 -0.0 0.77 perf-profile.calltrace.cycles-pp.___might_sleep.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
0.72 -0.0 0.70 perf-profile.calltrace.cycles-pp.__sb_end_write.pipe_write.new_sync_write.vfs_write.ksys_write
0.99 -0.0 0.97 perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read
0.00 +0.9 0.92 perf-profile.calltrace.cycles-pp.fsnotify_parent.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
49.09 +1.4 50.48 perf-profile.calltrace.cycles-pp.__GI___libc_write
0.00 +1.5 1.50 ± 5% perf-profile.calltrace.cycles-pp.fsnotify.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.33 +1.6 40.95 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write
38.18 +1.7 39.84 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
35.19 +1.8 37.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
30.77 +1.8 32.61 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
50.18 -1.4 48.80 perf-profile.children.cycles-pp.__GI___libc_read
36.87 -1.1 35.81 perf-profile.children.cycles-pp.ksys_read
32.66 -0.9 31.73 perf-profile.children.cycles-pp.vfs_read
24.84 -0.6 24.29 perf-profile.children.cycles-pp.new_sync_write
22.51 -0.5 22.00 perf-profile.children.cycles-pp.pipe_write
21.18 -0.5 20.69 perf-profile.children.cycles-pp.new_sync_read
18.94 -0.4 18.50 perf-profile.children.cycles-pp.pipe_read
5.44 -0.2 5.27 perf-profile.children.cycles-pp.mutex_lock
7.06 -0.2 6.90 perf-profile.children.cycles-pp.copy_page_from_iter
6.34 -0.2 6.19 perf-profile.children.cycles-pp.syscall_return_via_sysret
7.28 -0.1 7.13 perf-profile.children.cycles-pp.entry_SYSCALL_64
2.33 -0.1 2.21 perf-profile.children.cycles-pp.copyin
7.14 -0.1 7.03 perf-profile.children.cycles-pp.copy_page_to_iter
2.48 -0.1 2.36 ± 2% perf-profile.children.cycles-pp.copyout
4.78 -0.1 4.67 perf-profile.children.cycles-pp.___might_sleep
2.48 -0.1 2.39 perf-profile.children.cycles-pp.copy_user_generic_unrolled
2.23 -0.1 2.14 perf-profile.children.cycles-pp.__prepare_exit_to_usermode
3.56 -0.1 3.48 perf-profile.children.cycles-pp.__wake_up_common_lock
2.69 -0.1 2.63 perf-profile.children.cycles-pp.__might_sleep
2.41 -0.1 2.35 perf-profile.children.cycles-pp._raw_spin_lock_irq
2.08 -0.1 2.02 perf-profile.children.cycles-pp.__pthread_disable_asynccancel
1.88 -0.1 1.82 perf-profile.children.cycles-pp.fput_many
1.40 -0.0 1.35 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.34 -0.0 0.29 perf-profile.children.cycles-pp.fput
1.38 -0.0 1.34 perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.24 ± 7% -0.0 0.22 perf-profile.children.cycles-pp.tick_sched_handle
0.92 -0.0 0.90 perf-profile.children.cycles-pp.fpregs_assert_state_consistent
1.28 -0.0 1.26 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.75 -0.0 0.73 perf-profile.children.cycles-pp.__sb_end_write
80.17 +0.5 80.63 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
77.95 +0.5 78.47 perf-profile.children.cycles-pp.do_syscall_64
1.98 +0.9 2.88 perf-profile.children.cycles-pp.fsnotify_parent
3.04 +1.4 4.42 ± 2% perf-profile.children.cycles-pp.fsnotify
49.09 +1.4 50.48 perf-profile.children.cycles-pp.__GI___libc_write
35.45 +1.8 37.22 perf-profile.children.cycles-pp.ksys_write
30.96 +2.0 32.93 perf-profile.children.cycles-pp.vfs_write
6.28 -0.1 6.13 perf-profile.self.cycles-pp.syscall_return_via_sysret
4.68 -0.1 4.58 perf-profile.self.cycles-pp.___might_sleep
2.35 -0.1 2.26 perf-profile.self.cycles-pp.copy_user_generic_unrolled
2.26 -0.1 2.18 perf-profile.self.cycles-pp.mutex_lock
2.61 -0.1 2.54 perf-profile.self.cycles-pp.__GI___libc_write
3.09 -0.1 3.02 perf-profile.self.cycles-pp.pipe_write
2.57 -0.1 2.50 perf-profile.self.cycles-pp.__GI___libc_read
2.26 -0.1 2.19 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
3.74 -0.1 3.68 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.93 -0.1 0.87 perf-profile.self.cycles-pp.ksys_write
1.38 -0.1 1.32 perf-profile.self.cycles-pp.__prepare_exit_to_usermode
2.31 -0.1 2.25 perf-profile.self.cycles-pp._raw_spin_lock_irq
1.78 -0.1 1.72 perf-profile.self.cycles-pp.fput_many
1.99 -0.1 1.94 perf-profile.self.cycles-pp.__pthread_disable_asynccancel
2.37 -0.0 2.32 perf-profile.self.cycles-pp.__might_sleep
1.14 -0.0 1.10 perf-profile.self.cycles-pp.do_syscall_64
1.36 -0.0 1.31 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
2.22 -0.0 2.18 perf-profile.self.cycles-pp.__pthread_enable_asynccancel
1.87 -0.0 1.83 perf-profile.self.cycles-pp.mutex_unlock
1.34 -0.0 1.31 perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.91 -0.0 0.89 perf-profile.self.cycles-pp.file_update_time
0.85 -0.0 0.82 perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.44 -0.0 0.41 ± 2% perf-profile.self.cycles-pp.touch_atime
0.73 -0.0 0.71 perf-profile.self.cycles-pp.__sb_end_write
1.23 -0.0 1.21 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.21 -0.0 0.20 perf-profile.self.cycles-pp.kill_fasync
2.08 ± 2% +0.4 2.49 ± 4% perf-profile.self.cycles-pp.vfs_write
1.71 +0.8 2.51 perf-profile.self.cycles-pp.fsnotify_parent
2.94 +1.3 4.25 ± 2% perf-profile.self.cycles-pp.fsnotify
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months
[x86/copy_mc] a0ac629ebe: fio.read_iops -43.3% regression
by kernel test robot
Greeting,
FYI, we noticed a -43.3% regression of fio.read_iops due to commit:
commit: a0ac629ebe7b3d248cb93807782a00d9142fdb98 ("x86/copy_mc: Introduce copy_mc_generic()")
url: https://github.com/0day-ci/linux/commits/Dan-Williams/Renovate-memcpy_mcs...
in testcase: fio-basic
on test machine: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
with following parameters:
disk: 2pmem
fs: xfs
mount_option: dax
runtime: 200s
nr_task: 50%
time_based: tb
rw: read
bs: 2M
ioengine: libaio
test_size: 200G
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
In addition to that, the commit also has significant impact on the following tests:
+------------------+----------------------------------------------------------------------+
| testcase: change | fio-basic: fio.read_iops -55.6% regression |
| test machine | 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory |
| test parameters | bs=2M |
| | cpufreq_governor=performance |
| | disk=2pmem |
| | fs=xfs |
| | ioengine=sync |
| | mount_option=dax |
| | nr_task=50% |
| | runtime=200s |
| | rw=read |
| | test_size=200G |
| | time_based=tb |
| | ucode=0x5002f01 |
+------------------+----------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-9/performance/2pmem/xfs/libaio/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01
commit:
7476b91d4d ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()")
a0ac629ebe ("x86/copy_mc: Introduce copy_mc_generic()")
7476b91d4db369d8 a0ac629ebe7b3d248cb93807782
---------------- ---------------------------
%stddev %change %stddev
\ | \
97.22 -96.0 1.19 ± 21% fio.latency_100ms%
0.14 -0.1 0.05 fio.latency_10ms%
0.27 ± 13% -0.1 0.14 fio.latency_20ms%
0.04 ± 6% -0.0 0.03 ± 12% fio.latency_20us%
1.00 ± 28% +96.6 97.57 fio.latency_250ms%
0.05 -0.0 0.05 fio.latency_4ms%
0.02 ± 48% +0.3 0.31 ± 15% fio.latency_500ms%
1.25 ± 47% -0.6 0.63 ± 11% fio.latency_50ms%
0.01 ± 9% +0.0 0.02 ± 24% fio.latency_50us%
44292 -43.3% 25124 fio.read_bw_MBps
67895296 +76.8% 1.201e+08 fio.read_clat_90%_us
68681728 +76.7% 1.214e+08 fio.read_clat_95%_us
98304000 ± 19% +80.3% 1.772e+08 ± 4% fio.read_clat_99%_us
66674508 +76.2% 1.175e+08 fio.read_clat_mean_us
9950116 ± 12% +80.3% 17935634 fio.read_clat_stddev
22146 -43.3% 12562 fio.read_iops
2152824 +76.8% 3805428 fio.read_slat_mean_us
291719 ± 14% +86.6% 544324 fio.read_slat_stddev
12923 -2.5% 12594 fio.time.involuntary_context_switches
77.65 ± 3% -39.1% 47.29 fio.time.user_time
4429275 -43.3% 2512537 fio.workload
0.14 ± 3% +0.0 0.16 ± 4% mpstat.cpu.all.soft%
0.47 ± 3% -0.2 0.31 mpstat.cpu.all.usr%
53185 ± 91% +121.2% 117642 ± 40% numa-vmstat.node0.numa_other
122640 ± 39% -52.6% 58092 ± 81% numa-vmstat.node1.numa_other
60096 +1.5% 61021 proc-vmstat.nr_slab_unreclaimable
20103 ± 5% -17.9% 16495 ± 12% proc-vmstat.pgactivate
49.00 -2.0% 48.00 vmstat.cpu.id
1612 -1.6% 1587 vmstat.system.cs
2713 ± 4% +8.0% 2931 ± 4% slabinfo.PING.active_objs
2713 ± 4% +8.0% 2931 ± 4% slabinfo.PING.num_objs
1164 ± 9% +16.8% 1360 ± 6% slabinfo.task_group.active_objs
1164 ± 9% +16.8% 1360 ± 6% slabinfo.task_group.num_objs
379.25 ± 85% +279.7% 1439 ± 75% sched_debug.cfs_rq:/.exec_clock.min
29948 ± 5% -15.5% 25309 ± 5% sched_debug.cfs_rq:/.exec_clock.stddev
21606 ± 7% +25.1% 27034 ± 7% sched_debug.cfs_rq:/.min_vruntime.min
33321 ± 6% -16.5% 27820 ± 6% sched_debug.cfs_rq:/.min_vruntime.stddev
13783 ±109% +184.1% 39158 ± 20% sched_debug.cfs_rq:/.spread0.avg
-38497 -76.6% -9012 sched_debug.cfs_rq:/.spread0.min
33321 ± 6% -16.5% 27820 ± 6% sched_debug.cfs_rq:/.spread0.stddev
12.22 ± 10% +27.9% 15.62 ± 3% sched_debug.cpu.clock.stddev
3716 ±173% -100.0% 1.50 ± 57% softirqs.CPU10.NET_RX
17411 ± 36% -41.8% 10126 ± 19% softirqs.CPU24.SCHED
9179 ± 67% +87.1% 17173 ± 23% softirqs.CPU35.SCHED
9611 ± 34% -58.9% 3951 ± 10% softirqs.CPU48.SCHED
17177 ± 30% -42.6% 9864 ± 37% softirqs.CPU69.SCHED
86644 ± 29% -22.3% 67339 ± 5% softirqs.CPU76.TIMER
6339 ± 66% +115.9% 13686 ± 31% softirqs.CPU78.SCHED
10156 ± 64% +91.8% 19477 ± 25% softirqs.CPU81.SCHED
1239 ±172% -100.0% 0.00 interrupts.62:PCI-MSI.31981595-edge.i40e-eth0-TxRx-26
47482 +5.4% 50055 ± 4% interrupts.CAL:Function_call_interrupts
209.00 ± 23% -50.4% 103.75 ± 8% interrupts.CPU0.RES:Rescheduling_interrupts
146.25 ± 16% -27.4% 106.25 ± 16% interrupts.CPU15.RES:Rescheduling_interrupts
168.75 ± 81% -64.6% 59.75 ± 33% interrupts.CPU15.TLB:TLB_shootdowns
7321 ± 5% -52.7% 3461 ± 39% interrupts.CPU20.NMI:Non-maskable_interrupts
7321 ± 5% -52.7% 3461 ± 39% interrupts.CPU20.PMI:Performance_monitoring_interrupts
6665 ± 14% -61.2% 2586 ± 26% interrupts.CPU21.NMI:Non-maskable_interrupts
6665 ± 14% -61.2% 2586 ± 26% interrupts.CPU21.PMI:Performance_monitoring_interrupts
64.50 ± 23% +41.9% 91.50 ± 22% interrupts.CPU21.TLB:TLB_shootdowns
100.00 ± 41% +66.0% 166.00 ± 9% interrupts.CPU24.RES:Rescheduling_interrupts
1238 ±173% -100.0% 0.00 interrupts.CPU26.62:PCI-MSI.31981595-edge.i40e-eth0-TxRx-26
438.25 ± 4% +16.1% 509.00 ± 18% interrupts.CPU28.CAL:Function_call_interrupts
145.50 ± 20% -34.4% 95.50 ± 25% interrupts.CPU35.RES:Rescheduling_interrupts
7134 ± 11% -28.3% 5118 ± 19% interrupts.CPU41.NMI:Non-maskable_interrupts
7134 ± 11% -28.3% 5118 ± 19% interrupts.CPU41.PMI:Performance_monitoring_interrupts
107.75 ± 34% -47.3% 56.75 ± 40% interrupts.CPU93.RES:Rescheduling_interrupts
63.18 ± 12% -26.1 37.12 ± 15% perf-profile.calltrace.cycles-pp.copy_mc_fragile.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +3.7 3.72 ± 52% perf-profile.calltrace.cycles-pp.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +37.8 37.83 ± 12% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter
63.34 ± 12% -26.2 37.14 ± 15% perf-profile.children.cycles-pp.copy_mc_fragile
2.41 ±112% -2.2 0.25 ±108% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
2.26 ±109% -2.0 0.29 ± 89% perf-profile.children.cycles-pp.asm_call_on_stack
2.15 ±112% -1.9 0.23 ±110% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
2.12 ±113% -1.9 0.23 ±110% perf-profile.children.cycles-pp.hrtimer_interrupt
1.68 ±114% -1.5 0.17 ±119% perf-profile.children.cycles-pp.__hrtimer_run_queues
1.48 ±123% -1.3 0.15 ±121% perf-profile.children.cycles-pp.tick_sched_timer
1.34 ±120% -1.2 0.14 ±122% perf-profile.children.cycles-pp.tick_sched_handle
1.28 ±119% -1.1 0.14 ±122% perf-profile.children.cycles-pp.update_process_times
0.70 ±107% -0.6 0.10 ±120% perf-profile.children.cycles-pp.scheduler_tick
2.65 ±106% +16.5 19.13 ± 12% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +22.6 22.58 ± 7% perf-profile.children.cycles-pp.copy_mc_generic
62.52 ± 12% -25.5 37.00 ± 15% perf-profile.self.cycles-pp.copy_mc_fragile
0.00 +22.4 22.41 ± 6% perf-profile.self.cycles-pp.copy_mc_generic
42.43 +68.7% 71.58 perf-stat.i.MPKI
5.949e+09 -42.2% 3.44e+09 perf-stat.i.branch-instructions
0.07 +0.0 0.10 ± 5% perf-stat.i.branch-miss-rate%
3554006 ± 2% -7.5% 3286479 ± 3% perf-stat.i.branch-misses
95.02 -2.4 92.63 perf-stat.i.cache-miss-rate%
1.444e+09 -5.2% 1.369e+09 perf-stat.i.cache-misses
1.513e+09 -2.8% 1.471e+09 perf-stat.i.cache-references
3.81 +72.5% 6.58 perf-stat.i.cpi
102.49 +4.5% 107.13 perf-stat.i.cycles-between-cache-misses
0.00 ± 4% +0.0 0.00 ± 41% perf-stat.i.dTLB-load-miss-rate%
6.03e+09 -42.0% 3.495e+09 perf-stat.i.dTLB-loads
0.00 ± 5% +0.0 0.00 ± 7% perf-stat.i.dTLB-store-miss-rate%
5.909e+09 -42.5% 3.4e+09 perf-stat.i.dTLB-stores
47.00 +1.4 48.45 perf-stat.i.iTLB-load-miss-rate%
2270674 -11.0% 2021114 perf-stat.i.iTLB-load-misses
2563127 -16.0% 2151931 perf-stat.i.iTLB-loads
3.548e+10 -42.4% 2.044e+10 perf-stat.i.instructions
15634 -35.2% 10127 perf-stat.i.instructions-per-iTLB-miss
0.26 -41.6% 0.15 perf-stat.i.ipc
207.77 -37.5% 129.85 perf-stat.i.metric.M/sec
78061415 ± 13% +98.0% 1.546e+08 ± 20% perf-stat.i.node-load-misses
85582855 ± 11% +58.1% 1.353e+08 ± 20% perf-stat.i.node-loads
3.817e+08 -2.8% 3.709e+08 perf-stat.i.node-stores
42.66 +68.7% 71.96 perf-stat.overall.MPKI
0.06 +0.0 0.09 ± 3% perf-stat.overall.branch-miss-rate%
95.45 -2.4 93.07 perf-stat.overall.cache-miss-rate%
3.81 +73.0% 6.59 perf-stat.overall.cpi
93.55 +5.2% 98.41 perf-stat.overall.cycles-between-cache-misses
0.00 ± 5% +0.0 0.00 ± 13% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 5% +0.0 0.00 ± 2% perf-stat.overall.dTLB-store-miss-rate%
46.98 +1.5 48.43 perf-stat.overall.iTLB-load-miss-rate%
15639 -35.2% 10127 perf-stat.overall.instructions-per-iTLB-miss
0.26 -42.2% 0.15 perf-stat.overall.ipc
1605743 +1.5% 1630326 perf-stat.overall.path-length
5.919e+09 -42.2% 3.422e+09 perf-stat.ps.branch-instructions
3519866 ± 2% -7.8% 3245208 ± 3% perf-stat.ps.branch-misses
1.437e+09 -5.2% 1.362e+09 perf-stat.ps.cache-misses
1.506e+09 -2.8% 1.463e+09 perf-stat.ps.cache-references
1552 -1.4% 1530 perf-stat.ps.context-switches
6e+09 -42.1% 3.477e+09 perf-stat.ps.dTLB-loads
5.88e+09 -42.5% 3.382e+09 perf-stat.ps.dTLB-stores
2257568 -11.0% 2008542 perf-stat.ps.iTLB-load-misses
2547705 -16.1% 2138603 perf-stat.ps.iTLB-loads
3.53e+10 -42.4% 2.034e+10 perf-stat.ps.instructions
77685715 ± 13% +97.9% 1.538e+08 ± 20% perf-stat.ps.node-load-misses
85143339 ± 11% +58.1% 1.346e+08 ± 20% perf-stat.ps.node-loads
3.797e+08 -2.8% 3.69e+08 perf-stat.ps.node-stores
7.112e+12 -42.4% 4.096e+12 perf-stat.total.instructions
fio.read_bw_MBps
46000 +-------------------------------------------------------------------+
44000 |..+.+..+.+..+..+.+..+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
42000 |-+ |
40000 |-+ |
38000 |-+ |
36000 |-+ |
| |
34000 |-+ |
32000 |-+ |
30000 |-+ |
28000 |-+ |
| |
26000 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
24000 +-------------------------------------------------------------------+
fio.read_iops
23000 +-------------------------------------------------------------------+
22000 |..+.+..+.+..+..+.+..+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
21000 |-+ |
20000 |-+ |
19000 |-+ |
18000 |-+ |
| |
17000 |-+ |
16000 |-+ |
15000 |-+ |
14000 |-+ |
| |
13000 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
12000 +-------------------------------------------------------------------+
fio.read_clat_mean_us
1.2e+08 +-----------------------------------------------------------------+
| O O O O O O O O O O O O O O O O O O O O O O O O |
1.1e+08 |-+ |
| |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| |
| |
7e+07 |..+.+.. .+..+. .+..+.+..+..+.+..+.+.. |
| +.+..+ +..+ + |
6e+07 +-----------------------------------------------------------------+
fio.read_clat_90__us
1.3e+08 +-----------------------------------------------------------------+
| |
1.2e+08 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
| |
1.1e+08 |-+ |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| |
7e+07 |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+ |
| |
6e+07 +-----------------------------------------------------------------+
fio.read_clat_95__us
1.3e+08 +-----------------------------------------------------------------+
| O O O O O |
1.2e+08 |-+O O O O O O O O O O O O O O O O O O O O |
| |
1.1e+08 |-+ |
| |
1e+08 |-+ |
| |
9e+07 |-+ |
| |
8e+07 |-+ |
| .+. .+.. |
7e+07 |..+.+..+.+..+.+..+.+..+.+. +. +.+..+.+..+ |
| |
6e+07 +-----------------------------------------------------------------+
fio.read_slat_mean_us
4e+06 +-----------------------------------------------------------------+
3.8e+06 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
| |
3.6e+06 |-+ |
3.4e+06 |-+ |
| |
3.2e+06 |-+ |
3e+06 |-+ |
2.8e+06 |-+ |
| |
2.6e+06 |-+ |
2.4e+06 |-+ |
| |
2.2e+06 |..+.+..+.+..+.+..+.+..+.+..+.+..+..+.+..+.+..+ |
2e+06 +-----------------------------------------------------------------+
fio.latency_10ms_
0.15 +--------------------------------------------------------------------+
0.14 |..+.+..+..+.+..+..+.+..+..+.+..+..+.+..+.+..+..+ |
| |
0.13 |-+ |
0.12 |-+ |
0.11 |-+ |
0.1 |-+ |
| |
0.09 |-+ |
0.08 |-+ |
0.07 |-+ |
0.06 |-+ |
| |
0.05 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
0.04 +--------------------------------------------------------------------+
fio.latency_20ms_
0.55 +--------------------------------------------------------------------+
| + + + |
0.5 |-+ : + + :: |
0.45 |++ : + + + : : |
| + : + + : |
0.4 |-+ : : : : : |
0.35 |-+ : : : : : |
| : : : : : + |
0.3 |-+ : : : +. : + + |
0.25 |-+ : : .. +.. : + + |
| + +..+ +..+ +..+ |
0.2 |-+ |
0.15 |-+ |
| O O O O O O O O O O O O O O O O O O O O O O O O O |
0.1 +--------------------------------------------------------------------+
fio.latency_100ms_
100 +---------------------------------------------------------------------+
90 |-+ + +.+..+..+ +. |
| |
80 |-+ |
70 |-+ |
| |
60 |-+ |
50 |-+ |
40 |-+ |
| |
30 |-+ |
20 |-+ |
| |
10 |-+ |
0 +---------------------------------------------------------------------+
fio.latency_250ms_
100 +---------------------------------------------------------------------+
90 |-+ O |
| |
80 |-+ |
70 |-+ |
| |
60 |-+ |
50 |-+ |
40 |-+ |
| |
30 |-+ |
20 |-+ |
| |
10 |-+ |
0 +---------------------------------------------------------------------+
fio.workload
4.6e+06 +-----------------------------------------------------------------+
4.4e+06 |..+.+..+.+..+.+..+.+..+.+.. .+.. .+.+..+.+..+ |
| + +. |
4.2e+06 |-+ |
4e+06 |-+ |
3.8e+06 |-+ |
3.6e+06 |-+ |
| |
3.4e+06 |-+ |
3.2e+06 |-+ |
3e+06 |-+ |
2.8e+06 |-+ |
| |
2.6e+06 |-+O O O O O O O O O O O O O O O O O O O O O O O O O |
2.4e+06 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-csl-2sp6: 96 threads Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz with 256G memory
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/mount_option/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
2M/gcc-9/performance/2pmem/xfs/sync/x86_64-rhel-8.3/dax/50%/debian-10.4-x86_64-20200603.cgz/200s/read/lkp-csl-2sp6/200G/fio-basic/tb/0x5002f01
commit:
7476b91d4d ("x86, powerpc: Rename memcpy_mcsafe() to copy_mc_to_{user, kernel}()")
a0ac629ebe ("x86/copy_mc: Introduce copy_mc_generic()")
7476b91d4db369d8 a0ac629ebe7b3d248cb93807782
---------------- ---------------------------
%stddev %change %stddev
\ | \
0.61 ± 15% -0.4 0.22 ± 94% fio.latency_1000us%
0.01 ± 11% +1.3 1.27 ± 25% fio.latency_10ms%
96.06 -95.5 0.60 ± 80% fio.latency_2ms%
1.27 ± 33% +96.2 97.48 fio.latency_4ms%
1.29 ± 55% -1.2 0.05 ± 54% fio.latency_500us%
75143 -55.6% 33381 fio.read_bw_MBps
1372160 +118.5% 2998272 fio.read_clat_90%_us
1409024 +116.9% 3055616 fio.read_clat_95%_us
2142208 ± 19% +120.3% 4718592 ± 17% fio.read_clat_99%_us
1272849 +125.4% 2869293 fio.read_clat_mean_us
228201 ± 15% +103.6% 464620 ± 14% fio.read_clat_stddev
37571 -55.6% 16690 fio.read_iops
69.28 ± 2% -40.3% 41.38 ± 3% fio.time.user_time
7514438 -55.6% 3338252 fio.workload
0.11 ± 3% +0.0 0.14 ± 5% mpstat.cpu.all.soft%
0.43 ± 3% -0.1 0.28 ± 2% mpstat.cpu.all.usr%
115069 -2.3% 112454 proc-vmstat.nr_shmem
20846 ± 6% -27.8% 15052 ± 3% proc-vmstat.pgactivate
967.50 ± 27% -50.0% 483.75 ± 78% slabinfo.xfs_buf_item.active_objs
967.50 ± 27% -50.0% 483.75 ± 78% slabinfo.xfs_buf_item.num_objs
100.00 -2.0% 98.00 vmstat.io.bo
1672 -3.3% 1616 vmstat.system.cs
9.059e+09 ± 6% -32.3% 6.131e+09 ± 54% cpuidle.C1E.time
19004364 ± 3% -22.4% 14741281 ± 34% cpuidle.C1E.usage
4.034e+08 ±133% +713.0% 3.28e+09 ±100% cpuidle.C6.time
570211 ±122% +571.6% 3829822 ± 86% cpuidle.C6.usage
61.80 ± 9% -17.6 44.19 perf-profile.calltrace.cycles-pp.copy_mc_fragile.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +7.8 7.81 ± 6% perf-profile.calltrace.cycles-pp.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter.dax_iomap_actor
0.00 +29.2 29.21 ± 5% perf-profile.calltrace.cycles-pp.asm_sysvec_apic_timer_interrupt.copy_mc_generic.copy_mc_to_user.copyout_mc._copy_mc_to_iter
61.92 ± 9% -17.7 44.25 perf-profile.children.cycles-pp.copy_mc_fragile
3.47 ±132% +11.7 15.21 ± 5% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.00 +22.3 22.32 perf-profile.children.cycles-pp.copy_mc_generic
61.16 ± 9% -17.4 43.78 perf-profile.self.cycles-pp.copy_mc_fragile
0.00 +22.1 22.09 perf-profile.self.cycles-pp.copy_mc_generic
212.00 ± 38% +288.6% 823.90 ± 67% sched_debug.cfs_rq:/.exec_clock.min
34013 ± 3% -17.1% 28181 ± 2% sched_debug.cfs_rq:/.exec_clock.stddev
36118 ± 5% -15.0% 30710 ± 2% sched_debug.cfs_rq:/.min_vruntime.stddev
36118 ± 5% -15.0% 30707 ± 2% sched_debug.cfs_rq:/.spread0.stddev
9.52 ± 11% +33.8% 12.73 ± 9% sched_debug.cpu.clock.stddev
17832 ± 13% -47.5% 9368 ± 17% sched_debug.cpu.sched_count.max
2475 ± 9% -34.4% 1624 ± 8% sched_debug.cpu.sched_count.stddev
8858 ± 13% -48.3% 4577 ± 18% sched_debug.cpu.sched_goidle.max
1260 ± 9% -33.2% 841.68 ± 8% sched_debug.cpu.sched_goidle.stddev
8285 ± 16% -32.1% 5622 ± 7% sched_debug.cpu.ttwu_count.max
1169 ± 9% -24.9% 878.40 ± 4% sched_debug.cpu.ttwu_count.stddev
26587 ± 8% -21.9% 20773 ± 22% softirqs.CPU1.SCHED
19906 ± 37% -55.7% 8824 ± 96% softirqs.CPU10.SCHED
21997 ± 34% -82.2% 3910 ± 55% softirqs.CPU20.SCHED
5126 ± 70% +166.6% 13666 ± 15% softirqs.CPU30.SCHED
5567 ± 56% +165.3% 14772 ± 29% softirqs.CPU31.SCHED
10027 ± 35% +101.3% 20182 ± 18% softirqs.CPU33.SCHED
4868 ± 50% +112.6% 10349 ± 14% softirqs.CPU44.SCHED
6304 ± 60% +154.5% 16043 ± 22% softirqs.CPU46.SCHED
4127 ± 76% +198.6% 12326 ± 32% softirqs.CPU49.SCHED
6313 ± 62% +98.5% 12530 ± 19% softirqs.CPU51.SCHED
8249 ± 58% +148.7% 20515 ± 31% softirqs.CPU57.SCHED
6971 ±109% +268.6% 25698 ± 8% softirqs.CPU68.SCHED
25116 ± 15% -32.4% 16974 ± 12% softirqs.CPU78.SCHED
24757 ± 12% -36.8% 15657 ± 27% softirqs.CPU79.SCHED
20231 ± 14% -45.5% 11024 ± 24% softirqs.CPU81.SCHED
21830 ± 23% -55.4% 9733 ± 67% softirqs.CPU9.SCHED
24043 ± 16% -39.9% 14449 ± 23% softirqs.CPU94.SCHED
42.31 +68.3% 71.22 perf-stat.i.MPKI
9.958e+09 -54.7% 4.511e+09 perf-stat.i.branch-instructions
0.05 ± 2% +0.0 0.08 ± 4% perf-stat.i.branch-miss-rate%
3682118 ± 2% -8.2% 3381534 perf-stat.i.branch-misses
67.34 +10.4 77.74 perf-stat.i.cache-miss-rate%
1.709e+09 -12.2% 1.501e+09 perf-stat.i.cache-misses
2.531e+09 -24.0% 1.923e+09 perf-stat.i.cache-references
1639 -4.1% 1571 perf-stat.i.context-switches
2.25 +121.4% 4.98 perf-stat.i.cpi
99.03 -1.8% 97.24 perf-stat.i.cpu-migrations
85.60 +14.2% 97.78 perf-stat.i.cycles-between-cache-misses
0.00 ± 18% +0.0 0.00 ± 44% perf-stat.i.dTLB-load-miss-rate%
9.996e+09 -54.5% 4.549e+09 perf-stat.i.dTLB-loads
0.00 ± 7% +0.0 0.00 ± 6% perf-stat.i.dTLB-store-miss-rate%
9.904e+09 -54.9% 4.466e+09 perf-stat.i.dTLB-stores
44.79 +4.2 48.99 perf-stat.i.iTLB-load-miss-rate%
2535885 -13.8% 2185118 perf-stat.i.iTLB-load-misses
3134177 -27.3% 2278467 perf-stat.i.iTLB-loads
5.952e+10 -54.9% 2.687e+10 perf-stat.i.instructions
23480 -47.6% 12304 perf-stat.i.instructions-per-iTLB-miss
0.45 -54.6% 0.20 perf-stat.i.ipc
342.39 -51.0% 167.90 perf-stat.i.metric.M/sec
1.165e+08 ± 30% +72.8% 2.013e+08 ± 9% perf-stat.i.node-load-misses
1.257e+08 ± 26% +41.1% 1.773e+08 ± 9% perf-stat.i.node-loads
2.42e+08 +19.6% 2.895e+08 perf-stat.i.node-stores
42.53 +68.3% 71.58 perf-stat.overall.MPKI
0.04 ± 2% +0.0 0.07 perf-stat.overall.branch-miss-rate%
67.52 +10.5 78.07 perf-stat.overall.cache-miss-rate%
2.24 +122.4% 4.99 perf-stat.overall.cpi
78.17 +14.3% 89.34 perf-stat.overall.cycles-between-cache-misses
0.00 ± 25% +0.0 0.00 ± 12% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 13% +0.0 0.00 ± 10% perf-stat.overall.dTLB-store-miss-rate%
44.72 +4.2 48.96 perf-stat.overall.iTLB-load-miss-rate%
23499 -47.6% 12306 perf-stat.overall.instructions-per-iTLB-miss
0.45 -55.0% 0.20 perf-stat.overall.ipc
1587395 +1.5% 1611895 perf-stat.overall.path-length
9.912e+09 -54.7% 4.489e+09 perf-stat.ps.branch-instructions
3650903 ± 2% -8.4% 3345674 perf-stat.ps.branch-misses
1.701e+09 -12.2% 1.494e+09 perf-stat.ps.cache-misses
2.52e+09 -24.1% 1.914e+09 perf-stat.ps.cache-references
1616 -3.7% 1556 perf-stat.ps.context-switches
9.95e+09 -54.5% 4.526e+09 perf-stat.ps.dTLB-loads
9.859e+09 -54.9% 4.445e+09 perf-stat.ps.dTLB-stores
2521574 -13.8% 2172342 perf-stat.ps.iTLB-load-misses
3116655 -27.3% 2264894 perf-stat.ps.iTLB-loads
5.925e+10 -54.9% 2.673e+10 perf-stat.ps.instructions
1.159e+08 ± 30% +72.8% 2.003e+08 ± 9% perf-stat.ps.node-load-misses
1.25e+08 ± 26% +41.1% 1.764e+08 ± 9% perf-stat.ps.node-loads
2.407e+08 +19.6% 2.878e+08 perf-stat.ps.node-stores
1.193e+13 -54.9% 5.381e+12 perf-stat.total.instructions
0.00 +2.7e+105% 2689 ±171% interrupts.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
62.75 ± 27% +51.8% 95.25 ± 21% interrupts.CPU1.RES:Rescheduling_interrupts
6530 ± 17% -44.1% 3647 ± 35% interrupts.CPU17.NMI:Non-maskable_interrupts
6530 ± 17% -44.1% 3647 ± 35% interrupts.CPU17.PMI:Performance_monitoring_interrupts
62.00 ± 74% +187.9% 178.50 ± 5% interrupts.CPU20.RES:Rescheduling_interrupts
365.00 ± 78% -76.0% 87.50 ± 53% interrupts.CPU25.TLB:TLB_shootdowns
170.50 ± 15% -26.8% 124.75 ± 10% interrupts.CPU30.RES:Rescheduling_interrupts
7605 -43.3% 4316 ± 32% interrupts.CPU31.NMI:Non-maskable_interrupts
7605 -43.3% 4316 ± 32% interrupts.CPU31.PMI:Performance_monitoring_interrupts
169.00 ± 12% -37.1% 106.25 ± 23% interrupts.CPU31.RES:Rescheduling_interrupts
7145 ± 11% -33.0% 4786 ± 18% interrupts.CPU36.NMI:Non-maskable_interrupts
7145 ± 11% -33.0% 4786 ± 18% interrupts.CPU36.PMI:Performance_monitoring_interrupts
136.50 ± 27% -44.7% 75.50 ± 60% interrupts.CPU39.TLB:TLB_shootdowns
149.25 ± 24% -24.6% 112.50 ± 30% interrupts.CPU4.RES:Rescheduling_interrupts
7599 -46.6% 4061 ± 35% interrupts.CPU41.NMI:Non-maskable_interrupts
7599 -46.6% 4061 ± 35% interrupts.CPU41.PMI:Performance_monitoring_interrupts
6661 ± 24% -52.1% 3191 ± 51% interrupts.CPU44.NMI:Non-maskable_interrupts
6661 ± 24% -52.1% 3191 ± 51% interrupts.CPU44.PMI:Performance_monitoring_interrupts
7622 -43.5% 4307 ± 33% interrupts.CPU46.NMI:Non-maskable_interrupts
7622 -43.5% 4307 ± 33% interrupts.CPU46.PMI:Performance_monitoring_interrupts
7613 -43.1% 4331 ± 31% interrupts.CPU47.NMI:Non-maskable_interrupts
7613 -43.1% 4331 ± 31% interrupts.CPU47.PMI:Performance_monitoring_interrupts
5823 ± 32% -36.4% 3703 ± 34% interrupts.CPU5.NMI:Non-maskable_interrupts
5823 ± 32% -36.4% 3703 ± 34% interrupts.CPU5.PMI:Performance_monitoring_interrupts
89.25 ± 48% -61.1% 34.75 ± 31% interrupts.CPU53.TLB:TLB_shootdowns
5698 ± 33% -42.5% 3277 ± 49% interrupts.CPU55.NMI:Non-maskable_interrupts
5698 ± 33% -42.5% 3277 ± 49% interrupts.CPU55.PMI:Performance_monitoring_interrupts
172.00 ± 14% -35.2% 111.50 ± 41% interrupts.CPU56.RES:Rescheduling_interrupts
64.00 ± 42% -39.5% 38.75 ± 29% interrupts.CPU56.TLB:TLB_shootdowns
156.00 ± 17% -36.2% 99.50 ± 21% interrupts.CPU57.RES:Rescheduling_interrupts
146.25 ± 28% -48.9% 74.75 ± 67% interrupts.CPU58.RES:Rescheduling_interrupts
7627 -47.0% 4043 ± 31% interrupts.CPU62.NMI:Non-maskable_interrupts
7627 -47.0% 4043 ± 31% interrupts.CPU62.PMI:Performance_monitoring_interrupts
174.75 ± 12% -29.9% 122.50 ± 30% interrupts.CPU62.RES:Rescheduling_interrupts
76.00 ± 29% -48.4% 39.25 ± 29% interrupts.CPU62.TLB:TLB_shootdowns
7159 ± 11% -50.2% 3564 ± 32% interrupts.CPU63.NMI:Non-maskable_interrupts
7159 ± 11% -50.2% 3564 ± 32% interrupts.CPU63.PMI:Performance_monitoring_interrupts
7628 -62.9% 2831 interrupts.CPU66.NMI:Non-maskable_interrupts
7628 -62.9% 2831 interrupts.CPU66.PMI:Performance_monitoring_interrupts
174.50 ± 10% -36.4% 111.00 ± 50% interrupts.CPU66.RES:Rescheduling_interrupts
4370 ± 18% -34.7% 2853 interrupts.CPU69.NMI:Non-maskable_interrupts
4370 ± 18% -34.7% 2853 interrupts.CPU69.PMI:Performance_monitoring_interrupts
6885 ± 18% -45.8% 3731 ± 28% interrupts.CPU74.NMI:Non-maskable_interrupts
6885 ± 18% -45.8% 3731 ± 28% interrupts.CPU74.PMI:Performance_monitoring_interrupts
5900 ± 18% -57.5% 2510 ± 24% interrupts.CPU77.NMI:Non-maskable_interrupts
5900 ± 18% -57.5% 2510 ± 24% interrupts.CPU77.PMI:Performance_monitoring_interrupts
62.00 ± 41% +58.9% 98.50 ± 14% interrupts.CPU78.RES:Rescheduling_interrupts
0.00 +2.7e+105% 2689 ±171% interrupts.CPU79.115:PCI-MSI.31981648-edge.i40e-eth0-TxRx-79
49.75 ± 47% +119.6% 109.25 ± 28% interrupts.CPU79.RES:Rescheduling_interrupts
61.50 ± 54% +115.0% 132.25 ± 31% interrupts.CPU8.RES:Rescheduling_interrupts
5871 ± 19% -38.8% 3594 ± 32% interrupts.CPU80.NMI:Non-maskable_interrupts
5871 ± 19% -38.8% 3594 ± 32% interrupts.CPU80.PMI:Performance_monitoring_interrupts
60.50 ± 19% +120.2% 133.25 ± 14% interrupts.CPU81.RES:Rescheduling_interrupts
36.00 ± 79% +179.2% 100.50 ± 35% interrupts.CPU86.RES:Rescheduling_interrupts
6322 ± 21% -60.6% 2490 ± 25% interrupts.CPU88.NMI:Non-maskable_interrupts
6322 ± 21% -60.6% 2490 ± 25% interrupts.CPU88.PMI:Performance_monitoring_interrupts
32.50 ± 40% +150.0% 81.25 ± 41% interrupts.CPU92.RES:Rescheduling_interrupts
124.00 ± 11% -22.8% 95.75 ± 4% interrupts.IWI:IRQ_work_interrupts
538989 ± 8% -28.0% 387910 ± 2% interrupts.NMI:Non-maskable_interrupts
538989 ± 8% -28.0% 387910 ± 2% interrupts.PMI:Performance_monitoring_interrupts
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
6 months, 3 weeks
Re: [IB/srpt] c804af2c1d: last_state.test.blktests.exit_code.143
by Bart Van Assche
On 2020-08-03 00:27, Sagi Grimberg wrote:
>
>>>> Greeting,
>>>>
>>>> FYI, we noticed the following commit (built with gcc-9):
>>>>
>>>> commit: c804af2c1d3152c0cf877eeb50d60c2d49ac0cf0 ("IB/srpt: use new shared CQ mechanism")
>>>> https://git.kernel.org/cgit/linux/kernel/git/rdma/rdma.git for-next
>>>>
>>>>
>>>> in testcase: blktests
>>>> with following parameters:
>>>>
>>>> test: srp-group1
>>>> ucode: 0x21
>>>>
>>>>
>>>>
>>>> on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory
>>>>
>>>> caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
>>>>
>>>>
>>>>
>>>>
>>>> If you fix the issue, kindly add following tag
>>>> Reported-by: kernel test robot <rong.a.chen(a)intel.com>
>>>>
>>>>
>>>> user :notice: [ 44.688140] 2020-08-01 16:10:22 ./check srp/001 srp/002 srp/003 srp/004 srp/005 srp/006 srp/007 srp/008 srp/009 srp/010 srp/011 srp/012 srp/013 srp/015
>>>> user :notice: [ 44.706657] srp/001 (Create and remove LUNs)
>>>> user :notice: [ 44.718405] srp/001 (Create and remove LUNs) [passed]
>>>> user :notice: [ 44.729902] runtime ... 1.972s
>>>> user :notice: [ 99.038748] IPMI BMC is not supported on this machine, skip bmc-watchdog setup!
>>>> user :notice: [ 3699.039790] Sat Aug 1 17:11:22 UTC 2020 detected soft_timeout
>>>> user :notice: [ 3699.060341] kill 960 /usr/bin/time -v -o /tmp/lkp/blktests.time /lkp/lkp/src/tests/blktests
>>> Yamin and Max, can you take a look at this? The SRP tests from the
>>> blktests repository pass reliably with kernel version v5.7 and before.
>>> With label next-20200731 from linux-next however that test triggers the
>>> following hang:
>>
>> I will look into it.
>
> FWIW, I ran into this as well with nvme-rdma, but it also reproduces
> when I revert the shared CQ patch from nvme-rdma. Another data point
> is that my tests passes with siw.
Hi Jason,
The patch below is sufficient to unbreak blktests. I think that the
deadlock while unloading rdma_rxe happens because the RDMA core waits for
all ib_dev references to be dropped before dealloc_driver is called.
The rdma_rxe dealloc_driver implementation drops an ib_dev reference. The
dealloc_driver callback was introduced by commit d0899892edd0
("RDMA/device: Provide APIs from the core code to help unregistration").
Do you agree that this regression has been introduced by commits
d0899892edd0 and c367074b6c37 ("RDMA/rxe: Use driver_unregister and new
unregistration API")?
Thanks,
Bart.
---
drivers/infiniband/core/device.c | 7 +------
1 file changed, 1 insertion(+), 6 deletions(-)
diff --git a/drivers/infiniband/core/device.c b/drivers/infiniband/core/device.c
index dca2842a7872..5192f305b253 100644
--- a/drivers/infiniband/core/device.c
+++ b/drivers/infiniband/core/device.c
@@ -1287,13 +1287,8 @@ static void disable_device(struct ib_device *device)
/* Pairs with refcount_set in enable_device */
ib_device_put(device);
- wait_for_completion(&device->unreg_completion);
- /*
- * compat devices must be removed after device refcount drops to zero.
- * Otherwise init_net() may add more compatdevs after removing compat
- * devices and before device is disabled.
- */
+ /* To do: prevent init_net() to add more compat_devs. */
remove_compat_devs(device);
}
7 months
[PCI] 3233e41d3e: WARNING:at_drivers/pci/pci.c:#pci_reset_hotplug_slot
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 3233e41d3e8ebcd44e92da47ffed97fd49b84278 ("[PATCH] PCI: pciehp: Fix AB-BA deadlock between reset_lock and device_lock")
url: https://github.com/0day-ci/linux/commits/Lukas-Wunner/PCI-pciehp-Fix-AB-B...
base: https://git.kernel.org/cgit/linux/kernel/git/helgaas/pci.git next
in testcase: boot
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 8 -m 16G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------------------+------------+------------+
| | 8a445afd71 | 3233e41d3e |
+------------------------------------------------------+------------+------------+
| boot_successes | 4 | 0 |
| boot_failures | 0 | 4 |
| WARNING:at_drivers/pci/pci.c:#pci_reset_hotplug_slot | 0 | 4 |
| RIP:pci_reset_hotplug_slot | 0 | 4 |
+------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 0.971752] WARNING: CPU: 0 PID: 1 at drivers/pci/pci.c:4905 pci_reset_hotplug_slot+0x70/0x80
[ 0.971753] Modules linked in:
[ 0.971755] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.8.0-rc1-00053-g3233e41d3e8eb #1
[ 0.971756] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 0.971757] RIP: 0010:pci_reset_hotplug_slot+0x70/0x80
[ 0.971759] Code: 41 89 c4 48 8b 7b 20 e8 4e 3a c1 ff 44 89 e0 5b 5d 41 5c c3 48 8b 43 18 31 f6 48 8d b8 90 00 00 00 e8 e4 80 70 00 85 c0 75 ba <0f> 0b eb b6 41 bc e7 ff ff ff eb d6 0f 1f 40 00 66 66 66 66 90 48
[ 0.971759] RSP: 0000:ffffbd73c0013ab0 EFLAGS: 00010246
[ 0.971761] RAX: 0000000000000000 RBX: ffff9700475024c0 RCX: ffff9701048c9900
[ 0.971762] RDX: ffff970047e40000 RSI: ffff9701048c9990 RDI: 0000000000000246
[ 0.971763] RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000000
[ 0.971763] R10: 0000000000000001 R11: 0000000000000000 R12: ffff9700477320b0
[ 0.971764] R13: ffff970047732000 R14: ffff97004769c238 R15: 0000000000000000
[ 0.971765] FS: 0000000000000000(0000) GS:ffff97036fc00000(0000) knlGS:0000000000000000
[ 0.971766] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.971767] CR2: 0000000000000000 CR3: 00000001c2c10000 CR4: 00000000000006f0
[ 0.971767] Call Trace:
[ 0.971768] pci_probe_reset_function+0xc4/0xe0
[ 0.971769] pci_device_add+0x13f/0x2a0
[ 0.971769] pci_scan_single_device+0xa4/0xc0
[ 0.971770] pci_scan_slot+0x52/0x110
[ 0.971771] pci_scan_child_bus_extend+0x3a/0x2a0
[ 0.971771] acpi_pci_root_create+0x1f7/0x250
[ 0.971772] pci_acpi_scan_root+0x182/0x1b0
[ 0.971773] acpi_pci_root_add.cold+0x59/0x1b0
[ 0.971773] ? acpi_device_always_present+0x20/0x90
[ 0.971774] acpi_bus_attach+0xf6/0x200
[ 0.971775] acpi_bus_attach+0x6b/0x200
[ 0.971776] acpi_bus_scan+0x43/0x90
[ 0.971776] ? acpi_sleep_proc_init+0x24/0x24
[ 0.971777] acpi_scan_init+0x102/0x24b
[ 0.971778] acpi_init+0x2c7/0x329
[ 0.971778] do_one_initcall+0x5d/0x330
[ 0.971779] ? rcu_read_lock_sched_held+0x52/0x90
[ 0.971780] kernel_init_freeable+0x248/0x2c9
[ 0.971780] ? rest_init+0x23e/0x23e
[ 0.971781] kernel_init+0xa/0x112
[ 0.971782] ret_from_fork+0x22/0x30
[ 0.971782] irq event stamp: 109115
[ 0.971783] hardirqs last enabled at (109115): [<ffffffffaa71a514>] _raw_spin_unlock_irqrestore+0x54/0x70
[ 0.971784] hardirqs last disabled at (109114): [<ffffffffaa71ab01>] _raw_spin_lock_irqsave+0x21/0x60
[ 0.971785] softirqs last enabled at (109060): [<ffffffffaaa003aa>] __do_softirq+0x3aa/0x4af
[ 0.971786] softirqs last disabled at (109053): [<ffffffffaa8010b2>] asm_call_on_stack+0x12/0x20
[ 0.971787] ---[ end trace c3e4ce92dee5df5f ]---
To reproduce:
# build kernel
cd linux
cp config-5.8.0-rc1-00053-g3233e41d3e8eb .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
7 months, 1 week
[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-02
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
2019-07-26 16:52:03 make run_tests -C seccomp
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
In file included from seccomp_bpf.c:51:0:
seccomp_bpf.c: In function ‘tracer_ptrace’:
seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function)
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function)
: PTRACE_EVENTMSG_SYSCALL_EXIT, msg);
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
Makefile:12: recipe for target 'seccomp_bpf' failed
make: *** [seccomp_bpf] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
To reproduce:
# build kernel
cd linux
cp config-5.2.0-10889-g201766a20e30f9 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
7 months, 1 week
[rcuperf] 4e88ec4a9e: UBSAN:division-overflow_in_arch/x86/include/asm/div64.h
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 4e88ec4a9eb17527e640b063f79e5b875733eb53 ("rcuperf: Change rcuperf to rcuscale")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: trinity
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+---------------------------------------------------------+------------+------------+
| | 65bd77f554 | 4e88ec4a9e |
+---------------------------------------------------------+------------+------------+
| boot_successes | 13 | 0 |
| boot_failures | 0 | 14 |
| UBSAN:division-overflow_in_arch/x86/include/asm/div64.h | 0 | 14 |
| error:#[##] | 0 | 14 |
| EIP:main_func.cold | 0 | 14 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 14 |
+---------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 24.821408] UBSAN: division-overflow in arch/x86/include/asm/div64.h:54:20
[ 24.823429] division by zero
[ 24.824123] CPU: 0 PID: 64 Comm: main_func Not tainted 5.9.0-rc1-00019-g4e88ec4a9eb17 #1
[ 24.825684] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 24.827073] Call Trace:
[ 24.827523] dump_stack+0x45/0x66
[ 24.828139] ubsan_epilogue+0x16/0x75
[ 24.828843] __ubsan_handle_divrem_overflow.cold+0x10f/0x1b0
[ 24.834661] ? destroy_timer_on_stack+0x40/0x40
[ 24.835417] ? __might_sleep+0x88/0x140
[ 24.836116] main_func.cold+0xb3/0x36d
[ 24.836878] ? __lock_release+0x62/0x340
[ 24.837565] ? __kthread_parkme+0xd2/0x140
[ 24.838337] kthread+0x2c5/0x320
[ 24.838970] ? kzalloc+0x40/0x40
[ 24.839806] ? kthread_create_worker+0x40/0x40
[ 24.840709] ret_from_fork+0x1c/0x28
[ 24.841427] ================================================================================
[ 24.842989] divide_error: 0000 [#1]
[ 24.843812] CPU: 0 PID: 64 Comm: main_func Not tainted 5.9.0-rc1-00019-g4e88ec4a9eb17 #1
[ 24.845435] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 24.847007] EIP: main_func.cold+0xb7/0x36d
[ 24.847763] Code: 15 14 4b c5 c2 00 5e e9 f0 f7 ff ff 83 05 d0 3e c5 c2 01 b8 30 c6 5e c2 83 15 d4 3e c5 c2 00 31 c9 e8 6c 03 4c 00 89 d8 31 d2 <f7> f6 83 05 d8 3e c5 c2 01 89 da 83 15 dc 3e c5 c2 00 31 c9 89 c7
[ 24.850835] EAX: 00000000 EBX: 00000000 ECX: 00000000 EDX: 00000000
[ 24.851857] ESI: 00000000 EDI: 00000000 EBP: e8b91f84 ESP: e8b91ef8
[ 24.852969] DS: 007b ES: 007b FS: 0000 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 24.854183] CR0: 80050033 CR2: 096e3000 CR3: 29832160 CR4: 000406b0
[ 24.855270] Call Trace:
[ 24.855766] ? __lock_release+0x62/0x340
[ 24.856530] ? __kthread_parkme+0xd2/0x140
[ 24.857245] kthread+0x2c5/0x320
[ 24.857759] ? kzalloc+0x40/0x40
[ 24.858547] ? kthread_create_worker+0x40/0x40
[ 24.859387] ret_from_fork+0x1c/0x28
[ 24.860081] Modules linked in: serio_raw i2c_piix4 i2c_core
[ 24.861149] ---[ end trace 59d328920e1c098a ]---
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc1-00019-g4e88ec4a9eb17 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
lkp
7 months, 2 weeks