[sched/fair] 3c29e651e1: hackbench.throughput -15.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -15.2% regression of hackbench.throughput due to commit:
commit: 3c29e651e16dd3b3179cfb2d055ee9538e37515c ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: hackbench
on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
with following parameters:
nr_threads: 100%
mode: threads
ipc: pipe
cpufreq_governor: performance
ucode: 0xca
test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler.
test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc...
In addition to that, the commit also has significant impact on the following tests:
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -11.3% regression |
| test machine | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | iterations=18 |
| | mode=threads |
| | nr_threads=1600% |
| | ucode=0x43 |
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -10.6% regression |
| test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | mode=process |
| | nr_threads=50% |
| | ucode=0xb000038 |
+------------------+-----------------------------------------------------------------------+
| testcase: change | hackbench: hackbench.throughput -6.8% regression |
| test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory |
| test parameters | cpufreq_governor=performance |
| | ipc=pipe |
| | mode=threads |
| | nr_threads=100% |
| | ucode=0xb8 |
+------------------+-----------------------------------------------------------------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-20191114.cgz/lkp-cfl-e1/hackbench/0xca
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
%stddev %change %stddev
\ | \
69492 ± 3% -15.2% 58899 hackbench.throughput
7.727e+08 ± 7% +38.3% 1.069e+09 ± 2% hackbench.time.involuntary_context_switches
131708 ± 4% -15.4% 111382 hackbench.time.minor_page_faults
684.98 ± 2% -2.5% 667.72 hackbench.time.user_time
1.755e+09 ± 2% +11.2% 1.953e+09 hackbench.time.voluntary_context_switches
4.32e+08 ± 4% -15.6% 3.648e+08 hackbench.workload
186.50 ± 12% -16.6% 155.50 ± 11% interrupts.TLB:TLB_shootdowns
4147892 ± 5% +19.6% 4961643 vmstat.system.cs
0.72 ± 10% -0.1 0.58 ± 17% mpstat.cpu.all.idle%
0.00 ±156% +0.0 0.01 ± 62% mpstat.cpu.all.soft%
1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.active_objs
1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.num_objs
40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_hit
40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_local
40724029 ± 10% -34.5% 26661546 proc-vmstat.pgalloc_normal
763906 ± 2% -7.5% 706781 proc-vmstat.pgfault
40701242 ± 10% -34.6% 26637947 proc-vmstat.pgfree
1636058 ± 11% -45.0% 899629 ± 11% turbostat.C1
0.10 ± 7% -0.0 0.07 ± 7% turbostat.C1%
15201 ± 25% +113.3% 32422 ± 8% turbostat.C1E
0.01 ± 57% +0.0 0.03 ± 13% turbostat.C1E%
9093 ± 11% +217.2% 28841 ± 11% turbostat.C3
0.00 ±173% +0.0 0.03 ± 13% turbostat.C3%
16977 ± 2% -92.8% 1222 ± 98% turbostat.C8
0.17 ± 4% -0.2 0.01 ±110% turbostat.C8%
0.13 -100.0% 0.00 turbostat.CPU%c7
9623106 ± 5% -35.1% 6241651 ± 8% cpuidle.C1.time
1636668 ± 11% -45.0% 899926 ± 11% cpuidle.C1.usage
899494 ± 33% +272.6% 3351412 ± 5% cpuidle.C1E.time
15327 ± 25% +112.6% 32592 ± 8% cpuidle.C1E.usage
435674 ± 20% +643.5% 3239351 ± 14% cpuidle.C3.time
9179 ± 11% +215.2% 28933 ± 11% cpuidle.C3.usage
16850924 ± 2% -94.2% 979516 ± 86% cpuidle.C8.time
17150 ± 2% -92.0% 1378 ± 88% cpuidle.C8.usage
6022328 ± 18% -57.1% 2584773 ± 7% cpuidle.POLL.time
5339703 ± 19% -60.1% 2130089 ± 7% cpuidle.POLL.usage
14.06 ± 25% +64.0% 23.07 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.min
25.10 ± 5% -15.8% 21.12 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.stddev
13339 ± 24% +45.7% 19433 ± 7% sched_debug.cfs_rq:/.runnable_weight.min
166.51 ± 24% +48.9% 247.86 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.min
253.56 ± 2% -12.3% 222.30 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev
159602 ± 23% +55.6% 248411 ± 6% sched_debug.cpu.avg_idle.max
51567 ± 24% +49.1% 76905 ± 11% sched_debug.cpu.avg_idle.stddev
11576 ± 5% -9.5% 10478 sched_debug.cpu.curr->pid.avg
9976 ± 8% -16.8% 8305 ± 9% sched_debug.cpu.curr->pid.min
4.81 ± 15% +43.3% 6.89 ± 8% sched_debug.cpu.nr_running.min
6.68 ± 4% -17.0% 5.54 ± 8% sched_debug.cpu.nr_running.stddev
76003928 ± 2% +21.0% 91975904 sched_debug.cpu.nr_switches.avg
78115404 ± 2% +23.4% 96397551 sched_debug.cpu.nr_switches.max
74275755 ± 2% +19.0% 88359903 sched_debug.cpu.nr_switches.min
1043054 ± 15% +83.9% 1917752 ± 10% sched_debug.cpu.nr_switches.stddev
-685.26 +64.8% -1129 sched_debug.cpu.nr_uninterruptible.min
15470 ± 9% -31.0% 10674 ± 9% softirqs.CPU0.SCHED
13756 ± 10% -40.8% 8140 ± 4% softirqs.CPU1.SCHED
13645 ± 11% -39.7% 8227 ± 3% softirqs.CPU10.SCHED
14065 ± 12% -38.8% 8609 ± 2% softirqs.CPU11.SCHED
13948 ± 11% -39.9% 8381 ± 6% softirqs.CPU12.SCHED
13823 ± 12% -39.7% 8332 ± 9% softirqs.CPU13.SCHED
73323 +13.4% 83165 ± 11% softirqs.CPU14.RCU
13872 ± 12% -36.5% 8809 ± 7% softirqs.CPU14.SCHED
13726 ± 12% -40.4% 8175 ± 4% softirqs.CPU15.SCHED
13780 ± 11% -34.8% 8987 ± 12% softirqs.CPU2.SCHED
13895 ± 10% -40.5% 8266 ± 6% softirqs.CPU3.SCHED
221879 +13.4% 251589 ± 14% softirqs.CPU3.TIMER
13873 ± 10% -40.4% 8262 ± 6% softirqs.CPU4.SCHED
72625 +12.0% 81355 ± 10% softirqs.CPU5.RCU
13876 ± 13% -41.4% 8137 ± 3% softirqs.CPU5.SCHED
14118 ± 12% -40.8% 8354 ± 3% softirqs.CPU6.SCHED
13503 ± 12% -40.0% 8099 softirqs.CPU7.SCHED
13549 ± 11% -38.2% 8374 ± 8% softirqs.CPU8.SCHED
14273 ± 10% -40.1% 8548 ± 6% softirqs.CPU9.SCHED
223184 ± 11% -38.9% 136383 ± 5% softirqs.SCHED
55.80 +9.9% 61.34 perf-stat.i.MPKI
1.32 +0.1 1.39 perf-stat.i.branch-miss-rate%
1.066e+08 +3.7% 1.105e+08 perf-stat.i.branch-misses
0.15 ± 8% -0.0 0.11 ± 3% perf-stat.i.cache-miss-rate%
2521584 ± 9% -26.5% 1854353 ± 3% perf-stat.i.cache-misses
2.27e+09 +8.0% 2.451e+09 perf-stat.i.cache-references
4162783 ± 4% +19.7% 4981708 perf-stat.i.context-switches
1.52 +2.0% 1.55 perf-stat.i.cpi
227633 ± 14% +75.1% 398694 ± 3% perf-stat.i.cpu-migrations
33931 ± 12% +42.7% 48428 ± 2% perf-stat.i.cycles-between-cache-misses
0.01 -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate%
1408442 ± 2% -16.4% 1177727 ± 4% perf-stat.i.dTLB-load-misses
1.229e+10 -2.1% 1.203e+10 perf-stat.i.dTLB-loads
0.00 ± 2% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate%
35757 ± 2% -16.0% 30032 ± 3% perf-stat.i.dTLB-store-misses
7.449e+09 -2.3% 7.279e+09 perf-stat.i.dTLB-stores
39246970 -5.5% 37079604 perf-stat.i.iTLB-load-misses
76813 ± 8% -26.2% 56724 ± 6% perf-stat.i.iTLB-loads
4.096e+10 -1.9% 4.02e+10 perf-stat.i.instructions
1062 +4.1% 1105 perf-stat.i.instructions-per-iTLB-miss
0.66 -1.9% 0.65 perf-stat.i.ipc
1210 -7.1% 1124 perf-stat.i.minor-faults
105889 ± 10% -24.8% 79596 ± 5% perf-stat.i.node-loads
174508 ± 7% -24.4% 131915 perf-stat.i.node-stores
1210 -7.1% 1124 perf-stat.i.page-faults
55.42 ± 2% +10.0% 60.98 perf-stat.overall.MPKI
1.31 +0.1 1.38 perf-stat.overall.branch-miss-rate%
0.11 ± 11% -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate%
1.51 +1.9% 1.54 perf-stat.overall.cpi
24740 ± 9% +34.9% 33377 ± 3% perf-stat.overall.cycles-between-cache-misses
0.01 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate%
1043 +3.9% 1084 perf-stat.overall.instructions-per-iTLB-miss
0.66 -1.9% 0.65 perf-stat.overall.ipc
57826 ± 3% +15.6% 66874 perf-stat.overall.path-length
1.064e+08 +3.7% 1.104e+08 perf-stat.ps.branch-misses
2517524 ± 9% -26.5% 1851335 ± 3% perf-stat.ps.cache-misses
2.266e+09 +8.0% 2.447e+09 perf-stat.ps.cache-references
4155898 ± 4% +19.7% 4973458 perf-stat.ps.context-switches
227258 ± 14% +75.1% 398041 ± 3% perf-stat.ps.cpu-migrations
1406119 ± 2% -16.4% 1175777 ± 4% perf-stat.ps.dTLB-load-misses
1.227e+10 -2.1% 1.201e+10 perf-stat.ps.dTLB-loads
35699 ± 2% -16.0% 29983 ± 3% perf-stat.ps.dTLB-store-misses
7.437e+09 -2.3% 7.267e+09 perf-stat.ps.dTLB-stores
39182154 -5.5% 37018206 perf-stat.ps.iTLB-load-misses
76695 ± 8% -26.2% 56638 ± 6% perf-stat.ps.iTLB-loads
4.09e+10 -1.9% 4.013e+10 perf-stat.ps.instructions
1208 -7.1% 1122 perf-stat.ps.minor-faults
105716 ± 10% -24.8% 79465 ± 5% perf-stat.ps.node-loads
174234 ± 7% -24.4% 131703 perf-stat.ps.node-stores
1208 -7.1% 1122 perf-stat.ps.page-faults
29.39 ±104% -29.4 0.00 perf-profile.calltrace.cycles-pp.start_thread
15.86 ±105% -15.9 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread
14.88 ±105% -14.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
14.75 ±105% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
13.53 ±104% -13.5 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
12.51 ±104% -12.5 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread
12.45 ±105% -12.4 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
11.21 ±105% -11.2 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
11.08 ±105% -11.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
10.47 ±105% -10.5 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
9.73 ±105% -9.7 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
3.09 ± 12% -0.6 2.50 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
2.45 ± 14% -0.5 1.95 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
1.15 ± 13% -0.2 0.95 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
0.99 ± 13% -0.2 0.83 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.92 ± 15% +0.2 1.10 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.84 ± 23% +0.3 1.10 ± 2% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.99 ± 19% +0.3 1.26 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.97 ± 23% +0.3 1.26 perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.29 ±100% +0.3 0.60 ± 5% perf-profile.calltrace.cycles-pp.reschedule_interrupt.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write
0.55 ± 62% +0.3 0.89 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.28 ±100% +0.4 0.63 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop
0.33 ±100% +0.4 0.69 ± 2% perf-profile.calltrace.cycles-pp.prepare_to_wait.pipe_wait.pipe_read.new_sync_read.vfs_read
0.29 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
1.00 ± 27% +0.4 1.37 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.57 ± 63% +0.4 0.95 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.46 ± 31% +0.4 1.86 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.72 ± 60% +0.4 1.14 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.29 ±100% +0.4 0.70 ± 6% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.34 ±100% +0.4 0.77 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task
1.26 ± 26% +0.5 1.72 perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.13 ± 26% +0.5 1.59 perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.35 ±100% +0.5 0.83 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
1.28 ± 28% +0.5 1.77 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.40 ± 26% +0.5 1.89 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.44 ±100% +0.5 0.95 ± 7% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
2.33 ± 17% +0.5 2.88 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
1.33 ± 28% +0.6 1.89 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.40 ± 24% +0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
0.83 ± 66% +0.6 1.40 ± 5% perf-profile.calltrace.cycles-pp.native_write_msr
5.01 ± 8% +0.8 5.81 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.75 ± 39% +0.9 2.61 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
1.33 ± 67% +1.0 2.28 ± 6% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
6.56 ± 12% +1.2 7.78 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
8.96 ± 9% +1.6 10.56 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
9.23 ± 10% +1.6 10.87 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
9.17 ± 10% +1.6 10.81 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
3.72 ± 31% +1.7 5.44 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.37 ± 8% +1.9 5.32 ± 4% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.37 ± 10% +2.3 6.65 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
15.27 ± 10% +2.3 17.59 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
15.59 ± 10% +2.4 17.96 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
3.00 ± 71% +2.6 5.60 ± 6% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.10 ± 71% +2.7 5.77 ± 6% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
18.05 ± 10% +2.7 20.73 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
28.21 ± 5% +3.7 31.87 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
24.29 ± 8% +4.7 28.98 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
23.38 ± 8% +4.7 28.08 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
23.72 ± 8% +4.7 28.43 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
31.18 ± 41% +12.7 43.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
33.44 ± 40% +13.2 46.61 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
64.81 ± 43% +27.5 92.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
65.30 ± 43% +27.7 92.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
29.39 ±104% -29.4 0.00 perf-profile.children.cycles-pp.start_thread
16.09 ±105% -16.1 0.00 perf-profile.children.cycles-pp.__GI___libc_write
12.73 ±104% -12.7 0.00 perf-profile.children.cycles-pp.__GI___libc_read
5.80 ± 10% -1.2 4.65 ± 3% perf-profile.children.cycles-pp.security_file_permission
4.10 ± 16% -1.0 3.10 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
3.21 ± 9% -0.7 2.55 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter
1.45 ± 32% -0.6 0.83 perf-profile.children.cycles-pp.entry_SYSCALL_64
3.04 ± 8% -0.6 2.44 ± 3% perf-profile.children.cycles-pp.selinux_file_permission
2.56 ± 11% -0.6 1.99 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter
2.80 ± 9% -0.5 2.25 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
3.38 ± 7% -0.5 2.87 ± 2% perf-profile.children.cycles-pp.mutex_lock
1.62 ± 12% -0.4 1.23 ± 3% perf-profile.children.cycles-pp.___might_sleep
1.82 ± 9% -0.4 1.43 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret
2.09 ± 12% -0.4 1.72 ± 6% perf-profile.children.cycles-pp.file_has_perm
0.90 ± 20% -0.3 0.57 ± 9% perf-profile.children.cycles-pp.__mutex_lock
1.14 ± 12% -0.3 0.82 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate
1.26 ± 10% -0.3 0.97 ± 4% perf-profile.children.cycles-pp.copyin
1.70 ± 8% -0.3 1.42 ± 3% perf-profile.children.cycles-pp.copyout
1.25 ± 11% -0.3 0.98 ± 3% perf-profile.children.cycles-pp.fsnotify
1.04 ± 9% -0.3 0.79 ± 3% perf-profile.children.cycles-pp._cond_resched
1.72 ± 7% -0.2 1.48 ± 5% perf-profile.children.cycles-pp.fput_many
1.16 ± 10% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.__might_sleep
1.09 ± 8% -0.2 0.85 ± 3% perf-profile.children.cycles-pp.__fsnotify_parent
0.83 ± 13% -0.2 0.63 ± 3% perf-profile.children.cycles-pp.__might_fault
0.69 ± 11% -0.1 0.56 perf-profile.children.cycles-pp.current_time
0.99 ± 4% -0.1 0.89 ± 4% perf-profile.children.cycles-pp.touch_atime
0.65 ± 6% -0.1 0.55 ± 6% perf-profile.children.cycles-pp.atime_needs_update
0.39 ± 4% -0.1 0.29 perf-profile.children.cycles-pp.rcu_all_qs
0.13 ± 39% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.34 ± 11% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common
0.49 ± 5% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.wake_up_q
0.24 ± 2% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.30 ± 11% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.inode_has_perm
0.23 ± 8% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.__sb_end_write
0.21 ± 9% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x64_sys_read
0.29 ± 8% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.timespec64_trunc
0.21 ± 8% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.__x64_sys_write
0.22 ± 7% -0.0 0.19 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.08 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.bpf_fd_pass
0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.perf_exclude_event
0.08 ± 24% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.rcu_note_context_switch
0.17 ± 19% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.23 ± 19% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.resched_curr
0.61 ± 8% +0.1 0.72 ± 6% perf-profile.children.cycles-pp.sched_clock
0.67 ± 8% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu
0.50 ± 9% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.account_entity_enqueue
0.61 ± 10% +0.2 0.79 perf-profile.children.cycles-pp.account_entity_dequeue
0.43 ± 19% +0.2 0.62 ± 7% perf-profile.children.cycles-pp.clear_buddies
1.42 ± 14% +0.2 1.66 ± 4% perf-profile.children.cycles-pp.native_write_msr
1.17 ± 13% +0.3 1.42 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup
0.85 ± 23% +0.3 1.12 ± 7% perf-profile.children.cycles-pp.put_prev_entity
0.94 ± 14% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.___perf_sw_event
1.45 ± 11% +0.3 1.73 perf-profile.children.cycles-pp.check_preempt_curr
1.45 ± 12% +0.3 1.76 perf-profile.children.cycles-pp.__enqueue_entity
1.60 ± 10% +0.3 1.91 perf-profile.children.cycles-pp.ttwu_do_wakeup
1.12 ± 9% +0.3 1.43 ± 3% perf-profile.children.cycles-pp.__update_load_avg_se
1.64 ± 8% +0.4 2.00 ± 4% perf-profile.children.cycles-pp.available_idle_cpu
0.96 ± 19% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.pick_next_entity
1.85 ± 16% +0.4 2.28 ± 6% perf-profile.children.cycles-pp.switch_fpu_return
1.90 ± 9% +0.6 2.50 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
3.22 ± 13% +0.6 3.87 perf-profile.children.cycles-pp.update_load_avg
4.03 ± 10% +0.6 4.68 ± 2% perf-profile.children.cycles-pp.update_curr
3.01 ± 14% +0.7 3.72 ± 2% perf-profile.children.cycles-pp.reweight_entity
5.10 ± 8% +0.8 5.87 ± 2% perf-profile.children.cycles-pp.enqueue_entity
4.71 ± 15% +1.1 5.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair
6.73 ± 11% +1.1 7.87 perf-profile.children.cycles-pp.dequeue_task_fair
9.07 ± 9% +1.5 10.61 perf-profile.children.cycles-pp.enqueue_task_fair
9.32 ± 9% +1.6 10.89 perf-profile.children.cycles-pp.ttwu_do_activate
91.46 +1.6 93.03 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
9.27 ± 9% +1.6 10.84 perf-profile.children.cycles-pp.activate_task
4.16 ± 26% +1.6 5.78 ± 6% perf-profile.children.cycles-pp.exit_to_usermode_loop
90.79 +1.7 92.47 perf-profile.children.cycles-pp.do_syscall_64
3.48 ± 8% +1.9 5.36 ± 4% perf-profile.children.cycles-pp.select_idle_sibling
4.44 ± 10% +2.2 6.67 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair
18.46 ± 9% +2.3 20.81 perf-profile.children.cycles-pp.pipe_wait
19.93 ± 11% +3.5 23.46 ± 2% perf-profile.children.cycles-pp.__schedule
28.82 ± 5% +3.6 32.45 perf-profile.children.cycles-pp.__wake_up_common_lock
19.89 ± 12% +3.7 23.62 ± 2% perf-profile.children.cycles-pp.schedule
23.96 ± 8% +4.5 28.45 perf-profile.children.cycles-pp.try_to_wake_up
24.53 ± 8% +4.5 29.06 perf-profile.children.cycles-pp.__wake_up_common
23.91 ± 8% +4.6 28.46 perf-profile.children.cycles-pp.autoremove_wake_function
2.76 ± 9% -0.5 2.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.46 ± 18% -0.4 1.06 ± 4% perf-profile.self.cycles-pp.pipe_write
1.81 ± 9% -0.4 1.43 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret
1.56 ± 12% -0.4 1.19 ± 3% perf-profile.self.cycles-pp.___might_sleep
2.09 ± 7% -0.3 1.81 ± 2% perf-profile.self.cycles-pp.mutex_lock
1.86 ± 8% -0.3 1.59 ± 3% perf-profile.self.cycles-pp.selinux_file_permission
1.21 ± 11% -0.3 0.96 ± 2% perf-profile.self.cycles-pp.fsnotify
1.70 ± 7% -0.2 1.46 ± 5% perf-profile.self.cycles-pp.fput_many
1.04 ± 11% -0.2 0.82 ± 3% perf-profile.self.cycles-pp.__might_sleep
1.01 ± 8% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.__fsnotify_parent
1.04 ± 10% -0.2 0.83 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.67 ± 10% -0.2 0.51 ± 5% perf-profile.self.cycles-pp.new_sync_write
0.64 ± 12% -0.2 0.48 ± 5% perf-profile.self.cycles-pp.vfs_write
1.67 -0.2 1.51 ± 2% perf-profile.self.cycles-pp.pipe_read
0.58 ± 17% -0.1 0.44 ± 7% perf-profile.self.cycles-pp.file_has_perm
0.44 ± 18% -0.1 0.31 ± 7% perf-profile.self.cycles-pp.__mutex_lock
0.49 ± 13% -0.1 0.35 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter
0.64 ± 8% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.67 ± 7% -0.1 0.55 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.30 ± 5% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ksys_write
0.40 ± 12% -0.1 0.30 ± 7% perf-profile.self.cycles-pp.security_file_permission
0.13 ± 38% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.36 ± 11% -0.1 0.28 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate
0.76 ± 5% -0.1 0.68 ± 5% perf-profile.self.cycles-pp.do_syscall_64
0.21 ± 2% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.26 ± 12% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.inode_has_perm
0.29 ± 11% -0.1 0.23 ± 6% perf-profile.self.cycles-pp.current_time
0.28 ± 4% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.rcu_all_qs
0.22 ± 7% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.__sb_end_write
0.31 -0.1 0.26 ± 7% perf-profile.self.cycles-pp.ksys_read
0.20 ± 11% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_read
0.26 ± 8% -0.0 0.22 perf-profile.self.cycles-pp.timespec64_trunc
0.20 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_write
0.24 ± 9% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.__might_fault
0.09 ± 9% -0.0 0.06 perf-profile.self.cycles-pp.copyin
0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.07 ± 10% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.sched_clock
0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.20 ± 12% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__list_add_valid
0.10 ± 26% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.15 ± 16% +0.1 0.21 ± 11% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.35 ± 7% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.prepare_to_wait
0.55 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__fget_light
0.22 ± 18% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.resched_curr
0.48 ± 11% +0.1 0.58 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup
0.39 ± 11% +0.1 0.50 perf-profile.self.cycles-pp.account_entity_enqueue
0.85 ± 7% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.enqueue_entity
0.43 ± 16% +0.1 0.55 ± 7% perf-profile.self.cycles-pp.pick_next_entity
0.86 ± 10% +0.1 0.99 perf-profile.self.cycles-pp.dequeue_task_fair
0.37 ± 21% +0.2 0.55 ± 6% perf-profile.self.cycles-pp.clear_buddies
0.45 ± 14% +0.2 0.65 perf-profile.self.cycles-pp.account_entity_dequeue
1.41 ± 14% +0.2 1.65 ± 4% perf-profile.self.cycles-pp.native_write_msr
0.80 ± 16% +0.3 1.06 ± 4% perf-profile.self.cycles-pp.___perf_sw_event
1.09 ± 9% +0.3 1.39 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
0.84 ± 16% +0.3 1.14 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair
1.45 ± 12% +0.3 1.75 perf-profile.self.cycles-pp.__enqueue_entity
1.62 ± 8% +0.4 1.99 ± 4% perf-profile.self.cycles-pp.available_idle_cpu
1.83 ± 16% +0.4 2.25 ± 6% perf-profile.self.cycles-pp.switch_fpu_return
2.15 ± 11% +0.5 2.60 ± 5% perf-profile.self.cycles-pp.update_curr
1.88 ± 9% +0.6 2.45 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
1.21 ± 9% +0.7 1.92 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
0.80 ± 11% +1.4 2.17 ± 3% perf-profile.self.cycles-pp.select_idle_sibling
hackbench.throughput
74000 +-+-----------------------------------------------------------------+
72000 +-+.+ .+ |
|.+ : .+. .+. + |
70000 +-+ : + + +. : |
68000 +-+ + + : |
| :: |
66000 +-+ + |
64000 +-+ |
62000 +-+ |
| |
60000 +-+ O O OO O O O O O O O OO |
58000 +-+ O O O O O O O O O O O
| O O OO O O |
56000 O-+ O O O O O O |
54000 +-O-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
=========================================================================================
compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/18/x86_64-rhel-7.6/threads/1600%/debian-x86_64-2019-11-14.cgz/lkp-hsw-ep4/hackbench/0x43
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
176075 ± 2% -11.3% 156152 hackbench.throughput
891.79 +12.3% 1001 hackbench.time.elapsed_time
891.79 +12.3% 1001 hackbench.time.elapsed_time.max
1.889e+09 ± 9% +33.1% 2.514e+09 ± 3% hackbench.time.involuntary_context_switches
42939 ± 3% +17.0% 50220 hackbench.time.system_time
18978 +3.2% 19592 hackbench.time.user_time
2.558e+09 ± 8% +33.5% 3.415e+09 ± 2% hackbench.time.voluntary_context_switches
43133704 ± 3% +39.2% 60035429 ± 33% cpuidle.C1.time
48661 +17.7% 57274 ± 5% meminfo.Shmem
19043 ± 2% -14.0% 16377 meminfo.max_used_kB
2502 +1.3% 2535 turbostat.Avg_MHz
3.368e+08 ± 5% +30.6% 4.398e+08 turbostat.IRQ
66.25 +4.5% 69.25 vmstat.cpu.sy
29.25 ± 2% -8.5% 26.75 vmstat.cpu.us
5000443 ± 7% +18.6% 5931772 vmstat.system.cs
370173 ± 3% +17.1% 433616 vmstat.system.in
98786923 ± 7% +74.5% 1.724e+08 numa-numastat.node0.local_node
98791606 ± 7% +74.5% 1.724e+08 numa-numastat.node0.numa_hit
4686 ± 99% +297.8% 18642 ± 24% numa-numastat.node0.other_node
1.818e+08 -51.4% 88447082 ± 3% numa-numastat.node1.local_node
1.819e+08 -51.4% 88451833 ± 3% numa-numastat.node1.numa_hit
18694 ± 25% -74.6% 4753 ± 97% numa-numastat.node1.other_node
155316 +1.5% 157632 proc-vmstat.nr_active_anon
99.50 +5.3% 104.75 ± 3% proc-vmstat.nr_dirtied
4440 +4.9% 4656 proc-vmstat.nr_inactive_anon
6283 +1.9% 6399 proc-vmstat.nr_mapped
12248 ± 2% +16.6% 14276 ± 5% proc-vmstat.nr_shmem
98.75 +5.3% 104.00 proc-vmstat.nr_written
155316 +1.5% 157632 proc-vmstat.nr_zone_active_anon
4440 +4.9% 4656 proc-vmstat.nr_zone_inactive_anon
2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_hit
2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_local
2.83e+08 -7.0% 2.632e+08 proc-vmstat.pgalloc_normal
3212744 +2.3% 3286078 proc-vmstat.pgfault
2.828e+08 -7.0% 2.631e+08 proc-vmstat.pgfree
39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_active_anon
31864 ± 44% +211.0% 99111 ± 4% numa-vmstat.node0.nr_anon_pages
11183 ± 3% +4212.8% 482321 ± 2% numa-vmstat.node0.nr_kernel_stack
499.50 ± 23% +66.9% 833.75 ± 8% numa-vmstat.node0.nr_page_table_pages
11417 -63.5% 4168 ± 5% numa-vmstat.node0.nr_shmem
15905 ± 4% -12.5% 13919 ± 2% numa-vmstat.node0.nr_slab_reclaimable
17996 ± 3% +587.4% 123708 ± 2% numa-vmstat.node0.nr_slab_unreclaimable
39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_zone_active_anon
52471566 ± 6% +73.3% 90915199 numa-vmstat.node0.numa_hit
52466577 ± 6% +73.2% 90896366 numa-vmstat.node0.numa_local
4990 ± 92% +277.5% 18836 ± 24% numa-vmstat.node0.numa_other
114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_active_anon
114545 ± 12% -58.2% 47874 ± 9% numa-vmstat.node1.nr_anon_pages
462075 -96.6% 15717 ± 67% numa-vmstat.node1.nr_kernel_stack
871.50 ± 13% -37.9% 541.00 ± 13% numa-vmstat.node1.nr_page_table_pages
691.25 ± 22% +1359.2% 10086 ± 8% numa-vmstat.node1.nr_shmem
11357 ± 5% +18.1% 13414 ± 4% numa-vmstat.node1.nr_slab_reclaimable
118594 -85.5% 17243 ± 16% numa-vmstat.node1.nr_slab_unreclaimable
114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_zone_active_anon
93308245 ± 2% -49.7% 46921467 ± 4% numa-vmstat.node1.numa_hit
93128224 ± 2% -49.8% 46755145 ± 4% numa-vmstat.node1.numa_local
157762 ± 35% +154.4% 401333 ± 4% numa-meminfo.node0.Active
157721 ± 35% +154.4% 401249 ± 4% numa-meminfo.node0.Active(anon)
128232 ± 44% +212.2% 400337 ± 4% numa-meminfo.node0.AnonPages
63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.KReclaimable
11691 ± 3% +4087.3% 489553 numa-meminfo.node0.KernelStack
1170596 ± 4% +104.2% 2390602 numa-meminfo.node0.MemUsed
2003 ± 23% +67.3% 3351 ± 8% numa-meminfo.node0.PageTables
63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.SReclaimable
72301 ± 3% +591.9% 500285 numa-meminfo.node0.SUnreclaim
45749 -63.5% 16679 ± 5% numa-meminfo.node0.Shmem
135789 ± 3% +309.6% 556136 numa-meminfo.node0.Slab
460025 ± 11% -50.2% 228882 ± 7% numa-meminfo.node1.Active
459902 ± 11% -50.2% 228802 ± 7% numa-meminfo.node1.Active(anon)
459237 ± 11% -58.3% 191354 ± 9% numa-meminfo.node1.AnonPages
45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.KReclaimable
468955 -96.7% 15461 ± 67% numa-meminfo.node1.KernelStack
2241450 -51.9% 1078199 ± 3% numa-meminfo.node1.MemUsed
3493 ± 13% -38.0% 2167 ± 13% numa-meminfo.node1.PageTables
45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.SReclaimable
479010 -85.6% 68750 ± 16% numa-meminfo.node1.SUnreclaim
2779 ± 23% +1354.1% 40419 ± 8% numa-meminfo.node1.Shmem
524580 -76.7% 122408 ± 8% numa-meminfo.node1.Slab
250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.avg
1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev
20766 ± 9% -25.3% 15519 ± 10% sched_debug.cfs_rq:/.load.avg
43820 ± 19% -40.6% 26044 ± 36% sched_debug.cfs_rq:/.load.stddev
250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.max_vruntime.avg
1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev
10990567 ± 9% +22.9% 13509735 ± 7% sched_debug.cfs_rq:/.min_vruntime.stddev
102.40 -33.3% 68.27 ± 57% sched_debug.cfs_rq:/.removed.load_avg.max
4711 -33.4% 3136 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.max
19930 ± 10% -25.1% 14917 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg
-5713498 -349.0% 14226386 ± 8% sched_debug.cfs_rq:/.spread0.avg
8106715 ± 13% +283.1% 31057182 ± 6% sched_debug.cfs_rq:/.spread0.max
-18025645 -99.7% -45393 sched_debug.cfs_rq:/.spread0.min
11027552 ± 9% +21.5% 13398318 ± 7% sched_debug.cfs_rq:/.spread0.stddev
570.46 ± 5% +109.2% 1193 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg
2898 ± 11% +90.0% 5507 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.max
563.27 ± 13% +149.3% 1404 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev
155578 ± 22% +84.1% 286457 ± 28% sched_debug.cpu.avg_idle.min
1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock.stddev
1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock_task.stddev
50207 -12.3% 44017 ± 7% sched_debug.cpu.curr->pid.avg
82673 -19.5% 66514 ± 5% sched_debug.cpu.curr->pid.max
746.05 ± 11% +554.3% 4881 ± 8% sched_debug.cpu.curr->pid.min
30728 ± 2% -40.8% 18187 ± 10% sched_debug.cpu.curr->pid.stddev
0.00 ± 43% +106.0% 0.00 ± 31% sched_debug.cpu.next_balance.stddev
20.49 ± 6% +149.0% 51.03 ± 23% sched_debug.cpu.nr_running.avg
157.83 ± 9% +77.0% 279.37 ± 8% sched_debug.cpu.nr_running.max
29.95 ± 11% +137.1% 70.99 ± 16% sched_debug.cpu.nr_running.stddev
29803727 ± 9% +29.6% 38623870 ± 3% sched_debug.cpu.nr_switches.avg
51549476 ± 10% +22.5% 63171598 ± 3% sched_debug.cpu.nr_switches.max
12792570 ± 3% +43.5% 18362678 ± 4% sched_debug.cpu.nr_switches.min
2.03 ± 80% +430.9% 10.76 ± 38% sched_debug.cpu.nr_uninterruptible.avg
6.99 ± 4% +48.7% 10.39 ± 5% perf-stat.i.MPKI
7.49 ± 5% -1.2 6.29 ± 5% perf-stat.i.cache-miss-rate%
27202880 ± 2% +46.9% 39966416 ± 2% perf-stat.i.cache-misses
4.662e+08 ± 5% +71.5% 7.998e+08 ± 5% perf-stat.i.cache-references
5989366 ± 4% +12.9% 6760958 ± 5% perf-stat.i.context-switches
197319 ± 12% -19.2% 159504 ± 3% perf-stat.i.cpu-migrations
7415 ± 2% -23.3% 5686 ± 3% perf-stat.i.cycles-between-cache-misses
0.61 ± 35% -0.3 0.35 ± 4% perf-stat.i.dTLB-load-miss-rate%
1.506e+08 ± 38% -44.9% 82967539 ± 5% perf-stat.i.dTLB-load-misses
0.22 ± 4% -0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate%
35974976 ± 5% -13.4% 31146077 ± 6% perf-stat.i.dTLB-store-misses
1.627e+10 -3.7% 1.566e+10 perf-stat.i.dTLB-stores
46665620 ± 3% -14.6% 39838962 ± 5% perf-stat.i.iTLB-load-misses
2191 ± 5% +21.4% 2660 ± 7% perf-stat.i.instructions-per-iTLB-miss
0.47 -2.5% 0.46 perf-stat.i.ipc
6660 ± 8% -19.1% 5391 ± 8% perf-stat.i.minor-faults
62.93 ± 2% +11.1 74.06 perf-stat.i.node-load-miss-rate%
12537290 ± 4% +73.5% 21750457 ± 4% perf-stat.i.node-load-misses
5459308 ± 3% -7.4% 5057404 ± 3% perf-stat.i.node-loads
29.39 ± 2% -4.1 25.29 ± 2% perf-stat.i.node-store-miss-rate%
2863317 +13.6% 3252632 ± 2% perf-stat.i.node-store-misses
6381925 ± 2% +53.5% 9796918 ± 4% perf-stat.i.node-stores
6638 ± 8% -19.4% 5348 ± 8% perf-stat.i.page-faults
4.89 ± 4% +78.4% 8.73 perf-stat.overall.MPKI
1.29 +0.0 1.31 perf-stat.overall.branch-miss-rate%
6.13 ± 4% -1.3 4.85 ± 3% perf-stat.overall.cache-miss-rate%
2.15 +2.0% 2.19 perf-stat.overall.cpi
7187 -27.9% 5182 ± 3% perf-stat.overall.cycles-between-cache-misses
0.63 ± 39% -0.3 0.36 ± 5% perf-stat.overall.dTLB-load-miss-rate%
0.23 ± 5% -0.0 0.21 ± 3% perf-stat.overall.dTLB-store-miss-rate%
75.65 -1.1 74.59 perf-stat.overall.iTLB-load-miss-rate%
1604 ± 3% +15.9% 1859 perf-stat.overall.instructions-per-iTLB-miss
0.47 -2.0% 0.46 perf-stat.overall.ipc
63.14 ± 2% +13.6 76.70 perf-stat.overall.node-load-miss-rate%
30.74 ± 2% -5.1 25.62 ± 2% perf-stat.overall.node-store-miss-rate%
47754 ± 2% +11.7% 53321 perf-stat.overall.path-length
24893507 +40.6% 34993733 ± 2% perf-stat.ps.cache-misses
4.072e+08 ± 5% +77.1% 7.212e+08 perf-stat.ps.cache-references
4968881 ± 7% +19.0% 5911450 perf-stat.ps.context-switches
1.789e+11 +1.3% 1.812e+11 perf-stat.ps.cpu-cycles
1.59e+08 ± 39% -44.1% 88801351 ± 6% perf-stat.ps.dTLB-load-misses
2.502e+10 -2.0% 2.453e+10 perf-stat.ps.dTLB-loads
39138578 ± 5% -12.6% 34190129 ± 3% perf-stat.ps.dTLB-store-misses
1.675e+10 -3.7% 1.613e+10 perf-stat.ps.dTLB-stores
51901139 ± 2% -14.3% 44461197 perf-stat.ps.iTLB-load-misses
16711264 ± 2% -9.3% 15153636 ± 2% perf-stat.ps.iTLB-loads
3573 -9.1% 3247 perf-stat.ps.minor-faults
10427704 ± 3% +73.2% 18055799 ± 3% perf-stat.ps.node-load-misses
6085553 ± 4% -10.0% 5479559 ± 2% perf-stat.ps.node-loads
2566511 +13.4% 2911367 ± 2% perf-stat.ps.node-store-misses
5784179 ± 2% +46.2% 8454210 ± 3% perf-stat.ps.node-stores
3573 -9.1% 3247 perf-stat.ps.page-faults
7.427e+13 ± 2% +11.7% 8.293e+13 perf-stat.total.instructions
127394 ± 5% +19.1% 151665 softirqs.CPU0.RCU
321761 ± 3% +12.8% 363107 ± 3% softirqs.CPU0.TIMER
124526 ± 5% +20.2% 149697 ± 3% softirqs.CPU1.RCU
123768 ± 4% +24.7% 154332 ± 2% softirqs.CPU10.RCU
130315 ± 3% +22.1% 159111 softirqs.CPU11.RCU
126684 ± 4% +26.1% 159755 ± 2% softirqs.CPU12.RCU
124649 ± 9% +27.8% 159298 ± 3% softirqs.CPU13.RCU
124528 ± 4% +23.2% 153372 ± 2% softirqs.CPU14.RCU
326150 ± 2% +9.8% 357991 ± 4% softirqs.CPU14.TIMER
121923 ± 6% +17.4% 143119 softirqs.CPU15.RCU
122114 ± 5% +16.7% 142560 softirqs.CPU16.RCU
121425 ± 6% +14.4% 138861 ± 3% softirqs.CPU17.RCU
330948 ± 2% +22.2% 404570 ± 16% softirqs.CPU17.TIMER
312317 ± 4% +18.2% 369091 ± 2% softirqs.CPU18.TIMER
9626 ± 2% +15.0% 11067 ± 3% softirqs.CPU19.SCHED
322308 ± 3% +21.1% 390454 ± 2% softirqs.CPU19.TIMER
124281 ± 5% +22.8% 152670 ± 2% softirqs.CPU2.RCU
9452 ± 2% +15.6% 10927 ± 3% softirqs.CPU20.SCHED
318575 ± 6% +22.3% 389741 ± 3% softirqs.CPU20.TIMER
9327 ± 3% +16.7% 10886 ± 3% softirqs.CPU21.SCHED
306628 ± 4% +23.3% 377962 ± 3% softirqs.CPU21.TIMER
9424 ± 2% +17.6% 11084 ± 3% softirqs.CPU22.SCHED
302796 ± 4% +24.2% 376126 ± 2% softirqs.CPU22.TIMER
9798 ± 2% +11.4% 10918 ± 4% softirqs.CPU23.SCHED
312009 ± 4% +23.6% 385626 ± 2% softirqs.CPU23.TIMER
9559 ± 3% +13.2% 10822 softirqs.CPU24.SCHED
311772 ± 4% +23.0% 383504 ± 2% softirqs.CPU24.TIMER
9514 ± 4% +13.2% 10766 ± 3% softirqs.CPU25.SCHED
307640 ± 4% +22.4% 376479 ± 2% softirqs.CPU25.TIMER
9573 ± 3% +13.8% 10897 ± 3% softirqs.CPU26.SCHED
314607 ± 9% +21.1% 380918 ± 2% softirqs.CPU26.TIMER
9385 +13.7% 10668 ± 2% softirqs.CPU27.SCHED
301949 ± 4% +25.2% 378140 ± 3% softirqs.CPU27.TIMER
9483 ± 2% +14.7% 10874 ± 2% softirqs.CPU28.SCHED
299462 ± 3% +24.9% 373896 ± 2% softirqs.CPU28.TIMER
9505 ± 3% +13.3% 10769 ± 3% softirqs.CPU29.SCHED
307087 ± 4% +23.8% 380066 ± 3% softirqs.CPU29.TIMER
124992 ± 4% +19.4% 149253 ± 3% softirqs.CPU3.RCU
9630 ± 4% +13.6% 10942 ± 3% softirqs.CPU30.SCHED
306382 ± 4% +25.4% 384355 ± 2% softirqs.CPU30.TIMER
9496 ± 6% +12.6% 10689 ± 2% softirqs.CPU31.SCHED
308689 ± 4% +24.5% 384419 ± 2% softirqs.CPU31.TIMER
9714 +12.6% 10937 ± 4% softirqs.CPU32.SCHED
304137 ± 3% +35.9% 413240 ± 13% softirqs.CPU32.TIMER
9598 ± 2% +13.4% 10882 ± 5% softirqs.CPU33.SCHED
9622 +13.8% 10949 ± 3% softirqs.CPU34.SCHED
305882 ± 4% +27.3% 389315 ± 4% softirqs.CPU34.TIMER
9409 ± 4% +15.3% 10853 ± 4% softirqs.CPU35.SCHED
306215 ± 4% +23.8% 379222 ± 2% softirqs.CPU35.TIMER
321617 ± 4% +12.9% 362966 ± 3% softirqs.CPU36.TIMER
134843 ± 6% +19.4% 161053 ± 2% softirqs.CPU37.RCU
130954 ± 6% +20.4% 157716 softirqs.CPU38.RCU
129884 ± 5% +17.8% 152978 softirqs.CPU39.RCU
124744 ± 4% +20.1% 149790 ± 3% softirqs.CPU4.RCU
130593 ± 5% +17.2% 153092 ± 2% softirqs.CPU40.RCU
134185 ± 5% +19.9% 160871 softirqs.CPU41.RCU
133336 ± 5% +18.1% 157409 softirqs.CPU42.RCU
134486 ± 4% +18.7% 159613 softirqs.CPU43.RCU
129857 ± 6% +18.5% 153887 softirqs.CPU44.RCU
131816 ± 5% +20.0% 158144 softirqs.CPU45.RCU
132691 ± 5% +22.4% 162412 softirqs.CPU46.RCU
138736 ± 3% +18.7% 164668 softirqs.CPU47.RCU
132971 ± 6% +24.2% 165142 softirqs.CPU48.RCU
136142 ± 3% +20.8% 164493 softirqs.CPU49.RCU
125011 ± 4% +23.6% 154462 ± 2% softirqs.CPU5.RCU
133660 ± 4% +19.5% 159753 ± 2% softirqs.CPU50.RCU
140405 ± 4% +19.5% 167772 softirqs.CPU51.RCU
140523 ± 4% +18.5% 166576 softirqs.CPU52.RCU
138849 ± 5% +14.6% 159157 ± 4% softirqs.CPU53.RCU
332171 ± 2% +21.4% 403260 ± 15% softirqs.CPU53.TIMER
133037 ± 4% +14.0% 151688 ± 2% softirqs.CPU54.RCU
9584 +14.3% 10959 ± 2% softirqs.CPU54.SCHED
310603 ± 4% +18.0% 366472 ± 3% softirqs.CPU54.TIMER
9565 ± 2% +13.8% 10889 ± 5% softirqs.CPU55.SCHED
321231 ± 3% +21.3% 389753 ± 2% softirqs.CPU55.TIMER
136633 ± 4% +12.6% 153854 softirqs.CPU56.RCU
9430 ± 2% +15.6% 10898 ± 5% softirqs.CPU56.SCHED
319477 ± 6% +22.1% 390107 ± 2% softirqs.CPU56.TIMER
128672 ± 4% +14.1% 146791 ± 3% softirqs.CPU57.RCU
9493 ± 2% +13.1% 10740 ± 3% softirqs.CPU57.SCHED
305670 ± 4% +23.5% 377597 ± 2% softirqs.CPU57.TIMER
127129 ± 4% +15.6% 146993 ± 2% softirqs.CPU58.RCU
9529 ± 3% +13.6% 10824 ± 4% softirqs.CPU58.SCHED
302317 ± 4% +23.8% 374371 ± 3% softirqs.CPU58.TIMER
9427 ± 2% +14.0% 10743 ± 2% softirqs.CPU59.SCHED
310985 ± 4% +23.5% 384119 ± 3% softirqs.CPU59.TIMER
124937 ± 4% +21.6% 151962 ± 3% softirqs.CPU6.RCU
9435 ± 2% +14.1% 10762 ± 3% softirqs.CPU60.SCHED
311646 ± 4% +22.8% 382731 ± 3% softirqs.CPU60.TIMER
9585 +13.5% 10877 ± 2% softirqs.CPU61.SCHED
306857 ± 4% +22.8% 376944 ± 2% softirqs.CPU61.TIMER
9483 ± 2% +12.5% 10671 ± 5% softirqs.CPU62.SCHED
9620 ± 4% +9.7% 10556 ± 4% softirqs.CPU63.SCHED
301085 ± 3% +24.7% 375397 ± 3% softirqs.CPU63.TIMER
300443 ± 4% +23.7% 371620 ± 2% softirqs.CPU64.TIMER
9667 +11.2% 10751 ± 4% softirqs.CPU65.SCHED
307194 ± 4% +23.4% 379020 ± 3% softirqs.CPU65.TIMER
9493 ± 3% +14.9% 10909 ± 3% softirqs.CPU66.SCHED
306226 ± 4% +25.1% 383140 ± 3% softirqs.CPU66.TIMER
9674 ± 3% +13.8% 11010 ± 3% softirqs.CPU67.SCHED
307870 ± 4% +25.0% 384793 ± 2% softirqs.CPU67.TIMER
9451 ± 2% +14.2% 10792 ± 4% softirqs.CPU68.SCHED
301873 ± 3% +26.1% 380748 softirqs.CPU68.TIMER
9542 +13.8% 10863 ± 3% softirqs.CPU69.SCHED
316206 ± 8% +20.5% 380893 ± 2% softirqs.CPU69.TIMER
128158 ± 4% +20.4% 154294 ± 2% softirqs.CPU7.RCU
9456 ± 3% +13.5% 10734 ± 3% softirqs.CPU70.SCHED
305361 ± 4% +36.4% 416546 ± 15% softirqs.CPU70.TIMER
9520 ± 3% +15.4% 10985 ± 4% softirqs.CPU71.SCHED
305605 ± 4% +24.6% 380653 ± 2% softirqs.CPU71.TIMER
122831 ± 4% +21.6% 149424 ± 2% softirqs.CPU8.RCU
123870 ± 4% +23.0% 152396 ± 2% softirqs.CPU9.RCU
9357957 ± 5% +15.4% 10794953 softirqs.RCU
23209193 ± 3% +15.5% 26806870 ± 3% softirqs.TIMER
4.79 ± 4% -3.1 1.71 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.85 ± 4% -2.4 1.41 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.39 ± 4% -2.0 1.35 ± 16% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
2.93 ± 5% -1.9 0.99 ± 18% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
3.25 ± 4% -1.9 1.31 ± 10% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
2.14 ± 4% -1.2 0.91 ± 14% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
2.06 ± 4% -1.2 0.85 ± 13% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
1.52 ± 3% -1.0 0.49 ± 59% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
1.46 ± 3% -1.0 0.46 ± 59% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.53 ± 8% -0.8 0.68 ± 12% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read
1.65 ± 4% -0.8 0.81 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.45 ± 5% -0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.35 ± 5% -0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
1.28 ± 8% -0.7 0.62 ± 12% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write
1.29 ± 3% -0.6 0.67 ± 9% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
0.64 ± 11% +0.3 0.96 ± 7% perf-profile.calltrace.cycles-pp.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
0.00 +0.5 0.55 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.00 +0.6 0.57 ± 6% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.6 0.61 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.7 0.69 ± 8% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule
0.00 +0.7 0.72 ± 7% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.7 0.72 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.8 0.77 ± 5% perf-profile.calltrace.cycles-pp.__switch_to
0.00 +0.8 0.77 ± 6% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read
0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +0.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.find_next_bit.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up
0.00 +0.9 0.91 ± 4% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +0.9 0.91 ± 6% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr
0.00 +1.1 1.05 ± 5% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.1 1.08 ± 8% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.00 +1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
0.00 +1.2 1.20 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
0.00 +1.3 1.30 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +1.3 1.34 ± 6% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +1.6 1.62 ± 4% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.00 +1.9 1.88 ± 5% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read
0.00 +1.9 1.93 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.13 ±173% +2.0 2.11 ± 5% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
0.00 +2.1 2.06 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +2.4 2.38 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.00 +2.7 2.65 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.35 ±103% +4.3 4.67 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.22 ±173% +4.6 4.81 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.23 ±173% +4.7 4.97 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.72 ± 23% +5.0 5.72 ± 5% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
0.69 ± 17% +5.6 6.25 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.74 ± 17% +5.8 6.50 ± 4% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.74 ± 18% +5.8 6.55 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
15.99 ± 54% +7.5 23.51 ± 6% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.13 +7.7 20.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
11.49 +8.7 20.19 ± 3% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
1.31 ± 27% +8.9 10.21 ± 7% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
1.81 ± 22% +11.2 12.98 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
1.89 ± 22% +11.4 13.28 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
2.18 ± 21% +13.1 15.28 ± 4% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
2.01 ± 25% +19.1 21.10 ± 7% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
2.38 ± 23% +20.0 22.40 ± 7% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
19.01 ± 3% +23.4 42.38 ± 4% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
21.13 ± 53% +23.8 44.92 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
17.85 ± 4% +23.9 41.76 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
19.16 ± 53% +24.6 43.74 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.79 ± 22% +30.7 34.45 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
3.84 ± 23% +30.9 34.69 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
4.13 ± 21% +31.0 35.10 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
5.77 ± 14% +31.0 36.76 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
9.20 ± 6% -6.0 3.16 ± 12% perf-profile.children.cycles-pp.syscall_return_via_sysret
8.88 ± 3% -5.8 3.03 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64
8.72 ± 4% -5.3 3.39 ± 11% perf-profile.children.cycles-pp.security_file_permission
4.27 ± 3% -2.4 1.89 ± 9% perf-profile.children.cycles-pp.selinux_file_permission
3.15 ± 5% -2.1 1.07 ± 12% perf-profile.children.cycles-pp.file_has_perm
3.48 ± 3% -2.0 1.46 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter
3.80 ± 3% -2.0 1.79 ± 6% perf-profile.children.cycles-pp.mutex_unlock
3.02 ± 5% -1.9 1.11 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter
3.31 ± 4% -1.6 1.75 ± 6% perf-profile.children.cycles-pp.__fdget_pos
2.55 ± 3% -1.5 1.02 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
3.17 ± 4% -1.5 1.70 ± 6% perf-profile.children.cycles-pp.__fget_light
2.10 ± 7% -1.4 0.71 ± 12% perf-profile.children.cycles-pp.fsnotify
2.52 ± 6% -1.3 1.23 ± 8% perf-profile.children.cycles-pp.__fget
1.77 ± 6% -1.1 0.66 ± 9% perf-profile.children.cycles-pp.avc_has_perm
1.89 ± 5% -1.1 0.78 ± 10% perf-profile.children.cycles-pp.___might_sleep
1.44 ± 4% -1.0 0.48 ± 12% perf-profile.children.cycles-pp.__inode_security_revalidate
1.33 ± 10% -0.9 0.40 ± 16% perf-profile.children.cycles-pp.current_time
1.56 ± 3% -0.9 0.66 ± 9% perf-profile.children.cycles-pp.copyout
1.56 ± 8% -0.8 0.73 ± 7% perf-profile.children.cycles-pp.touch_atime
1.33 ± 4% -0.8 0.53 ± 10% perf-profile.children.cycles-pp.__might_sleep
1.15 ± 9% -0.8 0.36 ± 12% perf-profile.children.cycles-pp.atime_needs_update
1.17 ± 4% -0.7 0.43 ± 13% perf-profile.children.cycles-pp.copyin
1.23 ± 5% -0.7 0.54 ± 9% perf-profile.children.cycles-pp.__fsnotify_parent
1.32 ± 7% -0.6 0.68 ± 6% perf-profile.children.cycles-pp.file_update_time
1.02 ± 3% -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__might_fault
2.54 ± 2% -0.5 2.03 perf-profile.children.cycles-pp.mutex_lock
0.78 ± 4% -0.4 0.36 ± 10% perf-profile.children.cycles-pp.fput_many
0.53 ± 4% -0.4 0.15 ± 21% perf-profile.children.cycles-pp.inode_has_perm
0.44 ± 12% -0.3 0.11 ± 22% perf-profile.children.cycles-pp.iov_iter_init
0.76 ± 4% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.53 ± 3% -0.3 0.21 ± 6% perf-profile.children.cycles-pp.rcu_all_qs
0.41 ± 9% -0.3 0.12 ± 15% perf-profile.children.cycles-pp.timespec64_trunc
0.97 -0.3 0.71 ± 3% perf-profile.children.cycles-pp._cond_resched
0.47 ± 7% -0.3 0.21 ± 11% perf-profile.children.cycles-pp.__x64_sys_read
0.45 ± 7% -0.3 0.20 ± 10% perf-profile.children.cycles-pp.__x64_sys_write
0.37 ± 4% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.44 ± 12% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.__sb_start_write
0.31 ± 6% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__sb_end_write
0.36 ± 7% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.rw_verify_area
0.26 ± 14% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.29 ± 8% -0.2 0.14 ± 16% perf-profile.children.cycles-pp.fpregs_assert_state_consistent
0.22 ± 39% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.__vfs_read
0.21 ± 19% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_write
0.12 ± 17% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.wake_up_q
0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.kill_fasync
0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__mnt_want_write
0.07 ± 25% +0.1 0.12 ± 27% perf-profile.children.cycles-pp.__softirqentry_text_start
0.08 ± 30% +0.1 0.14 ± 26% perf-profile.children.cycles-pp.irq_exit
0.06 ± 6% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__mark_inode_dirty
0.00 +0.1 0.06 perf-profile.children.cycles-pp.preempt_schedule_common
0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.is_cpu_allowed
0.03 ±100% +0.1 0.10 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.01 ±173% +0.1 0.09 ± 32% perf-profile.children.cycles-pp.migrate_task_rq_fair
0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.05 ± 61% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret
0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.native_load_tls
0.03 ±102% +0.1 0.12 ± 30% perf-profile.children.cycles-pp.set_task_cpu
0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.check_cfs_rq_runtime
0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.rb_next
0.10 ± 10% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.generic_update_time
0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.__x64_sys_exit
0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.do_exit
0.00 +0.1 0.11 ± 8% perf-profile.children.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_add_valid
0.00 +0.1 0.12 ± 7% perf-profile.children.cycles-pp.resched_curr
0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off
0.13 ± 10% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid
0.00 +0.1 0.15 ± 4% perf-profile.children.cycles-pp.set_next_buddy
0.00 +0.2 0.16 ± 5% perf-profile.children.cycles-pp.deactivate_task
0.05 ± 9% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.rb_erase
0.06 ± 13% +0.2 0.23 ± 9% perf-profile.children.cycles-pp.finish_wait
0.17 ± 12% +0.2 0.35 ± 2% perf-profile.children.cycles-pp.anon_pipe_buf_release
0.01 ±173% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.cpumask_next
0.07 ± 22% +0.3 0.40 ± 2% perf-profile.children.cycles-pp.prepare_to_wait
0.03 ±100% +0.4 0.39 ± 5% perf-profile.children.cycles-pp.update_min_vruntime
0.03 ±100% +0.4 0.39 ± 12% perf-profile.children.cycles-pp.cpus_share_cache
0.06 ± 14% +0.4 0.43 perf-profile.children.cycles-pp.clear_buddies
0.11 ± 15% +0.4 0.50 ± 3% perf-profile.children.cycles-pp.cpuacct_charge
0.10 ± 26% +0.4 0.49 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.32 ± 31% +0.4 0.75 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending
0.11 ± 26% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.sched_clock
0.34 ± 32% +0.5 0.80 ± 15% perf-profile.children.cycles-pp.scheduler_ipi
0.08 ± 12% +0.5 0.56 ± 7% perf-profile.children.cycles-pp.account_entity_enqueue
0.13 ± 26% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.14 ± 26% +0.6 0.77 ± 4% perf-profile.children.cycles-pp.__calc_delta
0.08 ± 16% +0.7 0.76 ± 3% perf-profile.children.cycles-pp.account_entity_dequeue
0.84 ± 5% +0.7 1.54 ± 5% perf-profile.children.cycles-pp.__lock_text_start
1.61 ± 16% +0.7 2.31 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.42 ± 31% +0.7 1.12 ± 13% perf-profile.children.cycles-pp.reschedule_interrupt
0.24 ± 28% +0.9 1.11 perf-profile.children.cycles-pp.native_write_msr
0.14 ± 22% +0.9 1.01 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup
0.24 ± 32% +0.9 1.12 ± 7% perf-profile.children.cycles-pp.finish_task_switch
0.22 ± 22% +0.9 1.10 ± 3% perf-profile.children.cycles-pp.find_next_bit
0.22 ± 23% +0.9 1.12 ± 3% perf-profile.children.cycles-pp.set_next_entity
0.21 ± 32% +0.9 1.14 ± 3% perf-profile.children.cycles-pp.update_cfs_group
0.16 ± 23% +1.0 1.15 ± 4% perf-profile.children.cycles-pp.check_preempt_curr
0.21 ± 30% +1.0 1.21 ± 2% perf-profile.children.cycles-pp.__switch_to
0.23 ± 21% +1.0 1.24 ± 2% perf-profile.children.cycles-pp.__switch_to_asm
0.16 ± 38% +1.0 1.20 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
0.15 ± 26% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.pick_next_entity
0.15 ± 20% +1.1 1.21 ± 7% perf-profile.children.cycles-pp.put_prev_entity
0.17 ± 22% +1.1 1.23 ± 4% perf-profile.children.cycles-pp.ttwu_do_wakeup
0.19 ± 10% +1.3 1.52 ± 10% perf-profile.children.cycles-pp.__enqueue_entity
0.24 ± 23% +1.3 1.59 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se
0.28 ± 23% +1.4 1.64 ± 3% perf-profile.children.cycles-pp.switch_fpu_return
0.31 ± 29% +1.6 1.88 ± 5% perf-profile.children.cycles-pp.update_rq_clock
0.44 ± 21% +1.8 2.24 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap
0.33 ± 20% +1.8 2.15 ± 4% perf-profile.children.cycles-pp.dequeue_entity
23.28 ± 2% +2.0 25.27 ± 2% perf-profile.children.cycles-pp.ksys_read
0.46 ± 25% +2.1 2.57 ± 5% perf-profile.children.cycles-pp.update_load_avg
0.20 ± 21% +2.1 2.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.91 ± 29% +2.2 3.12 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.38 ± 23% +2.4 2.80 ± 4% perf-profile.children.cycles-pp.reweight_entity
0.52 ± 18% +2.4 2.96 ± 3% perf-profile.children.cycles-pp.enqueue_entity
0.49 ± 23% +2.6 3.10 ± 4% perf-profile.children.cycles-pp.update_curr
20.92 ± 2% +3.2 24.12 ± 2% perf-profile.children.cycles-pp.vfs_read
0.21 ± 23% +3.4 3.65 ± 7% perf-profile.children.cycles-pp._raw_spin_lock
0.76 ± 24% +3.9 4.67 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair
0.66 ± 34% +4.4 5.06 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop
0.81 ± 22% +5.0 5.83 ± 4% perf-profile.children.cycles-pp.dequeue_task_fair
1.01 ± 21% +5.8 6.86 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair
1.06 ± 21% +6.0 7.10 ± 4% perf-profile.children.cycles-pp.activate_task
1.07 ± 21% +6.1 7.15 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate
13.19 +7.8 20.94 ± 3% perf-profile.children.cycles-pp.new_sync_read
11.61 +8.7 20.29 ± 3% perf-profile.children.cycles-pp.pipe_read
1.45 ± 25% +8.9 10.35 ± 7% perf-profile.children.cycles-pp.available_idle_cpu
78.85 ± 2% +9.8 88.66 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
77.90 ± 2% +10.1 88.00 perf-profile.children.cycles-pp.do_syscall_64
2.42 ± 21% +12.9 15.36 ± 4% perf-profile.children.cycles-pp.pipe_wait
2.71 ± 26% +15.4 18.12 ± 4% perf-profile.children.cycles-pp.__schedule
2.76 ± 25% +15.6 18.37 ± 4% perf-profile.children.cycles-pp.schedule
27.59 ± 2% +18.6 46.17 ± 3% perf-profile.children.cycles-pp.ksys_write
2.26 ± 23% +19.0 21.30 ± 7% perf-profile.children.cycles-pp.select_idle_sibling
24.98 ± 2% +19.9 44.88 ± 3% perf-profile.children.cycles-pp.vfs_write
2.48 ± 23% +20.0 22.46 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair
19.06 ± 3% +23.3 42.41 ± 4% perf-profile.children.cycles-pp.new_sync_write
17.98 ± 4% +23.9 41.93 ± 4% perf-profile.children.cycles-pp.pipe_write
4.03 ± 21% +30.6 34.60 ± 6% perf-profile.children.cycles-pp.try_to_wake_up
3.97 ± 22% +30.8 34.74 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function
4.30 ± 20% +30.9 35.19 ± 6% perf-profile.children.cycles-pp.__wake_up_common
6.06 ± 14% +31.2 37.21 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock
24.41 ± 6% -15.5 8.96 ± 10% perf-profile.self.cycles-pp.do_syscall_64
9.18 ± 6% -6.0 3.14 ± 12% perf-profile.self.cycles-pp.syscall_return_via_sysret
8.63 ± 5% -5.6 3.00 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64
3.71 ± 3% -2.0 1.73 ± 6% perf-profile.self.cycles-pp.mutex_unlock
2.47 ± 3% -1.5 0.96 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
2.76 ± 3% -1.4 1.35 ± 8% perf-profile.self.cycles-pp.selinux_file_permission
2.02 ± 7% -1.3 0.68 ± 12% perf-profile.self.cycles-pp.fsnotify
2.46 ± 6% -1.3 1.20 ± 8% perf-profile.self.cycles-pp.__fget
1.75 ± 6% -1.1 0.64 ± 10% perf-profile.self.cycles-pp.avc_has_perm
1.83 ± 5% -1.1 0.75 ± 10% perf-profile.self.cycles-pp.___might_sleep
1.75 -1.0 0.77 ± 7% perf-profile.self.cycles-pp.pipe_write
1.20 ± 4% -0.7 0.46 ± 10% perf-profile.self.cycles-pp.__might_sleep
1.29 ± 11% -0.7 0.58 ± 10% perf-profile.self.cycles-pp.new_sync_read
1.63 ± 4% -0.7 0.97 ± 5% perf-profile.self.cycles-pp.pipe_read
1.15 ± 5% -0.7 0.49 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent
0.89 ± 8% -0.6 0.27 ± 15% perf-profile.self.cycles-pp.copy_page_from_iter
0.89 ± 4% -0.6 0.29 ± 14% perf-profile.self.cycles-pp.security_file_permission
0.83 ± 5% -0.6 0.24 ± 21% perf-profile.self.cycles-pp.file_has_perm
0.90 ± 2% -0.5 0.41 ± 6% perf-profile.self.cycles-pp.new_sync_write
0.67 ± 10% -0.5 0.20 ± 15% perf-profile.self.cycles-pp.current_time
0.83 ± 4% -0.4 0.41 ± 10% perf-profile.self.cycles-pp.copy_page_to_iter
0.72 ± 7% -0.4 0.31 ± 9% perf-profile.self.cycles-pp.vfs_write
0.75 ± 4% -0.4 0.35 ± 9% perf-profile.self.cycles-pp.fput_many
1.01 ± 4% -0.3 0.69 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.45 ± 4% -0.3 0.14 ± 19% perf-profile.self.cycles-pp.inode_has_perm
0.54 ± 8% -0.3 0.23 ± 9% perf-profile.self.cycles-pp.file_update_time
0.41 ± 10% -0.3 0.11 ± 20% perf-profile.self.cycles-pp.iov_iter_init
0.49 ± 7% -0.3 0.19 ± 8% perf-profile.self.cycles-pp.atime_needs_update
0.44 ± 7% -0.3 0.15 ± 14% perf-profile.self.cycles-pp.ksys_read
0.44 ± 7% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.__inode_security_revalidate
0.41 ± 2% -0.3 0.15 ± 8% perf-profile.self.cycles-pp.rcu_all_qs
0.43 ± 8% -0.3 0.17 ± 11% perf-profile.self.cycles-pp.ksys_write
0.37 ± 9% -0.3 0.11 ± 17% perf-profile.self.cycles-pp.timespec64_trunc
0.42 ± 14% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__sb_start_write
0.39 ± 7% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.__x64_sys_write
0.40 ± 8% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.__x64_sys_read
0.41 ± 7% -0.2 0.20 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock
0.60 ± 4% -0.2 0.39 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
1.25 -0.2 1.04 ± 2% perf-profile.self.cycles-pp.mutex_lock
0.30 ± 6% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.__sb_end_write
0.35 ± 5% -0.2 0.18 ± 3% perf-profile.self.cycles-pp.rw_verify_area
0.34 ± 7% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.touch_atime
0.32 ± 4% -0.2 0.15 ± 14% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.23 ± 16% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.74 ± 3% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.vfs_read
0.60 ± 3% -0.2 0.45 ± 4% perf-profile.self.cycles-pp.__fget_light
0.27 ± 8% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.fpregs_assert_state_consistent
0.18 ± 45% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__vfs_read
0.18 ± 5% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.__fdget_pos
0.25 ± 5% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.__might_fault
0.07 ± 7% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.kill_fasync
0.01 ±173% +0.1 0.06 ± 17% perf-profile.self.cycles-pp.sched_ttwu_pending
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.scheduler_ipi
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rcu_note_context_switch
0.01 ±173% +0.1 0.07 ± 12% perf-profile.self.cycles-pp.generic_update_time
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.cpumask_next
0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ttwu_do_activate
0.00 +0.1 0.06 perf-profile.self.cycles-pp.__mnt_want_write
0.06 ± 14% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mark_inode_dirty
0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.is_cpu_allowed
0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.sched_clock_cpu
0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.sched_clock
0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.00 +0.1 0.08 perf-profile.self.cycles-pp.native_load_tls
0.05 ± 60% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret
0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.rb_next
0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.__list_add_valid
0.00 +0.1 0.10 ± 10% perf-profile.self.cycles-pp.rb_insert_color
0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.resched_curr
0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off
0.32 ± 7% +0.1 0.45 ± 5% perf-profile.self.cycles-pp.__wake_up_common
0.13 ± 10% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid
0.00 +0.1 0.13 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function
0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.check_preempt_curr
0.04 ± 58% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.finish_wait
0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.set_next_buddy
0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.put_prev_entity
0.00 +0.2 0.15 ± 5% perf-profile.self.cycles-pp.deactivate_task
0.05 ± 9% +0.2 0.21 ± 5% perf-profile.self.cycles-pp.rb_erase
0.00 +0.2 0.15 ± 10% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.00 +0.2 0.18 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.15 ± 7% +0.2 0.34 ± 2% perf-profile.self.cycles-pp.anon_pipe_buf_release
0.03 ±100% +0.2 0.21 ± 8% perf-profile.self.cycles-pp.pipe_wait
0.03 ±102% +0.2 0.24 ± 3% perf-profile.self.cycles-pp.activate_task
0.01 ±173% +0.2 0.23 ± 3% perf-profile.self.cycles-pp.prepare_to_wait
0.03 ±100% +0.2 0.25 ± 4% perf-profile.self.cycles-pp.set_next_entity
0.02 ±173% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.04 ±100% +0.3 0.29 ± 5% perf-profile.self.cycles-pp.enqueue_entity
0.08 ± 31% +0.3 0.34 ± 2% perf-profile.self.cycles-pp.schedule
0.78 ± 3% +0.3 1.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.09 ± 24% +0.3 0.42 ± 7% perf-profile.self.cycles-pp.finish_task_switch
0.03 ±100% +0.3 0.37 ± 6% perf-profile.self.cycles-pp.update_min_vruntime
0.03 ±102% +0.4 0.38 perf-profile.self.cycles-pp.clear_buddies
0.03 ±100% +0.4 0.39 ± 12% perf-profile.self.cycles-pp.cpus_share_cache
0.07 ± 22% +0.4 0.44 ± 7% perf-profile.self.cycles-pp.check_preempt_wakeup
0.09 ± 27% +0.4 0.47 ± 4% perf-profile.self.cycles-pp.native_sched_clock
0.07 ± 26% +0.4 0.45 ± 4% perf-profile.self.cycles-pp.pick_next_entity
0.11 ± 15% +0.4 0.50 ± 3% perf-profile.self.cycles-pp.cpuacct_charge
0.07 ± 14% +0.5 0.53 ± 8% perf-profile.self.cycles-pp.account_entity_enqueue
0.10 ± 21% +0.5 0.56 ± 8% perf-profile.self.cycles-pp.try_to_wake_up
0.10 ± 18% +0.5 0.62 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair
0.14 ± 26% +0.6 0.75 ± 4% perf-profile.self.cycles-pp.__calc_delta
0.06 ± 13% +0.6 0.68 ± 4% perf-profile.self.cycles-pp.account_entity_dequeue
0.15 ± 27% +0.6 0.79 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair
0.17 ± 30% +0.7 0.89 ± 4% perf-profile.self.cycles-pp.update_load_avg
0.15 ± 20% +0.7 0.88 ± 4% perf-profile.self.cycles-pp.reweight_entity
0.15 ± 23% +0.8 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.24 ± 30% +0.9 1.11 perf-profile.self.cycles-pp.native_write_msr
0.20 ± 23% +0.9 1.07 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair
0.20 ± 22% +0.9 1.07 ± 3% perf-profile.self.cycles-pp.find_next_bit
0.20 ± 31% +0.9 1.11 ± 3% perf-profile.self.cycles-pp.update_cfs_group
0.20 ± 28% +0.9 1.14 ± 2% perf-profile.self.cycles-pp.__switch_to
0.14 ± 37% +1.0 1.13 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
0.21 ± 24% +1.0 1.22 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
0.24 ± 23% +1.1 1.31 ± 5% perf-profile.self.cycles-pp.cpumask_next_wrap
0.23 ± 32% +1.3 1.49 ± 5% perf-profile.self.cycles-pp.update_rq_clock
0.19 ± 28% +1.3 1.45 ± 6% perf-profile.self.cycles-pp.update_curr
0.23 ± 23% +1.3 1.55 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se
0.19 ± 10% +1.3 1.51 ± 10% perf-profile.self.cycles-pp.__enqueue_entity
0.28 ± 22% +1.3 1.63 ± 3% perf-profile.self.cycles-pp.switch_fpu_return
0.11 ± 19% +1.6 1.72 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.42 ± 26% +1.7 2.12 ± 2% perf-profile.self.cycles-pp.__schedule
0.19 ± 20% +2.0 2.20 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.91 ± 29% +2.2 3.11 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.32 ± 19% +7.7 8.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling
1.44 ± 25% +8.8 10.23 ± 7% perf-profile.self.cycles-pp.available_idle_cpu
439.00 +11.7% 490.50 interrupts.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
436.25 +13.1% 493.25 interrupts.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
436.25 +12.4% 490.50 interrupts.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
436.25 +12.4% 490.50 interrupts.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
509.75 ± 3% +545.1% 3288 ±120% interrupts.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
474.00 ± 3% +49.7% 709.75 ± 24% interrupts.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
588.00 ± 35% +196.0% 1740 ± 50% interrupts.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
436.75 ± 2% +12.8% 492.50 interrupts.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
438.00 +39.5% 611.00 ± 33% interrupts.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
436.25 +13.8% 496.25 interrupts.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
436.25 +14.3% 498.50 interrupts.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
437.00 +12.2% 490.50 interrupts.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
436.25 +12.4% 490.50 interrupts.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
436.25 +12.4% 490.50 interrupts.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
441.00 +11.8% 493.00 interrupts.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
437.25 +13.3% 495.25 interrupts.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
436.25 +12.4% 490.50 interrupts.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
436.25 +12.4% 490.50 interrupts.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
436.25 +12.4% 490.50 interrupts.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
436.25 +12.4% 490.50 interrupts.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
437.25 ± 2% +12.2% 490.50 interrupts.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
436.25 +12.6% 491.25 interrupts.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
440.75 +12.4% 495.25 ± 2% interrupts.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
436.25 +12.6% 491.00 interrupts.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
436.75 +13.1% 493.75 interrupts.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
436.25 +12.8% 492.25 interrupts.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
436.25 +12.8% 492.25 interrupts.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
436.25 +12.4% 490.50 interrupts.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
436.25 +12.4% 490.50 interrupts.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
436.25 +12.4% 490.50 interrupts.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
436.25 +14.6% 499.75 ± 2% interrupts.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
436.25 +12.4% 490.50 interrupts.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
436.50 +12.4% 490.50 interrupts.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
436.25 +12.4% 490.50 interrupts.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
436.25 +12.4% 490.50 interrupts.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
436.25 +12.6% 491.00 interrupts.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
441.25 ± 3% +14.1% 503.25 ± 3% interrupts.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
436.25 +12.4% 490.50 interrupts.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
436.25 +14.4% 499.25 ± 3% interrupts.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
439.25 ± 2% +11.7% 490.50 interrupts.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
437.75 +12.1% 490.50 interrupts.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
436.50 +12.4% 490.50 interrupts.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55
436.25 +12.4% 490.50 interrupts.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
439.00 ± 2% +11.7% 490.50 interrupts.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
442.25 ± 3% +10.9% 490.50 interrupts.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
1989 +12.1% 2230 interrupts.9:IR-IO-APIC.9-fasteoi.acpi
6596036 ± 10% -21.1% 5206443 ± 6% interrupts.CAL:Function_call_interrupts
86097 ± 13% -18.4% 70229 ± 7% interrupts.CPU0.CAL:Function_call_interrupts
1785369 +12.4% 2006294 interrupts.CPU0.LOC:Local_timer_interrupts
102823 ± 12% -18.3% 84031 ± 5% interrupts.CPU0.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496548 ± 4% interrupts.CPU0.TRM:Thermal_event_interrupts
1989 +12.1% 2230 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi
90370 ± 8% -21.4% 70995 ± 6% interrupts.CPU1.CAL:Function_call_interrupts
1784834 +12.4% 2006046 interrupts.CPU1.LOC:Local_timer_interrupts
3088391 ± 15% -20.9% 2441882 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts
105391 ± 8% -18.6% 85793 ± 6% interrupts.CPU1.TLB:TLB_shootdowns
388373 ± 12% +27.9% 496558 ± 4% interrupts.CPU1.TRM:Thermal_event_interrupts
88236 ± 11% -19.5% 71001 ± 7% interrupts.CPU10.CAL:Function_call_interrupts
1785471 +12.3% 2004740 interrupts.CPU10.LOC:Local_timer_interrupts
3040774 ± 4% -21.1% 2398396 ± 8% interrupts.CPU10.RES:Rescheduling_interrupts
104483 ± 10% -18.6% 85025 ± 5% interrupts.CPU10.TLB:TLB_shootdowns
388417 ± 12% +27.8% 496560 ± 4% interrupts.CPU10.TRM:Thermal_event_interrupts
1785039 +12.3% 2005274 interrupts.CPU11.LOC:Local_timer_interrupts
925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.NMI:Non-maskable_interrupts
925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.PMI:Performance_monitoring_interrupts
3094193 ± 14% -21.5% 2430228 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts
102398 ± 10% -16.1% 85884 ± 5% interrupts.CPU11.TLB:TLB_shootdowns
388413 ± 12% +27.8% 496474 ± 4% interrupts.CPU11.TRM:Thermal_event_interrupts
588.00 ± 35% +196.0% 1740 ± 50% interrupts.CPU12.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12
87438 ± 9% -18.1% 71580 ± 7% interrupts.CPU12.CAL:Function_call_interrupts
1785376 +12.3% 2005068 interrupts.CPU12.LOC:Local_timer_interrupts
103057 ± 10% -16.4% 86186 ± 6% interrupts.CPU12.TLB:TLB_shootdowns
388416 ± 12% +27.8% 496490 ± 4% interrupts.CPU12.TRM:Thermal_event_interrupts
88735 ± 9% -19.4% 71496 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1785460 +12.3% 2005445 interrupts.CPU13.LOC:Local_timer_interrupts
104230 ± 9% -17.5% 86038 ± 6% interrupts.CPU13.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU13.TRM:Thermal_event_interrupts
88327 ± 12% -19.0% 71507 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
1785094 +12.3% 2005441 interrupts.CPU14.LOC:Local_timer_interrupts
103317 ± 11% -18.1% 84581 ± 6% interrupts.CPU14.TLB:TLB_shootdowns
388412 ± 12% +27.8% 496564 ± 4% interrupts.CPU14.TRM:Thermal_event_interrupts
86972 ± 10% -17.3% 71898 ± 7% interrupts.CPU15.CAL:Function_call_interrupts
1784668 +12.4% 2005375 interrupts.CPU15.LOC:Local_timer_interrupts
3039799 ± 9% -21.1% 2397505 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts
103367 ± 12% -16.9% 85892 ± 6% interrupts.CPU15.TLB:TLB_shootdowns
388342 ± 12% +27.9% 496559 ± 4% interrupts.CPU15.TRM:Thermal_event_interrupts
436.75 ± 2% +12.8% 492.50 interrupts.CPU16.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16
1784750 +12.4% 2005330 interrupts.CPU16.LOC:Local_timer_interrupts
55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.NMI:Non-maskable_interrupts
55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.PMI:Performance_monitoring_interrupts
2969996 ± 10% -18.7% 2413141 ± 9% interrupts.CPU16.RES:Rescheduling_interrupts
102367 ± 11% -13.7% 88305 ± 3% interrupts.CPU16.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496563 ± 4% interrupts.CPU16.TRM:Thermal_event_interrupts
1785363 +12.3% 2005849 interrupts.CPU17.LOC:Local_timer_interrupts
388418 ± 12% +27.8% 496560 ± 4% interrupts.CPU17.TRM:Thermal_event_interrupts
438.00 +39.5% 611.00 ± 33% interrupts.CPU18.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18
88992 ± 12% -22.3% 69145 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1785398 +12.1% 2001316 interrupts.CPU18.LOC:Local_timer_interrupts
2199105 ± 7% +115.2% 4731679 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
108465 ± 8% -23.0% 83467 ± 5% interrupts.CPU18.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380131 ± 2% interrupts.CPU18.TRM:Thermal_event_interrupts
436.25 +13.8% 496.25 interrupts.CPU19.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19
92059 ± 6% -24.2% 69776 ± 5% interrupts.CPU19.CAL:Function_call_interrupts
1784665 +12.2% 2001579 interrupts.CPU19.LOC:Local_timer_interrupts
1284622 ± 7% +225.8% 4185208 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts
108492 ± 7% -23.4% 83059 ± 4% interrupts.CPU19.TLB:TLB_shootdowns
329166 ± 3% +15.5% 380141 ± 2% interrupts.CPU19.TRM:Thermal_event_interrupts
1784512 +12.4% 2005572 interrupts.CPU2.LOC:Local_timer_interrupts
3091132 ± 8% -22.3% 2401342 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts
102562 ± 9% -16.5% 85643 ± 5% interrupts.CPU2.TLB:TLB_shootdowns
388380 ± 12% +27.8% 496544 ± 4% interrupts.CPU2.TRM:Thermal_event_interrupts
436.25 +14.3% 498.50 interrupts.CPU20.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20
91785 ± 9% -23.9% 69853 ± 4% interrupts.CPU20.CAL:Function_call_interrupts
1785284 +12.1% 2001952 interrupts.CPU20.LOC:Local_timer_interrupts
1281621 ± 12% +203.6% 3890516 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts
106297 ± 9% -19.9% 85180 ± 3% interrupts.CPU20.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU20.TRM:Thermal_event_interrupts
437.00 +12.2% 490.50 interrupts.CPU21.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21
92635 ± 7% -23.9% 70512 ± 6% interrupts.CPU21.CAL:Function_call_interrupts
1785480 +12.1% 2001723 interrupts.CPU21.LOC:Local_timer_interrupts
1595447 ± 6% +187.8% 4591004 interrupts.CPU21.RES:Rescheduling_interrupts
107575 ± 7% -21.6% 84296 ± 5% interrupts.CPU21.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380086 ± 2% interrupts.CPU21.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU22.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22
92674 ± 10% -24.4% 70047 ± 6% interrupts.CPU22.CAL:Function_call_interrupts
1784881 +12.1% 2001482 interrupts.CPU22.LOC:Local_timer_interrupts
1605143 ± 6% +179.5% 4486945 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts
108749 ± 10% -23.3% 83409 ± 5% interrupts.CPU22.TLB:TLB_shootdowns
329295 ± 3% +15.4% 380136 ± 2% interrupts.CPU22.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU23.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23
90975 ± 11% -23.6% 69492 ± 5% interrupts.CPU23.CAL:Function_call_interrupts
1785236 +12.1% 2001811 interrupts.CPU23.LOC:Local_timer_interrupts
1600988 ± 2% +180.2% 4486667 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts
106819 ± 10% -22.3% 83007 ± 4% interrupts.CPU23.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380146 ± 2% interrupts.CPU23.TRM:Thermal_event_interrupts
441.00 +11.8% 493.00 interrupts.CPU24.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24
91919 ± 11% -23.1% 70689 ± 5% interrupts.CPU24.CAL:Function_call_interrupts
1784996 +12.1% 2000922 interrupts.CPU24.LOC:Local_timer_interrupts
1612237 ± 5% +188.4% 4649702 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts
108187 ± 8% -22.5% 83831 ± 3% interrupts.CPU24.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380122 ± 2% interrupts.CPU24.TRM:Thermal_event_interrupts
437.25 +13.3% 495.25 interrupts.CPU25.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25
88417 ± 12% -20.8% 70029 ± 4% interrupts.CPU25.CAL:Function_call_interrupts
1785404 +12.0% 1999622 interrupts.CPU25.LOC:Local_timer_interrupts
1612074 ± 2% +182.4% 4551699 interrupts.CPU25.RES:Rescheduling_interrupts
105324 ± 10% -20.1% 84201 ± 4% interrupts.CPU25.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU25.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU26.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26
92632 ± 12% -23.5% 70907 ± 5% interrupts.CPU26.CAL:Function_call_interrupts
1784730 +12.1% 2000264 interrupts.CPU26.LOC:Local_timer_interrupts
1639148 ± 6% +188.0% 4720149 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts
108099 ± 11% -21.6% 84727 ± 4% interrupts.CPU26.TLB:TLB_shootdowns
329271 ± 3% +15.4% 379984 ± 2% interrupts.CPU26.TRM:Thermal_event_interrupts
91312 ± 12% -23.4% 69963 ± 6% interrupts.CPU27.CAL:Function_call_interrupts
1785171 +12.1% 2001743 interrupts.CPU27.LOC:Local_timer_interrupts
1704847 ± 11% +179.4% 4764019 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts
106515 ± 11% -21.6% 83491 ± 5% interrupts.CPU27.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380080 ± 2% interrupts.CPU27.TRM:Thermal_event_interrupts
89178 ± 8% -22.0% 69565 ± 6% interrupts.CPU28.CAL:Function_call_interrupts
1785194 +12.1% 2001836 interrupts.CPU28.LOC:Local_timer_interrupts
1593025 ± 8% +185.7% 4551782 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts
104408 ± 10% -19.2% 84365 ± 6% interrupts.CPU28.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380142 ± 2% interrupts.CPU28.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU29.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29
90850 ± 12% -21.9% 70993 ± 6% interrupts.CPU29.CAL:Function_call_interrupts
1785398 +12.1% 2000545 interrupts.CPU29.LOC:Local_timer_interrupts
1657508 ± 5% +179.4% 4631109 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts
106383 ± 13% -20.8% 84284 ± 4% interrupts.CPU29.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380133 ± 2% interrupts.CPU29.TRM:Thermal_event_interrupts
509.75 ± 3% +545.1% 3288 ±120% interrupts.CPU3.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3
87008 ± 11% -19.9% 69730 ± 5% interrupts.CPU3.CAL:Function_call_interrupts
1784446 +12.4% 2005368 interrupts.CPU3.LOC:Local_timer_interrupts
3169674 ± 9% -20.4% 2521963 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts
103703 ± 12% -17.1% 85958 ± 7% interrupts.CPU3.TLB:TLB_shootdowns
388356 ± 12% +27.9% 496551 ± 4% interrupts.CPU3.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU30.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30
92043 ± 12% -24.2% 69765 ± 5% interrupts.CPU30.CAL:Function_call_interrupts
1785175 +12.1% 2001248 interrupts.CPU30.LOC:Local_timer_interrupts
1637465 ± 6% +191.3% 4769701 ± 3% interrupts.CPU30.RES:Rescheduling_interrupts
106836 ± 11% -22.5% 82810 ± 5% interrupts.CPU30.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU30.TRM:Thermal_event_interrupts
1785301 +12.0% 1999270 interrupts.CPU31.LOC:Local_timer_interrupts
62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.NMI:Non-maskable_interrupts
62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.PMI:Performance_monitoring_interrupts
1589500 ± 5% +184.1% 4515325 interrupts.CPU31.RES:Rescheduling_interrupts
106576 ± 11% -21.5% 83698 ± 5% interrupts.CPU31.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU31.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU32.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32
93348 ± 12% -24.4% 70595 ± 8% interrupts.CPU32.CAL:Function_call_interrupts
1785245 +12.1% 2001145 interrupts.CPU32.LOC:Local_timer_interrupts
1626883 ± 6% +176.3% 4494456 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts
107151 ± 12% -21.4% 84236 ± 5% interrupts.CPU32.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380131 ± 2% interrupts.CPU32.TRM:Thermal_event_interrupts
437.25 ± 2% +12.2% 490.50 interrupts.CPU33.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33
91487 ± 12% -22.7% 70758 ± 3% interrupts.CPU33.CAL:Function_call_interrupts
1785371 +12.0% 1999987 interrupts.CPU33.LOC:Local_timer_interrupts
1578761 ± 6% +201.2% 4754762 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts
106071 ± 12% -21.3% 83430 ± 2% interrupts.CPU33.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU33.TRM:Thermal_event_interrupts
436.25 +12.6% 491.25 interrupts.CPU34.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34
91116 ± 11% -24.3% 68943 ± 7% interrupts.CPU34.CAL:Function_call_interrupts
1785353 +12.1% 2001525 interrupts.CPU34.LOC:Local_timer_interrupts
1553828 ± 4% +186.0% 4444035 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts
105575 ± 10% -20.7% 83762 ± 4% interrupts.CPU34.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380139 ± 2% interrupts.CPU34.TRM:Thermal_event_interrupts
440.75 +12.4% 495.25 ± 2% interrupts.CPU35.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35
91396 ± 11% -25.6% 67986 ± 11% interrupts.CPU35.CAL:Function_call_interrupts
1785237 +12.0% 2000288 interrupts.CPU35.LOC:Local_timer_interrupts
1602616 ± 10% +189.7% 4642115 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts
107669 ± 12% -21.3% 84736 ± 4% interrupts.CPU35.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380145 ± 2% interrupts.CPU35.TRM:Thermal_event_interrupts
436.25 +12.6% 491.00 interrupts.CPU36.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36
93553 ± 8% -19.8% 75044 ± 6% interrupts.CPU36.CAL:Function_call_interrupts
1784636 +12.4% 2005389 interrupts.CPU36.LOC:Local_timer_interrupts
2588984 ± 6% +25.8% 3256348 ± 6% interrupts.CPU36.RES:Rescheduling_interrupts
107053 ± 7% -19.0% 86754 ± 5% interrupts.CPU36.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496560 ± 4% interrupts.CPU36.TRM:Thermal_event_interrupts
436.75 +13.1% 493.75 interrupts.CPU37.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37
89518 ± 10% -18.9% 72619 ± 5% interrupts.CPU37.CAL:Function_call_interrupts
1784648 +12.4% 2005651 interrupts.CPU37.LOC:Local_timer_interrupts
3112476 ± 10% -20.8% 2464103 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts
102576 ± 10% -14.4% 87794 ± 6% interrupts.CPU37.TLB:TLB_shootdowns
388412 ± 12% +27.8% 496563 ± 4% interrupts.CPU37.TRM:Thermal_event_interrupts
436.25 +12.8% 492.25 interrupts.CPU38.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38
92566 ± 10% -20.6% 73466 ± 6% interrupts.CPU38.CAL:Function_call_interrupts
1785132 +12.4% 2005883 interrupts.CPU38.LOC:Local_timer_interrupts
3163520 ± 10% -26.0% 2340820 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts
105823 ± 10% -18.0% 86785 ± 4% interrupts.CPU38.TLB:TLB_shootdowns
388401 ± 12% +27.8% 496554 ± 4% interrupts.CPU38.TRM:Thermal_event_interrupts
436.25 +12.8% 492.25 interrupts.CPU39.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39
93197 ± 11% -21.8% 72854 ± 7% interrupts.CPU39.CAL:Function_call_interrupts
1784624 +12.4% 2005934 interrupts.CPU39.LOC:Local_timer_interrupts
105953 ± 11% -20.8% 83925 ± 6% interrupts.CPU39.TLB:TLB_shootdowns
388418 ± 12% +27.8% 496548 ± 4% interrupts.CPU39.TRM:Thermal_event_interrupts
86883 ± 13% -17.8% 71449 ± 7% interrupts.CPU4.CAL:Function_call_interrupts
1784433 +12.4% 2005462 interrupts.CPU4.LOC:Local_timer_interrupts
3119739 ± 9% -21.1% 2461192 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts
104829 ± 11% -19.1% 84802 ± 5% interrupts.CPU4.TLB:TLB_shootdowns
388417 ± 12% +27.8% 496563 ± 4% interrupts.CPU4.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU40.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40
91881 ± 12% -21.4% 72188 ± 8% interrupts.CPU40.CAL:Function_call_interrupts
1784831 +12.4% 2005991 interrupts.CPU40.LOC:Local_timer_interrupts
3101288 ± 10% -17.2% 2569221 ± 7% interrupts.CPU40.RES:Rescheduling_interrupts
105884 ± 10% -19.0% 85772 ± 5% interrupts.CPU40.TLB:TLB_shootdowns
388415 ± 12% +27.8% 496564 ± 4% interrupts.CPU40.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU41.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41
91739 ± 12% -19.9% 73475 ± 7% interrupts.CPU41.CAL:Function_call_interrupts
1785231 +12.3% 2005398 interrupts.CPU41.LOC:Local_timer_interrupts
3223405 ± 11% -27.9% 2325498 ± 5% interrupts.CPU41.RES:Rescheduling_interrupts
104410 ± 11% -18.7% 84846 ± 6% interrupts.CPU41.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU41.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU42.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42
1785354 +12.4% 2006034 interrupts.CPU42.LOC:Local_timer_interrupts
3068304 ± 9% -22.1% 2390347 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts
107102 ± 9% -20.7% 84890 ± 4% interrupts.CPU42.TLB:TLB_shootdowns
388423 ± 12% +27.8% 496564 ± 4% interrupts.CPU42.TRM:Thermal_event_interrupts
436.25 +14.6% 499.75 ± 2% interrupts.CPU43.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43
92178 ± 9% -18.3% 75306 ± 7% interrupts.CPU43.CAL:Function_call_interrupts
1784759 +12.4% 2005986 interrupts.CPU43.LOC:Local_timer_interrupts
6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.NMI:Non-maskable_interrupts
6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.PMI:Performance_monitoring_interrupts
104773 ± 9% -17.6% 86369 ± 6% interrupts.CPU43.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496556 ± 4% interrupts.CPU43.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU44.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44
92387 ± 10% -21.9% 72174 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
1784603 +12.4% 2006064 interrupts.CPU44.LOC:Local_timer_interrupts
106179 ± 9% -18.6% 86461 ± 6% interrupts.CPU44.TLB:TLB_shootdowns
388426 ± 12% +27.8% 496562 ± 4% interrupts.CPU44.TRM:Thermal_event_interrupts
436.50 +12.4% 490.50 interrupts.CPU45.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45
93641 ± 7% -21.9% 73155 ± 6% interrupts.CPU45.CAL:Function_call_interrupts
1784811 +12.4% 2005654 interrupts.CPU45.LOC:Local_timer_interrupts
3093195 ± 11% -15.6% 2609354 ± 4% interrupts.CPU45.RES:Rescheduling_interrupts
106463 ± 7% -19.6% 85593 ± 5% interrupts.CPU45.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496564 ± 4% interrupts.CPU45.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU46.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46
91739 ± 10% -19.0% 74279 ± 8% interrupts.CPU46.CAL:Function_call_interrupts
1784609 +12.4% 2005522 interrupts.CPU46.LOC:Local_timer_interrupts
2970471 ± 9% -21.0% 2345439 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts
108158 ± 10% -19.8% 86743 ± 4% interrupts.CPU46.TLB:TLB_shootdowns
388368 ± 12% +27.8% 496469 ± 4% interrupts.CPU46.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU47.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47
92856 ± 9% -20.0% 74327 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1784835 +12.4% 2005700 interrupts.CPU47.LOC:Local_timer_interrupts
3116242 ± 10% -20.4% 2480398 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts
104927 ± 9% -18.6% 85393 ± 6% interrupts.CPU47.TLB:TLB_shootdowns
388403 ± 12% +27.8% 496554 ± 4% interrupts.CPU47.TRM:Thermal_event_interrupts
436.25 +12.6% 491.00 interrupts.CPU48.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48
92889 ± 11% -21.5% 72890 ± 8% interrupts.CPU48.CAL:Function_call_interrupts
1784772 +12.4% 2006205 interrupts.CPU48.LOC:Local_timer_interrupts
3175210 ± 12% -19.7% 2548339 ± 13% interrupts.CPU48.RES:Rescheduling_interrupts
105183 ± 11% -18.6% 85618 ± 6% interrupts.CPU48.TLB:TLB_shootdowns
388426 ± 12% +27.8% 496563 ± 4% interrupts.CPU48.TRM:Thermal_event_interrupts
441.25 ± 3% +14.1% 503.25 ± 3% interrupts.CPU49.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49
90134 ± 10% -18.3% 73603 ± 6% interrupts.CPU49.CAL:Function_call_interrupts
1785145 +12.4% 2005932 interrupts.CPU49.LOC:Local_timer_interrupts
3185609 ± 11% -17.6% 2626502 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts
102737 ± 10% -17.3% 84932 ± 5% interrupts.CPU49.TLB:TLB_shootdowns
388425 ± 12% +27.8% 496560 ± 4% interrupts.CPU49.TRM:Thermal_event_interrupts
92114 ± 9% -23.2% 70721 ± 6% interrupts.CPU5.CAL:Function_call_interrupts
1785135 +12.3% 2005424 interrupts.CPU5.LOC:Local_timer_interrupts
3210646 ± 12% -24.8% 2415935 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts
107349 ± 9% -18.9% 87034 ± 6% interrupts.CPU5.TLB:TLB_shootdowns
388418 ± 12% +27.8% 496564 ± 4% interrupts.CPU5.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU50.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50
92406 ± 10% -20.2% 73771 ± 8% interrupts.CPU50.CAL:Function_call_interrupts
1785200 +12.4% 2006488 interrupts.CPU50.LOC:Local_timer_interrupts
106635 ± 9% -19.9% 85431 ± 6% interrupts.CPU50.TLB:TLB_shootdowns
388419 ± 12% +27.8% 496564 ± 4% interrupts.CPU50.TRM:Thermal_event_interrupts
436.25 +14.4% 499.25 ± 3% interrupts.CPU51.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51
91491 ± 9% -20.8% 72473 ± 6% interrupts.CPU51.CAL:Function_call_interrupts
1784683 +12.4% 2005527 interrupts.CPU51.LOC:Local_timer_interrupts
3051494 ± 8% -24.6% 2300032 ± 10% interrupts.CPU51.RES:Rescheduling_interrupts
104954 ± 10% -20.2% 83775 ± 5% interrupts.CPU51.TLB:TLB_shootdowns
388264 ± 12% +27.9% 496462 ± 4% interrupts.CPU51.TRM:Thermal_event_interrupts
92938 ± 11% -19.7% 74594 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
1785451 +12.4% 2006171 interrupts.CPU52.LOC:Local_timer_interrupts
3025681 ± 11% -17.1% 2506800 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts
107100 ± 11% -19.6% 86156 ± 6% interrupts.CPU52.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU52.TRM:Thermal_event_interrupts
439.25 ± 2% +11.7% 490.50 interrupts.CPU53.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53
93000 ± 12% -20.9% 73549 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
1785290 +12.3% 2005441 interrupts.CPU53.LOC:Local_timer_interrupts
3212623 ± 7% -18.2% 2629478 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts
107303 ± 11% -17.5% 88567 ± 10% interrupts.CPU53.TLB:TLB_shootdowns
388411 ± 12% +27.8% 496557 ± 4% interrupts.CPU53.TRM:Thermal_event_interrupts
437.75 +12.1% 490.50 interrupts.CPU54.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54
1785170 +12.0% 2000090 interrupts.CPU54.LOC:Local_timer_interrupts
5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.NMI:Non-maskable_interrupts
5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.PMI:Performance_monitoring_interrupts
2066106 ± 9% +92.7% 3982033 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts
108150 ± 12% -21.2% 85173 ± 5% interrupts.CPU54.TLB:TLB_shootdowns
329294 ± 3% +15.4% 380146 ± 2% interrupts.CPU54.TRM:Thermal_event_interrupts
436.50 +12.4% 490.50 interrupts.CPU55.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55
91164 ± 12% -19.9% 73014 ± 6% interrupts.CPU55.CAL:Function_call_interrupts
1785472 +12.0% 1999115 interrupts.CPU55.LOC:Local_timer_interrupts
1286654 ± 6% +228.2% 4223034 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts
104943 ± 11% -19.5% 84466 ± 5% interrupts.CPU55.TLB:TLB_shootdowns
329292 ± 3% +15.4% 379918 ± 2% interrupts.CPU55.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU56.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56
96934 ± 13% -23.4% 74215 ± 6% interrupts.CPU56.CAL:Function_call_interrupts
1785010 +12.1% 2000169 interrupts.CPU56.LOC:Local_timer_interrupts
1312758 ± 17% +194.5% 3866522 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts
109307 ± 12% -22.1% 85139 ± 4% interrupts.CPU56.TLB:TLB_shootdowns
329273 ± 3% +15.4% 380131 ± 2% interrupts.CPU56.TRM:Thermal_event_interrupts
439.00 ± 2% +11.7% 490.50 interrupts.CPU57.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57
94960 ± 11% -22.9% 73246 ± 5% interrupts.CPU57.CAL:Function_call_interrupts
1785297 +12.0% 2000019 interrupts.CPU57.LOC:Local_timer_interrupts
1666475 ± 10% +169.6% 4492937 interrupts.CPU57.RES:Rescheduling_interrupts
107534 ± 10% -22.0% 83871 ± 5% interrupts.CPU57.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380136 ± 2% interrupts.CPU57.TRM:Thermal_event_interrupts
442.25 ± 3% +10.9% 490.50 interrupts.CPU58.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58
92818 ± 12% -21.4% 72925 ± 5% interrupts.CPU58.CAL:Function_call_interrupts
1785429 +12.0% 1999528 interrupts.CPU58.LOC:Local_timer_interrupts
1606064 ± 5% +176.8% 4446179 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts
103696 ± 11% -19.5% 83458 ± 4% interrupts.CPU58.TLB:TLB_shootdowns
329293 ± 3% +15.4% 380022 ± 2% interrupts.CPU58.TRM:Thermal_event_interrupts
439.00 +11.7% 490.50 interrupts.CPU59.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59
93876 ± 13% -20.5% 74669 ± 6% interrupts.CPU59.CAL:Function_call_interrupts
1785085 +12.0% 1999992 interrupts.CPU59.LOC:Local_timer_interrupts
1634812 ± 5% +174.5% 4487501 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts
106929 ± 13% -21.0% 84522 ± 5% interrupts.CPU59.TLB:TLB_shootdowns
329291 ± 3% +15.4% 380111 ± 2% interrupts.CPU59.TRM:Thermal_event_interrupts
89930 ± 9% -20.5% 71520 ± 7% interrupts.CPU6.CAL:Function_call_interrupts
1785220 +12.3% 2005468 interrupts.CPU6.LOC:Local_timer_interrupts
106307 ± 7% -19.9% 85162 ± 6% interrupts.CPU6.TLB:TLB_shootdowns
388421 ± 12% +27.8% 496564 ± 4% interrupts.CPU6.TRM:Thermal_event_interrupts
436.25 +13.1% 493.25 interrupts.CPU60.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60
98595 ± 11% -25.6% 73379 ± 5% interrupts.CPU60.CAL:Function_call_interrupts
1785116 +12.1% 2000324 interrupts.CPU60.LOC:Local_timer_interrupts
1604314 ± 8% +186.8% 4601614 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts
110442 ± 10% -24.0% 83932 ± 5% interrupts.CPU60.TLB:TLB_shootdowns
329256 ± 3% +15.5% 380140 ± 2% interrupts.CPU60.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU61.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61
93035 ± 11% -21.4% 73124 ± 5% interrupts.CPU61.CAL:Function_call_interrupts
1785176 +12.1% 2000960 interrupts.CPU61.LOC:Local_timer_interrupts
4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.NMI:Non-maskable_interrupts
4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.PMI:Performance_monitoring_interrupts
1575186 ± 6% +194.9% 4644760 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts
103665 ± 11% -18.7% 84259 ± 5% interrupts.CPU61.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380127 ± 2% interrupts.CPU61.TRM:Thermal_event_interrupts
436.25 +12.4% 490.50 interrupts.CPU62.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62
95188 ± 10% -21.9% 74376 ± 6% interrupts.CPU62.CAL:Function_call_interrupts
1785028 +12.0% 1999262 interrupts.CPU62.LOC:Local_timer_interrupts
1613834 ± 8% +184.7% 4594499 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts
106856 ± 10% -21.4% 83987 ± 5% interrupts.CPU62.TLB:TLB_shootdowns
329270 ± 3% +15.4% 380120 ± 2% interrupts.CPU62.TRM:Thermal_event_interrupts
94411 ± 11% -21.3% 74269 ± 5% interrupts.CPU63.CAL:Function_call_interrupts
1785322 +12.0% 1999237 interrupts.CPU63.LOC:Local_timer_interrupts
1669218 ± 6% +187.2% 4794684 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts
105402 ± 10% -18.3% 86076 ± 8% interrupts.CPU63.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU63.TRM:Thermal_event_interrupts
96060 ± 11% -22.6% 74364 ± 6% interrupts.CPU64.CAL:Function_call_interrupts
1785304 +12.0% 1999571 interrupts.CPU64.LOC:Local_timer_interrupts
1681501 ± 7% +164.0% 4438647 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts
108534 ± 10% -22.8% 83813 ± 5% interrupts.CPU64.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU64.TRM:Thermal_event_interrupts
94178 ± 9% -22.7% 72758 ± 8% interrupts.CPU65.CAL:Function_call_interrupts
1785338 +12.0% 1999408 interrupts.CPU65.LOC:Local_timer_interrupts
1674108 ± 4% +182.2% 4724262 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts
107519 ± 10% -22.6% 83259 ± 5% interrupts.CPU65.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380128 ± 2% interrupts.CPU65.TRM:Thermal_event_interrupts
95135 ± 8% -22.4% 73795 ± 5% interrupts.CPU66.CAL:Function_call_interrupts
1785413 +12.0% 2000352 interrupts.CPU66.LOC:Local_timer_interrupts
616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.NMI:Non-maskable_interrupts
616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts
1646166 ± 6% +169.3% 4433267 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts
107190 ± 7% -22.5% 83034 ± 4% interrupts.CPU66.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380118 ± 2% interrupts.CPU66.TRM:Thermal_event_interrupts
95627 ± 10% -22.4% 74249 ± 5% interrupts.CPU67.CAL:Function_call_interrupts
1785314 +12.1% 2001128 interrupts.CPU67.LOC:Local_timer_interrupts
1611146 ± 4% +183.8% 4572204 ± 7% interrupts.CPU67.RES:Rescheduling_interrupts
109283 ± 11% -22.5% 84676 ± 4% interrupts.CPU67.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU67.TRM:Thermal_event_interrupts
95479 ± 11% -20.6% 75791 ± 5% interrupts.CPU68.CAL:Function_call_interrupts
1785192 +12.0% 1999411 interrupts.CPU68.LOC:Local_timer_interrupts
1575394 ± 7% +189.9% 4567277 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts
106972 ± 12% -18.6% 87039 ± 5% interrupts.CPU68.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380129 ± 2% interrupts.CPU68.TRM:Thermal_event_interrupts
93067 ± 14% -19.9% 74538 ± 4% interrupts.CPU69.CAL:Function_call_interrupts
1785257 +12.1% 2000636 interrupts.CPU69.LOC:Local_timer_interrupts
1578663 ± 9% +192.9% 4624518 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts
104271 ± 12% -18.5% 84989 ± 2% interrupts.CPU69.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380066 ± 2% interrupts.CPU69.TRM:Thermal_event_interrupts
474.00 ± 3% +49.7% 709.75 ± 24% interrupts.CPU7.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7
92062 ± 9% -19.7% 73884 ± 5% interrupts.CPU7.CAL:Function_call_interrupts
1784266 +12.4% 2004980 interrupts.CPU7.LOC:Local_timer_interrupts
2505703 ± 11% -19.0% 2029187 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts
107698 ± 8% -18.4% 87903 ± 6% interrupts.CPU7.TLB:TLB_shootdowns
388281 ± 12% +27.8% 496347 ± 4% interrupts.CPU7.TRM:Thermal_event_interrupts
92337 ± 12% -17.5% 76148 ± 2% interrupts.CPU70.CAL:Function_call_interrupts
1785161 +12.0% 1999757 interrupts.CPU70.LOC:Local_timer_interrupts
1562360 ± 4% +170.8% 4231208 ± 9% interrupts.CPU70.RES:Rescheduling_interrupts
103862 ± 10% -18.8% 84371 ± 3% interrupts.CPU70.TLB:TLB_shootdowns
329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU70.TRM:Thermal_event_interrupts
95829 ± 12% -21.2% 75509 ± 5% interrupts.CPU71.CAL:Function_call_interrupts
1784894 +12.1% 2001287 interrupts.CPU71.LOC:Local_timer_interrupts
1533567 ± 7% +196.9% 4553440 interrupts.CPU71.RES:Rescheduling_interrupts
105883 ± 11% -20.4% 84248 ± 6% interrupts.CPU71.TLB:TLB_shootdowns
329247 ± 3% +15.5% 380146 ± 2% interrupts.CPU71.TRM:Thermal_event_interrupts
88372 ± 11% -20.3% 70389 ± 7% interrupts.CPU8.CAL:Function_call_interrupts
1784452 +12.4% 2005305 interrupts.CPU8.LOC:Local_timer_interrupts
103992 ± 10% -17.4% 85887 ± 5% interrupts.CPU8.TLB:TLB_shootdowns
388422 ± 12% +27.8% 496557 ± 4% interrupts.CPU8.TRM:Thermal_event_interrupts
89290 ± 10% -19.1% 72221 ± 6% interrupts.CPU9.CAL:Function_call_interrupts
1784716 +12.4% 2005710 interrupts.CPU9.LOC:Local_timer_interrupts
104340 ± 10% -16.0% 87647 ± 3% interrupts.CPU9.TLB:TLB_shootdowns
388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU9.TRM:Thermal_event_interrupts
1.285e+08 +12.2% 1.442e+08 interrupts.LOC:Local_timer_interrupts
144.00 +50.0% 216.00 interrupts.MCP:Machine_check_polls
1.677e+08 ± 8% +50.4% 2.522e+08 ± 2% interrupts.RES:Rescheduling_interrupts
7624145 ± 10% -19.7% 6125835 ± 5% interrupts.TLB:TLB_shootdowns
25836833 ± 8% +22.2% 31559779 ± 2% interrupts.TRM:Thermal_event_interrupts
***************************************************************************************************
lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-11-14.cgz/lkp-bdw-ep6/hackbench/0xb000038
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
134091 -10.6% 119937 hackbench.throughput
3.352e+09 ± 2% +16.1% 3.892e+09 ± 5% hackbench.time.involuntary_context_switches
535571 ± 2% +10.6% 592420 ± 4% numa-meminfo.node0.FilePages
12614 ± 18% -12.2% 11078 ± 18% numa-meminfo.node1.Mapped
3472 ± 6% +18.0% 4096 slabinfo.skbuff_head_cache.active_objs
3568 ± 6% +15.7% 4128 slabinfo.skbuff_head_cache.num_objs
730128 ± 5% -52.2% 349073 ± 25% turbostat.C3
14.77 +5.9% 15.64 turbostat.RAMWatt
24613783 ± 5% -32.6% 16595193 ± 23% numa-numastat.node0.local_node
24632941 ± 5% -32.6% 16605181 ± 23% numa-numastat.node0.numa_hit
25827961 -29.9% 18097150 ± 19% numa-numastat.node1.local_node
25836948 -29.9% 18115676 ± 19% numa-numastat.node1.numa_hit
50467170 ± 2% -31.2% 34733456 ± 10% proc-vmstat.numa_hit
50439016 ± 2% -31.2% 34704933 ± 10% proc-vmstat.numa_local
50622650 ± 2% -31.1% 34896854 ± 10% proc-vmstat.pgalloc_normal
50548381 ± 2% -31.1% 34834391 ± 10% proc-vmstat.pgfree
79.00 +2.2% 80.75 vmstat.cpu.sy
19.00 -7.9% 17.50 ± 2% vmstat.cpu.us
13450755 +4.6% 14066265 vmstat.system.cs
1122772 -12.1% 986681 vmstat.system.in
133891 ± 2% +10.6% 148112 ± 4% numa-vmstat.node0.nr_file_pages
12301995 ± 4% -30.2% 8590153 ± 22% numa-vmstat.node0.numa_hit
12282910 ± 4% -30.1% 8580139 ± 22% numa-vmstat.node0.numa_local
12964354 ± 5% -31.2% 8914408 ± 18% numa-vmstat.node1.numa_hit
12806275 ± 5% -31.7% 8746928 ± 19% numa-vmstat.node1.numa_local
33816695 ± 10% +17.8% 39824104 ± 7% cpuidle.C1.time
21015951 ± 3% +19.6% 25127666 ± 7% cpuidle.C1E.time
730991 ± 5% -52.1% 350128 ± 25% cpuidle.C3.usage
1.497e+08 ± 28% +41.6% 2.119e+08 ± 15% cpuidle.C6.time
6771631 ± 4% -45.2% 3713804 ± 15% cpuidle.POLL.time
5209842 ± 4% -44.1% 2913245 ± 15% cpuidle.POLL.usage
26965506 +5.5% 28444381 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg
28257967 +6.6% 30126596 ± 4% sched_debug.cfs_rq:/.min_vruntime.max
582203 ± 9% +37.0% 797353 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev
4.65 ± 8% -10.7% 4.16 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev
4620 ± 6% -10.7% 4125 ± 4% sched_debug.cfs_rq:/.runnable_weight.stddev
582831 ± 9% +36.6% 796226 ± 7% sched_debug.cfs_rq:/.spread0.stddev
154.73 ± 2% -14.7% 132.00 ± 3% sched_debug.cfs_rq:/.util_avg.stddev
120.93 ± 25% +43.5% 173.50 ± 19% sched_debug.cfs_rq:/.util_est_enqueued.min
184.93 ± 2% -11.0% 164.54 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev
182135 ± 18% +57.1% 286098 ± 6% sched_debug.cpu.avg_idle.avg
185926 ± 19% +49.0% 276991 ± 9% sched_debug.cpu.avg_idle.stddev
3.32 ± 4% -10.8% 2.96 ± 3% sched_debug.cpu.nr_running.stddev
46710201 +9.0% 50934866 ± 5% sched_debug.cpu.nr_switches.avg
50053735 +16.9% 58491253 ± 7% sched_debug.cpu.nr_switches.max
43751407 +6.7% 46698664 ± 4% sched_debug.cpu.nr_switches.min
1344975 ± 10% +78.9% 2406504 ± 16% sched_debug.cpu.nr_switches.stddev
1.63 ± 3% -32.8% 1.09 ± 11% sched_debug.cpu.nr_uninterruptible.avg
1017 ± 27% -59.5% 411.75 ± 16% interrupts.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6
6547829 -14.4% 5603910 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts
1017 ± 27% -59.5% 411.75 ± 16% interrupts.CPU19.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6
6801598 -14.7% 5801847 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts
6609756 -13.7% 5703084 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts
6446878 ± 2% -11.8% 5689145 ± 7% interrupts.CPU21.RES:Rescheduling_interrupts
6634825 -9.4% 6008203 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts
6801182 -9.8% 6137528 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts
6687252 -12.9% 5821967 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts
6583387 -10.9% 5868659 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts
6585631 -16.6% 5490726 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts
6759973 -13.2% 5867800 ± 7% interrupts.CPU57.RES:Rescheduling_interrupts
6459616 ± 2% -10.5% 5783047 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts
6580818 ± 2% -16.9% 5468945 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts
6723178 ± 2% -12.4% 5887778 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts
6480129 -16.2% 5427168 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts
6439558 ± 2% -11.6% 5691160 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts
6662822 -12.0% 5866390 ± 7% interrupts.CPU68.RES:Rescheduling_interrupts
6602178 -12.0% 5809760 ± 7% interrupts.CPU7.RES:Rescheduling_interrupts
6616172 -14.1% 5685889 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts
6591477 -13.8% 5682655 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts
13.97 +15.8% 16.18 perf-stat.i.MPKI
1.70 -0.1 1.65 perf-stat.i.branch-miss-rate%
3.607e+08 -1.6% 3.549e+08 perf-stat.i.branch-misses
1.22 ± 6% +0.8 2.01 ± 18% perf-stat.i.cache-miss-rate%
17650711 ± 6% +98.0% 34953649 ± 18% perf-stat.i.cache-misses
1.496e+09 +17.3% 1.755e+09 perf-stat.i.cache-references
13503062 +4.6% 14122231 perf-stat.i.context-switches
2946654 ± 2% +18.5% 3491269 ± 3% perf-stat.i.cpu-migrations
29365 ± 8% -30.6% 20387 ± 31% perf-stat.i.cycles-between-cache-misses
0.35 ± 2% +0.0 0.36 ± 2% perf-stat.i.dTLB-store-miss-rate%
2.048e+10 -2.1% 2.006e+10 perf-stat.i.dTLB-stores
55.17 -1.4 53.74 perf-stat.i.iTLB-load-miss-rate%
1.019e+08 +5.5% 1.074e+08 perf-stat.i.iTLB-loads
3068 -3.0% 2975 perf-stat.i.minor-faults
9195204 ± 6% +113.0% 19589023 ± 18% perf-stat.i.node-load-misses
28132 ± 6% +125.3% 63396 ± 13% perf-stat.i.node-loads
58.92 -9.8 49.16 ± 4% perf-stat.i.node-store-miss-rate%
3588196 ± 6% +60.5% 5760265 ± 19% perf-stat.i.node-store-misses
2415953 ± 5% +145.2% 5924042 ± 18% perf-stat.i.node-stores
3068 -3.0% 2975 perf-stat.i.page-faults
13.96 +16.0% 16.19 perf-stat.overall.MPKI
1.70 -0.1 1.65 perf-stat.overall.branch-miss-rate%
1.18 ± 6% +0.8 1.99 ± 18% perf-stat.overall.cache-miss-rate%
13823 ± 6% -47.8% 7221 ± 20% perf-stat.overall.cycles-between-cache-misses
0.35 ± 2% +0.0 0.36 ± 2% perf-stat.overall.dTLB-store-miss-rate%
55.11 -1.4 53.70 perf-stat.overall.iTLB-load-miss-rate%
59.75 -10.5 49.23 perf-stat.overall.node-store-miss-rate%
78192 +13.0% 88370 perf-stat.overall.path-length
3.601e+08 -1.6% 3.543e+08 perf-stat.ps.branch-misses
17622620 ± 6% +98.0% 34899005 ± 18% perf-stat.ps.cache-misses
1.494e+09 +17.3% 1.752e+09 perf-stat.ps.cache-references
13480657 +4.6% 14099389 perf-stat.ps.context-switches
2941749 ± 2% +18.5% 3485631 ± 3% perf-stat.ps.cpu-migrations
2.045e+10 -2.1% 2.003e+10 perf-stat.ps.dTLB-stores
1.017e+08 +5.5% 1.072e+08 perf-stat.ps.iTLB-loads
3064 -3.0% 2972 perf-stat.ps.minor-faults
9180406 ± 6% +113.0% 19558101 ± 18% perf-stat.ps.node-load-misses
28101 ± 6% +125.3% 63314 ± 13% perf-stat.ps.node-loads
3582467 ± 6% +60.5% 5751138 ± 19% perf-stat.ps.node-store-misses
2412086 ± 5% +145.2% 5914636 ± 18% perf-stat.ps.node-stores
3064 -3.0% 2972 perf-stat.ps.page-faults
223485 +9.1% 243918 ± 5% softirqs.CPU0.TIMER
11887 ± 11% -18.2% 9723 ± 6% softirqs.CPU1.SCHED
215411 +9.2% 235267 ± 6% softirqs.CPU1.TIMER
11293 ± 3% -14.3% 9678 ± 10% softirqs.CPU10.SCHED
213916 +9.8% 234908 ± 5% softirqs.CPU10.TIMER
212758 +7.7% 229141 ± 5% softirqs.CPU11.TIMER
11098 ± 2% -15.9% 9337 ± 10% softirqs.CPU12.SCHED
213397 +9.0% 232706 ± 6% softirqs.CPU12.TIMER
212419 +8.3% 230115 ± 6% softirqs.CPU13.TIMER
10777 ± 4% -12.5% 9426 ± 6% softirqs.CPU14.SCHED
11241 ± 3% -11.5% 9950 ± 6% softirqs.CPU15.SCHED
214604 +10.0% 236116 ± 8% softirqs.CPU15.TIMER
11147 -14.9% 9482 ± 8% softirqs.CPU16.SCHED
216343 +8.4% 234558 ± 6% softirqs.CPU16.TIMER
214680 +7.0% 229748 ± 6% softirqs.CPU17.TIMER
212791 +8.6% 230988 ± 6% softirqs.CPU19.TIMER
11211 ± 4% -10.4% 10047 ± 6% softirqs.CPU2.SCHED
219918 +9.8% 241513 ± 6% softirqs.CPU2.TIMER
10825 ± 4% -14.3% 9273 ± 7% softirqs.CPU20.SCHED
215209 +8.8% 234149 ± 6% softirqs.CPU20.TIMER
10869 ± 4% -11.4% 9626 ± 4% softirqs.CPU21.SCHED
216225 +8.7% 235105 ± 5% softirqs.CPU21.TIMER
13013 ± 7% -14.0% 11192 ± 3% softirqs.CPU22.SCHED
220406 +8.9% 239975 ± 4% softirqs.CPU22.TIMER
11925 -14.9% 10145 ± 3% softirqs.CPU23.SCHED
220231 +7.6% 237056 ± 5% softirqs.CPU23.TIMER
11140 ± 2% -11.5% 9861 ± 7% softirqs.CPU24.SCHED
216091 +8.9% 235338 ± 5% softirqs.CPU24.TIMER
11438 ± 3% -16.3% 9578 ± 2% softirqs.CPU25.SCHED
214880 +9.4% 235143 ± 5% softirqs.CPU25.TIMER
214888 +8.8% 233874 ± 5% softirqs.CPU26.TIMER
11410 ± 2% -13.7% 9844 ± 10% softirqs.CPU27.SCHED
215012 +8.7% 233790 ± 4% softirqs.CPU27.TIMER
220618 ± 2% +7.8% 237788 ± 5% softirqs.CPU28.TIMER
11096 ± 5% -14.3% 9513 ± 6% softirqs.CPU29.SCHED
217194 +8.2% 235108 ± 5% softirqs.CPU29.TIMER
11445 ± 2% -16.7% 9536 ± 10% softirqs.CPU3.SCHED
214692 +9.1% 234221 ± 5% softirqs.CPU3.TIMER
11127 ± 5% -10.5% 9954 ± 6% softirqs.CPU30.SCHED
214861 +8.2% 232532 ± 5% softirqs.CPU30.TIMER
217582 +7.8% 234493 ± 4% softirqs.CPU31.TIMER
214606 +8.8% 233473 ± 4% softirqs.CPU32.TIMER
11299 ± 2% -11.7% 9975 ± 6% softirqs.CPU33.SCHED
11503 ± 2% -16.0% 9658 ± 10% softirqs.CPU34.SCHED
213233 +10.9% 236580 ± 6% softirqs.CPU34.TIMER
11162 ± 2% -11.6% 9863 ± 7% softirqs.CPU35.SCHED
11241 -13.9% 9678 ± 6% softirqs.CPU36.SCHED
215681 +9.2% 235446 ± 4% softirqs.CPU36.TIMER
11358 -15.2% 9632 ± 6% softirqs.CPU37.SCHED
219691 ± 2% +18.5% 260407 ± 13% softirqs.CPU37.TIMER
11570 ± 2% -13.1% 10049 ± 6% softirqs.CPU38.SCHED
215559 +8.2% 233189 ± 4% softirqs.CPU38.TIMER
11295 ± 4% -13.1% 9819 ± 10% softirqs.CPU39.SCHED
215781 ± 2% +7.6% 232108 ± 5% softirqs.CPU39.TIMER
11506 ± 7% -13.0% 10006 ± 6% softirqs.CPU4.SCHED
214973 +8.9% 234140 ± 6% softirqs.CPU4.TIMER
11304 ± 2% -13.7% 9759 ± 10% softirqs.CPU40.SCHED
213827 +10.1% 235341 ± 4% softirqs.CPU40.TIMER
11462 ± 4% -13.5% 9913 ± 8% softirqs.CPU41.SCHED
215879 ± 2% +7.2% 231499 ± 5% softirqs.CPU41.TIMER
11418 ± 3% -14.0% 9819 ± 8% softirqs.CPU43.SCHED
214864 +8.3% 232698 ± 4% softirqs.CPU43.TIMER
10706 ± 5% -13.9% 9223 ± 11% softirqs.CPU44.SCHED
217386 +9.4% 237916 ± 5% softirqs.CPU44.TIMER
215191 +9.7% 236017 ± 5% softirqs.CPU45.TIMER
214214 +9.9% 235466 ± 6% softirqs.CPU46.TIMER
214471 +9.9% 235675 ± 5% softirqs.CPU47.TIMER
11251 ± 2% -13.0% 9785 ± 8% softirqs.CPU48.SCHED
212500 +9.9% 233451 ± 6% softirqs.CPU48.TIMER
213609 +9.9% 234657 ± 5% softirqs.CPU49.TIMER
216836 +8.5% 235225 ± 5% softirqs.CPU5.TIMER
215854 +9.0% 235383 ± 5% softirqs.CPU50.TIMER
11468 -14.4% 9820 ± 9% softirqs.CPU51.SCHED
215402 +8.8% 234360 ± 5% softirqs.CPU51.TIMER
214152 +8.8% 233038 ± 5% softirqs.CPU52.TIMER
212843 +8.4% 230716 ± 5% softirqs.CPU53.TIMER
11373 ± 2% -14.9% 9679 ± 7% softirqs.CPU54.SCHED
214008 +9.5% 234380 ± 5% softirqs.CPU54.TIMER
212619 +7.5% 228539 ± 5% softirqs.CPU55.TIMER
11136 ± 3% -16.4% 9312 ± 10% softirqs.CPU56.SCHED
213404 +9.0% 232542 ± 6% softirqs.CPU56.TIMER
11171 ± 4% -11.9% 9837 ± 7% softirqs.CPU57.SCHED
212283 +8.6% 230474 ± 6% softirqs.CPU57.TIMER
11137 ± 4% -14.0% 9578 ± 6% softirqs.CPU58.SCHED
11226 ± 4% -17.2% 9289 ± 4% softirqs.CPU59.SCHED
214193 +23.1% 263668 ± 22% softirqs.CPU59.TIMER
217195 +8.2% 234975 ± 5% softirqs.CPU6.TIMER
11051 ± 4% -14.4% 9460 ± 9% softirqs.CPU60.SCHED
217131 +7.9% 234363 ± 5% softirqs.CPU60.TIMER
11206 ± 3% -12.6% 9793 ± 9% softirqs.CPU61.SCHED
211791 +9.2% 231302 ± 5% softirqs.CPU61.TIMER
214956 ± 2% +8.8% 233770 ± 5% softirqs.CPU62.TIMER
11180 ± 3% -14.8% 9526 ± 8% softirqs.CPU63.SCHED
213413 +8.7% 232008 ± 5% softirqs.CPU63.TIMER
215674 +8.1% 233086 ± 6% softirqs.CPU64.TIMER
11032 ± 3% -15.7% 9302 ± 7% softirqs.CPU65.SCHED
215870 +9.0% 235386 ± 5% softirqs.CPU65.TIMER
11403 ± 3% -14.2% 9786 ± 6% softirqs.CPU66.SCHED
215968 +9.4% 236175 ± 5% softirqs.CPU66.TIMER
11098 ± 3% -7.4% 10281 ± 4% softirqs.CPU67.SCHED
217846 +8.3% 236023 ± 4% softirqs.CPU67.TIMER
11094 -12.2% 9740 ± 7% softirqs.CPU68.SCHED
215512 +9.4% 235796 ± 5% softirqs.CPU68.TIMER
11321 ± 3% -14.1% 9724 ± 3% softirqs.CPU69.SCHED
213564 +9.0% 232867 ± 5% softirqs.CPU69.TIMER
218822 +8.8% 238080 ± 6% softirqs.CPU7.TIMER
11507 ± 2% -12.9% 10026 ± 4% softirqs.CPU70.SCHED
213928 +9.4% 233986 ± 4% softirqs.CPU70.TIMER
213528 +8.5% 231730 ± 4% softirqs.CPU71.TIMER
11262 ± 2% -11.1% 10013 ± 5% softirqs.CPU72.SCHED
11145 ± 3% -14.5% 9533 ± 8% softirqs.CPU73.SCHED
216467 +9.2% 236409 ± 5% softirqs.CPU73.TIMER
11447 ± 4% -13.2% 9935 ± 4% softirqs.CPU74.SCHED
212578 +9.1% 232022 ± 5% softirqs.CPU74.TIMER
215256 +7.8% 232072 ± 5% softirqs.CPU75.TIMER
11324 ± 4% -11.8% 9985 ± 3% softirqs.CPU76.SCHED
214170 +8.6% 232516 ± 5% softirqs.CPU76.TIMER
11646 ± 5% -15.7% 9824 ± 7% softirqs.CPU77.SCHED
213706 +8.0% 230880 ± 5% softirqs.CPU77.TIMER
11397 ± 6% -13.9% 9810 ± 10% softirqs.CPU78.SCHED
11179 ± 3% -11.5% 9894 ± 5% softirqs.CPU79.SCHED
213215 +8.4% 231081 ± 5% softirqs.CPU79.TIMER
11165 ± 4% -16.2% 9352 ± 9% softirqs.CPU8.SCHED
216508 ± 2% +7.9% 233692 ± 6% softirqs.CPU8.TIMER
11203 ± 2% -13.9% 9643 ± 4% softirqs.CPU80.SCHED
216446 +8.3% 234469 ± 5% softirqs.CPU80.TIMER
11151 ± 2% -12.8% 9726 ± 6% softirqs.CPU81.SCHED
11373 ± 3% -13.0% 9891 ± 6% softirqs.CPU82.SCHED
213597 +8.9% 232608 ± 5% softirqs.CPU82.TIMER
214308 +7.9% 231148 ± 5% softirqs.CPU83.TIMER
214384 +20.9% 259149 ± 15% softirqs.CPU84.TIMER
11458 -13.7% 9887 ± 7% softirqs.CPU86.SCHED
215459 +8.2% 233058 ± 5% softirqs.CPU86.TIMER
11303 ± 4% -12.4% 9904 ± 8% softirqs.CPU87.SCHED
214185 +7.9% 231116 ± 5% softirqs.CPU87.TIMER
11030 ± 2% -13.0% 9593 ± 8% softirqs.CPU9.SCHED
213229 +8.7% 231788 ± 6% softirqs.CPU9.TIMER
995635 -12.9% 867688 ± 5% softirqs.SCHED
19052052 +8.6% 20690944 ± 5% softirqs.TIMER
31.32 -1.6 29.74 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.24 -1.5 28.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
26.06 -0.9 25.17 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.35 ± 5% -0.9 1.48 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
25.36 -0.8 24.54 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
2.75 ± 4% -0.7 2.08 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
3.84 ± 2% -0.6 3.23 ± 2% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
7.83 -0.5 7.30 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
1.68 ± 6% -0.5 1.17 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
8.02 -0.5 7.53 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
8.08 -0.5 7.60 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.63 ± 4% -0.4 1.19 ± 6% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write
1.71 ± 6% -0.4 1.28 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
2.60 -0.4 2.19 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
2.03 ± 2% -0.4 1.64 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.47 -0.4 3.08 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
1.95 ± 2% -0.3 1.61 ± 3% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write
2.67 -0.3 2.35 ± 3% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read
0.56 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.22 ± 4% -0.3 0.94 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write
1.25 ± 2% -0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64
2.64 -0.2 2.41 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.53 -0.2 1.34 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.59 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.34 -0.2 1.16 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
1.07 ± 2% -0.2 0.90 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write
1.05 -0.2 0.88 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule
1.56 ± 2% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read
1.17 ± 4% -0.2 1.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
0.98 ± 2% -0.2 0.82 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.66 ± 2% -0.2 1.50 ± 5% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read
0.73 ± 3% -0.2 0.57 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64
0.96 ± 2% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write
0.89 ± 2% -0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.66 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64
2.42 -0.1 2.32 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read
0.89 ± 4% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
0.74 ± 2% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.84 -0.1 0.79 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.56 -0.0 0.54 ± 2% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate
0.77 ± 2% +0.1 0.82 perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function
0.90 ± 3% +0.1 0.97 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr
0.69 ± 3% +0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
0.80 ± 4% +0.1 0.89 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
1.10 +0.1 1.20 perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.73 ± 3% +0.1 0.83 ± 2% perf-profile.calltrace.cycles-pp.__switch_to
1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
0.63 ± 4% +0.1 0.75 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
0.99 ± 5% +0.1 1.12 ± 5% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.07 ± 3% +0.1 1.21 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
0.64 ± 6% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read
1.41 ± 3% +0.2 1.59 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.77 ± 5% +0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop
1.73 ± 3% +0.3 2.06 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.14 ±173% +0.5 0.63 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read
0.13 ±173% +0.5 0.64 ± 5% perf-profile.calltrace.cycles-pp.migrate_task_rq_fair.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common
1.85 ± 5% +0.5 2.38 ± 3% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.00 +0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait
0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.15 ±173% +1.0 1.14 ± 12% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
87.66 +1.1 88.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
86.87 +1.1 88.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.59 ± 4% +1.4 6.96 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.72 ± 4% +1.4 7.12 ± 3% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.87 ± 4% +1.4 7.28 ± 3% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.70 ± 30% +3.3 4.98 ± 13% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function
35.54 +3.3 38.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
34.28 +3.5 37.82 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.12 +4.1 35.22 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.49 +4.1 34.63 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
22.54 ± 2% +6.0 28.53 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
20.58 ± 2% +6.2 26.76 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
19.97 ± 2% +6.2 26.16 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
19.73 ± 3% +6.2 25.92 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
3.39 ± 20% +7.0 10.39 ± 12% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.89 ± 15% +7.1 11.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
31.34 -1.6 29.75 perf-profile.children.cycles-pp.ksys_read
30.28 -1.5 28.79 perf-profile.children.cycles-pp.vfs_read
4.14 ± 5% -1.3 2.85 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
3.51 ± 6% -1.0 2.54 ± 8% perf-profile.children.cycles-pp.update_cfs_group
26.10 -0.9 25.20 perf-profile.children.cycles-pp.new_sync_read
25.42 -0.8 24.59 perf-profile.children.cycles-pp.pipe_read
4.67 -0.8 3.86 ± 4% perf-profile.children.cycles-pp.security_file_permission
4.33 ± 2% -0.7 3.65 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64
2.18 ± 4% -0.5 1.64 ± 6% perf-profile.children.cycles-pp.mutex_unlock
7.92 -0.5 7.40 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair
8.10 -0.5 7.61 ± 3% perf-profile.children.cycles-pp.activate_task
8.16 -0.5 7.67 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate
4.12 ± 4% -0.5 3.64 ± 5% perf-profile.children.cycles-pp._raw_spin_lock
2.83 -0.4 2.38 ± 6% perf-profile.children.cycles-pp.selinux_file_permission
3.53 -0.4 3.13 ± 4% perf-profile.children.cycles-pp.enqueue_entity
1.99 ± 2% -0.4 1.64 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter
3.34 -0.4 2.99 ± 4% perf-profile.children.cycles-pp.update_load_avg
2.70 -0.3 2.39 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter
2.66 -0.3 2.35 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.93 ± 3% -0.3 0.64 ± 7% perf-profile.children.cycles-pp.__mutex_unlock_slowpath
1.41 ± 4% -0.3 1.13 perf-profile.children.cycles-pp.file_has_perm
2.75 -0.3 2.49 ± 2% perf-profile.children.cycles-pp.dequeue_entity
2.29 ± 2% -0.3 2.04 perf-profile.children.cycles-pp.mutex_lock
0.85 ± 3% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate
0.59 ± 3% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.__mutex_lock
1.79 ± 2% -0.2 1.59 ± 4% perf-profile.children.cycles-pp.__fdget_pos
1.70 ± 2% -0.2 1.52 ± 3% perf-profile.children.cycles-pp.__fget_light
0.93 ± 3% -0.2 0.75 perf-profile.children.cycles-pp.avc_has_perm
1.09 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.copyin
2.33 ± 2% -0.2 2.16 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
0.96 ± 2% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.___might_sleep
1.67 ± 2% -0.2 1.52 ± 5% perf-profile.children.cycles-pp.copyout
0.83 ± 4% -0.1 0.68 ± 2% perf-profile.children.cycles-pp._cond_resched
0.33 ± 5% -0.1 0.19 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common
0.42 ± 2% -0.1 0.28 ± 8% perf-profile.children.cycles-pp.wake_up_q
0.70 ± 3% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.__might_sleep
0.65 ± 5% -0.1 0.54 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent
0.52 ± 2% -0.1 0.43 perf-profile.children.cycles-pp.__might_fault
0.53 ± 5% -0.1 0.45 perf-profile.children.cycles-pp.current_time
0.84 ± 3% -0.1 0.77 perf-profile.children.cycles-pp.fsnotify
0.16 ± 7% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.15 ± 8% -0.1 0.08 ± 15% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.22 ± 3% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.wake_q_add
0.49 -0.1 0.44 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.26 ± 4% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs
0.08 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_read
0.25 ± 4% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.finish_wait
0.38 -0.0 0.34 ± 2% perf-profile.children.cycles-pp.__x64_sys_read
0.43 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.prepare_to_wait
0.14 ± 20% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_tick_fair
0.16 ± 19% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.scheduler_tick
0.10 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner
0.18 ± 8% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.inode_has_perm
0.21 ± 2% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.deactivate_task
0.20 ± 2% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.generic_pipe_buf_confirm
0.13 ± 3% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__sb_end_write
0.14 ± 9% -0.0 0.12 perf-profile.children.cycles-pp.timespec64_trunc
0.14 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret
0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.17 ± 4% -0.0 0.15 perf-profile.children.cycles-pp.iov_iter_init
0.11 ± 4% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.native_apic_msr_eoi_write
0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.__x2apic_send_IPI_dest
0.15 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.rb_insert_color
0.18 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy
0.08 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.interrupt_entry
0.23 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.cpumask_next
0.67 +0.0 0.69 perf-profile.children.cycles-pp.__calc_delta
0.15 ± 3% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.17 ± 6% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.remove_entity_load_avg
0.14 ± 6% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg
0.80 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.check_preempt_wakeup
0.37 ± 3% +0.1 0.45 ± 4% perf-profile.children.cycles-pp.account_entity_enqueue
0.79 ± 4% +0.1 0.88 ± 5% perf-profile.children.cycles-pp.finish_task_switch
0.80 ± 2% +0.1 0.89 ± 2% perf-profile.children.cycles-pp.put_prev_entity
0.54 ± 2% +0.1 0.65 perf-profile.children.cycles-pp.account_entity_dequeue
1.01 ± 5% +0.1 1.13 ± 5% perf-profile.children.cycles-pp.set_task_cpu
0.49 ± 7% +0.2 0.65 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair
1.23 ± 4% +0.2 1.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm
1.15 ± 2% +0.2 1.33 ± 2% perf-profile.children.cycles-pp.__switch_to
1.42 ± 3% +0.2 1.59 ± 2% perf-profile.children.cycles-pp.switch_fpu_return
0.76 ± 3% +0.2 0.96 ± 3% perf-profile.children.cycles-pp.pick_next_entity
2.21 +0.2 2.42 perf-profile.children.cycles-pp.reweight_entity
2.20 ± 2% +0.2 2.43 perf-profile.children.cycles-pp.load_new_mm_cr3
1.24 ± 5% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.update_rq_clock
1.43 ± 5% +0.3 1.73 ± 2% perf-profile.children.cycles-pp.___perf_sw_event
0.38 ± 15% +0.3 0.68 ± 9% perf-profile.children.cycles-pp.find_next_bit
3.96 +0.3 4.28 perf-profile.children.cycles-pp.pick_next_task_fair
4.43 ± 2% +0.4 4.82 perf-profile.children.cycles-pp.switch_mm_irqs_off
1.11 +0.7 1.83 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.46 ± 26% +0.7 1.20 ± 12% perf-profile.children.cycles-pp.cpumask_next_wrap
87.76 +1.1 88.86 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
86.98 +1.1 88.10 perf-profile.children.cycles-pp.do_syscall_64
21.71 +1.3 23.02 perf-profile.children.cycles-pp.__schedule
5.98 ± 4% +1.4 7.34 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop
21.59 +1.4 22.99 perf-profile.children.cycles-pp.schedule
1.73 ± 29% +3.3 5.03 ± 13% perf-profile.children.cycles-pp.available_idle_cpu
35.55 +3.3 38.88 perf-profile.children.cycles-pp.ksys_write
34.31 +3.5 37.84 perf-profile.children.cycles-pp.vfs_write
31.14 +4.1 35.23 ± 2% perf-profile.children.cycles-pp.new_sync_write
30.56 +4.2 34.74 ± 2% perf-profile.children.cycles-pp.pipe_write
22.96 ± 2% +6.0 28.93 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock
20.11 ± 2% +6.1 26.19 ± 3% perf-profile.children.cycles-pp.try_to_wake_up
20.66 ± 2% +6.2 26.82 ± 3% perf-profile.children.cycles-pp.__wake_up_common
20.01 ± 2% +6.2 26.19 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function
3.46 ± 20% +7.0 10.48 ± 12% perf-profile.children.cycles-pp.select_idle_sibling
4.92 ± 15% +7.1 12.01 ± 10% perf-profile.children.cycles-pp.select_task_rq_fair
11.85 ± 3% -2.1 9.72 ± 2% perf-profile.self.cycles-pp.do_syscall_64
4.13 ± 5% -1.3 2.85 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
3.49 ± 6% -1.0 2.52 ± 9% perf-profile.self.cycles-pp.update_cfs_group
4.33 ± 2% -0.7 3.64 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
4.24 ± 2% -0.6 3.60 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64
2.12 ± 4% -0.5 1.59 ± 6% perf-profile.self.cycles-pp.mutex_unlock
2.61 -0.3 2.31 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.60 ± 2% -0.3 1.30 ± 5% perf-profile.self.cycles-pp.update_load_avg
1.48 ± 3% -0.2 1.25 ± 2% perf-profile.self.cycles-pp.mutex_lock
1.95 -0.2 1.75 ± 7% perf-profile.self.cycles-pp.selinux_file_permission
1.06 ± 5% -0.2 0.86 ± 5% perf-profile.self.cycles-pp.pipe_write
1.46 ± 2% -0.2 1.28 ± 3% perf-profile.self.cycles-pp.pipe_read
1.65 ± 2% -0.2 1.49 ± 4% perf-profile.self.cycles-pp.__fget_light
0.91 ± 3% -0.2 0.74 perf-profile.self.cycles-pp.avc_has_perm
0.94 ± 2% -0.2 0.79 ± 2% perf-profile.self.cycles-pp.___might_sleep
1.84 -0.1 1.72 ± 2% perf-profile.self.cycles-pp.update_curr
1.05 -0.1 0.93 perf-profile.self.cycles-pp.enqueue_task_fair
0.63 ± 3% -0.1 0.52 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.60 ± 6% -0.1 0.51 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent
0.29 ± 3% -0.1 0.19 ± 5% perf-profile.self.cycles-pp.__mutex_lock
0.38 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.vfs_write
0.51 -0.1 0.43 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter
0.49 -0.1 0.41 ± 2% perf-profile.self.cycles-pp.new_sync_write
0.35 ± 4% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter
0.81 ± 3% -0.1 0.75 perf-profile.self.cycles-pp.fsnotify
0.26 -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate
0.20 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.__mutex_unlock_slowpath
0.29 ± 5% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.file_has_perm
0.22 -0.1 0.16 ± 8% perf-profile.self.cycles-pp.wake_q_add
0.57 ± 2% -0.1 0.52 perf-profile.self.cycles-pp.new_sync_read
0.34 ± 2% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.dequeue_entity
0.26 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ksys_read
0.28 ± 2% -0.0 0.23 perf-profile.self.cycles-pp.security_file_permission
0.22 ± 3% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.finish_wait
0.25 -0.0 0.21 ± 4% perf-profile.self.cycles-pp.ksys_write
0.88 -0.0 0.83 perf-profile.self.cycles-pp.__lock_text_start
0.66 -0.0 0.61 perf-profile.self.cycles-pp.vfs_read
1.37 -0.0 1.32 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.29 ± 4% -0.0 0.25 perf-profile.self.cycles-pp.current_time
0.20 ± 3% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.rcu_all_qs
0.10 ± 8% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner
0.42 -0.0 0.39 ± 2% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.34 -0.0 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_read
0.09 ± 5% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.wake_up_q
0.15 ± 8% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.inode_has_perm
0.14 ± 5% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.rb_next
0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.deactivate_task
0.11 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__fdget_pos
0.24 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.prepare_to_wait
0.14 ± 3% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret
0.13 ± 7% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.timespec64_trunc
0.06 -0.0 0.04 ± 57% perf-profile.self.cycles-pp.copyout
0.11 ± 4% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.native_apic_msr_eoi_write
0.17 ± 4% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_pipe_buf_confirm
0.18 ± 2% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__might_fault
0.17 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy
0.15 ± 2% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.rb_insert_color
0.12 ± 3% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__sb_end_write
0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.06 -0.0 0.05 perf-profile.self.cycles-pp.kill_fasync
0.14 ± 3% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.66 +0.0 0.68 perf-profile.self.cycles-pp.__calc_delta
0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_load_tls
0.14 ± 3% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.67 +0.0 0.70 perf-profile.self.cycles-pp.dequeue_task_fair
0.23 ± 3% +0.0 0.27 ± 3% perf-profile.self.cycles-pp._cond_resched
0.17 ± 4% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.activate_task
0.35 ± 3% +0.0 0.40 perf-profile.self.cycles-pp.pick_next_entity
1.20 +0.1 1.25 perf-profile.self.cycles-pp.select_task_rq_fair
0.13 ± 6% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.attach_entity_load_avg
0.87 ± 3% +0.1 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair
0.27 ± 6% +0.1 0.36 ± 7% perf-profile.self.cycles-pp.migrate_task_rq_fair
0.32 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.account_entity_enqueue
0.42 +0.1 0.56 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue
2.24 ± 3% +0.1 2.39 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off
1.09 ± 2% +0.2 1.26 ± 2% perf-profile.self.cycles-pp.__switch_to
1.20 ± 4% +0.2 1.37 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
1.40 ± 3% +0.2 1.58 ± 2% perf-profile.self.cycles-pp.switch_fpu_return
0.89 ± 6% +0.2 1.10 ± 5% perf-profile.self.cycles-pp.update_rq_clock
2.19 ± 2% +0.2 2.42 perf-profile.self.cycles-pp.load_new_mm_cr3
1.33 ± 4% +0.3 1.60 ± 3% perf-profile.self.cycles-pp.___perf_sw_event
0.34 ± 14% +0.3 0.66 ± 9% perf-profile.self.cycles-pp.find_next_bit
0.28 ± 23% +0.4 0.72 ± 12% perf-profile.self.cycles-pp.cpumask_next_wrap
0.89 +0.7 1.58 ± 4% perf-profile.self.cycles-pp._raw_spin_lock
1.04 +0.7 1.76 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.65 ± 12% +3.0 3.61 ± 13% perf-profile.self.cycles-pp.select_idle_sibling
1.71 ± 29% +3.3 4.97 ± 13% perf-profile.self.cycles-pp.available_idle_cpu
***************************************************************************************************
lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory
=========================================================================================
compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode:
gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2018-04-03.cgz/lkp-cfl-e1/hackbench/0xb8
commit:
43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq")
3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found")
43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:2 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
1:2 -50% :4 dmesg.WARNING:stack_recursion
%stddev %change %stddev
\ | \
49845 -6.8% 46466 hackbench.throughput
8.776e+08 +13.0% 9.915e+08 hackbench.time.involuntary_context_switches
93623 -6.3% 87707 hackbench.time.minor_page_faults
740.08 -1.4% 729.54 hackbench.time.user_time
1.569e+09 +3.5% 1.623e+09 hackbench.time.voluntary_context_switches
3.072e+08 -6.2% 2.88e+08 hackbench.workload
21.45 -1.7% 21.09 boot-time.boot
1747212 ± 3% -31.0% 1205072 ± 14% cpuidle.POLL.time
1289000 ± 4% -36.6% 817407 ± 13% cpuidle.POLL.usage
4038486 +6.3% 4293331 vmstat.system.cs
394711 -10.2% 354353 ± 2% vmstat.system.in
10059 -15.4% 8506 ± 5% softirqs.CPU0.SCHED
309881 -27.3% 225159 ± 4% softirqs.CPU2.TIMER
220105 +20.8% 265807 ± 16% softirqs.CPU6.TIMER
120937 ± 3% -17.6% 99667 ± 2% softirqs.SCHED
18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.active_objs
18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.num_objs
8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.active_objs
8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.num_objs
5280 -8.8% 4814 slabinfo.task_delay_info.active_objs
5280 -8.8% 4814 slabinfo.task_delay_info.num_objs
65295 -1.9% 64034 proc-vmstat.nr_active_anon
60605 -1.5% 59671 proc-vmstat.nr_anon_pages
12935 -0.9% 12820 proc-vmstat.nr_slab_unreclaimable
65295 -1.9% 64034 proc-vmstat.nr_zone_active_anon
24774010 -16.6% 20673098 proc-vmstat.numa_hit
24774010 -16.6% 20673098 proc-vmstat.numa_local
24839243 -16.5% 20735391 proc-vmstat.pgalloc_normal
673024 -2.4% 657142 proc-vmstat.pgfault
24822112 -16.5% 20717243 proc-vmstat.pgfree
247.95 ± 14% -19.1% 200.59 ± 4% sched_debug.cfs_rq:/.load_avg.max
43.55 ± 3% -19.9% 34.86 ± 6% sched_debug.cfs_rq:/.load_avg.min
55.51 ± 6% -13.2% 48.19 ± 7% sched_debug.cfs_rq:/.load_avg.stddev
20116 ± 8% -12.8% 17531 ± 5% sched_debug.cfs_rq:/.runnable_weight.min
1288 +7.5% 1384 ± 3% sched_debug.cfs_rq:/.util_avg.max
54763 ± 9% +130.8% 126390 ± 25% sched_debug.cpu.avg_idle.avg
301502 ± 18% +84.9% 557406 ± 14% sched_debug.cpu.avg_idle.max
82233 ± 22% +122.7% 183153 ± 17% sched_debug.cpu.avg_idle.stddev
0.00 ± 12% -12.0% 0.00 ± 8% sched_debug.cpu.next_balance.stddev
16.92 ± 4% -7.5% 15.66 ± 3% sched_debug.cpu.nr_running.avg
2.92 ± 10% -20.9% 2.31 ± 16% sched_debug.cpu.nr_uninterruptible.avg
2977 ± 50% -73.0% 803.75 ± 43% interrupts.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1
546.00 ± 23% +119.5% 1198 ± 16% interrupts.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3
13831842 -11.5% 12245495 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts
45259 ± 5% +68.9% 76440 ± 15% interrupts.CPU0.TRM:Thermal_event_interrupts
2977 ± 50% -73.0% 803.75 ± 43% interrupts.CPU1.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1
45283 ± 5% +68.9% 76474 ± 15% interrupts.CPU1.TRM:Thermal_event_interrupts
13947887 -11.0% 12413074 interrupts.CPU10.RES:Rescheduling_interrupts
45275 ± 5% +68.9% 76468 ± 15% interrupts.CPU10.TRM:Thermal_event_interrupts
45286 ± 5% +68.9% 76468 ± 15% interrupts.CPU11.TRM:Thermal_event_interrupts
13579741 -11.8% 11974820 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU12.TRM:Thermal_event_interrupts
13668073 -11.3% 12119346 interrupts.CPU13.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76474 ± 15% interrupts.CPU13.TRM:Thermal_event_interrupts
13578574 -7.8% 12518854 interrupts.CPU14.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76467 ± 15% interrupts.CPU14.TRM:Thermal_event_interrupts
13689117 -10.1% 12310290 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76473 ± 15% interrupts.CPU15.TRM:Thermal_event_interrupts
13924470 -10.5% 12461613 interrupts.CPU2.RES:Rescheduling_interrupts
45264 ± 5% +68.9% 76467 ± 15% interrupts.CPU2.TRM:Thermal_event_interrupts
546.00 ± 23% +119.5% 1198 ± 16% interrupts.CPU3.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3
13951884 -10.5% 12480739 interrupts.CPU3.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76469 ± 15% interrupts.CPU3.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76471 ± 15% interrupts.CPU4.TRM:Thermal_event_interrupts
14030690 -10.7% 12530992 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts
45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU5.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76474 ± 15% interrupts.CPU6.TRM:Thermal_event_interrupts
45285 ± 5% +68.9% 76472 ± 15% interrupts.CPU7.TRM:Thermal_event_interrupts
13319036 -10.2% 11966842 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts
45280 ± 5% +68.7% 76405 ± 15% interrupts.CPU8.TRM:Thermal_event_interrupts
13401328 -11.8% 11816711 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts
45279 ± 5% +68.9% 76467 ± 15% interrupts.CPU9.TRM:Thermal_event_interrupts
2.19e+08 -9.8% 1.974e+08 ± 2% interrupts.RES:Rescheduling_interrupts
724498 ± 5% +68.9% 1223435 ± 15% interrupts.TRM:Thermal_event_interrupts
53.44 +4.2% 55.68 perf-stat.i.MPKI
0.12 -0.0 0.10 ± 4% perf-stat.i.cache-miss-rate%
1764490 -13.1% 1533362 ± 3% perf-stat.i.cache-misses
1.767e+09 +3.6% 1.832e+09 perf-stat.i.cache-references
4055785 +6.2% 4306694 perf-stat.i.context-switches
336154 +21.5% 408363 perf-stat.i.cpu-migrations
34528 ± 5% +10.3% 38089 ± 3% perf-stat.i.cycles-between-cache-misses
0.01 ± 6% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate%
899577 ± 6% -15.2% 762553 ± 5% perf-stat.i.dTLB-load-misses
0.00 ± 7% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate%
22816 ± 7% -13.2% 19813 ± 4% perf-stat.i.dTLB-store-misses
6.044e+09 -1.2% 5.969e+09 perf-stat.i.dTLB-stores
31331199 -3.4% 30272777 perf-stat.i.iTLB-load-misses
1061 +3.1% 1094 perf-stat.i.instructions-per-iTLB-miss
1077 -2.9% 1046 perf-stat.i.minor-faults
73018 -10.9% 65028 ± 2% perf-stat.i.node-loads
122632 ± 2% -9.3% 111181 ± 4% perf-stat.i.node-stores
1077 -2.9% 1046 perf-stat.i.page-faults
53.29 +4.2% 55.53 perf-stat.overall.MPKI
0.10 -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate%
25264 +15.3% 29120 ± 3% perf-stat.overall.cycles-between-cache-misses
0.01 ± 6% -0.0 0.01 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.00 ± 7% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate%
1058 +2.9% 1089 perf-stat.overall.instructions-per-iTLB-miss
65158 +6.6% 69487 perf-stat.overall.path-length
1761814 -13.1% 1530883 ± 3% perf-stat.ps.cache-misses
1.765e+09 +3.7% 1.829e+09 perf-stat.ps.cache-references
4048770 +6.2% 4299578 perf-stat.ps.context-switches
335565 +21.5% 407697 perf-stat.ps.cpu-migrations
898100 ± 6% -15.2% 761293 ± 5% perf-stat.ps.dTLB-load-misses
22777 ± 7% -13.2% 19781 ± 4% perf-stat.ps.dTLB-store-misses
6.034e+09 -1.2% 5.96e+09 perf-stat.ps.dTLB-stores
31277565 -3.4% 30222833 perf-stat.ps.iTLB-load-misses
1075 -2.9% 1044 perf-stat.ps.minor-faults
72896 -10.9% 64922 ± 2% perf-stat.ps.node-loads
122424 ± 2% -9.3% 111005 ± 4% perf-stat.ps.node-stores
1075 -2.9% 1044 perf-stat.ps.page-faults
39.37 ±100% -36.4 2.98 ±173% perf-profile.calltrace.cycles-pp.start_thread
21.68 ±100% -20.0 1.64 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread
20.14 ±100% -18.6 1.52 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
20.00 ±100% -18.5 1.51 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
18.05 ±100% -16.7 1.39 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread
16.75 ±100% -15.5 1.22 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread
16.70 ±100% -15.4 1.26 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write
14.98 ±100% -13.9 1.09 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
14.82 ±100% -13.7 1.08 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
13.95 ±100% -12.9 1.03 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread
12.97 ±100% -12.0 0.95 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read
3.69 ± 6% -1.1 2.58 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common
4.25 ± 6% -1.0 3.29 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
1.54 ± 10% -0.5 1.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
1.07 ± 16% -0.5 0.60 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write
1.14 ± 2% -0.3 0.80 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait
0.55 ± 4% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read
1.90 ± 9% -0.2 1.67 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.71 ± 7% -0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64
0.68 ± 19% +0.2 0.86 perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.67 ± 16% +0.2 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait
0.29 ±100% +0.4 0.64 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read
0.31 ±100% +0.4 0.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read
0.32 ±100% +0.4 0.68 ± 2% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait
0.29 ±100% +0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
0.35 ±100% +0.4 0.78 perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule
1.23 ± 19% +0.4 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait
1.29 ± 18% +0.5 1.76 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule
0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule
8.82 +0.6 9.46 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
0.47 ±100% +0.7 1.14 ± 6% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop
2.21 ± 10% +0.7 2.88 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64
9.10 +0.7 9.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
9.04 +0.7 9.74 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
0.71 ±100% +0.9 1.61 ± 3% perf-profile.calltrace.cycles-pp.__switch_to
0.68 ± 99% +0.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr
0.86 ±100% +0.9 1.79 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm
6.54 +1.0 7.53 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read
26.96 +1.0 27.95 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
1.42 ± 63% +1.0 2.47 ± 3% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.54 ± 8% +1.2 5.79 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
14.99 +1.3 16.27 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read
15.35 +1.3 16.66 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read
27.37 +1.3 28.70 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
17.81 +1.6 19.38 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read
3.95 ± 11% +1.8 5.70 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common
23.38 +2.0 25.33 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
24.30 +2.0 26.27 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
23.73 +2.0 25.71 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
5.24 ± 10% +2.0 7.25 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
3.23 ± 57% +2.7 5.98 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.34 ± 57% +2.8 6.18 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.37 ±100% -36.4 2.98 ±173% perf-profile.children.cycles-pp.start_thread
21.68 ±100% -20.0 1.67 ±173% perf-profile.children.cycles-pp.__GI___libc_write
16.99 ±100% -15.7 1.24 ±173% perf-profile.children.cycles-pp.__GI___libc_read
6.84 ± 10% -2.1 4.71 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
45.35 -1.0 44.33 perf-profile.children.cycles-pp.ksys_write
6.70 ± 5% -1.0 5.70 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
5.90 ± 4% -0.6 5.34 ± 6% perf-profile.children.cycles-pp.security_file_permission
3.35 ± 9% -0.6 2.80 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave
42.14 -0.5 41.63 perf-profile.children.cycles-pp.vfs_write
3.41 -0.4 3.01 perf-profile.children.cycles-pp.__fdget_pos
3.30 -0.4 2.91 perf-profile.children.cycles-pp.__fget_light
2.71 ± 2% -0.4 2.32 perf-profile.children.cycles-pp.__fget
3.23 -0.3 2.93 ± 3% perf-profile.children.cycles-pp.mutex_lock
2.76 -0.3 2.47 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
2.21 ± 3% -0.3 1.93 ± 10% perf-profile.children.cycles-pp.file_has_perm
3.14 -0.2 2.90 ± 4% perf-profile.children.cycles-pp.copy_page_to_iter
3.03 ± 5% -0.2 2.79 ± 5% perf-profile.children.cycles-pp.selinux_file_permission
2.10 ± 5% -0.2 1.86 ± 2% perf-profile.children.cycles-pp.mutex_unlock
1.50 ± 5% -0.2 1.27 ± 2% perf-profile.children.cycles-pp.fput_many
0.68 ± 17% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.__mutex_lock
1.66 ± 2% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.copyout
1.73 -0.2 1.57 ± 2% perf-profile.children.cycles-pp.update_rq_clock
1.30 -0.1 1.17 ± 6% perf-profile.children.cycles-pp.copyin
0.27 ± 11% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common
1.25 ± 3% -0.1 1.15 ± 3% perf-profile.children.cycles-pp.fsnotify
0.79 -0.1 0.69 ± 7% perf-profile.children.cycles-pp.reschedule_interrupt
0.91 -0.1 0.82 ± 5% perf-profile.children.cycles-pp._cond_resched
0.80 ± 2% -0.1 0.71 ± 3% perf-profile.children.cycles-pp.file_update_time
0.29 ± 10% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.rb_insert_color
0.26 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret
0.07 ± 14% -0.0 0.03 ±100% perf-profile.children.cycles-pp.scheduler_ipi
0.22 ± 6% -0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_read
0.25 ± 4% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
0.27 -0.0 0.23 ± 9% perf-profile.children.cycles-pp.timespec64_trunc
0.23 ± 2% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.smp_reschedule_interrupt
0.22 -0.0 0.20 ± 5% perf-profile.children.cycles-pp.__x64_sys_write
0.24 -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__list_add_valid
0.14 -0.0 0.12 ± 10% perf-profile.children.cycles-pp.kill_fasync
0.11 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_exclude_event
0.10 +0.0 0.12 ± 8% perf-profile.children.cycles-pp.is_cpu_allowed
0.09 ± 11% +0.0 0.12 ± 7% perf-profile.children.cycles-pp.rcu_note_context_switch
0.07 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.pkg_thermal_notify
0.17 ± 3% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.deactivate_task
0.16 +0.0 0.21 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load
0.08 ± 6% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.intel_thermal_interrupt
0.08 ± 6% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.smp_thermal_interrupt
0.08 +0.1 0.13 ± 9% perf-profile.children.cycles-pp.thermal_interrupt
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.default_wake_function
0.19 ± 5% +0.1 0.25 ± 5% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.19 ± 5% +0.1 0.27 ± 4% perf-profile.children.cycles-pp.finish_wait
0.89 ± 8% +0.1 0.98 perf-profile.children.cycles-pp.__calc_delta
0.63 ± 4% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.native_sched_clock
0.33 ± 4% +0.1 0.44 ± 5% perf-profile.children.cycles-pp.set_next_buddy
1.23 ± 3% +0.1 1.34 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup
0.69 ± 4% +0.1 0.81 ± 3% perf-profile.children.cycles-pp.sched_clock
1.52 ± 3% +0.1 1.65 ± 2% perf-profile.children.cycles-pp.set_next_entity
0.73 ± 4% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu
0.57 ± 4% +0.2 0.74 ± 4% perf-profile.children.cycles-pp.account_entity_dequeue
1.21 ± 4% +0.2 1.39 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se
1.99 ± 3% +0.2 2.20 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.64 ± 6% +0.2 1.85 perf-profile.children.cycles-pp.__switch_to_asm
1.60 ± 4% +0.2 1.85 ± 3% perf-profile.children.cycles-pp.update_cfs_group
3.91 +0.2 4.15 perf-profile.children.cycles-pp.update_curr
1.04 ± 5% +0.3 1.30 ± 4% perf-profile.children.cycles-pp.___perf_sw_event
1.05 ± 9% +0.3 1.33 ± 2% perf-profile.children.cycles-pp.pick_next_entity
1.57 ± 3% +0.3 1.89 perf-profile.children.cycles-pp.native_write_msr
1.01 ± 9% +0.3 1.33 ± 5% perf-profile.children.cycles-pp.put_prev_entity
2.17 ± 7% +0.3 2.51 ± 2% perf-profile.children.cycles-pp.switch_fpu_return
1.93 ± 7% +0.4 2.36 perf-profile.children.cycles-pp.__switch_to
2.99 ± 4% +0.6 3.57 ± 2% perf-profile.children.cycles-pp.reweight_entity
8.89 +0.6 9.50 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair
9.16 +0.7 9.82 perf-profile.children.cycles-pp.ttwu_do_activate
9.09 +0.7 9.76 ± 2% perf-profile.children.cycles-pp.activate_task
5.18 ± 6% +0.9 6.04 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair
6.71 +0.9 7.63 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair
28.01 +1.4 29.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock
4.91 ± 7% +1.4 6.28 ± 4% perf-profile.children.cycles-pp.exit_to_usermode_loop
18.19 +1.4 19.56 perf-profile.children.cycles-pp.pipe_wait
4.03 ± 11% +1.7 5.75 ± 6% perf-profile.children.cycles-pp.select_idle_sibling
23.86 ± 2% +1.8 25.71 ± 2% perf-profile.children.cycles-pp.try_to_wake_up
24.44 +1.9 26.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common
23.84 +1.9 25.77 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function
5.29 ± 10% +2.0 7.29 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair
20.27 ± 2% +2.3 22.53 ± 2% perf-profile.children.cycles-pp.__schedule
20.36 ± 2% +2.5 22.82 ± 2% perf-profile.children.cycles-pp.schedule
6.82 ± 10% -2.1 4.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
2.67 ± 2% -0.4 2.29 perf-profile.self.cycles-pp.__fget
2.72 -0.3 2.44 ± 5% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.22 -0.2 0.98 ± 2% perf-profile.self.cycles-pp.update_rq_clock
1.36 ± 3% -0.2 1.13 ± 6% perf-profile.self.cycles-pp.pipe_write
1.48 ± 5% -0.2 1.24 ± 3% perf-profile.self.cycles-pp.fput_many
2.04 ± 5% -0.2 1.81 ± 2% perf-profile.self.cycles-pp.mutex_unlock
1.96 -0.2 1.73 ± 2% perf-profile.self.cycles-pp.mutex_lock
0.66 ± 12% -0.1 0.53 ± 10% perf-profile.self.cycles-pp.file_has_perm
0.38 ± 10% -0.1 0.28 ± 9% perf-profile.self.cycles-pp.ksys_read
0.34 ± 7% -0.1 0.25 ± 7% perf-profile.self.cycles-pp.__mutex_lock
1.21 ± 3% -0.1 1.11 ± 2% perf-profile.self.cycles-pp.fsnotify
1.00 -0.1 0.94 ± 5% perf-profile.self.cycles-pp.__might_sleep
0.60 ± 4% -0.1 0.55 ± 8% perf-profile.self.cycles-pp.vfs_write
0.29 ± 10% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.rb_insert_color
0.32 ± 6% -0.0 0.28 ± 6% perf-profile.self.cycles-pp.ksys_write
0.26 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret
0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.scheduler_ipi
0.36 ± 5% -0.0 0.32 ± 8% perf-profile.self.cycles-pp.__sb_start_write
0.29 -0.0 0.24 ± 4% perf-profile.self.cycles-pp.check_preempt_curr
0.22 ± 6% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.__x64_sys_read
0.25 -0.0 0.21 ± 8% perf-profile.self.cycles-pp.timespec64_trunc
0.22 ± 4% -0.0 0.19 ± 11% perf-profile.self.cycles-pp.wake_q_add
0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__x64_sys_write
0.21 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__list_add_valid
0.11 ± 4% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.kill_fasync
0.06 ± 9% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.rcu_note_context_switch
0.15 +0.0 0.18 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function
0.16 +0.0 0.19 ± 9% perf-profile.self.cycles-pp.deactivate_task
0.26 +0.0 0.29 ± 5% perf-profile.self.cycles-pp.rcu_all_qs
0.33 ± 3% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.set_next_entity
0.18 ± 5% +0.0 0.22 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.03 ±100% +0.0 0.07 perf-profile.self.cycles-pp.check_cfs_rq_runtime
0.16 ± 6% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop
0.16 +0.0 0.20 ± 12% perf-profile.self.cycles-pp.update_cfs_rq_h_load
0.20 ± 2% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.activate_task
0.41 ± 6% +0.1 0.46 perf-profile.self.cycles-pp.schedule
1.96 +0.1 2.02 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.15 ± 6% +0.1 0.22 perf-profile.self.cycles-pp.finish_wait
0.41 ± 4% +0.1 0.48 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue
0.61 ± 4% +0.1 0.71 ± 3% perf-profile.self.cycles-pp.native_sched_clock
0.83 ± 2% +0.1 0.93 perf-profile.self.cycles-pp.dequeue_task_fair
0.29 ± 3% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.set_next_buddy
1.02 +0.1 1.15 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair
0.46 ± 12% +0.2 0.61 ± 3% perf-profile.self.cycles-pp.pick_next_entity
1.17 ± 4% +0.2 1.35 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se
0.42 ± 8% +0.2 0.61 ± 5% perf-profile.self.cycles-pp.account_entity_dequeue
1.64 ± 6% +0.2 1.85 perf-profile.self.cycles-pp.__switch_to_asm
1.23 ± 4% +0.2 1.46 ± 3% perf-profile.self.cycles-pp.reweight_entity
1.07 ± 5% +0.2 1.31 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair
0.90 ± 5% +0.2 1.14 ± 5% perf-profile.self.cycles-pp.___perf_sw_event
1.57 ± 4% +0.2 1.82 ± 2% perf-profile.self.cycles-pp.update_cfs_group
1.17 ± 3% +0.3 1.42 ± 7% perf-profile.self.cycles-pp.update_load_avg
1.56 ± 4% +0.3 1.88 perf-profile.self.cycles-pp.native_write_msr
2.16 ± 7% +0.3 2.50 ± 2% perf-profile.self.cycles-pp.switch_fpu_return
1.84 ± 8% +0.4 2.25 ± 2% perf-profile.self.cycles-pp.__switch_to
1.22 +0.5 1.73 ± 4% perf-profile.self.cycles-pp._raw_spin_lock
0.92 ± 11% +1.3 2.27 ± 6% perf-profile.self.cycles-pp.select_idle_sibling
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
2 months
[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail
by kernel test robot
FYI, we noticed the following commit (built with gcc-7):
commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request")
https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master
in testcase: kernel_selftests
with following parameters:
group: kselftests-02
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
2019-07-26 16:52:03 make run_tests -C seccomp
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf
In file included from seccomp_bpf.c:51:0:
seccomp_bpf.c: In function ‘tracer_ptrace’:
seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function)
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function)
: PTRACE_EVENTMSG_SYSCALL_EXIT, msg);
^
../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’
__typeof__(_expected) __exp = (_expected); \
^~~~~~~~~
seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’
EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY
^~~~~~~~~
Makefile:12: recipe for target 'seccomp_bpf' failed
make: *** [seccomp_bpf] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp'
To reproduce:
# build kernel
cd linux
cp config-5.2.0-10889-g201766a20e30f9 .config
make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
4 months, 2 weeks
[x86, sched] 1567c3e346: vm-scalability.median -15.8% regression
by kernel test robot
Greeting,
FYI, we noticed a -15.8% regression of vm-scalability.median due to commit:
commit: 1567c3e3467cddeb019a7b53ec632f834b6a9239 ("x86, sched: Add support for frequency invariance")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: vm-scalability
on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory
with following parameters:
runtime: 300s
size: 8T
test: anon-cow-seq
cpufreq_governor: performance
ucode: 0x16
test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us.
test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <oliver.sang(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-hsw-4ex1/anon-cow-seq/vm-scalability/0x16
commit:
2a4b03ffc6 ("sched/fair: Prevent unlimited runtime on throttled group")
1567c3e346 ("x86, sched: Add support for frequency invariance")
2a4b03ffc69f2ded 1567c3e3467cddeb019a7b53ec6
---------------- ---------------------------
%stddev %change %stddev
\ | \
210443 -15.8% 177280 vm-scalability.median
0.07 ± 6% -64.9% 0.02 ± 16% vm-scalability.median_stddev
30242018 -16.0% 25399016 vm-scalability.throughput
241304 ± 2% +53.9% 371337 ± 2% vm-scalability.time.involuntary_context_switches
8973780 -12.4% 7862380 vm-scalability.time.minor_page_faults
10507 +7.6% 11301 vm-scalability.time.percent_of_cpu_this_job_got
17759 +22.0% 21664 ± 2% vm-scalability.time.system_time
14801 -10.4% 13262 vm-scalability.time.user_time
1023965 -63.3% 376066 ± 4% vm-scalability.time.voluntary_context_switches
7.967e+09 -11.1% 7.082e+09 vm-scalability.workload
2985 -11.4% 2643 ± 3% slabinfo.khugepaged_mm_slot.active_objs
2985 -11.4% 2643 ± 3% slabinfo.khugepaged_mm_slot.num_objs
36024164 -42.9% 20554898 ± 40% cpuidle.C1.time
7.277e+09 ± 41% -72.1% 2.027e+09 ±160% cpuidle.C3.time
17453257 ± 19% -74.4% 4462508 ±163% cpuidle.C3.usage
214574 ± 10% -50.2% 106926 ± 33% cpuidle.POLL.usage
26.26 -5.4 20.89 mpstat.cpu.all.idle%
0.02 ± 8% -0.0 0.01 ± 9% mpstat.cpu.all.iowait%
40.23 +8.8 49.05 mpstat.cpu.all.sys%
33.49 -3.4 30.05 ± 2% mpstat.cpu.all.usr%
1989988 -20.6% 1580840 ± 4% numa-numastat.node1.local_node
2026261 -20.6% 1608134 ± 4% numa-numastat.node1.numa_hit
3614123 ± 44% -56.4% 1576459 ± 5% numa-numastat.node2.local_node
3641345 ± 44% -56.0% 1603122 ± 5% numa-numastat.node2.numa_hit
26.50 -20.8% 21.00 vmstat.cpu.id
32.75 -10.7% 29.25 vmstat.cpu.us
9800 -34.3% 6434 vmstat.system.cs
290059 -6.9% 270063 ± 4% vmstat.system.in
1.012e+08 +12.3% 1.136e+08 meminfo.Active
1.012e+08 +12.3% 1.136e+08 meminfo.Active(anon)
92123057 +14.3% 1.053e+08 meminfo.AnonHugePages
1.009e+08 +12.3% 1.134e+08 meminfo.AnonPages
1.044e+08 +11.9% 1.169e+08 meminfo.Memused
25495045 +15.0% 29324284 numa-meminfo.node1.Active
25494976 +15.0% 29324244 numa-meminfo.node1.Active(anon)
23164225 +17.2% 27157731 numa-meminfo.node1.AnonHugePages
25417044 +15.1% 29267058 numa-meminfo.node1.AnonPages
26192254 +14.4% 29959010 numa-meminfo.node1.MemUsed
24839952 ± 3% +18.2% 29349199 numa-meminfo.node2.Active
24839901 ± 3% +18.2% 29349040 numa-meminfo.node2.Active(anon)
22604826 ± 3% +20.4% 27211067 numa-meminfo.node2.AnonHugePages
24777054 ± 3% +18.2% 29282273 numa-meminfo.node2.AnonPages
25718372 ± 2% +16.8% 30037056 numa-meminfo.node2.MemUsed
199045 ± 96% -99.6% 743.75 ± 52% numa-meminfo.node2.PageTables
54935 ± 20% -24.5% 41470 ± 10% numa-meminfo.node2.SUnreclaim
418.75 ± 94% -93.7% 26.50 ± 49% numa-meminfo.node3.Inactive(file)
4864 ± 15% +26.9% 6174 ± 10% numa-meminfo.node3.KernelStack
95371 ±172% +233.9% 318444 ± 57% numa-meminfo.node3.PageTables
6347307 +14.8% 7286035 numa-vmstat.node1.nr_active_anon
6328825 +14.9% 7268986 numa-vmstat.node1.nr_anon_pages
11261 +16.9% 13162 numa-vmstat.node1.nr_anon_transparent_hugepages
6347284 +14.8% 7285923 numa-vmstat.node1.nr_zone_active_anon
1324428 -15.2% 1122566 ± 3% numa-vmstat.node1.numa_hit
1204942 -16.0% 1011998 ± 3% numa-vmstat.node1.numa_local
6207902 ± 3% +17.7% 7308204 numa-vmstat.node2.nr_active_anon
6195059 ± 3% +17.6% 7287608 numa-vmstat.node2.nr_anon_pages
11047 ± 3% +19.7% 13218 numa-vmstat.node2.nr_anon_transparent_hugepages
49684 ± 96% -99.6% 186.00 ± 52% numa-vmstat.node2.nr_page_table_pages
13734 ± 20% -24.5% 10367 ± 10% numa-vmstat.node2.nr_slab_unreclaimable
6207738 ± 3% +17.7% 7308081 numa-vmstat.node2.nr_zone_active_anon
2125860 ± 40% -49.5% 1073183 ± 2% numa-vmstat.node2.numa_hit
2015486 ± 42% -52.2% 963317 ± 2% numa-vmstat.node2.numa_local
104.25 ± 94% -93.8% 6.50 ± 49% numa-vmstat.node3.nr_inactive_file
4864 ± 15% +26.7% 6160 ± 10% numa-vmstat.node3.nr_kernel_stack
23833 ±172% +230.0% 78649 ± 57% numa-vmstat.node3.nr_page_table_pages
104.25 ± 94% -93.8% 6.50 ± 49% numa-vmstat.node3.nr_zone_inactive_file
1.12 ± 19% -0.5 0.63 ± 20% perf-profile.calltrace.cycles-pp.do_huge_pmd_numa_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault
0.81 ± 19% -0.2 0.60 ± 20% perf-profile.calltrace.cycles-pp.reuse_swap_page.do_huge_pmd_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault
0.33 ± 22% -0.2 0.17 ± 4% perf-profile.children.cycles-pp.migrate_misplaced_transhuge_page
0.09 ± 23% -0.1 0.03 ±100% perf-profile.children.cycles-pp.ttwu_do_activate
0.08 ± 19% -0.1 0.03 ±100% perf-profile.children.cycles-pp.enqueue_entity
0.15 ± 20% -0.1 0.10 ± 15% perf-profile.children.cycles-pp.wake_up_page_bit
0.14 ± 23% -0.1 0.09 ± 14% perf-profile.children.cycles-pp.__wake_up_common
0.14 ± 20% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.autoremove_wake_function
0.09 ± 20% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.enqueue_task_fair
0.09 ± 23% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.activate_task
0.15 ± 21% -0.0 0.11 ± 17% perf-profile.children.cycles-pp.try_to_wake_up
0.15 ± 11% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.schedule
0.08 ± 10% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.newidle_balance
0.12 ± 16% +0.0 0.17 ± 9% perf-profile.children.cycles-pp.find_busiest_group
0.00 +0.1 0.05 perf-profile.children.cycles-pp.balance_fair
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.arch_scale_freq_tick
0.18 ± 10% +0.1 0.23 ± 8% perf-profile.children.cycles-pp.load_balance
0.00 +0.1 0.09 ± 9% perf-profile.children.cycles-pp.smpboot_thread_fn
0.40 ± 23% +0.3 0.68 ± 17% perf-profile.children.cycles-pp.task_tick_fair
0.01 ±173% +0.3 0.30 ± 16% perf-profile.children.cycles-pp.update_cfs_group
0.01 ±173% +0.3 0.30 ± 16% perf-profile.self.cycles-pp.update_cfs_group
8587 ± 12% -29.1% 6090 ± 9% sched_debug.cfs_rq:/.exec_clock.stddev
0.53 ± 45% -92.1% 0.04 ±173% sched_debug.cfs_rq:/.load_avg.min
0.54 ± 19% +38.3% 0.74 ± 10% sched_debug.cfs_rq:/.nr_running.avg
4.81 ± 12% +266.0% 17.59 ± 7% sched_debug.cfs_rq:/.nr_spread_over.avg
24.01 ± 8% +96.4% 47.14 ± 4% sched_debug.cfs_rq:/.nr_spread_over.max
5.02 ± 10% +123.0% 11.19 ± 6% sched_debug.cfs_rq:/.nr_spread_over.stddev
313.47 ± 18% -55.1% 140.75 ± 51% sched_debug.cfs_rq:/.runnable_load_avg.max
31.75 ± 9% -54.5% 14.46 ± 44% sched_debug.cfs_rq:/.runnable_load_avg.stddev
0.54 ± 19% +38.2% 0.75 ± 9% sched_debug.cpu.nr_running.avg
11567 ± 8% -28.4% 8277 ± 7% sched_debug.cpu.nr_switches.avg
4760 ± 9% -31.7% 3249 ± 4% sched_debug.cpu.nr_switches.stddev
-36.17 +31.8% -47.65 sched_debug.cpu.nr_uninterruptible.min
15.88 ± 2% +35.3% 21.49 ± 7% sched_debug.cpu.nr_uninterruptible.stddev
9718 ± 10% -34.1% 6402 ± 9% sched_debug.cpu.sched_count.avg
4294 ± 8% -37.4% 2688 ± 8% sched_debug.cpu.sched_count.stddev
3936 ± 10% -53.6% 1828 ± 12% sched_debug.cpu.sched_goidle.avg
2045 ± 8% -50.0% 1022 ± 10% sched_debug.cpu.sched_goidle.stddev
4616 ± 9% -36.5% 2933 ± 10% sched_debug.cpu.ttwu_count.avg
11005 ± 9% -22.0% 8579 ± 9% sched_debug.cpu.ttwu_count.max
2271 ± 8% -34.9% 1478 ± 8% sched_debug.cpu.ttwu_count.stddev
915.40 ± 8% +42.3% 1303 ± 6% sched_debug.cpu.ttwu_local.avg
3927 ± 5% +19.3% 4684 ± 10% sched_debug.cpu.ttwu_local.max
488.40 ± 9% +40.6% 686.55 ± 3% sched_debug.cpu.ttwu_local.stddev
25252488 +12.3% 28363408 proc-vmstat.nr_active_anon
25182652 +12.4% 28296707 proc-vmstat.nr_anon_pages
44870 +14.2% 51263 proc-vmstat.nr_anon_transparent_hugepages
148.25 -5.7% 139.75 proc-vmstat.nr_dirtied
10541824 -3.0% 10230220 proc-vmstat.nr_dirty_background_threshold
21109424 -3.0% 20485453 proc-vmstat.nr_dirty_threshold
1.06e+08 -2.9% 1.029e+08 proc-vmstat.nr_free_pages
369.75 -1.4% 364.50 proc-vmstat.nr_inactive_file
99622 +7.3% 106911 proc-vmstat.nr_page_table_pages
144.25 -4.9% 137.25 proc-vmstat.nr_written
25252485 +12.3% 28363396 proc-vmstat.nr_zone_active_anon
369.75 -1.4% 364.50 proc-vmstat.nr_zone_inactive_file
2953466 -10.6% 2639571 proc-vmstat.numa_hint_faults
661711 -9.8% 596566 proc-vmstat.numa_hint_faults_local
11245421 -11.8% 9923535 proc-vmstat.numa_hit
5838019 -7.0% 5428548 proc-vmstat.numa_huge_pte_updates
11136518 -11.9% 9814538 proc-vmstat.numa_local
201352 ± 12% +234.6% 673715 ± 7% proc-vmstat.numa_pages_migrated
2.99e+09 -7.0% 2.78e+09 proc-vmstat.numa_pte_updates
2.382e+09 -18.9% 1.931e+09 proc-vmstat.pgalloc_normal
10046733 -11.2% 8918565 proc-vmstat.pgfault
2.381e+09 -18.9% 1.93e+09 proc-vmstat.pgfree
5.942e+08 -42.6% 3.409e+08 proc-vmstat.pgmigrate_fail
201352 ± 12% +234.6% 673715 ± 7% proc-vmstat.pgmigrate_success
3478289 -11.1% 3091553 proc-vmstat.thp_fault_alloc
8.85 ± 7% -16.8% 7.36 ± 12% perf-stat.i.MPKI
2.766e+10 ± 4% -7.4% 2.562e+10 ± 3% perf-stat.i.branch-instructions
20.31 +1.8 22.10 ± 4% perf-stat.i.cache-miss-rate%
5.089e+08 ± 4% -9.5% 4.604e+08 ± 3% perf-stat.i.cache-references
9955 ± 3% -33.0% 6665 ± 4% perf-stat.i.context-switches
1.73 ± 4% +9.9% 1.90 ± 3% perf-stat.i.cpi
1.287e+11 ± 4% +11.2% 1.431e+11 ± 3% perf-stat.i.cpu-cycles
485.10 ± 5% +108.0% 1009 ± 4% perf-stat.i.cpu-migrations
2.215e+10 ± 4% -6.5% 2.071e+10 ± 3% perf-stat.i.dTLB-loads
7.427e+09 ± 4% -8.1% 6.829e+09 ± 3% perf-stat.i.dTLB-stores
958368 ± 5% -30.9% 662160 ± 2% perf-stat.i.iTLB-load-misses
8.707e+10 ± 4% -7.4% 8.062e+10 ± 3% perf-stat.i.instructions
142754 ± 2% +11.3% 158927 ± 2% perf-stat.i.instructions-per-iTLB-miss
0.67 -13.3% 0.58 perf-stat.i.ipc
33135 ± 4% -10.7% 29584 ± 2% perf-stat.i.minor-faults
70.79 +6.0 76.82 perf-stat.i.node-load-miss-rate%
41028170 ± 4% +14.1% 46801837 ± 3% perf-stat.i.node-load-misses
1.85 ± 5% +0.9 2.78 ± 5% perf-stat.i.node-store-miss-rate%
584076 ± 6% +70.0% 992640 ± 4% perf-stat.i.node-store-misses
53445296 ± 5% -19.0% 43298204 ± 7% perf-stat.i.node-stores
33137 ± 4% -10.7% 29584 ± 2% perf-stat.i.page-faults
5.85 -1.7% 5.75 perf-stat.overall.MPKI
1.48 +19.6% 1.77 perf-stat.overall.cpi
1156 +16.7% 1350 ± 5% perf-stat.overall.cycles-between-cache-misses
92129 ± 6% +25.2% 115378 perf-stat.overall.instructions-per-iTLB-miss
0.67 -16.4% 0.56 perf-stat.overall.ipc
75.40 +3.8 79.17 perf-stat.overall.node-load-miss-rate%
1.11 ± 2% +1.2 2.33 ± 7% perf-stat.overall.node-store-miss-rate%
2.704e+10 -11.0% 2.408e+10 perf-stat.ps.branch-instructions
1.091e+08 -8.5% 99896941 ± 5% perf-stat.ps.cache-misses
4.979e+08 -12.5% 4.357e+08 perf-stat.ps.cache-references
9775 -34.7% 6386 ± 2% perf-stat.ps.context-switches
1.262e+11 +6.5% 1.344e+11 perf-stat.ps.cpu-cycles
487.90 ± 5% +100.7% 979.41 perf-stat.ps.cpu-migrations
2.166e+10 -10.1% 1.948e+10 perf-stat.ps.dTLB-loads
7.26e+09 -11.3% 6.441e+09 perf-stat.ps.dTLB-stores
927296 ± 5% -29.1% 657143 perf-stat.ps.iTLB-load-misses
8.512e+10 -10.9% 7.581e+10 perf-stat.ps.instructions
32070 -11.2% 28469 perf-stat.ps.minor-faults
40537100 +9.2% 44251909 ± 4% perf-stat.ps.node-load-misses
13226459 ± 2% -12.0% 11644845 ± 4% perf-stat.ps.node-loads
582949 ± 3% +65.6% 965166 ± 3% perf-stat.ps.node-store-misses
52116857 -21.9% 40679765 ± 8% perf-stat.ps.node-stores
32070 -11.2% 28469 perf-stat.ps.page-faults
2.642e+13 -11.4% 2.341e+13 perf-stat.total.instructions
229.75 ± 52% -34.1% 151.50 interrupts.102:PCI-MSI.1572921-edge.eth0-TxRx-57
156.75 ± 3% +27.3% 199.50 ± 29% interrupts.47:PCI-MSI.1572866-edge.eth0-TxRx-2
1603 ± 22% -15.6% 1352 ± 4% interrupts.CPU1.NMI:Non-maskable_interrupts
1603 ± 22% -15.6% 1352 ± 4% interrupts.CPU1.PMI:Performance_monitoring_interrupts
1007 ± 19% -38.7% 618.00 ± 8% interrupts.CPU100.RES:Rescheduling_interrupts
1016 ± 19% -33.9% 671.75 ± 5% interrupts.CPU101.RES:Rescheduling_interrupts
2046 ± 32% -19.9% 1638 ± 30% interrupts.CPU102.NMI:Non-maskable_interrupts
2046 ± 32% -19.9% 1638 ± 30% interrupts.CPU102.PMI:Performance_monitoring_interrupts
956.75 ± 18% -33.1% 640.25 ± 5% interrupts.CPU102.RES:Rescheduling_interrupts
909.50 ± 20% -29.1% 644.50 ± 12% interrupts.CPU103.RES:Rescheduling_interrupts
959.50 ± 24% -31.4% 658.00 ± 5% interrupts.CPU104.RES:Rescheduling_interrupts
921.75 ± 14% -30.2% 643.50 ± 3% interrupts.CPU105.RES:Rescheduling_interrupts
901.50 ± 14% -27.8% 650.50 ± 11% interrupts.CPU106.RES:Rescheduling_interrupts
2670 ± 3% -37.5% 1670 ± 31% interrupts.CPU107.NMI:Non-maskable_interrupts
2670 ± 3% -37.5% 1670 ± 31% interrupts.CPU107.PMI:Performance_monitoring_interrupts
2849 -31.6% 1948 ± 33% interrupts.CPU109.NMI:Non-maskable_interrupts
2849 -31.6% 1948 ± 33% interrupts.CPU109.PMI:Performance_monitoring_interrupts
1410 ± 20% +51.0% 2130 ± 23% interrupts.CPU11.NMI:Non-maskable_interrupts
1410 ± 20% +51.0% 2130 ± 23% interrupts.CPU11.PMI:Performance_monitoring_interrupts
2820 ± 3% -30.7% 1955 ± 32% interrupts.CPU111.NMI:Non-maskable_interrupts
2820 ± 3% -30.7% 1955 ± 32% interrupts.CPU111.PMI:Performance_monitoring_interrupts
2808 -30.1% 1962 ± 33% interrupts.CPU116.NMI:Non-maskable_interrupts
2808 -30.1% 1962 ± 33% interrupts.CPU116.PMI:Performance_monitoring_interrupts
622190 -8.1% 571803 ± 4% interrupts.CPU118.LOC:Local_timer_interrupts
621861 -8.3% 570381 ± 4% interrupts.CPU119.LOC:Local_timer_interrupts
2024 ± 25% -30.0% 1417 ± 8% interrupts.CPU12.NMI:Non-maskable_interrupts
2024 ± 25% -30.0% 1417 ± 8% interrupts.CPU12.PMI:Performance_monitoring_interrupts
622130 -8.0% 572612 ± 4% interrupts.CPU120.LOC:Local_timer_interrupts
622145 -8.4% 569980 ± 4% interrupts.CPU121.LOC:Local_timer_interrupts
2131 ± 33% -24.4% 1611 ± 26% interrupts.CPU121.NMI:Non-maskable_interrupts
2131 ± 33% -24.4% 1611 ± 26% interrupts.CPU121.PMI:Performance_monitoring_interrupts
621983 -7.9% 572602 ± 4% interrupts.CPU123.LOC:Local_timer_interrupts
2219 ± 34% -40.0% 1332 ± 5% interrupts.CPU124.NMI:Non-maskable_interrupts
2219 ± 34% -40.0% 1332 ± 5% interrupts.CPU124.PMI:Performance_monitoring_interrupts
622061 -8.0% 572198 ± 4% interrupts.CPU125.LOC:Local_timer_interrupts
817.50 ± 12% -46.8% 435.25 ± 13% interrupts.CPU128.RES:Rescheduling_interrupts
820.75 ± 24% -48.8% 420.00 ± 25% interrupts.CPU129.RES:Rescheduling_interrupts
750.00 ± 25% -42.9% 428.00 ± 24% interrupts.CPU130.RES:Rescheduling_interrupts
2285 ± 22% -38.1% 1415 interrupts.CPU133.NMI:Non-maskable_interrupts
2285 ± 22% -38.1% 1415 interrupts.CPU133.PMI:Performance_monitoring_interrupts
779.25 ± 30% -50.5% 386.00 ± 23% interrupts.CPU133.RES:Rescheduling_interrupts
741.25 ± 28% -47.7% 387.75 ± 28% interrupts.CPU135.RES:Rescheduling_interrupts
728.00 ± 28% -46.2% 391.50 ± 31% interrupts.CPU137.RES:Rescheduling_interrupts
749.25 ± 29% -53.9% 345.50 ± 27% interrupts.CPU138.RES:Rescheduling_interrupts
751.25 ± 27% -49.0% 383.00 ± 29% interrupts.CPU139.RES:Rescheduling_interrupts
688.00 ± 24% -49.8% 345.25 ± 27% interrupts.CPU140.RES:Rescheduling_interrupts
726.00 ± 27% -55.1% 326.25 ± 32% interrupts.CPU141.RES:Rescheduling_interrupts
2272 ± 29% -40.7% 1347 ± 2% interrupts.CPU15.NMI:Non-maskable_interrupts
2272 ± 29% -40.7% 1347 ± 2% interrupts.CPU15.PMI:Performance_monitoring_interrupts
156.75 ± 3% +27.3% 199.50 ± 29% interrupts.CPU2.47:PCI-MSI.1572866-edge.eth0-TxRx-2
2562 ± 9% -38.6% 1573 ± 28% interrupts.CPU2.NMI:Non-maskable_interrupts
2562 ± 9% -38.6% 1573 ± 28% interrupts.CPU2.PMI:Performance_monitoring_interrupts
1344 ± 20% -32.8% 903.75 interrupts.CPU20.RES:Rescheduling_interrupts
1314 ± 16% -29.9% 921.50 ± 5% interrupts.CPU21.RES:Rescheduling_interrupts
120.00 ± 47% -98.3% 2.00 ± 93% interrupts.CPU22.TLB:TLB_shootdowns
2245 ± 24% -35.7% 1443 ± 24% interrupts.CPU23.NMI:Non-maskable_interrupts
2245 ± 24% -35.7% 1443 ± 24% interrupts.CPU23.PMI:Performance_monitoring_interrupts
1338 ± 18% -32.8% 899.00 ± 12% interrupts.CPU23.RES:Rescheduling_interrupts
1243 ± 16% -28.0% 895.75 ± 4% interrupts.CPU24.RES:Rescheduling_interrupts
1360 ± 30% -33.3% 907.50 ± 15% interrupts.CPU25.RES:Rescheduling_interrupts
1273 ± 20% -34.0% 841.00 ± 6% interrupts.CPU26.RES:Rescheduling_interrupts
1273 ± 15% -31.3% 874.25 ± 8% interrupts.CPU27.RES:Rescheduling_interrupts
1315 ± 16% -40.6% 781.00 ± 11% interrupts.CPU28.RES:Rescheduling_interrupts
1320 ± 20% -39.2% 802.75 ± 5% interrupts.CPU29.RES:Rescheduling_interrupts
2094 ± 30% -22.7% 1619 ± 35% interrupts.CPU30.NMI:Non-maskable_interrupts
2094 ± 30% -22.7% 1619 ± 35% interrupts.CPU30.PMI:Performance_monitoring_interrupts
1301 ± 19% -40.6% 773.00 ± 5% interrupts.CPU30.RES:Rescheduling_interrupts
1253 ± 16% -36.9% 791.00 ± 15% interrupts.CPU31.RES:Rescheduling_interrupts
1231 ± 17% -41.5% 720.25 ± 8% interrupts.CPU32.RES:Rescheduling_interrupts
1177 ± 11% -37.3% 738.00 ± 14% interrupts.CPU33.RES:Rescheduling_interrupts
1195 ± 18% -34.0% 788.75 ± 21% interrupts.CPU34.RES:Rescheduling_interrupts
2567 ± 8% -35.1% 1666 ± 35% interrupts.CPU35.NMI:Non-maskable_interrupts
2567 ± 8% -35.1% 1666 ± 35% interrupts.CPU35.PMI:Performance_monitoring_interrupts
1223 ± 18% -39.1% 745.25 ± 8% interrupts.CPU35.RES:Rescheduling_interrupts
621753 -7.9% 572936 ± 4% interrupts.CPU36.LOC:Local_timer_interrupts
2527 ± 10% -34.0% 1667 ± 31% interrupts.CPU40.NMI:Non-maskable_interrupts
2527 ± 10% -34.0% 1667 ± 31% interrupts.CPU40.PMI:Performance_monitoring_interrupts
622323 -7.9% 572967 ± 4% interrupts.CPU45.LOC:Local_timer_interrupts
1832 ± 17% +38.7% 2541 ± 3% interrupts.CPU46.NMI:Non-maskable_interrupts
1832 ± 17% +38.7% 2541 ± 3% interrupts.CPU46.PMI:Performance_monitoring_interrupts
621906 -8.0% 572330 ± 4% interrupts.CPU47.LOC:Local_timer_interrupts
621972 -8.1% 571378 ± 4% interrupts.CPU48.LOC:Local_timer_interrupts
622275 -8.2% 571410 ± 4% interrupts.CPU49.LOC:Local_timer_interrupts
2459 ± 24% -41.7% 1433 ± 10% interrupts.CPU49.NMI:Non-maskable_interrupts
2459 ± 24% -41.7% 1433 ± 10% interrupts.CPU49.PMI:Performance_monitoring_interrupts
622031 -7.9% 573184 ± 4% interrupts.CPU50.LOC:Local_timer_interrupts
622081 -7.9% 572943 ± 4% interrupts.CPU51.LOC:Local_timer_interrupts
622140 -7.8% 573769 ± 4% interrupts.CPU52.LOC:Local_timer_interrupts
622074 -7.6% 574558 ± 4% interrupts.CPU53.LOC:Local_timer_interrupts
229.75 ± 52% -34.1% 151.50 interrupts.CPU57.102:PCI-MSI.1572921-edge.eth0-TxRx-57
1052 ± 38% -59.2% 429.25 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts
830.75 ± 26% -53.3% 388.25 ± 27% interrupts.CPU62.RES:Rescheduling_interrupts
774.50 ± 28% -48.5% 399.00 ± 32% interrupts.CPU67.RES:Rescheduling_interrupts
785.00 ± 33% -51.6% 380.00 ± 32% interrupts.CPU68.RES:Rescheduling_interrupts
890.25 ± 31% -58.3% 371.50 ± 27% interrupts.CPU69.RES:Rescheduling_interrupts
821.50 ± 30% -46.4% 440.50 ± 36% interrupts.CPU70.RES:Rescheduling_interrupts
1897 ± 24% +40.4% 2663 ± 4% interrupts.CPU84.NMI:Non-maskable_interrupts
1897 ± 24% +40.4% 2663 ± 4% interrupts.CPU84.PMI:Performance_monitoring_interrupts
1431 ± 17% -43.5% 808.25 ± 21% interrupts.CPU9.RES:Rescheduling_interrupts
1064 ± 18% -30.1% 743.75 ± 16% interrupts.CPU91.RES:Rescheduling_interrupts
1076 ± 10% -39.6% 649.75 ± 10% interrupts.CPU92.RES:Rescheduling_interrupts
1042 ± 17% -33.8% 690.00 ± 10% interrupts.CPU93.RES:Rescheduling_interrupts
1065 ± 18% -33.4% 709.50 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts
2731 ± 7% -30.6% 1894 ± 28% interrupts.CPU95.NMI:Non-maskable_interrupts
2731 ± 7% -30.6% 1894 ± 28% interrupts.CPU95.PMI:Performance_monitoring_interrupts
998.75 ± 19% -28.4% 715.00 ± 7% interrupts.CPU96.RES:Rescheduling_interrupts
2.75 ± 47% +2009.1% 58.00 ±148% interrupts.CPU96.TLB:TLB_shootdowns
952.50 ± 27% -29.5% 671.25 ± 12% interrupts.CPU97.RES:Rescheduling_interrupts
2667 ± 2% -50.2% 1328 ± 4% interrupts.CPU98.NMI:Non-maskable_interrupts
2667 ± 2% -50.2% 1328 ± 4% interrupts.CPU98.PMI:Performance_monitoring_interrupts
973.00 ± 20% -28.5% 695.25 ± 4% interrupts.CPU98.RES:Rescheduling_interrupts
991.00 ± 18% -34.0% 654.00 ± 8% interrupts.CPU99.RES:Rescheduling_interrupts
135782 ± 14% -25.0% 101847 ± 6% interrupts.RES:Rescheduling_interrupts
140559 ± 2% -11.8% 124028 ± 6% softirqs.CPU0.RCU
145814 ± 3% -13.7% 125791 ± 5% softirqs.CPU1.RCU
138760 -12.0% 122124 ± 6% softirqs.CPU100.RCU
16104 -17.4% 13295 ± 5% softirqs.CPU100.SCHED
16422 ± 5% -16.9% 13653 ± 7% softirqs.CPU101.SCHED
15960 -14.5% 13650 ± 6% softirqs.CPU102.SCHED
15804 ± 8% -13.8% 13616 ± 7% softirqs.CPU103.SCHED
139949 -12.0% 123102 ± 5% softirqs.CPU104.RCU
16155 ± 3% -14.6% 13799 ± 8% softirqs.CPU104.SCHED
16018 ± 4% -14.3% 13725 ± 10% softirqs.CPU105.SCHED
15392 ± 5% -14.4% 13172 ± 6% softirqs.CPU106.SCHED
15218 ± 4% -15.5% 12852 ± 9% softirqs.CPU107.SCHED
143209 -14.2% 122899 ± 5% softirqs.CPU108.RCU
144618 -14.9% 123106 ± 4% softirqs.CPU109.RCU
144915 -14.1% 124470 ± 6% softirqs.CPU110.RCU
145038 -14.7% 123770 ± 5% softirqs.CPU111.RCU
138846 -12.2% 121867 ± 6% softirqs.CPU112.RCU
137438 -11.9% 121069 ± 6% softirqs.CPU113.RCU
141152 -13.4% 122234 ± 5% softirqs.CPU114.RCU
141545 -13.8% 122025 ± 5% softirqs.CPU115.RCU
140302 -13.0% 122108 ± 6% softirqs.CPU116.RCU
140548 -13.2% 122057 ± 5% softirqs.CPU117.RCU
142370 -14.3% 122021 ± 5% softirqs.CPU118.RCU
136694 -11.3% 121193 ± 5% softirqs.CPU119.RCU
137193 -11.5% 121415 ± 6% softirqs.CPU120.RCU
137125 -11.9% 120804 ± 5% softirqs.CPU121.RCU
140181 -13.0% 121893 ± 5% softirqs.CPU122.RCU
137405 -11.4% 121741 ± 5% softirqs.CPU124.RCU
139350 -12.5% 122000 ± 6% softirqs.CPU125.RCU
145339 -14.5% 124311 ± 5% softirqs.CPU126.RCU
14887 ± 18% -32.6% 10033 ± 15% softirqs.CPU126.SCHED
144917 -15.6% 122366 ± 5% softirqs.CPU127.RCU
15094 ± 20% -37.0% 9516 ± 21% softirqs.CPU127.SCHED
140946 -13.8% 121461 ± 6% softirqs.CPU128.RCU
138106 ± 2% -11.1% 122812 ± 6% softirqs.CPU129.RCU
137799 -11.3% 122221 ± 6% softirqs.CPU130.RCU
137425 -10.9% 122427 ± 6% softirqs.CPU131.RCU
140854 -12.9% 122753 ± 5% softirqs.CPU132.RCU
15140 ± 24% -35.8% 9716 ± 19% softirqs.CPU132.SCHED
140764 ± 2% -12.7% 122926 ± 5% softirqs.CPU133.RCU
139626 -11.7% 123235 ± 5% softirqs.CPU134.RCU
141210 ± 2% -12.6% 123477 ± 5% softirqs.CPU135.RCU
142996 ± 3% -14.1% 122901 ± 5% softirqs.CPU136.RCU
140567 ± 2% -12.5% 123022 ± 6% softirqs.CPU140.RCU
14192 ± 21% -34.2% 9336 ± 18% softirqs.CPU141.SCHED
135514 ± 2% -9.8% 122294 ± 5% softirqs.CPU142.RCU
134945 ± 2% -10.0% 121501 ± 6% softirqs.CPU143.RCU
145145 -13.0% 126309 ± 5% softirqs.CPU16.RCU
144489 ± 2% -13.3% 125217 ± 5% softirqs.CPU17.RCU
138706 ± 4% -12.0% 122092 ± 5% softirqs.CPU18.RCU
15348 ± 4% -10.0% 13810 ± 8% softirqs.CPU18.SCHED
140870 ± 2% -10.7% 125784 ± 3% softirqs.CPU19.RCU
16392 ± 5% -15.5% 13843 ± 6% softirqs.CPU19.SCHED
143741 ± 2% -12.4% 125852 ± 5% softirqs.CPU2.RCU
140088 -11.7% 123663 ± 5% softirqs.CPU20.RCU
16092 ± 3% -14.0% 13832 ± 5% softirqs.CPU20.SCHED
138206 -11.0% 123025 ± 5% softirqs.CPU21.RCU
16877 ± 4% -16.7% 14051 ± 4% softirqs.CPU21.SCHED
138653 -11.4% 122814 ± 5% softirqs.CPU22.RCU
17090 ± 3% -19.3% 13797 ± 5% softirqs.CPU22.SCHED
16476 ± 3% -15.6% 13910 ± 5% softirqs.CPU23.SCHED
138808 -11.2% 123204 ± 5% softirqs.CPU24.RCU
16779 -16.2% 14060 ± 5% softirqs.CPU24.SCHED
139811 ± 2% -11.6% 123555 ± 5% softirqs.CPU25.RCU
16734 ± 3% -15.8% 14083 ± 7% softirqs.CPU25.SCHED
138909 -10.9% 123758 ± 5% softirqs.CPU26.RCU
16619 ± 5% -15.0% 14133 ± 5% softirqs.CPU26.SCHED
140257 -12.3% 123025 ± 5% softirqs.CPU27.RCU
17013 ± 5% -17.3% 14061 ± 5% softirqs.CPU27.SCHED
137824 -11.2% 122428 ± 6% softirqs.CPU28.RCU
16470 ± 2% -15.4% 13927 ± 6% softirqs.CPU28.SCHED
135294 ± 2% -9.6% 122364 ± 6% softirqs.CPU29.RCU
17350 ± 3% -20.7% 13761 ± 5% softirqs.CPU29.SCHED
142300 ± 3% -13.1% 123621 ± 6% softirqs.CPU3.RCU
17067 -18.4% 13927 ± 5% softirqs.CPU30.SCHED
17110 ± 4% -17.6% 14099 ± 7% softirqs.CPU31.SCHED
145321 -14.0% 124914 ± 4% softirqs.CPU32.RCU
16952 ± 3% -16.0% 14234 ± 7% softirqs.CPU32.SCHED
143684 -14.0% 123636 ± 5% softirqs.CPU33.RCU
16728 ± 2% -17.3% 13838 ± 5% softirqs.CPU33.SCHED
143358 -13.4% 124129 ± 5% softirqs.CPU34.RCU
17103 ± 3% -17.7% 14082 ± 6% softirqs.CPU34.SCHED
143430 -13.0% 124848 ± 5% softirqs.CPU35.RCU
16705 -17.7% 13746 ± 4% softirqs.CPU35.SCHED
139206 -11.5% 123261 ± 7% softirqs.CPU36.RCU
140252 -12.7% 122378 ± 5% softirqs.CPU37.RCU
140086 -12.6% 122492 ± 5% softirqs.CPU39.RCU
140518 -13.4% 121743 ± 5% softirqs.CPU40.RCU
138605 -11.7% 122371 ± 5% softirqs.CPU41.RCU
141519 -13.5% 122418 ± 5% softirqs.CPU42.RCU
142030 -13.7% 122637 ± 5% softirqs.CPU43.RCU
140694 -13.4% 121845 ± 4% softirqs.CPU44.RCU
140504 -12.1% 123545 ± 5% softirqs.CPU45.RCU
140277 -12.7% 122435 ± 5% softirqs.CPU46.RCU
137267 -11.2% 121919 ± 5% softirqs.CPU47.RCU
140839 -12.9% 122646 ± 5% softirqs.CPU48.RCU
141816 -13.3% 122937 ± 5% softirqs.CPU49.RCU
140868 ± 2% -12.0% 123942 ± 6% softirqs.CPU5.RCU
143643 -14.1% 123392 ± 6% softirqs.CPU50.RCU
141326 -12.7% 123403 ± 6% softirqs.CPU51.RCU
141583 -13.4% 122613 ± 6% softirqs.CPU52.RCU
141100 -13.2% 122445 ± 5% softirqs.CPU53.RCU
139956 -12.1% 122981 ± 6% softirqs.CPU54.RCU
14018 ± 17% -29.8% 9845 ± 15% softirqs.CPU54.SCHED
140114 -11.3% 124217 ± 5% softirqs.CPU55.RCU
143329 -14.8% 122069 ± 6% softirqs.CPU56.RCU
141146 -12.3% 123718 ± 6% softirqs.CPU57.RCU
15264 ± 19% -34.6% 9988 ± 17% softirqs.CPU57.SCHED
140809 -12.4% 123330 ± 6% softirqs.CPU58.RCU
15201 ± 20% -36.3% 9678 ± 20% softirqs.CPU58.SCHED
140568 -12.0% 123769 ± 5% softirqs.CPU59.RCU
15360 ± 20% -33.3% 10252 ± 19% softirqs.CPU59.SCHED
139409 ± 3% -10.9% 124281 ± 6% softirqs.CPU6.RCU
141767 -12.7% 123768 ± 5% softirqs.CPU60.RCU
142046 -12.9% 123657 ± 5% softirqs.CPU61.RCU
15389 ± 22% -34.7% 10052 ± 18% softirqs.CPU61.SCHED
141719 -12.7% 123700 ± 5% softirqs.CPU62.RCU
142499 -12.9% 124068 ± 6% softirqs.CPU63.RCU
14667 ± 20% -31.8% 10005 ± 18% softirqs.CPU63.SCHED
144437 -14.1% 124008 ± 6% softirqs.CPU64.RCU
15191 ± 22% -35.9% 9736 ± 19% softirqs.CPU64.SCHED
141156 ± 2% -12.3% 123743 ± 6% softirqs.CPU65.RCU
139859 -11.2% 124140 ± 6% softirqs.CPU66.RCU
141282 -11.9% 124535 ± 6% softirqs.CPU67.RCU
144228 -13.7% 124434 ± 6% softirqs.CPU68.RCU
142536 -13.0% 123956 ± 6% softirqs.CPU69.RCU
141103 ± 2% -11.8% 124488 ± 6% softirqs.CPU7.RCU
140849 ± 2% -11.5% 124683 ± 6% softirqs.CPU70.RCU
141347 ± 2% -12.4% 123863 ± 6% softirqs.CPU71.RCU
142552 ± 3% -13.6% 123193 ± 5% softirqs.CPU72.RCU
142074 ± 4% -12.3% 124659 ± 7% softirqs.CPU73.RCU
141649 -13.1% 123155 ± 5% softirqs.CPU74.RCU
143525 ± 4% -14.0% 123493 ± 6% softirqs.CPU75.RCU
142772 ± 3% -13.9% 122974 ± 7% softirqs.CPU76.RCU
141178 ± 3% -12.9% 122944 ± 5% softirqs.CPU77.RCU
140777 ± 3% -12.8% 122799 ± 5% softirqs.CPU78.RCU
143039 ± 4% -14.3% 122572 ± 5% softirqs.CPU79.RCU
140336 ± 3% -11.8% 123811 ± 6% softirqs.CPU8.RCU
141727 ± 2% -13.0% 123316 ± 5% softirqs.CPU80.RCU
141813 ± 2% -13.0% 123363 ± 5% softirqs.CPU81.RCU
141978 ± 2% -11.6% 125442 ± 3% softirqs.CPU82.RCU
137611 ± 3% -10.7% 122950 ± 5% softirqs.CPU84.RCU
141077 ± 2% -12.0% 124188 ± 5% softirqs.CPU86.RCU
139221 ± 3% -11.9% 122606 ± 5% softirqs.CPU87.RCU
138177 ± 4% -11.7% 121980 ± 5% softirqs.CPU88.RCU
139882 ± 2% -11.1% 124329 ± 5% softirqs.CPU9.RCU
141843 ± 2% -13.6% 122541 ± 6% softirqs.CPU90.RCU
16251 ± 3% -15.5% 13733 ± 5% softirqs.CPU90.SCHED
143175 ± 2% -14.1% 122926 ± 6% softirqs.CPU91.RCU
17088 ± 3% -19.2% 13801 ± 7% softirqs.CPU91.SCHED
144768 -14.7% 123542 ± 5% softirqs.CPU92.RCU
16851 ± 2% -20.1% 13462 ± 4% softirqs.CPU92.SCHED
144398 ± 4% -13.4% 125018 ± 3% softirqs.CPU93.RCU
16917 ± 3% -19.9% 13559 ± 6% softirqs.CPU93.SCHED
142957 ± 2% -13.6% 123521 ± 5% softirqs.CPU94.RCU
16758 ± 2% -17.8% 13773 ± 4% softirqs.CPU94.SCHED
141885 -13.8% 122263 ± 6% softirqs.CPU95.RCU
16161 ± 5% -14.7% 13789 ± 6% softirqs.CPU95.SCHED
138647 -11.8% 122297 ± 5% softirqs.CPU96.RCU
16590 ± 3% -16.8% 13804 ± 8% softirqs.CPU96.SCHED
15776 ± 3% -13.0% 13731 ± 7% softirqs.CPU97.SCHED
16115 -10.6% 14405 ± 5% softirqs.CPU98.SCHED
140827 -12.8% 122846 ± 5% softirqs.CPU99.RCU
16252 ± 4% -16.2% 13623 ± 5% softirqs.CPU99.SCHED
20190421 -12.1% 17737370 ± 5% softirqs.RCU
2133170 -16.1% 1790167 ± 4% softirqs.SCHED
vm-scalability.time.system_time
22500 +-------------------------------------------------------------------+
22000 |-+ O O |
| O O O O |
21500 |-O O O O O O O |
21000 |-+ O O O O O O O O O O O |
20500 |-+ O O O |
20000 |-+ |
| |
19500 |-+ |
19000 |-+ |
18500 |-+ + |
18000 |.+.. + + : |
| .. + + : .+.. +.. + |
17500 |-+ +.+ + + + + |
17000 +-------------------------------------------------------------------+
vm-scalability.time.percent_of_cpu_this_job_got
11400 +-------------------------------------------------------------------+
11300 |-+ O O O O O O O O O |
| O O O O O O O O O O O O O O O O |
11200 |-+ O O |
11100 |-+ |
| |
11000 |-+ |
10900 |-+ |
10800 |-+ |
| |
10700 |-+ |
10600 |-+ |
|.+.. .+. .+. .+.. .+ |
10500 |-+ +.+. +. +.+..+ + |
10400 +-------------------------------------------------------------------+
vm-scalability.time.voluntary_context_switches
1.3e+06 +-----------------------------------------------------------------+
1.2e+06 |.+ |
| + .+. .+. .+ |
1.1e+06 |-+ + .+.+..+.+..+ +. + |
1e+06 |-+ + |
| |
900000 |-+ |
800000 |-+ |
700000 |-+ |
| |
600000 |-+ |
500000 |-+ |
| O O O O O O O O O O O O O |
400000 |-O O O O O O O O O O O O O O |
300000 +-----------------------------------------------------------------+
vm-scalability.throughput
3.2e+07 +-----------------------------------------------------------------+
| +.. |
3.1e+07 |++ |
| +.+. .+. .+.+.+..+.+.+ |
3e+07 |-+ +. +. |
| |
2.9e+07 |-+ |
| |
2.8e+07 |-+ |
| |
2.7e+07 |-+ |
| |
2.6e+07 |-+ O O O O O O O O O O O O O |
| O O O O O O O O O O O O |
2.5e+07 +-----------------------------------------------------------------+
vm-scalability.median
220000 +------------------------------------------------------------------+
215000 |-+.. |
|+ +.+.. .+. .+.+..+.+.+..+ |
210000 |-+ + +. |
205000 |-+ |
| |
200000 |-+ |
195000 |-+ |
190000 |-+ |
| |
185000 |-+ |
180000 |-+ O O O O O O O O O O O O O O O |
| O O O O O O O O O O O |
175000 |-+ O |
170000 +------------------------------------------------------------------+
vm-scalability.workload
8.4e+09 +-----------------------------------------------------------------+
| + |
8.2e+09 |:++ |
|: + |
8e+09 |-+ +.+.+..+.+..+.+.+..+.+.+ |
| |
7.8e+09 |-+ |
| |
7.6e+09 |-+ |
| |
7.4e+09 |-+ |
| |
7.2e+09 |-+ |
| O O O O O O O O O O O O O O O O O O O O O O O O O O O |
7e+09 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Oliver Sang
6 months
[sched/fair] 070f5e860e: reaim.jobs_per_min -10.5% regression
by kernel test robot
Greeting,
FYI, we noticed a -10.5% regression of reaim.jobs_per_min due to commit:
commit: 070f5e860ee2bf588c99ef7b4c202451faa48236 ("sched/fair: Take into account runnable_avg to classify group")
https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
in testcase: reaim
on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory
with following parameters:
runtime: 300s
nr_task: 100%
test: five_sec
cpufreq_governor: performance
ucode: 0x21
test-description: REAIM is an updated and improved version of AIM 7 benchmark.
test-url: https://sourceforge.net/projects/re-aim-7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/100%/debian-x86_64-20191114.cgz/300s/lkp-ivb-d04/five_sec/reaim/0x21
commit:
9f68395333 ("sched/pelt: Add a new runnable average signal")
070f5e860e ("sched/fair: Take into account runnable_avg to classify group")
9f68395333ad7f5b 070f5e860ee2bf588c99ef7b4c2
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
4:4 -18% 3:4 perf-profile.children.cycles-pp.error_entry
3:4 -12% 3:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
0.68 -10.4% 0.61 reaim.child_systime
67235 -10.5% 60195 reaim.jobs_per_min
16808 -10.5% 15048 reaim.jobs_per_min_child
97.90 -1.2% 96.70 reaim.jti
72000 -10.8% 64216 reaim.max_jobs_per_min
0.36 +11.3% 0.40 reaim.parent_time
1.56 ± 3% +79.1% 2.80 ± 6% reaim.std_dev_percent
0.00 ± 7% +145.9% 0.01 ± 9% reaim.std_dev_time
104276 -16.0% 87616 reaim.time.involuntary_context_switches
15511157 -2.4% 15144312 reaim.time.minor_page_faults
55.00 -7.3% 51.00 reaim.time.percent_of_cpu_this_job_got
88.01 -12.4% 77.12 reaim.time.system_time
79.97 -3.2% 77.38 reaim.time.user_time
216380 -3.4% 208924 reaim.time.voluntary_context_switches
50800 -2.4% 49600 reaim.workload
30.40 ± 2% -4.7% 28.97 ± 2% boot-time.boot
9.38 -0.7 8.66 ± 3% mpstat.cpu.all.sys%
7452 +7.5% 8014 vmstat.system.cs
1457802 ± 16% +49.3% 2176122 ± 13% cpuidle.C1.time
48523684 +43.4% 69570233 ± 22% cpuidle.C1E.time
806543 ± 2% +20.7% 973406 ± 11% cpuidle.C1E.usage
14328 ± 6% +14.5% 16410 ± 8% cpuidle.POLL.time
43300 ± 4% +13.5% 49150 ± 5% softirqs.CPU0.SCHED
118751 -9.3% 107763 softirqs.CPU1.RCU
41679 ± 3% +14.1% 47546 ± 4% softirqs.CPU1.SCHED
42688 ± 3% +12.3% 47931 ± 4% softirqs.CPU2.SCHED
41730 ± 2% +17.7% 49115 ± 4% softirqs.CPU3.SCHED
169399 +14.4% 193744 ± 2% softirqs.SCHED
3419 +1.0% 3453 proc-vmstat.nr_kernel_stack
16365616 -1.8% 16077850 proc-vmstat.numa_hit
16365616 -1.8% 16077850 proc-vmstat.numa_local
93908 -1.6% 92389 proc-vmstat.pgactivate
16269664 -3.9% 15629529 ± 2% proc-vmstat.pgalloc_normal
15918803 -2.3% 15557936 proc-vmstat.pgfault
16644610 -2.0% 16310898 proc-vmstat.pgfree
20125 ±123% +161.7% 52662 ± 30% sched_debug.cfs_rq:/.load.min
348749 ± 10% -11.2% 309562 ± 11% sched_debug.cfs_rq:/.load.stddev
1096 ± 6% -14.4% 938.42 ± 7% sched_debug.cfs_rq:/.load_avg.max
448.46 ± 8% -17.5% 370.19 ± 10% sched_debug.cfs_rq:/.load_avg.stddev
117372 -10.2% 105432 sched_debug.cfs_rq:/.min_vruntime.avg
135242 ± 4% -9.2% 122811 sched_debug.cfs_rq:/.min_vruntime.max
0.53 ± 8% +17.6% 0.62 ± 6% sched_debug.cfs_rq:/.nr_running.avg
29.79 ± 30% -51.0% 14.58 ± 35% sched_debug.cfs_rq:/.nr_spread_over.max
10.21 ± 34% -59.7% 4.12 ± 52% sched_debug.cfs_rq:/.nr_spread_over.stddev
78.25 ± 40% +3304.7% 2664 ± 94% sched_debug.cpu.curr->pid.min
294309 ± 2% +34.3% 395172 ± 12% sched_debug.cpu.nr_switches.min
9.58 ± 35% +84.8% 17.71 ± 40% sched_debug.cpu.nr_uninterruptible.max
-6.88 +120.6% -15.17 sched_debug.cpu.nr_uninterruptible.min
6.41 ± 30% +95.2% 12.52 ± 33% sched_debug.cpu.nr_uninterruptible.stddev
286185 +33.4% 381734 ± 13% sched_debug.cpu.sched_count.min
180416 +11.0% 200247 sched_debug.cpu.sched_goidle.avg
116264 ± 3% +44.6% 168090 ± 15% sched_debug.cpu.sched_goidle.min
476.00 ± 8% +92.4% 915.75 ± 3% interrupts.CAL:Function_call_interrupts
110.50 ± 24% +101.1% 222.25 ± 4% interrupts.CPU0.CAL:Function_call_interrupts
1381 ± 29% +23.7% 1709 ± 26% interrupts.CPU0.NMI:Non-maskable_interrupts
1381 ± 29% +23.7% 1709 ± 26% interrupts.CPU0.PMI:Performance_monitoring_interrupts
3319 ± 9% +50.4% 4991 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts
41.25 ± 30% +274.5% 154.50 ± 15% interrupts.CPU0.TLB:TLB_shootdowns
116.25 ± 23% +96.1% 228.00 ± 16% interrupts.CPU1.CAL:Function_call_interrupts
1183 ± 10% +43.1% 1692 ± 23% interrupts.CPU1.NMI:Non-maskable_interrupts
1183 ± 10% +43.1% 1692 ± 23% interrupts.CPU1.PMI:Performance_monitoring_interrupts
3335 ± 7% +60.4% 5350 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts
36.25 ± 30% +344.1% 161.00 ± 8% interrupts.CPU1.TLB:TLB_shootdowns
131.25 ± 11% +81.1% 237.75 ± 11% interrupts.CPU2.CAL:Function_call_interrupts
3247 ± 2% +62.4% 5274 interrupts.CPU2.RES:Rescheduling_interrupts
34.50 ± 36% +357.2% 157.75 ± 7% interrupts.CPU2.TLB:TLB_shootdowns
118.00 ± 13% +93.0% 227.75 ± 9% interrupts.CPU3.CAL:Function_call_interrupts
3155 ± 4% +68.7% 5322 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts
38.50 ± 16% +303.9% 155.50 ± 3% interrupts.CPU3.TLB:TLB_shootdowns
13057 ± 2% +60.4% 20939 interrupts.RES:Rescheduling_interrupts
150.50 ± 27% +317.8% 628.75 ± 3% interrupts.TLB:TLB_shootdowns
2.00 +0.1 2.09 ± 3% perf-stat.i.branch-miss-rate%
10.26 +1.1 11.36 ± 7% perf-stat.i.cache-miss-rate%
2009706 ± 2% +5.4% 2117525 ± 3% perf-stat.i.cache-misses
16867421 -4.5% 16106908 perf-stat.i.cache-references
7514 +7.6% 8083 perf-stat.i.context-switches
1.51 -3.0% 1.47 perf-stat.i.cpi
2.523e+09 ± 3% -8.8% 2.301e+09 ± 2% perf-stat.i.cpu-cycles
124.54 +157.8% 321.08 perf-stat.i.cpu-migrations
1842 ± 10% -18.6% 1498 ± 6% perf-stat.i.cycles-between-cache-misses
752585 ± 2% -4.1% 721714 perf-stat.i.dTLB-store-misses
590441 +2.7% 606399 perf-stat.i.iTLB-load-misses
68766 +4.0% 71488 ± 2% perf-stat.i.iTLB-loads
1.847e+09 ± 3% -4.7% 1.76e+09 ± 2% perf-stat.i.instructions
3490 ± 4% -8.5% 3195 ± 3% perf-stat.i.instructions-per-iTLB-miss
0.68 +3.7% 0.70 perf-stat.i.ipc
51861 -2.1% 50797 perf-stat.i.minor-faults
51861 -2.1% 50797 perf-stat.i.page-faults
2.68 ± 2% +0.1 2.78 perf-stat.overall.branch-miss-rate%
11.91 +1.2 13.14 ± 2% perf-stat.overall.cache-miss-rate%
1.37 -4.3% 1.31 perf-stat.overall.cpi
1255 -13.4% 1087 ± 2% perf-stat.overall.cycles-between-cache-misses
3127 ± 3% -7.2% 2901 ± 2% perf-stat.overall.instructions-per-iTLB-miss
0.73 +4.5% 0.76 perf-stat.overall.ipc
2002763 ± 2% +5.4% 2110303 ± 3% perf-stat.ps.cache-misses
16809816 -4.5% 16051656 perf-stat.ps.cache-references
7489 +7.6% 8055 perf-stat.ps.context-switches
2.514e+09 ± 3% -8.8% 2.293e+09 ± 2% perf-stat.ps.cpu-cycles
124.12 +157.8% 319.95 perf-stat.ps.cpu-migrations
750010 ± 2% -4.1% 719223 perf-stat.ps.dTLB-store-misses
588424 +2.7% 604314 perf-stat.ps.iTLB-load-misses
68533 +4.0% 71246 ± 2% perf-stat.ps.iTLB-loads
1.841e+09 ± 3% -4.7% 1.754e+09 ± 2% perf-stat.ps.instructions
51683 -2.1% 50622 perf-stat.ps.minor-faults
51683 -2.1% 50622 perf-stat.ps.page-faults
5.577e+11 ± 3% -5.1% 5.292e+11 ± 2% perf-stat.total.instructions
7.35 ± 17% -2.7 4.60 ± 10% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit
7.74 ± 20% -2.7 5.00 ± 6% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock
10.14 ± 8% -2.7 7.44 ± 6% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.66 ± 8% -2.6 8.07 ± 8% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
7.10 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.write._fini
7.10 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp._fini
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write._fini
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.devkmsg_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write
7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write.ksys_write
6.20 ± 8% -2.1 4.08 ± 5% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write
5.15 ± 11% -1.8 3.38 ± 4% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write
5.05 ± 11% -1.7 3.31 ± 3% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit
7.41 ± 10% -1.1 6.29 ± 5% perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
7.57 ± 11% -1.1 6.46 ± 5% perf-profile.calltrace.cycles-pp.execve
7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve
7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve
7.03 ± 5% -1.1 5.95 ± 10% perf-profile.calltrace.cycles-pp.brk
5.90 ± 7% -0.9 4.98 ± 10% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.brk
5.84 ± 7% -0.9 4.93 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
15.77 ± 2% -0.9 14.88 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
15.86 ± 2% -0.9 14.97 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
3.88 ± 6% -0.9 2.99 ± 5% perf-profile.calltrace.cycles-pp.kill
1.70 ± 23% -0.8 0.90 ± 10% perf-profile.calltrace.cycles-pp.delay_tsc.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write
4.88 ± 8% -0.8 4.08 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk
2.39 ± 27% -0.7 1.67 ± 5% perf-profile.calltrace.cycles-pp.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve
2.29 ± 30% -0.7 1.59 ± 5% perf-profile.calltrace.cycles-pp.mmput.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file
2.27 ± 30% -0.7 1.58 ± 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler
3.11 ± 5% -0.6 2.47 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.kill
3.07 ± 5% -0.6 2.45 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
2.09 ± 18% -0.4 1.67 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read
2.82 ± 9% -0.4 2.40 ± 12% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64
2.80 ± 9% -0.4 2.38 ± 12% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group
1.11 ± 33% -0.4 0.71 ± 10% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.flush_old_exec.load_elf_binary
1.05 ± 15% -0.4 0.69 ± 13% perf-profile.calltrace.cycles-pp.vt_console_print.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write
1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.lf.vt_console_print.console_unlock.vprintk_emit.devkmsg_emit
1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.con_scroll.lf.vt_console_print.console_unlock.vprintk_emit
1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.fbcon_scroll.con_scroll.lf.vt_console_print.console_unlock
0.96 ± 16% -0.3 0.66 ± 12% perf-profile.calltrace.cycles-pp.fbcon_putcs.fbcon_redraw.fbcon_scroll.con_scroll.lf
1.85 ± 4% -0.3 1.58 ± 8% perf-profile.calltrace.cycles-pp.alloc_pages_vma.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault
0.89 ± 15% -0.3 0.62 ± 12% perf-profile.calltrace.cycles-pp.kill_pid_info.kill_something_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.67 ± 5% -0.3 1.41 ± 9% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault.handle_mm_fault
1.02 ± 7% -0.3 0.77 ± 12% perf-profile.calltrace.cycles-pp.do_signal.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
0.94 ± 16% -0.2 0.70 ± 5% perf-profile.calltrace.cycles-pp.clear_page_erms.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma
0.98 ± 16% -0.2 0.74 ± 7% perf-profile.calltrace.cycles-pp.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault
1.03 ± 6% -0.2 0.79 ± 10% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill
1.00 ± 10% -0.2 0.77 ± 9% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
0.87 ± 10% -0.2 0.66 ± 15% perf-profile.calltrace.cycles-pp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64
1.41 ± 3% -0.2 1.23 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault
1.88 ± 5% -0.1 1.73 perf-profile.calltrace.cycles-pp.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
1.87 ± 5% -0.1 1.73 perf-profile.calltrace.cycles-pp._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe
10.34 ± 11% +7.3 17.66 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
10.18 ± 11% +7.3 17.52 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
11.32 ± 9% +7.7 19.03 ± 8% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
11.32 ± 9% +7.7 19.05 ± 8% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
11.32 ± 9% +7.7 19.05 ± 8% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
11.02 ± 5% +8.1 19.14 ± 7% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
16.04 ± 6% +9.1 25.17 ± 8% perf-profile.calltrace.cycles-pp.secondary_startup_64
55.98 -7.0 48.94 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
55.67 -7.0 48.67 ± 4% perf-profile.children.cycles-pp.do_syscall_64
10.60 ± 16% -3.3 7.30 ± 8% perf-profile.children.cycles-pp.vprintk_emit
13.02 ± 7% -3.0 9.99 ± 7% perf-profile.children.cycles-pp.write
9.92 ± 13% -2.8 7.08 ± 4% perf-profile.children.cycles-pp.console_unlock
10.26 ± 8% -2.7 7.53 ± 6% perf-profile.children.cycles-pp.new_sync_write
10.79 ± 8% -2.6 8.18 ± 8% perf-profile.children.cycles-pp.vfs_write
10.95 ± 8% -2.6 8.36 ± 8% perf-profile.children.cycles-pp.ksys_write
7.17 ± 16% -2.5 4.69 ± 7% perf-profile.children.cycles-pp.devkmsg_write
7.17 ± 16% -2.5 4.69 ± 7% perf-profile.children.cycles-pp.devkmsg_emit
8.65 ± 16% -2.4 6.21 ± 4% perf-profile.children.cycles-pp.serial8250_console_write
8.53 ± 17% -2.4 6.11 ± 4% perf-profile.children.cycles-pp.uart_console_write
7.13 ± 17% -2.4 4.71 ± 6% perf-profile.children.cycles-pp._fini
8.46 ± 16% -2.4 6.07 ± 4% perf-profile.children.cycles-pp.wait_for_xmitr
8.34 ± 16% -2.4 5.97 ± 4% perf-profile.children.cycles-pp.serial8250_console_putchar
5.80 ± 16% -1.6 4.21 ± 6% perf-profile.children.cycles-pp.io_serial_in
7.85 ± 10% -1.2 6.67 ± 5% perf-profile.children.cycles-pp.execve
7.72 ± 11% -1.2 6.55 ± 5% perf-profile.children.cycles-pp.__do_execve_file
5.19 ± 13% -1.1 4.05 ± 8% perf-profile.children.cycles-pp.mmput
5.16 ± 13% -1.1 4.03 ± 8% perf-profile.children.cycles-pp.exit_mmap
7.76 ± 10% -1.1 6.64 ± 5% perf-profile.children.cycles-pp.__x64_sys_execve
7.11 ± 5% -1.1 6.01 ± 10% perf-profile.children.cycles-pp.brk
3.92 ± 6% -0.9 3.03 ± 5% perf-profile.children.cycles-pp.kill
2.63 ± 17% -0.8 1.85 perf-profile.children.cycles-pp.delay_tsc
4.89 ± 8% -0.8 4.12 ± 8% perf-profile.children.cycles-pp.__x64_sys_brk
2.48 ± 27% -0.7 1.74 ± 4% perf-profile.children.cycles-pp.flush_old_exec
3.02 ± 12% -0.7 2.28 ± 12% perf-profile.children.cycles-pp.unmap_page_range
3.15 ± 11% -0.7 2.40 ± 12% perf-profile.children.cycles-pp.unmap_vmas
2.25 ± 19% -0.5 1.75 ± 11% perf-profile.children.cycles-pp.unmap_region
1.27 ± 11% -0.4 0.86 ± 8% perf-profile.children.cycles-pp.vt_console_print
1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.lf
1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.con_scroll
1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.fbcon_scroll
1.79 ± 9% -0.4 1.41 ± 4% perf-profile.children.cycles-pp.release_pages
1.22 ± 11% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.fbcon_redraw
1.17 ± 12% -0.4 0.82 ± 10% perf-profile.children.cycles-pp.fbcon_putcs
1.16 ± 13% -0.3 0.82 ± 10% perf-profile.children.cycles-pp.bit_putcs
0.90 ± 16% -0.3 0.62 ± 12% perf-profile.children.cycles-pp.kill_pid_info
0.95 ± 10% -0.3 0.68 ± 6% perf-profile.children.cycles-pp.drm_fb_helper_cfb_imageblit
0.95 ± 11% -0.3 0.68 ± 6% perf-profile.children.cycles-pp.cfb_imageblit
1.24 ± 7% -0.2 1.01 ± 6% perf-profile.children.cycles-pp.new_sync_read
0.71 ± 4% -0.2 0.49 ± 23% perf-profile.children.cycles-pp.___perf_sw_event
0.55 ± 31% -0.2 0.33 ± 16% perf-profile.children.cycles-pp.unlink_anon_vmas
0.89 ± 11% -0.2 0.67 ± 15% perf-profile.children.cycles-pp.shmem_file_read_iter
0.60 ± 20% -0.2 0.39 ± 20% perf-profile.children.cycles-pp.__send_signal
1.06 ± 6% -0.2 0.85 ± 16% perf-profile.children.cycles-pp.pagevec_lru_move_fn
0.88 -0.2 0.68 ± 6% perf-profile.children.cycles-pp.__perf_sw_event
1.49 ± 5% -0.2 1.29 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.56 ± 12% -0.2 0.37 ± 11% perf-profile.children.cycles-pp.do_send_sig_info
1.65 ± 8% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.perf_event_mmap
0.69 ± 2% -0.2 0.52 ± 16% perf-profile.children.cycles-pp.page_remove_rmap
0.61 ± 5% -0.2 0.44 ± 15% perf-profile.children.cycles-pp.free_unref_page_list
0.60 ± 6% -0.2 0.43 ± 15% perf-profile.children.cycles-pp.__vm_munmap
0.77 ± 12% -0.2 0.62 ± 12% perf-profile.children.cycles-pp.__might_sleep
0.39 ± 12% -0.2 0.24 ± 18% perf-profile.children.cycles-pp.time
0.46 ± 14% -0.1 0.34 ± 14% perf-profile.children.cycles-pp.lru_add_drain_cpu
0.57 ± 8% -0.1 0.47 ± 14% perf-profile.children.cycles-pp.shmem_undo_range
0.41 ± 12% -0.1 0.30 ± 15% perf-profile.children.cycles-pp.copy_fpstate_to_sigframe
0.76 ± 7% -0.1 0.67 ± 8% perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn
0.26 ± 16% -0.1 0.17 ± 17% perf-profile.children.cycles-pp.mark_page_accessed
0.12 ± 47% -0.1 0.04 ±103% perf-profile.children.cycles-pp.sigaction
0.23 ± 12% -0.1 0.15 ± 11% perf-profile.children.cycles-pp.__vm_enough_memory
0.12 ± 18% -0.1 0.05 ±106% perf-profile.children.cycles-pp.__vsprintf_chk
0.23 ± 20% -0.1 0.17 ± 13% perf-profile.children.cycles-pp.d_add
0.13 ± 23% -0.1 0.07 ± 58% perf-profile.children.cycles-pp.fput_many
0.13 ± 14% -0.1 0.08 ± 24% perf-profile.children.cycles-pp.vfs_unlink
0.11 ± 20% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
0.04 ± 63% +0.0 0.08 ± 23% perf-profile.children.cycles-pp.uncharge_page
0.06 ± 22% +0.0 0.10 ± 36% perf-profile.children.cycles-pp.sched_exec
0.44 ± 4% +0.0 0.48 ± 4% perf-profile.children.cycles-pp.close
0.14 ± 22% +0.1 0.21 ± 17% perf-profile.children.cycles-pp.pick_next_task_fair
0.10 ± 17% +0.1 0.17 ± 23% perf-profile.children.cycles-pp.__anon_vma_prepare
0.00 +0.1 0.07 ± 24% perf-profile.children.cycles-pp.update_sd_lb_stats
0.07 ± 34% +0.1 0.15 ± 42% perf-profile.children.cycles-pp.file_free_rcu
0.15 ± 27% +0.1 0.23 ± 21% perf-profile.children.cycles-pp.__strcasecmp
0.20 ± 21% +0.1 0.29 ± 8% perf-profile.children.cycles-pp.__pte_alloc
0.14 ± 47% +0.1 0.23 ± 27% perf-profile.children.cycles-pp.update_blocked_averages
0.09 ± 44% +0.1 0.19 ± 18% perf-profile.children.cycles-pp.schedule_idle
0.00 +0.1 0.10 ± 33% perf-profile.children.cycles-pp.newidle_balance
0.00 +0.1 0.10 ± 18% perf-profile.children.cycles-pp.__vmalloc_node_range
0.21 ± 15% +0.1 0.32 ± 25% perf-profile.children.cycles-pp.__wake_up_common
0.63 ± 8% +0.1 0.77 ± 6% perf-profile.children.cycles-pp.rcu_do_batch
0.76 ± 14% +0.1 0.90 ± 9% perf-profile.children.cycles-pp.rcu_core
0.07 ± 90% +0.2 0.27 ±109% perf-profile.children.cycles-pp.security_mmap_addr
0.46 ± 26% +0.3 0.75 ± 13% perf-profile.children.cycles-pp.__sched_text_start
11.32 ± 9% +7.7 19.05 ± 8% perf-profile.children.cycles-pp.start_secondary
11.03 ± 5% +8.1 19.16 ± 7% perf-profile.children.cycles-pp.intel_idle
14.78 ± 6% +8.5 23.24 ± 8% perf-profile.children.cycles-pp.cpuidle_enter
14.76 ± 6% +8.5 23.24 ± 8% perf-profile.children.cycles-pp.cpuidle_enter_state
16.04 ± 6% +9.1 25.17 ± 8% perf-profile.children.cycles-pp.secondary_startup_64
16.04 ± 6% +9.1 25.17 ± 8% perf-profile.children.cycles-pp.cpu_startup_entry
16.04 ± 6% +9.1 25.19 ± 8% perf-profile.children.cycles-pp.do_idle
5.79 ± 16% -1.6 4.21 ± 6% perf-profile.self.cycles-pp.io_serial_in
2.62 ± 17% -0.8 1.85 perf-profile.self.cycles-pp.delay_tsc
5.11 ± 4% -0.6 4.56 ± 5% perf-profile.self.cycles-pp.do_syscall_64
1.44 ± 6% -0.3 1.15 ± 5% perf-profile.self.cycles-pp.unmap_page_range
0.94 ± 11% -0.3 0.68 ± 6% perf-profile.self.cycles-pp.cfb_imageblit
0.65 ± 6% -0.2 0.42 ± 23% perf-profile.self.cycles-pp.___perf_sw_event
1.42 ± 5% -0.2 1.22 ± 7% perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.65 ± 13% -0.2 0.47 ± 9% perf-profile.self.cycles-pp.do_page_fault
0.65 ± 9% -0.1 0.52 ± 5% perf-profile.self.cycles-pp.release_pages
0.24 ± 20% -0.1 0.15 ± 16% perf-profile.self.cycles-pp.mark_page_accessed
0.16 ± 28% -0.1 0.08 ± 69% perf-profile.self.cycles-pp.free_unref_page_commit
0.12 ± 24% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.__do_munmap
0.10 ± 24% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
0.04 ± 57% +0.0 0.07 ± 19% perf-profile.self.cycles-pp.__sbrk
0.04 ± 57% +0.0 0.08 ± 23% perf-profile.self.cycles-pp.update_load_avg
0.04 ± 57% +0.0 0.08 ± 23% perf-profile.self.cycles-pp.uncharge_page
0.26 ± 11% +0.1 0.39 ± 12% perf-profile.self.cycles-pp.copy_page
0.49 ± 13% +0.1 0.63 ± 13% perf-profile.self.cycles-pp.get_page_from_freelist
11.00 ± 5% +8.1 19.14 ± 7% perf-profile.self.cycles-pp.intel_idle
reaim.time.system_time
90 +----------------------------------------------------------------------+
| +. .+ ++.++ .+ |
88 |+.+ +.++. + +++ ++.++.+ +.++.+++.+++.+++. +.++ .+ + :.+++.+|
| + + + + ++.++ + |
86 |-+ |
| |
84 |-+ |
| |
82 |-+ O |
|O OO OO O |
80 |-+ |
| OO OOO O O O O |
78 |-+ O O O O O OO O |
| O OOO O |
76 +----------------------------------------------------------------------+
reaim.time.percent_of_cpu_this_job_got
55 +--------------------------------------------------------------------+
| |
54.5 |-+ |
54 |-+ |
| |
53.5 |-+ |
| |
53 |-+OOO OOO |
| |
52.5 |-+ |
52 |O+ OOO OOO OOO OOO OO |
| |
51.5 |-+ |
| |
51 +--------------------------------------------------------------------+
reaim.parent_time
0.405 +-------------------------------------------------------------------+
0.4 |-+ O |
| O O OOO OO |
0.395 |-+ |
0.39 |-+ |
0.385 |-+ OOOO OOO OOO OOO OOOO |
0.38 |O+OOO |
| |
0.375 |-+ |
0.37 |-+ |
0.365 |-+ |
0.36 |-+ |
| +. .+ +. .+ +. + +. +++. + .+|
0.355 |+. +.+ ++.++ +++. ++.++++ + +++ + +++ + ++.++ + +. : + + |
0.35 +-------------------------------------------------------------------+
reaim.child_systime
0.69 +--------------------------------------------------------------------+
| +.+ ++. ++.+ .++ ++ +++.+ .++ .+++. + .+ : + .+|
0.68 |+.++ ++.+ + ++ +.+ ++ + + +.+++ +.+ + |
0.67 |-+ |
| |
0.66 |-+ |
0.65 |-+ |
| O |
0.64 |O+O OOO |
0.63 |-+ O |
| O O O O O |
0.62 |-+ OO O O O O O |
0.61 |-+ O OO O OO OOOO |
| |
0.6 +--------------------------------------------------------------------+
reaim.jobs_per_min
69000 +-------------------------------------------------------------------+
68000 |-.++ + + .++ .+ .++ + .+++.+ |
|+ +.+ ++.+++.+ + + +++.+++.+++.+++.++++ +.+ + + +.+ +. |
67000 |-+ + + +|
66000 |-+ |
| |
65000 |-+ |
64000 |-+ |
63000 |-+ |
|O OOO O O O O |
62000 |-+ O O OOO O OOO OOOO |
61000 |-+ |
| O OOO OO |
60000 |-+ OO |
59000 +-------------------------------------------------------------------+
reaim.jobs_per_min_child
17500 +-------------------------------------------------------------------+
| |
17000 |-.+++. + + .+++.+ .++ + .+++.+ |
|+ + ++.+++.+ + +++.+++.+++.+++.++++ +.+ + + +.+++.+|
| + |
16500 |-+ |
| |
16000 |-+ |
| |
15500 |O+OOO OOOO OOO OOO OOO OOOO |
| |
| O O OO |
15000 |-+ OOO O |
| |
14500 +-------------------------------------------------------------------+
reaim.max_jobs_per_min
76000 +-------------------------------------------------------------------+
| |
74000 |-+ + + + |
| :: : :: |
| : : : :: : |
72000 |+.++ ++++.+++.+++.+++.++++.+++.+++.+++.++++.+++.+++.+ + +++.+++.+|
| |
70000 |-+ |
| |
68000 |-+ O |
| |
| |
66000 |O+O O OOOO OOO OOO OOO OOOO |
| |
64000 +-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
7 months, 1 week
[drm/i915/gen9] 53b9bd37af: phoronix-test-suite.clpeak.KernelLatency.us 22.5% regression
by kernel test robot
Greeting,
FYI, we noticed a 22.5% regression of phoronix-test-suite.clpeak.KernelLatency.us due to commit:
commit: 53b9bd37af59d1def99b20707536105857eb9bd0 ("drm/i915/gen9: Clear residual context state on context switch")
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git linux-5.4.y
in testcase: phoronix-test-suite
on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory
with following parameters:
test: clpeak-1.0.1
cpufreq_governor: performance
ucode: 0xca
test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added.
test-url: http://www.phoronix-test-suite.com/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-lck-7931/clear-x86_64-phoronix-30140/lkp-kbl-nuc1/clpeak-1.0.1/phoronix-test-suite/0xca
commit:
f58642c1bc ("netfilter: ipset: avoid null deref when IPSET_ATTR_LINENO is present")
53b9bd37af ("drm/i915/gen9: Clear residual context state on context switch")
f58642c1bc7946c3 53b9bd37af59d1def99b2070753
---------------- ---------------------------
%stddev %change %stddev
\ | \
24.49 ± 2% +22.5% 30.00 phoronix-test-suite.clpeak.KernelLatency.us
101.56 ± 38% -70.7% 29.74 phoronix-test-suite.time.elapsed_time
101.56 ± 38% -70.7% 29.74 phoronix-test-suite.time.elapsed_time.max
405511 ± 30% -54.3% 185150 phoronix-test-suite.time.minor_page_faults
53.33 ± 2% +4.4% 55.67 phoronix-test-suite.time.percent_of_cpu_this_job_got
47.70 ± 37% -70.2% 14.20 phoronix-test-suite.time.user_time
454369 ± 41% -75.5% 111123 phoronix-test-suite.time.voluntary_context_switches
11.39 ± 2% -6.7% 10.62 ± 6% boot-time.dhcp
591003 ± 3% -11.4% 523381 ± 2% meminfo.Active(file)
53200 ± 50% +159.9% 138245 meminfo.max_used_kB
61.00 ± 19% -38.3% 37.67 ± 2% vmstat.cpu.id
28.00 ± 32% +63.1% 45.67 vmstat.cpu.us
3583 ± 45% +106.2% 7390 ± 5% vmstat.io.bi
1.33 ± 35% +125.0% 3.00 vmstat.procs.r
60.90 ± 20% -24.2 36.70 ± 3% mpstat.cpu.all.idle%
0.89 ± 19% +0.6 1.46 ± 14% mpstat.cpu.all.iowait%
0.20 ± 23% +0.1 0.28 ± 2% mpstat.cpu.all.soft%
8.03 ± 35% +4.7 12.69 ± 2% mpstat.cpu.all.sys%
28.78 ± 32% +18.9 47.66 mpstat.cpu.all.usr%
13077984 ± 54% -54.8% 5908886 ± 43% cpuidle.C1.time
597419 ± 46% -74.3% 153496 ± 15% cpuidle.C1.usage
24140686 ± 20% -72.9% 6543020 ± 20% cpuidle.C1E.time
356666 ± 25% -71.4% 102084 ± 12% cpuidle.C1E.usage
1.55e+08 ± 53% -86.5% 20926680 ± 24% cpuidle.C6.time
262494 ± 49% -76.9% 60562 ± 25% cpuidle.C6.usage
3614603 ± 39% -73.6% 952459 ± 2% cpuidle.POLL.time
409623 ± 42% -74.9% 102932 cpuidle.POLL.usage
79.96 ± 16% +23.4% 98.63 turbostat.Any%C0
1499 ± 30% +57.6% 2363 turbostat.Avg_MHz
38.92 ± 31% +23.0 61.94 turbostat.Busy%
597382 ± 46% -74.3% 153485 ± 15% turbostat.C1
356611 ± 25% -71.4% 102035 ± 12% turbostat.C1E
262458 ± 49% -76.9% 60550 ± 25% turbostat.C6
34.42 ± 27% -17.6 16.77 ± 24% turbostat.C6%
18.75 ± 52% -84.2% 2.96 ± 52% turbostat.CPU%c6
14.41 ± 21% +37.9% 19.87 turbostat.CorWatt
1777243 ± 32% -66.4% 597719 turbostat.IRQ
21.36 ± 14% +27.9% 27.32 turbostat.PkgWatt
123.28 ± 25% +44.0% 177.47 turbostat.Totl%C0
38364 ± 4% -23.5% 29333 ± 7% softirqs.CPU0.RCU
16353 ± 31% -60.7% 6429 ± 6% softirqs.CPU0.SCHED
55438 ± 25% -55.7% 24533 ± 4% softirqs.CPU0.TIMER
38652 ± 7% -25.8% 28672 ± 3% softirqs.CPU1.RCU
17071 ± 29% -58.5% 7091 ± 6% softirqs.CPU1.SCHED
63131 ± 33% -58.6% 26110 ± 7% softirqs.CPU1.TIMER
233404 ± 40% -74.3% 60064 softirqs.CPU2.HI
36812 ± 5% -24.4% 27818 ± 3% softirqs.CPU2.RCU
17331 ± 30% -65.2% 6025 ± 7% softirqs.CPU2.SCHED
52817 ± 26% -51.1% 25847 softirqs.CPU2.TIMER
37680 ± 4% -27.2% 27418 ± 5% softirqs.CPU3.RCU
17232 ± 33% -57.0% 7413 ± 2% softirqs.CPU3.SCHED
54880 ± 31% -48.7% 28163 ± 7% softirqs.CPU3.TIMER
233405 ± 40% -74.3% 60065 softirqs.HI
151510 ± 5% -25.3% 113242 ± 2% softirqs.RCU
67989 ± 31% -60.3% 26959 softirqs.SCHED
226267 ± 28% -53.7% 104655 softirqs.TIMER
147753 ± 3% -11.4% 130854 ± 2% proc-vmstat.nr_active_file
386270 +4.2% 402302 proc-vmstat.nr_dirty_background_threshold
3804020 -2.4% 3713749 proc-vmstat.nr_dirty_threshold
803734 -3.1% 778927 proc-vmstat.nr_file_pages
452742 -1.6% 445477 proc-vmstat.nr_inactive_file
10099 +2.1% 10308 proc-vmstat.nr_kernel_stack
2628 +1.8% 2675 proc-vmstat.nr_page_table_pages
82010 -5.4% 77547 proc-vmstat.nr_slab_reclaimable
24816 -3.2% 24014 proc-vmstat.nr_slab_unreclaimable
780.67 -2.3% 763.00 proc-vmstat.nr_unevictable
147753 ± 3% -11.4% 130854 ± 2% proc-vmstat.nr_zone_active_file
452742 -1.6% 445477 proc-vmstat.nr_zone_inactive_file
780.67 -2.3% 763.00 proc-vmstat.nr_zone_unevictable
5659020 ± 6% -46.0% 3058187 proc-vmstat.numa_hit
5659020 ± 6% -46.0% 3058187 proc-vmstat.numa_local
47921 ± 3% -16.2% 40136 ± 2% proc-vmstat.pgactivate
5867768 ± 7% -44.8% 3238245 proc-vmstat.pgalloc_normal
8000575 ± 5% -44.9% 4409621 proc-vmstat.pgfault
5730760 ± 7% -45.9% 3102373 proc-vmstat.pgfree
285530 ± 4% -23.9% 217222 ± 4% proc-vmstat.pgpgin
49.48 ± 70% -49.5 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64
40.07 ± 70% -40.1 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
36.60 ± 79% -36.6 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
36.59 ± 79% -36.6 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
36.51 ± 79% -36.5 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
35.12 ± 79% -35.1 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
34.92 ± 79% -34.9 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
6.65 ± 72% -6.6 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
6.47 ± 72% -6.5 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.00 ± 72% -4.0 0.00 perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
49.48 ± 70% -49.5 0.00 perf-profile.children.cycles-pp.secondary_startup_64
49.48 ± 70% -49.5 0.00 perf-profile.children.cycles-pp.cpu_startup_entry
49.37 ± 70% -49.4 0.00 perf-profile.children.cycles-pp.do_idle
47.50 ± 70% -47.5 0.00 perf-profile.children.cycles-pp.cpuidle_enter
47.38 ± 70% -47.4 0.00 perf-profile.children.cycles-pp.cpuidle_enter_state
40.10 ± 70% -40.1 0.00 perf-profile.children.cycles-pp.intel_idle
36.60 ± 79% -36.6 0.00 perf-profile.children.cycles-pp.start_secondary
7.29 ± 72% -7.3 0.00 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
7.12 ± 71% -7.1 0.00 perf-profile.children.cycles-pp.do_syscall_64
4.03 ± 72% -4.0 0.00 perf-profile.children.cycles-pp.poll_idle
40.10 ± 70% -40.1 0.00 perf-profile.self.cycles-pp.intel_idle
3.76 ± 71% -3.8 0.00 perf-profile.self.cycles-pp.poll_idle
695942 ± 40% -74.6% 176670 interrupts.125:IR-PCI-MSI.32768-edge.i915
34054 ± 4% -17.3% 28167 ± 9% interrupts.127:IR-PCI-MSI.376832-edge.ahci[0000:00:17.0]
1050 ± 28% -49.8% 527.33 ± 39% interrupts.128:IR-PCI-MSI.520192-edge.eth0
222197 ± 4% -41.3% 130456 interrupts.CAL:Function_call_interrupts
55148 -39.6% 33319 ± 2% interrupts.CPU0.CAL:Function_call_interrupts
194140 ± 36% -68.0% 62087 interrupts.CPU0.LOC:Local_timer_interrupts
1863 ± 88% -97.8% 41.33 ± 89% interrupts.CPU0.NMI:Non-maskable_interrupts
1863 ± 88% -97.8% 41.33 ± 89% interrupts.CPU0.PMI:Performance_monitoring_interrupts
6030 ± 17% -32.2% 4089 ± 7% interrupts.CPU0.RES:Rescheduling_interrupts
5253 ± 37% -73.9% 1369 ± 23% interrupts.CPU0.TLB:TLB_shootdowns
34054 ± 4% -17.3% 28167 ± 9% interrupts.CPU1.127:IR-PCI-MSI.376832-edge.ahci[0000:00:17.0]
56244 ± 8% -42.8% 32144 ± 2% interrupts.CPU1.CAL:Function_call_interrupts
194206 ± 36% -68.0% 62051 interrupts.CPU1.LOC:Local_timer_interrupts
1228 ± 67% -96.7% 41.00 ± 82% interrupts.CPU1.NMI:Non-maskable_interrupts
1228 ± 67% -96.7% 41.00 ± 82% interrupts.CPU1.PMI:Performance_monitoring_interrupts
4247 ± 13% -41.7% 2477 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts
2396 ± 58% -76.5% 562.33 ± 30% interrupts.CPU1.TLB:TLB_shootdowns
695942 ± 40% -74.6% 176670 interrupts.CPU2.125:IR-PCI-MSI.32768-edge.i915
1050 ± 28% -49.8% 527.33 ± 39% interrupts.CPU2.128:IR-PCI-MSI.520192-edge.eth0
54618 -37.7% 34042 ± 4% interrupts.CPU2.CAL:Function_call_interrupts
194173 ± 36% -68.0% 62105 interrupts.CPU2.LOC:Local_timer_interrupts
1547 ± 73% -97.3% 41.00 ± 77% interrupts.CPU2.NMI:Non-maskable_interrupts
1547 ± 73% -97.3% 41.00 ± 77% interrupts.CPU2.PMI:Performance_monitoring_interrupts
9129 ± 17% -48.5% 4701 ± 6% interrupts.CPU2.RES:Rescheduling_interrupts
1957 ± 60% -91.0% 175.33 ± 37% interrupts.CPU2.TLB:TLB_shootdowns
56187 ± 5% -44.9% 30951 ± 6% interrupts.CPU3.CAL:Function_call_interrupts
194173 ± 36% -68.0% 62079 interrupts.CPU3.LOC:Local_timer_interrupts
947.00 ± 67% -95.8% 39.33 ± 85% interrupts.CPU3.NMI:Non-maskable_interrupts
947.00 ± 67% -95.8% 39.33 ± 85% interrupts.CPU3.PMI:Performance_monitoring_interrupts
5359 ± 32% -93.4% 353.67 ± 46% interrupts.CPU3.TLB:TLB_shootdowns
776694 ± 36% -68.0% 248323 interrupts.LOC:Local_timer_interrupts
5586 ± 72% -97.1% 162.67 ± 83% interrupts.NMI:Non-maskable_interrupts
5586 ± 72% -97.1% 162.67 ± 83% interrupts.PMI:Performance_monitoring_interrupts
24885 ± 7% -36.2% 15873 ± 7% interrupts.RES:Rescheduling_interrupts
14966 ± 11% -83.6% 2461 ± 9% interrupts.TLB:TLB_shootdowns
2014 ± 46% +58.2% 3187 ± 34% sched_debug.cfs_rq:/.load_avg.max
33405 ± 40% -58.6% 13826 ± 9% sched_debug.cfs_rq:/.min_vruntime.avg
37077 ± 38% -51.2% 18080 ± 26% sched_debug.cfs_rq:/.min_vruntime.max
29779 ± 42% -60.7% 11702 ± 6% sched_debug.cfs_rq:/.min_vruntime.min
1489 ± 10% +38.2% 2058 ± 14% sched_debug.cfs_rq:/.util_avg.max
94180 ± 5% -69.7% 28544 ± 69% sched_debug.cpu.avg_idle.min
62749 ± 44% -65.5% 21617 ± 5% sched_debug.cpu.clock.avg
62750 ± 44% -65.5% 21623 ± 5% sched_debug.cpu.clock.max
62748 ± 44% -65.6% 21612 ± 5% sched_debug.cpu.clock.min
0.72 ± 20% +490.0% 4.26 ± 45% sched_debug.cpu.clock.stddev
61991 ± 44% -65.5% 21373 ± 5% sched_debug.cpu.clock_task.avg
62297 ± 44% -65.4% 21536 ± 5% sched_debug.cpu.clock_task.max
61557 ± 44% -65.7% 21140 ± 5% sched_debug.cpu.clock_task.min
297.83 ± 30% -43.4% 168.59 sched_debug.cpu.clock_task.stddev
2455 ± 33% +43.8% 3529 ± 10% sched_debug.cpu.curr->pid.avg
0.00 ± 31% -51.7% 0.00 ± 38% sched_debug.cpu.next_balance.stddev
203929 ± 46% -67.6% 66070 ± 7% sched_debug.cpu.nr_switches.avg
258658 ± 46% -68.4% 81856 ± 4% sched_debug.cpu.nr_switches.max
155789 ± 46% -67.3% 51018 ± 8% sched_debug.cpu.nr_switches.min
0.17 ±108% +250.0% 0.58 ± 53% sched_debug.cpu.nr_uninterruptible.avg
60.22 ± 56% -70.7% 17.67 ± 49% sched_debug.cpu.nr_uninterruptible.max
-78.78 -83.5% -13.00 sched_debug.cpu.nr_uninterruptible.min
54.22 ± 53% -77.4% 12.27 ± 41% sched_debug.cpu.nr_uninterruptible.stddev
1225073 ± 70% -99.9% 749.58 ± 36% sched_debug.cpu.sched_count.avg
1935922 ± 72% -99.9% 1573 ± 36% sched_debug.cpu.sched_count.max
519540 ± 79% -99.9% 535.76 ± 41% sched_debug.cpu.sched_count.stddev
68013 ± 70% -99.4% 428.67 ± 39% sched_debug.cpu.ttwu_count.avg
117399 ± 70% -99.3% 848.33 ± 51% sched_debug.cpu.ttwu_count.max
30185 ± 70% -99.0% 305.10 ± 62% sched_debug.cpu.ttwu_count.stddev
18430 ± 69% -98.5% 277.08 ± 43% sched_debug.cpu.ttwu_local.avg
41983 ± 69% -98.7% 534.33 ± 48% sched_debug.cpu.ttwu_local.max
13660 ± 68% -98.6% 186.19 ± 63% sched_debug.cpu.ttwu_local.stddev
62747 ± 44% -65.6% 21609 ± 5% sched_debug.cpu_clk
62538 ± 44% -65.7% 21422 ± 5% sched_debug.ktime
0.02 ±141% +249.5% 0.06 ± 22% sched_debug.rt_rq:/.rt_time.avg
62766 ± 44% -65.6% 21619 ± 5% sched_debug.sched_clk
53.07 ± 25% -37.9% 32.94 perf-stat.i.MPKI
1.429e+09 ± 27% +52.1% 2.173e+09 perf-stat.i.branch-instructions
25378470 ± 48% +113.2% 54106012 perf-stat.i.branch-misses
41939756 ± 26% +38.8% 58213548 perf-stat.i.cache-misses
1.916e+08 ± 25% +47.4% 2.823e+08 ± 2% perf-stat.i.cache-references
13953 ± 10% +17.0% 16320 perf-stat.i.context-switches
1.39 ± 16% -22.4% 1.08 perf-stat.i.cpi
5.98e+09 ± 31% +57.8% 9.436e+09 perf-stat.i.cpu-cycles
253.07 ± 45% +98.8% 503.12 ± 6% perf-stat.i.cpu-migrations
0.22 ± 38% -0.1 0.10 perf-stat.i.dTLB-load-miss-rate%
1478975 ± 21% +34.7% 1991993 perf-stat.i.dTLB-load-misses
1.516e+09 ± 24% +42.7% 2.163e+09 perf-stat.i.dTLB-loads
0.14 ± 11% -0.0 0.11 perf-stat.i.dTLB-store-miss-rate%
565610 ± 24% +34.5% 760493 perf-stat.i.dTLB-store-misses
6.417e+08 ± 17% +31.0% 8.404e+08 perf-stat.i.dTLB-stores
611337 ± 19% +29.3% 790163 ± 2% perf-stat.i.iTLB-load-misses
5.981e+09 ± 26% +49.3% 8.931e+09 perf-stat.i.instructions
1.82 ± 51% +157.8% 4.70 perf-stat.i.major-faults
1.68 ± 19% +0.8 2.49 perf-stat.overall.branch-miss-rate%
21.83 -1.2 20.63 ± 2% perf-stat.overall.cache-miss-rate%
0.99 ± 4% +6.9% 1.06 perf-stat.overall.cpi
141.04 ± 4% +14.9% 162.09 perf-stat.overall.cycles-between-cache-misses
0.10 ± 3% -0.0 0.09 perf-stat.overall.dTLB-load-miss-rate%
14.55 ± 6% +1.2 15.77 ± 2% perf-stat.overall.iTLB-load-miss-rate%
9694 ± 9% +16.7% 11311 ± 2% perf-stat.overall.instructions-per-iTLB-miss
1.01 ± 4% -6.6% 0.95 perf-stat.overall.ipc
1.413e+09 ± 26% +50.9% 2.132e+09 perf-stat.ps.branch-instructions
25047514 ± 47% +112.0% 53091697 perf-stat.ps.branch-misses
41461384 ± 25% +37.8% 57118869 perf-stat.ps.cache-misses
1.894e+08 ± 24% +46.3% 2.77e+08 ± 2% perf-stat.ps.cache-references
13813 ± 9% +15.9% 16012 perf-stat.ps.context-switches
5.91e+09 ± 30% +56.7% 9.258e+09 perf-stat.ps.cpu-cycles
249.80 ± 45% +97.7% 493.74 ± 6% perf-stat.ps.cpu-migrations
1462753 ± 20% +33.6% 1954515 perf-stat.ps.dTLB-load-misses
1.499e+09 ± 23% +41.6% 2.122e+09 perf-stat.ps.dTLB-loads
6.349e+08 ± 16% +29.9% 8.246e+08 perf-stat.ps.dTLB-stores
3524622 ± 12% +17.5% 4140201 perf-stat.ps.iTLB-loads
5.913e+09 ± 25% +48.2% 8.763e+09 perf-stat.ps.instructions
1.80 ± 50% +156.4% 4.61 perf-stat.ps.major-faults
5.49e+11 ± 21% -50.6% 2.71e+11 perf-stat.total.instructions
phoronix-test-suite.clpeak.KernelLatency.us
35 +----------------------------------------------------------------------+
| |
30 |-+ O O O O |
| O O |
25 |-+ +...+ +...+ + + + + |
| : : : : : : : + : |
20 |:+ : : : : : : : : : : : : : :|
|: : : : : : : : : : : : : : : :|
15 |-: : : : : : : : : : : : : : : : |
| : : : : : : : : : : : : : : : : |
10 |-: : : : : : : : : : : : : : : : |
| : : : : : : : : : : : : : : : : |
5 |-+: : : : : : : : : : : : : : : : |
| : : : : : : : : |
0 +----------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
8 months, 4 weeks
b276527539 ("staging: most: move core files out of the staging .."): [ 12.247349] BUG: kernel NULL pointer dereference, address: 00000000
by kernel test robot
Greetings,
0day kernel testing robot got the below dmesg and the first bad commit is
https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git staging-next
commit b276527539188f1f61c082ebef27803db93e536d
Author: Christian Gromm <christian.gromm(a)microchip.com>
AuthorDate: Tue Mar 10 14:02:40 2020 +0100
Commit: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
CommitDate: Tue Mar 24 13:42:44 2020 +0100
staging: most: move core files out of the staging area
This patch moves the core module to the /drivers/most directory
and makes all necessary changes in order to not break the build.
Signed-off-by: Christian Gromm <christian.gromm(a)microchip.com>
Link: https://lore.kernel.org/r/1583845362-26707-2-git-send-email-christian.gro...
Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org>
22dd4acc80 Staging: speakup: Add identifier name to function declaration arguments.
b276527539 staging: most: move core files out of the staging area
e681bb287f staging: vt6656: Use DIV_ROUND_UP macro instead of specific code
+-------------------------------------------------------+------------+------------+------------+
| | 22dd4acc80 | b276527539 | e681bb287f |
+-------------------------------------------------------+------------+------------+------------+
| boot_successes | 26 | 0 | 0 |
| boot_failures | 8 | 11 | 11 |
| WARNING:possible_circular_locking_dependency_detected | 8 | | |
| BUG:kernel_NULL_pointer_dereference,address | 0 | 11 | 11 |
| Oops:#[##] | 0 | 11 | 11 |
| EIP:__list_add_valid | 0 | 11 | 11 |
| Kernel_panic-not_syncing:Fatal_exception | 0 | 11 | 11 |
+-------------------------------------------------------+------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
[ 12.242090] no options.
[ 12.245364] FPGA DOWNLOAD --->
[ 12.245723] FPGA image file name: xlinx_fpga_firmware.bit
[ 12.246548] GPIO INIT FAIL!!
[ 12.246995] most_sound: init()
[ 12.247349] BUG: kernel NULL pointer dereference, address: 00000000
[ 12.248032] #PF: supervisor read access in kernel mode
[ 12.248322] #PF: error_code(0x0000) - not-present page
[ 12.248322] *pdpt = 0000000000000000 *pde = f000ff53f000ff53
[ 12.248322] Oops: 0000 [#1] PREEMPT SMP
[ 12.248322] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc7-00376-gb276527539188 #1
[ 12.248322] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 12.248322] EIP: __list_add_valid+0x29/0x77
[ 12.248322] Code: c3 55 89 e5 56 53 83 ec 10 8b 59 04 39 d3 74 1a 89 4c 24 0c 89 5c 24 08 89 54 24 04 c7 04 24 00 cc bd c2 e8 84 9e b4 ff 0f 0b <8b> 33 39 ce 74 1a 89 5c 24 0c 89 74 24 08 89 4c 24 04 c7 04 24 7c
[ 12.248322] EAX: c2f45800 EBX: 00000000 ECX: c3e8df50 EDX: 00000000
[ 12.248322] ESI: 00000000 EDI: ec4a7f68 EBP: ec4a7ee8 ESP: ec4a7ed0
[ 12.248322] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010246
[ 12.248322] CR0: 80050033 CR2: 00000000 CR3: 03256000 CR4: 001406b0
[ 12.248322] Call Trace:
[ 12.248322] ? vprintk_func+0x9d/0xa7
[ 12.248322] most_register_component+0x33/0x53
[ 12.248322] ? wilc_spi_driver_init+0x11/0x11
[ 12.248322] audio_init+0x2c/0x76
[ 12.248322] do_one_initcall+0xf0/0x284
[ 12.248322] ? __might_sleep+0x70/0x77
[ 12.262064] kernel_init_freeable+0x141/0x1a5
[ 12.262064] ? rest_init+0x205/0x205
[ 12.262064] kernel_init+0xb/0xea
[ 12.262064] ? schedule_tail_wrapper+0x9/0xc
[ 12.262064] ret_from_fork+0x2e/0x38
[ 12.262064] Modules linked in:
[ 12.262064] CR2: 0000000000000000
[ 12.262064] ---[ end trace 7c7a2cb6d11f9c5d ]---
[ 12.262064] EIP: __list_add_valid+0x29/0x77
# HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD
git bisect start a400b349905677d3561138087528f36718cbd9fc 16fbf79b0f83bc752cee8589279f1ebfe57b3b6e --
git bisect bad 1fb7e4086438ac799b09b0fcafbe7d6d694d2229 # 10:44 B 0 9 25 0 Merge 'linux-review/Gustavo-A-R-Silva/chrome-cros_ec_chardev-Replace-zero-length-array-with-flexible-array-member/20200321-071741' into devel-hourly-2020032803
git bisect bad aa5a6dfe7759dceb732db7578fd8e93b558c583f # 11:03 B 0 6 22 0 Merge 'linux-review/Ravi-Bangoria/perf-dso-Fix-dso-comparison/20200324-152752' into devel-hourly-2020032803
git bisect good 9f5dc12221c99dd30a64be3fbce2130654d1fcc0 # 11:38 G 10 0 3 3 Merge 'linux-review/Mateusz-Gorski/Add-topologies-for-generic-HDA-DSP-machine-driver/20200326-075250' into devel-hourly-2020032803
git bisect good 4f2fa08ebb25660ca706626bf58b0ddccf6e35c6 # 12:13 G 10 0 3 3 Merge 'thermal/next' into devel-hourly-2020032803
git bisect bad 0b76d1986630dbd2332c91a2f94875ee93ca7e3a # 12:40 B 0 7 23 0 Merge 'linux-review/Weihang-Li/RDMA-hns-Update-some-configurations-related-to-hardware/20200327-030803' into devel-hourly-2020032803
git bisect good 3817f5f77f139d27a42ff92b0fcbae964665a471 # 13:34 G 11 0 2 2 Merge 'linux-review/Leon-Romanovsky/RDMA-rxe-Set-sys_image_guid-to-be-aligned-with-HW-IB-devices/20200323-235957' into devel-hourly-2020032803
git bisect good 37f479d4be63453151a4e8b1db0d18a102fa6e81 # 14:38 G 10 0 2 2 Merge 'linux-review/sunil-kovvuri-gmail-com/Miscellaneous-fixes/20200326-074148' into devel-hourly-2020032803
git bisect bad 9e25511121b25d3191aea63e9c2c65d8beed750e # 14:57 B 0 1 17 0 Merge 'linux-review/Rayagonda-Kokatanur/i2c-iproc-add-support-for-SMBUS-quick-cmd/20200323-045331' into devel-hourly-2020032803
git bisect good 676d651ef12f80cc35ef101bfe610d6a6f12ef10 # 15:40 G 10 0 5 5 Merge 'linux-review/Luiz-Augusto-von-Dentz/Bluetooth-Add-BT_MODE-socket-option/20200327-073534' into devel-hourly-2020032803
git bisect bad 25c4b63ca26e35d0793bdd2c1b24bed8cf95801d # 16:29 B 0 2 18 0 Merge 'linux-review/Simran-Singhal/staging-rtl8723bs-Clean-up-tests-if-NULL-returned-on-failure/20200327-012809' into devel-hourly-2020032803
git bisect good 8eb6b39c0ff38fd1bf21454c438fb3a2a52636f9 # 17:36 G 11 0 4 4 staging: qlge: qlge_dbg: remove unneeded spaces
git bisect good fe297f8f048a7a0663479dcf6447ec450b53b905 # 18:35 G 10 0 2 2 iio: dac: Kconfig: sort symbols alphabetically
git bisect good 9791ca0dcb46dcedc5fb93fb4e8791cedc91babd # 19:31 G 10 0 4 4 staging: mt7621-pci-phy: re-do 'xtal_mode' detection
git bisect bad c926f022c2560d801545e6b55dacc9082277d18e # 20:19 B 0 3 19 0 staging: rtl8723bs: Add line after variable declarations
git bisect bad b276527539188f1f61c082ebef27803db93e536d # 20:43 B 0 6 22 0 staging: most: move core files out of the staging area
git bisect good 759411a0e0f35c8ae1778a838a97cec9d0140f7f # 21:11 G 10 0 2 2 staging: wlan-ng: Fix third argument going over 80 characters
git bisect good 7385adece19190f28800e0fc2fa964b96706120a # 21:42 G 10 0 2 2 staging: hp100: Add space around operator +
git bisect good 7cf9a79dbae127776a6ddd455494faec9f3af9fb # 22:13 G 11 0 7 7 Staging: speakup: Use sizeof(*var) in kmalloc().
git bisect good 22dd4acc80495ea0fa3a54b36e49ef9dd5bad6c6 # 22:40 G 10 0 1 1 Staging: speakup: Add identifier name to function declaration arguments.
# first bad commit: [b276527539188f1f61c082ebef27803db93e536d] staging: most: move core files out of the staging area
git bisect good 22dd4acc80495ea0fa3a54b36e49ef9dd5bad6c6 # 22:45 G 31 0 7 8 Staging: speakup: Add identifier name to function declaration arguments.
# extra tests with debug options
git bisect good b276527539188f1f61c082ebef27803db93e536d # 23:05 G 11 0 0 0 staging: most: move core files out of the staging area
# extra tests on head commit of staging/staging-next
git bisect bad e681bb287f40e7a9dbcb04cef80fd87a2511ab86 # 23:49 B 0 1 17 0 staging: vt6656: Use DIV_ROUND_UP macro instead of specific code
# bad: [e681bb287f40e7a9dbcb04cef80fd87a2511ab86] staging: vt6656: Use DIV_ROUND_UP macro instead of specific code
# extra tests on revert first bad commit
git bisect good 099a66b938bb0368b91c81e4431180ef11a0b66b # 00:32 G 10 0 1 1 Revert "staging: most: move core files out of the staging area"
# good: [099a66b938bb0368b91c81e4431180ef11a0b66b] Revert "staging: most: move core files out of the staging area"
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/lkp@lists.01.org
9 months
[y2038] 412c53a680: will-it-scale.per_process_ops 11.7% improvement
by kernel test robot
Greeting,
FYI, we noticed a 11.7% improvement of will-it-scale.per_process_ops due to commit:
commit: 412c53a680a97cb1ae2c0ab60230e193bee86387 ("y2038: remove unused time32 interfaces")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory
with following parameters:
nr_task: 100%
mode: process
test: mmap2
cpufreq_governor: performance
ucode: 0x2000065
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
In addition to that, the commit also has significant impact on the following tests:
+------------------+---------------------------------------------+
| testcase: change | unixbench: unixbench.score 3.8% improvement |
| test machine | 104 threads Skylake with 192G memory |
| test parameters | cpufreq_governor=performance |
| | nr_task=30% |
| | runtime=300s |
| | test=context1 |
| | ucode=0x2000065 |
+------------------+---------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-skl-2sp7/mmap2/will-it-scale/0x2000065
commit:
595abbaff5 ("y2038: remove ktime to/from timespec/timeval conversion")
412c53a680 ("y2038: remove unused time32 interfaces")
595abbaff5db1214 412c53a680a97cb1ae2c0ab6023
---------------- ---------------------------
%stddev %change %stddev
\ | \
16079 +11.7% 17954 will-it-scale.per_process_ops
1157721 +11.7% 1292722 will-it-scale.workload
535.67 ±141% +201.4% 1614 meminfo.Mlocked
1.01 +0.1 1.12 mpstat.cpu.all.usr%
61269544 ± 54% +137.2% 1.453e+08 ± 48% cpuidle.C1E.time
143915 ± 36% +133.1% 335431 ± 57% cpuidle.C1E.usage
2261 ± 2% -15.1% 1920 ± 5% slabinfo.fsnotify_mark_connector.active_objs
2261 ± 2% -15.1% 1920 ± 5% slabinfo.fsnotify_mark_connector.num_objs
1649 ± 8% -12.5% 1443 ± 2% slabinfo.skbuff_ext_cache.active_objs
1649 ± 8% -12.5% 1443 ± 2% slabinfo.skbuff_ext_cache.num_objs
14258 ± 12% +25.5% 17895 ± 13% numa-meminfo.node0.Mapped
4132 ± 15% +20.3% 4970 numa-meminfo.node0.PageTables
5989 ± 10% -10.3% 5374 ± 4% numa-meminfo.node1.KernelStack
17133 ± 11% -18.9% 13895 ± 17% numa-meminfo.node1.Mapped
4799 ± 12% -18.1% 3928 numa-meminfo.node1.PageTables
7936 +1.4% 8050 proc-vmstat.nr_mapped
133.67 ±141% +201.5% 403.00 proc-vmstat.nr_mlock
18121 ± 2% +5.7% 19153 ± 3% proc-vmstat.nr_shmem
743293 +1.2% 752255 proc-vmstat.pgalloc_normal
700933 +1.4% 710819 proc-vmstat.pgfree
3601 ± 12% +27.9% 4605 ± 12% numa-vmstat.node0.nr_mapped
1033 ± 15% +20.3% 1243 numa-vmstat.node0.nr_page_table_pages
156534 ± 6% -33.2% 104509 ± 64% numa-vmstat.node0.numa_other
5989 ± 10% -10.3% 5375 ± 4% numa-vmstat.node1.nr_kernel_stack
56.00 ±141% +278.0% 211.67 ± 13% numa-vmstat.node1.nr_mlock
1199 ± 12% -18.1% 982.33 numa-vmstat.node1.nr_page_table_pages
5818 ± 35% +43.9% 8370 ± 6% interrupts.CPU26.NMI:Non-maskable_interrupts
5818 ± 35% +43.9% 8370 ± 6% interrupts.CPU26.PMI:Performance_monitoring_interrupts
414.67 ± 23% -24.8% 311.67 interrupts.CPU52.RES:Rescheduling_interrupts
5819 ± 35% +43.8% 8366 ± 6% interrupts.CPU56.NMI:Non-maskable_interrupts
5819 ± 35% +43.8% 8366 ± 6% interrupts.CPU56.PMI:Performance_monitoring_interrupts
5818 ± 35% +43.8% 8364 ± 6% interrupts.CPU59.NMI:Non-maskable_interrupts
5818 ± 35% +43.8% 8364 ± 6% interrupts.CPU59.PMI:Performance_monitoring_interrupts
471.33 ± 16% -26.0% 349.00 ± 6% interrupts.CPU9.RES:Rescheduling_interrupts
127.33 ± 3% +11.8% 142.33 ± 2% interrupts.IWI:IRQ_work_interrupts
10312 ± 92% -67.7% 3333 ± 6% sched_debug.cfs_rq:/.load.stddev
27.16 ± 7% -16.2% 22.76 ± 8% sched_debug.cfs_rq:/.load_avg.avg
47.12 ± 6% -17.5% 38.89 ± 10% sched_debug.cfs_rq:/.load_avg.stddev
8.99 ± 22% -46.6% 4.80 ± 41% sched_debug.cfs_rq:/.removed.load_avg.avg
37.49 ± 10% -27.4% 27.22 ± 21% sched_debug.cfs_rq:/.removed.load_avg.stddev
413.16 ± 23% -46.4% 221.38 ± 42% sched_debug.cfs_rq:/.removed.runnable_sum.avg
1722 ± 10% -27.2% 1253 ± 22% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
2.71 ± 22% -41.0% 1.60 ± 16% sched_debug.cfs_rq:/.removed.util_avg.avg
1480 ± 4% +11.4% 1648 sched_debug.cpu.curr->pid.min
0.19 ± 2% -13.4% 0.17 ± 7% sched_debug.cpu.nr_running.stddev
32567 ± 5% +13.4% 36931 ± 4% sched_debug.cpu.nr_switches.max
62048 ± 5% +21.0% 75097 ± 10% softirqs.CPU10.RCU
60821 +9.1% 66347 softirqs.CPU17.RCU
59451 ± 3% +19.2% 70891 ± 2% softirqs.CPU22.RCU
59548 +7.6% 64046 ± 2% softirqs.CPU34.RCU
59706 +16.4% 69478 ± 6% softirqs.CPU41.RCU
61173 ± 4% +22.1% 74662 ± 12% softirqs.CPU46.RCU
59827 +21.6% 72779 ± 14% softirqs.CPU5.RCU
60645 +11.0% 67300 ± 5% softirqs.CPU53.RCU
58533 ± 2% +9.0% 63779 ± 2% softirqs.CPU57.RCU
60026 ± 2% +15.7% 69444 ± 5% softirqs.CPU58.RCU
61127 ± 2% +11.4% 68125 ± 4% softirqs.CPU63.RCU
4.413e+09 +7.3% 4.733e+09 perf-stat.i.branch-instructions
19144349 +5.4% 20170738 perf-stat.i.branch-misses
41.28 -0.5 40.80 perf-stat.i.cache-miss-rate%
22256771 +3.9% 23133318 ± 2% perf-stat.i.cache-misses
53886473 +5.1% 56658720 ± 2% perf-stat.i.cache-references
11.77 -7.4% 10.90 perf-stat.i.cpi
9929 -4.3% 9499 ± 2% perf-stat.i.cycles-between-cache-misses
0.05 +0.0 0.05 ± 3% perf-stat.i.dTLB-load-miss-rate%
2370068 +11.2% 2635764 perf-stat.i.dTLB-load-misses
5.229e+09 +7.8% 5.638e+09 perf-stat.i.dTLB-loads
9837 ± 3% +11.8% 10995 ± 6% perf-stat.i.dTLB-store-misses
1.718e+09 +10.8% 1.903e+09 perf-stat.i.dTLB-stores
94.23 -9.1 85.12 perf-stat.i.iTLB-load-miss-rate%
2419173 -33.5% 1607709 perf-stat.i.iTLB-load-misses
146272 ± 3% +94.5% 284553 ± 8% perf-stat.i.iTLB-loads
1.88e+10 +7.4% 2.019e+10 perf-stat.i.instructions
7824 +61.4% 12627 perf-stat.i.instructions-per-iTLB-miss
0.09 +8.0% 0.09 perf-stat.i.ipc
5549135 +6.7% 5919869 perf-stat.i.node-load-misses
4803604 +6.0% 5091055 perf-stat.i.node-store-misses
0.43 -0.0 0.43 perf-stat.overall.branch-miss-rate%
41.30 -0.5 40.82 perf-stat.overall.cache-miss-rate%
11.76 -7.2% 10.91 perf-stat.overall.cpi
9937 -4.1% 9530 ± 2% perf-stat.overall.cycles-between-cache-misses
0.05 +0.0 0.05 perf-stat.overall.dTLB-load-miss-rate%
94.30 -9.3 84.96 perf-stat.overall.iTLB-load-miss-rate%
7770 +61.7% 12560 perf-stat.overall.instructions-per-iTLB-miss
0.09 +7.8% 0.09 perf-stat.overall.ipc
4887274 -3.3% 4723803 perf-stat.overall.path-length
4.398e+09 +7.3% 4.717e+09 perf-stat.ps.branch-instructions
19086955 +5.4% 20108226 perf-stat.ps.branch-misses
22182689 +3.9% 23057159 ± 2% perf-stat.ps.cache-misses
53712718 +5.2% 56486162 ± 2% perf-stat.ps.cache-references
2362866 +11.3% 2630144 perf-stat.ps.dTLB-load-misses
5.211e+09 +7.8% 5.62e+09 perf-stat.ps.dTLB-loads
9852 ± 3% +12.8% 11114 ± 6% perf-stat.ps.dTLB-store-misses
1.712e+09 +10.8% 1.897e+09 perf-stat.ps.dTLB-stores
2411038 -33.5% 1602293 perf-stat.ps.iTLB-load-misses
145843 ± 3% +94.6% 283843 ± 8% perf-stat.ps.iTLB-loads
1.873e+10 +7.4% 2.013e+10 perf-stat.ps.instructions
5530540 +6.7% 5900087 perf-stat.ps.node-load-misses
4787463 +6.0% 5073936 perf-stat.ps.node-store-misses
5.658e+12 +7.9% 6.107e+12 perf-stat.total.instructions
47.65 -0.8 46.85 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap.vm_mmap_pgoff
47.66 -0.8 46.85 perf-profile.calltrace.cycles-pp.__vm_enough_memory.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff
49.09 -0.8 48.31 perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe
48.84 -0.8 48.06 perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64
47.18 -0.8 46.40 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap
49.24 -0.8 48.47 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
49.31 -0.8 48.55 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
46.98 -0.8 46.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region
49.62 -0.7 48.89 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64
49.63 -0.7 48.90 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64
50.01 -0.7 49.31 perf-profile.calltrace.cycles-pp.mmap64
1.17 +0.1 1.28 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
1.27 +0.1 1.40 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
1.69 +0.2 1.84 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
46.89 +0.5 47.36 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
46.41 +0.5 46.89 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap
46.21 +0.5 46.70 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__do_munmap.__vm_munmap
0.00 +0.5 0.53 perf-profile.calltrace.cycles-pp.___might_sleep.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
49.26 +0.6 49.89 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
49.29 +0.6 49.92 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
48.83 +0.6 49.48 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
49.59 +0.7 50.26 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap
49.60 +0.7 50.27 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap
49.94 +0.7 50.64 perf-profile.calltrace.cycles-pp.munmap
47.66 -0.8 46.85 perf-profile.children.cycles-pp.__vm_enough_memory
49.10 -0.8 48.32 perf-profile.children.cycles-pp.do_mmap
48.84 -0.8 48.07 perf-profile.children.cycles-pp.mmap_region
49.24 -0.8 48.48 perf-profile.children.cycles-pp.vm_mmap_pgoff
49.31 -0.8 48.55 perf-profile.children.cycles-pp.ksys_mmap_pgoff
50.03 -0.7 49.33 perf-profile.children.cycles-pp.mmap64
94.55 -0.3 94.21 perf-profile.children.cycles-pp.percpu_counter_add_batch
93.61 -0.3 93.31 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
93.19 -0.3 92.92 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
99.23 -0.1 99.18 perf-profile.children.cycles-pp.do_syscall_64
99.26 -0.0 99.21 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.46 ± 2% -0.0 0.43 perf-profile.children.cycles-pp.vm_area_alloc
0.43 ± 3% -0.0 0.40 perf-profile.children.cycles-pp.kmem_cache_alloc
0.31 ± 2% -0.0 0.29 perf-profile.children.cycles-pp.apic_timer_interrupt
0.06 +0.0 0.07 perf-profile.children.cycles-pp.down_write_killable
0.08 ± 6% +0.0 0.09 perf-profile.children.cycles-pp.prepend_path
0.06 +0.0 0.07 ± 6% perf-profile.children.cycles-pp.shmem_mmap
0.10 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_iterate_sb
0.05 +0.0 0.07 perf-profile.children.cycles-pp.unlink_file_vma
0.33 +0.0 0.35 perf-profile.children.cycles-pp.syscall_return_via_sysret
0.40 ± 4% +0.0 0.43 perf-profile.children.cycles-pp.perf_event_mmap
0.08 ± 5% +0.0 0.11 perf-profile.children.cycles-pp.free_pgtables
0.34 +0.0 0.38 perf-profile.children.cycles-pp.entry_SYSCALL_64
0.00 +0.1 0.05 perf-profile.children.cycles-pp.up_write
0.00 +0.1 0.05 perf-profile.children.cycles-pp.touch_atime
0.53 +0.1 0.60 perf-profile.children.cycles-pp.___might_sleep
1.25 +0.1 1.35 perf-profile.children.cycles-pp.unmap_page_range
1.28 +0.1 1.40 perf-profile.children.cycles-pp.unmap_vmas
1.69 +0.2 1.85 perf-profile.children.cycles-pp.unmap_region
49.29 +0.6 49.93 perf-profile.children.cycles-pp.__x64_sys_munmap
49.26 +0.6 49.90 perf-profile.children.cycles-pp.__vm_munmap
48.84 +0.6 49.48 perf-profile.children.cycles-pp.__do_munmap
49.97 +0.7 50.67 perf-profile.children.cycles-pp.munmap
93.19 -0.3 92.92 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.42 ± 2% -0.0 0.38 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.40 ± 3% -0.0 0.38 perf-profile.self.cycles-pp.kmem_cache_alloc
0.12 -0.0 0.11 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.33 +0.0 0.35 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.30 +0.0 0.33 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.61 ± 2% +0.0 0.66 perf-profile.self.cycles-pp.unmap_page_range
0.58 +0.1 0.64 perf-profile.self.cycles-pp.do_syscall_64
0.50 +0.1 0.56 perf-profile.self.cycles-pp.___might_sleep
will-it-scale.per_process_ops
20000 +-------------------------------------------------------------------+
18000 |-+O O O O O O O O O O O O O O O O |
| |
16000 |..+.+..+.+..+..+.+..+..+.+..+.+..+..+.+..+.+..+..+.+..+..+.+..+ |
14000 |-+ : |
| : :|
12000 |-+ : :|
10000 |-+ : :|
8000 |-+ : :|
| : : |
6000 |-+ : : |
4000 |-+ : : |
| :: |
2000 |-+ : |
0 +-------------------------------------------------------------------+
will-it-scale.workload
1.4e+06 +-----------------------------------------------------------------+
| O O O O O O O O O O O O O O O O |
1.2e+06 |..+.+.. .+..+.+..+.+..+.+..+. .+..+.+..+. .+.+..+.+..+.+..+ |
| + +. +. : |
1e+06 |-+ : |
| : :|
800000 |-+ : :|
| : :|
600000 |-+ : :|
| : : |
400000 |-+ : : |
| :: |
200000 |-+ :: |
| : |
0 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
***************************************************************************************************
lkp-skl-fpga01: 104 threads Skylake with 192G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode:
gcc-7/performance/x86_64-rhel-7.6/30%/debian-x86_64-20191114.cgz/300s/lkp-skl-fpga01/context1/unixbench/0x2000065
commit:
595abbaff5 ("y2038: remove ktime to/from timespec/timeval conversion")
412c53a680 ("y2038: remove unused time32 interfaces")
595abbaff5db1214 412c53a680a97cb1ae2c0ab6023
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
3093 +3.8% 3209 unixbench.score
471.47 +36.6% 644.15 ± 40% unixbench.time.user_time
3.659e+08 +3.9% 3.801e+08 unixbench.time.voluntary_context_switches
4.838e+08 +3.8% 5.02e+08 unixbench.workload
3715604 +3.9% 3859617 vmstat.system.cs
1493573 -1.1% 1477832 proc-vmstat.numa_local
1620946 -1.3% 1599963 proc-vmstat.pgalloc_normal
1586478 -1.6% 1561368 proc-vmstat.pgfree
4505528 ± 8% -11.7% 3977138 ± 9% sched_debug.cfs_rq:/.MIN_vruntime.max
47683 ± 7% -11.2% 42356 ± 7% sched_debug.cfs_rq:/.exec_clock.min
4505528 ± 8% -11.7% 3977138 ± 9% sched_debug.cfs_rq:/.max_vruntime.max
2750853 ± 7% -11.4% 2435892 ± 7% sched_debug.cfs_rq:/.min_vruntime.min
0.31 ± 13% -33.1% 0.21 ± 25% sched_debug.cfs_rq:/.nr_spread_over.avg
3.18 ± 18% -28.1% 2.29 ± 22% sched_debug.cfs_rq:/.nr_spread_over.max
0.67 ± 11% -27.6% 0.48 ± 22% sched_debug.cfs_rq:/.nr_spread_over.stddev
133566 ± 48% +65.9% 221588 ± 31% sched_debug.cfs_rq:/.runnable_weight.max
899.95 ±173% +824.1% 8316 ±110% sched_debug.cpu.max_idle_balance_cost.stddev
317.01 ± 5% -9.2% 287.99 ± 7% sched_debug.cpu.ttwu_local.min
6.175e+09 +3.2% 6.375e+09 perf-stat.i.branch-instructions
6.846e+08 +2.4% 7.011e+08 perf-stat.i.cache-references
3734905 +3.9% 3879483 perf-stat.i.context-switches
7.582e+09 +3.4% 7.839e+09 perf-stat.i.dTLB-loads
4.468e+09 +3.6% 4.63e+09 perf-stat.i.dTLB-stores
4308069 +3.9% 4477501 perf-stat.i.iTLB-load-misses
28346961 +4.7% 29683898 perf-stat.i.iTLB-loads
2.823e+10 +3.3% 2.916e+10 perf-stat.i.instructions
3745153 +21.4% 4545801 perf-stat.i.node-load-misses
33138 +33.5% 44236 perf-stat.i.node-loads
3740245 +3.3% 3862099 perf-stat.i.node-store-misses
3.02 -2.9% 2.94 perf-stat.overall.cpi
0.33 +3.0% 0.34 perf-stat.overall.ipc
6.157e+09 +3.3% 6.357e+09 perf-stat.ps.branch-instructions
6.826e+08 +2.4% 6.991e+08 perf-stat.ps.cache-references
3723897 +3.9% 3868660 perf-stat.ps.context-switches
7.56e+09 +3.4% 7.817e+09 perf-stat.ps.dTLB-loads
4.454e+09 +3.7% 4.617e+09 perf-stat.ps.dTLB-stores
4295642 +3.9% 4465151 perf-stat.ps.iTLB-load-misses
28265235 +4.7% 29603109 perf-stat.ps.iTLB-loads
2.814e+10 +3.3% 2.908e+10 perf-stat.ps.instructions
3734083 +21.4% 4532943 perf-stat.ps.node-load-misses
33087 +33.4% 44149 perf-stat.ps.node-loads
3729103 +3.3% 3851108 perf-stat.ps.node-store-misses
1.103e+13 +3.3% 1.14e+13 perf-stat.total.instructions
372.25 ± 48% +114.4% 798.00 ± 28% interrupts.41:PCI-MSI.67633156-edge.eth0-TxRx-3
2920 ± 6% -12.3% 2561 ± 10% interrupts.CPU1.NMI:Non-maskable_interrupts
2920 ± 6% -12.3% 2561 ± 10% interrupts.CPU1.PMI:Performance_monitoring_interrupts
2691 ± 6% +18.2% 3179 ± 5% interrupts.CPU103.NMI:Non-maskable_interrupts
2691 ± 6% +18.2% 3179 ± 5% interrupts.CPU103.PMI:Performance_monitoring_interrupts
33.25 ± 37% +542.9% 213.75 ± 64% interrupts.CPU14.TLB:TLB_shootdowns
71.50 ±115% +202.8% 216.50 ± 83% interrupts.CPU16.TLB:TLB_shootdowns
3048 ± 13% -26.0% 2256 ± 14% interrupts.CPU18.NMI:Non-maskable_interrupts
3048 ± 13% -26.0% 2256 ± 14% interrupts.CPU18.PMI:Performance_monitoring_interrupts
2743 ± 4% -14.8% 2337 ± 18% interrupts.CPU19.NMI:Non-maskable_interrupts
2743 ± 4% -14.8% 2337 ± 18% interrupts.CPU19.PMI:Performance_monitoring_interrupts
26.00 ± 61% +1373.1% 383.00 ± 50% interrupts.CPU23.TLB:TLB_shootdowns
229.00 ± 45% +87.3% 429.00 ± 29% interrupts.CPU29.TLB:TLB_shootdowns
372.25 ± 48% +114.4% 798.00 ± 28% interrupts.CPU33.41:PCI-MSI.67633156-edge.eth0-TxRx-3
28.75 ± 50% +360.9% 132.50 ± 94% interrupts.CPU39.TLB:TLB_shootdowns
41.00 ±101% +600.0% 287.00 ± 37% interrupts.CPU49.TLB:TLB_shootdowns
39.75 ± 79% +140.9% 95.75 ± 13% interrupts.CPU50.TLB:TLB_shootdowns
3103 ± 9% -16.6% 2589 ± 10% interrupts.CPU53.NMI:Non-maskable_interrupts
3103 ± 9% -16.6% 2589 ± 10% interrupts.CPU53.PMI:Performance_monitoring_interrupts
163.75 ± 58% -58.2% 68.50 ±100% interrupts.CPU59.TLB:TLB_shootdowns
2469 ± 20% +25.1% 3089 ± 6% interrupts.CPU63.NMI:Non-maskable_interrupts
2469 ± 20% +25.1% 3089 ± 6% interrupts.CPU63.PMI:Performance_monitoring_interrupts
69.25 ±107% +210.1% 214.75 ± 57% interrupts.CPU7.TLB:TLB_shootdowns
49.75 ± 33% +165.3% 132.00 ± 74% interrupts.CPU76.TLB:TLB_shootdowns
27.75 ± 82% +272.1% 103.25 ± 66% interrupts.CPU78.TLB:TLB_shootdowns
2754 ± 23% +23.0% 3387 ± 7% interrupts.CPU84.NMI:Non-maskable_interrupts
2754 ± 23% +23.0% 3387 ± 7% interrupts.CPU84.PMI:Performance_monitoring_interrupts
22.50 ± 42% +687.8% 177.25 ±132% interrupts.CPU84.TLB:TLB_shootdowns
19.75 ± 35% +812.7% 180.25 ± 74% interrupts.CPU85.TLB:TLB_shootdowns
15.75 ± 18% +303.2% 63.50 ± 81% interrupts.CPU99.TLB:TLB_shootdowns
31.35 -0.6 30.72 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task
30.60 -0.6 29.98 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
34.93 -0.5 34.42 perf-profile.calltrace.cycles-pp.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate
36.88 -0.5 36.42 perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up
37.21 -0.4 36.77 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common
37.18 -0.4 36.75 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function
37.22 -0.4 36.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock
35.18 -0.4 34.79 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.30 -0.4 34.93 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
39.42 -0.4 39.06 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write
39.26 -0.4 38.91 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write
39.64 -0.3 39.30 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write
39.95 -0.3 39.62 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write
41.14 -0.3 40.83 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
41.08 -0.3 40.77 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64
0.79 ± 7% -0.1 0.69 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry
0.73 ± 8% -0.1 0.64 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle
0.69 +0.0 0.72 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret
0.70 ± 2% +0.0 0.73 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64
0.75 +0.0 0.78 perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry
0.98 ± 2% +0.0 1.02 perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle
1.26 +0.0 1.31 perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.pipe_read
1.47 +0.1 1.52 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.pipe_read.new_sync_read
2.38 +0.1 2.50 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary
2.43 +0.1 2.56 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
5.47 +0.1 5.61 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
3.32 +0.1 3.47 perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read
5.29 +0.2 5.46 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.94 ± 2% +0.2 6.14 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
5.83 ± 2% +0.2 6.04 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64
3.11 ± 8% +0.3 3.38 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.pipe_read.new_sync_read.vfs_read
0.13 ±173% +0.4 0.53 perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule
2.47 ± 9% +0.4 2.89 perf-profile.calltrace.cycles-pp.arch_stack_walk.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair
30.79 -0.6 30.14 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
32.21 -0.6 31.62 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
34.94 -0.5 34.43 perf-profile.children.cycles-pp.__account_scheduler_latency
36.94 -0.5 36.48 perf-profile.children.cycles-pp.enqueue_entity
37.19 -0.4 36.75 perf-profile.children.cycles-pp.enqueue_task_fair
37.22 -0.4 36.78 perf-profile.children.cycles-pp.activate_task
37.23 -0.4 36.79 perf-profile.children.cycles-pp.ttwu_do_activate
39.27 -0.4 38.91 perf-profile.children.cycles-pp.try_to_wake_up
39.65 -0.4 39.29 perf-profile.children.cycles-pp.__wake_up_common
39.42 -0.4 39.06 perf-profile.children.cycles-pp.autoremove_wake_function
39.95 -0.3 39.62 perf-profile.children.cycles-pp.__wake_up_common_lock
41.16 -0.3 40.85 perf-profile.children.cycles-pp.new_sync_write
41.08 -0.3 40.77 perf-profile.children.cycles-pp.pipe_write
41.45 -0.3 41.16 perf-profile.children.cycles-pp.vfs_write
41.59 -0.3 41.32 perf-profile.children.cycles-pp.ksys_write
0.05 +0.0 0.06 perf-profile.children.cycles-pp.apparmor_file_permission
0.17 ± 4% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.tick_nohz_idle_enter
0.33 +0.0 0.35 ± 3% perf-profile.children.cycles-pp.select_idle_sibling
0.41 +0.0 0.43 ± 2% perf-profile.children.cycles-pp.__next_timer_interrupt
0.25 ± 3% +0.0 0.27 ± 3% perf-profile.children.cycles-pp.rcu_idle_exit
0.38 +0.0 0.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm
0.49 ± 2% +0.0 0.51 perf-profile.children.cycles-pp.update_rq_clock
0.23 ± 3% +0.0 0.26 ± 3% perf-profile.children.cycles-pp.common_file_perm
0.57 +0.0 0.60 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.45 ± 2% +0.0 0.48 ± 2% perf-profile.children.cycles-pp.stack_trace_consume_entry_nosched
0.14 ± 13% +0.0 0.17 ± 7% perf-profile.children.cycles-pp.clockevents_program_event
0.70 ± 2% +0.0 0.73 perf-profile.children.cycles-pp.update_curr
0.44 ± 4% +0.0 0.48 ± 2% perf-profile.children.cycles-pp.copy_page_to_iter
0.76 +0.0 0.80 perf-profile.children.cycles-pp.tick_nohz_next_event
1.00 ± 2% +0.0 1.04 perf-profile.children.cycles-pp.set_next_entity
0.86 +0.0 0.90 perf-profile.children.cycles-pp.entry_SYSCALL_64
0.94 +0.0 0.98 perf-profile.children.cycles-pp.syscall_return_via_sysret
0.33 ± 2% +0.0 0.38 ± 3% perf-profile.children.cycles-pp.security_file_permission
0.00 +0.1 0.05 perf-profile.children.cycles-pp.tick_nohz_tick_stopped
1.45 ± 2% +0.1 1.51 perf-profile.children.cycles-pp.pick_next_task_fair
0.36 ± 8% +0.1 0.42 ± 6% perf-profile.children.cycles-pp.ktime_get
0.00 +0.1 0.06 perf-profile.children.cycles-pp.__x64_sys_read
1.25 ± 3% +0.1 1.31 perf-profile.children.cycles-pp.update_load_avg
1.56 ± 2% +0.1 1.63 perf-profile.children.cycles-pp.dequeue_entity
2.83 +0.1 2.91 perf-profile.children.cycles-pp.arch_stack_walk
1.78 ± 2% +0.1 1.88 perf-profile.children.cycles-pp.dequeue_task_fair
2.45 +0.1 2.57 perf-profile.children.cycles-pp.schedule_idle
3.33 +0.1 3.47 perf-profile.children.cycles-pp.schedule
5.94 ± 2% +0.2 6.14 perf-profile.children.cycles-pp.new_sync_read
5.85 +0.2 6.06 perf-profile.children.cycles-pp.pipe_read
6.58 ± 2% +0.2 6.80 perf-profile.children.cycles-pp.ksys_read
6.39 ± 2% +0.2 6.63 perf-profile.children.cycles-pp.vfs_read
5.70 +0.2 5.95 perf-profile.children.cycles-pp.__sched_text_start
30.79 -0.6 30.14 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath
0.09 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.__fsnotify_parent
0.09 -0.0 0.08 perf-profile.self.cycles-pp.ksys_read
0.08 ± 5% +0.0 0.09 perf-profile.self.cycles-pp.in_sched_functions
0.15 ± 2% +0.0 0.17 ± 7% perf-profile.self.cycles-pp.pipe_write
0.21 ± 2% +0.0 0.23 ± 3% perf-profile.self.cycles-pp._find_next_bit
0.20 ± 2% +0.0 0.22 ± 3% perf-profile.self.cycles-pp.common_file_perm
0.38 +0.0 0.40 ± 2% perf-profile.self.cycles-pp.__switch_to_asm
0.04 ± 57% +0.0 0.06 perf-profile.self.cycles-pp.apparmor_file_permission
0.30 ± 2% +0.0 0.32 ± 2% perf-profile.self.cycles-pp.stack_trace_consume_entry_nosched
0.14 ± 3% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.vfs_read
0.18 ± 2% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.__account_scheduler_latency
1.57 +0.0 1.61 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
1.15 +0.0 1.18 perf-profile.self.cycles-pp.__sched_text_start
0.84 +0.0 0.88 perf-profile.self.cycles-pp.entry_SYSCALL_64
0.94 +0.0 0.98 perf-profile.self.cycles-pp.syscall_return_via_sysret
0.00 +0.1 0.05 perf-profile.self.cycles-pp.ksys_write
0.00 +0.1 0.05 perf-profile.self.cycles-pp.rcu_idle_exit
0.18 ± 13% +0.1 0.23 ± 10% perf-profile.self.cycles-pp.ktime_get
0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.__x64_sys_read
1.59 ± 2% +0.1 1.66 perf-profile.self.cycles-pp.do_syscall_64
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months
[ext4] b1b4705d54: filebench.sum_bytes_mb/s -20.2% regression
by kernel test robot
Greeting,
FYI, we noticed a -20.2% regression of filebench.sum_bytes_mb/s due to commit:
commit: b1b4705d54abedfd69dcdf42779c521aa1e0fbd3 ("ext4: introduce direct I/O read using iomap infrastructure")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: filebench
on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory
with following parameters:
disk: 1HDD
fs: ext4
test: fivestreamreaddirect.f
cpufreq_governor: performance
ucode: 0x27
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/debian-x86_64-2019-11-14.cgz/lkp-hsw-d01/fivestreamreaddirect.f/filebench/0x27
commit:
09edf4d381 ("ext4: introduce new callback for IOMAP_REPORT")
b1b4705d54 ("ext4: introduce direct I/O read using iomap infrastructure")
09edf4d381957b14 b1b4705d54abedfd69dcdf42779
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x
:4 25% 1:4 dmesg.WARNING:stack_recursion
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:da
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:db
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:de
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:df
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ea
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:eb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ec
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ed
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ee
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ef
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fa
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fb
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fc
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fd
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fe
:4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ff
:4 25% 1:4 kmsg.file:libnss_files-#so_fault:filemap_fault_mmap:generic_file_mmap_readpage:simple_readpage
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a07fffffa0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a17fffffa1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a27fffffa2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a37fffffa3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a47fffffa4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a57fffffa5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a67fffffa6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a77fffffa7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff8aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff9aff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a87fffffa8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a97fffffa9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#aa7fffffaaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ab7fffffabff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ac7fffffacff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ad7fffffadff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ae7fffffaeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#af7fffffafff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b07fffffb0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b17fffffb1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b27fffffb2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b37fffffb3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b47fffffb4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b57fffffb5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b67fffffb6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b77fffffb7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff8bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff9bff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b87fffffb8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b97fffffb9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ba7fffffbaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bb7fffffbbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bc7fffffbcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bd7fffffbdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#be7fffffbeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bf7fffffbfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c07fffffc0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c17fffffc1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c27fffffc2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c37fffffc3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c47fffffc4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c57fffffc5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c67fffffc6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c77fffffc7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff8cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff9cff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c87fffffc8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c97fffffc9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ca7fffffcaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cb7fffffcbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cc7fffffccff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cd7fffffcdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ce7fffffceff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cf7fffffcfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d07fffffd0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d17fffffd1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d27fffffd2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d37fffffd3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d47fffffd4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d57fffffd5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d67fffffd6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d77fffffd7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff8dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff9dff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d87fffffd8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d97fffffd9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#da7fffffdaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#db7fffffdbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dc7fffffdcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dd7fffffddff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#de7fffffdeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#df7fffffdfff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e07fffffe0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e17fffffe1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e27fffffe2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e37fffffe3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e47fffffe4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e57fffffe5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e67fffffe6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e77fffffe7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff8eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff9eff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e87fffffe8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e97fffffe9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ea7fffffeaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#eb7fffffebff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ec7fffffecff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ed7fffffedff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ee7fffffeeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ef7fffffefff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f07ffffff0ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f17ffffff1ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f27ffffff2ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f37ffffff3ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f47ffffff4ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f57ffffff5ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f67ffffff6ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f77ffffff7ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff8fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff9fff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f87ffffff8ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f97ffffff9ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fa7ffffffaff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fb7ffffffbff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fc7ffffffcff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fd7ffffffdff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fe7ffffffeff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ff7fffffffff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff80ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff81ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff82ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff83ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff84ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff85ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff86ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff87ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff88ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff89ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff90ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff91ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff92ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff93ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff94ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff95ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff96ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff97ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff98ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff99ff
:4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffffffffff
%stddev %change %stddev
\ | \
58.30 ± 2% -20.2% 46.53 filebench.sum_bytes_mb/s
3503 ± 2% -20.2% 2796 filebench.sum_operations
58.39 ± 2% -20.2% 46.61 filebench.sum_operations/s
58.33 ± 2% -19.9% 46.75 filebench.sum_reads/s
84.34 ± 4% +27.1% 107.17 filebench.sum_time_ms/op
7174922 ± 2% -20.2% 5727328 filebench.time.file_system_inputs
10963 ± 2% -8.5% 10035 filebench.time.voluntary_context_switches
58.37 ± 3% -9.0 49.38 ± 3% mpstat.cpu.all.idle%
41.08 ± 5% +8.2 49.24 ± 2% mpstat.cpu.all.iowait%
3061 ± 6% +6.1% 3246 ± 4% turbostat.Bzy_MHz
5.48 ± 97% +260.7% 19.78 ± 30% turbostat.Pkg%pc2
27.33 ± 32% -52.0% 13.12 ± 19% sched_debug.cpu.nr_uninterruptible.max
-67.67 -60.8% -26.50 sched_debug.cpu.nr_uninterruptible.min
28.48 ± 30% -57.9% 11.98 ± 23% sched_debug.cpu.nr_uninterruptible.stddev
8210 ± 44% +76.1% 14454 ± 6% softirqs.CPU0.RCU
8200 ± 40% +70.2% 13954 ± 5% softirqs.CPU2.RCU
9772 ± 36% +59.6% 15597 ± 15% softirqs.CPU3.RCU
108.33 ± 27% +113.5% 231.25 ± 26% interrupts.CPU1.RES:Rescheduling_interrupts
1032 -23.9% 786.00 ± 30% interrupts.CPU5.CAL:Function_call_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.NMI:Non-maskable_interrupts
229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.PMI:Performance_monitoring_interrupts
457.00 ± 61% -74.2% 117.75 ± 41% interrupts.CPU5.RES:Rescheduling_interrupts
53.33 ± 60% +206.6% 163.50 ± 42% interrupts.CPU7.RES:Rescheduling_interrupts
58.99 ± 3% -14.9% 50.18 ± 3% iostat.cpu.idle
40.36 ± 5% +19.8% 48.35 ± 2% iostat.cpu.iowait
0.18 +386.3% 0.87 ±137% iostat.cpu.user
33012 ± 2% -19.7% 26520 iostat.sda.rkB/s
40.60 ± 6% +16.2% 47.18 ± 4% iostat.sda.w/s
43038 ± 4% +17.1% 50399 ± 2% iostat.sda.wkB/s
58.67 ± 3% -15.6% 49.50 ± 3% vmstat.cpu.id
40.00 ± 5% +19.4% 47.75 ± 2% vmstat.cpu.wa
33010 ± 2% -19.7% 26518 vmstat.io.bi
42837 ± 4% +17.2% 50200 ± 2% vmstat.io.bo
5519500 -15.5% 4664152 vmstat.memory.cache
2183603 +39.2% 3039439 ± 2% vmstat.memory.free
1065500 -19.2% 861375 slabinfo.buffer_head.active_objs
27320 -19.1% 22093 slabinfo.buffer_head.active_slabs
1065521 -19.1% 861671 slabinfo.buffer_head.num_objs
27320 -19.1% 22093 slabinfo.buffer_head.num_slabs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.active_objs
1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.num_objs
28277 -9.3% 25651 slabinfo.radix_tree_node.active_objs
5388173 -15.3% 4561843 meminfo.Cached
989375 ± 2% -22.6% 765315 ± 3% meminfo.Dirty
4310025 -19.2% 3483318 meminfo.Inactive
4261041 -19.4% 3434223 meminfo.Inactive(file)
162676 -13.7% 140448 meminfo.KReclaimable
2151883 ± 2% +39.5% 3000927 ± 2% meminfo.MemFree
5902816 -14.4% 5053772 meminfo.Memused
162676 -13.7% 140448 meminfo.SReclaimable
197794 -11.3% 175489 meminfo.Slab
80314276 ± 5% +6.5% 85566891 ± 3% perf-stat.i.branch-instructions
4810221 ± 5% +7.5% 5172535 perf-stat.i.cache-references
7.022e+08 ± 3% +6.9% 7.509e+08 ± 2% perf-stat.i.cpu-cycles
3.891e+08 ± 5% +6.3% 4.135e+08 ± 3% perf-stat.i.instructions
195094 ± 2% +10.8% 216142 perf-stat.i.node-loads
79563934 ± 5% +6.5% 84757473 ± 3% perf-stat.ps.branch-instructions
4765201 ± 5% +7.5% 5123552 perf-stat.ps.cache-references
6.956e+08 ± 3% +6.9% 7.438e+08 ± 2% perf-stat.ps.cpu-cycles
3.855e+08 ± 5% +6.3% 4.096e+08 ± 3% perf-stat.ps.instructions
193276 ± 2% +10.8% 214098 perf-stat.ps.node-loads
4.114e+10 ± 5% +5.3% 4.333e+10 ± 3% perf-stat.total.instructions
247900 ± 3% -22.9% 191187 ± 3% proc-vmstat.nr_dirty
1349353 -15.5% 1140545 proc-vmstat.nr_file_pages
535810 ± 2% +40.1% 750444 ± 2% proc-vmstat.nr_free_pages
1067252 -19.6% 858356 proc-vmstat.nr_inactive_file
40723 -13.8% 35112 proc-vmstat.nr_slab_reclaimable
1110681 ± 2% +17.1% 1300126 proc-vmstat.nr_written
1067251 -19.6% 858384 proc-vmstat.nr_zone_inactive_file
249604 ± 3% -22.7% 193058 ± 3% proc-vmstat.nr_zone_write_pending
660225 -3.8% 635276 proc-vmstat.pgalloc_dma32
871298 +2.7% 894930 proc-vmstat.pgalloc_normal
3587465 ± 2% -20.2% 2863670 proc-vmstat.pgpgin
4645372 ± 4% +16.6% 5414779 proc-vmstat.pgpgout
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.calltrace.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlinkat
13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat
13.02 ± 14% -6.5 6.48 ± 52% perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat.do_syscall_64
10.39 ± 36% -4.6 5.78 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel
11.21 ± 33% -4.6 6.64 ± 7% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64
5.71 ± 17% -2.8 2.95 ± 54% perf-profile.calltrace.cycles-pp.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
3.68 ± 14% -2.0 1.69 ± 50% perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict
3.70 ± 14% -2.0 1.74 ± 48% perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.90 ± 23% -1.6 1.32 ± 56% perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict
2.42 ± 20% -1.4 1.01 ± 75% perf-profile.calltrace.cycles-pp.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat
2.02 ± 17% -1.3 0.68 ±104% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode
1.33 ± 28% -0.9 0.45 ±100% perf-profile.calltrace.cycles-pp.free_buffer_head.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode
1.21 ± 22% -0.8 0.40 ±106% perf-profile.calltrace.cycles-pp.xas_store.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict
1.17 ± 3% +0.4 1.57 ± 13% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle
0.35 ±141% +0.7 1.07 ± 37% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues
3.71 ± 34% +1.6 5.34 ± 14% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter
64.92 ± 5% +9.2 74.17 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
63.80 ± 4% +9.4 73.19 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary
69.95 ± 5% +10.4 80.37 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
13.16 ± 14% -6.6 6.58 ± 52% perf-profile.children.cycles-pp.unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.do_unlinkat
13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.ext4_evict_inode
13.03 ± 14% -6.5 6.49 ± 53% perf-profile.children.cycles-pp.truncate_inode_pages_range
13.07 ± 15% -6.4 6.64 ± 53% perf-profile.children.cycles-pp.evict
11.21 ± 33% -4.6 6.65 ± 7% perf-profile.children.cycles-pp.start_kernel
5.71 ± 17% -2.7 2.99 ± 54% perf-profile.children.cycles-pp.truncate_cleanup_page
3.74 ± 13% -1.9 1.85 ± 49% perf-profile.children.cycles-pp.__pagevec_release
3.75 ± 12% -1.9 1.88 ± 49% perf-profile.children.cycles-pp.release_pages
2.91 ± 22% -1.5 1.40 ± 53% perf-profile.children.cycles-pp.try_to_free_buffers
2.43 ± 19% -1.3 1.12 ± 57% perf-profile.children.cycles-pp.delete_from_page_cache_batch
2.03 ± 17% -1.1 0.95 ± 58% perf-profile.children.cycles-pp.free_unref_page_list
1.34 ± 27% -0.7 0.65 ± 45% perf-profile.children.cycles-pp.free_buffer_head
1.23 ± 20% -0.6 0.64 ± 50% perf-profile.children.cycles-pp.xas_store
0.78 ± 15% -0.4 0.36 ± 52% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head
0.50 ± 39% -0.4 0.15 ± 53% perf-profile.children.cycles-pp.unaccount_page_cache_page
0.42 ± 24% -0.3 0.14 ± 93% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list
0.76 ± 19% -0.2 0.53 ± 6% perf-profile.children.cycles-pp.lapic_next_deadline
0.32 ± 22% -0.2 0.14 ± 69% perf-profile.children.cycles-pp.free_unref_page_commit
0.25 ± 25% -0.2 0.08 ±102% perf-profile.children.cycles-pp.uncharge_page
0.19 ± 47% -0.1 0.07 ± 68% perf-profile.children.cycles-pp.__might_sleep
0.16 ± 28% -0.1 0.05 ±112% perf-profile.children.cycles-pp.__libc_fork
0.15 ± 22% -0.1 0.04 ±102% perf-profile.children.cycles-pp.xas_find
0.13 ± 12% -0.1 0.08 ± 34% perf-profile.children.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.children.cycles-pp.irq_work_needs_cpu
0.02 ±141% +0.1 0.09 ± 29% perf-profile.children.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.cpuidle_not_available
0.09 ± 75% +0.1 0.18 ± 23% perf-profile.children.cycles-pp.__schedule
0.00 +0.1 0.10 ± 40% perf-profile.children.cycles-pp.link_path_walk
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.children.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.children.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.15 ± 33% perf-profile.children.cycles-pp.__test_set_page_writeback
0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.__delete_from_page_cache
0.05 ±141% +0.1 0.17 ± 60% perf-profile.children.cycles-pp.exit_mmap
0.19 ± 54% +0.1 0.32 ± 21% perf-profile.children.cycles-pp.hrtimer_get_next_event
0.05 ±141% +0.1 0.18 ± 64% perf-profile.children.cycles-pp.mmput
0.08 ± 30% +0.2 0.24 ± 33% perf-profile.children.cycles-pp.run_local_timers
0.05 ±141% +0.2 0.21 ± 41% perf-profile.children.cycles-pp.ext4_bio_write_page
0.14 ± 32% +0.2 0.30 ± 46% perf-profile.children.cycles-pp.update_rq_clock
0.15 ± 71% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.rcu_idle_exit
0.38 ± 29% +0.2 0.56 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu
0.14 ± 78% +0.2 0.33 ± 41% perf-profile.children.cycles-pp.mpage_map_and_submit_buffers
0.00 +0.2 0.20 ± 52% perf-profile.children.cycles-pp.invalidate_inode_pages2_range
0.07 ± 97% +0.2 0.30 ± 58% perf-profile.children.cycles-pp.__x64_sys_exit_group
0.07 ±141% +0.2 0.31 ± 37% perf-profile.children.cycles-pp.mpage_submit_page
0.38 ± 21% +0.3 0.64 ± 14% perf-profile.children.cycles-pp.cpuidle_governor_latency_req
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_group_exit
0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_exit
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.do_writepages
0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.ext4_writepages
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_workfn
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_writeback
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.writeback_sb_inodes
0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_single_inode
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.ext4_file_read_iter
0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.iomap_dio_rw
1.30 ± 4% +0.4 1.72 ± 14% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.74 ± 41% +0.5 1.25 ± 33% perf-profile.children.cycles-pp.scheduler_tick
75.40 +4.7 80.05 ± 4% perf-profile.children.cycles-pp.cpuidle_enter
75.35 +4.7 80.02 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state
81.28 +5.9 87.18 ± 4% perf-profile.children.cycles-pp.do_idle
70.02 ± 5% +10.4 80.46 ± 5% perf-profile.children.cycles-pp.start_secondary
1.16 ± 22% -0.6 0.56 ± 63% perf-profile.self.cycles-pp.free_pcppages_bulk
0.73 ± 16% -0.4 0.33 ± 52% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head
0.46 ± 21% -0.3 0.16 ± 36% perf-profile.self.cycles-pp.xas_store
0.70 ± 5% -0.3 0.45 ± 38% perf-profile.self.cycles-pp.release_pages
0.29 ± 17% -0.2 0.07 ± 66% perf-profile.self.cycles-pp.free_unref_page_commit
0.32 ± 26% -0.2 0.13 ± 86% perf-profile.self.cycles-pp.workingset_update_node
0.18 ± 14% -0.1 0.06 ±100% perf-profile.self.cycles-pp.tick_nohz_tick_stopped
0.11 -0.1 0.05 ± 60% perf-profile.self.cycles-pp.leave_mm
0.04 ± 71% +0.0 0.07 ± 14% perf-profile.self.cycles-pp.irq_work_needs_cpu
0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.__msecs_to_jiffies
0.04 ± 71% +0.1 0.11 ± 26% perf-profile.self.cycles-pp.cpuidle_not_available
0.08 ± 81% +0.1 0.18 ± 23% perf-profile.self.cycles-pp.rcu_idle_exit
0.09 ± 41% +0.1 0.20 ± 19% perf-profile.self.cycles-pp.pm_qos_read_value
0.04 ± 71% +0.1 0.14 ± 52% perf-profile.self.cycles-pp.raise_softirq
0.04 ±141% +0.1 0.17 ± 41% perf-profile.self.cycles-pp.calc_global_load_tick
0.16 ± 12% +0.2 0.32 ± 27% perf-profile.self.cycles-pp.run_timer_softirq
0.08 ± 81% +0.2 0.26 ± 46% perf-profile.self.cycles-pp.update_rq_clock
0.17 ± 49% +0.2 0.36 ± 22% perf-profile.self.cycles-pp.load_balance
0.26 ± 8% +0.2 0.47 ± 40% perf-profile.self.cycles-pp.hrtimer_interrupt
filebench.time.file_system_inputs
7.6e+06 +-+---------------------------------------------------------------+
7.4e+06 +-++ .+ +.+ |
|: ++ + +.+.+.+.++. .+ : : +.+.+. +.|
7.2e+06 +-+ : : + + .++ + : : .+ + + + |
7e+06 +-+ : : + + + +: +.+ + :+ + |
6.8e+06 +-+ + + + + +.+ : |
6.6e+06 +-+ +: |
| + |
6.4e+06 +-+ |
6.2e+06 +-+ |
6e+06 +-+ |
5.8e+06 O-+ O O O O |
| O OO O O O O O OO O O O OO O O OO O O O O |
5.6e+06 +-+ O O |
5.4e+06 +-+---------------------------------------------------------------+
filebench.sum_operations
3700 +-+------------------------------------------------------------------+
3600 +-++ .+. +.+ |
|: +. + +.+.+.+.++. .+ : : +.+.+. +.|
3500 +-+ + : + + .+.+ + : : : + + : |
3400 +-+ :: + + :+ : :.+.+ + + + : |
| + + + + + +.+ + |
3300 +-+ :+ |
3200 +-+ + |
3100 +-+ |
| |
3000 +-+ |
2900 +-+ |
O O O O O O O O |
2800 +-O O OO O O O O O O O O O O O OO O O O O |
2700 +-+---------------------------------------------O--------------------+
filebench.sum_operations_s
62 +-+--------------------------------------------------------------------+
| + .+ .+ |
60 +-++ + +.+.+.+.+. .+ + : +. .+. +.|
58 +-+ +.+ : +.+ + .+.+ + : : + + + + : |
|: + : + + :+ : :.+.+ + :+ : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O OO O O O O O O O O O |
| O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_reads_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ .++. .+. +.+ +. .+. .|
58 +-+ +.+ + +.+ +.+.+.+. .+.+ + : : + + + + + |
|: + + + + :+ : : .+ + :+ + |
56 +-+ + + + +.+ + +.+ + |
| : : |
54 +-+ : : |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ |
O O O O O O O O O O |
46 +-O-O-O-O---OO-O-----O---O-O-O-O-O-OO-O-O-O-O-----O--------------------+
filebench.sum_bytes_mb_s
62 +-+--------------------------------------------------------------------+
| + |
60 +-++ +.++.+.+.+.+. +.+ +. .+ +.|
58 +-+ +.+ : +.+.+. .+. : : + + + : |
|: + : + +. +. : : .+.+ +. +. : |
56 +-+ + + + + + +.+ + |
54 +-+ + + |
| + |
52 +-+ |
50 +-+ |
| |
48 +-+ O |
46 O-O O O O O O O O O O O O O O O O O O O O O O O O |
| O O O O |
44 +-+--------------------------------------------------------------------+
filebench.sum_time_ms_op
110 +-+-O------O----------------------O--------------O--------------------+
| O O OO O O O O O OO O O O O O O O OO O O O O |
105 O-+ O |
100 +-+ |
| |
95 +-+ |
| .+. |
90 +-+ + .+. .+ + .+. .++ + |
|: : : + .+.+.+ + + : +.+. + + + |
85 +-+ +. : +. +.+ :+ +. : +.+. + +.|
80 +-++ + +. .+. + : : + + |
| + + + : : |
75 +-+ : : |
| : |
70 +-+-------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 1 week
[f2fs] fe1897eaa6: aim7.jobs-per-min -60.9% regression
by kernel test robot
Greeting,
FYI, we noticed a -60.9% regression of aim7.jobs-per-min due to commit:
commit: fe1897eaa6646f5a64a4cee0e6473ed9887d324b ("f2fs: fix to update time in lazytime mode")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: aim7
on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory
with following parameters:
disk: 4BRD_12G
md: RAID1
fs: f2fs
test: sync_disk_rw
load: 200
cpufreq_governor: performance
ucode: 0x42e
test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system.
test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode:
gcc-7/performance/4BRD_12G/f2fs/x86_64-rhel-7.6/200/RAID1/debian-x86_64-2019-11-14.cgz/lkp-ivb-2ep1/sync_disk_rw/aim7/0x42e
commit:
b145b0eb20 (" ARM and x86 bugfixes of all kinds. The most visible one is that migrating")
fe1897eaa6 ("f2fs: fix to update time in lazytime mode")
b145b0eb2031a620 fe1897eaa6646f5a64a4cee0e64
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
1208 -60.9% 473.12 aim7.jobs-per-min
992.70 +155.6% 2537 aim7.time.elapsed_time
992.70 +155.6% 2537 aim7.time.elapsed_time.max
1.078e+08 +60.0% 1.724e+08 aim7.time.file_system_outputs
9646301 -4.2% 9238717 aim7.time.involuntary_context_switches
24107 ± 3% +58.2% 38130 ± 17% aim7.time.minor_page_faults
3295 +202.3% 9963 aim7.time.system_time
26.75 +51.4% 40.49 aim7.time.user_time
5.02e+08 +210.5% 1.559e+09 aim7.time.voluntary_context_switches
9294452 ± 4% -15.7% 7835497 ± 16% numa-numastat.node0.local_node
9298417 ± 4% -15.7% 7840991 ± 16% numa-numastat.node0.numa_hit
3.66 ± 3% +4.2 7.82 ± 18% mpstat.cpu.all.idle%
0.01 ± 32% -0.0 0.00 ± 31% mpstat.cpu.all.soft%
0.06 -0.0 0.04 mpstat.cpu.all.usr%
3.85 ± 3% +105.0% 7.89 ± 17% iostat.cpu.idle
89.80 -4.9% 85.36 iostat.cpu.iowait
6.29 +6.6% 6.70 iostat.cpu.system
23968 -48.4% 12356 iostat.md0.w/s
53780 -37.1% 33824 iostat.md0.wkB/s
89.00 -4.8% 84.75 vmstat.cpu.wa
53779 -37.1% 33833 vmstat.io.bo
29311046 -28.6% 20935254 vmstat.memory.free
1026388 +19.4% 1225080 vmstat.system.cs
103289 +8.5% 112038 vmstat.system.in
24013 ± 29% -58.5% 9976 ±120% numa-meminfo.node0.Dirty
119019 ± 10% +44.6% 172139 ± 13% numa-meminfo.node1.Active
3503 ± 56% +464.8% 19786 ± 38% numa-meminfo.node1.Active(file)
40477 ± 16% -38.5% 24896 ± 47% numa-meminfo.node1.Dirty
19966008 ± 2% -29.5% 14084594 ± 44% numa-meminfo.node1.MemFree
13014643 ± 4% +45.2% 18896052 ± 32% numa-meminfo.node1.MemUsed
12018 ± 41% +184.8% 34228 ± 35% numa-meminfo.node1.Shmem
2.09e+10 +127.4% 4.752e+10 ± 4% cpuidle.C1.time
2.558e+08 +140.1% 6.143e+08 ± 4% cpuidle.C1.usage
3.017e+09 ± 10% +301.2% 1.211e+10 ± 9% cpuidle.C1E.time
18225961 ± 9% +430.1% 96624314 ± 6% cpuidle.C1E.usage
4.627e+08 ± 14% +147.5% 1.145e+09 ± 20% cpuidle.C6.time
802166 ± 17% +186.9% 2301023 ± 26% cpuidle.C6.usage
1.799e+10 +148.9% 4.476e+10 cpuidle.POLL.time
2.616e+08 ± 2% +246.4% 9.063e+08 ± 2% cpuidle.POLL.usage
6017 ± 29% -58.5% 2496 ±120% numa-vmstat.node0.nr_dirty
5742 ± 29% -65.0% 2011 ±120% numa-vmstat.node0.nr_zone_write_pending
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_active_file
10139 ± 16% -38.6% 6224 ± 47% numa-vmstat.node1.nr_dirty
4992144 ± 2% -29.5% 3521200 ± 44% numa-vmstat.node1.nr_free_pages
3003 ± 41% +184.9% 8556 ± 35% numa-vmstat.node1.nr_shmem
874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_zone_active_file
9775 ± 16% -47.9% 5089 ± 47% numa-vmstat.node1.nr_zone_write_pending
2.558e+08 +140.1% 6.143e+08 ± 4% turbostat.C1
43.77 -4.8 38.98 ± 3% turbostat.C1%
18225642 ± 9% +430.2% 96624037 ± 6% turbostat.C1E
6.31 ± 10% +3.6 9.94 ± 9% turbostat.C1E%
795117 ± 17% +188.4% 2292823 ± 26% turbostat.C6
0.38 ± 42% -63.6% 0.14 ± 13% turbostat.CPU%c6
1.028e+08 +176.7% 2.844e+08 turbostat.IRQ
0.23 ± 33% -69.1% 0.07 ± 22% turbostat.Pkg%pc2
275064 +12.3% 308846 ± 4% meminfo.Active
8036 ± 21% +261.7% 29072 ± 52% meminfo.Active(file)
64400 -45.9% 34850 meminfo.Dirty
16481 +10.8% 18264 meminfo.Inactive(anon)
147067 +12.8% 165858 ± 2% meminfo.KReclaimable
29120417 -28.8% 20740075 meminfo.MemAvailable
29335165 -28.6% 20936804 meminfo.MemFree
36511133 +23.0% 44909491 meminfo.Memused
147067 +12.8% 165858 ± 2% meminfo.SReclaimable
36567 ± 6% +37.2% 50173 meminfo.Shmem
51580 -60.8% 20220 meminfo.max_used_kB
4.27 ± 7% -13.9% 3.67 ± 4% perf-stat.i.MPKI
0.71 ± 3% -0.0 0.67 perf-stat.i.branch-miss-rate%
17.93 +3.5 21.41 ± 2% perf-stat.i.cache-miss-rate%
1029337 +19.1% 1226025 perf-stat.i.context-switches
11421 ± 4% +41.1% 16120 ± 6% perf-stat.i.cpu-migrations
0.42 ± 2% +0.0 0.45 ± 6% perf-stat.i.dTLB-load-miss-rate%
0.11 ± 27% -0.0 0.08 ± 3% perf-stat.i.dTLB-store-miss-rate%
84.04 ± 2% -2.3 81.75 ± 2% perf-stat.i.iTLB-load-miss-rate%
7375571 +7.3% 7917282 perf-stat.i.iTLB-load-misses
3.91 ± 7% -9.2% 3.56 ± 4% perf-stat.overall.MPKI
17.90 +4.1 21.96 ± 3% perf-stat.overall.cache-miss-rate%
0.41 ± 2% +0.0 0.45 ± 6% perf-stat.overall.dTLB-load-miss-rate%
0.10 ± 27% -0.0 0.08 ± 3% perf-stat.overall.dTLB-store-miss-rate%
83.99 ± 2% -2.3 81.72 ± 2% perf-stat.overall.iTLB-load-miss-rate%
1028292 +19.2% 1225532 perf-stat.ps.context-switches
11410 ± 4% +41.2% 16114 ± 6% perf-stat.ps.cpu-migrations
7368087 +7.4% 7914119 perf-stat.ps.iTLB-load-misses
2.558e+13 ± 2% +157.0% 6.575e+13 ± 3% perf-stat.total.instructions
9128 ± 2% -25.3% 6821 ± 4% slabinfo.dmaengine-unmap-16.active_objs
9175 ± 2% -24.4% 6938 ± 4% slabinfo.dmaengine-unmap-16.num_objs
28309 +12.5% 31840 slabinfo.f2fs_extent_node.active_objs
28704 +13.0% 32429 slabinfo.f2fs_extent_node.num_objs
964.00 ± 5% -13.3% 836.00 ± 6% slabinfo.kmem_cache_node.active_objs
1008 ± 5% -12.7% 880.00 ± 6% slabinfo.kmem_cache_node.num_objs
13017 -33.1% 8707 slabinfo.nat_entry.active_objs
13036 -33.2% 8707 slabinfo.nat_entry.num_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.active_objs
433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.num_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.active_objs
697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.num_objs
153098 +22.3% 187296 slabinfo.radix_tree_node.active_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.active_slabs
153098 +22.3% 187296 slabinfo.radix_tree_node.num_objs
2733 +22.3% 3344 slabinfo.radix_tree_node.num_slabs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.active_objs
542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.num_objs
66757 +4.8% 69942 proc-vmstat.nr_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_active_file
96.00 +3.9% 99.75 proc-vmstat.nr_anon_transparent_hugepages
13466177 +60.0% 21542226 proc-vmstat.nr_dirtied
16154 -46.0% 8719 proc-vmstat.nr_dirty
726643 -28.8% 517541 proc-vmstat.nr_dirty_background_threshold
1455064 -28.8% 1036348 proc-vmstat.nr_dirty_threshold
311402 +2.4% 318957 proc-vmstat.nr_file_pages
7332853 -28.6% 5234441 proc-vmstat.nr_free_pages
4120 +10.8% 4565 proc-vmstat.nr_inactive_anon
13939 -1.2% 13772 proc-vmstat.nr_kernel_stack
2845 -1.5% 2803 proc-vmstat.nr_page_table_pages
9142 ± 6% +37.2% 12542 proc-vmstat.nr_shmem
36767 +12.8% 41463 ± 2% proc-vmstat.nr_slab_reclaimable
13387539 +60.3% 21463781 proc-vmstat.nr_written
66757 +4.8% 69942 proc-vmstat.nr_zone_active_anon
2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_zone_active_file
4120 +10.8% 4565 proc-vmstat.nr_zone_inactive_anon
15494 -54.2% 7089 proc-vmstat.nr_zone_write_pending
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_foreign
13506 ± 3% +114.5% 28974 ± 31% proc-vmstat.numa_hint_faults
8913 ± 6% +148.6% 22154 ± 35% proc-vmstat.numa_hint_faults_local
582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_miss
597885 ± 99% +611.6% 4254585 ± 6% proc-vmstat.numa_other
1014174 ± 20% -46.1% 547135 ± 57% proc-vmstat.pgalloc_dma32
16157005 +22.4% 19769138 proc-vmstat.pgalloc_normal
2516271 +152.8% 6360808 proc-vmstat.pgfault
4866589 +64.4% 8002519 proc-vmstat.pgfree
53553588 +60.3% 85856803 proc-vmstat.pgpgout
1.55 ± 4% +0.1 1.65 perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback
1.58 ± 4% +0.1 1.69 perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback
1.60 ± 4% +0.1 1.71 perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file
0.75 ± 2% +0.2 0.91 ± 2% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.70 ± 4% -0.3 0.36 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter
0.69 ± 4% -0.3 0.35 ± 7% perf-profile.children.cycles-pp.generic_perform_write
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.58 ± 5% -0.3 0.26 ± 7% perf-profile.children.cycles-pp.file_write_and_wait_range
0.57 ± 6% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.do_writepages
0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.f2fs_write_data_pages
0.64 ± 5% -0.3 0.34 ± 6% perf-profile.children.cycles-pp.f2fs_write_begin
0.57 ± 5% -0.2 0.32 ± 7% perf-profile.children.cycles-pp.f2fs_put_page
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_finish_plug
0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_flush_plug_list
0.30 ± 11% -0.2 0.11 ± 6% perf-profile.children.cycles-pp.raid1_unplug
0.26 -0.1 0.14 ± 12% perf-profile.children.cycles-pp.f2fs_write_cache_pages
0.22 ± 20% -0.1 0.11 ± 41% perf-profile.children.cycles-pp.brd_do_bvec
0.14 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.submit_bio_wait
0.21 ± 10% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.submit_bio
0.23 ± 17% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.worker_thread
0.20 ± 3% -0.1 0.11 ± 14% perf-profile.children.cycles-pp.__write_data_page
0.14 ± 15% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__submit_flush_wait
0.17 ± 12% -0.1 0.09 ± 12% perf-profile.children.cycles-pp.md_make_request
0.16 ± 11% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.md_handle_request
0.15 ± 9% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.raid1_make_request
0.14 ± 17% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.f2fs_issue_flush
0.14 ± 6% -0.1 0.08 ± 19% perf-profile.children.cycles-pp.f2fs_do_write_data_page
0.10 ± 33% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.reschedule_interrupt
0.09 ± 37% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.scheduler_ipi
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.up_read
0.10 ± 4% +0.0 0.12 ± 13% perf-profile.children.cycles-pp.pick_next_task_idle
0.20 ± 2% +0.0 0.23 ± 5% perf-profile.children.cycles-pp.get_next_timer_interrupt
0.16 ± 6% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.irq_exit
0.01 ±173% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.put_prev_task_fair
0.27 ± 3% +0.1 0.33 ± 4% perf-profile.children.cycles-pp.tick_nohz_next_event
0.35 ± 3% +0.1 0.42 ± 3% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length
1.60 ± 4% +0.1 1.72 perf-profile.children.cycles-pp.io_schedule
0.78 ± 2% +0.1 0.92 ± 2% perf-profile.children.cycles-pp.menu_select
0.06 ± 6% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.up_read
0.08 ± 5% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.cpuidle_enter_state
0.35 ± 5% +0.0 0.39 ± 5% perf-profile.self.cycles-pp.f2fs_wait_on_node_pages_writeback
0.35 ± 4% +0.1 0.41 ± 3% perf-profile.self.cycles-pp.menu_select
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.MIN_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.MIN_vruntime.stddev
44053 +192.3% 128774 sched_debug.cfs_rq:/.exec_clock.avg
140530 ± 9% +377.8% 671471 ± 29% sched_debug.cfs_rq:/.exec_clock.max
33108 ± 2% +109.9% 69484 ± 2% sched_debug.cfs_rq:/.exec_clock.min
20291 ± 7% +386.1% 98642 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev
0.25 ± 82% -88.4% 0.03 ±131% sched_debug.cfs_rq:/.load_avg.min
8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.max_vruntime.avg
158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.max_vruntime.max
32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.max_vruntime.stddev
184170 +218.6% 586859 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg
251516 ± 4% +254.5% 891702 ± 11% sched_debug.cfs_rq:/.min_vruntime.max
165614 +131.3% 383025 ± 2% sched_debug.cfs_rq:/.min_vruntime.min
15286 ± 9% +1057.3% 176903 ± 10% sched_debug.cfs_rq:/.min_vruntime.stddev
0.67 ± 6% +45.5% 0.98 ± 9% sched_debug.cfs_rq:/.nr_spread_over.avg
0.74 ± 20% +32.4% 0.98 ± 13% sched_debug.cfs_rq:/.nr_spread_over.stddev
3.43 ± 29% -60.7% 1.35 ±100% sched_debug.cfs_rq:/.removed.load_avg.avg
75.10 ± 34% -84.1% 11.91 ±100% sched_debug.cfs_rq:/.removed.load_avg.max
15.24 ± 23% -75.5% 3.74 ±100% sched_debug.cfs_rq:/.removed.load_avg.stddev
157.58 ± 29% -60.5% 62.19 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.avg
3459 ± 34% -84.0% 552.75 ± 99% sched_debug.cfs_rq:/.removed.runnable_sum.max
700.79 ± 23% -75.4% 172.66 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.stddev
34.81 ± 42% -87.0% 4.53 ±100% sched_debug.cfs_rq:/.removed.util_avg.max
5.89 ± 31% -79.0% 1.23 ±100% sched_debug.cfs_rq:/.removed.util_avg.stddev
59642 ± 58% +506.4% 361657 ± 61% sched_debug.cfs_rq:/.spread0.max
15286 ± 9% +1057.2% 176904 ± 10% sched_debug.cfs_rq:/.spread0.stddev
15.81 ± 15% +28.7% 20.34 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.avg
553.04 ± 15% +42.8% 789.60 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.max
83.37 ± 15% +37.7% 114.78 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev
130721 -17.9% 107295 sched_debug.cpu.avg_idle.avg
32650 ± 7% -16.9% 27145 ± 6% sched_debug.cpu.avg_idle.min
509212 +151.7% 1281639 sched_debug.cpu.clock.avg
509215 +151.7% 1281642 sched_debug.cpu.clock.max
509209 +151.7% 1281636 sched_debug.cpu.clock.min
509212 +151.7% 1281639 sched_debug.cpu.clock_task.avg
509215 +151.7% 1281642 sched_debug.cpu.clock_task.max
509209 +151.7% 1281636 sched_debug.cpu.clock_task.min
480.69 ± 2% +89.2% 909.42 ± 3% sched_debug.cpu.curr->pid.avg
14037 +135.4% 33040 sched_debug.cpu.curr->pid.max
2146 ± 2% +125.6% 4843 sched_debug.cpu.curr->pid.stddev
0.12 ± 5% +10.2% 0.13 ± 3% sched_debug.cpu.nr_running.avg
10174129 +208.2% 31360192 sched_debug.cpu.nr_switches.avg
10849045 +290.1% 42321541 ± 3% sched_debug.cpu.nr_switches.max
564008 ± 15% +1719.4% 10261545 ± 8% sched_debug.cpu.nr_switches.stddev
-87.16 +262.4% -315.83 sched_debug.cpu.nr_uninterruptible.min
66.84 ± 14% +143.1% 162.48 ± 21% sched_debug.cpu.nr_uninterruptible.stddev
10172460 +208.3% 31358619 sched_debug.cpu.sched_count.avg
10847113 +290.1% 42319816 ± 3% sched_debug.cpu.sched_count.max
563945 ± 15% +1719.6% 10261603 ± 8% sched_debug.cpu.sched_count.stddev
4957584 +211.1% 15425344 sched_debug.cpu.sched_goidle.avg
5286410 +291.7% 20709361 ± 3% sched_debug.cpu.sched_goidle.max
272603 ± 14% +1717.5% 4954528 ± 7% sched_debug.cpu.sched_goidle.stddev
5120024 +209.4% 15841794 sched_debug.cpu.ttwu_count.avg
58965381 ± 14% +567.2% 3.934e+08 ± 33% sched_debug.cpu.ttwu_count.max
132694 ± 2% -41.0% 78274 ± 5% sched_debug.cpu.ttwu_count.min
11266172 ± 8% +482.6% 65635267 ± 21% sched_debug.cpu.ttwu_count.stddev
114955 +26.1% 144930 ± 3% sched_debug.cpu.ttwu_local.max
83595 ± 4% -53.0% 39284 ± 30% sched_debug.cpu.ttwu_local.min
5967 ± 16% +460.9% 33468 ± 9% sched_debug.cpu.ttwu_local.stddev
509209 +151.7% 1281636 sched_debug.cpu_clk
506629 +152.5% 1279056 sched_debug.ktime
509627 +151.6% 1282053 sched_debug.sched_clk
18876 +138.7% 45051 ± 4% softirqs.CPU0.RCU
131251 ± 5% +153.2% 332324 ± 8% softirqs.CPU0.SCHED
327888 ± 3% +181.5% 923135 ± 10% softirqs.CPU0.TIMER
19003 ± 4% +121.7% 42132 ± 12% softirqs.CPU1.RCU
125587 ± 4% +130.3% 289267 ± 22% softirqs.CPU1.SCHED
331057 ± 2% +205.9% 1012636 ± 18% softirqs.CPU1.TIMER
19567 ± 5% +137.8% 46538 ± 2% softirqs.CPU10.RCU
131681 +161.3% 344047 softirqs.CPU10.SCHED
318774 +177.5% 884714 ± 8% softirqs.CPU10.TIMER
17876 ± 5% +158.8% 46258 softirqs.CPU11.RCU
128299 ± 4% +167.5% 343196 softirqs.CPU11.SCHED
320847 ± 3% +166.4% 854607 ± 8% softirqs.CPU11.TIMER
19129 ± 4% +121.1% 42292 ± 4% softirqs.CPU12.RCU
135251 +172.0% 367942 softirqs.CPU12.SCHED
310663 +201.8% 937514 ± 24% softirqs.CPU12.TIMER
17993 +143.9% 43882 ± 3% softirqs.CPU13.RCU
137747 ± 4% +157.5% 354715 ± 4% softirqs.CPU13.SCHED
359507 ± 23% +126.2% 813048 ± 13% softirqs.CPU13.TIMER
18173 +141.9% 43961 ± 3% softirqs.CPU14.RCU
134564 +163.8% 355019 ± 4% softirqs.CPU14.SCHED
310599 ± 2% +160.3% 808494 ± 13% softirqs.CPU14.TIMER
18180 +139.4% 43524 ± 3% softirqs.CPU15.RCU
134993 +160.8% 352061 ± 6% softirqs.CPU15.SCHED
308373 +164.7% 816152 ± 12% softirqs.CPU15.TIMER
19918 +144.6% 48716 softirqs.CPU16.RCU
134609 +162.6% 353418 ± 4% softirqs.CPU16.SCHED
307916 +163.0% 809772 ± 13% softirqs.CPU16.TIMER
19672 ± 2% +148.8% 48938 ± 3% softirqs.CPU17.RCU
132926 ± 2% +165.9% 353452 ± 4% softirqs.CPU17.SCHED
314316 +157.0% 807880 ± 13% softirqs.CPU17.TIMER
20021 ± 3% +141.2% 48298 ± 2% softirqs.CPU18.RCU
133347 ± 2% +164.0% 352038 ± 5% softirqs.CPU18.SCHED
312605 +158.6% 808358 ± 13% softirqs.CPU18.TIMER
19458 +139.3% 46570 ± 6% softirqs.CPU19.RCU
134001 +150.1% 335170 ± 14% softirqs.CPU19.SCHED
310521 +171.0% 841639 ± 14% softirqs.CPU19.TIMER
19131 ± 3% +138.3% 45586 ± 3% softirqs.CPU2.RCU
129111 ± 4% +154.4% 328419 ± 9% softirqs.CPU2.SCHED
329064 ± 3% +172.7% 897239 ± 10% softirqs.CPU2.TIMER
19249 ± 2% +145.5% 47257 ± 3% softirqs.CPU20.RCU
132532 +167.8% 354983 ± 3% softirqs.CPU20.SCHED
310659 +163.4% 818268 ± 13% softirqs.CPU20.TIMER
19473 +142.2% 47163 ± 4% softirqs.CPU21.RCU
133856 +166.0% 356005 ± 4% softirqs.CPU21.SCHED
308960 +164.3% 816637 ± 13% softirqs.CPU21.TIMER
19216 ± 2% +146.1% 47292 ± 3% softirqs.CPU22.RCU
133039 +165.4% 353140 ± 4% softirqs.CPU22.SCHED
307621 +166.0% 818260 ± 13% softirqs.CPU22.TIMER
19244 ± 2% +145.9% 47314 ± 2% softirqs.CPU23.RCU
133897 +163.1% 352326 ± 5% softirqs.CPU23.SCHED
310036 ± 2% +161.6% 811061 ± 13% softirqs.CPU23.TIMER
18954 ± 4% +143.9% 46235 ± 4% softirqs.CPU24.RCU
128754 ± 3% +157.8% 331927 ± 7% softirqs.CPU24.SCHED
323118 +169.7% 871609 ± 8% softirqs.CPU24.TIMER
18942 ± 6% +136.0% 44695 ± 3% softirqs.CPU25.RCU
129580 ± 3% +162.2% 339786 ± 3% softirqs.CPU25.SCHED
328907 +168.0% 881566 ± 7% softirqs.CPU25.TIMER
18913 ± 2% +147.2% 46757 ± 2% softirqs.CPU26.RCU
124714 ± 2% +176.4% 344692 softirqs.CPU26.SCHED
328463 ± 4% +170.2% 887463 ± 8% softirqs.CPU26.TIMER
19015 +137.2% 45107 ± 5% softirqs.CPU27.RCU
128551 ± 6% +152.0% 323916 ± 13% softirqs.CPU27.SCHED
325384 ± 2% +193.3% 954266 ± 13% softirqs.CPU27.TIMER
19094 +149.9% 47725 ± 2% softirqs.CPU28.RCU
131563 +161.9% 344579 ± 2% softirqs.CPU28.SCHED
319151 +176.7% 883105 ± 9% softirqs.CPU28.TIMER
19050 ± 4% +145.4% 46756 ± 2% softirqs.CPU29.RCU
124211 ± 6% +176.0% 342763 ± 2% softirqs.CPU29.SCHED
329526 ± 4% +158.9% 853085 ± 8% softirqs.CPU29.TIMER
22721 ± 8% +121.7% 50370 ± 4% softirqs.CPU3.RCU
132516 ± 2% +156.9% 340453 ± 4% softirqs.CPU3.SCHED
323552 ± 2% +174.3% 887612 ± 9% softirqs.CPU3.TIMER
19911 ± 3% +137.4% 47276 softirqs.CPU30.RCU
130778 +161.4% 341879 softirqs.CPU30.SCHED
317264 +181.0% 891407 ± 8% softirqs.CPU30.TIMER
19232 ± 4% +144.4% 47012 softirqs.CPU31.RCU
126109 ± 10% +174.9% 346628 ± 2% softirqs.CPU31.SCHED
329439 ± 4% +169.4% 887674 ± 9% softirqs.CPU31.TIMER
18139 ± 4% +141.4% 43795 ± 4% softirqs.CPU32.RCU
125566 ± 4% +170.9% 340112 ± 4% softirqs.CPU32.SCHED
328833 +184.0% 934027 ± 9% softirqs.CPU32.TIMER
19309 ± 10% +128.3% 44077 ± 3% softirqs.CPU33.RCU
128438 ± 3% +167.7% 343879 ± 2% softirqs.CPU33.SCHED
320949 +172.2% 873631 ± 9% softirqs.CPU33.TIMER
18041 ± 4% +146.1% 44399 ± 2% softirqs.CPU34.RCU
129791 +165.0% 343914 ± 2% softirqs.CPU34.SCHED
319403 ± 2% +177.3% 885711 ± 8% softirqs.CPU34.TIMER
17991 ± 5% +146.2% 44290 ± 2% softirqs.CPU35.RCU
137112 ± 7% +148.6% 340824 ± 3% softirqs.CPU35.SCHED
416552 ± 21% +119.5% 914435 ± 8% softirqs.CPU35.TIMER
20036 ± 4% +138.7% 47823 ± 4% softirqs.CPU36.RCU
138048 ± 3% +165.7% 366854 ± 7% softirqs.CPU36.SCHED
359442 ± 22% +196.1% 1064343 ± 19% softirqs.CPU36.TIMER
19421 ± 5% +153.4% 49203 ± 2% softirqs.CPU37.RCU
135462 +158.7% 350500 ± 6% softirqs.CPU37.SCHED
305531 +166.8% 815287 ± 12% softirqs.CPU37.TIMER
19963 +150.5% 50014 softirqs.CPU38.RCU
132257 +167.5% 353848 ± 5% softirqs.CPU38.SCHED
311979 ± 2% +160.3% 812118 ± 12% softirqs.CPU38.TIMER
20119 ± 2% +143.6% 49011 ± 2% softirqs.CPU39.RCU
135450 +161.9% 354713 ± 4% softirqs.CPU39.SCHED
307797 +164.3% 813377 ± 12% softirqs.CPU39.TIMER
20593 ± 9% +132.1% 47796 ± 5% softirqs.CPU4.RCU
129881 +159.2% 336643 ± 3% softirqs.CPU4.SCHED
321940 +175.0% 885297 ± 8% softirqs.CPU4.TIMER
19874 ± 2% +149.6% 49611 softirqs.CPU40.RCU
134660 +161.9% 352636 ± 5% softirqs.CPU40.SCHED
309870 +160.6% 807374 ± 13% softirqs.CPU40.TIMER
19897 ± 2% +147.2% 49186 ± 2% softirqs.CPU41.RCU
131978 ± 2% +167.4% 352864 ± 5% softirqs.CPU41.SCHED
315198 +157.6% 812013 ± 12% softirqs.CPU41.TIMER
19988 ± 2% +143.9% 48749 ± 2% softirqs.CPU42.RCU
133326 ± 2% +165.6% 354135 ± 4% softirqs.CPU42.SCHED
313042 +159.5% 812298 ± 12% softirqs.CPU42.TIMER
19723 ± 2% +141.9% 47704 ± 4% softirqs.CPU43.RCU
131944 +165.7% 350566 ± 5% softirqs.CPU43.SCHED
314428 +162.9% 826595 ± 13% softirqs.CPU43.TIMER
19824 ± 2% +143.0% 48177 ± 4% softirqs.CPU44.RCU
132047 ± 2% +157.8% 340362 ± 12% softirqs.CPU44.SCHED
312615 ± 2% +165.2% 829094 ± 12% softirqs.CPU44.TIMER
20140 ± 2% +138.6% 48049 ± 7% softirqs.CPU45.RCU
133846 +151.7% 336901 ± 14% softirqs.CPU45.SCHED
309086 ± 2% +170.8% 837094 ± 12% softirqs.CPU45.TIMER
20103 ± 3% +138.8% 48009 ± 3% softirqs.CPU46.RCU
133860 +154.1% 340108 ± 11% softirqs.CPU46.SCHED
311323 +166.1% 828498 ± 13% softirqs.CPU46.TIMER
19628 ± 2% +149.0% 48881 ± 2% softirqs.CPU47.RCU
132835 ± 2% +164.6% 351487 ± 5% softirqs.CPU47.SCHED
309264 +162.8% 812666 ± 13% softirqs.CPU47.TIMER
19773 ± 2% +139.0% 47263 ± 2% softirqs.CPU5.RCU
130891 ± 2% +163.7% 345185 softirqs.CPU5.SCHED
323979 ± 2% +182.2% 914392 ± 8% softirqs.CPU5.TIMER
19904 ± 3% +141.4% 48045 ± 4% softirqs.CPU6.RCU
131836 +160.9% 344009 softirqs.CPU6.SCHED
316150 +181.7% 890614 ± 8% softirqs.CPU6.TIMER
19077 ± 3% +145.1% 46751 ± 3% softirqs.CPU7.RCU
127906 ± 2% +157.6% 329548 ± 5% softirqs.CPU7.SCHED
325087 +181.6% 915494 ± 9% softirqs.CPU7.TIMER
20183 ± 2% +114.9% 43380 ± 10% softirqs.CPU8.RCU
131974 +118.3% 288124 ± 30% softirqs.CPU8.SCHED
319626 +190.1% 927161 ± 13% softirqs.CPU8.TIMER
19292 ± 3% +137.6% 45839 ± 3% softirqs.CPU9.RCU
129715 +158.9% 335788 ± 4% softirqs.CPU9.SCHED
321209 +178.0% 893035 ± 8% softirqs.CPU9.TIMER
931061 +140.9% 2242787 softirqs.RCU
6322319 +160.8% 16487173 softirqs.SCHED
15417545 +169.7% 41581783 ± 8% softirqs.TIMER
853.75 ± 12% +139.9% 2048 ± 7% interrupts.35:PCI-MSI.2621441-edge.eth0-TxRx-0
586.00 ± 14% +130.1% 1348 ± 5% interrupts.36:PCI-MSI.2621442-edge.eth0-TxRx-1
651.00 ± 9% +136.3% 1538 ± 9% interrupts.37:PCI-MSI.2621443-edge.eth0-TxRx-2
565.25 ± 14% +129.7% 1298 interrupts.38:PCI-MSI.2621444-edge.eth0-TxRx-3
591.00 ± 14% +123.8% 1322 ± 2% interrupts.39:PCI-MSI.2621445-edge.eth0-TxRx-4
660.25 ± 13% +111.4% 1396 ± 7% interrupts.40:PCI-MSI.2621446-edge.eth0-TxRx-5
529.50 ± 4% +227.1% 1732 ± 14% interrupts.41:PCI-MSI.2621447-edge.eth0-TxRx-6
585.75 ± 16% +163.2% 1541 ± 10% interrupts.42:PCI-MSI.2621448-edge.eth0-TxRx-7
404968 +145.6% 994665 ± 3% interrupts.CAL:Function_call_interrupts
8396 ± 6% +133.7% 19627 ± 6% interrupts.CPU0.CAL:Function_call_interrupts
1988743 +155.3% 5077644 interrupts.CPU0.LOC:Local_timer_interrupts
8698 ± 5% +121.4% 19255 ± 2% interrupts.CPU1.CAL:Function_call_interrupts
1986989 +155.5% 5077206 interrupts.CPU1.LOC:Local_timer_interrupts
8487 ± 5% +132.3% 19713 ± 8% interrupts.CPU10.CAL:Function_call_interrupts
1988375 +155.4% 5077322 interrupts.CPU10.LOC:Local_timer_interrupts
8770 ± 5% +149.6% 21889 ± 3% interrupts.CPU11.CAL:Function_call_interrupts
1987964 +155.4% 5077484 interrupts.CPU11.LOC:Local_timer_interrupts
78678 ± 8% +541.4% 504605 ±122% interrupts.CPU11.RES:Rescheduling_interrupts
8731 ± 9% +119.5% 19167 ± 6% interrupts.CPU12.CAL:Function_call_interrupts
1988222 +155.4% 5077524 interrupts.CPU12.LOC:Local_timer_interrupts
7960 ± 3% +153.5% 20176 ± 9% interrupts.CPU13.CAL:Function_call_interrupts
1988667 +155.3% 5076359 interrupts.CPU13.LOC:Local_timer_interrupts
8518 ± 6% +155.0% 21723 ± 7% interrupts.CPU14.CAL:Function_call_interrupts
1987350 +155.5% 5077482 interrupts.CPU14.LOC:Local_timer_interrupts
37.00 ± 31% +124.3% 83.00 ± 24% interrupts.CPU14.TLB:TLB_shootdowns
8831 ± 4% +137.4% 20964 ± 10% interrupts.CPU15.CAL:Function_call_interrupts
1987501 +155.5% 5077129 interrupts.CPU15.LOC:Local_timer_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.NMI:Non-maskable_interrupts
3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.PMI:Performance_monitoring_interrupts
23.25 ± 24% +253.8% 82.25 ± 28% interrupts.CPU15.TLB:TLB_shootdowns
8708 ± 8% +132.7% 20268 ± 7% interrupts.CPU16.CAL:Function_call_interrupts
1988418 +155.3% 5076923 interrupts.CPU16.LOC:Local_timer_interrupts
8548 ± 7% +148.3% 21226 ± 7% interrupts.CPU17.CAL:Function_call_interrupts
1988416 +155.3% 5077317 interrupts.CPU17.LOC:Local_timer_interrupts
33.00 ± 18% +147.0% 81.50 ± 21% interrupts.CPU17.TLB:TLB_shootdowns
8456 ± 9% +157.0% 21731 ± 6% interrupts.CPU18.CAL:Function_call_interrupts
1988574 +155.3% 5077430 interrupts.CPU18.LOC:Local_timer_interrupts
30.25 ± 15% +181.8% 85.25 ± 25% interrupts.CPU18.TLB:TLB_shootdowns
8884 ± 6% +144.5% 21720 ± 3% interrupts.CPU19.CAL:Function_call_interrupts
1987747 +155.4% 5077260 interrupts.CPU19.LOC:Local_timer_interrupts
37.00 ± 19% +127.0% 84.00 ± 26% interrupts.CPU19.TLB:TLB_shootdowns
8298 ± 11% +145.3% 20352 ± 9% interrupts.CPU2.CAL:Function_call_interrupts
1988446 +155.4% 5077740 interrupts.CPU2.LOC:Local_timer_interrupts
8594 ± 5% +146.1% 21149 ± 5% interrupts.CPU20.CAL:Function_call_interrupts
1987855 +155.4% 5077226 interrupts.CPU20.LOC:Local_timer_interrupts
8376 ± 8% +151.7% 21084 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
1988171 +155.4% 5077212 interrupts.CPU21.LOC:Local_timer_interrupts
41.25 ± 20% +106.1% 85.00 ± 27% interrupts.CPU21.TLB:TLB_shootdowns
9024 ± 2% +141.1% 21757 ± 3% interrupts.CPU22.CAL:Function_call_interrupts
1987613 +155.4% 5077125 interrupts.CPU22.LOC:Local_timer_interrupts
8095 ± 7% +154.9% 20633 ± 6% interrupts.CPU23.CAL:Function_call_interrupts
1989315 +155.2% 5077451 interrupts.CPU23.LOC:Local_timer_interrupts
853.75 ± 12% +139.9% 2048 ± 7% interrupts.CPU24.35:PCI-MSI.2621441-edge.eth0-TxRx-0
8463 ± 5% +147.0% 20901 ± 7% interrupts.CPU24.CAL:Function_call_interrupts
1988251 +155.4% 5077879 interrupts.CPU24.LOC:Local_timer_interrupts
74848 ± 8% +525.2% 467985 ±128% interrupts.CPU24.RES:Rescheduling_interrupts
586.00 ± 14% +130.1% 1348 ± 5% interrupts.CPU25.36:PCI-MSI.2621442-edge.eth0-TxRx-1
8156 ± 7% +170.0% 22019 ± 2% interrupts.CPU25.CAL:Function_call_interrupts
1988295 +155.3% 5077010 interrupts.CPU25.LOC:Local_timer_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.NMI:Non-maskable_interrupts
930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.PMI:Performance_monitoring_interrupts
72678 ± 7% +575.9% 491267 ±120% interrupts.CPU25.RES:Rescheduling_interrupts
651.00 ± 9% +136.3% 1538 ± 9% interrupts.CPU26.37:PCI-MSI.2621443-edge.eth0-TxRx-2
8538 ± 6% +141.5% 20625 ± 6% interrupts.CPU26.CAL:Function_call_interrupts
1988053 +155.4% 5077342 interrupts.CPU26.LOC:Local_timer_interrupts
75367 ± 8% +522.0% 468794 ±133% interrupts.CPU26.RES:Rescheduling_interrupts
565.25 ± 14% +129.7% 1298 interrupts.CPU27.38:PCI-MSI.2621444-edge.eth0-TxRx-3
8157 ± 5% +132.5% 18969 ± 3% interrupts.CPU27.CAL:Function_call_interrupts
1988437 +155.3% 5077428 interrupts.CPU27.LOC:Local_timer_interrupts
591.00 ± 14% +123.8% 1322 ± 2% interrupts.CPU28.39:PCI-MSI.2621445-edge.eth0-TxRx-4
8181 ± 6% +148.1% 20300 ± 3% interrupts.CPU28.CAL:Function_call_interrupts
1987807 +155.4% 5077738 interrupts.CPU28.LOC:Local_timer_interrupts
71071 ± 7% +581.0% 483987 ±128% interrupts.CPU28.RES:Rescheduling_interrupts
660.25 ± 13% +111.4% 1396 ± 7% interrupts.CPU29.40:PCI-MSI.2621446-edge.eth0-TxRx-5
8178 ± 5% +150.4% 20475 ± 7% interrupts.CPU29.CAL:Function_call_interrupts
1987852 +155.4% 5077382 interrupts.CPU29.LOC:Local_timer_interrupts
70205 ± 8% +585.5% 481286 ±127% interrupts.CPU29.RES:Rescheduling_interrupts
17.75 ± 36% +163.4% 46.75 ± 80% interrupts.CPU29.TLB:TLB_shootdowns
8684 ± 5% +151.7% 21860 ± 4% interrupts.CPU3.CAL:Function_call_interrupts
1988786 +155.3% 5077436 interrupts.CPU3.LOC:Local_timer_interrupts
80407 ± 4% +518.6% 497419 ±122% interrupts.CPU3.RES:Rescheduling_interrupts
529.50 ± 4% +227.1% 1732 ± 14% interrupts.CPU30.41:PCI-MSI.2621447-edge.eth0-TxRx-6
8415 ± 4% +138.0% 20031 ± 3% interrupts.CPU30.CAL:Function_call_interrupts
1988609 +155.3% 5077494 interrupts.CPU30.LOC:Local_timer_interrupts
74875 ± 8% +518.8% 463337 ±134% interrupts.CPU30.RES:Rescheduling_interrupts
585.75 ± 16% +163.2% 1541 ± 10% interrupts.CPU31.42:PCI-MSI.2621448-edge.eth0-TxRx-7
7917 +151.5% 19913 ± 3% interrupts.CPU31.CAL:Function_call_interrupts
1988677 +155.3% 5077502 interrupts.CPU31.LOC:Local_timer_interrupts
67596 ± 3% +596.5% 470844 ±134% interrupts.CPU31.RES:Rescheduling_interrupts
8230 ± 4% +161.9% 21551 ± 8% interrupts.CPU32.CAL:Function_call_interrupts
1988317 +155.4% 5077410 interrupts.CPU32.LOC:Local_timer_interrupts
70449 ± 5% +597.2% 491174 ±127% interrupts.CPU32.RES:Rescheduling_interrupts
8684 ± 6% +137.3% 20610 ± 6% interrupts.CPU33.CAL:Function_call_interrupts
1988440 +155.3% 5077281 interrupts.CPU33.LOC:Local_timer_interrupts
77401 ± 5% +495.8% 461151 ±134% interrupts.CPU33.RES:Rescheduling_interrupts
8412 ± 8% +150.3% 21061 ± 4% interrupts.CPU34.CAL:Function_call_interrupts
1988447 +155.4% 5077546 interrupts.CPU34.LOC:Local_timer_interrupts
75351 ± 8% +537.5% 480376 ±128% interrupts.CPU34.RES:Rescheduling_interrupts
7890 +147.7% 19543 ± 5% interrupts.CPU35.CAL:Function_call_interrupts
1988875 +155.3% 5077717 interrupts.CPU35.LOC:Local_timer_interrupts
70158 ± 2% +577.1% 475011 ±135% interrupts.CPU35.RES:Rescheduling_interrupts
7908 ± 2% +184.9% 22530 ± 3% interrupts.CPU36.CAL:Function_call_interrupts
1988403 +155.4% 5077694 interrupts.CPU36.LOC:Local_timer_interrupts
31.75 ± 20% +178.0% 88.25 ± 28% interrupts.CPU36.TLB:TLB_shootdowns
9078 ± 2% +133.8% 21229 ± 6% interrupts.CPU37.CAL:Function_call_interrupts
1986971 +155.5% 5076871 interrupts.CPU37.LOC:Local_timer_interrupts
34.75 ± 36% +154.7% 88.50 ± 25% interrupts.CPU37.TLB:TLB_shootdowns
8574 ± 7% +131.7% 19864 ± 3% interrupts.CPU38.CAL:Function_call_interrupts
1986780 +155.5% 5076769 interrupts.CPU38.LOC:Local_timer_interrupts
29.25 ± 35% +195.7% 86.50 ± 18% interrupts.CPU38.TLB:TLB_shootdowns
8184 ± 7% +158.0% 21112 ± 4% interrupts.CPU39.CAL:Function_call_interrupts
1986972 +155.5% 5077337 interrupts.CPU39.LOC:Local_timer_interrupts
8368 ± 8% +144.7% 20481 ± 12% interrupts.CPU4.CAL:Function_call_interrupts
1987774 +155.4% 5077506 interrupts.CPU4.LOC:Local_timer_interrupts
8165 ± 3% +163.7% 21528 ± 6% interrupts.CPU40.CAL:Function_call_interrupts
1987659 +155.4% 5077421 interrupts.CPU40.LOC:Local_timer_interrupts
32.00 ± 12% +154.7% 81.50 ± 29% interrupts.CPU40.TLB:TLB_shootdowns
8454 ± 6% +158.4% 21847 ± 8% interrupts.CPU41.CAL:Function_call_interrupts
1987504 +155.5% 5077467 interrupts.CPU41.LOC:Local_timer_interrupts
37.75 ± 45% +122.5% 84.00 ± 17% interrupts.CPU41.TLB:TLB_shootdowns
8529 ± 7% +136.4% 20164 ± 8% interrupts.CPU42.CAL:Function_call_interrupts
1988401 +155.4% 5077407 interrupts.CPU42.LOC:Local_timer_interrupts
32.75 ± 39% +171.8% 89.00 ± 37% interrupts.CPU42.TLB:TLB_shootdowns
8102 ± 5% +150.4% 20283 ± 9% interrupts.CPU43.CAL:Function_call_interrupts
1987414 +155.5% 5077903 interrupts.CPU43.LOC:Local_timer_interrupts
32.50 ± 18% +150.8% 81.50 ± 34% interrupts.CPU43.TLB:TLB_shootdowns
8206 ± 7% +157.3% 21112 ± 8% interrupts.CPU44.CAL:Function_call_interrupts
1987278 +155.5% 5077309 interrupts.CPU44.LOC:Local_timer_interrupts
37.00 ± 25% +119.6% 81.25 ± 24% interrupts.CPU44.TLB:TLB_shootdowns
8467 ± 7% +139.3% 20261 ± 5% interrupts.CPU45.CAL:Function_call_interrupts
1987858 +155.4% 5077234 interrupts.CPU45.LOC:Local_timer_interrupts
38.25 ± 22% +113.1% 81.50 ± 27% interrupts.CPU45.TLB:TLB_shootdowns
7773 ± 4% +162.0% 20367 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
1988584 +155.3% 5077417 interrupts.CPU46.LOC:Local_timer_interrupts
37.50 ± 14% +142.7% 91.00 ± 34% interrupts.CPU46.TLB:TLB_shootdowns
8746 ± 4% +138.7% 20880 ± 8% interrupts.CPU47.CAL:Function_call_interrupts
1988495 +155.3% 5077204 interrupts.CPU47.LOC:Local_timer_interrupts
8722 ± 7% +134.1% 20414 ± 10% interrupts.CPU5.CAL:Function_call_interrupts
1987875 +155.4% 5077418 interrupts.CPU5.LOC:Local_timer_interrupts
8435 ± 8% +143.9% 20570 ± 11% interrupts.CPU6.CAL:Function_call_interrupts
1988389 +155.4% 5077630 interrupts.CPU6.LOC:Local_timer_interrupts
8837 ± 5% +139.1% 21133 ± 9% interrupts.CPU7.CAL:Function_call_interrupts
1987853 +155.4% 5077370 interrupts.CPU7.LOC:Local_timer_interrupts
8852 ± 5% +123.8% 19810 ± 3% interrupts.CPU8.CAL:Function_call_interrupts
1988702 +155.3% 5077569 interrupts.CPU8.LOC:Local_timer_interrupts
8273 ± 6% +151.1% 20771 ± 9% interrupts.CPU9.CAL:Function_call_interrupts
1988033 +155.4% 5077588 interrupts.CPU9.LOC:Local_timer_interrupts
70739 ± 6% +578.9% 480243 ±129% interrupts.CPU9.RES:Rescheduling_interrupts
95428175 +155.4% 2.437e+08 interrupts.LOC:Local_timer_interrupts
144.00 +133.3% 336.00 interrupts.MCP:Machine_check_polls
6773889 ± 8% +482.9% 39483682 ± 7% interrupts.RES:Rescheduling_interrupts
1544 ± 5% +91.5% 2957 ± 11% interrupts.TLB:TLB_shootdowns
aim7.jobs-per-min
1300 +-+------------------------------------------------------------------+
1200 +-+.++.+.+.+.+. +.+.+.+. +.+.+ +.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| + + + |
1100 +-+ |
1000 +-+ |
| |
900 +-+ |
800 +-+ |
700 +-+ |
| |
600 +-+ |
500 +-+ O |
| O O O O O O O |
400 O-O OO O O O O OO O O O OO O O O O OO O O O O |
300 +-+------------------------------------------------------------------+
aim7.time.user_time
55 +-+--------------------------------------------------------------------+
| |
50 +-+ O |
| O |
| |
45 +-+ |
| O O O O O O O O O O |
40 O-O O OO O O O O O O O O O OO O O O O O |
| |
35 +-+ |
| |
| |
30 +-+ |
|.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.++.+. .+.+.+.+.++.+.+.|
25 +-+--------------------------------------------------------------------+
aim7.time.system_time
11000 +-+-----------------------------------------------------------------+
O O OO O O O OO O O OO O O O OO O O O OO O O |
10000 +-+ O OO O O OO O |
9000 +-+ |
| |
8000 +-+ |
| |
7000 +-+ |
| |
6000 +-+ |
5000 +-+ |
| |
4000 +-+ |
| .++. .+. .+.+. .+.+. .++. .+ .+.+.+ .+. .++.+.|
3000 +-+-----------------------------------------------------------------+
aim7.time.elapsed_time
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.elapsed_time.max
3500 +-+------------------------------------------------------------------+
O O OO O O O O OO O O OO O O O O OO O O O O |
3000 +-+ O |
| |
| O O O O O O |
2500 +-+ OO |
| |
2000 +-+ |
| |
1500 +-+ |
| |
| |
1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.|
| |
500 +-+------------------------------------------------------------------+
aim7.time.voluntary_context_switches
1.8e+09 +-+---------------------------------------------------------------+
O O O O OO O OO O O OO O O OO O OO O O |
1.6e+09 +-O O O O O O OO O O O |
| |
1.4e+09 +-+ |
| |
1.2e+09 +-+ |
| |
1e+09 +-+ |
| |
8e+08 +-+ |
| |
6e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
4e+08 +-+---------------------------------------------------------------+
aim7.time.file_system_outputs
1.8e+08 +-+---------------------------------------------------------------+
O OO O O OO O O OO O O OO O O OO O OO O O OO O O OO O O O |
1.7e+08 +-+ |
1.6e+08 +-+ |
| |
1.5e+08 +-+ |
| |
1.4e+08 +-+ |
| |
1.3e+08 +-+ |
1.2e+08 +-+ |
| |
1.1e+08 +-+ |
|.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.|
1e+08 +-+---------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 1 week
[btrfs] 8d47a0d8f7: fio.write_bw_MBps -28.6% regression
by kernel test robot
Greeting,
FYI, we noticed a -28.6% regression of fio.write_bw_MBps due to commit:
commit: 8d47a0d8f7947422dd359ac8e462687f81a7a137 ("btrfs: Do mandatory tree block check before submitting bio")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: fio-basic
on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory
with following parameters:
disk: 2pmem
fs: btrfs
runtime: 200s
nr_task: 50%
time_based: tb
rw: randwrite
bs: 4k
ioengine: libaio
test_size: 100G
cpufreq_governor: performance
ucode: 0x3d
test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user.
test-url: https://github.com/axboe/fio
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode:
4k/gcc-7/performance/2pmem/btrfs/libaio/x86_64-rhel-7.6/50%/debian-x86_64-2018-04-03.cgz/200s/randwrite/lkp-hsw-ep2/100G/fio-basic/tb/0x3d
commit:
ff2ac107fa ("btrfs: tree-checker: Remove comprehensive root owner check")
8d47a0d8f7 ("btrfs: Do mandatory tree block check before submitting bio")
ff2ac107fae2440b 8d47a0d8f7947422dd359ac8e46
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
:4 25% 1:4 dmesg.BUG:unable_to_handle_kernel
:4 25% 1:4 dmesg.Kernel_panic-not_syncing:Fatal_exception_in_interrupt
:4 25% 1:4 dmesg.Oops:#[##]
:4 25% 1:4 dmesg.RIP:cpuidle_enter_state
:4 25% 1:4 dmesg.RIP:native_write_msr
:4 25% 1:4 dmesg.RIP:perf_prepare_sample
1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x
%stddev %change %stddev
\ | \
0.01 ± 21% +0.3 0.28 ± 20% fio.latency_1000ms%
4.14 ± 13% +1.8 5.95 ± 9% fio.latency_10ms%
8.73 ± 15% +3.0 11.68 ± 8% fio.latency_20ms%
1.91 ± 5% -0.4 1.52 ± 3% fio.latency_250ms%
66.81 -5.9 60.90 ± 2% fio.latency_4ms%
0.18 ± 15% +0.1 0.29 ± 16% fio.latency_750us%
1.017e+08 -28.5% 72701525 fio.time.file_system_outputs
109.75 -26.8% 80.33 ± 2% fio.time.percent_of_cpu_this_job_got
197.97 -26.4% 145.79 ± 2% fio.time.system_time
38.66 ± 4% -24.0% 29.39 ± 3% fio.time.user_time
12388015 -28.5% 8858878 fio.time.voluntary_context_switches
12711059 -28.5% 9083954 fio.workload
247.91 -28.6% 176.96 fio.write_bw_MBps
309248 ± 2% +71.7% 531114 ± 4% fio.write_clat_99%_us
17574 +40.0% 24609 fio.write_clat_mean_us
57989 ± 4% +82.2% 105661 fio.write_clat_stddev
63463 -28.6% 45302 fio.write_iops
564.61 +40.2% 791.54 fio.write_slat_mean_us
9375 ± 7% +96.3% 18406 fio.write_slat_stddev
21.25 +8.5% 23.06 ± 9% boot-time.dhcp
93.32 +1.4% 94.58 iostat.cpu.idle
6.43 -18.6% 5.23 ± 2% iostat.cpu.system
6.48 -1.2 5.27 ± 2% mpstat.cpu.all.sys%
0.24 ± 5% -0.1 0.18 ± 2% mpstat.cpu.all.usr%
93.00 +1.1% 94.00 vmstat.cpu.id
591200 -30.0% 413921 vmstat.io.bo
31653776 ± 2% -10.9% 28196588 ± 2% vmstat.memory.cache
11927341 ± 7% +29.8% 15486805 ± 4% vmstat.memory.free
5.00 -26.7% 3.67 ± 12% vmstat.procs.r
635910 ± 2% -27.6% 460384 vmstat.system.cs
7822384 ± 2% -25.0% 5870498 numa-numastat.node0.local_node
7839861 ± 2% -24.8% 5893785 numa-numastat.node0.numa_hit
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node0.numa_miss
242465 ± 44% -84.3% 37964 ± 18% numa-numastat.node0.other_node
7762623 ± 12% -26.8% 5681358 ± 6% numa-numastat.node1.local_node
224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node1.numa_foreign
7768618 ± 12% -26.9% 5681573 ± 6% numa-numastat.node1.numa_hit
9.429e+08 ± 10% -35.4% 6.094e+08 cpuidle.C1.time
53504154 ± 6% -31.8% 36504731 cpuidle.C1.usage
1.093e+09 ± 22% -50.3% 5.435e+08 ± 3% cpuidle.C1E.time
16541925 ± 13% -45.0% 9103775 cpuidle.C1E.usage
5.366e+09 ± 36% +71.7% 9.214e+09 ± 18% cpuidle.C3.time
16163664 ± 33% +57.0% 25379738 ± 5% cpuidle.C3.usage
9697055 ± 16% -35.8% 6225168 ± 19% cpuidle.C6.usage
19313595 ± 17% -16.3% 16166123 cpuidle.POLL.time
10291838 ± 9% -26.8% 7537045 ± 7% meminfo.Active
10034782 ± 9% -27.4% 7280710 ± 7% meminfo.Active(file)
31524905 ± 2% -10.9% 28082249 ± 2% meminfo.Cached
115157 ± 4% +61.1% 185560 ± 3% meminfo.CmaFree
6430720 ± 8% +19.1% 7660885 ± 6% meminfo.DirectMap2M
534230 ± 5% -12.8% 465932 ± 6% meminfo.Dirty
11737322 ± 7% +30.2% 15281189 ± 4% meminfo.MemFree
37604344 ± 2% -9.4% 34060483 meminfo.Memused
806.50 ± 37% +137.4% 1914 ± 8% meminfo.Writeback
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.active_objs
370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.num_objs
15147 ± 5% -8.3% 13897 ± 2% slabinfo.blkdev_ioc.active_objs
17839 ± 6% -11.7% 15745 ± 2% slabinfo.blkdev_ioc.num_objs
12178 ± 4% -7.4% 11275 ± 2% slabinfo.kmalloc-512.num_objs
7794 ± 3% +25.4% 9774 slabinfo.proc_inode_cache.active_objs
8017 ± 2% +23.4% 9895 slabinfo.proc_inode_cache.num_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.active_objs
2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.num_objs
263.75 ± 2% -15.1% 224.00 ± 2% turbostat.Avg_MHz
9.46 ± 2% -1.4 8.07 ± 2% turbostat.Busy%
53502248 ± 6% -31.8% 36503898 turbostat.C1
5.86 ± 8% -2.0 3.82 turbostat.C1%
16540396 ± 13% -45.0% 9102733 turbostat.C1E
6.80 ± 22% -3.4 3.40 ± 4% turbostat.C1E%
16163332 ± 33% +57.0% 25379351 ± 5% turbostat.C3
33.55 ± 36% +24.1 57.61 ± 18% turbostat.C3%
9689445 ± 16% -35.8% 6221033 ± 19% turbostat.C6
18.04 ± 26% +49.1% 26.90 ± 9% turbostat.CPU%c3
53.25 ± 3% +10.8% 59.00 ± 4% turbostat.CoreTmp
4.67 ± 14% +62.7% 7.59 ± 26% turbostat.Pkg%pc2
56.50 +9.7% 62.00 ± 2% turbostat.PkgTmp
11.84 -4.8% 11.28 turbostat.RAMWatt
7232478 ± 10% -37.2% 4543037 ± 22% numa-meminfo.node0.Active
7072564 ± 10% -37.6% 4410135 ± 23% numa-meminfo.node0.Active(file)
308167 ± 6% -24.7% 232152 ± 10% numa-meminfo.node0.Dirty
8784028 ± 13% +21.7% 10686327 ± 2% numa-meminfo.node0.Inactive
8669548 ± 12% +20.7% 10465444 ± 2% numa-meminfo.node0.Inactive(file)
572.50 ± 22% +115.9% 1236 ± 25% numa-meminfo.node0.Writeback
15098140 ± 3% -17.7% 12425670 ± 3% numa-meminfo.node1.FilePages
11599549 ± 7% -22.3% 9018051 ± 4% numa-meminfo.node1.Inactive
105067 ± 92% -99.3% 725.33 ± 51% numa-meminfo.node1.Inactive(anon)
11494482 ± 7% -21.6% 9017325 ± 4% numa-meminfo.node1.Inactive(file)
5504 ± 9% -12.5% 4819 ± 2% numa-meminfo.node1.KernelStack
6709938 ± 11% +45.8% 9781880 ± 5% numa-meminfo.node1.MemFree
18029136 ± 4% -17.0% 14957200 ± 3% numa-meminfo.node1.MemUsed
4427 ± 69% -77.5% 996.00 ± 8% numa-meminfo.node1.PageTables
857660 ± 32% -37.2% 538646 ± 8% numa-meminfo.node1.SUnreclaim
109232 ± 88% -96.1% 4274 ± 10% numa-meminfo.node1.Shmem
986740 ± 35% -40.0% 592452 ± 10% numa-meminfo.node1.Slab
575.21 ± 15% +30.1% 748.58 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max
100.45 ± 15% +28.6% 129.19 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.stddev
830.19 ± 9% +15.7% 960.25 ± 6% sched_debug.cfs_rq:/.util_avg.max
29.97 ± 16% -25.2% 22.43 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.avg
591.17 ± 13% -22.2% 459.83 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max
107.52 ± 13% -23.0% 82.84 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock.stddev
2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock_task.stddev
24.51 ± 7% +21.5% 29.77 ± 8% sched_debug.cpu.cpu_load[0].avg
541.29 ± 15% +38.3% 748.58 ± 2% sched_debug.cpu.cpu_load[0].max
93.97 ± 7% +27.8% 120.09 ± 2% sched_debug.cpu.cpu_load[0].stddev
519.08 ± 6% +40.5% 729.42 ± 4% sched_debug.cpu.cpu_load[1].max
87.45 ± 5% +28.5% 112.39 ± 4% sched_debug.cpu.cpu_load[1].stddev
480.83 ± 7% +39.5% 670.92 ± 13% sched_debug.cpu.cpu_load[2].max
80.92 ± 5% +25.7% 101.75 ± 8% sched_debug.cpu.cpu_load[2].stddev
408.83 ± 4% +54.0% 629.58 ± 19% sched_debug.cpu.cpu_load[3].max
71.62 ± 2% +30.7% 93.61 ± 11% sched_debug.cpu.cpu_load[3].stddev
381.50 ± 8% +66.7% 636.00 ± 18% sched_debug.cpu.cpu_load[4].max
64.40 ± 6% +39.7% 89.96 ± 12% sched_debug.cpu.cpu_load[4].stddev
106.75 ± 20% -88.4% 12.33 ± 10% proc-vmstat.kswapd_high_wmark_hit_quickly
2508660 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_active_file
33315360 -30.4% 23192301 proc-vmstat.nr_dirtied
133447 ± 5% -12.6% 116580 ± 6% proc-vmstat.nr_dirty
7881043 ± 2% -10.9% 7019533 ± 2% proc-vmstat.nr_file_pages
28790 ± 4% +61.1% 46392 ± 3% proc-vmstat.nr_free_cma
2934503 ± 7% +30.2% 3821369 ± 4% proc-vmstat.nr_free_pages
5044168 -3.4% 4872965 proc-vmstat.nr_inactive_file
430282 -6.0% 404589 proc-vmstat.nr_slab_unreclaimable
197.75 ± 38% +136.5% 467.67 ± 10% proc-vmstat.nr_writeback
33243480 -30.4% 23143756 proc-vmstat.nr_written
2508665 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_zone_active_file
5044399 -3.4% 4873026 proc-vmstat.nr_zone_inactive_file
134972 ± 5% -12.4% 118262 ± 5% proc-vmstat.nr_zone_write_pending
15632301 ± 5% -25.8% 11598535 ± 3% proc-vmstat.numa_hit
15608826 ± 5% -25.8% 11575029 ± 3% proc-vmstat.numa_local
2300530 ± 10% -31.1% 1585363 ± 10% proc-vmstat.pgactivate
16955745 -26.3% 12499611 ± 2% proc-vmstat.pgalloc_normal
17488000 -25.9% 12959812 ± 2% proc-vmstat.pgfree
1.33e+08 -30.4% 92575440 proc-vmstat.pgpgout
5728177 ± 6% -78.5% 1232433 ± 25% proc-vmstat.pgscan_kswapd
5727967 ± 6% -78.5% 1232244 ± 25% proc-vmstat.pgsteal_kswapd
32760 -7.6% 30285 proc-vmstat.slabs_scanned
1768211 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_active_file
10410188 ± 7% -32.5% 7023939 ± 7% numa-vmstat.node0.nr_dirtied
76995 ± 6% -24.6% 58035 ± 10% numa-vmstat.node0.nr_dirty
2167457 ± 12% +20.7% 2616147 ± 2% numa-vmstat.node0.nr_inactive_file
142.00 ± 24% +120.4% 313.00 ± 24% numa-vmstat.node0.nr_writeback
10308758 ± 7% -32.6% 6949497 ± 7% numa-vmstat.node0.nr_written
1768216 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_zone_active_file
2167563 ± 12% +20.7% 2616189 ± 2% numa-vmstat.node0.nr_zone_inactive_file
77786 ± 7% -24.3% 58899 ± 10% numa-vmstat.node0.nr_zone_write_pending
5889240 ± 3% -14.3% 5046874 ± 3% numa-vmstat.node0.numa_hit
5837851 ± 4% -14.0% 5023120 ± 3% numa-vmstat.node0.numa_local
68376 ± 48% -96.9% 2149 ± 46% numa-vmstat.node0.numa_miss
119766 ± 42% -78.4% 25904 ± 3% numa-vmstat.node0.numa_other
30.50 ± 38% -100.0% 0.00 numa-vmstat.node0.workingset_nodes
6829126 ± 6% -22.8% 5269898 ± 10% numa-vmstat.node1.nr_dirtied
3774645 ± 3% -17.7% 3106081 ± 3% numa-vmstat.node1.nr_file_pages
28796 ± 4% +61.1% 46405 ± 3% numa-vmstat.node1.nr_free_cma
1677332 ± 11% +45.8% 2445778 ± 5% numa-vmstat.node1.nr_free_pages
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_inactive_anon
2873702 ± 7% -21.6% 2254116 ± 4% numa-vmstat.node1.nr_inactive_file
5506 ± 9% -12.5% 4820 ± 2% numa-vmstat.node1.nr_kernel_stack
29031 ± 85% -88.2% 3436 numa-vmstat.node1.nr_mapped
1106 ± 69% -77.5% 249.00 ± 8% numa-vmstat.node1.nr_page_table_pages
27308 ± 88% -96.1% 1068 ± 10% numa-vmstat.node1.nr_shmem
214418 ± 32% -37.2% 134663 ± 8% numa-vmstat.node1.nr_slab_unreclaimable
75.00 ± 35% +145.3% 184.00 ± 18% numa-vmstat.node1.nr_writeback
6761224 ± 6% -23.1% 5202586 ± 10% numa-vmstat.node1.nr_written
26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_zone_inactive_anon
2873810 ± 7% -21.6% 2254135 ± 4% numa-vmstat.node1.nr_zone_inactive_file
68381 ± 48% -96.9% 2152 ± 46% numa-vmstat.node1.numa_foreign
5035283 ± 15% -26.9% 3679318 ± 7% numa-vmstat.node1.numa_hit
4905249 ± 15% -28.2% 3522229 ± 8% numa-vmstat.node1.numa_local
18.43 ± 6% -28.3% 13.21 ± 3% perf-stat.i.MPKI
2.205e+09 ± 3% +10.5% 2.437e+09 perf-stat.i.branch-instructions
1.83 ± 5% -0.6 1.24 ± 7% perf-stat.i.branch-miss-rate%
37019551 ± 3% -29.4% 26140844 ± 5% perf-stat.i.branch-misses
28932020 ± 6% -31.1% 19933819 ± 3% perf-stat.i.cache-misses
2.03e+08 ± 4% -24.8% 1.526e+08 perf-stat.i.cache-references
645248 ± 2% -27.4% 468301 perf-stat.i.context-switches
1.97 ± 7% -22.1% 1.53 ± 2% perf-stat.i.cpi
1.983e+10 ± 3% -15.1% 1.684e+10 perf-stat.i.cpu-cycles
1.03 ± 5% -0.4 0.64 ± 10% perf-stat.i.dTLB-load-miss-rate%
31879502 ± 6% -37.9% 19807598 ± 9% perf-stat.i.dTLB-load-misses
3.201e+09 ± 3% +18.8% 3.804e+09 perf-stat.i.dTLB-loads
0.13 ± 7% -0.0 0.09 ± 10% perf-stat.i.dTLB-store-miss-rate%
2516181 ± 10% -28.3% 1805272 ± 10% perf-stat.i.dTLB-store-misses
2.143e+09 ± 3% +19.8% 2.568e+09 ± 2% perf-stat.i.dTLB-stores
13781489 ± 3% -28.4% 9866239 ± 3% perf-stat.i.iTLB-loads
1.164e+10 ± 3% +21.6% 1.415e+10 perf-stat.i.instructions
8435 ± 17% +42.5% 12023 ± 14% perf-stat.i.instructions-per-iTLB-miss
0.57 ± 3% +43.6% 0.82 perf-stat.i.ipc
16050361 ± 9% -30.7% 11127721 ± 3% perf-stat.i.node-load-misses
8120610 ± 9% -31.3% 5580996 perf-stat.i.node-loads
2591649 ± 13% -31.1% 1785932 ± 6% perf-stat.i.node-store-misses
1809775 ± 12% -31.2% 1244360 ± 2% perf-stat.i.node-stores
17.45 ± 4% -38.2% 10.79 perf-stat.overall.MPKI
1.68 ± 3% -0.6 1.07 ± 5% perf-stat.overall.branch-miss-rate%
1.70 ± 3% -30.2% 1.19 perf-stat.overall.cpi
689.69 ± 9% +22.6% 845.48 ± 2% perf-stat.overall.cycles-between-cache-misses
0.99 ± 5% -0.5 0.52 ± 10% perf-stat.overall.dTLB-load-miss-rate%
0.12 ± 8% -0.0 0.07 ± 10% perf-stat.overall.dTLB-store-miss-rate%
10.81 ± 14% +2.3 13.09 ± 10% perf-stat.overall.iTLB-load-miss-rate%
7119 ± 14% +35.9% 9679 ± 13% perf-stat.overall.instructions-per-iTLB-miss
0.59 ± 3% +43.1% 0.84 perf-stat.overall.ipc
202912 ± 2% +68.8% 342615 ± 3% perf-stat.overall.path-length
2.195e+09 ± 3% +10.5% 2.425e+09 perf-stat.ps.branch-instructions
36847010 ± 3% -29.4% 26015686 ± 5% perf-stat.ps.branch-misses
28799092 ± 6% -31.1% 19837306 ± 3% perf-stat.ps.cache-misses
2.021e+08 ± 4% -24.8% 1.519e+08 perf-stat.ps.cache-references
642266 ± 2% -27.4% 465968 perf-stat.ps.context-switches
1.974e+10 ± 3% -15.1% 1.676e+10 perf-stat.ps.cpu-cycles
31729472 ± 6% -37.9% 19715594 ± 9% perf-stat.ps.dTLB-load-misses
3.186e+09 ± 3% +18.8% 3.785e+09 perf-stat.ps.dTLB-loads
2504443 ± 10% -28.2% 1797154 ± 10% perf-stat.ps.dTLB-store-misses
2.133e+09 ± 3% +19.8% 2.556e+09 ± 2% perf-stat.ps.dTLB-stores
13717699 ± 3% -28.4% 9816979 ± 3% perf-stat.ps.iTLB-loads
1.159e+10 ± 3% +21.5% 1.408e+10 perf-stat.ps.instructions
15979371 ± 9% -30.7% 11073468 ± 3% perf-stat.ps.node-load-misses
8081205 ± 9% -31.3% 5554114 perf-stat.ps.node-loads
2580198 ± 13% -31.1% 1777169 ± 6% perf-stat.ps.node-store-misses
1801590 ± 12% -31.3% 1238321 ± 2% perf-stat.ps.node-stores
2.579e+12 +20.6% 3.111e+12 ± 2% perf-stat.total.instructions
36149 ± 4% -26.2% 26686 ± 5% softirqs.CPU0.RCU
37255 ± 3% -26.8% 27256 ± 2% softirqs.CPU1.RCU
37834 ± 5% -26.5% 27801 ± 9% softirqs.CPU10.RCU
39075 ± 3% -24.5% 29494 ± 2% softirqs.CPU11.RCU
38137 ± 6% -23.1% 29340 ± 4% softirqs.CPU12.RCU
37480 ± 3% -20.5% 29797 ± 2% softirqs.CPU13.RCU
37690 ± 5% -28.7% 26861 ± 4% softirqs.CPU14.RCU
33863 ± 9% -29.5% 23885 ± 5% softirqs.CPU15.RCU
34306 ± 9% -31.1% 23641 ± 12% softirqs.CPU16.RCU
34374 ± 9% -29.0% 24402 ± 7% softirqs.CPU17.RCU
44013 ± 9% -31.7% 30059 softirqs.CPU18.RCU
43114 ± 8% -29.9% 30219 softirqs.CPU19.RCU
36276 ± 8% -23.3% 27825 ± 5% softirqs.CPU2.RCU
67049 ± 7% +39.1% 93267 ± 18% softirqs.CPU2.TIMER
39721 ± 13% -31.4% 27233 ± 3% softirqs.CPU20.RCU
42340 ± 5% -28.6% 30215 ± 6% softirqs.CPU21.RCU
40896 ± 7% -30.3% 28506 ± 2% softirqs.CPU22.RCU
42422 ± 7% -33.2% 28357 ± 2% softirqs.CPU23.RCU
42330 ± 9% -33.5% 28150 ± 5% softirqs.CPU24.RCU
41406 ± 9% -31.4% 28398 softirqs.CPU25.RCU
39754 ± 5% -29.4% 28063 softirqs.CPU26.RCU
40207 ± 6% -27.9% 28995 ± 12% softirqs.CPU27.RCU
40351 ± 9% -29.4% 28487 ± 3% softirqs.CPU28.RCU
41880 ± 5% -29.6% 29472 softirqs.CPU29.RCU
36293 ± 4% -17.7% 29864 ± 7% softirqs.CPU3.RCU
39341 ± 9% -34.3% 25847 softirqs.CPU30.RCU
39687 ± 9% -33.5% 26400 ± 2% softirqs.CPU31.RCU
38062 ± 14% -39.9% 22878 ± 10% softirqs.CPU32.RCU
39769 ± 13% -33.8% 26314 ± 5% softirqs.CPU33.RCU
39504 ± 10% -37.2% 24824 ± 6% softirqs.CPU34.RCU
39591 ± 11% -38.6% 24300 ± 5% softirqs.CPU35.RCU
40398 ± 8% -27.0% 29488 ± 12% softirqs.CPU36.RCU
39464 ± 12% -25.3% 29478 ± 10% softirqs.CPU37.RCU
41177 ± 9% -34.0% 27190 ± 19% softirqs.CPU38.RCU
38734 ± 13% -24.8% 29145 ± 5% softirqs.CPU39.RCU
36804 ± 5% -23.0% 28354 softirqs.CPU4.RCU
40421 ± 12% -28.8% 28776 ± 6% softirqs.CPU40.RCU
40371 ± 11% -31.1% 27834 ± 8% softirqs.CPU41.RCU
40633 ± 10% -29.9% 28469 ± 10% softirqs.CPU42.RCU
41018 ± 9% -33.6% 27252 ± 6% softirqs.CPU43.RCU
38181 ± 12% -27.0% 27856 ± 10% softirqs.CPU44.RCU
39605 ± 12% -32.3% 26803 ± 12% softirqs.CPU45.RCU
42686 ± 11% -29.7% 30016 ± 9% softirqs.CPU47.RCU
40433 ± 9% -31.8% 27586 ± 11% softirqs.CPU48.RCU
38371 ± 12% -23.0% 29528 ± 7% softirqs.CPU49.RCU
37172 ± 2% -25.9% 27562 ± 3% softirqs.CPU5.RCU
39660 ± 12% -27.5% 28752 ± 11% softirqs.CPU50.RCU
40375 ± 10% -26.7% 29582 ± 3% softirqs.CPU51.RCU
40568 ± 9% -28.8% 28886 ± 6% softirqs.CPU52.RCU
39967 ± 10% -27.3% 29073 ± 11% softirqs.CPU53.RCU
40874 ± 11% -37.1% 25725 ± 6% softirqs.CPU54.RCU
41227 ± 14% -37.9% 25606 ± 6% softirqs.CPU55.RCU
38856 ± 13% -34.7% 25380 ± 8% softirqs.CPU56.RCU
39618 ± 13% -39.1% 24119 ± 4% softirqs.CPU57.RCU
39412 ± 12% -36.6% 24970 ± 3% softirqs.CPU58.RCU
41572 ± 14% -41.3% 24399 ± 10% softirqs.CPU59.RCU
36927 ± 3% -24.4% 27923 ± 22% softirqs.CPU6.RCU
37057 ± 10% -31.8% 25261 ± 2% softirqs.CPU60.RCU
36584 ± 8% -27.8% 26419 ± 2% softirqs.CPU61.RCU
35859 ± 4% -28.2% 25729 ± 5% softirqs.CPU62.RCU
35849 ± 7% -33.2% 23933 ± 5% softirqs.CPU63.RCU
35379 ± 6% -26.4% 26041 ± 4% softirqs.CPU64.RCU
36908 ± 6% -26.5% 27110 ± 2% softirqs.CPU65.RCU
37595 ± 3% -26.1% 27783 ± 4% softirqs.CPU66.RCU
36327 ± 2% -27.0% 26533 ± 2% softirqs.CPU67.RCU
35136 ± 7% -31.7% 23998 ± 9% softirqs.CPU68.RCU
35638 ± 4% -28.6% 25435 ± 4% softirqs.CPU69.RCU
37268 ± 6% -25.4% 27786 ± 4% softirqs.CPU7.RCU
36693 ± 4% -29.2% 25986 ± 5% softirqs.CPU70.RCU
36859 ± 3% -28.8% 26260 ± 4% softirqs.CPU71.RCU
37539 ± 7% -27.7% 27144 ± 4% softirqs.CPU8.RCU
37584 ± 4% -26.0% 27802 ± 15% softirqs.CPU9.RCU
2795313 ± 7% -29.7% 1964363 ± 4% softirqs.RCU
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.41:PCI-MSI.1572870-edge.eth0-TxRx-6
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.42:PCI-MSI.1572871-edge.eth0-TxRx-7
150.50 ± 43% -28.0% 108.33 interrupts.84:PCI-MSI.1572913-edge.eth0-TxRx-49
10393 ± 10% -26.4% 7653 ± 10% interrupts.CPU1.RES:Rescheduling_interrupts
12518 ± 15% -31.0% 8642 ± 21% interrupts.CPU10.RES:Rescheduling_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.NMI:Non-maskable_interrupts
945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.PMI:Performance_monitoring_interrupts
13068 ± 18% -30.6% 9075 ± 23% interrupts.CPU11.RES:Rescheduling_interrupts
11949 ± 11% -49.8% 6001 ± 29% interrupts.CPU12.RES:Rescheduling_interrupts
13711 ± 8% -19.4% 11052 ± 24% interrupts.CPU13.RES:Rescheduling_interrupts
13361 ± 6% -38.8% 8170 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts
7628 ± 12% -29.2% 5401 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts
7909 ± 25% -30.4% 5501 ± 37% interrupts.CPU19.RES:Rescheduling_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts
1482 ± 39% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts
2116 +23.2% 2606 ± 10% interrupts.CPU20.CAL:Function_call_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.NMI:Non-maskable_interrupts
194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.PMI:Performance_monitoring_interrupts
2096 ± 4% +18.0% 2474 ± 9% interrupts.CPU21.CAL:Function_call_interrupts
11313 ± 20% -56.4% 4935 ± 23% interrupts.CPU21.RES:Rescheduling_interrupts
2111 ± 3% +20.1% 2534 ± 11% interrupts.CPU22.CAL:Function_call_interrupts
2109 ± 4% +22.6% 2585 ± 10% interrupts.CPU23.CAL:Function_call_interrupts
10305 ± 16% -40.7% 6115 ± 17% interrupts.CPU23.RES:Rescheduling_interrupts
2128 ± 2% +17.8% 2507 ± 10% interrupts.CPU24.CAL:Function_call_interrupts
12271 ± 20% -39.0% 7488 ± 29% interrupts.CPU24.RES:Rescheduling_interrupts
2123 +19.7% 2541 ± 7% interrupts.CPU25.CAL:Function_call_interrupts
9780 ± 21% -32.9% 6563 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.NMI:Non-maskable_interrupts
980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.PMI:Performance_monitoring_interrupts
11163 ± 28% -49.8% 5605 ± 13% interrupts.CPU27.RES:Rescheduling_interrupts
11071 ± 12% -56.3% 4835 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts
728.50 ± 66% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts
10599 ± 10% -38.7% 6501 ± 16% interrupts.CPU31.RES:Rescheduling_interrupts
2213 ± 4% +14.2% 2527 ± 7% interrupts.CPU32.CAL:Function_call_interrupts
8862 ± 16% -39.6% 5354 ± 28% interrupts.CPU32.RES:Rescheduling_interrupts
2156 ± 6% +20.5% 2599 ± 9% interrupts.CPU33.CAL:Function_call_interrupts
2169 ± 5% +19.9% 2601 ± 9% interrupts.CPU34.CAL:Function_call_interrupts
10483 ± 18% -48.8% 5371 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts
2049 +24.4% 2549 ± 8% interrupts.CPU35.CAL:Function_call_interrupts
10929 ± 15% -49.0% 5578 ± 14% interrupts.CPU35.RES:Rescheduling_interrupts
1993 ± 4% +28.5% 2561 ± 12% interrupts.CPU36.CAL:Function_call_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.NMI:Non-maskable_interrupts
161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.PMI:Performance_monitoring_interrupts
1987 ± 4% +30.5% 2594 ± 13% interrupts.CPU37.CAL:Function_call_interrupts
1863 ± 10% +35.7% 2529 ± 14% interrupts.CPU38.CAL:Function_call_interrupts
1984 ± 6% +26.5% 2509 ± 12% interrupts.CPU41.CAL:Function_call_interrupts
12830 ± 30% -26.4% 9445 ± 9% interrupts.CPU43.RES:Rescheduling_interrupts
2067 ± 2% +27.4% 2632 ± 9% interrupts.CPU44.CAL:Function_call_interrupts
2066 ± 3% +27.2% 2629 ± 9% interrupts.CPU45.CAL:Function_call_interrupts
2158 ± 3% +19.6% 2582 ± 9% interrupts.CPU46.CAL:Function_call_interrupts
2036 ± 6% +30.3% 2652 ± 10% interrupts.CPU47.CAL:Function_call_interrupts
150.50 ± 43% -28.0% 108.33 interrupts.CPU49.84:PCI-MSI.1572913-edge.eth0-TxRx-49
11823 ± 12% -36.9% 7460 ± 18% interrupts.CPU49.RES:Rescheduling_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.NMI:Non-maskable_interrupts
1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.PMI:Performance_monitoring_interrupts
11748 ± 22% -40.0% 7048 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts
2102 ± 5% +19.6% 2515 ± 7% interrupts.CPU52.CAL:Function_call_interrupts
2115 ± 6% +19.5% 2527 ± 7% interrupts.CPU53.CAL:Function_call_interrupts
11318 ± 17% -20.4% 9004 ± 26% interrupts.CPU53.RES:Rescheduling_interrupts
2177 ± 5% +17.4% 2556 ± 9% interrupts.CPU54.CAL:Function_call_interrupts
10968 ± 23% -62.3% 4134 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts
2161 ± 5% +20.2% 2598 ± 9% interrupts.CPU55.CAL:Function_call_interrupts
9334 ± 19% -55.4% 4158 ± 34% interrupts.CPU55.RES:Rescheduling_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.NMI:Non-maskable_interrupts
1443 ± 22% -100.0% 0.00 interrupts.CPU56.PMI:Performance_monitoring_interrupts
11377 ± 31% -56.9% 4907 ± 43% interrupts.CPU56.RES:Rescheduling_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts
601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts
8295 ± 31% -47.7% 4340 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts
9182 ± 37% -59.7% 3701 ± 46% interrupts.CPU58.RES:Rescheduling_interrupts
249.50 ± 76% -50.4% 123.67 ± 2% interrupts.CPU6.41:PCI-MSI.1572870-edge.eth0-TxRx-6
12473 ± 27% -44.4% 6940 ± 13% interrupts.CPU6.RES:Rescheduling_interrupts
7886 ± 12% -42.2% 4555 ± 31% interrupts.CPU60.RES:Rescheduling_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts
971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts
9253 ± 26% -56.4% 4033 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts
2103 ± 5% +19.1% 2505 ± 11% interrupts.CPU63.CAL:Function_call_interrupts
2059 ± 2% +20.6% 2483 ± 10% interrupts.CPU64.CAL:Function_call_interrupts
8545 ± 17% -41.2% 5025 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts
7378 ± 23% -51.0% 3614 ± 30% interrupts.CPU68.RES:Rescheduling_interrupts
8573 ± 28% -50.1% 4277 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts
154.50 ± 13% -10.9% 137.67 ± 10% interrupts.CPU7.42:PCI-MSI.1572871-edge.eth0-TxRx-7
9910 ± 28% -51.2% 4839 ± 40% interrupts.CPU70.RES:Rescheduling_interrupts
8836 ± 26% -52.7% 4180 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.NMI:Non-maskable_interrupts
911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.PMI:Performance_monitoring_interrupts
12326 ± 15% -32.3% 8350 ± 10% interrupts.CPU8.RES:Rescheduling_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts
1392 ± 58% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.NMI:Non-maskable_interrupts
60185 ± 19% -30.5% 41822 ± 11% interrupts.PMI:Performance_monitoring_interrupts
749419 -31.9% 510225 ± 3% interrupts.RES:Rescheduling_interrupts
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread
5.99 ± 8% -1.6 4.37 ± 6% perf-profile.calltrace.cycles-pp.io_submit
5.83 ± 8% -1.6 4.25 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.io_submit
5.82 ± 8% -1.6 4.24 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.calltrace.cycles-pp.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit
5.65 ± 6% -1.6 4.08 ± 20% perf-profile.calltrace.cycles-pp.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes
5.58 ± 8% -1.5 4.07 ± 6% perf-profile.calltrace.cycles-pp.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe
4.50 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64
6.63 ± 8% -1.5 5.17 ± 10% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit
5.55 ± 8% -1.2 4.32 ± 10% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
4.98 ± 8% -1.1 3.86 ± 10% perf-profile.calltrace.cycles-pp.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages
5.00 ± 8% -1.1 3.88 ± 10% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages
2.80 ± 4% -1.0 1.78 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io
1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
2.32 ± 3% -0.6 1.72 ± 18% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.91 ± 8% -0.5 0.42 ± 73% perf-profile.calltrace.cycles-pp.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper
0.88 ± 8% -0.5 0.41 ± 74% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
0.83 ± 5% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums
1.90 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
1.89 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.82 ± 4% -0.4 0.42 ± 71% perf-profile.calltrace.cycles-pp.run_one_async_done.normal_work_helper.process_one_work.worker_thread.kthread
0.81 ± 4% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.normal_work_helper.process_one_work.worker_thread
0.74 ± 6% -0.4 0.36 ± 70% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks
1.19 ± 3% -0.4 0.83 ± 16% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.calltrace.cycles-pp.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.25 ± 6% -0.3 0.90 ± 7% perf-profile.calltrace.cycles-pp.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.93 ± 8% -0.3 0.59 ± 7% perf-profile.calltrace.cycles-pp.__set_extent_bit.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter
0.94 ± 7% -0.3 0.60 ± 6% perf-profile.calltrace.cycles-pp.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.70 ± 8% -0.3 0.37 ± 70% perf-profile.calltrace.cycles-pp.__lookup_extent_mapping.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter
0.79 ± 6% -0.3 0.47 ± 71% perf-profile.calltrace.cycles-pp.setup_items_for_insert.btrfs_duplicate_item.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper
0.71 ± 21% -0.3 0.43 ± 74% perf-profile.calltrace.cycles-pp.__btrfs_free_extent.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space
1.15 ± 11% -0.3 0.88 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.15 ± 11% -0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
1.43 ± 13% -0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one
1.06 ± 5% -0.3 0.81 ± 7% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.30 ± 15% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write
1.03 ± 13% -0.2 0.79 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range
1.20 ± 4% -0.2 0.96 ± 15% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.24 ± 3% -0.2 1.01 ± 15% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
1.05 ± 8% -0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.calltrace.cycles-pp.wait_reserve_ticket.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter
1.01 ± 9% -0.2 0.84 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc
1.01 ± 10% -0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage
0.72 ± 7% -0.2 0.56 ± 6% perf-profile.calltrace.cycles-pp.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write
0.75 ± 8% -0.1 0.60 ± 10% perf-profile.calltrace.cycles-pp.submit_extent_page.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages
0.92 ± 15% +0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt
1.01 ± 15% +0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt
1.15 ± 15% +0.3 1.44 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt
0.27 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt
1.74 ± 9% +0.4 2.11 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state
2.63 ± 10% +0.6 3.26 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle
0.00 +0.8 0.75 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread
0.00 +0.9 0.90 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook
4.71 ± 11% +1.0 5.74 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry
5.04 ± 10% +1.2 6.19 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
0.00 +1.3 1.30 ± 19% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range
0.00 +1.4 1.36 ± 19% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents
0.29 ±101% +1.4 1.67 ± 18% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work
0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space
0.00 +1.5 1.52 ± 18% perf-profile.calltrace.cycles-pp.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
0.00 +2.2 2.20 ± 14% perf-profile.calltrace.cycles-pp.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work.worker_thread
7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work
49.45 ± 2% +4.3 53.79 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64
1.07 ± 18% +5.0 6.06 ± 20% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback
0.93 ± 18% +5.0 5.95 ± 20% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes
61.26 ± 2% +5.1 66.36 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64
61.21 ± 2% +5.1 66.32 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.33 ±100% +5.3 5.68 ± 20% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode
55.45 ± 2% +5.4 60.88 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64
0.00 +6.4 6.39 ± 13% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page
0.00 +6.5 6.50 ± 13% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages
0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages
15.05 ± 6% -4.1 10.97 ± 20% perf-profile.children.cycles-pp.normal_work_helper
12.66 ± 5% -4.0 8.68 ± 20% perf-profile.children.cycles-pp.btrfs_finish_ordered_io
7.84 ± 5% -2.4 5.39 ± 10% perf-profile.children.cycles-pp.btrfs_search_slot
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_writepages
7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_write_cache_pages
7.03 ± 7% -1.6 5.38 ± 12% perf-profile.children.cycles-pp.__extent_writepage
6.19 ± 8% -1.6 4.56 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
6.17 ± 8% -1.6 4.55 ± 7% perf-profile.children.cycles-pp.do_syscall_64
6.01 ± 8% -1.6 4.39 ± 7% perf-profile.children.cycles-pp.io_submit
5.81 ± 8% -1.6 4.23 ± 6% perf-profile.children.cycles-pp.__x64_sys_io_submit
5.76 ± 8% -1.6 4.19 ± 6% perf-profile.children.cycles-pp.io_submit_one
5.66 ± 5% -1.6 4.09 ± 20% perf-profile.children.cycles-pp.btrfs_mark_extent_written
4.52 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.add_pending_csums
5.59 ± 8% -1.5 4.07 ± 6% perf-profile.children.cycles-pp.aio_write
4.51 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.btrfs_csum_file_blocks
5.44 ± 8% -1.5 3.96 ± 7% perf-profile.children.cycles-pp.btrfs_file_write_iter
5.34 ± 9% -1.4 3.90 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write
5.88 ± 7% -1.4 4.47 ± 11% perf-profile.children.cycles-pp.writepage_delalloc
5.30 ± 8% -1.3 4.01 ± 11% perf-profile.children.cycles-pp.btrfs_run_delalloc_range
5.28 ± 8% -1.3 3.99 ± 11% perf-profile.children.cycles-pp.run_delalloc_nocow
2.55 ± 2% -0.9 1.67 ± 16% perf-profile.children.cycles-pp.btrfs_cow_block
2.54 ± 2% -0.9 1.66 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block
2.15 ± 13% -0.8 1.31 ± 8% perf-profile.children.cycles-pp.__lookup_extent_mapping
1.87 ± 12% -0.8 1.04 ± 12% perf-profile.children.cycles-pp.__etree_search
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.printk
0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.vprintk_emit
2.60 ± 2% -0.7 1.92 ± 14% perf-profile.children.cycles-pp.btrfs_insert_empty_items
3.00 ± 5% -0.7 2.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock
1.89 ± 2% -0.7 1.23 ± 7% perf-profile.children.cycles-pp.__set_extent_bit
2.78 ± 4% -0.6 2.14 ± 14% perf-profile.children.cycles-pp.__schedule
1.57 ± 2% -0.5 1.02 ± 9% perf-profile.children.cycles-pp.lock_extent_bits
1.19 ± 14% -0.5 0.70 ± 11% perf-profile.children.cycles-pp.__clear_extent_bit
1.51 ± 7% -0.5 1.04 ± 9% perf-profile.children.cycles-pp.read_block_for_search
2.08 ± 5% -0.5 1.62 ± 18% perf-profile.children.cycles-pp.setup_items_for_insert
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.btrfs_run_delayed_refs
2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs
1.60 ± 6% -0.4 1.20 ± 12% perf-profile.children.cycles-pp.schedule
1.95 ± 11% -0.4 1.58 ± 12% perf-profile.children.cycles-pp.try_to_wake_up
1.31 ± 6% -0.4 0.94 ± 9% perf-profile.children.cycles-pp.create_io_em
0.99 ± 4% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush
1.00 ± 6% -0.4 0.65 ± 7% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need
1.27 ± 9% -0.4 0.92 ± 11% perf-profile.children.cycles-pp.__wake_up_common_lock
0.98 ± 3% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.btrfs_alloc_tree_block
1.27 ± 5% -0.4 0.92 ± 7% perf-profile.children.cycles-pp.btrfs_dirty_pages
1.34 ± 7% -0.3 0.99 ± 10% perf-profile.children.cycles-pp.btrfs_map_bio
1.11 ± 9% -0.3 0.77 ± 11% perf-profile.children.cycles-pp.generic_bin_search
1.16 ± 6% -0.3 0.82 ± 9% perf-profile.children.cycles-pp.find_extent_buffer
0.91 ± 7% -0.3 0.57 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_csum
1.21 ± 11% -0.3 0.90 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_csums_range
1.21 ± 11% -0.3 0.91 ± 9% perf-profile.children.cycles-pp.csum_exist_in_range
1.10 ± 10% -0.3 0.80 ± 9% perf-profile.children.cycles-pp.__wake_up_common
0.37 ± 70% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.test_range_bit
1.61 ± 4% -0.3 1.32 ± 15% perf-profile.children.cycles-pp.push_leaf_right
0.60 ± 4% -0.3 0.31 ± 25% perf-profile.children.cycles-pp.btrfs_extend_item
1.02 ± 9% -0.3 0.73 ± 10% perf-profile.children.cycles-pp.autoremove_wake_function
1.12 ± 4% -0.3 0.84 ± 9% perf-profile.children.cycles-pp.btrfs_drop_extent_cache
0.82 ± 6% -0.3 0.55 ± 4% perf-profile.children.cycles-pp.btrfs_release_path
0.85 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_extent_buffer_full
0.99 ± 4% -0.3 0.73 ± 13% perf-profile.children.cycles-pp.btrfs_set_token_32
1.44 ± 13% -0.3 1.18 ± 5% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata
0.84 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_page
0.87 ± 11% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.pagecache_get_page
0.84 ± 7% -0.2 0.60 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc
1.12 ± 7% -0.2 0.88 ± 17% perf-profile.children.cycles-pp.__extent_writepage_io
1.30 ± 15% -0.2 1.06 ± 5% perf-profile.children.cycles-pp.reserve_metadata_bytes
0.82 ± 4% -0.2 0.58 ± 13% perf-profile.children.cycles-pp.run_one_async_done
1.25 ± 3% -0.2 1.01 ± 15% perf-profile.children.cycles-pp.schedule_idle
0.34 ± 53% -0.2 0.10 ± 28% perf-profile.children.cycles-pp.delay_tsc
0.70 ± 13% -0.2 0.47 ± 11% perf-profile.children.cycles-pp.prepare_pages
0.50 ± 22% -0.2 0.28 ± 13% perf-profile.children.cycles-pp.clear_state_bit
0.84 ± 5% -0.2 0.63 ± 6% perf-profile.children.cycles-pp.btrfs_get_extent
0.54 ± 11% -0.2 0.34 ± 4% perf-profile.children.cycles-pp.btrfs_free_path
1.16 ± 11% -0.2 0.96 ± 19% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath
0.53 ± 9% -0.2 0.33 ± 18% perf-profile.children.cycles-pp.unpin_extent_cache
1.07 ± 10% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_file_extent
0.50 ± 4% -0.2 0.31 ± 15% perf-profile.children.cycles-pp.clear_extent_bit
0.45 ± 13% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.kmem_cache_free
0.40 ± 14% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.__writeback_inodes_wb
0.64 ± 7% -0.2 0.45 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair
0.50 ± 8% -0.2 0.33 ± 23% perf-profile.children.cycles-pp.btrfs_add_delayed_tree_ref
0.88 ± 13% -0.2 0.71 ± 2% perf-profile.children.cycles-pp.wait_reserve_ticket
0.52 ± 9% -0.2 0.35 ± 7% perf-profile.children.cycles-pp.__radix_tree_lookup
0.52 ± 8% -0.2 0.36 ± 8% perf-profile.children.cycles-pp.xas_load
0.78 ± 19% -0.2 0.62 ± 23% perf-profile.children.cycles-pp.__btrfs_free_extent
0.56 ± 17% -0.1 0.42 ± 15% perf-profile.children.cycles-pp.btrfs_del_items
0.51 ± 5% -0.1 0.36 ± 9% perf-profile.children.cycles-pp.dequeue_entity
0.39 ± 7% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers
0.82 ± 14% -0.1 0.68 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate
0.55 ± 8% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.btrfs_submit_bio_hook
0.53 ± 5% -0.1 0.40 ± 13% perf-profile.children.cycles-pp.update_load_avg
0.40 ± 8% -0.1 0.27 ± 27% perf-profile.children.cycles-pp.add_delayed_ref_head
0.35 ± 13% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.set_extent_bit
0.52 ± 9% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.btrfs_wq_submit_bio
0.35 ± 3% -0.1 0.23 ± 32% perf-profile.children.cycles-pp.queued_write_lock_slowpath
0.33 ± 6% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.__slab_alloc
0.63 ± 19% -0.1 0.51 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode
0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode_fallback
0.57 ± 5% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.find_lock_delalloc_range
0.45 ± 5% -0.1 0.34 ± 9% perf-profile.children.cycles-pp.btrfs_cross_ref_exist
0.29 ± 3% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.free_extent_buffer
0.38 ± 3% -0.1 0.26 ± 8% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty
0.46 ± 5% -0.1 0.35 ± 13% perf-profile.children.cycles-pp.find_delalloc_range
0.41 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.btrfs_copy_from_user
0.40 ± 9% -0.1 0.29 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic
0.38 ± 10% -0.1 0.27 perf-profile.children.cycles-pp.find_get_entry
0.31 ± 12% -0.1 0.20 ± 31% perf-profile.children.cycles-pp.btrfs_remove_ordered_extent
0.30 ± 6% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.___slab_alloc
0.26 ± 8% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.merge_state
0.38 ± 7% -0.1 0.28 ± 3% perf-profile.children.cycles-pp.copyin
0.26 ± 17% -0.1 0.15 ± 26% perf-profile.children.cycles-pp.btrfs_delayed_update_inode
0.25 ± 4% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.insert_state
0.40 ± 13% -0.1 0.30 ± 13% perf-profile.children.cycles-pp.extent_clear_unlock_delalloc
0.36 ± 9% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.alloc_extent_state
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
0.39 ± 3% -0.1 0.29 ± 14% perf-profile.children.cycles-pp.alloc_extent_buffer
0.31 ± 4% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.set_extent_buffer_dirty
0.26 ± 11% -0.1 0.17 ± 23% perf-profile.children.cycles-pp.unlock_up
0.32 ± 3% -0.1 0.24 ± 13% perf-profile.children.cycles-pp.__btrfs_map_block
0.23 ± 13% -0.1 0.15 ± 28% perf-profile.children.cycles-pp.btrfs_add_delayed_data_ref
0.25 ± 12% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free
0.47 ± 6% -0.1 0.39 ± 15% perf-profile.children.cycles-pp.__push_leaf_right
0.26 ± 9% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.__test_set_page_writeback
0.32 ± 11% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.btrfs_add_ordered_extent
0.15 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__xa_set_mark
0.23 ± 7% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.release_extent_buffer
0.18 ± 8% -0.1 0.11 ± 26% perf-profile.children.cycles-pp.alloc_extent_map
0.32 ± 10% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.__btrfs_add_ordered_extent
0.32 ± 17% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.btrfs_tree_read_lock_atomic
0.20 ± 13% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.btrfs_inc_extent_ref
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.find_ref_head
0.13 ± 32% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.rb_erase_cached
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_submit_bio_start
0.53 ± 12% -0.1 0.46 ± 16% perf-profile.children.cycles-pp.btrfs_tree_read_lock
0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_csum_one_bio
0.17 ± 16% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.rb_erase
0.21 ± 7% -0.1 0.14 ± 16% perf-profile.children.cycles-pp.run_one_async_start
0.22 ± 3% -0.1 0.15 ± 22% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock
0.21 ± 9% -0.1 0.15 ± 21% perf-profile.children.cycles-pp.btrfs_leaf_free_space
0.15 ± 23% -0.1 0.09 ± 35% perf-profile.children.cycles-pp.free_extent_map
0.20 ± 9% -0.1 0.14 ± 20% perf-profile.children.cycles-pp.leaf_space_used
0.19 ± 8% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.check_delayed_ref
0.17 ± 13% -0.1 0.12 ± 31% perf-profile.children.cycles-pp.get_page_from_freelist
0.13 ± 8% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.__update_load_avg_se
0.17 ± 2% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.btrfs_set_range_writeback
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 11% -0.1 0.15 ± 35% perf-profile.children.cycles-pp.__alloc_pages_nodemask
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_unlock
0.14 ± 11% -0.1 0.09 ± 28% perf-profile.children.cycles-pp.btrfs_reserve_extent
0.10 ± 33% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__btrfs_release_delayed_node
0.22 ± 9% -0.0 0.17 ± 25% perf-profile.children.cycles-pp.mark_extent_buffer_accessed
0.09 ± 13% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_read_unlock_blocking
0.18 ± 8% -0.0 0.14 ± 24% perf-profile.children.cycles-pp.__list_del_entry_valid
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.children.cycles-pp.verify_parent_transid
0.11 ± 19% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.22 ± 5% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.check_committed_ref
0.11 ± 19% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.select_idle_sibling
0.14 ± 11% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.do_io_getevents
0.15 ± 7% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__x64_sys_io_getevents
0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_release_extent_buffer_pages
0.17 ± 18% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.clear_page_dirty_for_io
0.14 ± 5% -0.0 0.10 ± 25% perf-profile.children.cycles-pp.block_group_cache_tree_search
0.18 ± 4% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.btrfs_get_chunk_map
0.11 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.btrfs_buffer_uptodate
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.children.cycles-pp.xas_find_marked
0.12 ± 15% -0.0 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_unlock_up_safe
0.12 ± 14% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.btrfs_set_path_blocking
0.11 ± 11% -0.0 0.07 ± 28% perf-profile.children.cycles-pp.find_free_extent
0.24 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.btrfs_root_node
0.07 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.extent_mergeable
0.13 ± 12% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__switch_to
0.10 ± 18% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.rb_insert_color_cached
0.07 ± 14% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.rb_prev
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.get_io_u
0.10 ± 14% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.available_idle_cpu
0.17 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp._raw_write_lock
0.10 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__might_sleep
0.17 ± 7% -0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_read_lock
0.17 ± 10% -0.0 0.14 ± 12% perf-profile.children.cycles-pp.can_overcommit
0.10 ± 11% -0.0 0.07 perf-profile.children.cycles-pp.__list_add_valid
0.09 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.aio_read_events
0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.___perf_sw_event
0.11 ± 9% -0.0 0.09 ± 18% perf-profile.children.cycles-pp.get_alloc_profile
0.11 ± 12% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.pick_next_task_idle
0.10 ± 13% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.read_events
0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.replace_extent_mapping
0.10 ± 5% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.menu_reflect
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.run_posix_cpu_timers
0.05 ± 58% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.rcu_irq_enter
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.children.cycles-pp.native_irq_return_iret
0.20 ± 14% +0.1 0.27 ± 8% perf-profile.children.cycles-pp.btrfs_comp_cpu_keys
0.38 ± 9% +0.1 0.46 perf-profile.children.cycles-pp.load_balance
0.30 ± 8% +0.1 0.38 ± 15% perf-profile.children.cycles-pp.irq_enter
0.21 ± 24% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq
0.22 ± 3% +0.1 0.32 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline
0.52 ± 12% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.rebalance_domains
1.90 ± 10% +0.3 2.23 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.23 ± 12% +0.4 0.67 ± 15% perf-profile.children.cycles-pp.read_extent_buffer
2.88 ± 11% +0.5 3.41 perf-profile.children.cycles-pp.hrtimer_interrupt
0.09 ± 12% +0.7 0.83 ± 15% perf-profile.children.cycles-pp.btrfs_get_token_8
2.37 ± 19% +0.8 3.19 ± 5% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space
2.33 ± 19% +0.8 3.17 ± 5% perf-profile.children.cycles-pp.flush_space
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_marked_extents
0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range
0.42 ± 43% +1.3 1.67 ± 18% perf-profile.children.cycles-pp.btrfs_commit_transaction
0.25 ± 5% +1.9 2.17 ± 10% perf-profile.children.cycles-pp.btrfs_get_token_64
1.71 ± 5% +2.1 3.79 ± 5% perf-profile.children.cycles-pp.btrfs_get_token_32
0.43 ± 8% +2.4 2.84 ± 10% perf-profile.children.cycles-pp.map_private_extent_buffer
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_writeback
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.writeback_sb_inodes
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.__writeback_single_inode
8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_workfn
50.06 ± 2% +3.8 53.82 ± 4% perf-profile.children.cycles-pp.intel_idle
61.99 ± 2% +4.4 66.39 ± 3% perf-profile.children.cycles-pp.do_idle
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64
61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry
8.51 ± 7% +4.5 13.04 ± 12% perf-profile.children.cycles-pp.do_writepages
56.43 ± 3% +4.9 61.30 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state
61.27 ± 2% +5.1 66.37 ± 3% perf-profile.children.cycles-pp.start_secondary
1.43 ± 7% +6.2 7.64 ± 13% perf-profile.children.cycles-pp.submit_extent_page
1.20 ± 17% +6.2 7.45 ± 13% perf-profile.children.cycles-pp.btree_write_cache_pages
1.05 ± 17% +6.3 7.31 ± 13% perf-profile.children.cycles-pp.write_one_eb
1.11 ± 7% +6.3 7.37 ± 13% perf-profile.children.cycles-pp.submit_one_bio
0.55 ± 16% +6.4 6.95 ± 13% perf-profile.children.cycles-pp.btree_submit_bio_hook
0.00 +6.5 6.48 ± 13% perf-profile.children.cycles-pp.check_leaf
0.00 +6.5 6.53 ± 13% perf-profile.children.cycles-pp.btree_csum_one_bio
2.12 ± 11% -0.8 1.29 ± 7% perf-profile.self.cycles-pp.__lookup_extent_mapping
1.83 ± 12% -0.8 1.03 ± 13% perf-profile.self.cycles-pp.__etree_search
2.06 ± 5% -0.5 1.59 ± 5% perf-profile.self.cycles-pp._raw_spin_lock
0.81 ± 9% -0.3 0.52 ± 10% perf-profile.self.cycles-pp.generic_bin_search
0.84 ± 7% -0.3 0.58 ± 17% perf-profile.self.cycles-pp.copy_page
0.33 ± 54% -0.2 0.10 ± 28% perf-profile.self.cycles-pp.delay_tsc
0.89 ± 9% -0.2 0.69 ± 14% perf-profile.self.cycles-pp.btrfs_set_token_32
0.81 ± 4% -0.2 0.63 ± 14% perf-profile.self.cycles-pp.__schedule
0.51 ± 9% -0.2 0.35 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup
0.45 ± 8% -0.1 0.32 ± 10% perf-profile.self.cycles-pp.xas_load
0.43 ± 4% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.find_extent_buffer
0.77 ± 5% -0.1 0.66 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.31 ± 10% -0.1 0.20 ± 28% perf-profile.self.cycles-pp.add_delayed_ref_head
0.39 ± 16% -0.1 0.29 ± 16% perf-profile.self.cycles-pp.try_to_wake_up
0.31 ± 4% -0.1 0.20 ± 26% perf-profile.self.cycles-pp.queued_write_lock_slowpath
0.38 ± 7% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
0.20 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.kmem_cache_free
0.25 ± 11% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.__slab_free
0.16 ± 9% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.free_extent_buffer
0.28 ± 17% -0.1 0.20 ± 11% perf-profile.self.cycles-pp.btrfs_search_slot
0.31 ± 4% -0.1 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc
0.14 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.find_ref_head
0.16 ± 10% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.insert_state
0.12 ± 8% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.__update_load_avg_se
0.16 ± 15% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.rb_erase
0.10 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_unlock
0.15 ± 13% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.___slab_alloc
0.28 ± 13% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.queued_read_lock_slowpath
0.15 ± 21% -0.1 0.09 ± 35% perf-profile.self.cycles-pp.free_extent_map
0.18 ± 6% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.update_load_avg
0.10 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.btrfs_extend_item
0.15 ± 23% -0.1 0.09 ± 9% perf-profile.self.cycles-pp.add_extent_mapping
0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.xas_set_mark
0.18 ± 6% -0.1 0.13 ± 16% perf-profile.self.cycles-pp.___might_sleep
0.18 ± 6% -0.0 0.13 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid
0.18 ± 4% -0.0 0.14 ± 22% perf-profile.self.cycles-pp.pick_next_task_fair
0.09 ± 13% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.verify_parent_transid
0.09 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_read_unlock_blocking
0.16 ± 13% -0.0 0.12 ± 24% perf-profile.self.cycles-pp.mark_page_accessed
0.12 ± 14% -0.0 0.08 ± 20% perf-profile.self.cycles-pp.dequeue_task_fair
0.10 ± 19% -0.0 0.07 ± 18% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.08 ± 14% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.unlock_up
0.15 ± 15% -0.0 0.11 ± 19% perf-profile.self.cycles-pp.xas_find_marked
0.24 ± 6% -0.0 0.20 ± 8% perf-profile.self.cycles-pp.btrfs_root_node
0.17 ± 6% -0.0 0.13 ± 10% perf-profile.self.cycles-pp._raw_read_lock
0.10 ± 13% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.btrfs_csum_one_bio
0.16 ± 7% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.dequeue_entity
0.07 -0.0 0.04 ± 71% perf-profile.self.cycles-pp.io_u_queued_complete
0.08 -0.0 0.05 ± 70% perf-profile.self.cycles-pp.mutex_lock
0.11 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.get_io_u
0.16 ± 9% -0.0 0.13 perf-profile.self.cycles-pp._raw_write_lock
0.07 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.btrfs_tree_read_lock
0.12 ± 8% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.__switch_to
0.10 ± 11% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.__clear_extent_bit
0.07 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.rb_prev
0.10 ± 15% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.find_get_pages_range_tag
0.10 ± 12% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.available_idle_cpu
0.10 ± 11% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid
0.08 ± 13% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__might_sleep
0.12 ± 17% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.release_extent_buffer
0.09 ± 17% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event
0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.__push_leaf_right
0.11 ± 7% -0.0 0.09 ± 9% perf-profile.self.cycles-pp.set_extent_buffer_dirty
0.11 ± 4% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.__set_extent_bit
0.09 ± 7% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.hrtimer_interrupt
0.07 ± 17% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.run_posix_cpu_timers
0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.load_balance
0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline
0.00 +0.1 0.06 ± 23% perf-profile.self.cycles-pp.find_next_and_bit
0.19 ± 11% +0.1 0.26 ± 17% perf-profile.self.cycles-pp.native_irq_return_iret
0.19 ± 24% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq
0.50 ± 8% +0.2 0.69 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state
0.07 ± 11% +0.4 0.46 ± 17% perf-profile.self.cycles-pp.btrfs_get_token_8
0.22 ± 13% +0.4 0.66 ± 16% perf-profile.self.cycles-pp.read_extent_buffer
0.00 +0.6 0.65 ± 14% perf-profile.self.cycles-pp.check_leaf
0.15 ± 11% +1.0 1.13 ± 11% perf-profile.self.cycles-pp.btrfs_get_token_64
1.48 ± 7% +1.0 2.51 ± 5% perf-profile.self.cycles-pp.btrfs_get_token_32
0.41 ± 8% +2.2 2.61 ± 9% perf-profile.self.cycles-pp.map_private_extent_buffer
49.99 ± 2% +3.7 53.74 ± 4% perf-profile.self.cycles-pp.intel_idle
fio.write_bw_MBps
300 +-+-------------------------------------------------------------------+
| |
250 +-++.+ +. .+.. .+..+..+.+.. .+..+.. .+ |
| : : +. +..+ + +..+ +. + |
| : : : : : : |
200 +-+ : : : : : : O O |
O O O: O :O O O O O O O O O O: O:O O: O: O O O O O
150 +-+ : : : : : : |
| : : : : : : |
100 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
50 +-+ :: :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_iops
70000 +-+-----------------------------------------------------------------+
|..+.+ +..+..+.+.. .+.+..+.+..+ +.. +..+.+..+..+ |
60000 +-+ : : +. : : + : |
| : : : : : : |
50000 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
40000 +-+ : O: : : : : |
| : : : : : : |
30000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
10000 +-+ : : : |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_mean_us
30000 +-+-----------------------------------------------------------------+
| |
25000 O-+O O O O O O O O O O O O O O O O O O O |
| O O O O O
| |
20000 +-+ |
|..+.+ +..+..+.+..+..+.+..+.+..+ +..+ +..+.+..+..+ |
15000 +-+ : : : : : : |
| : : : : : : |
10000 +-+ : : : : : : |
| : : : : : : |
| : : : : : : |
5000 +-+ : : : : :: |
| : : : |
0 +-+--------------------------------------------O------------O-------+
fio.write_clat_stddev
160000 +-+----------------------------------------------------------------+
| |
140000 O-+O O O O O O O O O O O O O O |
120000 +-+ O O |
| |
100000 +-+ O O O O O O O O
| |
80000 +-+ |
| .+.. |
60000 +-++.+ +..+.+. +.+..+.+..+..+ +.+ +..+..+.+..+ |
40000 +-+ : : : : : : |
| : : : : : : |
20000 +-+ : : : : : : |
| :: :: :: |
0 +-+-------------------------------------------O------------O-------+
fio.write_slat_mean_us
900 +-+-------------------------------------------------------------------+
O O O O O O O O O O O O O O O O O O |
800 +-+O O O O O O O
700 +-+ |
| |
600 +-++.+ +.+..+..+..+.+..+..+.+..+ +..+ +..+..+..+.+ |
500 +-+ : : : : : : |
| : : : : : : |
400 +-+ : : : : : : |
300 +-+ : : : : : : |
| : : : : : : |
200 +-+ : : : : : : |
100 +-+ : : :: :: |
| : : : |
0 +-+---------------------------------------------O-------------O-------+
fio.write_slat_stddev
25000 +-+-----O---------O-------------------------------------------------+
O O O O O O O O O O O O O O O |
| |
20000 +-+ |
| O O O O O O O O
| |
15000 +-+ |
| |
10000 +-++. .+.. .+.. .+..+ .+ .+..+ |
|. + +..+..+ +..+ + : +. : +..+.+. |
| : : : : : : |
5000 +-+ : : : : : : |
| : : : : : : |
| :: : :: |
0 +-+--------------------------------------------O------------O-------+
fio.latency_1000ms_
0.4 +-+------------------------------------------------------------------+
| |
0.35 +-+ O |
0.3 +-+ |
| O O |
0.25 +-+ O O |
| O O O
0.2 O-+ O O O O O O O O O O |
| O O |
0.15 +-+O O O O |
0.1 +-+ |
| |
0.05 +-+ |
| .+. .+..+..+.+..+.. .+. .+.+..+.. |
0 +-+---------------------------------------------O------------O-------+
fio.workload
1.4e+07 +-+---------------------------------------------------------------+
|..+.+ +..+.+..+.+..+.+..+.+..+ +.. +.+..+.+..+ |
1.2e+07 +-+ : : : : + : |
| : : : : : : |
1e+07 +-+ : : : : : : |
O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O
8e+06 +-+ : O: : : : : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| :: :: :: |
2e+06 +-+ : : : |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.voluntary_context_switches
1.4e+07 +-+---------------------------------------------------------------+
| |
1.2e+07 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
1e+07 +-+ : : : : : : |
| O O: : O O O : : :O : O O O O O O O
8e+06 O-+ : O:O O O O O O O: O:O O: : |
| : : : : : : |
6e+06 +-+ : : : : : : |
| : : : : : : |
4e+06 +-+ : : : : : : |
| : : : : :: |
2e+06 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
fio.time.file_system_outputs
1.2e+08 +-+---------------------------------------------------------------+
| |
1e+08 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ |
| : : : : : : |
| : : : : : : |
8e+07 +-+ : : : : : : O O O O O
O O O: O:O O O O O O O O O O: O:O O:O : O O |
6e+07 +-+ : : : : : : |
| : : : : : : |
4e+07 +-+ : : : : : : |
| : : : : : : |
| : : : : :: |
2e+07 +-+ : : :: |
| : : : |
0 +-+-------------------------------------------O-----------O-------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
9 months, 1 week