March 2020 - LKP - Ml01.01.Org

[sched/fair] 3c29e651e1: hackbench.throughput -15.2% regression

by kernel test robot

Greeting, FYI, we noticed a -15.2% regression of hackbench.throughput due to commit: commit: 3c29e651e16dd3b3179cfb2d055ee9538e37515c ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: hackbench on test machine: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory with following parameters: nr_threads: 100% mode: threads ipc: pipe cpufreq_governor: performance ucode: 0xca test-description: Hackbench is both a benchmark and a stress test for the Linux kernel scheduler. test-url: https://github.com/linux-test-project/ltp/blob/master/testcases/kernel/sc... In addition to that, the commit also has significant impact on the following tests: +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -11.3% regression | | test machine | 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | iterations=18 | | | mode=threads | | | nr_threads=1600% | | | ucode=0x43 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -10.6% regression | | test machine | 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=process | | | nr_threads=50% | | | ucode=0xb000038 | +------------------+-----------------------------------------------------------------------+ | testcase: change | hackbench: hackbench.throughput -6.8% regression | | test machine | 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory | | test parameters | cpufreq_governor=performance | | | ipc=pipe | | | mode=threads | | | nr_threads=100% | | | ucode=0xb8 | +------------------+-----------------------------------------------------------------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-20191114.cgz/lkp-cfl-e1/hackbench/0xca commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- %stddev %change %stddev \ | \ 69492 ± 3% -15.2% 58899 hackbench.throughput 7.727e+08 ± 7% +38.3% 1.069e+09 ± 2% hackbench.time.involuntary_context_switches 131708 ± 4% -15.4% 111382 hackbench.time.minor_page_faults 684.98 ± 2% -2.5% 667.72 hackbench.time.user_time 1.755e+09 ± 2% +11.2% 1.953e+09 hackbench.time.voluntary_context_switches 4.32e+08 ± 4% -15.6% 3.648e+08 hackbench.workload 186.50 ± 12% -16.6% 155.50 ± 11% interrupts.TLB:TLB_shootdowns 4147892 ± 5% +19.6% 4961643 vmstat.system.cs 0.72 ± 10% -0.1 0.58 ± 17% mpstat.cpu.all.idle% 0.00 ±156% +0.0 0.01 ± 62% mpstat.cpu.all.soft% 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.active_objs 1038 ± 9% +18.5% 1230 ± 4% slabinfo.avc_xperms_data.num_objs 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_hit 40647036 ± 10% -34.6% 26589504 proc-vmstat.numa_local 40724029 ± 10% -34.5% 26661546 proc-vmstat.pgalloc_normal 763906 ± 2% -7.5% 706781 proc-vmstat.pgfault 40701242 ± 10% -34.6% 26637947 proc-vmstat.pgfree 1636058 ± 11% -45.0% 899629 ± 11% turbostat.C1 0.10 ± 7% -0.0 0.07 ± 7% turbostat.C1% 15201 ± 25% +113.3% 32422 ± 8% turbostat.C1E 0.01 ± 57% +0.0 0.03 ± 13% turbostat.C1E% 9093 ± 11% +217.2% 28841 ± 11% turbostat.C3 0.00 ±173% +0.0 0.03 ± 13% turbostat.C3% 16977 ± 2% -92.8% 1222 ± 98% turbostat.C8 0.17 ± 4% -0.2 0.01 ±110% turbostat.C8% 0.13 -100.0% 0.00 turbostat.CPU%c7 9623106 ± 5% -35.1% 6241651 ± 8% cpuidle.C1.time 1636668 ± 11% -45.0% 899926 ± 11% cpuidle.C1.usage 899494 ± 33% +272.6% 3351412 ± 5% cpuidle.C1E.time 15327 ± 25% +112.6% 32592 ± 8% cpuidle.C1E.usage 435674 ± 20% +643.5% 3239351 ± 14% cpuidle.C3.time 9179 ± 11% +215.2% 28933 ± 11% cpuidle.C3.usage 16850924 ± 2% -94.2% 979516 ± 86% cpuidle.C8.time 17150 ± 2% -92.0% 1378 ± 88% cpuidle.C8.usage 6022328 ± 18% -57.1% 2584773 ± 7% cpuidle.POLL.time 5339703 ± 19% -60.1% 2130089 ± 7% cpuidle.POLL.usage 14.06 ± 25% +64.0% 23.07 ± 13% sched_debug.cfs_rq:/.runnable_load_avg.min 25.10 ± 5% -15.8% 21.12 ± 7% sched_debug.cfs_rq:/.runnable_load_avg.stddev 13339 ± 24% +45.7% 19433 ± 7% sched_debug.cfs_rq:/.runnable_weight.min 166.51 ± 24% +48.9% 247.86 ± 7% sched_debug.cfs_rq:/.util_est_enqueued.min 253.56 ± 2% -12.3% 222.30 ± 9% sched_debug.cfs_rq:/.util_est_enqueued.stddev 159602 ± 23% +55.6% 248411 ± 6% sched_debug.cpu.avg_idle.max 51567 ± 24% +49.1% 76905 ± 11% sched_debug.cpu.avg_idle.stddev 11576 ± 5% -9.5% 10478 sched_debug.cpu.curr->pid.avg 9976 ± 8% -16.8% 8305 ± 9% sched_debug.cpu.curr->pid.min 4.81 ± 15% +43.3% 6.89 ± 8% sched_debug.cpu.nr_running.min 6.68 ± 4% -17.0% 5.54 ± 8% sched_debug.cpu.nr_running.stddev 76003928 ± 2% +21.0% 91975904 sched_debug.cpu.nr_switches.avg 78115404 ± 2% +23.4% 96397551 sched_debug.cpu.nr_switches.max 74275755 ± 2% +19.0% 88359903 sched_debug.cpu.nr_switches.min 1043054 ± 15% +83.9% 1917752 ± 10% sched_debug.cpu.nr_switches.stddev -685.26 +64.8% -1129 sched_debug.cpu.nr_uninterruptible.min 15470 ± 9% -31.0% 10674 ± 9% softirqs.CPU0.SCHED 13756 ± 10% -40.8% 8140 ± 4% softirqs.CPU1.SCHED 13645 ± 11% -39.7% 8227 ± 3% softirqs.CPU10.SCHED 14065 ± 12% -38.8% 8609 ± 2% softirqs.CPU11.SCHED 13948 ± 11% -39.9% 8381 ± 6% softirqs.CPU12.SCHED 13823 ± 12% -39.7% 8332 ± 9% softirqs.CPU13.SCHED 73323 +13.4% 83165 ± 11% softirqs.CPU14.RCU 13872 ± 12% -36.5% 8809 ± 7% softirqs.CPU14.SCHED 13726 ± 12% -40.4% 8175 ± 4% softirqs.CPU15.SCHED 13780 ± 11% -34.8% 8987 ± 12% softirqs.CPU2.SCHED 13895 ± 10% -40.5% 8266 ± 6% softirqs.CPU3.SCHED 221879 +13.4% 251589 ± 14% softirqs.CPU3.TIMER 13873 ± 10% -40.4% 8262 ± 6% softirqs.CPU4.SCHED 72625 +12.0% 81355 ± 10% softirqs.CPU5.RCU 13876 ± 13% -41.4% 8137 ± 3% softirqs.CPU5.SCHED 14118 ± 12% -40.8% 8354 ± 3% softirqs.CPU6.SCHED 13503 ± 12% -40.0% 8099 softirqs.CPU7.SCHED 13549 ± 11% -38.2% 8374 ± 8% softirqs.CPU8.SCHED 14273 ± 10% -40.1% 8548 ± 6% softirqs.CPU9.SCHED 223184 ± 11% -38.9% 136383 ± 5% softirqs.SCHED 55.80 +9.9% 61.34 perf-stat.i.MPKI 1.32 +0.1 1.39 perf-stat.i.branch-miss-rate% 1.066e+08 +3.7% 1.105e+08 perf-stat.i.branch-misses 0.15 ± 8% -0.0 0.11 ± 3% perf-stat.i.cache-miss-rate% 2521584 ± 9% -26.5% 1854353 ± 3% perf-stat.i.cache-misses 2.27e+09 +8.0% 2.451e+09 perf-stat.i.cache-references 4162783 ± 4% +19.7% 4981708 perf-stat.i.context-switches 1.52 +2.0% 1.55 perf-stat.i.cpi 227633 ± 14% +75.1% 398694 ± 3% perf-stat.i.cpu-migrations 33931 ± 12% +42.7% 48428 ± 2% perf-stat.i.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.i.dTLB-load-miss-rate% 1408442 ± 2% -16.4% 1177727 ± 4% perf-stat.i.dTLB-load-misses 1.229e+10 -2.1% 1.203e+10 perf-stat.i.dTLB-loads 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 35757 ± 2% -16.0% 30032 ± 3% perf-stat.i.dTLB-store-misses 7.449e+09 -2.3% 7.279e+09 perf-stat.i.dTLB-stores 39246970 -5.5% 37079604 perf-stat.i.iTLB-load-misses 76813 ± 8% -26.2% 56724 ± 6% perf-stat.i.iTLB-loads 4.096e+10 -1.9% 4.02e+10 perf-stat.i.instructions 1062 +4.1% 1105 perf-stat.i.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.i.ipc 1210 -7.1% 1124 perf-stat.i.minor-faults 105889 ± 10% -24.8% 79596 ± 5% perf-stat.i.node-loads 174508 ± 7% -24.4% 131915 perf-stat.i.node-stores 1210 -7.1% 1124 perf-stat.i.page-faults 55.42 ± 2% +10.0% 60.98 perf-stat.overall.MPKI 1.31 +0.1 1.38 perf-stat.overall.branch-miss-rate% 0.11 ± 11% -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 1.51 +1.9% 1.54 perf-stat.overall.cpi 24740 ± 9% +34.9% 33377 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 -0.0 0.01 ± 4% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 2% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1043 +3.9% 1084 perf-stat.overall.instructions-per-iTLB-miss 0.66 -1.9% 0.65 perf-stat.overall.ipc 57826 ± 3% +15.6% 66874 perf-stat.overall.path-length 1.064e+08 +3.7% 1.104e+08 perf-stat.ps.branch-misses 2517524 ± 9% -26.5% 1851335 ± 3% perf-stat.ps.cache-misses 2.266e+09 +8.0% 2.447e+09 perf-stat.ps.cache-references 4155898 ± 4% +19.7% 4973458 perf-stat.ps.context-switches 227258 ± 14% +75.1% 398041 ± 3% perf-stat.ps.cpu-migrations 1406119 ± 2% -16.4% 1175777 ± 4% perf-stat.ps.dTLB-load-misses 1.227e+10 -2.1% 1.201e+10 perf-stat.ps.dTLB-loads 35699 ± 2% -16.0% 29983 ± 3% perf-stat.ps.dTLB-store-misses 7.437e+09 -2.3% 7.267e+09 perf-stat.ps.dTLB-stores 39182154 -5.5% 37018206 perf-stat.ps.iTLB-load-misses 76695 ± 8% -26.2% 56638 ± 6% perf-stat.ps.iTLB-loads 4.09e+10 -1.9% 4.013e+10 perf-stat.ps.instructions 1208 -7.1% 1122 perf-stat.ps.minor-faults 105716 ± 10% -24.8% 79465 ± 5% perf-stat.ps.node-loads 174234 ± 7% -24.4% 131703 perf-stat.ps.node-stores 1208 -7.1% 1122 perf-stat.ps.page-faults 29.39 ±104% -29.4 0.00 perf-profile.calltrace.cycles-pp.start_thread 15.86 ±105% -15.9 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 14.88 ±105% -14.9 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 14.75 ±105% -14.8 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 13.53 ±104% -13.5 0.00 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 12.51 ±104% -12.5 0.00 perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 12.45 ±105% -12.4 0.00 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 11.21 ±105% -11.2 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 11.08 ±105% -11.1 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 10.47 ±105% -10.5 0.00 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 9.73 ±105% -9.7 0.00 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.09 ± 12% -0.6 2.50 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.45 ± 14% -0.5 1.95 ± 2% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 1.15 ± 13% -0.2 0.95 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 0.99 ± 13% -0.2 0.83 ± 5% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.92 ± 15% +0.2 1.10 perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.84 ± 23% +0.3 1.10 ± 2% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.99 ± 19% +0.3 1.26 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.97 ± 23% +0.3 1.26 perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.29 ±100% +0.3 0.60 ± 5% perf-profile.calltrace.cycles-pp.reschedule_interrupt.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write 0.55 ± 62% +0.3 0.89 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.28 ±100% +0.4 0.63 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop 0.33 ±100% +0.4 0.69 ± 2% perf-profile.calltrace.cycles-pp.prepare_to_wait.pipe_wait.pipe_read.new_sync_read.vfs_read 0.29 ±100% +0.4 0.65 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.00 ± 27% +0.4 1.37 ± 2% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.57 ± 63% +0.4 0.95 ± 5% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.46 ± 31% +0.4 1.86 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 60% +0.4 1.14 ± 2% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.29 ±100% +0.4 0.70 ± 6% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.34 ±100% +0.4 0.77 ± 7% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task 1.26 ± 26% +0.5 1.72 perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.13 ± 26% +0.5 1.59 perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.35 ±100% +0.5 0.83 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.28 ± 28% +0.5 1.77 ± 3% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.40 ± 26% +0.5 1.89 perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.44 ±100% +0.5 0.95 ± 7% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.33 ± 17% +0.5 2.88 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 1.33 ± 28% +0.6 1.89 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.40 ± 24% +0.6 1.97 ± 4% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.83 ± 66% +0.6 1.40 ± 5% perf-profile.calltrace.cycles-pp.native_write_msr 5.01 ± 8% +0.8 5.81 ± 2% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.75 ± 39% +0.9 2.61 ± 6% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 1.33 ± 67% +1.0 2.28 ± 6% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 6.56 ± 12% +1.2 7.78 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 8.96 ± 9% +1.6 10.56 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 9.23 ± 10% +1.6 10.87 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.17 ± 10% +1.6 10.81 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 3.72 ± 31% +1.7 5.44 ± 6% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.37 ± 8% +1.9 5.32 ± 4% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.37 ± 10% +2.3 6.65 ± 5% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.27 ± 10% +2.3 17.59 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.59 ± 10% +2.4 17.96 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 3.00 ± 71% +2.6 5.60 ± 6% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.10 ± 71% +2.7 5.77 ± 6% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 18.05 ± 10% +2.7 20.73 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 28.21 ± 5% +3.7 31.87 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 24.29 ± 8% +4.7 28.98 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.38 ± 8% +4.7 28.08 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 23.72 ± 8% +4.7 28.43 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 31.18 ± 41% +12.7 43.86 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 33.44 ± 40% +13.2 46.61 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 64.81 ± 43% +27.5 92.34 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 65.30 ± 43% +27.7 92.99 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 29.39 ±104% -29.4 0.00 perf-profile.children.cycles-pp.start_thread 16.09 ±105% -16.1 0.00 perf-profile.children.cycles-pp.__GI___libc_write 12.73 ±104% -12.7 0.00 perf-profile.children.cycles-pp.__GI___libc_read 5.80 ± 10% -1.2 4.65 ± 3% perf-profile.children.cycles-pp.security_file_permission 4.10 ± 16% -1.0 3.10 ± 6% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 3.21 ± 9% -0.7 2.55 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 1.45 ± 32% -0.6 0.83 perf-profile.children.cycles-pp.entry_SYSCALL_64 3.04 ± 8% -0.6 2.44 ± 3% perf-profile.children.cycles-pp.selinux_file_permission 2.56 ± 11% -0.6 1.99 ± 2% perf-profile.children.cycles-pp.copy_page_from_iter 2.80 ± 9% -0.5 2.25 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.38 ± 7% -0.5 2.87 ± 2% perf-profile.children.cycles-pp.mutex_lock 1.62 ± 12% -0.4 1.23 ± 3% perf-profile.children.cycles-pp.___might_sleep 1.82 ± 9% -0.4 1.43 ± 4% perf-profile.children.cycles-pp.syscall_return_via_sysret 2.09 ± 12% -0.4 1.72 ± 6% perf-profile.children.cycles-pp.file_has_perm 0.90 ± 20% -0.3 0.57 ± 9% perf-profile.children.cycles-pp.__mutex_lock 1.14 ± 12% -0.3 0.82 ± 3% perf-profile.children.cycles-pp.__inode_security_revalidate 1.26 ± 10% -0.3 0.97 ± 4% perf-profile.children.cycles-pp.copyin 1.70 ± 8% -0.3 1.42 ± 3% perf-profile.children.cycles-pp.copyout 1.25 ± 11% -0.3 0.98 ± 3% perf-profile.children.cycles-pp.fsnotify 1.04 ± 9% -0.3 0.79 ± 3% perf-profile.children.cycles-pp._cond_resched 1.72 ± 7% -0.2 1.48 ± 5% perf-profile.children.cycles-pp.fput_many 1.16 ± 10% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.__might_sleep 1.09 ± 8% -0.2 0.85 ± 3% perf-profile.children.cycles-pp.__fsnotify_parent 0.83 ± 13% -0.2 0.63 ± 3% perf-profile.children.cycles-pp.__might_fault 0.69 ± 11% -0.1 0.56 perf-profile.children.cycles-pp.current_time 0.99 ± 4% -0.1 0.89 ± 4% perf-profile.children.cycles-pp.touch_atime 0.65 ± 6% -0.1 0.55 ± 6% perf-profile.children.cycles-pp.atime_needs_update 0.39 ± 4% -0.1 0.29 perf-profile.children.cycles-pp.rcu_all_qs 0.13 ± 39% -0.1 0.04 ± 59% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.34 ± 11% -0.1 0.26 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 0.49 ± 5% -0.1 0.41 ± 2% perf-profile.children.cycles-pp.wake_up_q 0.24 ± 2% -0.1 0.17 ± 4% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.30 ± 11% -0.1 0.23 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.23 ± 8% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.__sb_end_write 0.21 ± 9% -0.1 0.16 ± 9% perf-profile.children.cycles-pp.__x64_sys_read 0.29 ± 8% -0.0 0.24 ± 2% perf-profile.children.cycles-pp.timespec64_trunc 0.21 ± 8% -0.0 0.17 ± 8% perf-profile.children.cycles-pp.__x64_sys_write 0.22 ± 7% -0.0 0.19 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.08 ± 16% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.bpf_fd_pass 0.10 ± 4% +0.0 0.11 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.08 ± 24% +0.0 0.13 ± 5% perf-profile.children.cycles-pp.rcu_note_context_switch 0.17 ± 19% +0.1 0.24 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.23 ± 19% +0.1 0.32 ± 3% perf-profile.children.cycles-pp.resched_curr 0.61 ± 8% +0.1 0.72 ± 6% perf-profile.children.cycles-pp.sched_clock 0.67 ± 8% +0.1 0.78 ± 6% perf-profile.children.cycles-pp.sched_clock_cpu 0.50 ± 9% +0.1 0.62 ± 3% perf-profile.children.cycles-pp.account_entity_enqueue 0.61 ± 10% +0.2 0.79 perf-profile.children.cycles-pp.account_entity_dequeue 0.43 ± 19% +0.2 0.62 ± 7% perf-profile.children.cycles-pp.clear_buddies 1.42 ± 14% +0.2 1.66 ± 4% perf-profile.children.cycles-pp.native_write_msr 1.17 ± 13% +0.3 1.42 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup 0.85 ± 23% +0.3 1.12 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.94 ± 14% +0.3 1.22 ± 5% perf-profile.children.cycles-pp.___perf_sw_event 1.45 ± 11% +0.3 1.73 perf-profile.children.cycles-pp.check_preempt_curr 1.45 ± 12% +0.3 1.76 perf-profile.children.cycles-pp.__enqueue_entity 1.60 ± 10% +0.3 1.91 perf-profile.children.cycles-pp.ttwu_do_wakeup 1.12 ± 9% +0.3 1.43 ± 3% perf-profile.children.cycles-pp.__update_load_avg_se 1.64 ± 8% +0.4 2.00 ± 4% perf-profile.children.cycles-pp.available_idle_cpu 0.96 ± 19% +0.4 1.34 ± 7% perf-profile.children.cycles-pp.pick_next_entity 1.85 ± 16% +0.4 2.28 ± 6% perf-profile.children.cycles-pp.switch_fpu_return 1.90 ± 9% +0.6 2.50 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 3.22 ± 13% +0.6 3.87 perf-profile.children.cycles-pp.update_load_avg 4.03 ± 10% +0.6 4.68 ± 2% perf-profile.children.cycles-pp.update_curr 3.01 ± 14% +0.7 3.72 ± 2% perf-profile.children.cycles-pp.reweight_entity 5.10 ± 8% +0.8 5.87 ± 2% perf-profile.children.cycles-pp.enqueue_entity 4.71 ± 15% +1.1 5.77 ± 4% perf-profile.children.cycles-pp.pick_next_task_fair 6.73 ± 11% +1.1 7.87 perf-profile.children.cycles-pp.dequeue_task_fair 9.07 ± 9% +1.5 10.61 perf-profile.children.cycles-pp.enqueue_task_fair 9.32 ± 9% +1.6 10.89 perf-profile.children.cycles-pp.ttwu_do_activate 91.46 +1.6 93.03 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 9.27 ± 9% +1.6 10.84 perf-profile.children.cycles-pp.activate_task 4.16 ± 26% +1.6 5.78 ± 6% perf-profile.children.cycles-pp.exit_to_usermode_loop 90.79 +1.7 92.47 perf-profile.children.cycles-pp.do_syscall_64 3.48 ± 8% +1.9 5.36 ± 4% perf-profile.children.cycles-pp.select_idle_sibling 4.44 ± 10% +2.2 6.67 ± 5% perf-profile.children.cycles-pp.select_task_rq_fair 18.46 ± 9% +2.3 20.81 perf-profile.children.cycles-pp.pipe_wait 19.93 ± 11% +3.5 23.46 ± 2% perf-profile.children.cycles-pp.__schedule 28.82 ± 5% +3.6 32.45 perf-profile.children.cycles-pp.__wake_up_common_lock 19.89 ± 12% +3.7 23.62 ± 2% perf-profile.children.cycles-pp.schedule 23.96 ± 8% +4.5 28.45 perf-profile.children.cycles-pp.try_to_wake_up 24.53 ± 8% +4.5 29.06 perf-profile.children.cycles-pp.__wake_up_common 23.91 ± 8% +4.6 28.46 perf-profile.children.cycles-pp.autoremove_wake_function 2.76 ± 9% -0.5 2.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.46 ± 18% -0.4 1.06 ± 4% perf-profile.self.cycles-pp.pipe_write 1.81 ± 9% -0.4 1.43 ± 4% perf-profile.self.cycles-pp.syscall_return_via_sysret 1.56 ± 12% -0.4 1.19 ± 3% perf-profile.self.cycles-pp.___might_sleep 2.09 ± 7% -0.3 1.81 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.86 ± 8% -0.3 1.59 ± 3% perf-profile.self.cycles-pp.selinux_file_permission 1.21 ± 11% -0.3 0.96 ± 2% perf-profile.self.cycles-pp.fsnotify 1.70 ± 7% -0.2 1.46 ± 5% perf-profile.self.cycles-pp.fput_many 1.04 ± 11% -0.2 0.82 ± 3% perf-profile.self.cycles-pp.__might_sleep 1.01 ± 8% -0.2 0.80 ± 3% perf-profile.self.cycles-pp.__fsnotify_parent 1.04 ± 10% -0.2 0.83 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.67 ± 10% -0.2 0.51 ± 5% perf-profile.self.cycles-pp.new_sync_write 0.64 ± 12% -0.2 0.48 ± 5% perf-profile.self.cycles-pp.vfs_write 1.67 -0.2 1.51 ± 2% perf-profile.self.cycles-pp.pipe_read 0.58 ± 17% -0.1 0.44 ± 7% perf-profile.self.cycles-pp.file_has_perm 0.44 ± 18% -0.1 0.31 ± 7% perf-profile.self.cycles-pp.__mutex_lock 0.49 ± 13% -0.1 0.35 ± 3% perf-profile.self.cycles-pp.copy_page_from_iter 0.64 ± 8% -0.1 0.52 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.67 ± 7% -0.1 0.55 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.30 ± 5% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.ksys_write 0.40 ± 12% -0.1 0.30 ± 7% perf-profile.self.cycles-pp.security_file_permission 0.13 ± 38% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.36 ± 11% -0.1 0.28 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.76 ± 5% -0.1 0.68 ± 5% perf-profile.self.cycles-pp.do_syscall_64 0.21 ± 2% -0.1 0.14 ± 8% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.26 ± 12% -0.1 0.20 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.29 ± 11% -0.1 0.23 ± 6% perf-profile.self.cycles-pp.current_time 0.28 ± 4% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.rcu_all_qs 0.22 ± 7% -0.1 0.17 ± 7% perf-profile.self.cycles-pp.__sb_end_write 0.31 -0.1 0.26 ± 7% perf-profile.self.cycles-pp.ksys_read 0.20 ± 11% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_read 0.26 ± 8% -0.0 0.22 perf-profile.self.cycles-pp.timespec64_trunc 0.20 ± 5% -0.0 0.16 ± 9% perf-profile.self.cycles-pp.__x64_sys_write 0.24 ± 9% -0.0 0.21 ± 6% perf-profile.self.cycles-pp.__might_fault 0.09 ± 9% -0.0 0.06 perf-profile.self.cycles-pp.copyin 0.14 ± 5% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.07 ± 10% +0.0 0.09 ± 4% perf-profile.self.cycles-pp.sched_clock 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.20 ± 12% +0.0 0.24 ± 5% perf-profile.self.cycles-pp.__list_add_valid 0.10 ± 26% +0.0 0.14 ± 10% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.15 ± 16% +0.1 0.21 ± 11% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.35 ± 7% +0.1 0.43 ± 4% perf-profile.self.cycles-pp.prepare_to_wait 0.55 ± 2% +0.1 0.63 ± 3% perf-profile.self.cycles-pp.__fget_light 0.22 ± 18% +0.1 0.30 ± 3% perf-profile.self.cycles-pp.resched_curr 0.48 ± 11% +0.1 0.58 ± 3% perf-profile.self.cycles-pp.check_preempt_wakeup 0.39 ± 11% +0.1 0.50 perf-profile.self.cycles-pp.account_entity_enqueue 0.85 ± 7% +0.1 0.97 ± 4% perf-profile.self.cycles-pp.enqueue_entity 0.43 ± 16% +0.1 0.55 ± 7% perf-profile.self.cycles-pp.pick_next_entity 0.86 ± 10% +0.1 0.99 perf-profile.self.cycles-pp.dequeue_task_fair 0.37 ± 21% +0.2 0.55 ± 6% perf-profile.self.cycles-pp.clear_buddies 0.45 ± 14% +0.2 0.65 perf-profile.self.cycles-pp.account_entity_dequeue 1.41 ± 14% +0.2 1.65 ± 4% perf-profile.self.cycles-pp.native_write_msr 0.80 ± 16% +0.3 1.06 ± 4% perf-profile.self.cycles-pp.___perf_sw_event 1.09 ± 9% +0.3 1.39 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.84 ± 16% +0.3 1.14 ± 7% perf-profile.self.cycles-pp.select_task_rq_fair 1.45 ± 12% +0.3 1.75 perf-profile.self.cycles-pp.__enqueue_entity 1.62 ± 8% +0.4 1.99 ± 4% perf-profile.self.cycles-pp.available_idle_cpu 1.83 ± 16% +0.4 2.25 ± 6% perf-profile.self.cycles-pp.switch_fpu_return 2.15 ± 11% +0.5 2.60 ± 5% perf-profile.self.cycles-pp.update_curr 1.88 ± 9% +0.6 2.45 ± 3% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 1.21 ± 9% +0.7 1.92 ± 3% perf-profile.self.cycles-pp._raw_spin_lock 0.80 ± 11% +1.4 2.17 ± 3% perf-profile.self.cycles-pp.select_idle_sibling hackbench.throughput 74000 +-+-----------------------------------------------------------------+ 72000 +-+.+ .+ | |.+ : .+. .+. + | 70000 +-+ : + + +. : | 68000 +-+ + + : | | :: | 66000 +-+ + | 64000 +-+ | 62000 +-+ | | | 60000 +-+ O O OO O O O O O O O OO | 58000 +-+ O O O O O O O O O O O | O O OO O O | 56000 O-+ O O O O O O | 54000 +-O-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-hsw-ep4: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory ========================================================================================= compiler/cpufreq_governor/ipc/iterations/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/18/x86_64-rhel-7.6/threads/1600%/debian-x86_64-2019-11-14.cgz/lkp-hsw-ep4/hackbench/0x43 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 176075 ± 2% -11.3% 156152 hackbench.throughput 891.79 +12.3% 1001 hackbench.time.elapsed_time 891.79 +12.3% 1001 hackbench.time.elapsed_time.max 1.889e+09 ± 9% +33.1% 2.514e+09 ± 3% hackbench.time.involuntary_context_switches 42939 ± 3% +17.0% 50220 hackbench.time.system_time 18978 +3.2% 19592 hackbench.time.user_time 2.558e+09 ± 8% +33.5% 3.415e+09 ± 2% hackbench.time.voluntary_context_switches 43133704 ± 3% +39.2% 60035429 ± 33% cpuidle.C1.time 48661 +17.7% 57274 ± 5% meminfo.Shmem 19043 ± 2% -14.0% 16377 meminfo.max_used_kB 2502 +1.3% 2535 turbostat.Avg_MHz 3.368e+08 ± 5% +30.6% 4.398e+08 turbostat.IRQ 66.25 +4.5% 69.25 vmstat.cpu.sy 29.25 ± 2% -8.5% 26.75 vmstat.cpu.us 5000443 ± 7% +18.6% 5931772 vmstat.system.cs 370173 ± 3% +17.1% 433616 vmstat.system.in 98786923 ± 7% +74.5% 1.724e+08 numa-numastat.node0.local_node 98791606 ± 7% +74.5% 1.724e+08 numa-numastat.node0.numa_hit 4686 ± 99% +297.8% 18642 ± 24% numa-numastat.node0.other_node 1.818e+08 -51.4% 88447082 ± 3% numa-numastat.node1.local_node 1.819e+08 -51.4% 88451833 ± 3% numa-numastat.node1.numa_hit 18694 ± 25% -74.6% 4753 ± 97% numa-numastat.node1.other_node 155316 +1.5% 157632 proc-vmstat.nr_active_anon 99.50 +5.3% 104.75 ± 3% proc-vmstat.nr_dirtied 4440 +4.9% 4656 proc-vmstat.nr_inactive_anon 6283 +1.9% 6399 proc-vmstat.nr_mapped 12248 ± 2% +16.6% 14276 ± 5% proc-vmstat.nr_shmem 98.75 +5.3% 104.00 proc-vmstat.nr_written 155316 +1.5% 157632 proc-vmstat.nr_zone_active_anon 4440 +4.9% 4656 proc-vmstat.nr_zone_inactive_anon 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_hit 2.807e+08 -7.1% 2.609e+08 proc-vmstat.numa_local 2.83e+08 -7.0% 2.632e+08 proc-vmstat.pgalloc_normal 3212744 +2.3% 3286078 proc-vmstat.pgfault 2.828e+08 -7.0% 2.631e+08 proc-vmstat.pgfree 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_active_anon 31864 ± 44% +211.0% 99111 ± 4% numa-vmstat.node0.nr_anon_pages 11183 ± 3% +4212.8% 482321 ± 2% numa-vmstat.node0.nr_kernel_stack 499.50 ± 23% +66.9% 833.75 ± 8% numa-vmstat.node0.nr_page_table_pages 11417 -63.5% 4168 ± 5% numa-vmstat.node0.nr_shmem 15905 ± 4% -12.5% 13919 ± 2% numa-vmstat.node0.nr_slab_reclaimable 17996 ± 3% +587.4% 123708 ± 2% numa-vmstat.node0.nr_slab_unreclaimable 39216 ± 35% +153.3% 99342 ± 4% numa-vmstat.node0.nr_zone_active_anon 52471566 ± 6% +73.3% 90915199 numa-vmstat.node0.numa_hit 52466577 ± 6% +73.2% 90896366 numa-vmstat.node0.numa_local 4990 ± 92% +277.5% 18836 ± 24% numa-vmstat.node0.numa_other 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_active_anon 114545 ± 12% -58.2% 47874 ± 9% numa-vmstat.node1.nr_anon_pages 462075 -96.6% 15717 ± 67% numa-vmstat.node1.nr_kernel_stack 871.50 ± 13% -37.9% 541.00 ± 13% numa-vmstat.node1.nr_page_table_pages 691.25 ± 22% +1359.2% 10086 ± 8% numa-vmstat.node1.nr_shmem 11357 ± 5% +18.1% 13414 ± 4% numa-vmstat.node1.nr_slab_reclaimable 118594 -85.5% 17243 ± 16% numa-vmstat.node1.nr_slab_unreclaimable 114712 ± 12% -50.1% 57234 ± 7% numa-vmstat.node1.nr_zone_active_anon 93308245 ± 2% -49.7% 46921467 ± 4% numa-vmstat.node1.numa_hit 93128224 ± 2% -49.8% 46755145 ± 4% numa-vmstat.node1.numa_local 157762 ± 35% +154.4% 401333 ± 4% numa-meminfo.node0.Active 157721 ± 35% +154.4% 401249 ± 4% numa-meminfo.node0.Active(anon) 128232 ± 44% +212.2% 400337 ± 4% numa-meminfo.node0.AnonPages 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.KReclaimable 11691 ± 3% +4087.3% 489553 numa-meminfo.node0.KernelStack 1170596 ± 4% +104.2% 2390602 numa-meminfo.node0.MemUsed 2003 ± 23% +67.3% 3351 ± 8% numa-meminfo.node0.PageTables 63487 ± 4% -12.0% 55850 ± 2% numa-meminfo.node0.SReclaimable 72301 ± 3% +591.9% 500285 numa-meminfo.node0.SUnreclaim 45749 -63.5% 16679 ± 5% numa-meminfo.node0.Shmem 135789 ± 3% +309.6% 556136 numa-meminfo.node0.Slab 460025 ± 11% -50.2% 228882 ± 7% numa-meminfo.node1.Active 459902 ± 11% -50.2% 228802 ± 7% numa-meminfo.node1.Active(anon) 459237 ± 11% -58.3% 191354 ± 9% numa-meminfo.node1.AnonPages 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.KReclaimable 468955 -96.7% 15461 ± 67% numa-meminfo.node1.KernelStack 2241450 -51.9% 1078199 ± 3% numa-meminfo.node1.MemUsed 3493 ± 13% -38.0% 2167 ± 13% numa-meminfo.node1.PageTables 45568 ± 5% +17.8% 53657 ± 4% numa-meminfo.node1.SReclaimable 479010 -85.6% 68750 ± 16% numa-meminfo.node1.SUnreclaim 2779 ± 23% +1354.1% 40419 ± 8% numa-meminfo.node1.Shmem 524580 -76.7% 122408 ± 8% numa-meminfo.node1.Slab 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.MIN_vruntime.stddev 20766 ± 9% -25.3% 15519 ± 10% sched_debug.cfs_rq:/.load.avg 43820 ± 19% -40.6% 26044 ± 36% sched_debug.cfs_rq:/.load.stddev 250988 ± 19% -52.0% 120407 ± 33% sched_debug.cfs_rq:/.max_vruntime.avg 1469922 ± 14% -32.1% 998119 ± 33% sched_debug.cfs_rq:/.max_vruntime.stddev 10990567 ± 9% +22.9% 13509735 ± 7% sched_debug.cfs_rq:/.min_vruntime.stddev 102.40 -33.3% 68.27 ± 57% sched_debug.cfs_rq:/.removed.load_avg.max 4711 -33.4% 3136 ± 57% sched_debug.cfs_rq:/.removed.runnable_sum.max 19930 ± 10% -25.1% 14917 ± 10% sched_debug.cfs_rq:/.runnable_weight.avg -5713498 -349.0% 14226386 ± 8% sched_debug.cfs_rq:/.spread0.avg 8106715 ± 13% +283.1% 31057182 ± 6% sched_debug.cfs_rq:/.spread0.max -18025645 -99.7% -45393 sched_debug.cfs_rq:/.spread0.min 11027552 ± 9% +21.5% 13398318 ± 7% sched_debug.cfs_rq:/.spread0.stddev 570.46 ± 5% +109.2% 1193 ± 21% sched_debug.cfs_rq:/.util_est_enqueued.avg 2898 ± 11% +90.0% 5507 ± 10% sched_debug.cfs_rq:/.util_est_enqueued.max 563.27 ± 13% +149.3% 1404 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.stddev 155578 ± 22% +84.1% 286457 ± 28% sched_debug.cpu.avg_idle.min 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock.stddev 1678 ± 43% +105.9% 3457 ± 31% sched_debug.cpu.clock_task.stddev 50207 -12.3% 44017 ± 7% sched_debug.cpu.curr->pid.avg 82673 -19.5% 66514 ± 5% sched_debug.cpu.curr->pid.max 746.05 ± 11% +554.3% 4881 ± 8% sched_debug.cpu.curr->pid.min 30728 ± 2% -40.8% 18187 ± 10% sched_debug.cpu.curr->pid.stddev 0.00 ± 43% +106.0% 0.00 ± 31% sched_debug.cpu.next_balance.stddev 20.49 ± 6% +149.0% 51.03 ± 23% sched_debug.cpu.nr_running.avg 157.83 ± 9% +77.0% 279.37 ± 8% sched_debug.cpu.nr_running.max 29.95 ± 11% +137.1% 70.99 ± 16% sched_debug.cpu.nr_running.stddev 29803727 ± 9% +29.6% 38623870 ± 3% sched_debug.cpu.nr_switches.avg 51549476 ± 10% +22.5% 63171598 ± 3% sched_debug.cpu.nr_switches.max 12792570 ± 3% +43.5% 18362678 ± 4% sched_debug.cpu.nr_switches.min 2.03 ± 80% +430.9% 10.76 ± 38% sched_debug.cpu.nr_uninterruptible.avg 6.99 ± 4% +48.7% 10.39 ± 5% perf-stat.i.MPKI 7.49 ± 5% -1.2 6.29 ± 5% perf-stat.i.cache-miss-rate% 27202880 ± 2% +46.9% 39966416 ± 2% perf-stat.i.cache-misses 4.662e+08 ± 5% +71.5% 7.998e+08 ± 5% perf-stat.i.cache-references 5989366 ± 4% +12.9% 6760958 ± 5% perf-stat.i.context-switches 197319 ± 12% -19.2% 159504 ± 3% perf-stat.i.cpu-migrations 7415 ± 2% -23.3% 5686 ± 3% perf-stat.i.cycles-between-cache-misses 0.61 ± 35% -0.3 0.35 ± 4% perf-stat.i.dTLB-load-miss-rate% 1.506e+08 ± 38% -44.9% 82967539 ± 5% perf-stat.i.dTLB-load-misses 0.22 ± 4% -0.0 0.20 ± 4% perf-stat.i.dTLB-store-miss-rate% 35974976 ± 5% -13.4% 31146077 ± 6% perf-stat.i.dTLB-store-misses 1.627e+10 -3.7% 1.566e+10 perf-stat.i.dTLB-stores 46665620 ± 3% -14.6% 39838962 ± 5% perf-stat.i.iTLB-load-misses 2191 ± 5% +21.4% 2660 ± 7% perf-stat.i.instructions-per-iTLB-miss 0.47 -2.5% 0.46 perf-stat.i.ipc 6660 ± 8% -19.1% 5391 ± 8% perf-stat.i.minor-faults 62.93 ± 2% +11.1 74.06 perf-stat.i.node-load-miss-rate% 12537290 ± 4% +73.5% 21750457 ± 4% perf-stat.i.node-load-misses 5459308 ± 3% -7.4% 5057404 ± 3% perf-stat.i.node-loads 29.39 ± 2% -4.1 25.29 ± 2% perf-stat.i.node-store-miss-rate% 2863317 +13.6% 3252632 ± 2% perf-stat.i.node-store-misses 6381925 ± 2% +53.5% 9796918 ± 4% perf-stat.i.node-stores 6638 ± 8% -19.4% 5348 ± 8% perf-stat.i.page-faults 4.89 ± 4% +78.4% 8.73 perf-stat.overall.MPKI 1.29 +0.0 1.31 perf-stat.overall.branch-miss-rate% 6.13 ± 4% -1.3 4.85 ± 3% perf-stat.overall.cache-miss-rate% 2.15 +2.0% 2.19 perf-stat.overall.cpi 7187 -27.9% 5182 ± 3% perf-stat.overall.cycles-between-cache-misses 0.63 ± 39% -0.3 0.36 ± 5% perf-stat.overall.dTLB-load-miss-rate% 0.23 ± 5% -0.0 0.21 ± 3% perf-stat.overall.dTLB-store-miss-rate% 75.65 -1.1 74.59 perf-stat.overall.iTLB-load-miss-rate% 1604 ± 3% +15.9% 1859 perf-stat.overall.instructions-per-iTLB-miss 0.47 -2.0% 0.46 perf-stat.overall.ipc 63.14 ± 2% +13.6 76.70 perf-stat.overall.node-load-miss-rate% 30.74 ± 2% -5.1 25.62 ± 2% perf-stat.overall.node-store-miss-rate% 47754 ± 2% +11.7% 53321 perf-stat.overall.path-length 24893507 +40.6% 34993733 ± 2% perf-stat.ps.cache-misses 4.072e+08 ± 5% +77.1% 7.212e+08 perf-stat.ps.cache-references 4968881 ± 7% +19.0% 5911450 perf-stat.ps.context-switches 1.789e+11 +1.3% 1.812e+11 perf-stat.ps.cpu-cycles 1.59e+08 ± 39% -44.1% 88801351 ± 6% perf-stat.ps.dTLB-load-misses 2.502e+10 -2.0% 2.453e+10 perf-stat.ps.dTLB-loads 39138578 ± 5% -12.6% 34190129 ± 3% perf-stat.ps.dTLB-store-misses 1.675e+10 -3.7% 1.613e+10 perf-stat.ps.dTLB-stores 51901139 ± 2% -14.3% 44461197 perf-stat.ps.iTLB-load-misses 16711264 ± 2% -9.3% 15153636 ± 2% perf-stat.ps.iTLB-loads 3573 -9.1% 3247 perf-stat.ps.minor-faults 10427704 ± 3% +73.2% 18055799 ± 3% perf-stat.ps.node-load-misses 6085553 ± 4% -10.0% 5479559 ± 2% perf-stat.ps.node-loads 2566511 +13.4% 2911367 ± 2% perf-stat.ps.node-store-misses 5784179 ± 2% +46.2% 8454210 ± 3% perf-stat.ps.node-stores 3573 -9.1% 3247 perf-stat.ps.page-faults 7.427e+13 ± 2% +11.7% 8.293e+13 perf-stat.total.instructions 127394 ± 5% +19.1% 151665 softirqs.CPU0.RCU 321761 ± 3% +12.8% 363107 ± 3% softirqs.CPU0.TIMER 124526 ± 5% +20.2% 149697 ± 3% softirqs.CPU1.RCU 123768 ± 4% +24.7% 154332 ± 2% softirqs.CPU10.RCU 130315 ± 3% +22.1% 159111 softirqs.CPU11.RCU 126684 ± 4% +26.1% 159755 ± 2% softirqs.CPU12.RCU 124649 ± 9% +27.8% 159298 ± 3% softirqs.CPU13.RCU 124528 ± 4% +23.2% 153372 ± 2% softirqs.CPU14.RCU 326150 ± 2% +9.8% 357991 ± 4% softirqs.CPU14.TIMER 121923 ± 6% +17.4% 143119 softirqs.CPU15.RCU 122114 ± 5% +16.7% 142560 softirqs.CPU16.RCU 121425 ± 6% +14.4% 138861 ± 3% softirqs.CPU17.RCU 330948 ± 2% +22.2% 404570 ± 16% softirqs.CPU17.TIMER 312317 ± 4% +18.2% 369091 ± 2% softirqs.CPU18.TIMER 9626 ± 2% +15.0% 11067 ± 3% softirqs.CPU19.SCHED 322308 ± 3% +21.1% 390454 ± 2% softirqs.CPU19.TIMER 124281 ± 5% +22.8% 152670 ± 2% softirqs.CPU2.RCU 9452 ± 2% +15.6% 10927 ± 3% softirqs.CPU20.SCHED 318575 ± 6% +22.3% 389741 ± 3% softirqs.CPU20.TIMER 9327 ± 3% +16.7% 10886 ± 3% softirqs.CPU21.SCHED 306628 ± 4% +23.3% 377962 ± 3% softirqs.CPU21.TIMER 9424 ± 2% +17.6% 11084 ± 3% softirqs.CPU22.SCHED 302796 ± 4% +24.2% 376126 ± 2% softirqs.CPU22.TIMER 9798 ± 2% +11.4% 10918 ± 4% softirqs.CPU23.SCHED 312009 ± 4% +23.6% 385626 ± 2% softirqs.CPU23.TIMER 9559 ± 3% +13.2% 10822 softirqs.CPU24.SCHED 311772 ± 4% +23.0% 383504 ± 2% softirqs.CPU24.TIMER 9514 ± 4% +13.2% 10766 ± 3% softirqs.CPU25.SCHED 307640 ± 4% +22.4% 376479 ± 2% softirqs.CPU25.TIMER 9573 ± 3% +13.8% 10897 ± 3% softirqs.CPU26.SCHED 314607 ± 9% +21.1% 380918 ± 2% softirqs.CPU26.TIMER 9385 +13.7% 10668 ± 2% softirqs.CPU27.SCHED 301949 ± 4% +25.2% 378140 ± 3% softirqs.CPU27.TIMER 9483 ± 2% +14.7% 10874 ± 2% softirqs.CPU28.SCHED 299462 ± 3% +24.9% 373896 ± 2% softirqs.CPU28.TIMER 9505 ± 3% +13.3% 10769 ± 3% softirqs.CPU29.SCHED 307087 ± 4% +23.8% 380066 ± 3% softirqs.CPU29.TIMER 124992 ± 4% +19.4% 149253 ± 3% softirqs.CPU3.RCU 9630 ± 4% +13.6% 10942 ± 3% softirqs.CPU30.SCHED 306382 ± 4% +25.4% 384355 ± 2% softirqs.CPU30.TIMER 9496 ± 6% +12.6% 10689 ± 2% softirqs.CPU31.SCHED 308689 ± 4% +24.5% 384419 ± 2% softirqs.CPU31.TIMER 9714 +12.6% 10937 ± 4% softirqs.CPU32.SCHED 304137 ± 3% +35.9% 413240 ± 13% softirqs.CPU32.TIMER 9598 ± 2% +13.4% 10882 ± 5% softirqs.CPU33.SCHED 9622 +13.8% 10949 ± 3% softirqs.CPU34.SCHED 305882 ± 4% +27.3% 389315 ± 4% softirqs.CPU34.TIMER 9409 ± 4% +15.3% 10853 ± 4% softirqs.CPU35.SCHED 306215 ± 4% +23.8% 379222 ± 2% softirqs.CPU35.TIMER 321617 ± 4% +12.9% 362966 ± 3% softirqs.CPU36.TIMER 134843 ± 6% +19.4% 161053 ± 2% softirqs.CPU37.RCU 130954 ± 6% +20.4% 157716 softirqs.CPU38.RCU 129884 ± 5% +17.8% 152978 softirqs.CPU39.RCU 124744 ± 4% +20.1% 149790 ± 3% softirqs.CPU4.RCU 130593 ± 5% +17.2% 153092 ± 2% softirqs.CPU40.RCU 134185 ± 5% +19.9% 160871 softirqs.CPU41.RCU 133336 ± 5% +18.1% 157409 softirqs.CPU42.RCU 134486 ± 4% +18.7% 159613 softirqs.CPU43.RCU 129857 ± 6% +18.5% 153887 softirqs.CPU44.RCU 131816 ± 5% +20.0% 158144 softirqs.CPU45.RCU 132691 ± 5% +22.4% 162412 softirqs.CPU46.RCU 138736 ± 3% +18.7% 164668 softirqs.CPU47.RCU 132971 ± 6% +24.2% 165142 softirqs.CPU48.RCU 136142 ± 3% +20.8% 164493 softirqs.CPU49.RCU 125011 ± 4% +23.6% 154462 ± 2% softirqs.CPU5.RCU 133660 ± 4% +19.5% 159753 ± 2% softirqs.CPU50.RCU 140405 ± 4% +19.5% 167772 softirqs.CPU51.RCU 140523 ± 4% +18.5% 166576 softirqs.CPU52.RCU 138849 ± 5% +14.6% 159157 ± 4% softirqs.CPU53.RCU 332171 ± 2% +21.4% 403260 ± 15% softirqs.CPU53.TIMER 133037 ± 4% +14.0% 151688 ± 2% softirqs.CPU54.RCU 9584 +14.3% 10959 ± 2% softirqs.CPU54.SCHED 310603 ± 4% +18.0% 366472 ± 3% softirqs.CPU54.TIMER 9565 ± 2% +13.8% 10889 ± 5% softirqs.CPU55.SCHED 321231 ± 3% +21.3% 389753 ± 2% softirqs.CPU55.TIMER 136633 ± 4% +12.6% 153854 softirqs.CPU56.RCU 9430 ± 2% +15.6% 10898 ± 5% softirqs.CPU56.SCHED 319477 ± 6% +22.1% 390107 ± 2% softirqs.CPU56.TIMER 128672 ± 4% +14.1% 146791 ± 3% softirqs.CPU57.RCU 9493 ± 2% +13.1% 10740 ± 3% softirqs.CPU57.SCHED 305670 ± 4% +23.5% 377597 ± 2% softirqs.CPU57.TIMER 127129 ± 4% +15.6% 146993 ± 2% softirqs.CPU58.RCU 9529 ± 3% +13.6% 10824 ± 4% softirqs.CPU58.SCHED 302317 ± 4% +23.8% 374371 ± 3% softirqs.CPU58.TIMER 9427 ± 2% +14.0% 10743 ± 2% softirqs.CPU59.SCHED 310985 ± 4% +23.5% 384119 ± 3% softirqs.CPU59.TIMER 124937 ± 4% +21.6% 151962 ± 3% softirqs.CPU6.RCU 9435 ± 2% +14.1% 10762 ± 3% softirqs.CPU60.SCHED 311646 ± 4% +22.8% 382731 ± 3% softirqs.CPU60.TIMER 9585 +13.5% 10877 ± 2% softirqs.CPU61.SCHED 306857 ± 4% +22.8% 376944 ± 2% softirqs.CPU61.TIMER 9483 ± 2% +12.5% 10671 ± 5% softirqs.CPU62.SCHED 9620 ± 4% +9.7% 10556 ± 4% softirqs.CPU63.SCHED 301085 ± 3% +24.7% 375397 ± 3% softirqs.CPU63.TIMER 300443 ± 4% +23.7% 371620 ± 2% softirqs.CPU64.TIMER 9667 +11.2% 10751 ± 4% softirqs.CPU65.SCHED 307194 ± 4% +23.4% 379020 ± 3% softirqs.CPU65.TIMER 9493 ± 3% +14.9% 10909 ± 3% softirqs.CPU66.SCHED 306226 ± 4% +25.1% 383140 ± 3% softirqs.CPU66.TIMER 9674 ± 3% +13.8% 11010 ± 3% softirqs.CPU67.SCHED 307870 ± 4% +25.0% 384793 ± 2% softirqs.CPU67.TIMER 9451 ± 2% +14.2% 10792 ± 4% softirqs.CPU68.SCHED 301873 ± 3% +26.1% 380748 softirqs.CPU68.TIMER 9542 +13.8% 10863 ± 3% softirqs.CPU69.SCHED 316206 ± 8% +20.5% 380893 ± 2% softirqs.CPU69.TIMER 128158 ± 4% +20.4% 154294 ± 2% softirqs.CPU7.RCU 9456 ± 3% +13.5% 10734 ± 3% softirqs.CPU70.SCHED 305361 ± 4% +36.4% 416546 ± 15% softirqs.CPU70.TIMER 9520 ± 3% +15.4% 10985 ± 4% softirqs.CPU71.SCHED 305605 ± 4% +24.6% 380653 ± 2% softirqs.CPU71.TIMER 122831 ± 4% +21.6% 149424 ± 2% softirqs.CPU8.RCU 123870 ± 4% +23.0% 152396 ± 2% softirqs.CPU9.RCU 9357957 ± 5% +15.4% 10794953 softirqs.RCU 23209193 ± 3% +15.5% 26806870 ± 3% softirqs.TIMER 4.79 ± 4% -3.1 1.71 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.85 ± 4% -2.4 1.41 ± 16% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.39 ± 4% -2.0 1.35 ± 16% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 2.93 ± 5% -1.9 0.99 ± 18% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 3.25 ± 4% -1.9 1.31 ± 10% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 2.14 ± 4% -1.2 0.91 ± 14% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 2.06 ± 4% -1.2 0.85 ± 13% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 1.52 ± 3% -1.0 0.49 ± 59% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 1.46 ± 3% -1.0 0.46 ± 59% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.53 ± 8% -0.8 0.68 ± 12% perf-profile.calltrace.cycles-pp.touch_atime.pipe_read.new_sync_read.vfs_read.ksys_read 1.65 ± 4% -0.8 0.81 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.45 ± 5% -0.7 0.74 ± 10% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.35 ± 5% -0.7 0.68 ± 10% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 1.28 ± 8% -0.7 0.62 ± 12% perf-profile.calltrace.cycles-pp.file_update_time.pipe_write.new_sync_write.vfs_write.ksys_write 1.29 ± 3% -0.6 0.67 ± 9% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 0.64 ± 11% +0.3 0.96 ± 7% perf-profile.calltrace.cycles-pp.__lock_text_start.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.00 +0.5 0.55 ± 3% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +0.6 0.55 ± 5% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.6 0.57 ± 6% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.6 0.61 ± 7% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.7 0.69 ± 8% perf-profile.calltrace.cycles-pp.finish_task_switch.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.7 0.70 ± 6% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.71 ± 11% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.7 0.72 ± 7% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.7 0.72 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.8 0.76 ± 10% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.00 +0.8 0.77 ± 2% perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.77 ± 5% perf-profile.calltrace.cycles-pp.__switch_to 0.00 +0.8 0.77 ± 6% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.79 ± 5% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 0.00 +0.8 0.81 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.00 +0.8 0.82 ± 6% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.9 0.90 ± 3% perf-profile.calltrace.cycles-pp.find_next_bit.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up 0.00 +0.9 0.91 ± 4% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +0.9 0.91 ± 6% perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.00 +0.9 0.92 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.00 +1.1 1.05 ± 5% perf-profile.calltrace.cycles-pp.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.1 1.08 ± 8% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.00 +1.1 1.13 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.00 +1.2 1.17 ± 9% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.00 +1.2 1.18 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.00 +1.2 1.20 ± 9% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.3 1.30 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +1.3 1.34 ± 6% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +1.6 1.62 ± 4% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.00 +1.9 1.88 ± 5% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.00 +1.9 1.93 ± 8% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.13 ±173% +2.0 2.11 ± 5% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 0.00 +2.1 2.06 ± 4% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +2.4 2.38 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +2.7 2.65 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.35 ±103% +4.3 4.67 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.22 ±173% +4.6 4.81 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.23 ±173% +4.7 4.97 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.72 ± 23% +5.0 5.72 ± 5% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 0.69 ± 17% +5.6 6.25 ± 4% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.74 ± 17% +5.8 6.50 ± 4% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.74 ± 18% +5.8 6.55 ± 5% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 15.99 ± 54% +7.5 23.51 ± 6% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.13 +7.7 20.86 ± 3% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 11.49 +8.7 20.19 ± 3% perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.31 ± 27% +8.9 10.21 ± 7% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 1.81 ± 22% +11.2 12.98 ± 5% perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 1.89 ± 22% +11.4 13.28 ± 5% perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 2.18 ± 21% +13.1 15.28 ± 4% perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 2.01 ± 25% +19.1 21.10 ± 7% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 2.38 ± 23% +20.0 22.40 ± 7% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 19.01 ± 3% +23.4 42.38 ± 4% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 21.13 ± 53% +23.8 44.92 ± 5% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 17.85 ± 4% +23.9 41.76 ± 4% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 19.16 ± 53% +24.6 43.74 ± 5% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.79 ± 22% +30.7 34.45 ± 6% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.84 ± 23% +30.9 34.69 ± 6% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 4.13 ± 21% +31.0 35.10 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 5.77 ± 14% +31.0 36.76 ± 6% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 9.20 ± 6% -6.0 3.16 ± 12% perf-profile.children.cycles-pp.syscall_return_via_sysret 8.88 ± 3% -5.8 3.03 ± 12% perf-profile.children.cycles-pp.entry_SYSCALL_64 8.72 ± 4% -5.3 3.39 ± 11% perf-profile.children.cycles-pp.security_file_permission 4.27 ± 3% -2.4 1.89 ± 9% perf-profile.children.cycles-pp.selinux_file_permission 3.15 ± 5% -2.1 1.07 ± 12% perf-profile.children.cycles-pp.file_has_perm 3.48 ± 3% -2.0 1.46 ± 10% perf-profile.children.cycles-pp.copy_page_to_iter 3.80 ± 3% -2.0 1.79 ± 6% perf-profile.children.cycles-pp.mutex_unlock 3.02 ± 5% -1.9 1.11 ± 11% perf-profile.children.cycles-pp.copy_page_from_iter 3.31 ± 4% -1.6 1.75 ± 6% perf-profile.children.cycles-pp.__fdget_pos 2.55 ± 3% -1.5 1.02 ± 10% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 3.17 ± 4% -1.5 1.70 ± 6% perf-profile.children.cycles-pp.__fget_light 2.10 ± 7% -1.4 0.71 ± 12% perf-profile.children.cycles-pp.fsnotify 2.52 ± 6% -1.3 1.23 ± 8% perf-profile.children.cycles-pp.__fget 1.77 ± 6% -1.1 0.66 ± 9% perf-profile.children.cycles-pp.avc_has_perm 1.89 ± 5% -1.1 0.78 ± 10% perf-profile.children.cycles-pp.___might_sleep 1.44 ± 4% -1.0 0.48 ± 12% perf-profile.children.cycles-pp.__inode_security_revalidate 1.33 ± 10% -0.9 0.40 ± 16% perf-profile.children.cycles-pp.current_time 1.56 ± 3% -0.9 0.66 ± 9% perf-profile.children.cycles-pp.copyout 1.56 ± 8% -0.8 0.73 ± 7% perf-profile.children.cycles-pp.touch_atime 1.33 ± 4% -0.8 0.53 ± 10% perf-profile.children.cycles-pp.__might_sleep 1.15 ± 9% -0.8 0.36 ± 12% perf-profile.children.cycles-pp.atime_needs_update 1.17 ± 4% -0.7 0.43 ± 13% perf-profile.children.cycles-pp.copyin 1.23 ± 5% -0.7 0.54 ± 9% perf-profile.children.cycles-pp.__fsnotify_parent 1.32 ± 7% -0.6 0.68 ± 6% perf-profile.children.cycles-pp.file_update_time 1.02 ± 3% -0.6 0.38 ± 12% perf-profile.children.cycles-pp.__might_fault 2.54 ± 2% -0.5 2.03 perf-profile.children.cycles-pp.mutex_lock 0.78 ± 4% -0.4 0.36 ± 10% perf-profile.children.cycles-pp.fput_many 0.53 ± 4% -0.4 0.15 ± 21% perf-profile.children.cycles-pp.inode_has_perm 0.44 ± 12% -0.3 0.11 ± 22% perf-profile.children.cycles-pp.iov_iter_init 0.76 ± 4% -0.3 0.44 ± 8% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.53 ± 3% -0.3 0.21 ± 6% perf-profile.children.cycles-pp.rcu_all_qs 0.41 ± 9% -0.3 0.12 ± 15% perf-profile.children.cycles-pp.timespec64_trunc 0.97 -0.3 0.71 ± 3% perf-profile.children.cycles-pp._cond_resched 0.47 ± 7% -0.3 0.21 ± 11% perf-profile.children.cycles-pp.__x64_sys_read 0.45 ± 7% -0.3 0.20 ± 10% perf-profile.children.cycles-pp.__x64_sys_write 0.37 ± 4% -0.2 0.17 ± 16% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.44 ± 12% -0.2 0.24 ± 3% perf-profile.children.cycles-pp.__sb_start_write 0.31 ± 6% -0.2 0.12 ± 13% perf-profile.children.cycles-pp.__sb_end_write 0.36 ± 7% -0.2 0.18 ± 6% perf-profile.children.cycles-pp.rw_verify_area 0.26 ± 14% -0.2 0.09 ± 14% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64 0.29 ± 8% -0.2 0.14 ± 16% perf-profile.children.cycles-pp.fpregs_assert_state_consistent 0.22 ± 39% -0.2 0.06 ± 11% perf-profile.children.cycles-pp.__vfs_read 0.21 ± 19% -0.1 0.11 ± 9% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 0.12 ± 9% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_write 0.12 ± 17% -0.0 0.08 ± 11% perf-profile.children.cycles-pp.wake_up_q 0.08 ± 10% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 7% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.09 ± 5% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.01 ±173% +0.1 0.06 ± 6% perf-profile.children.cycles-pp.__mnt_want_write 0.07 ± 25% +0.1 0.12 ± 27% perf-profile.children.cycles-pp.__softirqentry_text_start 0.08 ± 30% +0.1 0.14 ± 26% perf-profile.children.cycles-pp.irq_exit 0.06 ± 6% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 perf-profile.children.cycles-pp.preempt_schedule_common 0.00 +0.1 0.07 ± 6% perf-profile.children.cycles-pp.is_cpu_allowed 0.03 ±100% +0.1 0.10 ± 8% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 5% perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.01 ±173% +0.1 0.09 ± 32% perf-profile.children.cycles-pp.migrate_task_rq_fair 0.00 +0.1 0.07 ± 14% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.05 ± 61% +0.1 0.13 ± 5% perf-profile.children.cycles-pp.native_irq_return_iret 0.00 +0.1 0.08 ± 8% perf-profile.children.cycles-pp.rcu_note_context_switch 0.00 +0.1 0.08 ± 13% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.native_load_tls 0.03 ±102% +0.1 0.12 ± 30% perf-profile.children.cycles-pp.set_task_cpu 0.00 +0.1 0.09 ± 11% perf-profile.children.cycles-pp.check_cfs_rq_runtime 0.00 +0.1 0.10 ± 5% perf-profile.children.cycles-pp.rb_next 0.10 ± 10% +0.1 0.20 ± 9% perf-profile.children.cycles-pp.generic_update_time 0.00 +0.1 0.10 ± 29% perf-profile.children.cycles-pp.__x64_sys_exit 0.00 +0.1 0.10 ± 28% perf-profile.children.cycles-pp.do_exit 0.00 +0.1 0.11 ± 8% perf-profile.children.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 7% perf-profile.children.cycles-pp.__list_add_valid 0.00 +0.1 0.12 ± 7% perf-profile.children.cycles-pp.resched_curr 0.00 +0.1 0.12 ± 6% perf-profile.children.cycles-pp.switch_mm_irqs_off 0.13 ± 10% +0.1 0.26 ± 4% perf-profile.children.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.15 ± 4% perf-profile.children.cycles-pp.set_next_buddy 0.00 +0.2 0.16 ± 5% perf-profile.children.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.22 ± 7% perf-profile.children.cycles-pp.rb_erase 0.06 ± 13% +0.2 0.23 ± 9% perf-profile.children.cycles-pp.finish_wait 0.17 ± 12% +0.2 0.35 ± 2% perf-profile.children.cycles-pp.anon_pipe_buf_release 0.01 ±173% +0.2 0.20 ± 8% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.00 +0.2 0.21 ± 4% perf-profile.children.cycles-pp.cpumask_next 0.07 ± 22% +0.3 0.40 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.03 ±100% +0.4 0.39 ± 5% perf-profile.children.cycles-pp.update_min_vruntime 0.03 ±100% +0.4 0.39 ± 12% perf-profile.children.cycles-pp.cpus_share_cache 0.06 ± 14% +0.4 0.43 perf-profile.children.cycles-pp.clear_buddies 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.children.cycles-pp.cpuacct_charge 0.10 ± 26% +0.4 0.49 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.32 ± 31% +0.4 0.75 ± 14% perf-profile.children.cycles-pp.sched_ttwu_pending 0.11 ± 26% +0.4 0.56 ± 3% perf-profile.children.cycles-pp.sched_clock 0.34 ± 32% +0.5 0.80 ± 15% perf-profile.children.cycles-pp.scheduler_ipi 0.08 ± 12% +0.5 0.56 ± 7% perf-profile.children.cycles-pp.account_entity_enqueue 0.13 ± 26% +0.5 0.62 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.14 ± 26% +0.6 0.77 ± 4% perf-profile.children.cycles-pp.__calc_delta 0.08 ± 16% +0.7 0.76 ± 3% perf-profile.children.cycles-pp.account_entity_dequeue 0.84 ± 5% +0.7 1.54 ± 5% perf-profile.children.cycles-pp.__lock_text_start 1.61 ± 16% +0.7 2.31 ± 9% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.42 ± 31% +0.7 1.12 ± 13% perf-profile.children.cycles-pp.reschedule_interrupt 0.24 ± 28% +0.9 1.11 perf-profile.children.cycles-pp.native_write_msr 0.14 ± 22% +0.9 1.01 ± 5% perf-profile.children.cycles-pp.check_preempt_wakeup 0.24 ± 32% +0.9 1.12 ± 7% perf-profile.children.cycles-pp.finish_task_switch 0.22 ± 22% +0.9 1.10 ± 3% perf-profile.children.cycles-pp.find_next_bit 0.22 ± 23% +0.9 1.12 ± 3% perf-profile.children.cycles-pp.set_next_entity 0.21 ± 32% +0.9 1.14 ± 3% perf-profile.children.cycles-pp.update_cfs_group 0.16 ± 23% +1.0 1.15 ± 4% perf-profile.children.cycles-pp.check_preempt_curr 0.21 ± 30% +1.0 1.21 ± 2% perf-profile.children.cycles-pp.__switch_to 0.23 ± 21% +1.0 1.24 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.16 ± 38% +1.0 1.20 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 0.15 ± 26% +1.0 1.20 ± 3% perf-profile.children.cycles-pp.pick_next_entity 0.15 ± 20% +1.1 1.21 ± 7% perf-profile.children.cycles-pp.put_prev_entity 0.17 ± 22% +1.1 1.23 ± 4% perf-profile.children.cycles-pp.ttwu_do_wakeup 0.19 ± 10% +1.3 1.52 ± 10% perf-profile.children.cycles-pp.__enqueue_entity 0.24 ± 23% +1.3 1.59 ± 6% perf-profile.children.cycles-pp.__update_load_avg_se 0.28 ± 23% +1.4 1.64 ± 3% perf-profile.children.cycles-pp.switch_fpu_return 0.31 ± 29% +1.6 1.88 ± 5% perf-profile.children.cycles-pp.update_rq_clock 0.44 ± 21% +1.8 2.24 ± 4% perf-profile.children.cycles-pp.cpumask_next_wrap 0.33 ± 20% +1.8 2.15 ± 4% perf-profile.children.cycles-pp.dequeue_entity 23.28 ± 2% +2.0 25.27 ± 2% perf-profile.children.cycles-pp.ksys_read 0.46 ± 25% +2.1 2.57 ± 5% perf-profile.children.cycles-pp.update_load_avg 0.20 ± 21% +2.1 2.34 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.12 ± 11% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.38 ± 23% +2.4 2.80 ± 4% perf-profile.children.cycles-pp.reweight_entity 0.52 ± 18% +2.4 2.96 ± 3% perf-profile.children.cycles-pp.enqueue_entity 0.49 ± 23% +2.6 3.10 ± 4% perf-profile.children.cycles-pp.update_curr 20.92 ± 2% +3.2 24.12 ± 2% perf-profile.children.cycles-pp.vfs_read 0.21 ± 23% +3.4 3.65 ± 7% perf-profile.children.cycles-pp._raw_spin_lock 0.76 ± 24% +3.9 4.67 ± 3% perf-profile.children.cycles-pp.pick_next_task_fair 0.66 ± 34% +4.4 5.06 ± 2% perf-profile.children.cycles-pp.exit_to_usermode_loop 0.81 ± 22% +5.0 5.83 ± 4% perf-profile.children.cycles-pp.dequeue_task_fair 1.01 ± 21% +5.8 6.86 ± 4% perf-profile.children.cycles-pp.enqueue_task_fair 1.06 ± 21% +6.0 7.10 ± 4% perf-profile.children.cycles-pp.activate_task 1.07 ± 21% +6.1 7.15 ± 4% perf-profile.children.cycles-pp.ttwu_do_activate 13.19 +7.8 20.94 ± 3% perf-profile.children.cycles-pp.new_sync_read 11.61 +8.7 20.29 ± 3% perf-profile.children.cycles-pp.pipe_read 1.45 ± 25% +8.9 10.35 ± 7% perf-profile.children.cycles-pp.available_idle_cpu 78.85 ± 2% +9.8 88.66 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 77.90 ± 2% +10.1 88.00 perf-profile.children.cycles-pp.do_syscall_64 2.42 ± 21% +12.9 15.36 ± 4% perf-profile.children.cycles-pp.pipe_wait 2.71 ± 26% +15.4 18.12 ± 4% perf-profile.children.cycles-pp.__schedule 2.76 ± 25% +15.6 18.37 ± 4% perf-profile.children.cycles-pp.schedule 27.59 ± 2% +18.6 46.17 ± 3% perf-profile.children.cycles-pp.ksys_write 2.26 ± 23% +19.0 21.30 ± 7% perf-profile.children.cycles-pp.select_idle_sibling 24.98 ± 2% +19.9 44.88 ± 3% perf-profile.children.cycles-pp.vfs_write 2.48 ± 23% +20.0 22.46 ± 7% perf-profile.children.cycles-pp.select_task_rq_fair 19.06 ± 3% +23.3 42.41 ± 4% perf-profile.children.cycles-pp.new_sync_write 17.98 ± 4% +23.9 41.93 ± 4% perf-profile.children.cycles-pp.pipe_write 4.03 ± 21% +30.6 34.60 ± 6% perf-profile.children.cycles-pp.try_to_wake_up 3.97 ± 22% +30.8 34.74 ± 6% perf-profile.children.cycles-pp.autoremove_wake_function 4.30 ± 20% +30.9 35.19 ± 6% perf-profile.children.cycles-pp.__wake_up_common 6.06 ± 14% +31.2 37.21 ± 5% perf-profile.children.cycles-pp.__wake_up_common_lock 24.41 ± 6% -15.5 8.96 ± 10% perf-profile.self.cycles-pp.do_syscall_64 9.18 ± 6% -6.0 3.14 ± 12% perf-profile.self.cycles-pp.syscall_return_via_sysret 8.63 ± 5% -5.6 3.00 ± 12% perf-profile.self.cycles-pp.entry_SYSCALL_64 3.71 ± 3% -2.0 1.73 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.47 ± 3% -1.5 0.96 ± 10% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 2.76 ± 3% -1.4 1.35 ± 8% perf-profile.self.cycles-pp.selinux_file_permission 2.02 ± 7% -1.3 0.68 ± 12% perf-profile.self.cycles-pp.fsnotify 2.46 ± 6% -1.3 1.20 ± 8% perf-profile.self.cycles-pp.__fget 1.75 ± 6% -1.1 0.64 ± 10% perf-profile.self.cycles-pp.avc_has_perm 1.83 ± 5% -1.1 0.75 ± 10% perf-profile.self.cycles-pp.___might_sleep 1.75 -1.0 0.77 ± 7% perf-profile.self.cycles-pp.pipe_write 1.20 ± 4% -0.7 0.46 ± 10% perf-profile.self.cycles-pp.__might_sleep 1.29 ± 11% -0.7 0.58 ± 10% perf-profile.self.cycles-pp.new_sync_read 1.63 ± 4% -0.7 0.97 ± 5% perf-profile.self.cycles-pp.pipe_read 1.15 ± 5% -0.7 0.49 ± 8% perf-profile.self.cycles-pp.__fsnotify_parent 0.89 ± 8% -0.6 0.27 ± 15% perf-profile.self.cycles-pp.copy_page_from_iter 0.89 ± 4% -0.6 0.29 ± 14% perf-profile.self.cycles-pp.security_file_permission 0.83 ± 5% -0.6 0.24 ± 21% perf-profile.self.cycles-pp.file_has_perm 0.90 ± 2% -0.5 0.41 ± 6% perf-profile.self.cycles-pp.new_sync_write 0.67 ± 10% -0.5 0.20 ± 15% perf-profile.self.cycles-pp.current_time 0.83 ± 4% -0.4 0.41 ± 10% perf-profile.self.cycles-pp.copy_page_to_iter 0.72 ± 7% -0.4 0.31 ± 9% perf-profile.self.cycles-pp.vfs_write 0.75 ± 4% -0.4 0.35 ± 9% perf-profile.self.cycles-pp.fput_many 1.01 ± 4% -0.3 0.69 ± 5% perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe 0.45 ± 4% -0.3 0.14 ± 19% perf-profile.self.cycles-pp.inode_has_perm 0.54 ± 8% -0.3 0.23 ± 9% perf-profile.self.cycles-pp.file_update_time 0.41 ± 10% -0.3 0.11 ± 20% perf-profile.self.cycles-pp.iov_iter_init 0.49 ± 7% -0.3 0.19 ± 8% perf-profile.self.cycles-pp.atime_needs_update 0.44 ± 7% -0.3 0.15 ± 14% perf-profile.self.cycles-pp.ksys_read 0.44 ± 7% -0.3 0.17 ± 9% perf-profile.self.cycles-pp.__inode_security_revalidate 0.41 ± 2% -0.3 0.15 ± 8% perf-profile.self.cycles-pp.rcu_all_qs 0.43 ± 8% -0.3 0.17 ± 11% perf-profile.self.cycles-pp.ksys_write 0.37 ± 9% -0.3 0.11 ± 17% perf-profile.self.cycles-pp.timespec64_trunc 0.42 ± 14% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.39 ± 7% -0.2 0.17 ± 11% perf-profile.self.cycles-pp.__x64_sys_write 0.40 ± 8% -0.2 0.17 ± 8% perf-profile.self.cycles-pp.__x64_sys_read 0.41 ± 7% -0.2 0.20 ± 11% perf-profile.self.cycles-pp.__wake_up_common_lock 0.60 ± 4% -0.2 0.39 ± 7% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 1.25 -0.2 1.04 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.30 ± 6% -0.2 0.12 ± 15% perf-profile.self.cycles-pp.__sb_end_write 0.35 ± 5% -0.2 0.18 ± 3% perf-profile.self.cycles-pp.rw_verify_area 0.34 ± 7% -0.2 0.18 ± 8% perf-profile.self.cycles-pp.touch_atime 0.32 ± 4% -0.2 0.15 ± 14% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.23 ± 16% -0.2 0.08 ± 11% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.74 ± 3% -0.2 0.59 ± 5% perf-profile.self.cycles-pp.vfs_read 0.60 ± 3% -0.2 0.45 ± 4% perf-profile.self.cycles-pp.__fget_light 0.27 ± 8% -0.1 0.12 ± 15% perf-profile.self.cycles-pp.fpregs_assert_state_consistent 0.18 ± 45% -0.1 0.04 ± 58% perf-profile.self.cycles-pp.__vfs_read 0.18 ± 5% -0.1 0.06 ± 17% perf-profile.self.cycles-pp.__fdget_pos 0.25 ± 5% -0.1 0.14 ± 11% perf-profile.self.cycles-pp.__might_fault 0.07 ± 7% -0.0 0.04 ± 57% perf-profile.self.cycles-pp.kill_fasync 0.01 ±173% +0.1 0.06 ± 17% perf-profile.self.cycles-pp.sched_ttwu_pending 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.scheduler_ipi 0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.rcu_note_context_switch 0.01 ±173% +0.1 0.07 ± 12% perf-profile.self.cycles-pp.generic_update_time 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.cpumask_next 0.00 +0.1 0.06 ± 11% perf-profile.self.cycles-pp.ttwu_do_activate 0.00 +0.1 0.06 perf-profile.self.cycles-pp.__mnt_want_write 0.06 ± 14% +0.1 0.12 ± 10% perf-profile.self.cycles-pp.__mark_inode_dirty 0.00 +0.1 0.06 ± 13% perf-profile.self.cycles-pp.is_cpu_allowed 0.00 +0.1 0.07 ± 13% perf-profile.self.cycles-pp.sched_clock_cpu 0.00 +0.1 0.07 ± 6% perf-profile.self.cycles-pp.sched_clock 0.03 ±100% +0.1 0.10 ± 11% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.00 +0.1 0.08 perf-profile.self.cycles-pp.native_load_tls 0.05 ± 60% +0.1 0.13 ± 5% perf-profile.self.cycles-pp.native_irq_return_iret 0.00 +0.1 0.09 ± 9% perf-profile.self.cycles-pp.ttwu_do_wakeup 0.00 +0.1 0.09 ± 11% perf-profile.self.cycles-pp.rb_next 0.00 +0.1 0.10 ± 8% perf-profile.self.cycles-pp.__list_add_valid 0.00 +0.1 0.10 ± 10% perf-profile.self.cycles-pp.rb_insert_color 0.00 +0.1 0.11 ± 9% perf-profile.self.cycles-pp.resched_curr 0.00 +0.1 0.11 ± 7% perf-profile.self.cycles-pp.switch_mm_irqs_off 0.32 ± 7% +0.1 0.45 ± 5% perf-profile.self.cycles-pp.__wake_up_common 0.13 ± 10% +0.1 0.25 ± 5% perf-profile.self.cycles-pp.__list_del_entry_valid 0.00 +0.1 0.13 ± 5% perf-profile.self.cycles-pp.autoremove_wake_function 0.00 +0.1 0.13 ± 8% perf-profile.self.cycles-pp.check_preempt_curr 0.04 ± 58% +0.1 0.18 ± 10% perf-profile.self.cycles-pp.finish_wait 0.00 +0.1 0.14 ± 5% perf-profile.self.cycles-pp.set_next_buddy 0.00 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.put_prev_entity 0.00 +0.2 0.15 ± 5% perf-profile.self.cycles-pp.deactivate_task 0.05 ± 9% +0.2 0.21 ± 5% perf-profile.self.cycles-pp.rb_erase 0.00 +0.2 0.15 ± 10% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.00 +0.2 0.18 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.15 ± 7% +0.2 0.34 ± 2% perf-profile.self.cycles-pp.anon_pipe_buf_release 0.03 ±100% +0.2 0.21 ± 8% perf-profile.self.cycles-pp.pipe_wait 0.03 ±102% +0.2 0.24 ± 3% perf-profile.self.cycles-pp.activate_task 0.01 ±173% +0.2 0.23 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.03 ±100% +0.2 0.25 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.02 ±173% +0.2 0.26 ± 9% perf-profile.self.cycles-pp.dequeue_entity 0.04 ±100% +0.3 0.29 ± 5% perf-profile.self.cycles-pp.enqueue_entity 0.08 ± 31% +0.3 0.34 ± 2% perf-profile.self.cycles-pp.schedule 0.78 ± 3% +0.3 1.09 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.09 ± 24% +0.3 0.42 ± 7% perf-profile.self.cycles-pp.finish_task_switch 0.03 ±100% +0.3 0.37 ± 6% perf-profile.self.cycles-pp.update_min_vruntime 0.03 ±102% +0.4 0.38 perf-profile.self.cycles-pp.clear_buddies 0.03 ±100% +0.4 0.39 ± 12% perf-profile.self.cycles-pp.cpus_share_cache 0.07 ± 22% +0.4 0.44 ± 7% perf-profile.self.cycles-pp.check_preempt_wakeup 0.09 ± 27% +0.4 0.47 ± 4% perf-profile.self.cycles-pp.native_sched_clock 0.07 ± 26% +0.4 0.45 ± 4% perf-profile.self.cycles-pp.pick_next_entity 0.11 ± 15% +0.4 0.50 ± 3% perf-profile.self.cycles-pp.cpuacct_charge 0.07 ± 14% +0.5 0.53 ± 8% perf-profile.self.cycles-pp.account_entity_enqueue 0.10 ± 21% +0.5 0.56 ± 8% perf-profile.self.cycles-pp.try_to_wake_up 0.10 ± 18% +0.5 0.62 ± 6% perf-profile.self.cycles-pp.dequeue_task_fair 0.14 ± 26% +0.6 0.75 ± 4% perf-profile.self.cycles-pp.__calc_delta 0.06 ± 13% +0.6 0.68 ± 4% perf-profile.self.cycles-pp.account_entity_dequeue 0.15 ± 27% +0.6 0.79 ± 6% perf-profile.self.cycles-pp.enqueue_task_fair 0.17 ± 30% +0.7 0.89 ± 4% perf-profile.self.cycles-pp.update_load_avg 0.15 ± 20% +0.7 0.88 ± 4% perf-profile.self.cycles-pp.reweight_entity 0.15 ± 23% +0.8 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.24 ± 30% +0.9 1.11 perf-profile.self.cycles-pp.native_write_msr 0.20 ± 23% +0.9 1.07 ± 5% perf-profile.self.cycles-pp.select_task_rq_fair 0.20 ± 22% +0.9 1.07 ± 3% perf-profile.self.cycles-pp.find_next_bit 0.20 ± 31% +0.9 1.11 ± 3% perf-profile.self.cycles-pp.update_cfs_group 0.20 ± 28% +0.9 1.14 ± 2% perf-profile.self.cycles-pp.__switch_to 0.14 ± 37% +1.0 1.13 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 0.21 ± 24% +1.0 1.22 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.24 ± 23% +1.1 1.31 ± 5% perf-profile.self.cycles-pp.cpumask_next_wrap 0.23 ± 32% +1.3 1.49 ± 5% perf-profile.self.cycles-pp.update_rq_clock 0.19 ± 28% +1.3 1.45 ± 6% perf-profile.self.cycles-pp.update_curr 0.23 ± 23% +1.3 1.55 ± 6% perf-profile.self.cycles-pp.__update_load_avg_se 0.19 ± 10% +1.3 1.51 ± 10% perf-profile.self.cycles-pp.__enqueue_entity 0.28 ± 22% +1.3 1.63 ± 3% perf-profile.self.cycles-pp.switch_fpu_return 0.11 ± 19% +1.6 1.72 ± 5% perf-profile.self.cycles-pp._raw_spin_lock 0.42 ± 26% +1.7 2.12 ± 2% perf-profile.self.cycles-pp.__schedule 0.19 ± 20% +2.0 2.20 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.91 ± 29% +2.2 3.11 ± 12% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.32 ± 19% +7.7 8.05 ± 7% perf-profile.self.cycles-pp.select_idle_sibling 1.44 ± 25% +8.8 10.23 ± 7% perf-profile.self.cycles-pp.available_idle_cpu 439.00 +11.7% 490.50 interrupts.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 436.25 +13.1% 493.25 interrupts.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 436.25 +12.4% 490.50 interrupts.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 436.25 +12.4% 490.50 interrupts.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 509.75 ± 3% +545.1% 3288 ±120% interrupts.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 588.00 ± 35% +196.0% 1740 ± 50% interrupts.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 436.75 ± 2% +12.8% 492.50 interrupts.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 438.00 +39.5% 611.00 ± 33% interrupts.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 436.25 +13.8% 496.25 interrupts.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 436.25 +14.3% 498.50 interrupts.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 437.00 +12.2% 490.50 interrupts.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 436.25 +12.4% 490.50 interrupts.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 436.25 +12.4% 490.50 interrupts.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 441.00 +11.8% 493.00 interrupts.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 437.25 +13.3% 495.25 interrupts.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 436.25 +12.4% 490.50 interrupts.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 436.25 +12.4% 490.50 interrupts.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 436.25 +12.4% 490.50 interrupts.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 436.25 +12.4% 490.50 interrupts.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 437.25 ± 2% +12.2% 490.50 interrupts.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 436.25 +12.6% 491.25 interrupts.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 440.75 +12.4% 495.25 ± 2% interrupts.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 436.25 +12.6% 491.00 interrupts.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 436.75 +13.1% 493.75 interrupts.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 436.25 +12.8% 492.25 interrupts.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 436.25 +12.8% 492.25 interrupts.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 436.25 +12.4% 490.50 interrupts.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 436.25 +12.4% 490.50 interrupts.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 436.25 +12.4% 490.50 interrupts.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 436.25 +14.6% 499.75 ± 2% interrupts.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 436.25 +12.4% 490.50 interrupts.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 436.50 +12.4% 490.50 interrupts.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 436.25 +12.4% 490.50 interrupts.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 436.25 +12.4% 490.50 interrupts.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 436.25 +12.6% 491.00 interrupts.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 436.25 +12.4% 490.50 interrupts.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 436.25 +14.4% 499.25 ± 3% interrupts.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 439.25 ± 2% +11.7% 490.50 interrupts.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 437.75 +12.1% 490.50 interrupts.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 436.50 +12.4% 490.50 interrupts.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 436.25 +12.4% 490.50 interrupts.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 439.00 ± 2% +11.7% 490.50 interrupts.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 442.25 ± 3% +10.9% 490.50 interrupts.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 1989 +12.1% 2230 interrupts.9:IR-IO-APIC.9-fasteoi.acpi 6596036 ± 10% -21.1% 5206443 ± 6% interrupts.CAL:Function_call_interrupts 86097 ± 13% -18.4% 70229 ± 7% interrupts.CPU0.CAL:Function_call_interrupts 1785369 +12.4% 2006294 interrupts.CPU0.LOC:Local_timer_interrupts 102823 ± 12% -18.3% 84031 ± 5% interrupts.CPU0.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496548 ± 4% interrupts.CPU0.TRM:Thermal_event_interrupts 1989 +12.1% 2230 interrupts.CPU1.9:IR-IO-APIC.9-fasteoi.acpi 90370 ± 8% -21.4% 70995 ± 6% interrupts.CPU1.CAL:Function_call_interrupts 1784834 +12.4% 2006046 interrupts.CPU1.LOC:Local_timer_interrupts 3088391 ± 15% -20.9% 2441882 ± 7% interrupts.CPU1.RES:Rescheduling_interrupts 105391 ± 8% -18.6% 85793 ± 6% interrupts.CPU1.TLB:TLB_shootdowns 388373 ± 12% +27.9% 496558 ± 4% interrupts.CPU1.TRM:Thermal_event_interrupts 88236 ± 11% -19.5% 71001 ± 7% interrupts.CPU10.CAL:Function_call_interrupts 1785471 +12.3% 2004740 interrupts.CPU10.LOC:Local_timer_interrupts 3040774 ± 4% -21.1% 2398396 ± 8% interrupts.CPU10.RES:Rescheduling_interrupts 104483 ± 10% -18.6% 85025 ± 5% interrupts.CPU10.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496560 ± 4% interrupts.CPU10.TRM:Thermal_event_interrupts 1785039 +12.3% 2005274 interrupts.CPU11.LOC:Local_timer_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.NMI:Non-maskable_interrupts 925.75 ±173% +461.3% 5196 ± 58% interrupts.CPU11.PMI:Performance_monitoring_interrupts 3094193 ± 14% -21.5% 2430228 ± 8% interrupts.CPU11.RES:Rescheduling_interrupts 102398 ± 10% -16.1% 85884 ± 5% interrupts.CPU11.TLB:TLB_shootdowns 388413 ± 12% +27.8% 496474 ± 4% interrupts.CPU11.TRM:Thermal_event_interrupts 588.00 ± 35% +196.0% 1740 ± 50% interrupts.CPU12.51:IR-PCI-MSI.1572876-edge.eth0-TxRx-12 87438 ± 9% -18.1% 71580 ± 7% interrupts.CPU12.CAL:Function_call_interrupts 1785376 +12.3% 2005068 interrupts.CPU12.LOC:Local_timer_interrupts 103057 ± 10% -16.4% 86186 ± 6% interrupts.CPU12.TLB:TLB_shootdowns 388416 ± 12% +27.8% 496490 ± 4% interrupts.CPU12.TRM:Thermal_event_interrupts 88735 ± 9% -19.4% 71496 ± 9% interrupts.CPU13.CAL:Function_call_interrupts 1785460 +12.3% 2005445 interrupts.CPU13.LOC:Local_timer_interrupts 104230 ± 9% -17.5% 86038 ± 6% interrupts.CPU13.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU13.TRM:Thermal_event_interrupts 88327 ± 12% -19.0% 71507 ± 7% interrupts.CPU14.CAL:Function_call_interrupts 1785094 +12.3% 2005441 interrupts.CPU14.LOC:Local_timer_interrupts 103317 ± 11% -18.1% 84581 ± 6% interrupts.CPU14.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496564 ± 4% interrupts.CPU14.TRM:Thermal_event_interrupts 86972 ± 10% -17.3% 71898 ± 7% interrupts.CPU15.CAL:Function_call_interrupts 1784668 +12.4% 2005375 interrupts.CPU15.LOC:Local_timer_interrupts 3039799 ± 9% -21.1% 2397505 ± 9% interrupts.CPU15.RES:Rescheduling_interrupts 103367 ± 12% -16.9% 85892 ± 6% interrupts.CPU15.TLB:TLB_shootdowns 388342 ± 12% +27.9% 496559 ± 4% interrupts.CPU15.TRM:Thermal_event_interrupts 436.75 ± 2% +12.8% 492.50 interrupts.CPU16.55:IR-PCI-MSI.1572880-edge.eth0-TxRx-16 1784750 +12.4% 2005330 interrupts.CPU16.LOC:Local_timer_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.NMI:Non-maskable_interrupts 55.25 ±171% +7696.8% 4307 ± 64% interrupts.CPU16.PMI:Performance_monitoring_interrupts 2969996 ± 10% -18.7% 2413141 ± 9% interrupts.CPU16.RES:Rescheduling_interrupts 102367 ± 11% -13.7% 88305 ± 3% interrupts.CPU16.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496563 ± 4% interrupts.CPU16.TRM:Thermal_event_interrupts 1785363 +12.3% 2005849 interrupts.CPU17.LOC:Local_timer_interrupts 388418 ± 12% +27.8% 496560 ± 4% interrupts.CPU17.TRM:Thermal_event_interrupts 438.00 +39.5% 611.00 ± 33% interrupts.CPU18.57:IR-PCI-MSI.1572882-edge.eth0-TxRx-18 88992 ± 12% -22.3% 69145 ± 6% interrupts.CPU18.CAL:Function_call_interrupts 1785398 +12.1% 2001316 interrupts.CPU18.LOC:Local_timer_interrupts 2199105 ± 7% +115.2% 4731679 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts 108465 ± 8% -23.0% 83467 ± 5% interrupts.CPU18.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380131 ± 2% interrupts.CPU18.TRM:Thermal_event_interrupts 436.25 +13.8% 496.25 interrupts.CPU19.58:IR-PCI-MSI.1572883-edge.eth0-TxRx-19 92059 ± 6% -24.2% 69776 ± 5% interrupts.CPU19.CAL:Function_call_interrupts 1784665 +12.2% 2001579 interrupts.CPU19.LOC:Local_timer_interrupts 1284622 ± 7% +225.8% 4185208 ± 4% interrupts.CPU19.RES:Rescheduling_interrupts 108492 ± 7% -23.4% 83059 ± 4% interrupts.CPU19.TLB:TLB_shootdowns 329166 ± 3% +15.5% 380141 ± 2% interrupts.CPU19.TRM:Thermal_event_interrupts 1784512 +12.4% 2005572 interrupts.CPU2.LOC:Local_timer_interrupts 3091132 ± 8% -22.3% 2401342 ± 4% interrupts.CPU2.RES:Rescheduling_interrupts 102562 ± 9% -16.5% 85643 ± 5% interrupts.CPU2.TLB:TLB_shootdowns 388380 ± 12% +27.8% 496544 ± 4% interrupts.CPU2.TRM:Thermal_event_interrupts 436.25 +14.3% 498.50 interrupts.CPU20.59:IR-PCI-MSI.1572884-edge.eth0-TxRx-20 91785 ± 9% -23.9% 69853 ± 4% interrupts.CPU20.CAL:Function_call_interrupts 1785284 +12.1% 2001952 interrupts.CPU20.LOC:Local_timer_interrupts 1281621 ± 12% +203.6% 3890516 ± 7% interrupts.CPU20.RES:Rescheduling_interrupts 106297 ± 9% -19.9% 85180 ± 3% interrupts.CPU20.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU20.TRM:Thermal_event_interrupts 437.00 +12.2% 490.50 interrupts.CPU21.60:IR-PCI-MSI.1572885-edge.eth0-TxRx-21 92635 ± 7% -23.9% 70512 ± 6% interrupts.CPU21.CAL:Function_call_interrupts 1785480 +12.1% 2001723 interrupts.CPU21.LOC:Local_timer_interrupts 1595447 ± 6% +187.8% 4591004 interrupts.CPU21.RES:Rescheduling_interrupts 107575 ± 7% -21.6% 84296 ± 5% interrupts.CPU21.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380086 ± 2% interrupts.CPU21.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU22.61:IR-PCI-MSI.1572886-edge.eth0-TxRx-22 92674 ± 10% -24.4% 70047 ± 6% interrupts.CPU22.CAL:Function_call_interrupts 1784881 +12.1% 2001482 interrupts.CPU22.LOC:Local_timer_interrupts 1605143 ± 6% +179.5% 4486945 ± 5% interrupts.CPU22.RES:Rescheduling_interrupts 108749 ± 10% -23.3% 83409 ± 5% interrupts.CPU22.TLB:TLB_shootdowns 329295 ± 3% +15.4% 380136 ± 2% interrupts.CPU22.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU23.62:IR-PCI-MSI.1572887-edge.eth0-TxRx-23 90975 ± 11% -23.6% 69492 ± 5% interrupts.CPU23.CAL:Function_call_interrupts 1785236 +12.1% 2001811 interrupts.CPU23.LOC:Local_timer_interrupts 1600988 ± 2% +180.2% 4486667 ± 5% interrupts.CPU23.RES:Rescheduling_interrupts 106819 ± 10% -22.3% 83007 ± 4% interrupts.CPU23.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380146 ± 2% interrupts.CPU23.TRM:Thermal_event_interrupts 441.00 +11.8% 493.00 interrupts.CPU24.63:IR-PCI-MSI.1572888-edge.eth0-TxRx-24 91919 ± 11% -23.1% 70689 ± 5% interrupts.CPU24.CAL:Function_call_interrupts 1784996 +12.1% 2000922 interrupts.CPU24.LOC:Local_timer_interrupts 1612237 ± 5% +188.4% 4649702 ± 2% interrupts.CPU24.RES:Rescheduling_interrupts 108187 ± 8% -22.5% 83831 ± 3% interrupts.CPU24.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380122 ± 2% interrupts.CPU24.TRM:Thermal_event_interrupts 437.25 +13.3% 495.25 interrupts.CPU25.64:IR-PCI-MSI.1572889-edge.eth0-TxRx-25 88417 ± 12% -20.8% 70029 ± 4% interrupts.CPU25.CAL:Function_call_interrupts 1785404 +12.0% 1999622 interrupts.CPU25.LOC:Local_timer_interrupts 1612074 ± 2% +182.4% 4551699 interrupts.CPU25.RES:Rescheduling_interrupts 105324 ± 10% -20.1% 84201 ± 4% interrupts.CPU25.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU25.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU26.65:IR-PCI-MSI.1572890-edge.eth0-TxRx-26 92632 ± 12% -23.5% 70907 ± 5% interrupts.CPU26.CAL:Function_call_interrupts 1784730 +12.1% 2000264 interrupts.CPU26.LOC:Local_timer_interrupts 1639148 ± 6% +188.0% 4720149 ± 4% interrupts.CPU26.RES:Rescheduling_interrupts 108099 ± 11% -21.6% 84727 ± 4% interrupts.CPU26.TLB:TLB_shootdowns 329271 ± 3% +15.4% 379984 ± 2% interrupts.CPU26.TRM:Thermal_event_interrupts 91312 ± 12% -23.4% 69963 ± 6% interrupts.CPU27.CAL:Function_call_interrupts 1785171 +12.1% 2001743 interrupts.CPU27.LOC:Local_timer_interrupts 1704847 ± 11% +179.4% 4764019 ± 4% interrupts.CPU27.RES:Rescheduling_interrupts 106515 ± 11% -21.6% 83491 ± 5% interrupts.CPU27.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380080 ± 2% interrupts.CPU27.TRM:Thermal_event_interrupts 89178 ± 8% -22.0% 69565 ± 6% interrupts.CPU28.CAL:Function_call_interrupts 1785194 +12.1% 2001836 interrupts.CPU28.LOC:Local_timer_interrupts 1593025 ± 8% +185.7% 4551782 ± 5% interrupts.CPU28.RES:Rescheduling_interrupts 104408 ± 10% -19.2% 84365 ± 6% interrupts.CPU28.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380142 ± 2% interrupts.CPU28.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU29.68:IR-PCI-MSI.1572893-edge.eth0-TxRx-29 90850 ± 12% -21.9% 70993 ± 6% interrupts.CPU29.CAL:Function_call_interrupts 1785398 +12.1% 2000545 interrupts.CPU29.LOC:Local_timer_interrupts 1657508 ± 5% +179.4% 4631109 ± 3% interrupts.CPU29.RES:Rescheduling_interrupts 106383 ± 13% -20.8% 84284 ± 4% interrupts.CPU29.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380133 ± 2% interrupts.CPU29.TRM:Thermal_event_interrupts 509.75 ± 3% +545.1% 3288 ±120% interrupts.CPU3.42:IR-PCI-MSI.1572867-edge.eth0-TxRx-3 87008 ± 11% -19.9% 69730 ± 5% interrupts.CPU3.CAL:Function_call_interrupts 1784446 +12.4% 2005368 interrupts.CPU3.LOC:Local_timer_interrupts 3169674 ± 9% -20.4% 2521963 ± 8% interrupts.CPU3.RES:Rescheduling_interrupts 103703 ± 12% -17.1% 85958 ± 7% interrupts.CPU3.TLB:TLB_shootdowns 388356 ± 12% +27.9% 496551 ± 4% interrupts.CPU3.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU30.69:IR-PCI-MSI.1572894-edge.eth0-TxRx-30 92043 ± 12% -24.2% 69765 ± 5% interrupts.CPU30.CAL:Function_call_interrupts 1785175 +12.1% 2001248 interrupts.CPU30.LOC:Local_timer_interrupts 1637465 ± 6% +191.3% 4769701 ± 3% interrupts.CPU30.RES:Rescheduling_interrupts 106836 ± 11% -22.5% 82810 ± 5% interrupts.CPU30.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU30.TRM:Thermal_event_interrupts 1785301 +12.0% 1999270 interrupts.CPU31.LOC:Local_timer_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.NMI:Non-maskable_interrupts 62.75 ±173% +4261.4% 2736 ±113% interrupts.CPU31.PMI:Performance_monitoring_interrupts 1589500 ± 5% +184.1% 4515325 interrupts.CPU31.RES:Rescheduling_interrupts 106576 ± 11% -21.5% 83698 ± 5% interrupts.CPU31.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380131 ± 2% interrupts.CPU31.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU32.71:IR-PCI-MSI.1572896-edge.eth0-TxRx-32 93348 ± 12% -24.4% 70595 ± 8% interrupts.CPU32.CAL:Function_call_interrupts 1785245 +12.1% 2001145 interrupts.CPU32.LOC:Local_timer_interrupts 1626883 ± 6% +176.3% 4494456 ± 5% interrupts.CPU32.RES:Rescheduling_interrupts 107151 ± 12% -21.4% 84236 ± 5% interrupts.CPU32.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380131 ± 2% interrupts.CPU32.TRM:Thermal_event_interrupts 437.25 ± 2% +12.2% 490.50 interrupts.CPU33.74:IR-PCI-MSI.1572897-edge.eth0-TxRx-33 91487 ± 12% -22.7% 70758 ± 3% interrupts.CPU33.CAL:Function_call_interrupts 1785371 +12.0% 1999987 interrupts.CPU33.LOC:Local_timer_interrupts 1578761 ± 6% +201.2% 4754762 ± 6% interrupts.CPU33.RES:Rescheduling_interrupts 106071 ± 12% -21.3% 83430 ± 2% interrupts.CPU33.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380130 ± 2% interrupts.CPU33.TRM:Thermal_event_interrupts 436.25 +12.6% 491.25 interrupts.CPU34.75:IR-PCI-MSI.1572898-edge.eth0-TxRx-34 91116 ± 11% -24.3% 68943 ± 7% interrupts.CPU34.CAL:Function_call_interrupts 1785353 +12.1% 2001525 interrupts.CPU34.LOC:Local_timer_interrupts 1553828 ± 4% +186.0% 4444035 ± 3% interrupts.CPU34.RES:Rescheduling_interrupts 105575 ± 10% -20.7% 83762 ± 4% interrupts.CPU34.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380139 ± 2% interrupts.CPU34.TRM:Thermal_event_interrupts 440.75 +12.4% 495.25 ± 2% interrupts.CPU35.76:IR-PCI-MSI.1572899-edge.eth0-TxRx-35 91396 ± 11% -25.6% 67986 ± 11% interrupts.CPU35.CAL:Function_call_interrupts 1785237 +12.0% 2000288 interrupts.CPU35.LOC:Local_timer_interrupts 1602616 ± 10% +189.7% 4642115 ± 3% interrupts.CPU35.RES:Rescheduling_interrupts 107669 ± 12% -21.3% 84736 ± 4% interrupts.CPU35.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380145 ± 2% interrupts.CPU35.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU36.77:IR-PCI-MSI.1572900-edge.eth0-TxRx-36 93553 ± 8% -19.8% 75044 ± 6% interrupts.CPU36.CAL:Function_call_interrupts 1784636 +12.4% 2005389 interrupts.CPU36.LOC:Local_timer_interrupts 2588984 ± 6% +25.8% 3256348 ± 6% interrupts.CPU36.RES:Rescheduling_interrupts 107053 ± 7% -19.0% 86754 ± 5% interrupts.CPU36.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496560 ± 4% interrupts.CPU36.TRM:Thermal_event_interrupts 436.75 +13.1% 493.75 interrupts.CPU37.78:IR-PCI-MSI.1572901-edge.eth0-TxRx-37 89518 ± 10% -18.9% 72619 ± 5% interrupts.CPU37.CAL:Function_call_interrupts 1784648 +12.4% 2005651 interrupts.CPU37.LOC:Local_timer_interrupts 3112476 ± 10% -20.8% 2464103 ± 5% interrupts.CPU37.RES:Rescheduling_interrupts 102576 ± 10% -14.4% 87794 ± 6% interrupts.CPU37.TLB:TLB_shootdowns 388412 ± 12% +27.8% 496563 ± 4% interrupts.CPU37.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU38.79:IR-PCI-MSI.1572902-edge.eth0-TxRx-38 92566 ± 10% -20.6% 73466 ± 6% interrupts.CPU38.CAL:Function_call_interrupts 1785132 +12.4% 2005883 interrupts.CPU38.LOC:Local_timer_interrupts 3163520 ± 10% -26.0% 2340820 ± 11% interrupts.CPU38.RES:Rescheduling_interrupts 105823 ± 10% -18.0% 86785 ± 4% interrupts.CPU38.TLB:TLB_shootdowns 388401 ± 12% +27.8% 496554 ± 4% interrupts.CPU38.TRM:Thermal_event_interrupts 436.25 +12.8% 492.25 interrupts.CPU39.80:IR-PCI-MSI.1572903-edge.eth0-TxRx-39 93197 ± 11% -21.8% 72854 ± 7% interrupts.CPU39.CAL:Function_call_interrupts 1784624 +12.4% 2005934 interrupts.CPU39.LOC:Local_timer_interrupts 105953 ± 11% -20.8% 83925 ± 6% interrupts.CPU39.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496548 ± 4% interrupts.CPU39.TRM:Thermal_event_interrupts 86883 ± 13% -17.8% 71449 ± 7% interrupts.CPU4.CAL:Function_call_interrupts 1784433 +12.4% 2005462 interrupts.CPU4.LOC:Local_timer_interrupts 3119739 ± 9% -21.1% 2461192 ± 4% interrupts.CPU4.RES:Rescheduling_interrupts 104829 ± 11% -19.1% 84802 ± 5% interrupts.CPU4.TLB:TLB_shootdowns 388417 ± 12% +27.8% 496563 ± 4% interrupts.CPU4.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU40.81:IR-PCI-MSI.1572904-edge.eth0-TxRx-40 91881 ± 12% -21.4% 72188 ± 8% interrupts.CPU40.CAL:Function_call_interrupts 1784831 +12.4% 2005991 interrupts.CPU40.LOC:Local_timer_interrupts 3101288 ± 10% -17.2% 2569221 ± 7% interrupts.CPU40.RES:Rescheduling_interrupts 105884 ± 10% -19.0% 85772 ± 5% interrupts.CPU40.TLB:TLB_shootdowns 388415 ± 12% +27.8% 496564 ± 4% interrupts.CPU40.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU41.82:IR-PCI-MSI.1572905-edge.eth0-TxRx-41 91739 ± 12% -19.9% 73475 ± 7% interrupts.CPU41.CAL:Function_call_interrupts 1785231 +12.3% 2005398 interrupts.CPU41.LOC:Local_timer_interrupts 3223405 ± 11% -27.9% 2325498 ± 5% interrupts.CPU41.RES:Rescheduling_interrupts 104410 ± 11% -18.7% 84846 ± 6% interrupts.CPU41.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU41.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU42.83:IR-PCI-MSI.1572906-edge.eth0-TxRx-42 1785354 +12.4% 2006034 interrupts.CPU42.LOC:Local_timer_interrupts 3068304 ± 9% -22.1% 2390347 ± 6% interrupts.CPU42.RES:Rescheduling_interrupts 107102 ± 9% -20.7% 84890 ± 4% interrupts.CPU42.TLB:TLB_shootdowns 388423 ± 12% +27.8% 496564 ± 4% interrupts.CPU42.TRM:Thermal_event_interrupts 436.25 +14.6% 499.75 ± 2% interrupts.CPU43.84:IR-PCI-MSI.1572907-edge.eth0-TxRx-43 92178 ± 9% -18.3% 75306 ± 7% interrupts.CPU43.CAL:Function_call_interrupts 1784759 +12.4% 2005986 interrupts.CPU43.LOC:Local_timer_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.NMI:Non-maskable_interrupts 6434 ± 7% -64.3% 2297 ±106% interrupts.CPU43.PMI:Performance_monitoring_interrupts 104773 ± 9% -17.6% 86369 ± 6% interrupts.CPU43.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496556 ± 4% interrupts.CPU43.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU44.85:IR-PCI-MSI.1572908-edge.eth0-TxRx-44 92387 ± 10% -21.9% 72174 ± 9% interrupts.CPU44.CAL:Function_call_interrupts 1784603 +12.4% 2006064 interrupts.CPU44.LOC:Local_timer_interrupts 106179 ± 9% -18.6% 86461 ± 6% interrupts.CPU44.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496562 ± 4% interrupts.CPU44.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU45.86:IR-PCI-MSI.1572909-edge.eth0-TxRx-45 93641 ± 7% -21.9% 73155 ± 6% interrupts.CPU45.CAL:Function_call_interrupts 1784811 +12.4% 2005654 interrupts.CPU45.LOC:Local_timer_interrupts 3093195 ± 11% -15.6% 2609354 ± 4% interrupts.CPU45.RES:Rescheduling_interrupts 106463 ± 7% -19.6% 85593 ± 5% interrupts.CPU45.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496564 ± 4% interrupts.CPU45.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU46.87:IR-PCI-MSI.1572910-edge.eth0-TxRx-46 91739 ± 10% -19.0% 74279 ± 8% interrupts.CPU46.CAL:Function_call_interrupts 1784609 +12.4% 2005522 interrupts.CPU46.LOC:Local_timer_interrupts 2970471 ± 9% -21.0% 2345439 ± 8% interrupts.CPU46.RES:Rescheduling_interrupts 108158 ± 10% -19.8% 86743 ± 4% interrupts.CPU46.TLB:TLB_shootdowns 388368 ± 12% +27.8% 496469 ± 4% interrupts.CPU46.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU47.88:IR-PCI-MSI.1572911-edge.eth0-TxRx-47 92856 ± 9% -20.0% 74327 ± 8% interrupts.CPU47.CAL:Function_call_interrupts 1784835 +12.4% 2005700 interrupts.CPU47.LOC:Local_timer_interrupts 3116242 ± 10% -20.4% 2480398 ± 5% interrupts.CPU47.RES:Rescheduling_interrupts 104927 ± 9% -18.6% 85393 ± 6% interrupts.CPU47.TLB:TLB_shootdowns 388403 ± 12% +27.8% 496554 ± 4% interrupts.CPU47.TRM:Thermal_event_interrupts 436.25 +12.6% 491.00 interrupts.CPU48.89:IR-PCI-MSI.1572912-edge.eth0-TxRx-48 92889 ± 11% -21.5% 72890 ± 8% interrupts.CPU48.CAL:Function_call_interrupts 1784772 +12.4% 2006205 interrupts.CPU48.LOC:Local_timer_interrupts 3175210 ± 12% -19.7% 2548339 ± 13% interrupts.CPU48.RES:Rescheduling_interrupts 105183 ± 11% -18.6% 85618 ± 6% interrupts.CPU48.TLB:TLB_shootdowns 388426 ± 12% +27.8% 496563 ± 4% interrupts.CPU48.TRM:Thermal_event_interrupts 441.25 ± 3% +14.1% 503.25 ± 3% interrupts.CPU49.90:IR-PCI-MSI.1572913-edge.eth0-TxRx-49 90134 ± 10% -18.3% 73603 ± 6% interrupts.CPU49.CAL:Function_call_interrupts 1785145 +12.4% 2005932 interrupts.CPU49.LOC:Local_timer_interrupts 3185609 ± 11% -17.6% 2626502 ± 4% interrupts.CPU49.RES:Rescheduling_interrupts 102737 ± 10% -17.3% 84932 ± 5% interrupts.CPU49.TLB:TLB_shootdowns 388425 ± 12% +27.8% 496560 ± 4% interrupts.CPU49.TRM:Thermal_event_interrupts 92114 ± 9% -23.2% 70721 ± 6% interrupts.CPU5.CAL:Function_call_interrupts 1785135 +12.3% 2005424 interrupts.CPU5.LOC:Local_timer_interrupts 3210646 ± 12% -24.8% 2415935 ± 4% interrupts.CPU5.RES:Rescheduling_interrupts 107349 ± 9% -18.9% 87034 ± 6% interrupts.CPU5.TLB:TLB_shootdowns 388418 ± 12% +27.8% 496564 ± 4% interrupts.CPU5.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU50.91:IR-PCI-MSI.1572914-edge.eth0-TxRx-50 92406 ± 10% -20.2% 73771 ± 8% interrupts.CPU50.CAL:Function_call_interrupts 1785200 +12.4% 2006488 interrupts.CPU50.LOC:Local_timer_interrupts 106635 ± 9% -19.9% 85431 ± 6% interrupts.CPU50.TLB:TLB_shootdowns 388419 ± 12% +27.8% 496564 ± 4% interrupts.CPU50.TRM:Thermal_event_interrupts 436.25 +14.4% 499.25 ± 3% interrupts.CPU51.92:IR-PCI-MSI.1572915-edge.eth0-TxRx-51 91491 ± 9% -20.8% 72473 ± 6% interrupts.CPU51.CAL:Function_call_interrupts 1784683 +12.4% 2005527 interrupts.CPU51.LOC:Local_timer_interrupts 3051494 ± 8% -24.6% 2300032 ± 10% interrupts.CPU51.RES:Rescheduling_interrupts 104954 ± 10% -20.2% 83775 ± 5% interrupts.CPU51.TLB:TLB_shootdowns 388264 ± 12% +27.9% 496462 ± 4% interrupts.CPU51.TRM:Thermal_event_interrupts 92938 ± 11% -19.7% 74594 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 1785451 +12.4% 2006171 interrupts.CPU52.LOC:Local_timer_interrupts 3025681 ± 11% -17.1% 2506800 ± 10% interrupts.CPU52.RES:Rescheduling_interrupts 107100 ± 11% -19.6% 86156 ± 6% interrupts.CPU52.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496563 ± 4% interrupts.CPU52.TRM:Thermal_event_interrupts 439.25 ± 2% +11.7% 490.50 interrupts.CPU53.94:IR-PCI-MSI.1572917-edge.eth0-TxRx-53 93000 ± 12% -20.9% 73549 ± 7% interrupts.CPU53.CAL:Function_call_interrupts 1785290 +12.3% 2005441 interrupts.CPU53.LOC:Local_timer_interrupts 3212623 ± 7% -18.2% 2629478 ± 10% interrupts.CPU53.RES:Rescheduling_interrupts 107303 ± 11% -17.5% 88567 ± 10% interrupts.CPU53.TLB:TLB_shootdowns 388411 ± 12% +27.8% 496557 ± 4% interrupts.CPU53.TRM:Thermal_event_interrupts 437.75 +12.1% 490.50 interrupts.CPU54.95:IR-PCI-MSI.1572918-edge.eth0-TxRx-54 1785170 +12.0% 2000090 interrupts.CPU54.LOC:Local_timer_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.NMI:Non-maskable_interrupts 5351 ± 26% -64.5% 1900 ± 99% interrupts.CPU54.PMI:Performance_monitoring_interrupts 2066106 ± 9% +92.7% 3982033 ± 6% interrupts.CPU54.RES:Rescheduling_interrupts 108150 ± 12% -21.2% 85173 ± 5% interrupts.CPU54.TLB:TLB_shootdowns 329294 ± 3% +15.4% 380146 ± 2% interrupts.CPU54.TRM:Thermal_event_interrupts 436.50 +12.4% 490.50 interrupts.CPU55.96:IR-PCI-MSI.1572919-edge.eth0-TxRx-55 91164 ± 12% -19.9% 73014 ± 6% interrupts.CPU55.CAL:Function_call_interrupts 1785472 +12.0% 1999115 interrupts.CPU55.LOC:Local_timer_interrupts 1286654 ± 6% +228.2% 4223034 ± 5% interrupts.CPU55.RES:Rescheduling_interrupts 104943 ± 11% -19.5% 84466 ± 5% interrupts.CPU55.TLB:TLB_shootdowns 329292 ± 3% +15.4% 379918 ± 2% interrupts.CPU55.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU56.97:IR-PCI-MSI.1572920-edge.eth0-TxRx-56 96934 ± 13% -23.4% 74215 ± 6% interrupts.CPU56.CAL:Function_call_interrupts 1785010 +12.1% 2000169 interrupts.CPU56.LOC:Local_timer_interrupts 1312758 ± 17% +194.5% 3866522 ± 3% interrupts.CPU56.RES:Rescheduling_interrupts 109307 ± 12% -22.1% 85139 ± 4% interrupts.CPU56.TLB:TLB_shootdowns 329273 ± 3% +15.4% 380131 ± 2% interrupts.CPU56.TRM:Thermal_event_interrupts 439.00 ± 2% +11.7% 490.50 interrupts.CPU57.98:IR-PCI-MSI.1572921-edge.eth0-TxRx-57 94960 ± 11% -22.9% 73246 ± 5% interrupts.CPU57.CAL:Function_call_interrupts 1785297 +12.0% 2000019 interrupts.CPU57.LOC:Local_timer_interrupts 1666475 ± 10% +169.6% 4492937 interrupts.CPU57.RES:Rescheduling_interrupts 107534 ± 10% -22.0% 83871 ± 5% interrupts.CPU57.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380136 ± 2% interrupts.CPU57.TRM:Thermal_event_interrupts 442.25 ± 3% +10.9% 490.50 interrupts.CPU58.99:IR-PCI-MSI.1572922-edge.eth0-TxRx-58 92818 ± 12% -21.4% 72925 ± 5% interrupts.CPU58.CAL:Function_call_interrupts 1785429 +12.0% 1999528 interrupts.CPU58.LOC:Local_timer_interrupts 1606064 ± 5% +176.8% 4446179 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 103696 ± 11% -19.5% 83458 ± 4% interrupts.CPU58.TLB:TLB_shootdowns 329293 ± 3% +15.4% 380022 ± 2% interrupts.CPU58.TRM:Thermal_event_interrupts 439.00 +11.7% 490.50 interrupts.CPU59.100:IR-PCI-MSI.1572923-edge.eth0-TxRx-59 93876 ± 13% -20.5% 74669 ± 6% interrupts.CPU59.CAL:Function_call_interrupts 1785085 +12.0% 1999992 interrupts.CPU59.LOC:Local_timer_interrupts 1634812 ± 5% +174.5% 4487501 ± 4% interrupts.CPU59.RES:Rescheduling_interrupts 106929 ± 13% -21.0% 84522 ± 5% interrupts.CPU59.TLB:TLB_shootdowns 329291 ± 3% +15.4% 380111 ± 2% interrupts.CPU59.TRM:Thermal_event_interrupts 89930 ± 9% -20.5% 71520 ± 7% interrupts.CPU6.CAL:Function_call_interrupts 1785220 +12.3% 2005468 interrupts.CPU6.LOC:Local_timer_interrupts 106307 ± 7% -19.9% 85162 ± 6% interrupts.CPU6.TLB:TLB_shootdowns 388421 ± 12% +27.8% 496564 ± 4% interrupts.CPU6.TRM:Thermal_event_interrupts 436.25 +13.1% 493.25 interrupts.CPU60.101:IR-PCI-MSI.1572924-edge.eth0-TxRx-60 98595 ± 11% -25.6% 73379 ± 5% interrupts.CPU60.CAL:Function_call_interrupts 1785116 +12.1% 2000324 interrupts.CPU60.LOC:Local_timer_interrupts 1604314 ± 8% +186.8% 4601614 ± 5% interrupts.CPU60.RES:Rescheduling_interrupts 110442 ± 10% -24.0% 83932 ± 5% interrupts.CPU60.TLB:TLB_shootdowns 329256 ± 3% +15.5% 380140 ± 2% interrupts.CPU60.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU61.102:IR-PCI-MSI.1572925-edge.eth0-TxRx-61 93035 ± 11% -21.4% 73124 ± 5% interrupts.CPU61.CAL:Function_call_interrupts 1785176 +12.1% 2000960 interrupts.CPU61.LOC:Local_timer_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.NMI:Non-maskable_interrupts 4567 ± 67% -79.2% 952.00 ±171% interrupts.CPU61.PMI:Performance_monitoring_interrupts 1575186 ± 6% +194.9% 4644760 ± 4% interrupts.CPU61.RES:Rescheduling_interrupts 103665 ± 11% -18.7% 84259 ± 5% interrupts.CPU61.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380127 ± 2% interrupts.CPU61.TRM:Thermal_event_interrupts 436.25 +12.4% 490.50 interrupts.CPU62.103:IR-PCI-MSI.1572926-edge.eth0-TxRx-62 95188 ± 10% -21.9% 74376 ± 6% interrupts.CPU62.CAL:Function_call_interrupts 1785028 +12.0% 1999262 interrupts.CPU62.LOC:Local_timer_interrupts 1613834 ± 8% +184.7% 4594499 ± 3% interrupts.CPU62.RES:Rescheduling_interrupts 106856 ± 10% -21.4% 83987 ± 5% interrupts.CPU62.TLB:TLB_shootdowns 329270 ± 3% +15.4% 380120 ± 2% interrupts.CPU62.TRM:Thermal_event_interrupts 94411 ± 11% -21.3% 74269 ± 5% interrupts.CPU63.CAL:Function_call_interrupts 1785322 +12.0% 1999237 interrupts.CPU63.LOC:Local_timer_interrupts 1669218 ± 6% +187.2% 4794684 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 105402 ± 10% -18.3% 86076 ± 8% interrupts.CPU63.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU63.TRM:Thermal_event_interrupts 96060 ± 11% -22.6% 74364 ± 6% interrupts.CPU64.CAL:Function_call_interrupts 1785304 +12.0% 1999571 interrupts.CPU64.LOC:Local_timer_interrupts 1681501 ± 7% +164.0% 4438647 ± 3% interrupts.CPU64.RES:Rescheduling_interrupts 108534 ± 10% -22.8% 83813 ± 5% interrupts.CPU64.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU64.TRM:Thermal_event_interrupts 94178 ± 9% -22.7% 72758 ± 8% interrupts.CPU65.CAL:Function_call_interrupts 1785338 +12.0% 1999408 interrupts.CPU65.LOC:Local_timer_interrupts 1674108 ± 4% +182.2% 4724262 ± 3% interrupts.CPU65.RES:Rescheduling_interrupts 107519 ± 10% -22.6% 83259 ± 5% interrupts.CPU65.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380128 ± 2% interrupts.CPU65.TRM:Thermal_event_interrupts 95135 ± 8% -22.4% 73795 ± 5% interrupts.CPU66.CAL:Function_call_interrupts 1785413 +12.0% 2000352 interrupts.CPU66.LOC:Local_timer_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.NMI:Non-maskable_interrupts 616.75 ±171% +902.4% 6182 ± 28% interrupts.CPU66.PMI:Performance_monitoring_interrupts 1646166 ± 6% +169.3% 4433267 ± 6% interrupts.CPU66.RES:Rescheduling_interrupts 107190 ± 7% -22.5% 83034 ± 4% interrupts.CPU66.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380118 ± 2% interrupts.CPU66.TRM:Thermal_event_interrupts 95627 ± 10% -22.4% 74249 ± 5% interrupts.CPU67.CAL:Function_call_interrupts 1785314 +12.1% 2001128 interrupts.CPU67.LOC:Local_timer_interrupts 1611146 ± 4% +183.8% 4572204 ± 7% interrupts.CPU67.RES:Rescheduling_interrupts 109283 ± 11% -22.5% 84676 ± 4% interrupts.CPU67.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380142 ± 2% interrupts.CPU67.TRM:Thermal_event_interrupts 95479 ± 11% -20.6% 75791 ± 5% interrupts.CPU68.CAL:Function_call_interrupts 1785192 +12.0% 1999411 interrupts.CPU68.LOC:Local_timer_interrupts 1575394 ± 7% +189.9% 4567277 ± 4% interrupts.CPU68.RES:Rescheduling_interrupts 106972 ± 12% -18.6% 87039 ± 5% interrupts.CPU68.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380129 ± 2% interrupts.CPU68.TRM:Thermal_event_interrupts 93067 ± 14% -19.9% 74538 ± 4% interrupts.CPU69.CAL:Function_call_interrupts 1785257 +12.1% 2000636 interrupts.CPU69.LOC:Local_timer_interrupts 1578663 ± 9% +192.9% 4624518 ± 3% interrupts.CPU69.RES:Rescheduling_interrupts 104271 ± 12% -18.5% 84989 ± 2% interrupts.CPU69.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380066 ± 2% interrupts.CPU69.TRM:Thermal_event_interrupts 474.00 ± 3% +49.7% 709.75 ± 24% interrupts.CPU7.46:IR-PCI-MSI.1572871-edge.eth0-TxRx-7 92062 ± 9% -19.7% 73884 ± 5% interrupts.CPU7.CAL:Function_call_interrupts 1784266 +12.4% 2004980 interrupts.CPU7.LOC:Local_timer_interrupts 2505703 ± 11% -19.0% 2029187 ± 5% interrupts.CPU7.RES:Rescheduling_interrupts 107698 ± 8% -18.4% 87903 ± 6% interrupts.CPU7.TLB:TLB_shootdowns 388281 ± 12% +27.8% 496347 ± 4% interrupts.CPU7.TRM:Thermal_event_interrupts 92337 ± 12% -17.5% 76148 ± 2% interrupts.CPU70.CAL:Function_call_interrupts 1785161 +12.0% 1999757 interrupts.CPU70.LOC:Local_timer_interrupts 1562360 ± 4% +170.8% 4231208 ± 9% interrupts.CPU70.RES:Rescheduling_interrupts 103862 ± 10% -18.8% 84371 ± 3% interrupts.CPU70.TLB:TLB_shootdowns 329296 ± 3% +15.4% 380141 ± 2% interrupts.CPU70.TRM:Thermal_event_interrupts 95829 ± 12% -21.2% 75509 ± 5% interrupts.CPU71.CAL:Function_call_interrupts 1784894 +12.1% 2001287 interrupts.CPU71.LOC:Local_timer_interrupts 1533567 ± 7% +196.9% 4553440 interrupts.CPU71.RES:Rescheduling_interrupts 105883 ± 11% -20.4% 84248 ± 6% interrupts.CPU71.TLB:TLB_shootdowns 329247 ± 3% +15.5% 380146 ± 2% interrupts.CPU71.TRM:Thermal_event_interrupts 88372 ± 11% -20.3% 70389 ± 7% interrupts.CPU8.CAL:Function_call_interrupts 1784452 +12.4% 2005305 interrupts.CPU8.LOC:Local_timer_interrupts 103992 ± 10% -17.4% 85887 ± 5% interrupts.CPU8.TLB:TLB_shootdowns 388422 ± 12% +27.8% 496557 ± 4% interrupts.CPU8.TRM:Thermal_event_interrupts 89290 ± 10% -19.1% 72221 ± 6% interrupts.CPU9.CAL:Function_call_interrupts 1784716 +12.4% 2005710 interrupts.CPU9.LOC:Local_timer_interrupts 104340 ± 10% -16.0% 87647 ± 3% interrupts.CPU9.TLB:TLB_shootdowns 388408 ± 12% +27.8% 496557 ± 4% interrupts.CPU9.TRM:Thermal_event_interrupts 1.285e+08 +12.2% 1.442e+08 interrupts.LOC:Local_timer_interrupts 144.00 +50.0% 216.00 interrupts.MCP:Machine_check_polls 1.677e+08 ± 8% +50.4% 2.522e+08 ± 2% interrupts.RES:Rescheduling_interrupts 7624145 ± 10% -19.7% 6125835 ± 5% interrupts.TLB:TLB_shootdowns 25836833 ± 8% +22.2% 31559779 ± 2% interrupts.TRM:Thermal_event_interrupts *************************************************************************************************** lkp-bdw-ep6: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz with 128G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/process/50%/debian-x86_64-2019-11-14.cgz/lkp-bdw-ep6/hackbench/0xb000038 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 134091 -10.6% 119937 hackbench.throughput 3.352e+09 ± 2% +16.1% 3.892e+09 ± 5% hackbench.time.involuntary_context_switches 535571 ± 2% +10.6% 592420 ± 4% numa-meminfo.node0.FilePages 12614 ± 18% -12.2% 11078 ± 18% numa-meminfo.node1.Mapped 3472 ± 6% +18.0% 4096 slabinfo.skbuff_head_cache.active_objs 3568 ± 6% +15.7% 4128 slabinfo.skbuff_head_cache.num_objs 730128 ± 5% -52.2% 349073 ± 25% turbostat.C3 14.77 +5.9% 15.64 turbostat.RAMWatt 24613783 ± 5% -32.6% 16595193 ± 23% numa-numastat.node0.local_node 24632941 ± 5% -32.6% 16605181 ± 23% numa-numastat.node0.numa_hit 25827961 -29.9% 18097150 ± 19% numa-numastat.node1.local_node 25836948 -29.9% 18115676 ± 19% numa-numastat.node1.numa_hit 50467170 ± 2% -31.2% 34733456 ± 10% proc-vmstat.numa_hit 50439016 ± 2% -31.2% 34704933 ± 10% proc-vmstat.numa_local 50622650 ± 2% -31.1% 34896854 ± 10% proc-vmstat.pgalloc_normal 50548381 ± 2% -31.1% 34834391 ± 10% proc-vmstat.pgfree 79.00 +2.2% 80.75 vmstat.cpu.sy 19.00 -7.9% 17.50 ± 2% vmstat.cpu.us 13450755 +4.6% 14066265 vmstat.system.cs 1122772 -12.1% 986681 vmstat.system.in 133891 ± 2% +10.6% 148112 ± 4% numa-vmstat.node0.nr_file_pages 12301995 ± 4% -30.2% 8590153 ± 22% numa-vmstat.node0.numa_hit 12282910 ± 4% -30.1% 8580139 ± 22% numa-vmstat.node0.numa_local 12964354 ± 5% -31.2% 8914408 ± 18% numa-vmstat.node1.numa_hit 12806275 ± 5% -31.7% 8746928 ± 19% numa-vmstat.node1.numa_local 33816695 ± 10% +17.8% 39824104 ± 7% cpuidle.C1.time 21015951 ± 3% +19.6% 25127666 ± 7% cpuidle.C1E.time 730991 ± 5% -52.1% 350128 ± 25% cpuidle.C3.usage 1.497e+08 ± 28% +41.6% 2.119e+08 ± 15% cpuidle.C6.time 6771631 ± 4% -45.2% 3713804 ± 15% cpuidle.POLL.time 5209842 ± 4% -44.1% 2913245 ± 15% cpuidle.POLL.usage 26965506 +5.5% 28444381 ± 4% sched_debug.cfs_rq:/.min_vruntime.avg 28257967 +6.6% 30126596 ± 4% sched_debug.cfs_rq:/.min_vruntime.max 582203 ± 9% +37.0% 797353 ± 8% sched_debug.cfs_rq:/.min_vruntime.stddev 4.65 ± 8% -10.7% 4.16 ± 6% sched_debug.cfs_rq:/.runnable_load_avg.stddev 4620 ± 6% -10.7% 4125 ± 4% sched_debug.cfs_rq:/.runnable_weight.stddev 582831 ± 9% +36.6% 796226 ± 7% sched_debug.cfs_rq:/.spread0.stddev 154.73 ± 2% -14.7% 132.00 ± 3% sched_debug.cfs_rq:/.util_avg.stddev 120.93 ± 25% +43.5% 173.50 ± 19% sched_debug.cfs_rq:/.util_est_enqueued.min 184.93 ± 2% -11.0% 164.54 ± 3% sched_debug.cfs_rq:/.util_est_enqueued.stddev 182135 ± 18% +57.1% 286098 ± 6% sched_debug.cpu.avg_idle.avg 185926 ± 19% +49.0% 276991 ± 9% sched_debug.cpu.avg_idle.stddev 3.32 ± 4% -10.8% 2.96 ± 3% sched_debug.cpu.nr_running.stddev 46710201 +9.0% 50934866 ± 5% sched_debug.cpu.nr_switches.avg 50053735 +16.9% 58491253 ± 7% sched_debug.cpu.nr_switches.max 43751407 +6.7% 46698664 ± 4% sched_debug.cpu.nr_switches.min 1344975 ± 10% +78.9% 2406504 ± 16% sched_debug.cpu.nr_switches.stddev 1.63 ± 3% -32.8% 1.09 ± 11% sched_debug.cpu.nr_uninterruptible.avg 1017 ± 27% -59.5% 411.75 ± 16% interrupts.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6547829 -14.4% 5603910 ± 5% interrupts.CPU16.RES:Rescheduling_interrupts 1017 ± 27% -59.5% 411.75 ± 16% interrupts.CPU19.40:IR-PCI-MSI.1572871-edge.eth0-TxRx-6 6801598 -14.7% 5801847 ± 5% interrupts.CPU19.RES:Rescheduling_interrupts 6609756 -13.7% 5703084 ± 8% interrupts.CPU20.RES:Rescheduling_interrupts 6446878 ± 2% -11.8% 5689145 ± 7% interrupts.CPU21.RES:Rescheduling_interrupts 6634825 -9.4% 6008203 ± 6% interrupts.CPU24.RES:Rescheduling_interrupts 6801182 -9.8% 6137528 ± 3% interrupts.CPU25.RES:Rescheduling_interrupts 6687252 -12.9% 5821967 ± 6% interrupts.CPU28.RES:Rescheduling_interrupts 6583387 -10.9% 5868659 ± 6% interrupts.CPU29.RES:Rescheduling_interrupts 6585631 -16.6% 5490726 ± 9% interrupts.CPU37.RES:Rescheduling_interrupts 6759973 -13.2% 5867800 ± 7% interrupts.CPU57.RES:Rescheduling_interrupts 6459616 ± 2% -10.5% 5783047 ± 6% interrupts.CPU58.RES:Rescheduling_interrupts 6580818 ± 2% -16.9% 5468945 ± 3% interrupts.CPU60.RES:Rescheduling_interrupts 6723178 ± 2% -12.4% 5887778 ± 3% interrupts.CPU63.RES:Rescheduling_interrupts 6480129 -16.2% 5427168 ± 7% interrupts.CPU64.RES:Rescheduling_interrupts 6439558 ± 2% -11.6% 5691160 ± 5% interrupts.CPU65.RES:Rescheduling_interrupts 6662822 -12.0% 5866390 ± 7% interrupts.CPU68.RES:Rescheduling_interrupts 6602178 -12.0% 5809760 ± 7% interrupts.CPU7.RES:Rescheduling_interrupts 6616172 -14.1% 5685889 ± 7% interrupts.CPU73.RES:Rescheduling_interrupts 6591477 -13.8% 5682655 ± 8% interrupts.CPU81.RES:Rescheduling_interrupts 13.97 +15.8% 16.18 perf-stat.i.MPKI 1.70 -0.1 1.65 perf-stat.i.branch-miss-rate% 3.607e+08 -1.6% 3.549e+08 perf-stat.i.branch-misses 1.22 ± 6% +0.8 2.01 ± 18% perf-stat.i.cache-miss-rate% 17650711 ± 6% +98.0% 34953649 ± 18% perf-stat.i.cache-misses 1.496e+09 +17.3% 1.755e+09 perf-stat.i.cache-references 13503062 +4.6% 14122231 perf-stat.i.context-switches 2946654 ± 2% +18.5% 3491269 ± 3% perf-stat.i.cpu-migrations 29365 ± 8% -30.6% 20387 ± 31% perf-stat.i.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.i.dTLB-store-miss-rate% 2.048e+10 -2.1% 2.006e+10 perf-stat.i.dTLB-stores 55.17 -1.4 53.74 perf-stat.i.iTLB-load-miss-rate% 1.019e+08 +5.5% 1.074e+08 perf-stat.i.iTLB-loads 3068 -3.0% 2975 perf-stat.i.minor-faults 9195204 ± 6% +113.0% 19589023 ± 18% perf-stat.i.node-load-misses 28132 ± 6% +125.3% 63396 ± 13% perf-stat.i.node-loads 58.92 -9.8 49.16 ± 4% perf-stat.i.node-store-miss-rate% 3588196 ± 6% +60.5% 5760265 ± 19% perf-stat.i.node-store-misses 2415953 ± 5% +145.2% 5924042 ± 18% perf-stat.i.node-stores 3068 -3.0% 2975 perf-stat.i.page-faults 13.96 +16.0% 16.19 perf-stat.overall.MPKI 1.70 -0.1 1.65 perf-stat.overall.branch-miss-rate% 1.18 ± 6% +0.8 1.99 ± 18% perf-stat.overall.cache-miss-rate% 13823 ± 6% -47.8% 7221 ± 20% perf-stat.overall.cycles-between-cache-misses 0.35 ± 2% +0.0 0.36 ± 2% perf-stat.overall.dTLB-store-miss-rate% 55.11 -1.4 53.70 perf-stat.overall.iTLB-load-miss-rate% 59.75 -10.5 49.23 perf-stat.overall.node-store-miss-rate% 78192 +13.0% 88370 perf-stat.overall.path-length 3.601e+08 -1.6% 3.543e+08 perf-stat.ps.branch-misses 17622620 ± 6% +98.0% 34899005 ± 18% perf-stat.ps.cache-misses 1.494e+09 +17.3% 1.752e+09 perf-stat.ps.cache-references 13480657 +4.6% 14099389 perf-stat.ps.context-switches 2941749 ± 2% +18.5% 3485631 ± 3% perf-stat.ps.cpu-migrations 2.045e+10 -2.1% 2.003e+10 perf-stat.ps.dTLB-stores 1.017e+08 +5.5% 1.072e+08 perf-stat.ps.iTLB-loads 3064 -3.0% 2972 perf-stat.ps.minor-faults 9180406 ± 6% +113.0% 19558101 ± 18% perf-stat.ps.node-load-misses 28101 ± 6% +125.3% 63314 ± 13% perf-stat.ps.node-loads 3582467 ± 6% +60.5% 5751138 ± 19% perf-stat.ps.node-store-misses 2412086 ± 5% +145.2% 5914636 ± 18% perf-stat.ps.node-stores 3064 -3.0% 2972 perf-stat.ps.page-faults 223485 +9.1% 243918 ± 5% softirqs.CPU0.TIMER 11887 ± 11% -18.2% 9723 ± 6% softirqs.CPU1.SCHED 215411 +9.2% 235267 ± 6% softirqs.CPU1.TIMER 11293 ± 3% -14.3% 9678 ± 10% softirqs.CPU10.SCHED 213916 +9.8% 234908 ± 5% softirqs.CPU10.TIMER 212758 +7.7% 229141 ± 5% softirqs.CPU11.TIMER 11098 ± 2% -15.9% 9337 ± 10% softirqs.CPU12.SCHED 213397 +9.0% 232706 ± 6% softirqs.CPU12.TIMER 212419 +8.3% 230115 ± 6% softirqs.CPU13.TIMER 10777 ± 4% -12.5% 9426 ± 6% softirqs.CPU14.SCHED 11241 ± 3% -11.5% 9950 ± 6% softirqs.CPU15.SCHED 214604 +10.0% 236116 ± 8% softirqs.CPU15.TIMER 11147 -14.9% 9482 ± 8% softirqs.CPU16.SCHED 216343 +8.4% 234558 ± 6% softirqs.CPU16.TIMER 214680 +7.0% 229748 ± 6% softirqs.CPU17.TIMER 212791 +8.6% 230988 ± 6% softirqs.CPU19.TIMER 11211 ± 4% -10.4% 10047 ± 6% softirqs.CPU2.SCHED 219918 +9.8% 241513 ± 6% softirqs.CPU2.TIMER 10825 ± 4% -14.3% 9273 ± 7% softirqs.CPU20.SCHED 215209 +8.8% 234149 ± 6% softirqs.CPU20.TIMER 10869 ± 4% -11.4% 9626 ± 4% softirqs.CPU21.SCHED 216225 +8.7% 235105 ± 5% softirqs.CPU21.TIMER 13013 ± 7% -14.0% 11192 ± 3% softirqs.CPU22.SCHED 220406 +8.9% 239975 ± 4% softirqs.CPU22.TIMER 11925 -14.9% 10145 ± 3% softirqs.CPU23.SCHED 220231 +7.6% 237056 ± 5% softirqs.CPU23.TIMER 11140 ± 2% -11.5% 9861 ± 7% softirqs.CPU24.SCHED 216091 +8.9% 235338 ± 5% softirqs.CPU24.TIMER 11438 ± 3% -16.3% 9578 ± 2% softirqs.CPU25.SCHED 214880 +9.4% 235143 ± 5% softirqs.CPU25.TIMER 214888 +8.8% 233874 ± 5% softirqs.CPU26.TIMER 11410 ± 2% -13.7% 9844 ± 10% softirqs.CPU27.SCHED 215012 +8.7% 233790 ± 4% softirqs.CPU27.TIMER 220618 ± 2% +7.8% 237788 ± 5% softirqs.CPU28.TIMER 11096 ± 5% -14.3% 9513 ± 6% softirqs.CPU29.SCHED 217194 +8.2% 235108 ± 5% softirqs.CPU29.TIMER 11445 ± 2% -16.7% 9536 ± 10% softirqs.CPU3.SCHED 214692 +9.1% 234221 ± 5% softirqs.CPU3.TIMER 11127 ± 5% -10.5% 9954 ± 6% softirqs.CPU30.SCHED 214861 +8.2% 232532 ± 5% softirqs.CPU30.TIMER 217582 +7.8% 234493 ± 4% softirqs.CPU31.TIMER 214606 +8.8% 233473 ± 4% softirqs.CPU32.TIMER 11299 ± 2% -11.7% 9975 ± 6% softirqs.CPU33.SCHED 11503 ± 2% -16.0% 9658 ± 10% softirqs.CPU34.SCHED 213233 +10.9% 236580 ± 6% softirqs.CPU34.TIMER 11162 ± 2% -11.6% 9863 ± 7% softirqs.CPU35.SCHED 11241 -13.9% 9678 ± 6% softirqs.CPU36.SCHED 215681 +9.2% 235446 ± 4% softirqs.CPU36.TIMER 11358 -15.2% 9632 ± 6% softirqs.CPU37.SCHED 219691 ± 2% +18.5% 260407 ± 13% softirqs.CPU37.TIMER 11570 ± 2% -13.1% 10049 ± 6% softirqs.CPU38.SCHED 215559 +8.2% 233189 ± 4% softirqs.CPU38.TIMER 11295 ± 4% -13.1% 9819 ± 10% softirqs.CPU39.SCHED 215781 ± 2% +7.6% 232108 ± 5% softirqs.CPU39.TIMER 11506 ± 7% -13.0% 10006 ± 6% softirqs.CPU4.SCHED 214973 +8.9% 234140 ± 6% softirqs.CPU4.TIMER 11304 ± 2% -13.7% 9759 ± 10% softirqs.CPU40.SCHED 213827 +10.1% 235341 ± 4% softirqs.CPU40.TIMER 11462 ± 4% -13.5% 9913 ± 8% softirqs.CPU41.SCHED 215879 ± 2% +7.2% 231499 ± 5% softirqs.CPU41.TIMER 11418 ± 3% -14.0% 9819 ± 8% softirqs.CPU43.SCHED 214864 +8.3% 232698 ± 4% softirqs.CPU43.TIMER 10706 ± 5% -13.9% 9223 ± 11% softirqs.CPU44.SCHED 217386 +9.4% 237916 ± 5% softirqs.CPU44.TIMER 215191 +9.7% 236017 ± 5% softirqs.CPU45.TIMER 214214 +9.9% 235466 ± 6% softirqs.CPU46.TIMER 214471 +9.9% 235675 ± 5% softirqs.CPU47.TIMER 11251 ± 2% -13.0% 9785 ± 8% softirqs.CPU48.SCHED 212500 +9.9% 233451 ± 6% softirqs.CPU48.TIMER 213609 +9.9% 234657 ± 5% softirqs.CPU49.TIMER 216836 +8.5% 235225 ± 5% softirqs.CPU5.TIMER 215854 +9.0% 235383 ± 5% softirqs.CPU50.TIMER 11468 -14.4% 9820 ± 9% softirqs.CPU51.SCHED 215402 +8.8% 234360 ± 5% softirqs.CPU51.TIMER 214152 +8.8% 233038 ± 5% softirqs.CPU52.TIMER 212843 +8.4% 230716 ± 5% softirqs.CPU53.TIMER 11373 ± 2% -14.9% 9679 ± 7% softirqs.CPU54.SCHED 214008 +9.5% 234380 ± 5% softirqs.CPU54.TIMER 212619 +7.5% 228539 ± 5% softirqs.CPU55.TIMER 11136 ± 3% -16.4% 9312 ± 10% softirqs.CPU56.SCHED 213404 +9.0% 232542 ± 6% softirqs.CPU56.TIMER 11171 ± 4% -11.9% 9837 ± 7% softirqs.CPU57.SCHED 212283 +8.6% 230474 ± 6% softirqs.CPU57.TIMER 11137 ± 4% -14.0% 9578 ± 6% softirqs.CPU58.SCHED 11226 ± 4% -17.2% 9289 ± 4% softirqs.CPU59.SCHED 214193 +23.1% 263668 ± 22% softirqs.CPU59.TIMER 217195 +8.2% 234975 ± 5% softirqs.CPU6.TIMER 11051 ± 4% -14.4% 9460 ± 9% softirqs.CPU60.SCHED 217131 +7.9% 234363 ± 5% softirqs.CPU60.TIMER 11206 ± 3% -12.6% 9793 ± 9% softirqs.CPU61.SCHED 211791 +9.2% 231302 ± 5% softirqs.CPU61.TIMER 214956 ± 2% +8.8% 233770 ± 5% softirqs.CPU62.TIMER 11180 ± 3% -14.8% 9526 ± 8% softirqs.CPU63.SCHED 213413 +8.7% 232008 ± 5% softirqs.CPU63.TIMER 215674 +8.1% 233086 ± 6% softirqs.CPU64.TIMER 11032 ± 3% -15.7% 9302 ± 7% softirqs.CPU65.SCHED 215870 +9.0% 235386 ± 5% softirqs.CPU65.TIMER 11403 ± 3% -14.2% 9786 ± 6% softirqs.CPU66.SCHED 215968 +9.4% 236175 ± 5% softirqs.CPU66.TIMER 11098 ± 3% -7.4% 10281 ± 4% softirqs.CPU67.SCHED 217846 +8.3% 236023 ± 4% softirqs.CPU67.TIMER 11094 -12.2% 9740 ± 7% softirqs.CPU68.SCHED 215512 +9.4% 235796 ± 5% softirqs.CPU68.TIMER 11321 ± 3% -14.1% 9724 ± 3% softirqs.CPU69.SCHED 213564 +9.0% 232867 ± 5% softirqs.CPU69.TIMER 218822 +8.8% 238080 ± 6% softirqs.CPU7.TIMER 11507 ± 2% -12.9% 10026 ± 4% softirqs.CPU70.SCHED 213928 +9.4% 233986 ± 4% softirqs.CPU70.TIMER 213528 +8.5% 231730 ± 4% softirqs.CPU71.TIMER 11262 ± 2% -11.1% 10013 ± 5% softirqs.CPU72.SCHED 11145 ± 3% -14.5% 9533 ± 8% softirqs.CPU73.SCHED 216467 +9.2% 236409 ± 5% softirqs.CPU73.TIMER 11447 ± 4% -13.2% 9935 ± 4% softirqs.CPU74.SCHED 212578 +9.1% 232022 ± 5% softirqs.CPU74.TIMER 215256 +7.8% 232072 ± 5% softirqs.CPU75.TIMER 11324 ± 4% -11.8% 9985 ± 3% softirqs.CPU76.SCHED 214170 +8.6% 232516 ± 5% softirqs.CPU76.TIMER 11646 ± 5% -15.7% 9824 ± 7% softirqs.CPU77.SCHED 213706 +8.0% 230880 ± 5% softirqs.CPU77.TIMER 11397 ± 6% -13.9% 9810 ± 10% softirqs.CPU78.SCHED 11179 ± 3% -11.5% 9894 ± 5% softirqs.CPU79.SCHED 213215 +8.4% 231081 ± 5% softirqs.CPU79.TIMER 11165 ± 4% -16.2% 9352 ± 9% softirqs.CPU8.SCHED 216508 ± 2% +7.9% 233692 ± 6% softirqs.CPU8.TIMER 11203 ± 2% -13.9% 9643 ± 4% softirqs.CPU80.SCHED 216446 +8.3% 234469 ± 5% softirqs.CPU80.TIMER 11151 ± 2% -12.8% 9726 ± 6% softirqs.CPU81.SCHED 11373 ± 3% -13.0% 9891 ± 6% softirqs.CPU82.SCHED 213597 +8.9% 232608 ± 5% softirqs.CPU82.TIMER 214308 +7.9% 231148 ± 5% softirqs.CPU83.TIMER 214384 +20.9% 259149 ± 15% softirqs.CPU84.TIMER 11458 -13.7% 9887 ± 7% softirqs.CPU86.SCHED 215459 +8.2% 233058 ± 5% softirqs.CPU86.TIMER 11303 ± 4% -12.4% 9904 ± 8% softirqs.CPU87.SCHED 214185 +7.9% 231116 ± 5% softirqs.CPU87.TIMER 11030 ± 2% -13.0% 9593 ± 8% softirqs.CPU9.SCHED 213229 +8.7% 231788 ± 6% softirqs.CPU9.TIMER 995635 -12.9% 867688 ± 5% softirqs.SCHED 19052052 +8.6% 20690944 ± 5% softirqs.TIMER 31.32 -1.6 29.74 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.24 -1.5 28.76 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 26.06 -0.9 25.17 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.35 ± 5% -0.9 1.48 ± 12% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 25.36 -0.8 24.54 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 2.75 ± 4% -0.7 2.08 ± 7% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 3.84 ± 2% -0.6 3.23 ± 2% perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 7.83 -0.5 7.30 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 1.68 ± 6% -0.5 1.17 ± 10% perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 8.02 -0.5 7.53 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 8.08 -0.5 7.60 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.63 ± 4% -0.4 1.19 ± 6% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_write.new_sync_write.vfs_write.ksys_write 1.71 ± 6% -0.4 1.28 ± 8% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 2.60 -0.4 2.19 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 2.03 ± 2% -0.4 1.64 ± 4% perf-profile.calltrace.cycles-pp.security_file_permission.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.47 -0.4 3.08 ± 4% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 1.95 ± 2% -0.3 1.61 ± 3% perf-profile.calltrace.cycles-pp.copy_page_from_iter.pipe_write.new_sync_write.vfs_write.ksys_write 2.67 -0.3 2.35 ± 3% perf-profile.calltrace.cycles-pp.copy_page_to_iter.pipe_read.new_sync_read.vfs_read.ksys_read 0.56 -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.fsnotify.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.22 ± 4% -0.3 0.94 ± 2% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_write.new_sync_write.vfs_write.ksys_write 1.25 ± 2% -0.3 1.00 ± 5% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.64 -0.2 2.41 ± 2% perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.53 -0.2 1.34 ± 7% perf-profile.calltrace.cycles-pp.selinux_file_permission.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.59 ± 4% -0.2 0.40 ± 57% perf-profile.calltrace.cycles-pp.__enqueue_entity.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.34 -0.2 1.16 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 1.07 ± 2% -0.2 0.90 ± 4% perf-profile.calltrace.cycles-pp.copyin.copy_page_from_iter.pipe_write.new_sync_write.vfs_write 1.05 -0.2 0.88 ± 6% perf-profile.calltrace.cycles-pp.update_load_avg.dequeue_entity.dequeue_task_fair.__schedule.schedule 1.56 ± 2% -0.2 1.38 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout.copy_page_to_iter.pipe_read.new_sync_read 1.17 ± 4% -0.2 1.01 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 0.98 ± 2% -0.2 0.82 ± 4% perf-profile.calltrace.cycles-pp.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.66 ± 2% -0.2 1.50 ± 5% perf-profile.calltrace.cycles-pp.copyout.copy_page_to_iter.pipe_read.new_sync_read.vfs_read 0.73 ± 3% -0.2 0.57 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_read.ksys_read.do_syscall_64 0.96 ± 2% -0.1 0.82 ± 3% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyin.copy_page_from_iter.pipe_write.new_sync_write 0.89 ± 2% -0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.66 ± 6% -0.1 0.54 ± 2% perf-profile.calltrace.cycles-pp.file_has_perm.security_file_permission.vfs_write.ksys_write.do_syscall_64 2.42 -0.1 2.32 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.pipe_wait.pipe_read 0.89 ± 4% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 0.74 ± 2% -0.1 0.66 ± 3% perf-profile.calltrace.cycles-pp.update_curr.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.84 -0.1 0.79 ± 3% perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.56 -0.0 0.54 ± 2% perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate 0.77 ± 2% +0.1 0.82 perf-profile.calltrace.cycles-pp.check_preempt_wakeup.check_preempt_curr.ttwu_do_wakeup.try_to_wake_up.autoremove_wake_function 0.90 ± 3% +0.1 0.97 ± 2% perf-profile.calltrace.cycles-pp.native_write_msr 0.69 ± 3% +0.1 0.77 ± 3% perf-profile.calltrace.cycles-pp.update_rq_clock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 0.80 ± 4% +0.1 0.89 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 1.10 +0.1 1.20 perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.73 ± 3% +0.1 0.83 ± 2% perf-profile.calltrace.cycles-pp.__switch_to 1.05 +0.1 1.16 perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 0.63 ± 4% +0.1 0.75 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 0.99 ± 5% +0.1 1.12 ± 5% perf-profile.calltrace.cycles-pp.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.07 ± 3% +0.1 1.21 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 0.64 ± 6% +0.2 0.80 ± 2% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.pipe_wait.pipe_read 1.41 ± 3% +0.2 1.59 ± 2% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.77 ± 5% +0.2 1.00 ± 5% perf-profile.calltrace.cycles-pp.load_new_mm_cr3.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop 1.73 ± 3% +0.3 2.06 ± 3% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.14 ±173% +0.5 0.63 ± 5% perf-profile.calltrace.cycles-pp.update_rq_clock.__schedule.schedule.pipe_wait.pipe_read 0.13 ±173% +0.5 0.64 ± 5% perf-profile.calltrace.cycles-pp.migrate_task_rq_fair.set_task_cpu.try_to_wake_up.autoremove_wake_function.__wake_up_common 1.85 ± 5% +0.5 2.38 ± 3% perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.00 +0.6 0.59 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.00 +0.6 0.60 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.dequeue_task_fair.__schedule.schedule.pipe_wait 0.00 +0.7 0.67 ± 5% perf-profile.calltrace.cycles-pp.__update_load_avg_cfs_rq.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.15 ±173% +1.0 1.14 ± 12% perf-profile.calltrace.cycles-pp.cpumask_next_wrap.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 87.66 +1.1 88.78 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 86.87 +1.1 88.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.59 ± 4% +1.4 6.96 ± 3% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.72 ± 4% +1.4 7.12 ± 3% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.87 ± 4% +1.4 7.28 ± 3% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.70 ± 30% +3.3 4.98 ± 13% perf-profile.calltrace.cycles-pp.available_idle_cpu.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function 35.54 +3.3 38.87 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 34.28 +3.5 37.82 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 31.12 +4.1 35.22 ± 2% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 30.49 +4.1 34.63 ± 2% perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 22.54 ± 2% +6.0 28.53 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 20.58 ± 2% +6.2 26.76 ± 3% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 19.97 ± 2% +6.2 26.16 ± 3% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 19.73 ± 3% +6.2 25.92 ± 3% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 3.39 ± 20% +7.0 10.39 ± 12% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.89 ± 15% +7.1 11.98 ± 10% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 31.34 -1.6 29.75 perf-profile.children.cycles-pp.ksys_read 30.28 -1.5 28.79 perf-profile.children.cycles-pp.vfs_read 4.14 ± 5% -1.3 2.85 ± 7% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 3.51 ± 6% -1.0 2.54 ± 8% perf-profile.children.cycles-pp.update_cfs_group 26.10 -0.9 25.20 perf-profile.children.cycles-pp.new_sync_read 25.42 -0.8 24.59 perf-profile.children.cycles-pp.pipe_read 4.67 -0.8 3.86 ± 4% perf-profile.children.cycles-pp.security_file_permission 4.33 ± 2% -0.7 3.65 ± 2% perf-profile.children.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.children.cycles-pp.entry_SYSCALL_64 2.18 ± 4% -0.5 1.64 ± 6% perf-profile.children.cycles-pp.mutex_unlock 7.92 -0.5 7.40 ± 3% perf-profile.children.cycles-pp.enqueue_task_fair 8.10 -0.5 7.61 ± 3% perf-profile.children.cycles-pp.activate_task 8.16 -0.5 7.67 ± 3% perf-profile.children.cycles-pp.ttwu_do_activate 4.12 ± 4% -0.5 3.64 ± 5% perf-profile.children.cycles-pp._raw_spin_lock 2.83 -0.4 2.38 ± 6% perf-profile.children.cycles-pp.selinux_file_permission 3.53 -0.4 3.13 ± 4% perf-profile.children.cycles-pp.enqueue_entity 1.99 ± 2% -0.4 1.64 ± 3% perf-profile.children.cycles-pp.copy_page_from_iter 3.34 -0.4 2.99 ± 4% perf-profile.children.cycles-pp.update_load_avg 2.70 -0.3 2.39 ± 3% perf-profile.children.cycles-pp.copy_page_to_iter 2.66 -0.3 2.35 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.93 ± 3% -0.3 0.64 ± 7% perf-profile.children.cycles-pp.__mutex_unlock_slowpath 1.41 ± 4% -0.3 1.13 perf-profile.children.cycles-pp.file_has_perm 2.75 -0.3 2.49 ± 2% perf-profile.children.cycles-pp.dequeue_entity 2.29 ± 2% -0.3 2.04 perf-profile.children.cycles-pp.mutex_lock 0.85 ± 3% -0.2 0.61 ± 4% perf-profile.children.cycles-pp.__inode_security_revalidate 0.59 ± 3% -0.2 0.39 ± 6% perf-profile.children.cycles-pp.__mutex_lock 1.79 ± 2% -0.2 1.59 ± 4% perf-profile.children.cycles-pp.__fdget_pos 1.70 ± 2% -0.2 1.52 ± 3% perf-profile.children.cycles-pp.__fget_light 0.93 ± 3% -0.2 0.75 perf-profile.children.cycles-pp.avc_has_perm 1.09 ± 2% -0.2 0.92 ± 4% perf-profile.children.cycles-pp.copyin 2.33 ± 2% -0.2 2.16 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 0.96 ± 2% -0.2 0.81 ± 2% perf-profile.children.cycles-pp.___might_sleep 1.67 ± 2% -0.2 1.52 ± 5% perf-profile.children.cycles-pp.copyout 0.83 ± 4% -0.1 0.68 ± 2% perf-profile.children.cycles-pp._cond_resched 0.33 ± 5% -0.1 0.19 ± 10% perf-profile.children.cycles-pp.preempt_schedule_common 0.42 ± 2% -0.1 0.28 ± 8% perf-profile.children.cycles-pp.wake_up_q 0.70 ± 3% -0.1 0.59 ± 3% perf-profile.children.cycles-pp.__might_sleep 0.65 ± 5% -0.1 0.54 ± 7% perf-profile.children.cycles-pp.__fsnotify_parent 0.52 ± 2% -0.1 0.43 perf-profile.children.cycles-pp.__might_fault 0.53 ± 5% -0.1 0.45 perf-profile.children.cycles-pp.current_time 0.84 ± 3% -0.1 0.77 perf-profile.children.cycles-pp.fsnotify 0.16 ± 7% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode 0.15 ± 8% -0.1 0.08 ± 15% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.22 ± 3% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.wake_q_add 0.49 -0.1 0.44 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.26 ± 4% -0.1 0.21 ± 2% perf-profile.children.cycles-pp.rcu_all_qs 0.08 ± 14% -0.1 0.03 ±100% perf-profile.children.cycles-pp.__vfs_read 0.25 ± 4% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.finish_wait 0.38 -0.0 0.34 ± 2% perf-profile.children.cycles-pp.__x64_sys_read 0.43 ± 3% -0.0 0.39 ± 2% perf-profile.children.cycles-pp.prepare_to_wait 0.14 ± 20% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.task_tick_fair 0.16 ± 19% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.scheduler_tick 0.10 ± 8% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.mutex_spin_on_owner 0.18 ± 8% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.inode_has_perm 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.children.cycles-pp.deactivate_task 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.children.cycles-pp.generic_pipe_buf_confirm 0.13 ± 3% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__sb_end_write 0.14 ± 9% -0.0 0.12 perf-profile.children.cycles-pp.timespec64_trunc 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.native_irq_return_iret 0.11 ± 4% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.17 ± 4% -0.0 0.15 perf-profile.children.cycles-pp.iov_iter_init 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.children.cycles-pp.native_apic_msr_eoi_write 0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.__x2apic_send_IPI_dest 0.15 -0.0 0.13 ± 3% perf-profile.children.cycles-pp.rb_insert_color 0.18 ± 2% -0.0 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy 0.08 ± 5% -0.0 0.07 perf-profile.children.cycles-pp.interrupt_entry 0.23 ± 2% +0.0 0.24 perf-profile.children.cycles-pp.cpumask_next 0.67 +0.0 0.69 perf-profile.children.cycles-pp.__calc_delta 0.15 ± 3% +0.0 0.18 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.17 ± 6% +0.1 0.23 ± 3% perf-profile.children.cycles-pp.remove_entity_load_avg 0.14 ± 6% +0.1 0.19 ± 7% perf-profile.children.cycles-pp.attach_entity_load_avg 0.80 ± 3% +0.1 0.86 perf-profile.children.cycles-pp.check_preempt_wakeup 0.37 ± 3% +0.1 0.45 ± 4% perf-profile.children.cycles-pp.account_entity_enqueue 0.79 ± 4% +0.1 0.88 ± 5% perf-profile.children.cycles-pp.finish_task_switch 0.80 ± 2% +0.1 0.89 ± 2% perf-profile.children.cycles-pp.put_prev_entity 0.54 ± 2% +0.1 0.65 perf-profile.children.cycles-pp.account_entity_dequeue 1.01 ± 5% +0.1 1.13 ± 5% perf-profile.children.cycles-pp.set_task_cpu 0.49 ± 7% +0.2 0.65 ± 5% perf-profile.children.cycles-pp.migrate_task_rq_fair 1.23 ± 4% +0.2 1.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 1.15 ± 2% +0.2 1.33 ± 2% perf-profile.children.cycles-pp.__switch_to 1.42 ± 3% +0.2 1.59 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 0.76 ± 3% +0.2 0.96 ± 3% perf-profile.children.cycles-pp.pick_next_entity 2.21 +0.2 2.42 perf-profile.children.cycles-pp.reweight_entity 2.20 ± 2% +0.2 2.43 perf-profile.children.cycles-pp.load_new_mm_cr3 1.24 ± 5% +0.2 1.48 ± 4% perf-profile.children.cycles-pp.update_rq_clock 1.43 ± 5% +0.3 1.73 ± 2% perf-profile.children.cycles-pp.___perf_sw_event 0.38 ± 15% +0.3 0.68 ± 9% perf-profile.children.cycles-pp.find_next_bit 3.96 +0.3 4.28 perf-profile.children.cycles-pp.pick_next_task_fair 4.43 ± 2% +0.4 4.82 perf-profile.children.cycles-pp.switch_mm_irqs_off 1.11 +0.7 1.83 ± 4% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.46 ± 26% +0.7 1.20 ± 12% perf-profile.children.cycles-pp.cpumask_next_wrap 87.76 +1.1 88.86 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 86.98 +1.1 88.10 perf-profile.children.cycles-pp.do_syscall_64 21.71 +1.3 23.02 perf-profile.children.cycles-pp.__schedule 5.98 ± 4% +1.4 7.34 ± 3% perf-profile.children.cycles-pp.exit_to_usermode_loop 21.59 +1.4 22.99 perf-profile.children.cycles-pp.schedule 1.73 ± 29% +3.3 5.03 ± 13% perf-profile.children.cycles-pp.available_idle_cpu 35.55 +3.3 38.88 perf-profile.children.cycles-pp.ksys_write 34.31 +3.5 37.84 perf-profile.children.cycles-pp.vfs_write 31.14 +4.1 35.23 ± 2% perf-profile.children.cycles-pp.new_sync_write 30.56 +4.2 34.74 ± 2% perf-profile.children.cycles-pp.pipe_write 22.96 ± 2% +6.0 28.93 ± 3% perf-profile.children.cycles-pp.__wake_up_common_lock 20.11 ± 2% +6.1 26.19 ± 3% perf-profile.children.cycles-pp.try_to_wake_up 20.66 ± 2% +6.2 26.82 ± 3% perf-profile.children.cycles-pp.__wake_up_common 20.01 ± 2% +6.2 26.19 ± 3% perf-profile.children.cycles-pp.autoremove_wake_function 3.46 ± 20% +7.0 10.48 ± 12% perf-profile.children.cycles-pp.select_idle_sibling 4.92 ± 15% +7.1 12.01 ± 10% perf-profile.children.cycles-pp.select_task_rq_fair 11.85 ± 3% -2.1 9.72 ± 2% perf-profile.self.cycles-pp.do_syscall_64 4.13 ± 5% -1.3 2.85 ± 8% perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 3.49 ± 6% -1.0 2.52 ± 9% perf-profile.self.cycles-pp.update_cfs_group 4.33 ± 2% -0.7 3.64 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret 4.24 ± 2% -0.6 3.60 ± 3% perf-profile.self.cycles-pp.entry_SYSCALL_64 2.12 ± 4% -0.5 1.59 ± 6% perf-profile.self.cycles-pp.mutex_unlock 2.61 -0.3 2.31 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.60 ± 2% -0.3 1.30 ± 5% perf-profile.self.cycles-pp.update_load_avg 1.48 ± 3% -0.2 1.25 ± 2% perf-profile.self.cycles-pp.mutex_lock 1.95 -0.2 1.75 ± 7% perf-profile.self.cycles-pp.selinux_file_permission 1.06 ± 5% -0.2 0.86 ± 5% perf-profile.self.cycles-pp.pipe_write 1.46 ± 2% -0.2 1.28 ± 3% perf-profile.self.cycles-pp.pipe_read 1.65 ± 2% -0.2 1.49 ± 4% perf-profile.self.cycles-pp.__fget_light 0.91 ± 3% -0.2 0.74 perf-profile.self.cycles-pp.avc_has_perm 0.94 ± 2% -0.2 0.79 ± 2% perf-profile.self.cycles-pp.___might_sleep 1.84 -0.1 1.72 ± 2% perf-profile.self.cycles-pp.update_curr 1.05 -0.1 0.93 perf-profile.self.cycles-pp.enqueue_task_fair 0.63 ± 3% -0.1 0.52 ± 2% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 6% -0.1 0.51 ± 7% perf-profile.self.cycles-pp.__fsnotify_parent 0.29 ± 3% -0.1 0.19 ± 5% perf-profile.self.cycles-pp.__mutex_lock 0.38 ± 4% -0.1 0.29 ± 4% perf-profile.self.cycles-pp.vfs_write 0.51 -0.1 0.43 ± 4% perf-profile.self.cycles-pp.copy_page_to_iter 0.49 -0.1 0.41 ± 2% perf-profile.self.cycles-pp.new_sync_write 0.35 ± 4% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.copy_page_from_iter 0.81 ± 3% -0.1 0.75 perf-profile.self.cycles-pp.fsnotify 0.26 -0.1 0.19 ± 3% perf-profile.self.cycles-pp.__inode_security_revalidate 0.20 ± 4% -0.1 0.13 ± 6% perf-profile.self.cycles-pp.__mutex_unlock_slowpath 0.29 ± 5% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.file_has_perm 0.22 -0.1 0.16 ± 8% perf-profile.self.cycles-pp.wake_q_add 0.57 ± 2% -0.1 0.52 perf-profile.self.cycles-pp.new_sync_read 0.34 ± 2% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.dequeue_entity 0.26 ± 3% -0.1 0.21 ± 3% perf-profile.self.cycles-pp.ksys_read 0.28 ± 2% -0.0 0.23 perf-profile.self.cycles-pp.security_file_permission 0.22 ± 3% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.finish_wait 0.25 -0.0 0.21 ± 4% perf-profile.self.cycles-pp.ksys_write 0.88 -0.0 0.83 perf-profile.self.cycles-pp.__lock_text_start 0.66 -0.0 0.61 perf-profile.self.cycles-pp.vfs_read 1.37 -0.0 1.32 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.29 ± 4% -0.0 0.25 perf-profile.self.cycles-pp.current_time 0.20 ± 3% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.rcu_all_qs 0.10 ± 8% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.mutex_spin_on_owner 0.42 -0.0 0.39 ± 2% perf-profile.self.cycles-pp.__x86_indirect_thunk_rax 0.34 -0.0 0.31 ± 2% perf-profile.self.cycles-pp.__x64_sys_read 0.09 ± 5% -0.0 0.05 ± 9% perf-profile.self.cycles-pp.wake_up_q 0.15 ± 8% -0.0 0.12 ± 4% perf-profile.self.cycles-pp.inode_has_perm 0.14 ± 5% -0.0 0.11 ± 9% perf-profile.self.cycles-pp.rb_next 0.20 ± 2% -0.0 0.17 ± 4% perf-profile.self.cycles-pp.deactivate_task 0.11 ± 4% -0.0 0.08 ± 8% perf-profile.self.cycles-pp.__fdget_pos 0.24 ± 2% -0.0 0.22 ± 3% perf-profile.self.cycles-pp.prepare_to_wait 0.14 ± 3% -0.0 0.11 ± 3% perf-profile.self.cycles-pp.native_irq_return_iret 0.13 ± 7% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.timespec64_trunc 0.06 -0.0 0.04 ± 57% perf-profile.self.cycles-pp.copyout 0.11 ± 4% -0.0 0.09 ± 10% perf-profile.self.cycles-pp.native_apic_msr_eoi_write 0.17 ± 4% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.generic_pipe_buf_confirm 0.18 ± 2% -0.0 0.16 ± 4% perf-profile.self.cycles-pp.__might_fault 0.17 ± 2% -0.0 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy 0.15 ± 2% -0.0 0.13 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.12 ± 3% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.__sb_end_write 0.10 ± 8% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64 0.06 -0.0 0.05 perf-profile.self.cycles-pp.kill_fasync 0.14 ± 3% +0.0 0.16 ± 5% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.66 +0.0 0.68 perf-profile.self.cycles-pp.__calc_delta 0.16 ± 5% +0.0 0.18 ± 2% perf-profile.self.cycles-pp.native_load_tls 0.14 ± 3% +0.0 0.17 ± 4% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.67 +0.0 0.70 perf-profile.self.cycles-pp.dequeue_task_fair 0.23 ± 3% +0.0 0.27 ± 3% perf-profile.self.cycles-pp._cond_resched 0.17 ± 4% +0.0 0.21 ± 6% perf-profile.self.cycles-pp.activate_task 0.35 ± 3% +0.0 0.40 perf-profile.self.cycles-pp.pick_next_entity 1.20 +0.1 1.25 perf-profile.self.cycles-pp.select_task_rq_fair 0.13 ± 6% +0.1 0.18 ± 6% perf-profile.self.cycles-pp.attach_entity_load_avg 0.87 ± 3% +0.1 0.96 ± 3% perf-profile.self.cycles-pp.pick_next_task_fair 0.27 ± 6% +0.1 0.36 ± 7% perf-profile.self.cycles-pp.migrate_task_rq_fair 0.32 ± 2% +0.1 0.41 ± 4% perf-profile.self.cycles-pp.account_entity_enqueue 0.42 +0.1 0.56 ± 2% perf-profile.self.cycles-pp.account_entity_dequeue 2.24 ± 3% +0.1 2.39 ± 2% perf-profile.self.cycles-pp.switch_mm_irqs_off 1.09 ± 2% +0.2 1.26 ± 2% perf-profile.self.cycles-pp.__switch_to 1.20 ± 4% +0.2 1.37 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 1.40 ± 3% +0.2 1.58 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 0.89 ± 6% +0.2 1.10 ± 5% perf-profile.self.cycles-pp.update_rq_clock 2.19 ± 2% +0.2 2.42 perf-profile.self.cycles-pp.load_new_mm_cr3 1.33 ± 4% +0.3 1.60 ± 3% perf-profile.self.cycles-pp.___perf_sw_event 0.34 ± 14% +0.3 0.66 ± 9% perf-profile.self.cycles-pp.find_next_bit 0.28 ± 23% +0.4 0.72 ± 12% perf-profile.self.cycles-pp.cpumask_next_wrap 0.89 +0.7 1.58 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 1.04 +0.7 1.76 ± 4% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.65 ± 12% +3.0 3.61 ± 13% perf-profile.self.cycles-pp.select_idle_sibling 1.71 ± 29% +3.3 4.97 ± 13% perf-profile.self.cycles-pp.available_idle_cpu *************************************************************************************************** lkp-cfl-e1: 16 threads Intel(R) Xeon(R) E-2278G CPU @ 3.40GHz with 32G memory ========================================================================================= compiler/cpufreq_governor/ipc/kconfig/mode/nr_threads/rootfs/tbox_group/testcase/ucode: gcc-7/performance/pipe/x86_64-rhel-7.6/threads/100%/debian-x86_64-2018-04-03.cgz/lkp-cfl-e1/hackbench/0xb8 commit: 43e9f7f231 ("sched/fair: Start tracking SCHED_IDLE tasks count in cfs_rq") 3c29e651e1 ("sched/fair: Fall back to sched-idle CPU if idle CPU isn't found") 43e9f7f231e40e45 3c29e651e16dd3b3179cfb2d055 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:2 -50% :4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x 1:2 -50% :4 dmesg.WARNING:stack_recursion %stddev %change %stddev \ | \ 49845 -6.8% 46466 hackbench.throughput 8.776e+08 +13.0% 9.915e+08 hackbench.time.involuntary_context_switches 93623 -6.3% 87707 hackbench.time.minor_page_faults 740.08 -1.4% 729.54 hackbench.time.user_time 1.569e+09 +3.5% 1.623e+09 hackbench.time.voluntary_context_switches 3.072e+08 -6.2% 2.88e+08 hackbench.workload 21.45 -1.7% 21.09 boot-time.boot 1747212 ± 3% -31.0% 1205072 ± 14% cpuidle.POLL.time 1289000 ± 4% -36.6% 817407 ± 13% cpuidle.POLL.usage 4038486 +6.3% 4293331 vmstat.system.cs 394711 -10.2% 354353 ± 2% vmstat.system.in 10059 -15.4% 8506 ± 5% softirqs.CPU0.SCHED 309881 -27.3% 225159 ± 4% softirqs.CPU2.TIMER 220105 +20.8% 265807 ± 16% softirqs.CPU6.TIMER 120937 ± 3% -17.6% 99667 ± 2% softirqs.SCHED 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.active_objs 18883 -8.9% 17200 ± 3% slabinfo.kmalloc-32.num_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.active_objs 8573 ± 2% -7.6% 7924 slabinfo.lsm_file_cache.num_objs 5280 -8.8% 4814 slabinfo.task_delay_info.active_objs 5280 -8.8% 4814 slabinfo.task_delay_info.num_objs 65295 -1.9% 64034 proc-vmstat.nr_active_anon 60605 -1.5% 59671 proc-vmstat.nr_anon_pages 12935 -0.9% 12820 proc-vmstat.nr_slab_unreclaimable 65295 -1.9% 64034 proc-vmstat.nr_zone_active_anon 24774010 -16.6% 20673098 proc-vmstat.numa_hit 24774010 -16.6% 20673098 proc-vmstat.numa_local 24839243 -16.5% 20735391 proc-vmstat.pgalloc_normal 673024 -2.4% 657142 proc-vmstat.pgfault 24822112 -16.5% 20717243 proc-vmstat.pgfree 247.95 ± 14% -19.1% 200.59 ± 4% sched_debug.cfs_rq:/.load_avg.max 43.55 ± 3% -19.9% 34.86 ± 6% sched_debug.cfs_rq:/.load_avg.min 55.51 ± 6% -13.2% 48.19 ± 7% sched_debug.cfs_rq:/.load_avg.stddev 20116 ± 8% -12.8% 17531 ± 5% sched_debug.cfs_rq:/.runnable_weight.min 1288 +7.5% 1384 ± 3% sched_debug.cfs_rq:/.util_avg.max 54763 ± 9% +130.8% 126390 ± 25% sched_debug.cpu.avg_idle.avg 301502 ± 18% +84.9% 557406 ± 14% sched_debug.cpu.avg_idle.max 82233 ± 22% +122.7% 183153 ± 17% sched_debug.cpu.avg_idle.stddev 0.00 ± 12% -12.0% 0.00 ± 8% sched_debug.cpu.next_balance.stddev 16.92 ± 4% -7.5% 15.66 ± 3% sched_debug.cpu.nr_running.avg 2.92 ± 10% -20.9% 2.31 ± 16% sched_debug.cpu.nr_uninterruptible.avg 2977 ± 50% -73.0% 803.75 ± 43% interrupts.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 546.00 ± 23% +119.5% 1198 ± 16% interrupts.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13831842 -11.5% 12245495 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts 45259 ± 5% +68.9% 76440 ± 15% interrupts.CPU0.TRM:Thermal_event_interrupts 2977 ± 50% -73.0% 803.75 ± 43% interrupts.CPU1.133:IR-PCI-MSI.2097154-edge.eth1-TxRx-1 45283 ± 5% +68.9% 76474 ± 15% interrupts.CPU1.TRM:Thermal_event_interrupts 13947887 -11.0% 12413074 interrupts.CPU10.RES:Rescheduling_interrupts 45275 ± 5% +68.9% 76468 ± 15% interrupts.CPU10.TRM:Thermal_event_interrupts 45286 ± 5% +68.9% 76468 ± 15% interrupts.CPU11.TRM:Thermal_event_interrupts 13579741 -11.8% 11974820 ± 5% interrupts.CPU12.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU12.TRM:Thermal_event_interrupts 13668073 -11.3% 12119346 interrupts.CPU13.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76474 ± 15% interrupts.CPU13.TRM:Thermal_event_interrupts 13578574 -7.8% 12518854 interrupts.CPU14.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76467 ± 15% interrupts.CPU14.TRM:Thermal_event_interrupts 13689117 -10.1% 12310290 ± 2% interrupts.CPU15.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76473 ± 15% interrupts.CPU15.TRM:Thermal_event_interrupts 13924470 -10.5% 12461613 interrupts.CPU2.RES:Rescheduling_interrupts 45264 ± 5% +68.9% 76467 ± 15% interrupts.CPU2.TRM:Thermal_event_interrupts 546.00 ± 23% +119.5% 1198 ± 16% interrupts.CPU3.135:IR-PCI-MSI.2097156-edge.eth1-TxRx-3 13951884 -10.5% 12480739 interrupts.CPU3.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76469 ± 15% interrupts.CPU3.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76471 ± 15% interrupts.CPU4.TRM:Thermal_event_interrupts 14030690 -10.7% 12530992 ± 3% interrupts.CPU5.RES:Rescheduling_interrupts 45286 ± 5% +68.9% 76471 ± 15% interrupts.CPU5.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76474 ± 15% interrupts.CPU6.TRM:Thermal_event_interrupts 45285 ± 5% +68.9% 76472 ± 15% interrupts.CPU7.TRM:Thermal_event_interrupts 13319036 -10.2% 11966842 ± 2% interrupts.CPU8.RES:Rescheduling_interrupts 45280 ± 5% +68.7% 76405 ± 15% interrupts.CPU8.TRM:Thermal_event_interrupts 13401328 -11.8% 11816711 ± 3% interrupts.CPU9.RES:Rescheduling_interrupts 45279 ± 5% +68.9% 76467 ± 15% interrupts.CPU9.TRM:Thermal_event_interrupts 2.19e+08 -9.8% 1.974e+08 ± 2% interrupts.RES:Rescheduling_interrupts 724498 ± 5% +68.9% 1223435 ± 15% interrupts.TRM:Thermal_event_interrupts 53.44 +4.2% 55.68 perf-stat.i.MPKI 0.12 -0.0 0.10 ± 4% perf-stat.i.cache-miss-rate% 1764490 -13.1% 1533362 ± 3% perf-stat.i.cache-misses 1.767e+09 +3.6% 1.832e+09 perf-stat.i.cache-references 4055785 +6.2% 4306694 perf-stat.i.context-switches 336154 +21.5% 408363 perf-stat.i.cpu-migrations 34528 ± 5% +10.3% 38089 ± 3% perf-stat.i.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.i.dTLB-load-miss-rate% 899577 ± 6% -15.2% 762553 ± 5% perf-stat.i.dTLB-load-misses 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.i.dTLB-store-miss-rate% 22816 ± 7% -13.2% 19813 ± 4% perf-stat.i.dTLB-store-misses 6.044e+09 -1.2% 5.969e+09 perf-stat.i.dTLB-stores 31331199 -3.4% 30272777 perf-stat.i.iTLB-load-misses 1061 +3.1% 1094 perf-stat.i.instructions-per-iTLB-miss 1077 -2.9% 1046 perf-stat.i.minor-faults 73018 -10.9% 65028 ± 2% perf-stat.i.node-loads 122632 ± 2% -9.3% 111181 ± 4% perf-stat.i.node-stores 1077 -2.9% 1046 perf-stat.i.page-faults 53.29 +4.2% 55.53 perf-stat.overall.MPKI 0.10 -0.0 0.08 ± 3% perf-stat.overall.cache-miss-rate% 25264 +15.3% 29120 ± 3% perf-stat.overall.cycles-between-cache-misses 0.01 ± 6% -0.0 0.01 ± 6% perf-stat.overall.dTLB-load-miss-rate% 0.00 ± 7% -0.0 0.00 ± 4% perf-stat.overall.dTLB-store-miss-rate% 1058 +2.9% 1089 perf-stat.overall.instructions-per-iTLB-miss 65158 +6.6% 69487 perf-stat.overall.path-length 1761814 -13.1% 1530883 ± 3% perf-stat.ps.cache-misses 1.765e+09 +3.7% 1.829e+09 perf-stat.ps.cache-references 4048770 +6.2% 4299578 perf-stat.ps.context-switches 335565 +21.5% 407697 perf-stat.ps.cpu-migrations 898100 ± 6% -15.2% 761293 ± 5% perf-stat.ps.dTLB-load-misses 22777 ± 7% -13.2% 19781 ± 4% perf-stat.ps.dTLB-store-misses 6.034e+09 -1.2% 5.96e+09 perf-stat.ps.dTLB-stores 31277565 -3.4% 30222833 perf-stat.ps.iTLB-load-misses 1075 -2.9% 1044 perf-stat.ps.minor-faults 72896 -10.9% 64922 ± 2% perf-stat.ps.node-loads 122424 ± 2% -9.3% 111005 ± 4% perf-stat.ps.node-stores 1075 -2.9% 1044 perf-stat.ps.page-faults 39.37 ±100% -36.4 2.98 ±173% perf-profile.calltrace.cycles-pp.start_thread 21.68 ±100% -20.0 1.64 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_write.start_thread 20.14 ±100% -18.6 1.52 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 20.00 ±100% -18.5 1.51 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 18.05 ±100% -16.7 1.39 ±173% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write.start_thread 16.75 ±100% -15.5 1.22 ±173% perf-profile.calltrace.cycles-pp.__GI___libc_read.start_thread 16.70 ±100% -15.4 1.26 ±173% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_write 14.98 ±100% -13.9 1.09 ±173% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 14.82 ±100% -13.7 1.08 ±173% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 13.95 ±100% -12.9 1.03 ±173% perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read.start_thread 12.97 ±100% -12.0 0.95 ±173% perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.__GI___libc_read 3.69 ± 6% -1.1 2.58 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common 4.25 ± 6% -1.0 3.29 ± 6% perf-profile.calltrace.cycles-pp._raw_spin_lock.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 1.54 ± 10% -0.5 1.05 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 1.07 ± 16% -0.5 0.60 ± 5% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__wake_up_common_lock.pipe_write.new_sync_write 1.14 ± 2% -0.3 0.80 ± 7% perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.__schedule.schedule.pipe_wait 0.55 ± 4% -0.3 0.26 ±100% perf-profile.calltrace.cycles-pp.mutex_unlock.pipe_read.new_sync_read.vfs_read.ksys_read 1.90 ± 9% -0.2 1.67 ± 7% perf-profile.calltrace.cycles-pp.__fget_light.__fdget_pos.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.71 ± 7% -0.2 1.48 ± 7% perf-profile.calltrace.cycles-pp.__fget.__fget_light.__fdget_pos.ksys_write.do_syscall_64 0.68 ± 19% +0.2 0.86 perf-profile.calltrace.cycles-pp.update_cfs_group.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.67 ± 16% +0.2 0.88 ± 4% perf-profile.calltrace.cycles-pp.update_cfs_group.dequeue_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.64 ± 4% perf-profile.calltrace.cycles-pp.___perf_sw_event.__schedule.schedule.pipe_wait.pipe_read 0.31 ±100% +0.4 0.67 ± 4% perf-profile.calltrace.cycles-pp.mutex_lock.pipe_wait.pipe_read.new_sync_read.vfs_read 0.32 ±100% +0.4 0.68 ± 2% perf-profile.calltrace.cycles-pp.pick_next_entity.pick_next_task_fair.__schedule.schedule.pipe_wait 0.29 ±100% +0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 0.35 ±100% +0.4 0.78 perf-profile.calltrace.cycles-pp.update_curr.reweight_entity.dequeue_task_fair.__schedule.schedule 1.23 ± 19% +0.4 1.68 ± 2% perf-profile.calltrace.cycles-pp.reweight_entity.dequeue_task_fair.__schedule.schedule.pipe_wait 1.29 ± 18% +0.5 1.76 ± 5% perf-profile.calltrace.cycles-pp.reweight_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 0.00 +0.5 0.53 ± 2% perf-profile.calltrace.cycles-pp.update_load_avg.set_next_entity.pick_next_task_fair.__schedule.schedule 0.00 +0.6 0.59 ± 7% perf-profile.calltrace.cycles-pp.__enqueue_entity.put_prev_entity.pick_next_task_fair.__schedule.schedule 8.82 +0.6 9.46 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 0.47 ±100% +0.7 1.14 ± 6% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop 2.21 ± 10% +0.7 2.88 ± 4% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_usermode_loop.do_syscall_64 9.10 +0.7 9.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 9.04 +0.7 9.74 ± 2% perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 0.71 ±100% +0.9 1.61 ± 3% perf-profile.calltrace.cycles-pp.__switch_to 0.68 ± 99% +0.9 1.60 ± 3% perf-profile.calltrace.cycles-pp.native_write_msr 0.86 ±100% +0.9 1.79 ± 2% perf-profile.calltrace.cycles-pp.__switch_to_asm 6.54 +1.0 7.53 ± 2% perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.pipe_wait.pipe_read 26.96 +1.0 27.95 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.42 ± 63% +1.0 2.47 ± 3% perf-profile.calltrace.cycles-pp.switch_fpu_return.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.54 ± 8% +1.2 5.79 ± 4% perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 14.99 +1.3 16.27 perf-profile.calltrace.cycles-pp.__schedule.schedule.pipe_wait.pipe_read.new_sync_read 15.35 +1.3 16.66 perf-profile.calltrace.cycles-pp.schedule.pipe_wait.pipe_read.new_sync_read.vfs_read 27.37 +1.3 28.70 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 17.81 +1.6 19.38 perf-profile.calltrace.cycles-pp.pipe_wait.pipe_read.new_sync_read.vfs_read.ksys_read 3.95 ± 11% +1.8 5.70 ± 6% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common 23.38 +2.0 25.33 ± 2% perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 24.30 +2.0 26.27 ± 2% perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 23.73 +2.0 25.71 ± 2% perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 5.24 ± 10% +2.0 7.25 ± 6% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 3.23 ± 57% +2.7 5.98 ± 4% perf-profile.calltrace.cycles-pp.schedule.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.34 ± 57% +2.8 6.18 ± 4% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.37 ±100% -36.4 2.98 ±173% perf-profile.children.cycles-pp.start_thread 21.68 ±100% -20.0 1.67 ±173% perf-profile.children.cycles-pp.__GI___libc_write 16.99 ±100% -15.7 1.24 ±173% perf-profile.children.cycles-pp.__GI___libc_read 6.84 ± 10% -2.1 4.71 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 45.35 -1.0 44.33 perf-profile.children.cycles-pp.ksys_write 6.70 ± 5% -1.0 5.70 ± 2% perf-profile.children.cycles-pp._raw_spin_lock 5.90 ± 4% -0.6 5.34 ± 6% perf-profile.children.cycles-pp.security_file_permission 3.35 ± 9% -0.6 2.80 ± 3% perf-profile.children.cycles-pp._raw_spin_lock_irqsave 42.14 -0.5 41.63 perf-profile.children.cycles-pp.vfs_write 3.41 -0.4 3.01 perf-profile.children.cycles-pp.__fdget_pos 3.30 -0.4 2.91 perf-profile.children.cycles-pp.__fget_light 2.71 ± 2% -0.4 2.32 perf-profile.children.cycles-pp.__fget 3.23 -0.3 2.93 ± 3% perf-profile.children.cycles-pp.mutex_lock 2.76 -0.3 2.47 ± 5% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 2.21 ± 3% -0.3 1.93 ± 10% perf-profile.children.cycles-pp.file_has_perm 3.14 -0.2 2.90 ± 4% perf-profile.children.cycles-pp.copy_page_to_iter 3.03 ± 5% -0.2 2.79 ± 5% perf-profile.children.cycles-pp.selinux_file_permission 2.10 ± 5% -0.2 1.86 ± 2% perf-profile.children.cycles-pp.mutex_unlock 1.50 ± 5% -0.2 1.27 ± 2% perf-profile.children.cycles-pp.fput_many 0.68 ± 17% -0.2 0.48 ± 4% perf-profile.children.cycles-pp.__mutex_lock 1.66 ± 2% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.copyout 1.73 -0.2 1.57 ± 2% perf-profile.children.cycles-pp.update_rq_clock 1.30 -0.1 1.17 ± 6% perf-profile.children.cycles-pp.copyin 0.27 ± 11% -0.1 0.15 ± 9% perf-profile.children.cycles-pp.preempt_schedule_common 1.25 ± 3% -0.1 1.15 ± 3% perf-profile.children.cycles-pp.fsnotify 0.79 -0.1 0.69 ± 7% perf-profile.children.cycles-pp.reschedule_interrupt 0.91 -0.1 0.82 ± 5% perf-profile.children.cycles-pp._cond_resched 0.80 ± 2% -0.1 0.71 ± 3% perf-profile.children.cycles-pp.file_update_time 0.29 ± 10% -0.1 0.24 ± 5% perf-profile.children.cycles-pp.rb_insert_color 0.26 -0.0 0.21 ± 2% perf-profile.children.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.children.cycles-pp.scheduler_ipi 0.22 ± 6% -0.0 0.18 ± 4% perf-profile.children.cycles-pp.__x64_sys_read 0.25 ± 4% -0.0 0.21 ± 2% perf-profile.children.cycles-pp.__x86_indirect_thunk_rax 0.27 -0.0 0.23 ± 9% perf-profile.children.cycles-pp.timespec64_trunc 0.23 ± 2% -0.0 0.20 ± 5% perf-profile.children.cycles-pp.smp_reschedule_interrupt 0.22 -0.0 0.20 ± 5% perf-profile.children.cycles-pp.__x64_sys_write 0.24 -0.0 0.22 ± 3% perf-profile.children.cycles-pp.__list_add_valid 0.14 -0.0 0.12 ± 10% perf-profile.children.cycles-pp.kill_fasync 0.11 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_exclude_event 0.10 +0.0 0.12 ± 8% perf-profile.children.cycles-pp.is_cpu_allowed 0.09 ± 11% +0.0 0.12 ± 7% perf-profile.children.cycles-pp.rcu_note_context_switch 0.07 +0.0 0.10 ± 10% perf-profile.children.cycles-pp.pkg_thermal_notify 0.17 ± 3% +0.0 0.20 ± 9% perf-profile.children.cycles-pp.deactivate_task 0.16 +0.0 0.21 ± 10% perf-profile.children.cycles-pp.update_cfs_rq_h_load 0.08 ± 6% +0.0 0.12 ± 10% perf-profile.children.cycles-pp.intel_thermal_interrupt 0.08 ± 6% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.smp_thermal_interrupt 0.08 +0.1 0.13 ± 9% perf-profile.children.cycles-pp.thermal_interrupt 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.default_wake_function 0.19 ± 5% +0.1 0.25 ± 5% perf-profile.children.cycles-pp.wakeup_preempt_entity 0.19 ± 5% +0.1 0.27 ± 4% perf-profile.children.cycles-pp.finish_wait 0.89 ± 8% +0.1 0.98 perf-profile.children.cycles-pp.__calc_delta 0.63 ± 4% +0.1 0.74 ± 4% perf-profile.children.cycles-pp.native_sched_clock 0.33 ± 4% +0.1 0.44 ± 5% perf-profile.children.cycles-pp.set_next_buddy 1.23 ± 3% +0.1 1.34 ± 3% perf-profile.children.cycles-pp.check_preempt_wakeup 0.69 ± 4% +0.1 0.81 ± 3% perf-profile.children.cycles-pp.sched_clock 1.52 ± 3% +0.1 1.65 ± 2% perf-profile.children.cycles-pp.set_next_entity 0.73 ± 4% +0.1 0.88 ± 3% perf-profile.children.cycles-pp.sched_clock_cpu 0.57 ± 4% +0.2 0.74 ± 4% perf-profile.children.cycles-pp.account_entity_dequeue 1.21 ± 4% +0.2 1.39 ± 2% perf-profile.children.cycles-pp.__update_load_avg_se 1.99 ± 3% +0.2 2.20 ± 3% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 1.64 ± 6% +0.2 1.85 perf-profile.children.cycles-pp.__switch_to_asm 1.60 ± 4% +0.2 1.85 ± 3% perf-profile.children.cycles-pp.update_cfs_group 3.91 +0.2 4.15 perf-profile.children.cycles-pp.update_curr 1.04 ± 5% +0.3 1.30 ± 4% perf-profile.children.cycles-pp.___perf_sw_event 1.05 ± 9% +0.3 1.33 ± 2% perf-profile.children.cycles-pp.pick_next_entity 1.57 ± 3% +0.3 1.89 perf-profile.children.cycles-pp.native_write_msr 1.01 ± 9% +0.3 1.33 ± 5% perf-profile.children.cycles-pp.put_prev_entity 2.17 ± 7% +0.3 2.51 ± 2% perf-profile.children.cycles-pp.switch_fpu_return 1.93 ± 7% +0.4 2.36 perf-profile.children.cycles-pp.__switch_to 2.99 ± 4% +0.6 3.57 ± 2% perf-profile.children.cycles-pp.reweight_entity 8.89 +0.6 9.50 ± 2% perf-profile.children.cycles-pp.enqueue_task_fair 9.16 +0.7 9.82 perf-profile.children.cycles-pp.ttwu_do_activate 9.09 +0.7 9.76 ± 2% perf-profile.children.cycles-pp.activate_task 5.18 ± 6% +0.9 6.04 ± 2% perf-profile.children.cycles-pp.pick_next_task_fair 6.71 +0.9 7.63 ± 2% perf-profile.children.cycles-pp.dequeue_task_fair 28.01 +1.4 29.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common_lock 4.91 ± 7% +1.4 6.28 ± 4% perf-profile.children.cycles-pp.exit_to_usermode_loop 18.19 +1.4 19.56 perf-profile.children.cycles-pp.pipe_wait 4.03 ± 11% +1.7 5.75 ± 6% perf-profile.children.cycles-pp.select_idle_sibling 23.86 ± 2% +1.8 25.71 ± 2% perf-profile.children.cycles-pp.try_to_wake_up 24.44 +1.9 26.36 ± 2% perf-profile.children.cycles-pp.__wake_up_common 23.84 +1.9 25.77 ± 2% perf-profile.children.cycles-pp.autoremove_wake_function 5.29 ± 10% +2.0 7.29 ± 6% perf-profile.children.cycles-pp.select_task_rq_fair 20.27 ± 2% +2.3 22.53 ± 2% perf-profile.children.cycles-pp.__schedule 20.36 ± 2% +2.5 22.82 ± 2% perf-profile.children.cycles-pp.schedule 6.82 ± 10% -2.1 4.69 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 2.67 ± 2% -0.4 2.29 perf-profile.self.cycles-pp.__fget 2.72 -0.3 2.44 ± 5% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 1.22 -0.2 0.98 ± 2% perf-profile.self.cycles-pp.update_rq_clock 1.36 ± 3% -0.2 1.13 ± 6% perf-profile.self.cycles-pp.pipe_write 1.48 ± 5% -0.2 1.24 ± 3% perf-profile.self.cycles-pp.fput_many 2.04 ± 5% -0.2 1.81 ± 2% perf-profile.self.cycles-pp.mutex_unlock 1.96 -0.2 1.73 ± 2% perf-profile.self.cycles-pp.mutex_lock 0.66 ± 12% -0.1 0.53 ± 10% perf-profile.self.cycles-pp.file_has_perm 0.38 ± 10% -0.1 0.28 ± 9% perf-profile.self.cycles-pp.ksys_read 0.34 ± 7% -0.1 0.25 ± 7% perf-profile.self.cycles-pp.__mutex_lock 1.21 ± 3% -0.1 1.11 ± 2% perf-profile.self.cycles-pp.fsnotify 1.00 -0.1 0.94 ± 5% perf-profile.self.cycles-pp.__might_sleep 0.60 ± 4% -0.1 0.55 ± 8% perf-profile.self.cycles-pp.vfs_write 0.29 ± 10% -0.1 0.23 ± 3% perf-profile.self.cycles-pp.rb_insert_color 0.32 ± 6% -0.0 0.28 ± 6% perf-profile.self.cycles-pp.ksys_write 0.26 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.native_irq_return_iret 0.07 ± 14% -0.0 0.03 ±100% perf-profile.self.cycles-pp.scheduler_ipi 0.36 ± 5% -0.0 0.32 ± 8% perf-profile.self.cycles-pp.__sb_start_write 0.29 -0.0 0.24 ± 4% perf-profile.self.cycles-pp.check_preempt_curr 0.22 ± 6% -0.0 0.17 ± 6% perf-profile.self.cycles-pp.__x64_sys_read 0.25 -0.0 0.21 ± 8% perf-profile.self.cycles-pp.timespec64_trunc 0.22 ± 4% -0.0 0.19 ± 11% perf-profile.self.cycles-pp.wake_q_add 0.21 ± 2% -0.0 0.18 ± 6% perf-profile.self.cycles-pp.__x64_sys_write 0.21 ± 2% -0.0 0.20 perf-profile.self.cycles-pp.__list_add_valid 0.11 ± 4% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.kill_fasync 0.06 ± 9% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.rcu_note_context_switch 0.15 +0.0 0.18 ± 6% perf-profile.self.cycles-pp.autoremove_wake_function 0.16 +0.0 0.19 ± 9% perf-profile.self.cycles-pp.deactivate_task 0.26 +0.0 0.29 ± 5% perf-profile.self.cycles-pp.rcu_all_qs 0.33 ± 3% +0.0 0.37 ± 4% perf-profile.self.cycles-pp.set_next_entity 0.18 ± 5% +0.0 0.22 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity 0.03 ±100% +0.0 0.07 perf-profile.self.cycles-pp.check_cfs_rq_runtime 0.16 ± 6% +0.0 0.20 ± 6% perf-profile.self.cycles-pp.exit_to_usermode_loop 0.16 +0.0 0.20 ± 12% perf-profile.self.cycles-pp.update_cfs_rq_h_load 0.20 ± 2% +0.1 0.25 ± 2% perf-profile.self.cycles-pp.activate_task 0.41 ± 6% +0.1 0.46 perf-profile.self.cycles-pp.schedule 1.96 +0.1 2.02 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.15 ± 6% +0.1 0.22 perf-profile.self.cycles-pp.finish_wait 0.41 ± 4% +0.1 0.48 ± 6% perf-profile.self.cycles-pp.account_entity_enqueue 0.61 ± 4% +0.1 0.71 ± 3% perf-profile.self.cycles-pp.native_sched_clock 0.83 ± 2% +0.1 0.93 perf-profile.self.cycles-pp.dequeue_task_fair 0.29 ± 3% +0.1 0.40 ± 5% perf-profile.self.cycles-pp.set_next_buddy 1.02 +0.1 1.15 ± 7% perf-profile.self.cycles-pp.enqueue_task_fair 0.46 ± 12% +0.2 0.61 ± 3% perf-profile.self.cycles-pp.pick_next_entity 1.17 ± 4% +0.2 1.35 ± 2% perf-profile.self.cycles-pp.__update_load_avg_se 0.42 ± 8% +0.2 0.61 ± 5% perf-profile.self.cycles-pp.account_entity_dequeue 1.64 ± 6% +0.2 1.85 perf-profile.self.cycles-pp.__switch_to_asm 1.23 ± 4% +0.2 1.46 ± 3% perf-profile.self.cycles-pp.reweight_entity 1.07 ± 5% +0.2 1.31 ± 4% perf-profile.self.cycles-pp.select_task_rq_fair 0.90 ± 5% +0.2 1.14 ± 5% perf-profile.self.cycles-pp.___perf_sw_event 1.57 ± 4% +0.2 1.82 ± 2% perf-profile.self.cycles-pp.update_cfs_group 1.17 ± 3% +0.3 1.42 ± 7% perf-profile.self.cycles-pp.update_load_avg 1.56 ± 4% +0.3 1.88 perf-profile.self.cycles-pp.native_write_msr 2.16 ± 7% +0.3 2.50 ± 2% perf-profile.self.cycles-pp.switch_fpu_return 1.84 ± 8% +0.4 2.25 ± 2% perf-profile.self.cycles-pp.__switch_to 1.22 +0.5 1.73 ± 4% perf-profile.self.cycles-pp._raw_spin_lock 0.92 ± 11% +1.3 2.27 ± 6% perf-profile.self.cycles-pp.select_idle_sibling Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

2 months

4
6
0 / 0

[ptrace] 201766a20e: kernel_selftests.seccomp.make_fail

by kernel test robot

FYI, we noticed the following commit (built with gcc-7): commit: 201766a20e30f982ccfe36bebfad9602c3ff574a ("ptrace: add PTRACE_GET_SYSCALL_INFO request") https://kernel.googlesource.com/pub/scm/linux/kernel/git/torvalds/linux.git master in testcase: kernel_selftests with following parameters: group: kselftests-02 test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel. test-url: https://www.kernel.org/doc/Documentation/kselftest.txt on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace): If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> 2019-07-26 16:52:03 make run_tests -C seccomp make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp' gcc -Wl,-no-as-needed -Wall seccomp_bpf.c -lpthread -o seccomp_bpf In file included from seccomp_bpf.c:51:0: seccomp_bpf.c: In function ‘tracer_ptrace’: seccomp_bpf.c:1787:20: error: ‘PTRACE_EVENTMSG_SYSCALL_ENTRY’ undeclared (first use in this function) EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ seccomp_bpf.c:1787:20: note: each undeclared identifier is reported only once for each function it appears in EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ seccomp_bpf.c:1788:6: error: ‘PTRACE_EVENTMSG_SYSCALL_EXIT’ undeclared (first use in this function) : PTRACE_EVENTMSG_SYSCALL_EXIT, msg); ^ ../kselftest_harness.h:608:13: note: in definition of macro ‘__EXPECT’ __typeof__(_expected) __exp = (_expected); \ ^~~~~~~~~ seccomp_bpf.c:1787:2: note: in expansion of macro ‘EXPECT_EQ’ EXPECT_EQ(entry ? PTRACE_EVENTMSG_SYSCALL_ENTRY ^~~~~~~~~ Makefile:12: recipe for target 'seccomp_bpf' failed make: *** [seccomp_bpf] Error 1 make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-201766a20e30f982ccfe36bebfad9602c3ff574a/tools/testing/selftests/seccomp' To reproduce: # build kernel cd linux cp config-5.2.0-10889-g201766a20e30f9 .config make HOSTCC=gcc-7 CC=gcc-7 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email Thanks, Rong Chen

4 months, 2 weeks

3
3
0 / 0

[x86, sched] 1567c3e346: vm-scalability.median -15.8% regression

by kernel test robot

Greeting, FYI, we noticed a -15.8% regression of vm-scalability.median due to commit: commit: 1567c3e3467cddeb019a7b53ec632f834b6a9239 ("x86, sched: Add support for frequency invariance") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: vm-scalability on test machine: 144 threads Intel(R) Xeon(R) CPU E7-8890 v3 @ 2.50GHz with 512G memory with following parameters: runtime: 300s size: 8T test: anon-cow-seq cpufreq_governor: performance ucode: 0x16 test-description: The motivation behind this suite is to exercise functions and regions of the mm/ of the Linux kernel which are of interest to us. test-url: https://git.kernel.org/cgit/linux/kernel/git/wfg/vm-scalability.git/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <oliver.sang(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/runtime/size/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/debian-x86_64-20191114.cgz/300s/8T/lkp-hsw-4ex1/anon-cow-seq/vm-scalability/0x16 commit: 2a4b03ffc6 ("sched/fair: Prevent unlimited runtime on throttled group") 1567c3e346 ("x86, sched: Add support for frequency invariance") 2a4b03ffc69f2ded 1567c3e3467cddeb019a7b53ec6 ---------------- --------------------------- %stddev %change %stddev \ | \ 210443 -15.8% 177280 vm-scalability.median 0.07 ± 6% -64.9% 0.02 ± 16% vm-scalability.median_stddev 30242018 -16.0% 25399016 vm-scalability.throughput 241304 ± 2% +53.9% 371337 ± 2% vm-scalability.time.involuntary_context_switches 8973780 -12.4% 7862380 vm-scalability.time.minor_page_faults 10507 +7.6% 11301 vm-scalability.time.percent_of_cpu_this_job_got 17759 +22.0% 21664 ± 2% vm-scalability.time.system_time 14801 -10.4% 13262 vm-scalability.time.user_time 1023965 -63.3% 376066 ± 4% vm-scalability.time.voluntary_context_switches 7.967e+09 -11.1% 7.082e+09 vm-scalability.workload 2985 -11.4% 2643 ± 3% slabinfo.khugepaged_mm_slot.active_objs 2985 -11.4% 2643 ± 3% slabinfo.khugepaged_mm_slot.num_objs 36024164 -42.9% 20554898 ± 40% cpuidle.C1.time 7.277e+09 ± 41% -72.1% 2.027e+09 ±160% cpuidle.C3.time 17453257 ± 19% -74.4% 4462508 ±163% cpuidle.C3.usage 214574 ± 10% -50.2% 106926 ± 33% cpuidle.POLL.usage 26.26 -5.4 20.89 mpstat.cpu.all.idle% 0.02 ± 8% -0.0 0.01 ± 9% mpstat.cpu.all.iowait% 40.23 +8.8 49.05 mpstat.cpu.all.sys% 33.49 -3.4 30.05 ± 2% mpstat.cpu.all.usr% 1989988 -20.6% 1580840 ± 4% numa-numastat.node1.local_node 2026261 -20.6% 1608134 ± 4% numa-numastat.node1.numa_hit 3614123 ± 44% -56.4% 1576459 ± 5% numa-numastat.node2.local_node 3641345 ± 44% -56.0% 1603122 ± 5% numa-numastat.node2.numa_hit 26.50 -20.8% 21.00 vmstat.cpu.id 32.75 -10.7% 29.25 vmstat.cpu.us 9800 -34.3% 6434 vmstat.system.cs 290059 -6.9% 270063 ± 4% vmstat.system.in 1.012e+08 +12.3% 1.136e+08 meminfo.Active 1.012e+08 +12.3% 1.136e+08 meminfo.Active(anon) 92123057 +14.3% 1.053e+08 meminfo.AnonHugePages 1.009e+08 +12.3% 1.134e+08 meminfo.AnonPages 1.044e+08 +11.9% 1.169e+08 meminfo.Memused 25495045 +15.0% 29324284 numa-meminfo.node1.Active 25494976 +15.0% 29324244 numa-meminfo.node1.Active(anon) 23164225 +17.2% 27157731 numa-meminfo.node1.AnonHugePages 25417044 +15.1% 29267058 numa-meminfo.node1.AnonPages 26192254 +14.4% 29959010 numa-meminfo.node1.MemUsed 24839952 ± 3% +18.2% 29349199 numa-meminfo.node2.Active 24839901 ± 3% +18.2% 29349040 numa-meminfo.node2.Active(anon) 22604826 ± 3% +20.4% 27211067 numa-meminfo.node2.AnonHugePages 24777054 ± 3% +18.2% 29282273 numa-meminfo.node2.AnonPages 25718372 ± 2% +16.8% 30037056 numa-meminfo.node2.MemUsed 199045 ± 96% -99.6% 743.75 ± 52% numa-meminfo.node2.PageTables 54935 ± 20% -24.5% 41470 ± 10% numa-meminfo.node2.SUnreclaim 418.75 ± 94% -93.7% 26.50 ± 49% numa-meminfo.node3.Inactive(file) 4864 ± 15% +26.9% 6174 ± 10% numa-meminfo.node3.KernelStack 95371 ±172% +233.9% 318444 ± 57% numa-meminfo.node3.PageTables 6347307 +14.8% 7286035 numa-vmstat.node1.nr_active_anon 6328825 +14.9% 7268986 numa-vmstat.node1.nr_anon_pages 11261 +16.9% 13162 numa-vmstat.node1.nr_anon_transparent_hugepages 6347284 +14.8% 7285923 numa-vmstat.node1.nr_zone_active_anon 1324428 -15.2% 1122566 ± 3% numa-vmstat.node1.numa_hit 1204942 -16.0% 1011998 ± 3% numa-vmstat.node1.numa_local 6207902 ± 3% +17.7% 7308204 numa-vmstat.node2.nr_active_anon 6195059 ± 3% +17.6% 7287608 numa-vmstat.node2.nr_anon_pages 11047 ± 3% +19.7% 13218 numa-vmstat.node2.nr_anon_transparent_hugepages 49684 ± 96% -99.6% 186.00 ± 52% numa-vmstat.node2.nr_page_table_pages 13734 ± 20% -24.5% 10367 ± 10% numa-vmstat.node2.nr_slab_unreclaimable 6207738 ± 3% +17.7% 7308081 numa-vmstat.node2.nr_zone_active_anon 2125860 ± 40% -49.5% 1073183 ± 2% numa-vmstat.node2.numa_hit 2015486 ± 42% -52.2% 963317 ± 2% numa-vmstat.node2.numa_local 104.25 ± 94% -93.8% 6.50 ± 49% numa-vmstat.node3.nr_inactive_file 4864 ± 15% +26.7% 6160 ± 10% numa-vmstat.node3.nr_kernel_stack 23833 ±172% +230.0% 78649 ± 57% numa-vmstat.node3.nr_page_table_pages 104.25 ± 94% -93.8% 6.50 ± 49% numa-vmstat.node3.nr_zone_inactive_file 1.12 ± 19% -0.5 0.63 ± 20% perf-profile.calltrace.cycles-pp.do_huge_pmd_numa_page.__handle_mm_fault.handle_mm_fault.__do_page_fault.do_page_fault 0.81 ± 19% -0.2 0.60 ± 20% perf-profile.calltrace.cycles-pp.reuse_swap_page.do_huge_pmd_wp_page.__handle_mm_fault.handle_mm_fault.__do_page_fault 0.33 ± 22% -0.2 0.17 ± 4% perf-profile.children.cycles-pp.migrate_misplaced_transhuge_page 0.09 ± 23% -0.1 0.03 ±100% perf-profile.children.cycles-pp.ttwu_do_activate 0.08 ± 19% -0.1 0.03 ±100% perf-profile.children.cycles-pp.enqueue_entity 0.15 ± 20% -0.1 0.10 ± 15% perf-profile.children.cycles-pp.wake_up_page_bit 0.14 ± 23% -0.1 0.09 ± 14% perf-profile.children.cycles-pp.__wake_up_common 0.14 ± 20% -0.1 0.09 ± 13% perf-profile.children.cycles-pp.autoremove_wake_function 0.09 ± 20% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.enqueue_task_fair 0.09 ± 23% -0.0 0.04 ± 58% perf-profile.children.cycles-pp.activate_task 0.15 ± 21% -0.0 0.11 ± 17% perf-profile.children.cycles-pp.try_to_wake_up 0.15 ± 11% +0.0 0.17 ± 4% perf-profile.children.cycles-pp.schedule 0.08 ± 10% +0.0 0.12 ± 6% perf-profile.children.cycles-pp.newidle_balance 0.12 ± 16% +0.0 0.17 ± 9% perf-profile.children.cycles-pp.find_busiest_group 0.00 +0.1 0.05 perf-profile.children.cycles-pp.balance_fair 0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.arch_scale_freq_tick 0.18 ± 10% +0.1 0.23 ± 8% perf-profile.children.cycles-pp.load_balance 0.00 +0.1 0.09 ± 9% perf-profile.children.cycles-pp.smpboot_thread_fn 0.40 ± 23% +0.3 0.68 ± 17% perf-profile.children.cycles-pp.task_tick_fair 0.01 ±173% +0.3 0.30 ± 16% perf-profile.children.cycles-pp.update_cfs_group 0.01 ±173% +0.3 0.30 ± 16% perf-profile.self.cycles-pp.update_cfs_group 8587 ± 12% -29.1% 6090 ± 9% sched_debug.cfs_rq:/.exec_clock.stddev 0.53 ± 45% -92.1% 0.04 ±173% sched_debug.cfs_rq:/.load_avg.min 0.54 ± 19% +38.3% 0.74 ± 10% sched_debug.cfs_rq:/.nr_running.avg 4.81 ± 12% +266.0% 17.59 ± 7% sched_debug.cfs_rq:/.nr_spread_over.avg 24.01 ± 8% +96.4% 47.14 ± 4% sched_debug.cfs_rq:/.nr_spread_over.max 5.02 ± 10% +123.0% 11.19 ± 6% sched_debug.cfs_rq:/.nr_spread_over.stddev 313.47 ± 18% -55.1% 140.75 ± 51% sched_debug.cfs_rq:/.runnable_load_avg.max 31.75 ± 9% -54.5% 14.46 ± 44% sched_debug.cfs_rq:/.runnable_load_avg.stddev 0.54 ± 19% +38.2% 0.75 ± 9% sched_debug.cpu.nr_running.avg 11567 ± 8% -28.4% 8277 ± 7% sched_debug.cpu.nr_switches.avg 4760 ± 9% -31.7% 3249 ± 4% sched_debug.cpu.nr_switches.stddev -36.17 +31.8% -47.65 sched_debug.cpu.nr_uninterruptible.min 15.88 ± 2% +35.3% 21.49 ± 7% sched_debug.cpu.nr_uninterruptible.stddev 9718 ± 10% -34.1% 6402 ± 9% sched_debug.cpu.sched_count.avg 4294 ± 8% -37.4% 2688 ± 8% sched_debug.cpu.sched_count.stddev 3936 ± 10% -53.6% 1828 ± 12% sched_debug.cpu.sched_goidle.avg 2045 ± 8% -50.0% 1022 ± 10% sched_debug.cpu.sched_goidle.stddev 4616 ± 9% -36.5% 2933 ± 10% sched_debug.cpu.ttwu_count.avg 11005 ± 9% -22.0% 8579 ± 9% sched_debug.cpu.ttwu_count.max 2271 ± 8% -34.9% 1478 ± 8% sched_debug.cpu.ttwu_count.stddev 915.40 ± 8% +42.3% 1303 ± 6% sched_debug.cpu.ttwu_local.avg 3927 ± 5% +19.3% 4684 ± 10% sched_debug.cpu.ttwu_local.max 488.40 ± 9% +40.6% 686.55 ± 3% sched_debug.cpu.ttwu_local.stddev 25252488 +12.3% 28363408 proc-vmstat.nr_active_anon 25182652 +12.4% 28296707 proc-vmstat.nr_anon_pages 44870 +14.2% 51263 proc-vmstat.nr_anon_transparent_hugepages 148.25 -5.7% 139.75 proc-vmstat.nr_dirtied 10541824 -3.0% 10230220 proc-vmstat.nr_dirty_background_threshold 21109424 -3.0% 20485453 proc-vmstat.nr_dirty_threshold 1.06e+08 -2.9% 1.029e+08 proc-vmstat.nr_free_pages 369.75 -1.4% 364.50 proc-vmstat.nr_inactive_file 99622 +7.3% 106911 proc-vmstat.nr_page_table_pages 144.25 -4.9% 137.25 proc-vmstat.nr_written 25252485 +12.3% 28363396 proc-vmstat.nr_zone_active_anon 369.75 -1.4% 364.50 proc-vmstat.nr_zone_inactive_file 2953466 -10.6% 2639571 proc-vmstat.numa_hint_faults 661711 -9.8% 596566 proc-vmstat.numa_hint_faults_local 11245421 -11.8% 9923535 proc-vmstat.numa_hit 5838019 -7.0% 5428548 proc-vmstat.numa_huge_pte_updates 11136518 -11.9% 9814538 proc-vmstat.numa_local 201352 ± 12% +234.6% 673715 ± 7% proc-vmstat.numa_pages_migrated 2.99e+09 -7.0% 2.78e+09 proc-vmstat.numa_pte_updates 2.382e+09 -18.9% 1.931e+09 proc-vmstat.pgalloc_normal 10046733 -11.2% 8918565 proc-vmstat.pgfault 2.381e+09 -18.9% 1.93e+09 proc-vmstat.pgfree 5.942e+08 -42.6% 3.409e+08 proc-vmstat.pgmigrate_fail 201352 ± 12% +234.6% 673715 ± 7% proc-vmstat.pgmigrate_success 3478289 -11.1% 3091553 proc-vmstat.thp_fault_alloc 8.85 ± 7% -16.8% 7.36 ± 12% perf-stat.i.MPKI 2.766e+10 ± 4% -7.4% 2.562e+10 ± 3% perf-stat.i.branch-instructions 20.31 +1.8 22.10 ± 4% perf-stat.i.cache-miss-rate% 5.089e+08 ± 4% -9.5% 4.604e+08 ± 3% perf-stat.i.cache-references 9955 ± 3% -33.0% 6665 ± 4% perf-stat.i.context-switches 1.73 ± 4% +9.9% 1.90 ± 3% perf-stat.i.cpi 1.287e+11 ± 4% +11.2% 1.431e+11 ± 3% perf-stat.i.cpu-cycles 485.10 ± 5% +108.0% 1009 ± 4% perf-stat.i.cpu-migrations 2.215e+10 ± 4% -6.5% 2.071e+10 ± 3% perf-stat.i.dTLB-loads 7.427e+09 ± 4% -8.1% 6.829e+09 ± 3% perf-stat.i.dTLB-stores 958368 ± 5% -30.9% 662160 ± 2% perf-stat.i.iTLB-load-misses 8.707e+10 ± 4% -7.4% 8.062e+10 ± 3% perf-stat.i.instructions 142754 ± 2% +11.3% 158927 ± 2% perf-stat.i.instructions-per-iTLB-miss 0.67 -13.3% 0.58 perf-stat.i.ipc 33135 ± 4% -10.7% 29584 ± 2% perf-stat.i.minor-faults 70.79 +6.0 76.82 perf-stat.i.node-load-miss-rate% 41028170 ± 4% +14.1% 46801837 ± 3% perf-stat.i.node-load-misses 1.85 ± 5% +0.9 2.78 ± 5% perf-stat.i.node-store-miss-rate% 584076 ± 6% +70.0% 992640 ± 4% perf-stat.i.node-store-misses 53445296 ± 5% -19.0% 43298204 ± 7% perf-stat.i.node-stores 33137 ± 4% -10.7% 29584 ± 2% perf-stat.i.page-faults 5.85 -1.7% 5.75 perf-stat.overall.MPKI 1.48 +19.6% 1.77 perf-stat.overall.cpi 1156 +16.7% 1350 ± 5% perf-stat.overall.cycles-between-cache-misses 92129 ± 6% +25.2% 115378 perf-stat.overall.instructions-per-iTLB-miss 0.67 -16.4% 0.56 perf-stat.overall.ipc 75.40 +3.8 79.17 perf-stat.overall.node-load-miss-rate% 1.11 ± 2% +1.2 2.33 ± 7% perf-stat.overall.node-store-miss-rate% 2.704e+10 -11.0% 2.408e+10 perf-stat.ps.branch-instructions 1.091e+08 -8.5% 99896941 ± 5% perf-stat.ps.cache-misses 4.979e+08 -12.5% 4.357e+08 perf-stat.ps.cache-references 9775 -34.7% 6386 ± 2% perf-stat.ps.context-switches 1.262e+11 +6.5% 1.344e+11 perf-stat.ps.cpu-cycles 487.90 ± 5% +100.7% 979.41 perf-stat.ps.cpu-migrations 2.166e+10 -10.1% 1.948e+10 perf-stat.ps.dTLB-loads 7.26e+09 -11.3% 6.441e+09 perf-stat.ps.dTLB-stores 927296 ± 5% -29.1% 657143 perf-stat.ps.iTLB-load-misses 8.512e+10 -10.9% 7.581e+10 perf-stat.ps.instructions 32070 -11.2% 28469 perf-stat.ps.minor-faults 40537100 +9.2% 44251909 ± 4% perf-stat.ps.node-load-misses 13226459 ± 2% -12.0% 11644845 ± 4% perf-stat.ps.node-loads 582949 ± 3% +65.6% 965166 ± 3% perf-stat.ps.node-store-misses 52116857 -21.9% 40679765 ± 8% perf-stat.ps.node-stores 32070 -11.2% 28469 perf-stat.ps.page-faults 2.642e+13 -11.4% 2.341e+13 perf-stat.total.instructions 229.75 ± 52% -34.1% 151.50 interrupts.102:PCI-MSI.1572921-edge.eth0-TxRx-57 156.75 ± 3% +27.3% 199.50 ± 29% interrupts.47:PCI-MSI.1572866-edge.eth0-TxRx-2 1603 ± 22% -15.6% 1352 ± 4% interrupts.CPU1.NMI:Non-maskable_interrupts 1603 ± 22% -15.6% 1352 ± 4% interrupts.CPU1.PMI:Performance_monitoring_interrupts 1007 ± 19% -38.7% 618.00 ± 8% interrupts.CPU100.RES:Rescheduling_interrupts 1016 ± 19% -33.9% 671.75 ± 5% interrupts.CPU101.RES:Rescheduling_interrupts 2046 ± 32% -19.9% 1638 ± 30% interrupts.CPU102.NMI:Non-maskable_interrupts 2046 ± 32% -19.9% 1638 ± 30% interrupts.CPU102.PMI:Performance_monitoring_interrupts 956.75 ± 18% -33.1% 640.25 ± 5% interrupts.CPU102.RES:Rescheduling_interrupts 909.50 ± 20% -29.1% 644.50 ± 12% interrupts.CPU103.RES:Rescheduling_interrupts 959.50 ± 24% -31.4% 658.00 ± 5% interrupts.CPU104.RES:Rescheduling_interrupts 921.75 ± 14% -30.2% 643.50 ± 3% interrupts.CPU105.RES:Rescheduling_interrupts 901.50 ± 14% -27.8% 650.50 ± 11% interrupts.CPU106.RES:Rescheduling_interrupts 2670 ± 3% -37.5% 1670 ± 31% interrupts.CPU107.NMI:Non-maskable_interrupts 2670 ± 3% -37.5% 1670 ± 31% interrupts.CPU107.PMI:Performance_monitoring_interrupts 2849 -31.6% 1948 ± 33% interrupts.CPU109.NMI:Non-maskable_interrupts 2849 -31.6% 1948 ± 33% interrupts.CPU109.PMI:Performance_monitoring_interrupts 1410 ± 20% +51.0% 2130 ± 23% interrupts.CPU11.NMI:Non-maskable_interrupts 1410 ± 20% +51.0% 2130 ± 23% interrupts.CPU11.PMI:Performance_monitoring_interrupts 2820 ± 3% -30.7% 1955 ± 32% interrupts.CPU111.NMI:Non-maskable_interrupts 2820 ± 3% -30.7% 1955 ± 32% interrupts.CPU111.PMI:Performance_monitoring_interrupts 2808 -30.1% 1962 ± 33% interrupts.CPU116.NMI:Non-maskable_interrupts 2808 -30.1% 1962 ± 33% interrupts.CPU116.PMI:Performance_monitoring_interrupts 622190 -8.1% 571803 ± 4% interrupts.CPU118.LOC:Local_timer_interrupts 621861 -8.3% 570381 ± 4% interrupts.CPU119.LOC:Local_timer_interrupts 2024 ± 25% -30.0% 1417 ± 8% interrupts.CPU12.NMI:Non-maskable_interrupts 2024 ± 25% -30.0% 1417 ± 8% interrupts.CPU12.PMI:Performance_monitoring_interrupts 622130 -8.0% 572612 ± 4% interrupts.CPU120.LOC:Local_timer_interrupts 622145 -8.4% 569980 ± 4% interrupts.CPU121.LOC:Local_timer_interrupts 2131 ± 33% -24.4% 1611 ± 26% interrupts.CPU121.NMI:Non-maskable_interrupts 2131 ± 33% -24.4% 1611 ± 26% interrupts.CPU121.PMI:Performance_monitoring_interrupts 621983 -7.9% 572602 ± 4% interrupts.CPU123.LOC:Local_timer_interrupts 2219 ± 34% -40.0% 1332 ± 5% interrupts.CPU124.NMI:Non-maskable_interrupts 2219 ± 34% -40.0% 1332 ± 5% interrupts.CPU124.PMI:Performance_monitoring_interrupts 622061 -8.0% 572198 ± 4% interrupts.CPU125.LOC:Local_timer_interrupts 817.50 ± 12% -46.8% 435.25 ± 13% interrupts.CPU128.RES:Rescheduling_interrupts 820.75 ± 24% -48.8% 420.00 ± 25% interrupts.CPU129.RES:Rescheduling_interrupts 750.00 ± 25% -42.9% 428.00 ± 24% interrupts.CPU130.RES:Rescheduling_interrupts 2285 ± 22% -38.1% 1415 interrupts.CPU133.NMI:Non-maskable_interrupts 2285 ± 22% -38.1% 1415 interrupts.CPU133.PMI:Performance_monitoring_interrupts 779.25 ± 30% -50.5% 386.00 ± 23% interrupts.CPU133.RES:Rescheduling_interrupts 741.25 ± 28% -47.7% 387.75 ± 28% interrupts.CPU135.RES:Rescheduling_interrupts 728.00 ± 28% -46.2% 391.50 ± 31% interrupts.CPU137.RES:Rescheduling_interrupts 749.25 ± 29% -53.9% 345.50 ± 27% interrupts.CPU138.RES:Rescheduling_interrupts 751.25 ± 27% -49.0% 383.00 ± 29% interrupts.CPU139.RES:Rescheduling_interrupts 688.00 ± 24% -49.8% 345.25 ± 27% interrupts.CPU140.RES:Rescheduling_interrupts 726.00 ± 27% -55.1% 326.25 ± 32% interrupts.CPU141.RES:Rescheduling_interrupts 2272 ± 29% -40.7% 1347 ± 2% interrupts.CPU15.NMI:Non-maskable_interrupts 2272 ± 29% -40.7% 1347 ± 2% interrupts.CPU15.PMI:Performance_monitoring_interrupts 156.75 ± 3% +27.3% 199.50 ± 29% interrupts.CPU2.47:PCI-MSI.1572866-edge.eth0-TxRx-2 2562 ± 9% -38.6% 1573 ± 28% interrupts.CPU2.NMI:Non-maskable_interrupts 2562 ± 9% -38.6% 1573 ± 28% interrupts.CPU2.PMI:Performance_monitoring_interrupts 1344 ± 20% -32.8% 903.75 interrupts.CPU20.RES:Rescheduling_interrupts 1314 ± 16% -29.9% 921.50 ± 5% interrupts.CPU21.RES:Rescheduling_interrupts 120.00 ± 47% -98.3% 2.00 ± 93% interrupts.CPU22.TLB:TLB_shootdowns 2245 ± 24% -35.7% 1443 ± 24% interrupts.CPU23.NMI:Non-maskable_interrupts 2245 ± 24% -35.7% 1443 ± 24% interrupts.CPU23.PMI:Performance_monitoring_interrupts 1338 ± 18% -32.8% 899.00 ± 12% interrupts.CPU23.RES:Rescheduling_interrupts 1243 ± 16% -28.0% 895.75 ± 4% interrupts.CPU24.RES:Rescheduling_interrupts 1360 ± 30% -33.3% 907.50 ± 15% interrupts.CPU25.RES:Rescheduling_interrupts 1273 ± 20% -34.0% 841.00 ± 6% interrupts.CPU26.RES:Rescheduling_interrupts 1273 ± 15% -31.3% 874.25 ± 8% interrupts.CPU27.RES:Rescheduling_interrupts 1315 ± 16% -40.6% 781.00 ± 11% interrupts.CPU28.RES:Rescheduling_interrupts 1320 ± 20% -39.2% 802.75 ± 5% interrupts.CPU29.RES:Rescheduling_interrupts 2094 ± 30% -22.7% 1619 ± 35% interrupts.CPU30.NMI:Non-maskable_interrupts 2094 ± 30% -22.7% 1619 ± 35% interrupts.CPU30.PMI:Performance_monitoring_interrupts 1301 ± 19% -40.6% 773.00 ± 5% interrupts.CPU30.RES:Rescheduling_interrupts 1253 ± 16% -36.9% 791.00 ± 15% interrupts.CPU31.RES:Rescheduling_interrupts 1231 ± 17% -41.5% 720.25 ± 8% interrupts.CPU32.RES:Rescheduling_interrupts 1177 ± 11% -37.3% 738.00 ± 14% interrupts.CPU33.RES:Rescheduling_interrupts 1195 ± 18% -34.0% 788.75 ± 21% interrupts.CPU34.RES:Rescheduling_interrupts 2567 ± 8% -35.1% 1666 ± 35% interrupts.CPU35.NMI:Non-maskable_interrupts 2567 ± 8% -35.1% 1666 ± 35% interrupts.CPU35.PMI:Performance_monitoring_interrupts 1223 ± 18% -39.1% 745.25 ± 8% interrupts.CPU35.RES:Rescheduling_interrupts 621753 -7.9% 572936 ± 4% interrupts.CPU36.LOC:Local_timer_interrupts 2527 ± 10% -34.0% 1667 ± 31% interrupts.CPU40.NMI:Non-maskable_interrupts 2527 ± 10% -34.0% 1667 ± 31% interrupts.CPU40.PMI:Performance_monitoring_interrupts 622323 -7.9% 572967 ± 4% interrupts.CPU45.LOC:Local_timer_interrupts 1832 ± 17% +38.7% 2541 ± 3% interrupts.CPU46.NMI:Non-maskable_interrupts 1832 ± 17% +38.7% 2541 ± 3% interrupts.CPU46.PMI:Performance_monitoring_interrupts 621906 -8.0% 572330 ± 4% interrupts.CPU47.LOC:Local_timer_interrupts 621972 -8.1% 571378 ± 4% interrupts.CPU48.LOC:Local_timer_interrupts 622275 -8.2% 571410 ± 4% interrupts.CPU49.LOC:Local_timer_interrupts 2459 ± 24% -41.7% 1433 ± 10% interrupts.CPU49.NMI:Non-maskable_interrupts 2459 ± 24% -41.7% 1433 ± 10% interrupts.CPU49.PMI:Performance_monitoring_interrupts 622031 -7.9% 573184 ± 4% interrupts.CPU50.LOC:Local_timer_interrupts 622081 -7.9% 572943 ± 4% interrupts.CPU51.LOC:Local_timer_interrupts 622140 -7.8% 573769 ± 4% interrupts.CPU52.LOC:Local_timer_interrupts 622074 -7.6% 574558 ± 4% interrupts.CPU53.LOC:Local_timer_interrupts 229.75 ± 52% -34.1% 151.50 interrupts.CPU57.102:PCI-MSI.1572921-edge.eth0-TxRx-57 1052 ± 38% -59.2% 429.25 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts 830.75 ± 26% -53.3% 388.25 ± 27% interrupts.CPU62.RES:Rescheduling_interrupts 774.50 ± 28% -48.5% 399.00 ± 32% interrupts.CPU67.RES:Rescheduling_interrupts 785.00 ± 33% -51.6% 380.00 ± 32% interrupts.CPU68.RES:Rescheduling_interrupts 890.25 ± 31% -58.3% 371.50 ± 27% interrupts.CPU69.RES:Rescheduling_interrupts 821.50 ± 30% -46.4% 440.50 ± 36% interrupts.CPU70.RES:Rescheduling_interrupts 1897 ± 24% +40.4% 2663 ± 4% interrupts.CPU84.NMI:Non-maskable_interrupts 1897 ± 24% +40.4% 2663 ± 4% interrupts.CPU84.PMI:Performance_monitoring_interrupts 1431 ± 17% -43.5% 808.25 ± 21% interrupts.CPU9.RES:Rescheduling_interrupts 1064 ± 18% -30.1% 743.75 ± 16% interrupts.CPU91.RES:Rescheduling_interrupts 1076 ± 10% -39.6% 649.75 ± 10% interrupts.CPU92.RES:Rescheduling_interrupts 1042 ± 17% -33.8% 690.00 ± 10% interrupts.CPU93.RES:Rescheduling_interrupts 1065 ± 18% -33.4% 709.50 ± 5% interrupts.CPU94.RES:Rescheduling_interrupts 2731 ± 7% -30.6% 1894 ± 28% interrupts.CPU95.NMI:Non-maskable_interrupts 2731 ± 7% -30.6% 1894 ± 28% interrupts.CPU95.PMI:Performance_monitoring_interrupts 998.75 ± 19% -28.4% 715.00 ± 7% interrupts.CPU96.RES:Rescheduling_interrupts 2.75 ± 47% +2009.1% 58.00 ±148% interrupts.CPU96.TLB:TLB_shootdowns 952.50 ± 27% -29.5% 671.25 ± 12% interrupts.CPU97.RES:Rescheduling_interrupts 2667 ± 2% -50.2% 1328 ± 4% interrupts.CPU98.NMI:Non-maskable_interrupts 2667 ± 2% -50.2% 1328 ± 4% interrupts.CPU98.PMI:Performance_monitoring_interrupts 973.00 ± 20% -28.5% 695.25 ± 4% interrupts.CPU98.RES:Rescheduling_interrupts 991.00 ± 18% -34.0% 654.00 ± 8% interrupts.CPU99.RES:Rescheduling_interrupts 135782 ± 14% -25.0% 101847 ± 6% interrupts.RES:Rescheduling_interrupts 140559 ± 2% -11.8% 124028 ± 6% softirqs.CPU0.RCU 145814 ± 3% -13.7% 125791 ± 5% softirqs.CPU1.RCU 138760 -12.0% 122124 ± 6% softirqs.CPU100.RCU 16104 -17.4% 13295 ± 5% softirqs.CPU100.SCHED 16422 ± 5% -16.9% 13653 ± 7% softirqs.CPU101.SCHED 15960 -14.5% 13650 ± 6% softirqs.CPU102.SCHED 15804 ± 8% -13.8% 13616 ± 7% softirqs.CPU103.SCHED 139949 -12.0% 123102 ± 5% softirqs.CPU104.RCU 16155 ± 3% -14.6% 13799 ± 8% softirqs.CPU104.SCHED 16018 ± 4% -14.3% 13725 ± 10% softirqs.CPU105.SCHED 15392 ± 5% -14.4% 13172 ± 6% softirqs.CPU106.SCHED 15218 ± 4% -15.5% 12852 ± 9% softirqs.CPU107.SCHED 143209 -14.2% 122899 ± 5% softirqs.CPU108.RCU 144618 -14.9% 123106 ± 4% softirqs.CPU109.RCU 144915 -14.1% 124470 ± 6% softirqs.CPU110.RCU 145038 -14.7% 123770 ± 5% softirqs.CPU111.RCU 138846 -12.2% 121867 ± 6% softirqs.CPU112.RCU 137438 -11.9% 121069 ± 6% softirqs.CPU113.RCU 141152 -13.4% 122234 ± 5% softirqs.CPU114.RCU 141545 -13.8% 122025 ± 5% softirqs.CPU115.RCU 140302 -13.0% 122108 ± 6% softirqs.CPU116.RCU 140548 -13.2% 122057 ± 5% softirqs.CPU117.RCU 142370 -14.3% 122021 ± 5% softirqs.CPU118.RCU 136694 -11.3% 121193 ± 5% softirqs.CPU119.RCU 137193 -11.5% 121415 ± 6% softirqs.CPU120.RCU 137125 -11.9% 120804 ± 5% softirqs.CPU121.RCU 140181 -13.0% 121893 ± 5% softirqs.CPU122.RCU 137405 -11.4% 121741 ± 5% softirqs.CPU124.RCU 139350 -12.5% 122000 ± 6% softirqs.CPU125.RCU 145339 -14.5% 124311 ± 5% softirqs.CPU126.RCU 14887 ± 18% -32.6% 10033 ± 15% softirqs.CPU126.SCHED 144917 -15.6% 122366 ± 5% softirqs.CPU127.RCU 15094 ± 20% -37.0% 9516 ± 21% softirqs.CPU127.SCHED 140946 -13.8% 121461 ± 6% softirqs.CPU128.RCU 138106 ± 2% -11.1% 122812 ± 6% softirqs.CPU129.RCU 137799 -11.3% 122221 ± 6% softirqs.CPU130.RCU 137425 -10.9% 122427 ± 6% softirqs.CPU131.RCU 140854 -12.9% 122753 ± 5% softirqs.CPU132.RCU 15140 ± 24% -35.8% 9716 ± 19% softirqs.CPU132.SCHED 140764 ± 2% -12.7% 122926 ± 5% softirqs.CPU133.RCU 139626 -11.7% 123235 ± 5% softirqs.CPU134.RCU 141210 ± 2% -12.6% 123477 ± 5% softirqs.CPU135.RCU 142996 ± 3% -14.1% 122901 ± 5% softirqs.CPU136.RCU 140567 ± 2% -12.5% 123022 ± 6% softirqs.CPU140.RCU 14192 ± 21% -34.2% 9336 ± 18% softirqs.CPU141.SCHED 135514 ± 2% -9.8% 122294 ± 5% softirqs.CPU142.RCU 134945 ± 2% -10.0% 121501 ± 6% softirqs.CPU143.RCU 145145 -13.0% 126309 ± 5% softirqs.CPU16.RCU 144489 ± 2% -13.3% 125217 ± 5% softirqs.CPU17.RCU 138706 ± 4% -12.0% 122092 ± 5% softirqs.CPU18.RCU 15348 ± 4% -10.0% 13810 ± 8% softirqs.CPU18.SCHED 140870 ± 2% -10.7% 125784 ± 3% softirqs.CPU19.RCU 16392 ± 5% -15.5% 13843 ± 6% softirqs.CPU19.SCHED 143741 ± 2% -12.4% 125852 ± 5% softirqs.CPU2.RCU 140088 -11.7% 123663 ± 5% softirqs.CPU20.RCU 16092 ± 3% -14.0% 13832 ± 5% softirqs.CPU20.SCHED 138206 -11.0% 123025 ± 5% softirqs.CPU21.RCU 16877 ± 4% -16.7% 14051 ± 4% softirqs.CPU21.SCHED 138653 -11.4% 122814 ± 5% softirqs.CPU22.RCU 17090 ± 3% -19.3% 13797 ± 5% softirqs.CPU22.SCHED 16476 ± 3% -15.6% 13910 ± 5% softirqs.CPU23.SCHED 138808 -11.2% 123204 ± 5% softirqs.CPU24.RCU 16779 -16.2% 14060 ± 5% softirqs.CPU24.SCHED 139811 ± 2% -11.6% 123555 ± 5% softirqs.CPU25.RCU 16734 ± 3% -15.8% 14083 ± 7% softirqs.CPU25.SCHED 138909 -10.9% 123758 ± 5% softirqs.CPU26.RCU 16619 ± 5% -15.0% 14133 ± 5% softirqs.CPU26.SCHED 140257 -12.3% 123025 ± 5% softirqs.CPU27.RCU 17013 ± 5% -17.3% 14061 ± 5% softirqs.CPU27.SCHED 137824 -11.2% 122428 ± 6% softirqs.CPU28.RCU 16470 ± 2% -15.4% 13927 ± 6% softirqs.CPU28.SCHED 135294 ± 2% -9.6% 122364 ± 6% softirqs.CPU29.RCU 17350 ± 3% -20.7% 13761 ± 5% softirqs.CPU29.SCHED 142300 ± 3% -13.1% 123621 ± 6% softirqs.CPU3.RCU 17067 -18.4% 13927 ± 5% softirqs.CPU30.SCHED 17110 ± 4% -17.6% 14099 ± 7% softirqs.CPU31.SCHED 145321 -14.0% 124914 ± 4% softirqs.CPU32.RCU 16952 ± 3% -16.0% 14234 ± 7% softirqs.CPU32.SCHED 143684 -14.0% 123636 ± 5% softirqs.CPU33.RCU 16728 ± 2% -17.3% 13838 ± 5% softirqs.CPU33.SCHED 143358 -13.4% 124129 ± 5% softirqs.CPU34.RCU 17103 ± 3% -17.7% 14082 ± 6% softirqs.CPU34.SCHED 143430 -13.0% 124848 ± 5% softirqs.CPU35.RCU 16705 -17.7% 13746 ± 4% softirqs.CPU35.SCHED 139206 -11.5% 123261 ± 7% softirqs.CPU36.RCU 140252 -12.7% 122378 ± 5% softirqs.CPU37.RCU 140086 -12.6% 122492 ± 5% softirqs.CPU39.RCU 140518 -13.4% 121743 ± 5% softirqs.CPU40.RCU 138605 -11.7% 122371 ± 5% softirqs.CPU41.RCU 141519 -13.5% 122418 ± 5% softirqs.CPU42.RCU 142030 -13.7% 122637 ± 5% softirqs.CPU43.RCU 140694 -13.4% 121845 ± 4% softirqs.CPU44.RCU 140504 -12.1% 123545 ± 5% softirqs.CPU45.RCU 140277 -12.7% 122435 ± 5% softirqs.CPU46.RCU 137267 -11.2% 121919 ± 5% softirqs.CPU47.RCU 140839 -12.9% 122646 ± 5% softirqs.CPU48.RCU 141816 -13.3% 122937 ± 5% softirqs.CPU49.RCU 140868 ± 2% -12.0% 123942 ± 6% softirqs.CPU5.RCU 143643 -14.1% 123392 ± 6% softirqs.CPU50.RCU 141326 -12.7% 123403 ± 6% softirqs.CPU51.RCU 141583 -13.4% 122613 ± 6% softirqs.CPU52.RCU 141100 -13.2% 122445 ± 5% softirqs.CPU53.RCU 139956 -12.1% 122981 ± 6% softirqs.CPU54.RCU 14018 ± 17% -29.8% 9845 ± 15% softirqs.CPU54.SCHED 140114 -11.3% 124217 ± 5% softirqs.CPU55.RCU 143329 -14.8% 122069 ± 6% softirqs.CPU56.RCU 141146 -12.3% 123718 ± 6% softirqs.CPU57.RCU 15264 ± 19% -34.6% 9988 ± 17% softirqs.CPU57.SCHED 140809 -12.4% 123330 ± 6% softirqs.CPU58.RCU 15201 ± 20% -36.3% 9678 ± 20% softirqs.CPU58.SCHED 140568 -12.0% 123769 ± 5% softirqs.CPU59.RCU 15360 ± 20% -33.3% 10252 ± 19% softirqs.CPU59.SCHED 139409 ± 3% -10.9% 124281 ± 6% softirqs.CPU6.RCU 141767 -12.7% 123768 ± 5% softirqs.CPU60.RCU 142046 -12.9% 123657 ± 5% softirqs.CPU61.RCU 15389 ± 22% -34.7% 10052 ± 18% softirqs.CPU61.SCHED 141719 -12.7% 123700 ± 5% softirqs.CPU62.RCU 142499 -12.9% 124068 ± 6% softirqs.CPU63.RCU 14667 ± 20% -31.8% 10005 ± 18% softirqs.CPU63.SCHED 144437 -14.1% 124008 ± 6% softirqs.CPU64.RCU 15191 ± 22% -35.9% 9736 ± 19% softirqs.CPU64.SCHED 141156 ± 2% -12.3% 123743 ± 6% softirqs.CPU65.RCU 139859 -11.2% 124140 ± 6% softirqs.CPU66.RCU 141282 -11.9% 124535 ± 6% softirqs.CPU67.RCU 144228 -13.7% 124434 ± 6% softirqs.CPU68.RCU 142536 -13.0% 123956 ± 6% softirqs.CPU69.RCU 141103 ± 2% -11.8% 124488 ± 6% softirqs.CPU7.RCU 140849 ± 2% -11.5% 124683 ± 6% softirqs.CPU70.RCU 141347 ± 2% -12.4% 123863 ± 6% softirqs.CPU71.RCU 142552 ± 3% -13.6% 123193 ± 5% softirqs.CPU72.RCU 142074 ± 4% -12.3% 124659 ± 7% softirqs.CPU73.RCU 141649 -13.1% 123155 ± 5% softirqs.CPU74.RCU 143525 ± 4% -14.0% 123493 ± 6% softirqs.CPU75.RCU 142772 ± 3% -13.9% 122974 ± 7% softirqs.CPU76.RCU 141178 ± 3% -12.9% 122944 ± 5% softirqs.CPU77.RCU 140777 ± 3% -12.8% 122799 ± 5% softirqs.CPU78.RCU 143039 ± 4% -14.3% 122572 ± 5% softirqs.CPU79.RCU 140336 ± 3% -11.8% 123811 ± 6% softirqs.CPU8.RCU 141727 ± 2% -13.0% 123316 ± 5% softirqs.CPU80.RCU 141813 ± 2% -13.0% 123363 ± 5% softirqs.CPU81.RCU 141978 ± 2% -11.6% 125442 ± 3% softirqs.CPU82.RCU 137611 ± 3% -10.7% 122950 ± 5% softirqs.CPU84.RCU 141077 ± 2% -12.0% 124188 ± 5% softirqs.CPU86.RCU 139221 ± 3% -11.9% 122606 ± 5% softirqs.CPU87.RCU 138177 ± 4% -11.7% 121980 ± 5% softirqs.CPU88.RCU 139882 ± 2% -11.1% 124329 ± 5% softirqs.CPU9.RCU 141843 ± 2% -13.6% 122541 ± 6% softirqs.CPU90.RCU 16251 ± 3% -15.5% 13733 ± 5% softirqs.CPU90.SCHED 143175 ± 2% -14.1% 122926 ± 6% softirqs.CPU91.RCU 17088 ± 3% -19.2% 13801 ± 7% softirqs.CPU91.SCHED 144768 -14.7% 123542 ± 5% softirqs.CPU92.RCU 16851 ± 2% -20.1% 13462 ± 4% softirqs.CPU92.SCHED 144398 ± 4% -13.4% 125018 ± 3% softirqs.CPU93.RCU 16917 ± 3% -19.9% 13559 ± 6% softirqs.CPU93.SCHED 142957 ± 2% -13.6% 123521 ± 5% softirqs.CPU94.RCU 16758 ± 2% -17.8% 13773 ± 4% softirqs.CPU94.SCHED 141885 -13.8% 122263 ± 6% softirqs.CPU95.RCU 16161 ± 5% -14.7% 13789 ± 6% softirqs.CPU95.SCHED 138647 -11.8% 122297 ± 5% softirqs.CPU96.RCU 16590 ± 3% -16.8% 13804 ± 8% softirqs.CPU96.SCHED 15776 ± 3% -13.0% 13731 ± 7% softirqs.CPU97.SCHED 16115 -10.6% 14405 ± 5% softirqs.CPU98.SCHED 140827 -12.8% 122846 ± 5% softirqs.CPU99.RCU 16252 ± 4% -16.2% 13623 ± 5% softirqs.CPU99.SCHED 20190421 -12.1% 17737370 ± 5% softirqs.RCU 2133170 -16.1% 1790167 ± 4% softirqs.SCHED vm-scalability.time.system_time 22500 +-------------------------------------------------------------------+ 22000 |-+ O O | | O O O O | 21500 |-O O O O O O O | 21000 |-+ O O O O O O O O O O O | 20500 |-+ O O O | 20000 |-+ | | | 19500 |-+ | 19000 |-+ | 18500 |-+ + | 18000 |.+.. + + : | | .. + + : .+.. +.. + | 17500 |-+ +.+ + + + + | 17000 +-------------------------------------------------------------------+ vm-scalability.time.percent_of_cpu_this_job_got 11400 +-------------------------------------------------------------------+ 11300 |-+ O O O O O O O O O | | O O O O O O O O O O O O O O O O | 11200 |-+ O O | 11100 |-+ | | | 11000 |-+ | 10900 |-+ | 10800 |-+ | | | 10700 |-+ | 10600 |-+ | |.+.. .+. .+. .+.. .+ | 10500 |-+ +.+. +. +.+..+ + | 10400 +-------------------------------------------------------------------+ vm-scalability.time.voluntary_context_switches 1.3e+06 +-----------------------------------------------------------------+ 1.2e+06 |.+ | | + .+. .+. .+ | 1.1e+06 |-+ + .+.+..+.+..+ +. + | 1e+06 |-+ + | | | 900000 |-+ | 800000 |-+ | 700000 |-+ | | | 600000 |-+ | 500000 |-+ | | O O O O O O O O O O O O O | 400000 |-O O O O O O O O O O O O O O | 300000 +-----------------------------------------------------------------+ vm-scalability.throughput 3.2e+07 +-----------------------------------------------------------------+ | +.. | 3.1e+07 |++ | | +.+. .+. .+.+.+..+.+.+ | 3e+07 |-+ +. +. | | | 2.9e+07 |-+ | | | 2.8e+07 |-+ | | | 2.7e+07 |-+ | | | 2.6e+07 |-+ O O O O O O O O O O O O O | | O O O O O O O O O O O O | 2.5e+07 +-----------------------------------------------------------------+ vm-scalability.median 220000 +------------------------------------------------------------------+ 215000 |-+.. | |+ +.+.. .+. .+.+..+.+.+..+ | 210000 |-+ + +. | 205000 |-+ | | | 200000 |-+ | 195000 |-+ | 190000 |-+ | | | 185000 |-+ | 180000 |-+ O O O O O O O O O O O O O O O | | O O O O O O O O O O O | 175000 |-+ O | 170000 +------------------------------------------------------------------+ vm-scalability.workload 8.4e+09 +-----------------------------------------------------------------+ | + | 8.2e+09 |:++ | |: + | 8e+09 |-+ +.+.+..+.+..+.+.+..+.+.+ | | | 7.8e+09 |-+ | | | 7.6e+09 |-+ | | | 7.4e+09 |-+ | | | 7.2e+09 |-+ | | O O O O O O O O O O O O O O O O O O O O O O O O O O O | 7e+09 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Oliver Sang

6 months

3
6
0 / 0

[sched/fair] 070f5e860e: reaim.jobs_per_min -10.5% regression

by kernel test robot

Greeting, FYI, we noticed a -10.5% regression of reaim.jobs_per_min due to commit: commit: 070f5e860ee2bf588c99ef7b4c202451faa48236 ("sched/fair: Take into account runnable_avg to classify group") https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master in testcase: reaim on test machine: 4 threads Intel(R) Core(TM) i3-3220 CPU @ 3.30GHz with 4G memory with following parameters: runtime: 300s nr_task: 100% test: five_sec cpufreq_governor: performance ucode: 0x21 test-description: REAIM is an updated and improved version of AIM 7 benchmark. test-url: https://sourceforge.net/projects/re-aim-7/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/100%/debian-x86_64-20191114.cgz/300s/lkp-ivb-d04/five_sec/reaim/0x21 commit: 9f68395333 ("sched/pelt: Add a new runnable average signal") 070f5e860e ("sched/fair: Take into account runnable_avg to classify group") 9f68395333ad7f5b 070f5e860ee2bf588c99ef7b4c2 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 4:4 -18% 3:4 perf-profile.children.cycles-pp.error_entry 3:4 -12% 3:4 perf-profile.self.cycles-pp.error_entry %stddev %change %stddev \ | \ 0.68 -10.4% 0.61 reaim.child_systime 67235 -10.5% 60195 reaim.jobs_per_min 16808 -10.5% 15048 reaim.jobs_per_min_child 97.90 -1.2% 96.70 reaim.jti 72000 -10.8% 64216 reaim.max_jobs_per_min 0.36 +11.3% 0.40 reaim.parent_time 1.56 ± 3% +79.1% 2.80 ± 6% reaim.std_dev_percent 0.00 ± 7% +145.9% 0.01 ± 9% reaim.std_dev_time 104276 -16.0% 87616 reaim.time.involuntary_context_switches 15511157 -2.4% 15144312 reaim.time.minor_page_faults 55.00 -7.3% 51.00 reaim.time.percent_of_cpu_this_job_got 88.01 -12.4% 77.12 reaim.time.system_time 79.97 -3.2% 77.38 reaim.time.user_time 216380 -3.4% 208924 reaim.time.voluntary_context_switches 50800 -2.4% 49600 reaim.workload 30.40 ± 2% -4.7% 28.97 ± 2% boot-time.boot 9.38 -0.7 8.66 ± 3% mpstat.cpu.all.sys% 7452 +7.5% 8014 vmstat.system.cs 1457802 ± 16% +49.3% 2176122 ± 13% cpuidle.C1.time 48523684 +43.4% 69570233 ± 22% cpuidle.C1E.time 806543 ± 2% +20.7% 973406 ± 11% cpuidle.C1E.usage 14328 ± 6% +14.5% 16410 ± 8% cpuidle.POLL.time 43300 ± 4% +13.5% 49150 ± 5% softirqs.CPU0.SCHED 118751 -9.3% 107763 softirqs.CPU1.RCU 41679 ± 3% +14.1% 47546 ± 4% softirqs.CPU1.SCHED 42688 ± 3% +12.3% 47931 ± 4% softirqs.CPU2.SCHED 41730 ± 2% +17.7% 49115 ± 4% softirqs.CPU3.SCHED 169399 +14.4% 193744 ± 2% softirqs.SCHED 3419 +1.0% 3453 proc-vmstat.nr_kernel_stack 16365616 -1.8% 16077850 proc-vmstat.numa_hit 16365616 -1.8% 16077850 proc-vmstat.numa_local 93908 -1.6% 92389 proc-vmstat.pgactivate 16269664 -3.9% 15629529 ± 2% proc-vmstat.pgalloc_normal 15918803 -2.3% 15557936 proc-vmstat.pgfault 16644610 -2.0% 16310898 proc-vmstat.pgfree 20125 ±123% +161.7% 52662 ± 30% sched_debug.cfs_rq:/.load.min 348749 ± 10% -11.2% 309562 ± 11% sched_debug.cfs_rq:/.load.stddev 1096 ± 6% -14.4% 938.42 ± 7% sched_debug.cfs_rq:/.load_avg.max 448.46 ± 8% -17.5% 370.19 ± 10% sched_debug.cfs_rq:/.load_avg.stddev 117372 -10.2% 105432 sched_debug.cfs_rq:/.min_vruntime.avg 135242 ± 4% -9.2% 122811 sched_debug.cfs_rq:/.min_vruntime.max 0.53 ± 8% +17.6% 0.62 ± 6% sched_debug.cfs_rq:/.nr_running.avg 29.79 ± 30% -51.0% 14.58 ± 35% sched_debug.cfs_rq:/.nr_spread_over.max 10.21 ± 34% -59.7% 4.12 ± 52% sched_debug.cfs_rq:/.nr_spread_over.stddev 78.25 ± 40% +3304.7% 2664 ± 94% sched_debug.cpu.curr->pid.min 294309 ± 2% +34.3% 395172 ± 12% sched_debug.cpu.nr_switches.min 9.58 ± 35% +84.8% 17.71 ± 40% sched_debug.cpu.nr_uninterruptible.max -6.88 +120.6% -15.17 sched_debug.cpu.nr_uninterruptible.min 6.41 ± 30% +95.2% 12.52 ± 33% sched_debug.cpu.nr_uninterruptible.stddev 286185 +33.4% 381734 ± 13% sched_debug.cpu.sched_count.min 180416 +11.0% 200247 sched_debug.cpu.sched_goidle.avg 116264 ± 3% +44.6% 168090 ± 15% sched_debug.cpu.sched_goidle.min 476.00 ± 8% +92.4% 915.75 ± 3% interrupts.CAL:Function_call_interrupts 110.50 ± 24% +101.1% 222.25 ± 4% interrupts.CPU0.CAL:Function_call_interrupts 1381 ± 29% +23.7% 1709 ± 26% interrupts.CPU0.NMI:Non-maskable_interrupts 1381 ± 29% +23.7% 1709 ± 26% interrupts.CPU0.PMI:Performance_monitoring_interrupts 3319 ± 9% +50.4% 4991 ± 2% interrupts.CPU0.RES:Rescheduling_interrupts 41.25 ± 30% +274.5% 154.50 ± 15% interrupts.CPU0.TLB:TLB_shootdowns 116.25 ± 23% +96.1% 228.00 ± 16% interrupts.CPU1.CAL:Function_call_interrupts 1183 ± 10% +43.1% 1692 ± 23% interrupts.CPU1.NMI:Non-maskable_interrupts 1183 ± 10% +43.1% 1692 ± 23% interrupts.CPU1.PMI:Performance_monitoring_interrupts 3335 ± 7% +60.4% 5350 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts 36.25 ± 30% +344.1% 161.00 ± 8% interrupts.CPU1.TLB:TLB_shootdowns 131.25 ± 11% +81.1% 237.75 ± 11% interrupts.CPU2.CAL:Function_call_interrupts 3247 ± 2% +62.4% 5274 interrupts.CPU2.RES:Rescheduling_interrupts 34.50 ± 36% +357.2% 157.75 ± 7% interrupts.CPU2.TLB:TLB_shootdowns 118.00 ± 13% +93.0% 227.75 ± 9% interrupts.CPU3.CAL:Function_call_interrupts 3155 ± 4% +68.7% 5322 ± 3% interrupts.CPU3.RES:Rescheduling_interrupts 38.50 ± 16% +303.9% 155.50 ± 3% interrupts.CPU3.TLB:TLB_shootdowns 13057 ± 2% +60.4% 20939 interrupts.RES:Rescheduling_interrupts 150.50 ± 27% +317.8% 628.75 ± 3% interrupts.TLB:TLB_shootdowns 2.00 +0.1 2.09 ± 3% perf-stat.i.branch-miss-rate% 10.26 +1.1 11.36 ± 7% perf-stat.i.cache-miss-rate% 2009706 ± 2% +5.4% 2117525 ± 3% perf-stat.i.cache-misses 16867421 -4.5% 16106908 perf-stat.i.cache-references 7514 +7.6% 8083 perf-stat.i.context-switches 1.51 -3.0% 1.47 perf-stat.i.cpi 2.523e+09 ± 3% -8.8% 2.301e+09 ± 2% perf-stat.i.cpu-cycles 124.54 +157.8% 321.08 perf-stat.i.cpu-migrations 1842 ± 10% -18.6% 1498 ± 6% perf-stat.i.cycles-between-cache-misses 752585 ± 2% -4.1% 721714 perf-stat.i.dTLB-store-misses 590441 +2.7% 606399 perf-stat.i.iTLB-load-misses 68766 +4.0% 71488 ± 2% perf-stat.i.iTLB-loads 1.847e+09 ± 3% -4.7% 1.76e+09 ± 2% perf-stat.i.instructions 3490 ± 4% -8.5% 3195 ± 3% perf-stat.i.instructions-per-iTLB-miss 0.68 +3.7% 0.70 perf-stat.i.ipc 51861 -2.1% 50797 perf-stat.i.minor-faults 51861 -2.1% 50797 perf-stat.i.page-faults 2.68 ± 2% +0.1 2.78 perf-stat.overall.branch-miss-rate% 11.91 +1.2 13.14 ± 2% perf-stat.overall.cache-miss-rate% 1.37 -4.3% 1.31 perf-stat.overall.cpi 1255 -13.4% 1087 ± 2% perf-stat.overall.cycles-between-cache-misses 3127 ± 3% -7.2% 2901 ± 2% perf-stat.overall.instructions-per-iTLB-miss 0.73 +4.5% 0.76 perf-stat.overall.ipc 2002763 ± 2% +5.4% 2110303 ± 3% perf-stat.ps.cache-misses 16809816 -4.5% 16051656 perf-stat.ps.cache-references 7489 +7.6% 8055 perf-stat.ps.context-switches 2.514e+09 ± 3% -8.8% 2.293e+09 ± 2% perf-stat.ps.cpu-cycles 124.12 +157.8% 319.95 perf-stat.ps.cpu-migrations 750010 ± 2% -4.1% 719223 perf-stat.ps.dTLB-store-misses 588424 +2.7% 604314 perf-stat.ps.iTLB-load-misses 68533 +4.0% 71246 ± 2% perf-stat.ps.iTLB-loads 1.841e+09 ± 3% -4.7% 1.754e+09 ± 2% perf-stat.ps.instructions 51683 -2.1% 50622 perf-stat.ps.minor-faults 51683 -2.1% 50622 perf-stat.ps.page-faults 5.577e+11 ± 3% -5.1% 5.292e+11 ± 2% perf-stat.total.instructions 7.35 ± 17% -2.7 4.60 ± 10% perf-profile.calltrace.cycles-pp.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit 7.74 ± 20% -2.7 5.00 ± 6% perf-profile.calltrace.cycles-pp.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write.console_unlock 10.14 ± 8% -2.7 7.44 ± 6% perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.66 ± 8% -2.6 8.07 ± 8% perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write 7.10 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.write._fini 7.10 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp._fini 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.write._fini 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write._fini 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.devkmsg_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write 7.09 ± 17% -2.4 4.69 ± 7% perf-profile.calltrace.cycles-pp.devkmsg_emit.devkmsg_write.new_sync_write.vfs_write.ksys_write 6.20 ± 8% -2.1 4.08 ± 5% perf-profile.calltrace.cycles-pp.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write.new_sync_write 5.15 ± 11% -1.8 3.38 ± 4% perf-profile.calltrace.cycles-pp.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write 5.05 ± 11% -1.7 3.31 ± 3% perf-profile.calltrace.cycles-pp.uart_console_write.serial8250_console_write.console_unlock.vprintk_emit.devkmsg_emit 7.41 ± 10% -1.1 6.29 ± 5% perf-profile.calltrace.cycles-pp.__do_execve_file.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 7.57 ± 11% -1.1 6.46 ± 5% perf-profile.calltrace.cycles-pp.execve 7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.__x64_sys_execve.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.execve 7.46 ± 10% -1.1 6.37 ± 5% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.execve 7.03 ± 5% -1.1 5.95 ± 10% perf-profile.calltrace.cycles-pp.brk 5.90 ± 7% -0.9 4.98 ± 10% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.brk 5.84 ± 7% -0.9 4.93 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk 15.77 ± 2% -0.9 14.88 ± 2% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 15.86 ± 2% -0.9 14.97 ± 2% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 3.88 ± 6% -0.9 2.99 ± 5% perf-profile.calltrace.cycles-pp.kill 1.70 ± 23% -0.8 0.90 ± 10% perf-profile.calltrace.cycles-pp.delay_tsc.wait_for_xmitr.serial8250_console_putchar.uart_console_write.serial8250_console_write 4.88 ± 8% -0.8 4.08 ± 8% perf-profile.calltrace.cycles-pp.__x64_sys_brk.do_syscall_64.entry_SYSCALL_64_after_hwframe.brk 2.39 ± 27% -0.7 1.67 ± 5% perf-profile.calltrace.cycles-pp.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file.__x64_sys_execve 2.29 ± 30% -0.7 1.59 ± 5% perf-profile.calltrace.cycles-pp.mmput.flush_old_exec.load_elf_binary.search_binary_handler.__do_execve_file 2.27 ± 30% -0.7 1.58 ± 5% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.flush_old_exec.load_elf_binary.search_binary_handler 3.11 ± 5% -0.6 2.47 ± 9% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.kill 3.07 ± 5% -0.6 2.45 ± 9% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill 2.09 ± 18% -0.4 1.67 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.read 2.82 ± 9% -0.4 2.40 ± 12% perf-profile.calltrace.cycles-pp.mmput.do_exit.do_group_exit.__x64_sys_exit_group.do_syscall_64 2.80 ± 9% -0.4 2.38 ± 12% perf-profile.calltrace.cycles-pp.exit_mmap.mmput.do_exit.do_group_exit.__x64_sys_exit_group 1.11 ± 33% -0.4 0.71 ± 10% perf-profile.calltrace.cycles-pp.unmap_vmas.exit_mmap.mmput.flush_old_exec.load_elf_binary 1.05 ± 15% -0.4 0.69 ± 13% perf-profile.calltrace.cycles-pp.vt_console_print.console_unlock.vprintk_emit.devkmsg_emit.devkmsg_write 1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.lf.vt_console_print.console_unlock.vprintk_emit.devkmsg_emit 1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.con_scroll.lf.vt_console_print.console_unlock.vprintk_emit 1.03 ± 17% -0.4 0.68 ± 13% perf-profile.calltrace.cycles-pp.fbcon_scroll.con_scroll.lf.vt_console_print.console_unlock 0.96 ± 16% -0.3 0.66 ± 12% perf-profile.calltrace.cycles-pp.fbcon_putcs.fbcon_redraw.fbcon_scroll.con_scroll.lf 1.85 ± 4% -0.3 1.58 ± 8% perf-profile.calltrace.cycles-pp.alloc_pages_vma.handle_pte_fault.__handle_mm_fault.handle_mm_fault.do_page_fault 0.89 ± 15% -0.3 0.62 ± 12% perf-profile.calltrace.cycles-pp.kill_pid_info.kill_something_info.__x64_sys_kill.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.67 ± 5% -0.3 1.41 ± 9% perf-profile.calltrace.cycles-pp.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault.handle_mm_fault 1.02 ± 7% -0.3 0.77 ± 12% perf-profile.calltrace.cycles-pp.do_signal.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill 0.94 ± 16% -0.2 0.70 ± 5% perf-profile.calltrace.cycles-pp.clear_page_erms.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma 0.98 ± 16% -0.2 0.74 ± 7% perf-profile.calltrace.cycles-pp.prep_new_page.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault 1.03 ± 6% -0.2 0.79 ± 10% perf-profile.calltrace.cycles-pp.exit_to_usermode_loop.do_syscall_64.entry_SYSCALL_64_after_hwframe.kill 1.00 ± 10% -0.2 0.77 ± 9% perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 0.87 ± 10% -0.2 0.66 ± 15% perf-profile.calltrace.cycles-pp.shmem_file_read_iter.new_sync_read.vfs_read.ksys_read.do_syscall_64 1.41 ± 3% -0.2 1.23 ± 7% perf-profile.calltrace.cycles-pp.get_page_from_freelist.__alloc_pages_nodemask.alloc_pages_vma.handle_pte_fault.__handle_mm_fault 1.88 ± 5% -0.1 1.73 perf-profile.calltrace.cycles-pp.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 1.87 ± 5% -0.1 1.73 perf-profile.calltrace.cycles-pp._do_fork.__x64_sys_clone.do_syscall_64.entry_SYSCALL_64_after_hwframe 10.34 ± 11% +7.3 17.66 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 10.18 ± 11% +7.3 17.52 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 11.32 ± 9% +7.7 19.03 ± 8% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 11.32 ± 9% +7.7 19.05 ± 8% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 11.32 ± 9% +7.7 19.05 ± 8% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 11.02 ± 5% +8.1 19.14 ± 7% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 16.04 ± 6% +9.1 25.17 ± 8% perf-profile.calltrace.cycles-pp.secondary_startup_64 55.98 -7.0 48.94 ± 4% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 55.67 -7.0 48.67 ± 4% perf-profile.children.cycles-pp.do_syscall_64 10.60 ± 16% -3.3 7.30 ± 8% perf-profile.children.cycles-pp.vprintk_emit 13.02 ± 7% -3.0 9.99 ± 7% perf-profile.children.cycles-pp.write 9.92 ± 13% -2.8 7.08 ± 4% perf-profile.children.cycles-pp.console_unlock 10.26 ± 8% -2.7 7.53 ± 6% perf-profile.children.cycles-pp.new_sync_write 10.79 ± 8% -2.6 8.18 ± 8% perf-profile.children.cycles-pp.vfs_write 10.95 ± 8% -2.6 8.36 ± 8% perf-profile.children.cycles-pp.ksys_write 7.17 ± 16% -2.5 4.69 ± 7% perf-profile.children.cycles-pp.devkmsg_write 7.17 ± 16% -2.5 4.69 ± 7% perf-profile.children.cycles-pp.devkmsg_emit 8.65 ± 16% -2.4 6.21 ± 4% perf-profile.children.cycles-pp.serial8250_console_write 8.53 ± 17% -2.4 6.11 ± 4% perf-profile.children.cycles-pp.uart_console_write 7.13 ± 17% -2.4 4.71 ± 6% perf-profile.children.cycles-pp._fini 8.46 ± 16% -2.4 6.07 ± 4% perf-profile.children.cycles-pp.wait_for_xmitr 8.34 ± 16% -2.4 5.97 ± 4% perf-profile.children.cycles-pp.serial8250_console_putchar 5.80 ± 16% -1.6 4.21 ± 6% perf-profile.children.cycles-pp.io_serial_in 7.85 ± 10% -1.2 6.67 ± 5% perf-profile.children.cycles-pp.execve 7.72 ± 11% -1.2 6.55 ± 5% perf-profile.children.cycles-pp.__do_execve_file 5.19 ± 13% -1.1 4.05 ± 8% perf-profile.children.cycles-pp.mmput 5.16 ± 13% -1.1 4.03 ± 8% perf-profile.children.cycles-pp.exit_mmap 7.76 ± 10% -1.1 6.64 ± 5% perf-profile.children.cycles-pp.__x64_sys_execve 7.11 ± 5% -1.1 6.01 ± 10% perf-profile.children.cycles-pp.brk 3.92 ± 6% -0.9 3.03 ± 5% perf-profile.children.cycles-pp.kill 2.63 ± 17% -0.8 1.85 perf-profile.children.cycles-pp.delay_tsc 4.89 ± 8% -0.8 4.12 ± 8% perf-profile.children.cycles-pp.__x64_sys_brk 2.48 ± 27% -0.7 1.74 ± 4% perf-profile.children.cycles-pp.flush_old_exec 3.02 ± 12% -0.7 2.28 ± 12% perf-profile.children.cycles-pp.unmap_page_range 3.15 ± 11% -0.7 2.40 ± 12% perf-profile.children.cycles-pp.unmap_vmas 2.25 ± 19% -0.5 1.75 ± 11% perf-profile.children.cycles-pp.unmap_region 1.27 ± 11% -0.4 0.86 ± 8% perf-profile.children.cycles-pp.vt_console_print 1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.lf 1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.con_scroll 1.24 ± 12% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.fbcon_scroll 1.79 ± 9% -0.4 1.41 ± 4% perf-profile.children.cycles-pp.release_pages 1.22 ± 11% -0.4 0.85 ± 9% perf-profile.children.cycles-pp.fbcon_redraw 1.17 ± 12% -0.4 0.82 ± 10% perf-profile.children.cycles-pp.fbcon_putcs 1.16 ± 13% -0.3 0.82 ± 10% perf-profile.children.cycles-pp.bit_putcs 0.90 ± 16% -0.3 0.62 ± 12% perf-profile.children.cycles-pp.kill_pid_info 0.95 ± 10% -0.3 0.68 ± 6% perf-profile.children.cycles-pp.drm_fb_helper_cfb_imageblit 0.95 ± 11% -0.3 0.68 ± 6% perf-profile.children.cycles-pp.cfb_imageblit 1.24 ± 7% -0.2 1.01 ± 6% perf-profile.children.cycles-pp.new_sync_read 0.71 ± 4% -0.2 0.49 ± 23% perf-profile.children.cycles-pp.___perf_sw_event 0.55 ± 31% -0.2 0.33 ± 16% perf-profile.children.cycles-pp.unlink_anon_vmas 0.89 ± 11% -0.2 0.67 ± 15% perf-profile.children.cycles-pp.shmem_file_read_iter 0.60 ± 20% -0.2 0.39 ± 20% perf-profile.children.cycles-pp.__send_signal 1.06 ± 6% -0.2 0.85 ± 16% perf-profile.children.cycles-pp.pagevec_lru_move_fn 0.88 -0.2 0.68 ± 6% perf-profile.children.cycles-pp.__perf_sw_event 1.49 ± 5% -0.2 1.29 ± 7% perf-profile.children.cycles-pp.prepare_exit_to_usermode 0.56 ± 12% -0.2 0.37 ± 11% perf-profile.children.cycles-pp.do_send_sig_info 1.65 ± 8% -0.2 1.47 ± 4% perf-profile.children.cycles-pp.perf_event_mmap 0.69 ± 2% -0.2 0.52 ± 16% perf-profile.children.cycles-pp.page_remove_rmap 0.61 ± 5% -0.2 0.44 ± 15% perf-profile.children.cycles-pp.free_unref_page_list 0.60 ± 6% -0.2 0.43 ± 15% perf-profile.children.cycles-pp.__vm_munmap 0.77 ± 12% -0.2 0.62 ± 12% perf-profile.children.cycles-pp.__might_sleep 0.39 ± 12% -0.2 0.24 ± 18% perf-profile.children.cycles-pp.time 0.46 ± 14% -0.1 0.34 ± 14% perf-profile.children.cycles-pp.lru_add_drain_cpu 0.57 ± 8% -0.1 0.47 ± 14% perf-profile.children.cycles-pp.shmem_undo_range 0.41 ± 12% -0.1 0.30 ± 15% perf-profile.children.cycles-pp.copy_fpstate_to_sigframe 0.76 ± 7% -0.1 0.67 ± 8% perf-profile.children.cycles-pp.__x64_sys_rt_sigreturn 0.26 ± 16% -0.1 0.17 ± 17% perf-profile.children.cycles-pp.mark_page_accessed 0.12 ± 47% -0.1 0.04 ±103% perf-profile.children.cycles-pp.sigaction 0.23 ± 12% -0.1 0.15 ± 11% perf-profile.children.cycles-pp.__vm_enough_memory 0.12 ± 18% -0.1 0.05 ±106% perf-profile.children.cycles-pp.__vsprintf_chk 0.23 ± 20% -0.1 0.17 ± 13% perf-profile.children.cycles-pp.d_add 0.13 ± 23% -0.1 0.07 ± 58% perf-profile.children.cycles-pp.fput_many 0.13 ± 14% -0.1 0.08 ± 24% perf-profile.children.cycles-pp.vfs_unlink 0.11 ± 20% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.__update_load_avg_cfs_rq 0.04 ± 63% +0.0 0.08 ± 23% perf-profile.children.cycles-pp.uncharge_page 0.06 ± 22% +0.0 0.10 ± 36% perf-profile.children.cycles-pp.sched_exec 0.44 ± 4% +0.0 0.48 ± 4% perf-profile.children.cycles-pp.close 0.14 ± 22% +0.1 0.21 ± 17% perf-profile.children.cycles-pp.pick_next_task_fair 0.10 ± 17% +0.1 0.17 ± 23% perf-profile.children.cycles-pp.__anon_vma_prepare 0.00 +0.1 0.07 ± 24% perf-profile.children.cycles-pp.update_sd_lb_stats 0.07 ± 34% +0.1 0.15 ± 42% perf-profile.children.cycles-pp.file_free_rcu 0.15 ± 27% +0.1 0.23 ± 21% perf-profile.children.cycles-pp.__strcasecmp 0.20 ± 21% +0.1 0.29 ± 8% perf-profile.children.cycles-pp.__pte_alloc 0.14 ± 47% +0.1 0.23 ± 27% perf-profile.children.cycles-pp.update_blocked_averages 0.09 ± 44% +0.1 0.19 ± 18% perf-profile.children.cycles-pp.schedule_idle 0.00 +0.1 0.10 ± 33% perf-profile.children.cycles-pp.newidle_balance 0.00 +0.1 0.10 ± 18% perf-profile.children.cycles-pp.__vmalloc_node_range 0.21 ± 15% +0.1 0.32 ± 25% perf-profile.children.cycles-pp.__wake_up_common 0.63 ± 8% +0.1 0.77 ± 6% perf-profile.children.cycles-pp.rcu_do_batch 0.76 ± 14% +0.1 0.90 ± 9% perf-profile.children.cycles-pp.rcu_core 0.07 ± 90% +0.2 0.27 ±109% perf-profile.children.cycles-pp.security_mmap_addr 0.46 ± 26% +0.3 0.75 ± 13% perf-profile.children.cycles-pp.__sched_text_start 11.32 ± 9% +7.7 19.05 ± 8% perf-profile.children.cycles-pp.start_secondary 11.03 ± 5% +8.1 19.16 ± 7% perf-profile.children.cycles-pp.intel_idle 14.78 ± 6% +8.5 23.24 ± 8% perf-profile.children.cycles-pp.cpuidle_enter 14.76 ± 6% +8.5 23.24 ± 8% perf-profile.children.cycles-pp.cpuidle_enter_state 16.04 ± 6% +9.1 25.17 ± 8% perf-profile.children.cycles-pp.secondary_startup_64 16.04 ± 6% +9.1 25.17 ± 8% perf-profile.children.cycles-pp.cpu_startup_entry 16.04 ± 6% +9.1 25.19 ± 8% perf-profile.children.cycles-pp.do_idle 5.79 ± 16% -1.6 4.21 ± 6% perf-profile.self.cycles-pp.io_serial_in 2.62 ± 17% -0.8 1.85 perf-profile.self.cycles-pp.delay_tsc 5.11 ± 4% -0.6 4.56 ± 5% perf-profile.self.cycles-pp.do_syscall_64 1.44 ± 6% -0.3 1.15 ± 5% perf-profile.self.cycles-pp.unmap_page_range 0.94 ± 11% -0.3 0.68 ± 6% perf-profile.self.cycles-pp.cfb_imageblit 0.65 ± 6% -0.2 0.42 ± 23% perf-profile.self.cycles-pp.___perf_sw_event 1.42 ± 5% -0.2 1.22 ± 7% perf-profile.self.cycles-pp.prepare_exit_to_usermode 0.65 ± 13% -0.2 0.47 ± 9% perf-profile.self.cycles-pp.do_page_fault 0.65 ± 9% -0.1 0.52 ± 5% perf-profile.self.cycles-pp.release_pages 0.24 ± 20% -0.1 0.15 ± 16% perf-profile.self.cycles-pp.mark_page_accessed 0.16 ± 28% -0.1 0.08 ± 69% perf-profile.self.cycles-pp.free_unref_page_commit 0.12 ± 24% -0.1 0.04 ± 59% perf-profile.self.cycles-pp.__do_munmap 0.10 ± 24% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.__update_load_avg_cfs_rq 0.04 ± 57% +0.0 0.07 ± 19% perf-profile.self.cycles-pp.__sbrk 0.04 ± 57% +0.0 0.08 ± 23% perf-profile.self.cycles-pp.update_load_avg 0.04 ± 57% +0.0 0.08 ± 23% perf-profile.self.cycles-pp.uncharge_page 0.26 ± 11% +0.1 0.39 ± 12% perf-profile.self.cycles-pp.copy_page 0.49 ± 13% +0.1 0.63 ± 13% perf-profile.self.cycles-pp.get_page_from_freelist 11.00 ± 5% +8.1 19.14 ± 7% perf-profile.self.cycles-pp.intel_idle reaim.time.system_time 90 +----------------------------------------------------------------------+ | +. .+ ++.++ .+ | 88 |+.+ +.++. + +++ ++.++.+ +.++.+++.+++.+++. +.++ .+ + :.+++.+| | + + + + ++.++ + | 86 |-+ | | | 84 |-+ | | | 82 |-+ O | |O OO OO O | 80 |-+ | | OO OOO O O O O | 78 |-+ O O O O O OO O | | O OOO O | 76 +----------------------------------------------------------------------+ reaim.time.percent_of_cpu_this_job_got 55 +--------------------------------------------------------------------+ | | 54.5 |-+ | 54 |-+ | | | 53.5 |-+ | | | 53 |-+OOO OOO | | | 52.5 |-+ | 52 |O+ OOO OOO OOO OOO OO | | | 51.5 |-+ | | | 51 +--------------------------------------------------------------------+ reaim.parent_time 0.405 +-------------------------------------------------------------------+ 0.4 |-+ O | | O O OOO OO | 0.395 |-+ | 0.39 |-+ | 0.385 |-+ OOOO OOO OOO OOO OOOO | 0.38 |O+OOO | | | 0.375 |-+ | 0.37 |-+ | 0.365 |-+ | 0.36 |-+ | | +. .+ +. .+ +. + +. +++. + .+| 0.355 |+. +.+ ++.++ +++. ++.++++ + +++ + +++ + ++.++ + +. : + + | 0.35 +-------------------------------------------------------------------+ reaim.child_systime 0.69 +--------------------------------------------------------------------+ | +.+ ++. ++.+ .++ ++ +++.+ .++ .+++. + .+ : + .+| 0.68 |+.++ ++.+ + ++ +.+ ++ + + +.+++ +.+ + | 0.67 |-+ | | | 0.66 |-+ | 0.65 |-+ | | O | 0.64 |O+O OOO | 0.63 |-+ O | | O O O O O | 0.62 |-+ OO O O O O O | 0.61 |-+ O OO O OO OOOO | | | 0.6 +--------------------------------------------------------------------+ reaim.jobs_per_min 69000 +-------------------------------------------------------------------+ 68000 |-.++ + + .++ .+ .++ + .+++.+ | |+ +.+ ++.+++.+ + + +++.+++.+++.+++.++++ +.+ + + +.+ +. | 67000 |-+ + + +| 66000 |-+ | | | 65000 |-+ | 64000 |-+ | 63000 |-+ | |O OOO O O O O | 62000 |-+ O O OOO O OOO OOOO | 61000 |-+ | | O OOO OO | 60000 |-+ OO | 59000 +-------------------------------------------------------------------+ reaim.jobs_per_min_child 17500 +-------------------------------------------------------------------+ | | 17000 |-.+++. + + .+++.+ .++ + .+++.+ | |+ + ++.+++.+ + +++.+++.+++.+++.++++ +.+ + + +.+++.+| | + | 16500 |-+ | | | 16000 |-+ | | | 15500 |O+OOO OOOO OOO OOO OOO OOOO | | | | O O OO | 15000 |-+ OOO O | | | 14500 +-------------------------------------------------------------------+ reaim.max_jobs_per_min 76000 +-------------------------------------------------------------------+ | | 74000 |-+ + + + | | :: : :: | | : : : :: : | 72000 |+.++ ++++.+++.+++.+++.++++.+++.+++.+++.++++.+++.+++.+ + +++.+++.+| | | 70000 |-+ | | | 68000 |-+ O | | | | | 66000 |O+O O OOOO OOO OOO OOO OOOO | | | 64000 +-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

7 months, 1 week

3
4
0 / 0

[drm/i915/gen9] 53b9bd37af: phoronix-test-suite.clpeak.KernelLatency.us 22.5% regression

by kernel test robot

Greeting, FYI, we noticed a 22.5% regression of phoronix-test-suite.clpeak.KernelLatency.us due to commit: commit: 53b9bd37af59d1def99b20707536105857eb9bd0 ("drm/i915/gen9: Clear residual context state on context switch") https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable.git linux-5.4.y in testcase: phoronix-test-suite on test machine: 4 threads Intel(R) Core(TM) i7-7567U CPU @ 3.50GHz with 32G memory with following parameters: test: clpeak-1.0.1 cpufreq_governor: performance ucode: 0xca test-description: The Phoronix Test Suite is the most comprehensive testing and benchmarking platform available that provides an extensible framework for which new tests can be easily added. test-url: http://www.phoronix-test-suite.com/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-lck-7931/clear-x86_64-phoronix-30140/lkp-kbl-nuc1/clpeak-1.0.1/phoronix-test-suite/0xca commit: f58642c1bc ("netfilter: ipset: avoid null deref when IPSET_ATTR_LINENO is present") 53b9bd37af ("drm/i915/gen9: Clear residual context state on context switch") f58642c1bc7946c3 53b9bd37af59d1def99b2070753 ---------------- --------------------------- %stddev %change %stddev \ | \ 24.49 ± 2% +22.5% 30.00 phoronix-test-suite.clpeak.KernelLatency.us 101.56 ± 38% -70.7% 29.74 phoronix-test-suite.time.elapsed_time 101.56 ± 38% -70.7% 29.74 phoronix-test-suite.time.elapsed_time.max 405511 ± 30% -54.3% 185150 phoronix-test-suite.time.minor_page_faults 53.33 ± 2% +4.4% 55.67 phoronix-test-suite.time.percent_of_cpu_this_job_got 47.70 ± 37% -70.2% 14.20 phoronix-test-suite.time.user_time 454369 ± 41% -75.5% 111123 phoronix-test-suite.time.voluntary_context_switches 11.39 ± 2% -6.7% 10.62 ± 6% boot-time.dhcp 591003 ± 3% -11.4% 523381 ± 2% meminfo.Active(file) 53200 ± 50% +159.9% 138245 meminfo.max_used_kB 61.00 ± 19% -38.3% 37.67 ± 2% vmstat.cpu.id 28.00 ± 32% +63.1% 45.67 vmstat.cpu.us 3583 ± 45% +106.2% 7390 ± 5% vmstat.io.bi 1.33 ± 35% +125.0% 3.00 vmstat.procs.r 60.90 ± 20% -24.2 36.70 ± 3% mpstat.cpu.all.idle% 0.89 ± 19% +0.6 1.46 ± 14% mpstat.cpu.all.iowait% 0.20 ± 23% +0.1 0.28 ± 2% mpstat.cpu.all.soft% 8.03 ± 35% +4.7 12.69 ± 2% mpstat.cpu.all.sys% 28.78 ± 32% +18.9 47.66 mpstat.cpu.all.usr% 13077984 ± 54% -54.8% 5908886 ± 43% cpuidle.C1.time 597419 ± 46% -74.3% 153496 ± 15% cpuidle.C1.usage 24140686 ± 20% -72.9% 6543020 ± 20% cpuidle.C1E.time 356666 ± 25% -71.4% 102084 ± 12% cpuidle.C1E.usage 1.55e+08 ± 53% -86.5% 20926680 ± 24% cpuidle.C6.time 262494 ± 49% -76.9% 60562 ± 25% cpuidle.C6.usage 3614603 ± 39% -73.6% 952459 ± 2% cpuidle.POLL.time 409623 ± 42% -74.9% 102932 cpuidle.POLL.usage 79.96 ± 16% +23.4% 98.63 turbostat.Any%C0 1499 ± 30% +57.6% 2363 turbostat.Avg_MHz 38.92 ± 31% +23.0 61.94 turbostat.Busy% 597382 ± 46% -74.3% 153485 ± 15% turbostat.C1 356611 ± 25% -71.4% 102035 ± 12% turbostat.C1E 262458 ± 49% -76.9% 60550 ± 25% turbostat.C6 34.42 ± 27% -17.6 16.77 ± 24% turbostat.C6% 18.75 ± 52% -84.2% 2.96 ± 52% turbostat.CPU%c6 14.41 ± 21% +37.9% 19.87 turbostat.CorWatt 1777243 ± 32% -66.4% 597719 turbostat.IRQ 21.36 ± 14% +27.9% 27.32 turbostat.PkgWatt 123.28 ± 25% +44.0% 177.47 turbostat.Totl%C0 38364 ± 4% -23.5% 29333 ± 7% softirqs.CPU0.RCU 16353 ± 31% -60.7% 6429 ± 6% softirqs.CPU0.SCHED 55438 ± 25% -55.7% 24533 ± 4% softirqs.CPU0.TIMER 38652 ± 7% -25.8% 28672 ± 3% softirqs.CPU1.RCU 17071 ± 29% -58.5% 7091 ± 6% softirqs.CPU1.SCHED 63131 ± 33% -58.6% 26110 ± 7% softirqs.CPU1.TIMER 233404 ± 40% -74.3% 60064 softirqs.CPU2.HI 36812 ± 5% -24.4% 27818 ± 3% softirqs.CPU2.RCU 17331 ± 30% -65.2% 6025 ± 7% softirqs.CPU2.SCHED 52817 ± 26% -51.1% 25847 softirqs.CPU2.TIMER 37680 ± 4% -27.2% 27418 ± 5% softirqs.CPU3.RCU 17232 ± 33% -57.0% 7413 ± 2% softirqs.CPU3.SCHED 54880 ± 31% -48.7% 28163 ± 7% softirqs.CPU3.TIMER 233405 ± 40% -74.3% 60065 softirqs.HI 151510 ± 5% -25.3% 113242 ± 2% softirqs.RCU 67989 ± 31% -60.3% 26959 softirqs.SCHED 226267 ± 28% -53.7% 104655 softirqs.TIMER 147753 ± 3% -11.4% 130854 ± 2% proc-vmstat.nr_active_file 386270 +4.2% 402302 proc-vmstat.nr_dirty_background_threshold 3804020 -2.4% 3713749 proc-vmstat.nr_dirty_threshold 803734 -3.1% 778927 proc-vmstat.nr_file_pages 452742 -1.6% 445477 proc-vmstat.nr_inactive_file 10099 +2.1% 10308 proc-vmstat.nr_kernel_stack 2628 +1.8% 2675 proc-vmstat.nr_page_table_pages 82010 -5.4% 77547 proc-vmstat.nr_slab_reclaimable 24816 -3.2% 24014 proc-vmstat.nr_slab_unreclaimable 780.67 -2.3% 763.00 proc-vmstat.nr_unevictable 147753 ± 3% -11.4% 130854 ± 2% proc-vmstat.nr_zone_active_file 452742 -1.6% 445477 proc-vmstat.nr_zone_inactive_file 780.67 -2.3% 763.00 proc-vmstat.nr_zone_unevictable 5659020 ± 6% -46.0% 3058187 proc-vmstat.numa_hit 5659020 ± 6% -46.0% 3058187 proc-vmstat.numa_local 47921 ± 3% -16.2% 40136 ± 2% proc-vmstat.pgactivate 5867768 ± 7% -44.8% 3238245 proc-vmstat.pgalloc_normal 8000575 ± 5% -44.9% 4409621 proc-vmstat.pgfault 5730760 ± 7% -45.9% 3102373 proc-vmstat.pgfree 285530 ± 4% -23.9% 217222 ± 4% proc-vmstat.pgpgin 49.48 ± 70% -49.5 0.00 perf-profile.calltrace.cycles-pp.secondary_startup_64 40.07 ± 70% -40.1 0.00 perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 36.60 ± 79% -36.6 0.00 perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 36.59 ± 79% -36.6 0.00 perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 36.51 ± 79% -36.5 0.00 perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 35.12 ± 79% -35.1 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 34.92 ± 79% -34.9 0.00 perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 6.65 ± 72% -6.6 0.00 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe 6.47 ± 72% -6.5 0.00 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.00 ± 72% -4.0 0.00 perf-profile.calltrace.cycles-pp.poll_idle.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 49.48 ± 70% -49.5 0.00 perf-profile.children.cycles-pp.secondary_startup_64 49.48 ± 70% -49.5 0.00 perf-profile.children.cycles-pp.cpu_startup_entry 49.37 ± 70% -49.4 0.00 perf-profile.children.cycles-pp.do_idle 47.50 ± 70% -47.5 0.00 perf-profile.children.cycles-pp.cpuidle_enter 47.38 ± 70% -47.4 0.00 perf-profile.children.cycles-pp.cpuidle_enter_state 40.10 ± 70% -40.1 0.00 perf-profile.children.cycles-pp.intel_idle 36.60 ± 79% -36.6 0.00 perf-profile.children.cycles-pp.start_secondary 7.29 ± 72% -7.3 0.00 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 7.12 ± 71% -7.1 0.00 perf-profile.children.cycles-pp.do_syscall_64 4.03 ± 72% -4.0 0.00 perf-profile.children.cycles-pp.poll_idle 40.10 ± 70% -40.1 0.00 perf-profile.self.cycles-pp.intel_idle 3.76 ± 71% -3.8 0.00 perf-profile.self.cycles-pp.poll_idle 695942 ± 40% -74.6% 176670 interrupts.125:IR-PCI-MSI.32768-edge.i915 34054 ± 4% -17.3% 28167 ± 9% interrupts.127:IR-PCI-MSI.376832-edge.ahci[0000:00:17.0] 1050 ± 28% -49.8% 527.33 ± 39% interrupts.128:IR-PCI-MSI.520192-edge.eth0 222197 ± 4% -41.3% 130456 interrupts.CAL:Function_call_interrupts 55148 -39.6% 33319 ± 2% interrupts.CPU0.CAL:Function_call_interrupts 194140 ± 36% -68.0% 62087 interrupts.CPU0.LOC:Local_timer_interrupts 1863 ± 88% -97.8% 41.33 ± 89% interrupts.CPU0.NMI:Non-maskable_interrupts 1863 ± 88% -97.8% 41.33 ± 89% interrupts.CPU0.PMI:Performance_monitoring_interrupts 6030 ± 17% -32.2% 4089 ± 7% interrupts.CPU0.RES:Rescheduling_interrupts 5253 ± 37% -73.9% 1369 ± 23% interrupts.CPU0.TLB:TLB_shootdowns 34054 ± 4% -17.3% 28167 ± 9% interrupts.CPU1.127:IR-PCI-MSI.376832-edge.ahci[0000:00:17.0] 56244 ± 8% -42.8% 32144 ± 2% interrupts.CPU1.CAL:Function_call_interrupts 194206 ± 36% -68.0% 62051 interrupts.CPU1.LOC:Local_timer_interrupts 1228 ± 67% -96.7% 41.00 ± 82% interrupts.CPU1.NMI:Non-maskable_interrupts 1228 ± 67% -96.7% 41.00 ± 82% interrupts.CPU1.PMI:Performance_monitoring_interrupts 4247 ± 13% -41.7% 2477 ± 5% interrupts.CPU1.RES:Rescheduling_interrupts 2396 ± 58% -76.5% 562.33 ± 30% interrupts.CPU1.TLB:TLB_shootdowns 695942 ± 40% -74.6% 176670 interrupts.CPU2.125:IR-PCI-MSI.32768-edge.i915 1050 ± 28% -49.8% 527.33 ± 39% interrupts.CPU2.128:IR-PCI-MSI.520192-edge.eth0 54618 -37.7% 34042 ± 4% interrupts.CPU2.CAL:Function_call_interrupts 194173 ± 36% -68.0% 62105 interrupts.CPU2.LOC:Local_timer_interrupts 1547 ± 73% -97.3% 41.00 ± 77% interrupts.CPU2.NMI:Non-maskable_interrupts 1547 ± 73% -97.3% 41.00 ± 77% interrupts.CPU2.PMI:Performance_monitoring_interrupts 9129 ± 17% -48.5% 4701 ± 6% interrupts.CPU2.RES:Rescheduling_interrupts 1957 ± 60% -91.0% 175.33 ± 37% interrupts.CPU2.TLB:TLB_shootdowns 56187 ± 5% -44.9% 30951 ± 6% interrupts.CPU3.CAL:Function_call_interrupts 194173 ± 36% -68.0% 62079 interrupts.CPU3.LOC:Local_timer_interrupts 947.00 ± 67% -95.8% 39.33 ± 85% interrupts.CPU3.NMI:Non-maskable_interrupts 947.00 ± 67% -95.8% 39.33 ± 85% interrupts.CPU3.PMI:Performance_monitoring_interrupts 5359 ± 32% -93.4% 353.67 ± 46% interrupts.CPU3.TLB:TLB_shootdowns 776694 ± 36% -68.0% 248323 interrupts.LOC:Local_timer_interrupts 5586 ± 72% -97.1% 162.67 ± 83% interrupts.NMI:Non-maskable_interrupts 5586 ± 72% -97.1% 162.67 ± 83% interrupts.PMI:Performance_monitoring_interrupts 24885 ± 7% -36.2% 15873 ± 7% interrupts.RES:Rescheduling_interrupts 14966 ± 11% -83.6% 2461 ± 9% interrupts.TLB:TLB_shootdowns 2014 ± 46% +58.2% 3187 ± 34% sched_debug.cfs_rq:/.load_avg.max 33405 ± 40% -58.6% 13826 ± 9% sched_debug.cfs_rq:/.min_vruntime.avg 37077 ± 38% -51.2% 18080 ± 26% sched_debug.cfs_rq:/.min_vruntime.max 29779 ± 42% -60.7% 11702 ± 6% sched_debug.cfs_rq:/.min_vruntime.min 1489 ± 10% +38.2% 2058 ± 14% sched_debug.cfs_rq:/.util_avg.max 94180 ± 5% -69.7% 28544 ± 69% sched_debug.cpu.avg_idle.min 62749 ± 44% -65.5% 21617 ± 5% sched_debug.cpu.clock.avg 62750 ± 44% -65.5% 21623 ± 5% sched_debug.cpu.clock.max 62748 ± 44% -65.6% 21612 ± 5% sched_debug.cpu.clock.min 0.72 ± 20% +490.0% 4.26 ± 45% sched_debug.cpu.clock.stddev 61991 ± 44% -65.5% 21373 ± 5% sched_debug.cpu.clock_task.avg 62297 ± 44% -65.4% 21536 ± 5% sched_debug.cpu.clock_task.max 61557 ± 44% -65.7% 21140 ± 5% sched_debug.cpu.clock_task.min 297.83 ± 30% -43.4% 168.59 sched_debug.cpu.clock_task.stddev 2455 ± 33% +43.8% 3529 ± 10% sched_debug.cpu.curr->pid.avg 0.00 ± 31% -51.7% 0.00 ± 38% sched_debug.cpu.next_balance.stddev 203929 ± 46% -67.6% 66070 ± 7% sched_debug.cpu.nr_switches.avg 258658 ± 46% -68.4% 81856 ± 4% sched_debug.cpu.nr_switches.max 155789 ± 46% -67.3% 51018 ± 8% sched_debug.cpu.nr_switches.min 0.17 ±108% +250.0% 0.58 ± 53% sched_debug.cpu.nr_uninterruptible.avg 60.22 ± 56% -70.7% 17.67 ± 49% sched_debug.cpu.nr_uninterruptible.max -78.78 -83.5% -13.00 sched_debug.cpu.nr_uninterruptible.min 54.22 ± 53% -77.4% 12.27 ± 41% sched_debug.cpu.nr_uninterruptible.stddev 1225073 ± 70% -99.9% 749.58 ± 36% sched_debug.cpu.sched_count.avg 1935922 ± 72% -99.9% 1573 ± 36% sched_debug.cpu.sched_count.max 519540 ± 79% -99.9% 535.76 ± 41% sched_debug.cpu.sched_count.stddev 68013 ± 70% -99.4% 428.67 ± 39% sched_debug.cpu.ttwu_count.avg 117399 ± 70% -99.3% 848.33 ± 51% sched_debug.cpu.ttwu_count.max 30185 ± 70% -99.0% 305.10 ± 62% sched_debug.cpu.ttwu_count.stddev 18430 ± 69% -98.5% 277.08 ± 43% sched_debug.cpu.ttwu_local.avg 41983 ± 69% -98.7% 534.33 ± 48% sched_debug.cpu.ttwu_local.max 13660 ± 68% -98.6% 186.19 ± 63% sched_debug.cpu.ttwu_local.stddev 62747 ± 44% -65.6% 21609 ± 5% sched_debug.cpu_clk 62538 ± 44% -65.7% 21422 ± 5% sched_debug.ktime 0.02 ±141% +249.5% 0.06 ± 22% sched_debug.rt_rq:/.rt_time.avg 62766 ± 44% -65.6% 21619 ± 5% sched_debug.sched_clk 53.07 ± 25% -37.9% 32.94 perf-stat.i.MPKI 1.429e+09 ± 27% +52.1% 2.173e+09 perf-stat.i.branch-instructions 25378470 ± 48% +113.2% 54106012 perf-stat.i.branch-misses 41939756 ± 26% +38.8% 58213548 perf-stat.i.cache-misses 1.916e+08 ± 25% +47.4% 2.823e+08 ± 2% perf-stat.i.cache-references 13953 ± 10% +17.0% 16320 perf-stat.i.context-switches 1.39 ± 16% -22.4% 1.08 perf-stat.i.cpi 5.98e+09 ± 31% +57.8% 9.436e+09 perf-stat.i.cpu-cycles 253.07 ± 45% +98.8% 503.12 ± 6% perf-stat.i.cpu-migrations 0.22 ± 38% -0.1 0.10 perf-stat.i.dTLB-load-miss-rate% 1478975 ± 21% +34.7% 1991993 perf-stat.i.dTLB-load-misses 1.516e+09 ± 24% +42.7% 2.163e+09 perf-stat.i.dTLB-loads 0.14 ± 11% -0.0 0.11 perf-stat.i.dTLB-store-miss-rate% 565610 ± 24% +34.5% 760493 perf-stat.i.dTLB-store-misses 6.417e+08 ± 17% +31.0% 8.404e+08 perf-stat.i.dTLB-stores 611337 ± 19% +29.3% 790163 ± 2% perf-stat.i.iTLB-load-misses 5.981e+09 ± 26% +49.3% 8.931e+09 perf-stat.i.instructions 1.82 ± 51% +157.8% 4.70 perf-stat.i.major-faults 1.68 ± 19% +0.8 2.49 perf-stat.overall.branch-miss-rate% 21.83 -1.2 20.63 ± 2% perf-stat.overall.cache-miss-rate% 0.99 ± 4% +6.9% 1.06 perf-stat.overall.cpi 141.04 ± 4% +14.9% 162.09 perf-stat.overall.cycles-between-cache-misses 0.10 ± 3% -0.0 0.09 perf-stat.overall.dTLB-load-miss-rate% 14.55 ± 6% +1.2 15.77 ± 2% perf-stat.overall.iTLB-load-miss-rate% 9694 ± 9% +16.7% 11311 ± 2% perf-stat.overall.instructions-per-iTLB-miss 1.01 ± 4% -6.6% 0.95 perf-stat.overall.ipc 1.413e+09 ± 26% +50.9% 2.132e+09 perf-stat.ps.branch-instructions 25047514 ± 47% +112.0% 53091697 perf-stat.ps.branch-misses 41461384 ± 25% +37.8% 57118869 perf-stat.ps.cache-misses 1.894e+08 ± 24% +46.3% 2.77e+08 ± 2% perf-stat.ps.cache-references 13813 ± 9% +15.9% 16012 perf-stat.ps.context-switches 5.91e+09 ± 30% +56.7% 9.258e+09 perf-stat.ps.cpu-cycles 249.80 ± 45% +97.7% 493.74 ± 6% perf-stat.ps.cpu-migrations 1462753 ± 20% +33.6% 1954515 perf-stat.ps.dTLB-load-misses 1.499e+09 ± 23% +41.6% 2.122e+09 perf-stat.ps.dTLB-loads 6.349e+08 ± 16% +29.9% 8.246e+08 perf-stat.ps.dTLB-stores 3524622 ± 12% +17.5% 4140201 perf-stat.ps.iTLB-loads 5.913e+09 ± 25% +48.2% 8.763e+09 perf-stat.ps.instructions 1.80 ± 50% +156.4% 4.61 perf-stat.ps.major-faults 5.49e+11 ± 21% -50.6% 2.71e+11 perf-stat.total.instructions phoronix-test-suite.clpeak.KernelLatency.us 35 +----------------------------------------------------------------------+ | | 30 |-+ O O O O | | O O | 25 |-+ +...+ +...+ + + + + | | : : : : : : : + : | 20 |:+ : : : : : : : : : : : : : :| |: : : : : : : : : : : : : : : :| 15 |-: : : : : : : : : : : : : : : : | | : : : : : : : : : : : : : : : : | 10 |-: : : : : : : : : : : : : : : : | | : : : : : : : : : : : : : : : : | 5 |-+: : : : : : : : : : : : : : : : | | : : : : : : : : | 0 +----------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

8 months, 4 weeks

3
3
1 / 0

b276527539 ("staging: most: move core files out of the staging .."): [ 12.247349] BUG: kernel NULL pointer dereference, address: 00000000

by kernel test robot

Greetings, 0day kernel testing robot got the below dmesg and the first bad commit is https://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git staging-next commit b276527539188f1f61c082ebef27803db93e536d Author: Christian Gromm <christian.gromm(a)microchip.com> AuthorDate: Tue Mar 10 14:02:40 2020 +0100 Commit: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> CommitDate: Tue Mar 24 13:42:44 2020 +0100 staging: most: move core files out of the staging area This patch moves the core module to the /drivers/most directory and makes all necessary changes in order to not break the build. Signed-off-by: Christian Gromm <christian.gromm(a)microchip.com> Link: https://lore.kernel.org/r/1583845362-26707-2-git-send-email-christian.gro... Signed-off-by: Greg Kroah-Hartman <gregkh(a)linuxfoundation.org> 22dd4acc80 Staging: speakup: Add identifier name to function declaration arguments. b276527539 staging: most: move core files out of the staging area e681bb287f staging: vt6656: Use DIV_ROUND_UP macro instead of specific code +-------------------------------------------------------+------------+------------+------------+ | | 22dd4acc80 | b276527539 | e681bb287f | +-------------------------------------------------------+------------+------------+------------+ | boot_successes | 26 | 0 | 0 | | boot_failures | 8 | 11 | 11 | | WARNING:possible_circular_locking_dependency_detected | 8 | | | | BUG:kernel_NULL_pointer_dereference,address | 0 | 11 | 11 | | Oops:#[##] | 0 | 11 | 11 | | EIP:__list_add_valid | 0 | 11 | 11 | | Kernel_panic-not_syncing:Fatal_exception | 0 | 11 | 11 | +-------------------------------------------------------+------------+------------+------------+ If you fix the issue, kindly add following tag Reported-by: kernel test robot <lkp(a)intel.com> [ 12.242090] no options. [ 12.245364] FPGA DOWNLOAD ---> [ 12.245723] FPGA image file name: xlinx_fpga_firmware.bit [ 12.246548] GPIO INIT FAIL!! [ 12.246995] most_sound: init() [ 12.247349] BUG: kernel NULL pointer dereference, address: 00000000 [ 12.248032] #PF: supervisor read access in kernel mode [ 12.248322] #PF: error_code(0x0000) - not-present page [ 12.248322] *pdpt = 0000000000000000 *pde = f000ff53f000ff53 [ 12.248322] Oops: 0000 [#1] PREEMPT SMP [ 12.248322] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.6.0-rc7-00376-gb276527539188 #1 [ 12.248322] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014 [ 12.248322] EIP: __list_add_valid+0x29/0x77 [ 12.248322] Code: c3 55 89 e5 56 53 83 ec 10 8b 59 04 39 d3 74 1a 89 4c 24 0c 89 5c 24 08 89 54 24 04 c7 04 24 00 cc bd c2 e8 84 9e b4 ff 0f 0b <8b> 33 39 ce 74 1a 89 5c 24 0c 89 74 24 08 89 4c 24 04 c7 04 24 7c [ 12.248322] EAX: c2f45800 EBX: 00000000 ECX: c3e8df50 EDX: 00000000 [ 12.248322] ESI: 00000000 EDI: ec4a7f68 EBP: ec4a7ee8 ESP: ec4a7ed0 [ 12.248322] DS: 007b ES: 007b FS: 00d8 GS: 00e0 SS: 0068 EFLAGS: 00010246 [ 12.248322] CR0: 80050033 CR2: 00000000 CR3: 03256000 CR4: 001406b0 [ 12.248322] Call Trace: [ 12.248322] ? vprintk_func+0x9d/0xa7 [ 12.248322] most_register_component+0x33/0x53 [ 12.248322] ? wilc_spi_driver_init+0x11/0x11 [ 12.248322] audio_init+0x2c/0x76 [ 12.248322] do_one_initcall+0xf0/0x284 [ 12.248322] ? __might_sleep+0x70/0x77 [ 12.262064] kernel_init_freeable+0x141/0x1a5 [ 12.262064] ? rest_init+0x205/0x205 [ 12.262064] kernel_init+0xb/0xea [ 12.262064] ? schedule_tail_wrapper+0x9/0xc [ 12.262064] ret_from_fork+0x2e/0x38 [ 12.262064] Modules linked in: [ 12.262064] CR2: 0000000000000000 [ 12.262064] ---[ end trace 7c7a2cb6d11f9c5d ]--- [ 12.262064] EIP: __list_add_valid+0x29/0x77 # HH:MM RESULT GOOD BAD GOOD_BUT_DIRTY DIRTY_NOT_BAD git bisect start a400b349905677d3561138087528f36718cbd9fc 16fbf79b0f83bc752cee8589279f1ebfe57b3b6e -- git bisect bad 1fb7e4086438ac799b09b0fcafbe7d6d694d2229 # 10:44 B 0 9 25 0 Merge 'linux-review/Gustavo-A-R-Silva/chrome-cros_ec_chardev-Replace-zero-length-array-with-flexible-array-member/20200321-071741' into devel-hourly-2020032803 git bisect bad aa5a6dfe7759dceb732db7578fd8e93b558c583f # 11:03 B 0 6 22 0 Merge 'linux-review/Ravi-Bangoria/perf-dso-Fix-dso-comparison/20200324-152752' into devel-hourly-2020032803 git bisect good 9f5dc12221c99dd30a64be3fbce2130654d1fcc0 # 11:38 G 10 0 3 3 Merge 'linux-review/Mateusz-Gorski/Add-topologies-for-generic-HDA-DSP-machine-driver/20200326-075250' into devel-hourly-2020032803 git bisect good 4f2fa08ebb25660ca706626bf58b0ddccf6e35c6 # 12:13 G 10 0 3 3 Merge 'thermal/next' into devel-hourly-2020032803 git bisect bad 0b76d1986630dbd2332c91a2f94875ee93ca7e3a # 12:40 B 0 7 23 0 Merge 'linux-review/Weihang-Li/RDMA-hns-Update-some-configurations-related-to-hardware/20200327-030803' into devel-hourly-2020032803 git bisect good 3817f5f77f139d27a42ff92b0fcbae964665a471 # 13:34 G 11 0 2 2 Merge 'linux-review/Leon-Romanovsky/RDMA-rxe-Set-sys_image_guid-to-be-aligned-with-HW-IB-devices/20200323-235957' into devel-hourly-2020032803 git bisect good 37f479d4be63453151a4e8b1db0d18a102fa6e81 # 14:38 G 10 0 2 2 Merge 'linux-review/sunil-kovvuri-gmail-com/Miscellaneous-fixes/20200326-074148' into devel-hourly-2020032803 git bisect bad 9e25511121b25d3191aea63e9c2c65d8beed750e # 14:57 B 0 1 17 0 Merge 'linux-review/Rayagonda-Kokatanur/i2c-iproc-add-support-for-SMBUS-quick-cmd/20200323-045331' into devel-hourly-2020032803 git bisect good 676d651ef12f80cc35ef101bfe610d6a6f12ef10 # 15:40 G 10 0 5 5 Merge 'linux-review/Luiz-Augusto-von-Dentz/Bluetooth-Add-BT_MODE-socket-option/20200327-073534' into devel-hourly-2020032803 git bisect bad 25c4b63ca26e35d0793bdd2c1b24bed8cf95801d # 16:29 B 0 2 18 0 Merge 'linux-review/Simran-Singhal/staging-rtl8723bs-Clean-up-tests-if-NULL-returned-on-failure/20200327-012809' into devel-hourly-2020032803 git bisect good 8eb6b39c0ff38fd1bf21454c438fb3a2a52636f9 # 17:36 G 11 0 4 4 staging: qlge: qlge_dbg: remove unneeded spaces git bisect good fe297f8f048a7a0663479dcf6447ec450b53b905 # 18:35 G 10 0 2 2 iio: dac: Kconfig: sort symbols alphabetically git bisect good 9791ca0dcb46dcedc5fb93fb4e8791cedc91babd # 19:31 G 10 0 4 4 staging: mt7621-pci-phy: re-do 'xtal_mode' detection git bisect bad c926f022c2560d801545e6b55dacc9082277d18e # 20:19 B 0 3 19 0 staging: rtl8723bs: Add line after variable declarations git bisect bad b276527539188f1f61c082ebef27803db93e536d # 20:43 B 0 6 22 0 staging: most: move core files out of the staging area git bisect good 759411a0e0f35c8ae1778a838a97cec9d0140f7f # 21:11 G 10 0 2 2 staging: wlan-ng: Fix third argument going over 80 characters git bisect good 7385adece19190f28800e0fc2fa964b96706120a # 21:42 G 10 0 2 2 staging: hp100: Add space around operator + git bisect good 7cf9a79dbae127776a6ddd455494faec9f3af9fb # 22:13 G 11 0 7 7 Staging: speakup: Use sizeof(*var) in kmalloc(). git bisect good 22dd4acc80495ea0fa3a54b36e49ef9dd5bad6c6 # 22:40 G 10 0 1 1 Staging: speakup: Add identifier name to function declaration arguments. # first bad commit: [b276527539188f1f61c082ebef27803db93e536d] staging: most: move core files out of the staging area git bisect good 22dd4acc80495ea0fa3a54b36e49ef9dd5bad6c6 # 22:45 G 31 0 7 8 Staging: speakup: Add identifier name to function declaration arguments. # extra tests with debug options git bisect good b276527539188f1f61c082ebef27803db93e536d # 23:05 G 11 0 0 0 staging: most: move core files out of the staging area # extra tests on head commit of staging/staging-next git bisect bad e681bb287f40e7a9dbcb04cef80fd87a2511ab86 # 23:49 B 0 1 17 0 staging: vt6656: Use DIV_ROUND_UP macro instead of specific code # bad: [e681bb287f40e7a9dbcb04cef80fd87a2511ab86] staging: vt6656: Use DIV_ROUND_UP macro instead of specific code # extra tests on revert first bad commit git bisect good 099a66b938bb0368b91c81e4431180ef11a0b66b # 00:32 G 10 0 1 1 Revert "staging: most: move core files out of the staging area" # good: [099a66b938bb0368b91c81e4431180ef11a0b66b] Revert "staging: most: move core files out of the staging area" --- 0-DAY CI Kernel Test Service, Intel Corporation https://lists.01.org/hyperkitty/list/lkp@lists.01.org

9 months

3
5
0 / 0

[y2038] 412c53a680: will-it-scale.per_process_ops 11.7% improvement

by kernel test robot

Greeting, FYI, we noticed a 11.7% improvement of will-it-scale.per_process_ops due to commit: commit: 412c53a680a97cb1ae2c0ab60230e193bee86387 ("y2038: remove unused time32 interfaces") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: will-it-scale on test machine: 72 threads Intel(R) Xeon(R) Gold 6139 CPU @ 2.30GHz with 128G memory with following parameters: nr_task: 100% mode: process test: mmap2 cpufreq_governor: performance ucode: 0x2000065 test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two. test-url: https://github.com/antonblanchard/will-it-scale In addition to that, the commit also has significant impact on the following tests: +------------------+---------------------------------------------+ | testcase: change | unixbench: unixbench.score 3.8% improvement | | test machine | 104 threads Skylake with 192G memory | | test parameters | cpufreq_governor=performance | | | nr_task=30% | | | runtime=300s | | | test=context1 | | | ucode=0x2000065 | +------------------+---------------------------------------------+ Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/process/100%/debian-x86_64-20191114.cgz/lkp-skl-2sp7/mmap2/will-it-scale/0x2000065 commit: 595abbaff5 ("y2038: remove ktime to/from timespec/timeval conversion") 412c53a680 ("y2038: remove unused time32 interfaces") 595abbaff5db1214 412c53a680a97cb1ae2c0ab6023 ---------------- --------------------------- %stddev %change %stddev \ | \ 16079 +11.7% 17954 will-it-scale.per_process_ops 1157721 +11.7% 1292722 will-it-scale.workload 535.67 ±141% +201.4% 1614 meminfo.Mlocked 1.01 +0.1 1.12 mpstat.cpu.all.usr% 61269544 ± 54% +137.2% 1.453e+08 ± 48% cpuidle.C1E.time 143915 ± 36% +133.1% 335431 ± 57% cpuidle.C1E.usage 2261 ± 2% -15.1% 1920 ± 5% slabinfo.fsnotify_mark_connector.active_objs 2261 ± 2% -15.1% 1920 ± 5% slabinfo.fsnotify_mark_connector.num_objs 1649 ± 8% -12.5% 1443 ± 2% slabinfo.skbuff_ext_cache.active_objs 1649 ± 8% -12.5% 1443 ± 2% slabinfo.skbuff_ext_cache.num_objs 14258 ± 12% +25.5% 17895 ± 13% numa-meminfo.node0.Mapped 4132 ± 15% +20.3% 4970 numa-meminfo.node0.PageTables 5989 ± 10% -10.3% 5374 ± 4% numa-meminfo.node1.KernelStack 17133 ± 11% -18.9% 13895 ± 17% numa-meminfo.node1.Mapped 4799 ± 12% -18.1% 3928 numa-meminfo.node1.PageTables 7936 +1.4% 8050 proc-vmstat.nr_mapped 133.67 ±141% +201.5% 403.00 proc-vmstat.nr_mlock 18121 ± 2% +5.7% 19153 ± 3% proc-vmstat.nr_shmem 743293 +1.2% 752255 proc-vmstat.pgalloc_normal 700933 +1.4% 710819 proc-vmstat.pgfree 3601 ± 12% +27.9% 4605 ± 12% numa-vmstat.node0.nr_mapped 1033 ± 15% +20.3% 1243 numa-vmstat.node0.nr_page_table_pages 156534 ± 6% -33.2% 104509 ± 64% numa-vmstat.node0.numa_other 5989 ± 10% -10.3% 5375 ± 4% numa-vmstat.node1.nr_kernel_stack 56.00 ±141% +278.0% 211.67 ± 13% numa-vmstat.node1.nr_mlock 1199 ± 12% -18.1% 982.33 numa-vmstat.node1.nr_page_table_pages 5818 ± 35% +43.9% 8370 ± 6% interrupts.CPU26.NMI:Non-maskable_interrupts 5818 ± 35% +43.9% 8370 ± 6% interrupts.CPU26.PMI:Performance_monitoring_interrupts 414.67 ± 23% -24.8% 311.67 interrupts.CPU52.RES:Rescheduling_interrupts 5819 ± 35% +43.8% 8366 ± 6% interrupts.CPU56.NMI:Non-maskable_interrupts 5819 ± 35% +43.8% 8366 ± 6% interrupts.CPU56.PMI:Performance_monitoring_interrupts 5818 ± 35% +43.8% 8364 ± 6% interrupts.CPU59.NMI:Non-maskable_interrupts 5818 ± 35% +43.8% 8364 ± 6% interrupts.CPU59.PMI:Performance_monitoring_interrupts 471.33 ± 16% -26.0% 349.00 ± 6% interrupts.CPU9.RES:Rescheduling_interrupts 127.33 ± 3% +11.8% 142.33 ± 2% interrupts.IWI:IRQ_work_interrupts 10312 ± 92% -67.7% 3333 ± 6% sched_debug.cfs_rq:/.load.stddev 27.16 ± 7% -16.2% 22.76 ± 8% sched_debug.cfs_rq:/.load_avg.avg 47.12 ± 6% -17.5% 38.89 ± 10% sched_debug.cfs_rq:/.load_avg.stddev 8.99 ± 22% -46.6% 4.80 ± 41% sched_debug.cfs_rq:/.removed.load_avg.avg 37.49 ± 10% -27.4% 27.22 ± 21% sched_debug.cfs_rq:/.removed.load_avg.stddev 413.16 ± 23% -46.4% 221.38 ± 42% sched_debug.cfs_rq:/.removed.runnable_sum.avg 1722 ± 10% -27.2% 1253 ± 22% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 2.71 ± 22% -41.0% 1.60 ± 16% sched_debug.cfs_rq:/.removed.util_avg.avg 1480 ± 4% +11.4% 1648 sched_debug.cpu.curr->pid.min 0.19 ± 2% -13.4% 0.17 ± 7% sched_debug.cpu.nr_running.stddev 32567 ± 5% +13.4% 36931 ± 4% sched_debug.cpu.nr_switches.max 62048 ± 5% +21.0% 75097 ± 10% softirqs.CPU10.RCU 60821 +9.1% 66347 softirqs.CPU17.RCU 59451 ± 3% +19.2% 70891 ± 2% softirqs.CPU22.RCU 59548 +7.6% 64046 ± 2% softirqs.CPU34.RCU 59706 +16.4% 69478 ± 6% softirqs.CPU41.RCU 61173 ± 4% +22.1% 74662 ± 12% softirqs.CPU46.RCU 59827 +21.6% 72779 ± 14% softirqs.CPU5.RCU 60645 +11.0% 67300 ± 5% softirqs.CPU53.RCU 58533 ± 2% +9.0% 63779 ± 2% softirqs.CPU57.RCU 60026 ± 2% +15.7% 69444 ± 5% softirqs.CPU58.RCU 61127 ± 2% +11.4% 68125 ± 4% softirqs.CPU63.RCU 4.413e+09 +7.3% 4.733e+09 perf-stat.i.branch-instructions 19144349 +5.4% 20170738 perf-stat.i.branch-misses 41.28 -0.5 40.80 perf-stat.i.cache-miss-rate% 22256771 +3.9% 23133318 ± 2% perf-stat.i.cache-misses 53886473 +5.1% 56658720 ± 2% perf-stat.i.cache-references 11.77 -7.4% 10.90 perf-stat.i.cpi 9929 -4.3% 9499 ± 2% perf-stat.i.cycles-between-cache-misses 0.05 +0.0 0.05 ± 3% perf-stat.i.dTLB-load-miss-rate% 2370068 +11.2% 2635764 perf-stat.i.dTLB-load-misses 5.229e+09 +7.8% 5.638e+09 perf-stat.i.dTLB-loads 9837 ± 3% +11.8% 10995 ± 6% perf-stat.i.dTLB-store-misses 1.718e+09 +10.8% 1.903e+09 perf-stat.i.dTLB-stores 94.23 -9.1 85.12 perf-stat.i.iTLB-load-miss-rate% 2419173 -33.5% 1607709 perf-stat.i.iTLB-load-misses 146272 ± 3% +94.5% 284553 ± 8% perf-stat.i.iTLB-loads 1.88e+10 +7.4% 2.019e+10 perf-stat.i.instructions 7824 +61.4% 12627 perf-stat.i.instructions-per-iTLB-miss 0.09 +8.0% 0.09 perf-stat.i.ipc 5549135 +6.7% 5919869 perf-stat.i.node-load-misses 4803604 +6.0% 5091055 perf-stat.i.node-store-misses 0.43 -0.0 0.43 perf-stat.overall.branch-miss-rate% 41.30 -0.5 40.82 perf-stat.overall.cache-miss-rate% 11.76 -7.2% 10.91 perf-stat.overall.cpi 9937 -4.1% 9530 ± 2% perf-stat.overall.cycles-between-cache-misses 0.05 +0.0 0.05 perf-stat.overall.dTLB-load-miss-rate% 94.30 -9.3 84.96 perf-stat.overall.iTLB-load-miss-rate% 7770 +61.7% 12560 perf-stat.overall.instructions-per-iTLB-miss 0.09 +7.8% 0.09 perf-stat.overall.ipc 4887274 -3.3% 4723803 perf-stat.overall.path-length 4.398e+09 +7.3% 4.717e+09 perf-stat.ps.branch-instructions 19086955 +5.4% 20108226 perf-stat.ps.branch-misses 22182689 +3.9% 23057159 ± 2% perf-stat.ps.cache-misses 53712718 +5.2% 56486162 ± 2% perf-stat.ps.cache-references 2362866 +11.3% 2630144 perf-stat.ps.dTLB-load-misses 5.211e+09 +7.8% 5.62e+09 perf-stat.ps.dTLB-loads 9852 ± 3% +12.8% 11114 ± 6% perf-stat.ps.dTLB-store-misses 1.712e+09 +10.8% 1.897e+09 perf-stat.ps.dTLB-stores 2411038 -33.5% 1602293 perf-stat.ps.iTLB-load-misses 145843 ± 3% +94.6% 283843 ± 8% perf-stat.ps.iTLB-loads 1.873e+10 +7.4% 2.013e+10 perf-stat.ps.instructions 5530540 +6.7% 5900087 perf-stat.ps.node-load-misses 4787463 +6.0% 5073936 perf-stat.ps.node-store-misses 5.658e+12 +7.9% 6.107e+12 perf-stat.total.instructions 47.65 -0.8 46.85 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap.vm_mmap_pgoff 47.66 -0.8 46.85 perf-profile.calltrace.cycles-pp.__vm_enough_memory.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff 49.09 -0.8 48.31 perf-profile.calltrace.cycles-pp.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe 48.84 -0.8 48.06 perf-profile.calltrace.cycles-pp.mmap_region.do_mmap.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64 47.18 -0.8 46.40 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region.do_mmap 49.24 -0.8 48.47 perf-profile.calltrace.cycles-pp.vm_mmap_pgoff.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 49.31 -0.8 48.55 perf-profile.calltrace.cycles-pp.ksys_mmap_pgoff.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 46.98 -0.8 46.23 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__vm_enough_memory.mmap_region 49.62 -0.7 48.89 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.mmap64 49.63 -0.7 48.90 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.mmap64 50.01 -0.7 49.31 perf-profile.calltrace.cycles-pp.mmap64 1.17 +0.1 1.28 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap 1.27 +0.1 1.40 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap 1.69 +0.2 1.84 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 46.89 +0.5 47.36 perf-profile.calltrace.cycles-pp.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64 46.41 +0.5 46.89 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.percpu_counter_add_batch.__do_munmap.__vm_munmap.__x64_sys_munmap 46.21 +0.5 46.70 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.percpu_counter_add_batch.__do_munmap.__vm_munmap 0.00 +0.5 0.53 perf-profile.calltrace.cycles-pp.___might_sleep.unmap_page_range.unmap_vmas.unmap_region.__do_munmap 49.26 +0.6 49.89 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 49.29 +0.6 49.92 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 48.83 +0.6 49.48 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe 49.59 +0.7 50.26 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.munmap 49.60 +0.7 50.27 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.munmap 49.94 +0.7 50.64 perf-profile.calltrace.cycles-pp.munmap 47.66 -0.8 46.85 perf-profile.children.cycles-pp.__vm_enough_memory 49.10 -0.8 48.32 perf-profile.children.cycles-pp.do_mmap 48.84 -0.8 48.07 perf-profile.children.cycles-pp.mmap_region 49.24 -0.8 48.48 perf-profile.children.cycles-pp.vm_mmap_pgoff 49.31 -0.8 48.55 perf-profile.children.cycles-pp.ksys_mmap_pgoff 50.03 -0.7 49.33 perf-profile.children.cycles-pp.mmap64 94.55 -0.3 94.21 perf-profile.children.cycles-pp.percpu_counter_add_batch 93.61 -0.3 93.31 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 93.19 -0.3 92.92 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 99.23 -0.1 99.18 perf-profile.children.cycles-pp.do_syscall_64 99.26 -0.0 99.21 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 0.46 ± 2% -0.0 0.43 perf-profile.children.cycles-pp.vm_area_alloc 0.43 ± 3% -0.0 0.40 perf-profile.children.cycles-pp.kmem_cache_alloc 0.31 ± 2% -0.0 0.29 perf-profile.children.cycles-pp.apic_timer_interrupt 0.06 +0.0 0.07 perf-profile.children.cycles-pp.down_write_killable 0.08 ± 6% +0.0 0.09 perf-profile.children.cycles-pp.prepend_path 0.06 +0.0 0.07 ± 6% perf-profile.children.cycles-pp.shmem_mmap 0.10 ± 4% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.perf_iterate_sb 0.05 +0.0 0.07 perf-profile.children.cycles-pp.unlink_file_vma 0.33 +0.0 0.35 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.40 ± 4% +0.0 0.43 perf-profile.children.cycles-pp.perf_event_mmap 0.08 ± 5% +0.0 0.11 perf-profile.children.cycles-pp.free_pgtables 0.34 +0.0 0.38 perf-profile.children.cycles-pp.entry_SYSCALL_64 0.00 +0.1 0.05 perf-profile.children.cycles-pp.up_write 0.00 +0.1 0.05 perf-profile.children.cycles-pp.touch_atime 0.53 +0.1 0.60 perf-profile.children.cycles-pp.___might_sleep 1.25 +0.1 1.35 perf-profile.children.cycles-pp.unmap_page_range 1.28 +0.1 1.40 perf-profile.children.cycles-pp.unmap_vmas 1.69 +0.2 1.85 perf-profile.children.cycles-pp.unmap_region 49.29 +0.6 49.93 perf-profile.children.cycles-pp.__x64_sys_munmap 49.26 +0.6 49.90 perf-profile.children.cycles-pp.__vm_munmap 48.84 +0.6 49.48 perf-profile.children.cycles-pp.__do_munmap 49.97 +0.7 50.67 perf-profile.children.cycles-pp.munmap 93.19 -0.3 92.92 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.42 ± 2% -0.0 0.38 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.40 ± 3% -0.0 0.38 perf-profile.self.cycles-pp.kmem_cache_alloc 0.12 -0.0 0.11 perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore 0.33 +0.0 0.35 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.30 +0.0 0.33 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.61 ± 2% +0.0 0.66 perf-profile.self.cycles-pp.unmap_page_range 0.58 +0.1 0.64 perf-profile.self.cycles-pp.do_syscall_64 0.50 +0.1 0.56 perf-profile.self.cycles-pp.___might_sleep will-it-scale.per_process_ops 20000 +-------------------------------------------------------------------+ 18000 |-+O O O O O O O O O O O O O O O O | | | 16000 |..+.+..+.+..+..+.+..+..+.+..+.+..+..+.+..+.+..+..+.+..+..+.+..+ | 14000 |-+ : | | : :| 12000 |-+ : :| 10000 |-+ : :| 8000 |-+ : :| | : : | 6000 |-+ : : | 4000 |-+ : : | | :: | 2000 |-+ : | 0 +-------------------------------------------------------------------+ will-it-scale.workload 1.4e+06 +-----------------------------------------------------------------+ | O O O O O O O O O O O O O O O O | 1.2e+06 |..+.+.. .+..+.+..+.+..+.+..+. .+..+.+..+. .+.+..+.+..+.+..+ | | + +. +. : | 1e+06 |-+ : | | : :| 800000 |-+ : :| | : :| 600000 |-+ : :| | : : | 400000 |-+ : : | | :: | 200000 |-+ :: | | : | 0 +-----------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample *************************************************************************************************** lkp-skl-fpga01: 104 threads Skylake with 192G memory ========================================================================================= compiler/cpufreq_governor/kconfig/nr_task/rootfs/runtime/tbox_group/test/testcase/ucode: gcc-7/performance/x86_64-rhel-7.6/30%/debian-x86_64-20191114.cgz/300s/lkp-skl-fpga01/context1/unixbench/0x2000065 commit: 595abbaff5 ("y2038: remove ktime to/from timespec/timeval conversion") 412c53a680 ("y2038: remove unused time32 interfaces") 595abbaff5db1214 412c53a680a97cb1ae2c0ab6023 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 3093 +3.8% 3209 unixbench.score 471.47 +36.6% 644.15 ± 40% unixbench.time.user_time 3.659e+08 +3.9% 3.801e+08 unixbench.time.voluntary_context_switches 4.838e+08 +3.8% 5.02e+08 unixbench.workload 3715604 +3.9% 3859617 vmstat.system.cs 1493573 -1.1% 1477832 proc-vmstat.numa_local 1620946 -1.3% 1599963 proc-vmstat.pgalloc_normal 1586478 -1.6% 1561368 proc-vmstat.pgfree 4505528 ± 8% -11.7% 3977138 ± 9% sched_debug.cfs_rq:/.MIN_vruntime.max 47683 ± 7% -11.2% 42356 ± 7% sched_debug.cfs_rq:/.exec_clock.min 4505528 ± 8% -11.7% 3977138 ± 9% sched_debug.cfs_rq:/.max_vruntime.max 2750853 ± 7% -11.4% 2435892 ± 7% sched_debug.cfs_rq:/.min_vruntime.min 0.31 ± 13% -33.1% 0.21 ± 25% sched_debug.cfs_rq:/.nr_spread_over.avg 3.18 ± 18% -28.1% 2.29 ± 22% sched_debug.cfs_rq:/.nr_spread_over.max 0.67 ± 11% -27.6% 0.48 ± 22% sched_debug.cfs_rq:/.nr_spread_over.stddev 133566 ± 48% +65.9% 221588 ± 31% sched_debug.cfs_rq:/.runnable_weight.max 899.95 ±173% +824.1% 8316 ±110% sched_debug.cpu.max_idle_balance_cost.stddev 317.01 ± 5% -9.2% 287.99 ± 7% sched_debug.cpu.ttwu_local.min 6.175e+09 +3.2% 6.375e+09 perf-stat.i.branch-instructions 6.846e+08 +2.4% 7.011e+08 perf-stat.i.cache-references 3734905 +3.9% 3879483 perf-stat.i.context-switches 7.582e+09 +3.4% 7.839e+09 perf-stat.i.dTLB-loads 4.468e+09 +3.6% 4.63e+09 perf-stat.i.dTLB-stores 4308069 +3.9% 4477501 perf-stat.i.iTLB-load-misses 28346961 +4.7% 29683898 perf-stat.i.iTLB-loads 2.823e+10 +3.3% 2.916e+10 perf-stat.i.instructions 3745153 +21.4% 4545801 perf-stat.i.node-load-misses 33138 +33.5% 44236 perf-stat.i.node-loads 3740245 +3.3% 3862099 perf-stat.i.node-store-misses 3.02 -2.9% 2.94 perf-stat.overall.cpi 0.33 +3.0% 0.34 perf-stat.overall.ipc 6.157e+09 +3.3% 6.357e+09 perf-stat.ps.branch-instructions 6.826e+08 +2.4% 6.991e+08 perf-stat.ps.cache-references 3723897 +3.9% 3868660 perf-stat.ps.context-switches 7.56e+09 +3.4% 7.817e+09 perf-stat.ps.dTLB-loads 4.454e+09 +3.7% 4.617e+09 perf-stat.ps.dTLB-stores 4295642 +3.9% 4465151 perf-stat.ps.iTLB-load-misses 28265235 +4.7% 29603109 perf-stat.ps.iTLB-loads 2.814e+10 +3.3% 2.908e+10 perf-stat.ps.instructions 3734083 +21.4% 4532943 perf-stat.ps.node-load-misses 33087 +33.4% 44149 perf-stat.ps.node-loads 3729103 +3.3% 3851108 perf-stat.ps.node-store-misses 1.103e+13 +3.3% 1.14e+13 perf-stat.total.instructions 372.25 ± 48% +114.4% 798.00 ± 28% interrupts.41:PCI-MSI.67633156-edge.eth0-TxRx-3 2920 ± 6% -12.3% 2561 ± 10% interrupts.CPU1.NMI:Non-maskable_interrupts 2920 ± 6% -12.3% 2561 ± 10% interrupts.CPU1.PMI:Performance_monitoring_interrupts 2691 ± 6% +18.2% 3179 ± 5% interrupts.CPU103.NMI:Non-maskable_interrupts 2691 ± 6% +18.2% 3179 ± 5% interrupts.CPU103.PMI:Performance_monitoring_interrupts 33.25 ± 37% +542.9% 213.75 ± 64% interrupts.CPU14.TLB:TLB_shootdowns 71.50 ±115% +202.8% 216.50 ± 83% interrupts.CPU16.TLB:TLB_shootdowns 3048 ± 13% -26.0% 2256 ± 14% interrupts.CPU18.NMI:Non-maskable_interrupts 3048 ± 13% -26.0% 2256 ± 14% interrupts.CPU18.PMI:Performance_monitoring_interrupts 2743 ± 4% -14.8% 2337 ± 18% interrupts.CPU19.NMI:Non-maskable_interrupts 2743 ± 4% -14.8% 2337 ± 18% interrupts.CPU19.PMI:Performance_monitoring_interrupts 26.00 ± 61% +1373.1% 383.00 ± 50% interrupts.CPU23.TLB:TLB_shootdowns 229.00 ± 45% +87.3% 429.00 ± 29% interrupts.CPU29.TLB:TLB_shootdowns 372.25 ± 48% +114.4% 798.00 ± 28% interrupts.CPU33.41:PCI-MSI.67633156-edge.eth0-TxRx-3 28.75 ± 50% +360.9% 132.50 ± 94% interrupts.CPU39.TLB:TLB_shootdowns 41.00 ±101% +600.0% 287.00 ± 37% interrupts.CPU49.TLB:TLB_shootdowns 39.75 ± 79% +140.9% 95.75 ± 13% interrupts.CPU50.TLB:TLB_shootdowns 3103 ± 9% -16.6% 2589 ± 10% interrupts.CPU53.NMI:Non-maskable_interrupts 3103 ± 9% -16.6% 2589 ± 10% interrupts.CPU53.PMI:Performance_monitoring_interrupts 163.75 ± 58% -58.2% 68.50 ±100% interrupts.CPU59.TLB:TLB_shootdowns 2469 ± 20% +25.1% 3089 ± 6% interrupts.CPU63.NMI:Non-maskable_interrupts 2469 ± 20% +25.1% 3089 ± 6% interrupts.CPU63.PMI:Performance_monitoring_interrupts 69.25 ±107% +210.1% 214.75 ± 57% interrupts.CPU7.TLB:TLB_shootdowns 49.75 ± 33% +165.3% 132.00 ± 74% interrupts.CPU76.TLB:TLB_shootdowns 27.75 ± 82% +272.1% 103.25 ± 66% interrupts.CPU78.TLB:TLB_shootdowns 2754 ± 23% +23.0% 3387 ± 7% interrupts.CPU84.NMI:Non-maskable_interrupts 2754 ± 23% +23.0% 3387 ± 7% interrupts.CPU84.PMI:Performance_monitoring_interrupts 22.50 ± 42% +687.8% 177.25 ±132% interrupts.CPU84.TLB:TLB_shootdowns 19.75 ± 35% +812.7% 180.25 ± 74% interrupts.CPU85.TLB:TLB_shootdowns 15.75 ± 18% +303.2% 63.50 ± 81% interrupts.CPU99.TLB:TLB_shootdowns 31.35 -0.6 30.72 perf-profile.calltrace.cycles-pp._raw_spin_lock_irqsave.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task 30.60 -0.6 29.98 perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.__account_scheduler_latency.enqueue_entity.enqueue_task_fair 34.93 -0.5 34.42 perf-profile.calltrace.cycles-pp.__account_scheduler_latency.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate 36.88 -0.5 36.42 perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up 37.21 -0.4 36.77 perf-profile.calltrace.cycles-pp.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common 37.18 -0.4 36.75 perf-profile.calltrace.cycles-pp.enqueue_task_fair.activate_task.ttwu_do_activate.try_to_wake_up.autoremove_wake_function 37.22 -0.4 36.79 perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock 35.18 -0.4 34.79 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 35.30 -0.4 34.93 perf-profile.calltrace.cycles-pp.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 39.42 -0.4 39.06 perf-profile.calltrace.cycles-pp.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write 39.26 -0.4 38.91 perf-profile.calltrace.cycles-pp.try_to_wake_up.autoremove_wake_function.__wake_up_common.__wake_up_common_lock.pipe_write 39.64 -0.3 39.30 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write 39.95 -0.3 39.62 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.pipe_write.new_sync_write.vfs_write.ksys_write 41.14 -0.3 40.83 perf-profile.calltrace.cycles-pp.new_sync_write.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe 41.08 -0.3 40.77 perf-profile.calltrace.cycles-pp.pipe_write.new_sync_write.vfs_write.ksys_write.do_syscall_64 0.79 ± 7% -0.1 0.69 ± 9% perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry 0.73 ± 8% -0.1 0.64 ± 9% perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter.do_idle 0.69 +0.0 0.72 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret 0.70 ± 2% +0.0 0.73 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64 0.75 +0.0 0.78 perf-profile.calltrace.cycles-pp.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle.cpu_startup_entry 0.98 ± 2% +0.0 1.02 perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__sched_text_start.schedule_idle.do_idle 1.26 +0.0 1.31 perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule.pipe_read 1.47 +0.1 1.52 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__sched_text_start.schedule.pipe_read.new_sync_read 2.38 +0.1 2.50 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule_idle.do_idle.cpu_startup_entry.start_secondary 2.43 +0.1 2.56 perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 5.47 +0.1 5.61 perf-profile.calltrace.cycles-pp.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 3.32 +0.1 3.47 perf-profile.calltrace.cycles-pp.schedule.pipe_read.new_sync_read.vfs_read.ksys_read 5.29 +0.2 5.46 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.94 ± 2% +0.2 6.14 perf-profile.calltrace.cycles-pp.new_sync_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe 5.83 ± 2% +0.2 6.04 perf-profile.calltrace.cycles-pp.pipe_read.new_sync_read.vfs_read.ksys_read.do_syscall_64 3.11 ± 8% +0.3 3.38 perf-profile.calltrace.cycles-pp.__sched_text_start.schedule.pipe_read.new_sync_read.vfs_read 0.13 ±173% +0.4 0.53 perf-profile.calltrace.cycles-pp.update_curr.dequeue_entity.dequeue_task_fair.__sched_text_start.schedule 2.47 ± 9% +0.4 2.89 perf-profile.calltrace.cycles-pp.arch_stack_walk.stack_trace_save_tsk.__account_scheduler_latency.enqueue_entity.enqueue_task_fair 30.79 -0.6 30.14 perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 32.21 -0.6 31.62 perf-profile.children.cycles-pp._raw_spin_lock_irqsave 34.94 -0.5 34.43 perf-profile.children.cycles-pp.__account_scheduler_latency 36.94 -0.5 36.48 perf-profile.children.cycles-pp.enqueue_entity 37.19 -0.4 36.75 perf-profile.children.cycles-pp.enqueue_task_fair 37.22 -0.4 36.78 perf-profile.children.cycles-pp.activate_task 37.23 -0.4 36.79 perf-profile.children.cycles-pp.ttwu_do_activate 39.27 -0.4 38.91 perf-profile.children.cycles-pp.try_to_wake_up 39.65 -0.4 39.29 perf-profile.children.cycles-pp.__wake_up_common 39.42 -0.4 39.06 perf-profile.children.cycles-pp.autoremove_wake_function 39.95 -0.3 39.62 perf-profile.children.cycles-pp.__wake_up_common_lock 41.16 -0.3 40.85 perf-profile.children.cycles-pp.new_sync_write 41.08 -0.3 40.77 perf-profile.children.cycles-pp.pipe_write 41.45 -0.3 41.16 perf-profile.children.cycles-pp.vfs_write 41.59 -0.3 41.32 perf-profile.children.cycles-pp.ksys_write 0.05 +0.0 0.06 perf-profile.children.cycles-pp.apparmor_file_permission 0.17 ± 4% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.tick_nohz_idle_enter 0.33 +0.0 0.35 ± 3% perf-profile.children.cycles-pp.select_idle_sibling 0.41 +0.0 0.43 ± 2% perf-profile.children.cycles-pp.__next_timer_interrupt 0.25 ± 3% +0.0 0.27 ± 3% perf-profile.children.cycles-pp.rcu_idle_exit 0.38 +0.0 0.40 ± 2% perf-profile.children.cycles-pp.__switch_to_asm 0.49 ± 2% +0.0 0.51 perf-profile.children.cycles-pp.update_rq_clock 0.23 ± 3% +0.0 0.26 ± 3% perf-profile.children.cycles-pp.common_file_perm 0.57 +0.0 0.60 ± 2% perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore 0.45 ± 2% +0.0 0.48 ± 2% perf-profile.children.cycles-pp.stack_trace_consume_entry_nosched 0.14 ± 13% +0.0 0.17 ± 7% perf-profile.children.cycles-pp.clockevents_program_event 0.70 ± 2% +0.0 0.73 perf-profile.children.cycles-pp.update_curr 0.44 ± 4% +0.0 0.48 ± 2% perf-profile.children.cycles-pp.copy_page_to_iter 0.76 +0.0 0.80 perf-profile.children.cycles-pp.tick_nohz_next_event 1.00 ± 2% +0.0 1.04 perf-profile.children.cycles-pp.set_next_entity 0.86 +0.0 0.90 perf-profile.children.cycles-pp.entry_SYSCALL_64 0.94 +0.0 0.98 perf-profile.children.cycles-pp.syscall_return_via_sysret 0.33 ± 2% +0.0 0.38 ± 3% perf-profile.children.cycles-pp.security_file_permission 0.00 +0.1 0.05 perf-profile.children.cycles-pp.tick_nohz_tick_stopped 1.45 ± 2% +0.1 1.51 perf-profile.children.cycles-pp.pick_next_task_fair 0.36 ± 8% +0.1 0.42 ± 6% perf-profile.children.cycles-pp.ktime_get 0.00 +0.1 0.06 perf-profile.children.cycles-pp.__x64_sys_read 1.25 ± 3% +0.1 1.31 perf-profile.children.cycles-pp.update_load_avg 1.56 ± 2% +0.1 1.63 perf-profile.children.cycles-pp.dequeue_entity 2.83 +0.1 2.91 perf-profile.children.cycles-pp.arch_stack_walk 1.78 ± 2% +0.1 1.88 perf-profile.children.cycles-pp.dequeue_task_fair 2.45 +0.1 2.57 perf-profile.children.cycles-pp.schedule_idle 3.33 +0.1 3.47 perf-profile.children.cycles-pp.schedule 5.94 ± 2% +0.2 6.14 perf-profile.children.cycles-pp.new_sync_read 5.85 +0.2 6.06 perf-profile.children.cycles-pp.pipe_read 6.58 ± 2% +0.2 6.80 perf-profile.children.cycles-pp.ksys_read 6.39 ± 2% +0.2 6.63 perf-profile.children.cycles-pp.vfs_read 5.70 +0.2 5.95 perf-profile.children.cycles-pp.__sched_text_start 30.79 -0.6 30.14 perf-profile.self.cycles-pp.native_queued_spin_lock_slowpath 0.09 -0.0 0.08 ± 5% perf-profile.self.cycles-pp.__fsnotify_parent 0.09 -0.0 0.08 perf-profile.self.cycles-pp.ksys_read 0.08 ± 5% +0.0 0.09 perf-profile.self.cycles-pp.in_sched_functions 0.15 ± 2% +0.0 0.17 ± 7% perf-profile.self.cycles-pp.pipe_write 0.21 ± 2% +0.0 0.23 ± 3% perf-profile.self.cycles-pp._find_next_bit 0.20 ± 2% +0.0 0.22 ± 3% perf-profile.self.cycles-pp.common_file_perm 0.38 +0.0 0.40 ± 2% perf-profile.self.cycles-pp.__switch_to_asm 0.04 ± 57% +0.0 0.06 perf-profile.self.cycles-pp.apparmor_file_permission 0.30 ± 2% +0.0 0.32 ± 2% perf-profile.self.cycles-pp.stack_trace_consume_entry_nosched 0.14 ± 3% +0.0 0.16 ± 7% perf-profile.self.cycles-pp.vfs_read 0.18 ± 2% +0.0 0.20 ± 4% perf-profile.self.cycles-pp.__account_scheduler_latency 1.57 +0.0 1.61 perf-profile.self.cycles-pp._raw_spin_lock_irqsave 1.15 +0.0 1.18 perf-profile.self.cycles-pp.__sched_text_start 0.84 +0.0 0.88 perf-profile.self.cycles-pp.entry_SYSCALL_64 0.94 +0.0 0.98 perf-profile.self.cycles-pp.syscall_return_via_sysret 0.00 +0.1 0.05 perf-profile.self.cycles-pp.ksys_write 0.00 +0.1 0.05 perf-profile.self.cycles-pp.rcu_idle_exit 0.18 ± 13% +0.1 0.23 ± 10% perf-profile.self.cycles-pp.ktime_get 0.00 +0.1 0.06 ± 7% perf-profile.self.cycles-pp.__x64_sys_read 1.59 ± 2% +0.1 1.66 perf-profile.self.cycles-pp.do_syscall_64 Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

9 months

3
2
0 / 0

[ext4] b1b4705d54: filebench.sum_bytes_mb/s -20.2% regression

by kernel test robot

Greeting, FYI, we noticed a -20.2% regression of filebench.sum_bytes_mb/s due to commit: commit: b1b4705d54abedfd69dcdf42779c521aa1e0fbd3 ("ext4: introduce direct I/O read using iomap infrastructure") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: filebench on test machine: 8 threads Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz with 8G memory with following parameters: disk: 1HDD fs: ext4 test: fivestreamreaddirect.f cpufreq_governor: performance ucode: 0x27 If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/1HDD/ext4/x86_64-rhel-7.6/debian-x86_64-2019-11-14.cgz/lkp-hsw-d01/fivestreamreaddirect.f/filebench/0x27 commit: 09edf4d381 ("ext4: introduce new callback for IOMAP_REPORT") b1b4705d54 ("ext4: introduce direct I/O read using iomap infrastructure") 09edf4d381957b14 b1b4705d54abedfd69dcdf42779 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.WARNING:at#for_ip_swapgs_restore_regs_and_return_to_usermode/0x :4 25% 1:4 dmesg.WARNING:stack_recursion :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:da :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:db :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dc :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:dd :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:de :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:df :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ea :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:eb :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ec :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ed :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ee :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ef :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fa :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fb :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fc :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fd :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:fe :4 25% 1:4 kmsg.addr:#vm_flags:#anon_vma:#mapping:#index:ff :4 25% 1:4 kmsg.file:libnss_files-#so_fault:filemap_fault_mmap:generic_file_mmap_readpage:simple_readpage :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a07fffffa0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a17fffffa1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a27fffffa2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a37fffffa3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a47fffffa4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a57fffffa5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a67fffffa6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a77fffffa7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff8aff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a7fffff9aff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a87fffffa8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#a97fffffa9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#aa7fffffaaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ab7fffffabff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ac7fffffacff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ad7fffffadff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ae7fffffaeff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#af7fffffafff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b07fffffb0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b17fffffb1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b27fffffb2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b37fffffb3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b47fffffb4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b57fffffb5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b67fffffb6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b77fffffb7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff8bff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b7fffff9bff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b87fffffb8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#b97fffffb9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ba7fffffbaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bb7fffffbbff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bc7fffffbcff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bd7fffffbdff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#be7fffffbeff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#bf7fffffbfff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c07fffffc0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c17fffffc1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c27fffffc2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c37fffffc3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c47fffffc4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c57fffffc5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c67fffffc6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c77fffffc7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff8cff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c7fffff9cff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c87fffffc8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#c97fffffc9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ca7fffffcaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cb7fffffcbff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cc7fffffccff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cd7fffffcdff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ce7fffffceff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#cf7fffffcfff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d07fffffd0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d17fffffd1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d27fffffd2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d37fffffd3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d47fffffd4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d57fffffd5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d67fffffd6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d77fffffd7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff8dff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d7fffff9dff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d87fffffd8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#d97fffffd9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#da7fffffdaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#db7fffffdbff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dc7fffffdcff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#dd7fffffddff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#de7fffffdeff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#df7fffffdfff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e07fffffe0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e17fffffe1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e27fffffe2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e37fffffe3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e47fffffe4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e57fffffe5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e67fffffe6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e77fffffe7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff8eff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e7fffff9eff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e87fffffe8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#e97fffffe9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ea7fffffeaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#eb7fffffebff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ec7fffffecff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ed7fffffedff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ee7fffffeeff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ef7fffffefff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f07ffffff0ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f17ffffff1ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f27ffffff2ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f37ffffff3ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f47ffffff4ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f57ffffff5ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f67ffffff6ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f77ffffff7ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff8fff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f7fffff9fff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f87ffffff8ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#f97ffffff9ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fa7ffffffaff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fb7ffffffbff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fc7ffffffcff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fd7ffffffdff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fe7ffffffeff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#ff7fffffffff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff80ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff81ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff82ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff83ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff84ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff85ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff86ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff87ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff88ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff89ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff90ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff91ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff92ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff93ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff94ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff95ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff96ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff97ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff98ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffff99ff :4 25% 1:4 kmsg.swap_info_get:Bad_swap_file_entry#fffffffffff %stddev %change %stddev \ | \ 58.30 ± 2% -20.2% 46.53 filebench.sum_bytes_mb/s 3503 ± 2% -20.2% 2796 filebench.sum_operations 58.39 ± 2% -20.2% 46.61 filebench.sum_operations/s 58.33 ± 2% -19.9% 46.75 filebench.sum_reads/s 84.34 ± 4% +27.1% 107.17 filebench.sum_time_ms/op 7174922 ± 2% -20.2% 5727328 filebench.time.file_system_inputs 10963 ± 2% -8.5% 10035 filebench.time.voluntary_context_switches 58.37 ± 3% -9.0 49.38 ± 3% mpstat.cpu.all.idle% 41.08 ± 5% +8.2 49.24 ± 2% mpstat.cpu.all.iowait% 3061 ± 6% +6.1% 3246 ± 4% turbostat.Bzy_MHz 5.48 ± 97% +260.7% 19.78 ± 30% turbostat.Pkg%pc2 27.33 ± 32% -52.0% 13.12 ± 19% sched_debug.cpu.nr_uninterruptible.max -67.67 -60.8% -26.50 sched_debug.cpu.nr_uninterruptible.min 28.48 ± 30% -57.9% 11.98 ± 23% sched_debug.cpu.nr_uninterruptible.stddev 8210 ± 44% +76.1% 14454 ± 6% softirqs.CPU0.RCU 8200 ± 40% +70.2% 13954 ± 5% softirqs.CPU2.RCU 9772 ± 36% +59.6% 15597 ± 15% softirqs.CPU3.RCU 108.33 ± 27% +113.5% 231.25 ± 26% interrupts.CPU1.RES:Rescheduling_interrupts 1032 -23.9% 786.00 ± 30% interrupts.CPU5.CAL:Function_call_interrupts 229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.NMI:Non-maskable_interrupts 229.33 ± 46% +50.1% 344.25 ± 31% interrupts.CPU5.PMI:Performance_monitoring_interrupts 457.00 ± 61% -74.2% 117.75 ± 41% interrupts.CPU5.RES:Rescheduling_interrupts 53.33 ± 60% +206.6% 163.50 ± 42% interrupts.CPU7.RES:Rescheduling_interrupts 58.99 ± 3% -14.9% 50.18 ± 3% iostat.cpu.idle 40.36 ± 5% +19.8% 48.35 ± 2% iostat.cpu.iowait 0.18 +386.3% 0.87 ±137% iostat.cpu.user 33012 ± 2% -19.7% 26520 iostat.sda.rkB/s 40.60 ± 6% +16.2% 47.18 ± 4% iostat.sda.w/s 43038 ± 4% +17.1% 50399 ± 2% iostat.sda.wkB/s 58.67 ± 3% -15.6% 49.50 ± 3% vmstat.cpu.id 40.00 ± 5% +19.4% 47.75 ± 2% vmstat.cpu.wa 33010 ± 2% -19.7% 26518 vmstat.io.bi 42837 ± 4% +17.2% 50200 ± 2% vmstat.io.bo 5519500 -15.5% 4664152 vmstat.memory.cache 2183603 +39.2% 3039439 ± 2% vmstat.memory.free 1065500 -19.2% 861375 slabinfo.buffer_head.active_objs 27320 -19.1% 22093 slabinfo.buffer_head.active_slabs 1065521 -19.1% 861671 slabinfo.buffer_head.num_objs 27320 -19.1% 22093 slabinfo.buffer_head.num_slabs 1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.active_objs 1306 +18.4% 1546 ± 7% slabinfo.kmalloc-512.num_objs 28277 -9.3% 25651 slabinfo.radix_tree_node.active_objs 5388173 -15.3% 4561843 meminfo.Cached 989375 ± 2% -22.6% 765315 ± 3% meminfo.Dirty 4310025 -19.2% 3483318 meminfo.Inactive 4261041 -19.4% 3434223 meminfo.Inactive(file) 162676 -13.7% 140448 meminfo.KReclaimable 2151883 ± 2% +39.5% 3000927 ± 2% meminfo.MemFree 5902816 -14.4% 5053772 meminfo.Memused 162676 -13.7% 140448 meminfo.SReclaimable 197794 -11.3% 175489 meminfo.Slab 80314276 ± 5% +6.5% 85566891 ± 3% perf-stat.i.branch-instructions 4810221 ± 5% +7.5% 5172535 perf-stat.i.cache-references 7.022e+08 ± 3% +6.9% 7.509e+08 ± 2% perf-stat.i.cpu-cycles 3.891e+08 ± 5% +6.3% 4.135e+08 ± 3% perf-stat.i.instructions 195094 ± 2% +10.8% 216142 perf-stat.i.node-loads 79563934 ± 5% +6.5% 84757473 ± 3% perf-stat.ps.branch-instructions 4765201 ± 5% +7.5% 5123552 perf-stat.ps.cache-references 6.956e+08 ± 3% +6.9% 7.438e+08 ± 2% perf-stat.ps.cpu-cycles 3.855e+08 ± 5% +6.3% 4.096e+08 ± 3% perf-stat.ps.instructions 193276 ± 2% +10.8% 214098 perf-stat.ps.node-loads 4.114e+10 ± 5% +5.3% 4.333e+10 ± 3% perf-stat.total.instructions 247900 ± 3% -22.9% 191187 ± 3% proc-vmstat.nr_dirty 1349353 -15.5% 1140545 proc-vmstat.nr_file_pages 535810 ± 2% +40.1% 750444 ± 2% proc-vmstat.nr_free_pages 1067252 -19.6% 858356 proc-vmstat.nr_inactive_file 40723 -13.8% 35112 proc-vmstat.nr_slab_reclaimable 1110681 ± 2% +17.1% 1300126 proc-vmstat.nr_written 1067251 -19.6% 858384 proc-vmstat.nr_zone_inactive_file 249604 ± 3% -22.7% 193058 ± 3% proc-vmstat.nr_zone_write_pending 660225 -3.8% 635276 proc-vmstat.pgalloc_dma32 871298 +2.7% 894930 proc-vmstat.pgalloc_normal 3587465 ± 2% -20.2% 2863670 proc-vmstat.pgpgin 4645372 ± 4% +16.6% 5414779 proc-vmstat.pgpgout 13.16 ± 14% -6.6 6.58 ± 52% perf-profile.calltrace.cycles-pp.unlinkat 13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 13.06 ± 15% -6.6 6.49 ± 53% perf-profile.calltrace.cycles-pp.ext4_evict_inode.evict.do_unlinkat.do_syscall_64.entry_SYSCALL_64_after_hwframe 13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.unlinkat 13.06 ± 15% -6.6 6.50 ± 52% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.unlinkat 13.02 ± 14% -6.5 6.48 ± 52% perf-profile.calltrace.cycles-pp.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat.do_syscall_64 10.39 ± 36% -4.6 5.78 ± 8% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_kernel 11.21 ± 33% -4.6 6.64 ± 7% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_kernel.secondary_startup_64 11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_kernel.secondary_startup_64 11.21 ± 33% -4.6 6.65 ± 7% perf-profile.calltrace.cycles-pp.start_kernel.secondary_startup_64 5.71 ± 17% -2.8 2.95 ± 54% perf-profile.calltrace.cycles-pp.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat 3.68 ± 14% -2.0 1.69 ± 50% perf-profile.calltrace.cycles-pp.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict 3.70 ± 14% -2.0 1.74 ± 48% perf-profile.calltrace.cycles-pp.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat 2.90 ± 23% -1.6 1.32 ± 56% perf-profile.calltrace.cycles-pp.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode.evict 2.42 ± 20% -1.4 1.01 ± 75% perf-profile.calltrace.cycles-pp.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict.do_unlinkat 2.02 ± 17% -1.3 0.68 ±104% perf-profile.calltrace.cycles-pp.free_unref_page_list.release_pages.__pagevec_release.truncate_inode_pages_range.ext4_evict_inode 1.33 ± 28% -0.9 0.45 ±100% perf-profile.calltrace.cycles-pp.free_buffer_head.try_to_free_buffers.truncate_cleanup_page.truncate_inode_pages_range.ext4_evict_inode 1.21 ± 22% -0.8 0.40 ±106% perf-profile.calltrace.cycles-pp.xas_store.delete_from_page_cache_batch.truncate_inode_pages_range.ext4_evict_inode.evict 1.17 ± 3% +0.4 1.57 ± 13% perf-profile.calltrace.cycles-pp.get_next_timer_interrupt.tick_nohz_next_event.tick_nohz_get_sleep_length.menu_select.do_idle 0.35 ±141% +0.7 1.07 ± 37% perf-profile.calltrace.cycles-pp.scheduler_tick.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues 3.71 ± 34% +1.6 5.34 ± 14% perf-profile.calltrace.cycles-pp.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.cpuidle_enter 64.92 ± 5% +9.2 74.17 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 63.80 ± 4% +9.4 73.19 ± 4% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.cpuidle_enter.do_idle.cpu_startup_entry.start_secondary 69.95 ± 5% +10.4 80.37 ± 5% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 70.02 ± 5% +10.4 80.46 ± 5% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 13.16 ± 14% -6.6 6.58 ± 52% perf-profile.children.cycles-pp.unlinkat 13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.do_unlinkat 13.06 ± 15% -6.6 6.49 ± 53% perf-profile.children.cycles-pp.ext4_evict_inode 13.03 ± 14% -6.5 6.49 ± 53% perf-profile.children.cycles-pp.truncate_inode_pages_range 13.07 ± 15% -6.4 6.64 ± 53% perf-profile.children.cycles-pp.evict 11.21 ± 33% -4.6 6.65 ± 7% perf-profile.children.cycles-pp.start_kernel 5.71 ± 17% -2.7 2.99 ± 54% perf-profile.children.cycles-pp.truncate_cleanup_page 3.74 ± 13% -1.9 1.85 ± 49% perf-profile.children.cycles-pp.__pagevec_release 3.75 ± 12% -1.9 1.88 ± 49% perf-profile.children.cycles-pp.release_pages 2.91 ± 22% -1.5 1.40 ± 53% perf-profile.children.cycles-pp.try_to_free_buffers 2.43 ± 19% -1.3 1.12 ± 57% perf-profile.children.cycles-pp.delete_from_page_cache_batch 2.03 ± 17% -1.1 0.95 ± 58% perf-profile.children.cycles-pp.free_unref_page_list 1.34 ± 27% -0.7 0.65 ± 45% perf-profile.children.cycles-pp.free_buffer_head 1.23 ± 20% -0.6 0.64 ± 50% perf-profile.children.cycles-pp.xas_store 0.78 ± 15% -0.4 0.36 ± 52% perf-profile.children.cycles-pp.jbd2_journal_try_to_free_buffers 0.73 ± 16% -0.4 0.33 ± 52% perf-profile.children.cycles-pp.jbd2_journal_grab_journal_head 0.50 ± 39% -0.4 0.15 ± 53% perf-profile.children.cycles-pp.unaccount_page_cache_page 0.42 ± 24% -0.3 0.14 ± 93% perf-profile.children.cycles-pp.mem_cgroup_uncharge_list 0.76 ± 19% -0.2 0.53 ± 6% perf-profile.children.cycles-pp.lapic_next_deadline 0.32 ± 22% -0.2 0.14 ± 69% perf-profile.children.cycles-pp.free_unref_page_commit 0.25 ± 25% -0.2 0.08 ±102% perf-profile.children.cycles-pp.uncharge_page 0.19 ± 47% -0.1 0.07 ± 68% perf-profile.children.cycles-pp.__might_sleep 0.16 ± 28% -0.1 0.05 ±112% perf-profile.children.cycles-pp.__libc_fork 0.15 ± 22% -0.1 0.04 ±102% perf-profile.children.cycles-pp.xas_find 0.13 ± 12% -0.1 0.08 ± 34% perf-profile.children.cycles-pp.leave_mm 0.04 ± 71% +0.0 0.07 ± 14% perf-profile.children.cycles-pp.irq_work_needs_cpu 0.02 ±141% +0.1 0.09 ± 29% perf-profile.children.cycles-pp.__msecs_to_jiffies 0.04 ± 71% +0.1 0.12 ± 10% perf-profile.children.cycles-pp.cpuidle_not_available 0.09 ± 75% +0.1 0.18 ± 23% perf-profile.children.cycles-pp.__schedule 0.00 +0.1 0.10 ± 40% perf-profile.children.cycles-pp.link_path_walk 0.09 ± 41% +0.1 0.20 ± 19% perf-profile.children.cycles-pp.pm_qos_read_value 0.04 ± 71% +0.1 0.14 ± 52% perf-profile.children.cycles-pp.raise_softirq 0.04 ±141% +0.1 0.15 ± 33% perf-profile.children.cycles-pp.__test_set_page_writeback 0.00 +0.1 0.12 ± 17% perf-profile.children.cycles-pp.__delete_from_page_cache 0.05 ±141% +0.1 0.17 ± 60% perf-profile.children.cycles-pp.exit_mmap 0.19 ± 54% +0.1 0.32 ± 21% perf-profile.children.cycles-pp.hrtimer_get_next_event 0.05 ±141% +0.1 0.18 ± 64% perf-profile.children.cycles-pp.mmput 0.08 ± 30% +0.2 0.24 ± 33% perf-profile.children.cycles-pp.run_local_timers 0.05 ±141% +0.2 0.21 ± 41% perf-profile.children.cycles-pp.ext4_bio_write_page 0.14 ± 32% +0.2 0.30 ± 46% perf-profile.children.cycles-pp.update_rq_clock 0.15 ± 71% +0.2 0.33 ± 17% perf-profile.children.cycles-pp.rcu_idle_exit 0.38 ± 29% +0.2 0.56 ± 16% perf-profile.children.cycles-pp.sched_clock_cpu 0.14 ± 78% +0.2 0.33 ± 41% perf-profile.children.cycles-pp.mpage_map_and_submit_buffers 0.00 +0.2 0.20 ± 52% perf-profile.children.cycles-pp.invalidate_inode_pages2_range 0.07 ± 97% +0.2 0.30 ± 58% perf-profile.children.cycles-pp.__x64_sys_exit_group 0.07 ±141% +0.2 0.31 ± 37% perf-profile.children.cycles-pp.mpage_submit_page 0.38 ± 21% +0.3 0.64 ± 14% perf-profile.children.cycles-pp.cpuidle_governor_latency_req 0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_group_exit 0.08 ±101% +0.3 0.34 ± 51% perf-profile.children.cycles-pp.do_exit 0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.do_writepages 0.23 ± 29% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.ext4_writepages 0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_workfn 0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.wb_writeback 0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_inodes_wb 0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.writeback_sb_inodes 0.22 ± 35% +0.3 0.50 ± 34% perf-profile.children.cycles-pp.__writeback_single_inode 0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.ext4_file_read_iter 0.00 +0.3 0.32 ± 49% perf-profile.children.cycles-pp.iomap_dio_rw 1.30 ± 4% +0.4 1.72 ± 14% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.74 ± 41% +0.5 1.25 ± 33% perf-profile.children.cycles-pp.scheduler_tick 75.40 +4.7 80.05 ± 4% perf-profile.children.cycles-pp.cpuidle_enter 75.35 +4.7 80.02 ± 4% perf-profile.children.cycles-pp.cpuidle_enter_state 81.28 +5.9 87.18 ± 4% perf-profile.children.cycles-pp.do_idle 70.02 ± 5% +10.4 80.46 ± 5% perf-profile.children.cycles-pp.start_secondary 1.16 ± 22% -0.6 0.56 ± 63% perf-profile.self.cycles-pp.free_pcppages_bulk 0.73 ± 16% -0.4 0.33 ± 52% perf-profile.self.cycles-pp.jbd2_journal_grab_journal_head 0.46 ± 21% -0.3 0.16 ± 36% perf-profile.self.cycles-pp.xas_store 0.70 ± 5% -0.3 0.45 ± 38% perf-profile.self.cycles-pp.release_pages 0.29 ± 17% -0.2 0.07 ± 66% perf-profile.self.cycles-pp.free_unref_page_commit 0.32 ± 26% -0.2 0.13 ± 86% perf-profile.self.cycles-pp.workingset_update_node 0.18 ± 14% -0.1 0.06 ±100% perf-profile.self.cycles-pp.tick_nohz_tick_stopped 0.11 -0.1 0.05 ± 60% perf-profile.self.cycles-pp.leave_mm 0.04 ± 71% +0.0 0.07 ± 14% perf-profile.self.cycles-pp.irq_work_needs_cpu 0.00 +0.1 0.07 ± 14% perf-profile.self.cycles-pp.__msecs_to_jiffies 0.04 ± 71% +0.1 0.11 ± 26% perf-profile.self.cycles-pp.cpuidle_not_available 0.08 ± 81% +0.1 0.18 ± 23% perf-profile.self.cycles-pp.rcu_idle_exit 0.09 ± 41% +0.1 0.20 ± 19% perf-profile.self.cycles-pp.pm_qos_read_value 0.04 ± 71% +0.1 0.14 ± 52% perf-profile.self.cycles-pp.raise_softirq 0.04 ±141% +0.1 0.17 ± 41% perf-profile.self.cycles-pp.calc_global_load_tick 0.16 ± 12% +0.2 0.32 ± 27% perf-profile.self.cycles-pp.run_timer_softirq 0.08 ± 81% +0.2 0.26 ± 46% perf-profile.self.cycles-pp.update_rq_clock 0.17 ± 49% +0.2 0.36 ± 22% perf-profile.self.cycles-pp.load_balance 0.26 ± 8% +0.2 0.47 ± 40% perf-profile.self.cycles-pp.hrtimer_interrupt filebench.time.file_system_inputs 7.6e+06 +-+---------------------------------------------------------------+ 7.4e+06 +-++ .+ +.+ | |: ++ + +.+.+.+.++. .+ : : +.+.+. +.| 7.2e+06 +-+ : : + + .++ + : : .+ + + + | 7e+06 +-+ : : + + + +: +.+ + :+ + | 6.8e+06 +-+ + + + + +.+ : | 6.6e+06 +-+ +: | | + | 6.4e+06 +-+ | 6.2e+06 +-+ | 6e+06 +-+ | 5.8e+06 O-+ O O O O | | O OO O O O O O OO O O O OO O O OO O O O O | 5.6e+06 +-+ O O | 5.4e+06 +-+---------------------------------------------------------------+ filebench.sum_operations 3700 +-+------------------------------------------------------------------+ 3600 +-++ .+. +.+ | |: +. + +.+.+.+.++. .+ : : +.+.+. +.| 3500 +-+ + : + + .+.+ + : : : + + : | 3400 +-+ :: + + :+ : :.+.+ + + + : | | + + + + + +.+ + | 3300 +-+ :+ | 3200 +-+ + | 3100 +-+ | | | 3000 +-+ | 2900 +-+ | O O O O O O O O | 2800 +-O O OO O O O O O O O O O O O OO O O O O | 2700 +-+---------------------------------------------O--------------------+ filebench.sum_operations_s 62 +-+--------------------------------------------------------------------+ | + .+ .+ | 60 +-++ + +.+.+.+.+. .+ + : +. .+. +.| 58 +-+ +.+ : +.+ + .+.+ + : : + + + + : | |: + : + + :+ : :.+.+ + :+ : | 56 +-+ + + + + + +.+ + | 54 +-+ + + | | + | 52 +-+ | 50 +-+ | | | 48 +-+ O | 46 O-O O O O O O O O O O O O O O O OO O O O O O O O O O | | O O | 44 +-+--------------------------------------------------------------------+ filebench.sum_reads_s 62 +-+--------------------------------------------------------------------+ | + | 60 +-++ .++. .+. +.+ +. .+. .| 58 +-+ +.+ + +.+ +.+.+.+. .+.+ + : : + + + + + | |: + + + + :+ : : .+ + :+ + | 56 +-+ + + + +.+ + +.+ + | | : : | 54 +-+ : : | | + | 52 +-+ | 50 +-+ | | | 48 +-+ | O O O O O O O O O O | 46 +-O-O-O-O---OO-O-----O---O-O-O-O-O-OO-O-O-O-O-----O--------------------+ filebench.sum_bytes_mb_s 62 +-+--------------------------------------------------------------------+ | + | 60 +-++ +.++.+.+.+.+. +.+ +. .+ +.| 58 +-+ +.+ : +.+.+. .+. : : + + + : | |: + : + +. +. : : .+.+ +. +. : | 56 +-+ + + + + + +.+ + | 54 +-+ + + | | + | 52 +-+ | 50 +-+ | | | 48 +-+ O | 46 O-O O O O O O O O O O O O O O O O O O O O O O O O | | O O O O | 44 +-+--------------------------------------------------------------------+ filebench.sum_time_ms_op 110 +-+-O------O----------------------O--------------O--------------------+ | O O OO O O O O O OO O O O O O O O OO O O O O | 105 O-+ O | 100 +-+ | | | 95 +-+ | | .+. | 90 +-+ + .+. .+ + .+. .++ + | |: : : + .+.+.+ + + : +.+. + + + | 85 +-+ +. : +. +.+ :+ +. : +.+. + +.| 80 +-++ + +. .+. + : : + + | | + + + : : | 75 +-+ : : | | : | 70 +-+-------------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

9 months, 1 week

5
10
0 / 0

[f2fs] fe1897eaa6: aim7.jobs-per-min -60.9% regression

by kernel test robot

Greeting, FYI, we noticed a -60.9% regression of aim7.jobs-per-min due to commit: commit: fe1897eaa6646f5a64a4cee0e6473ed9887d324b ("f2fs: fix to update time in lazytime mode") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: aim7 on test machine: 48 threads Intel(R) Xeon(R) CPU E5-2697 v2 @ 2.70GHz with 64G memory with following parameters: disk: 4BRD_12G md: RAID1 fs: f2fs test: sync_disk_rw load: 200 cpufreq_governor: performance ucode: 0x42e test-description: AIM7 is a traditional UNIX system level benchmark suite which is used to test and measure the performance of multiuser system. test-url: https://sourceforge.net/projects/aimbench/files/aim-suite7/ If you fix the issue, kindly add following tag Reported-by: kernel test robot <rong.a.chen(a)intel.com> Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= compiler/cpufreq_governor/disk/fs/kconfig/load/md/rootfs/tbox_group/test/testcase/ucode: gcc-7/performance/4BRD_12G/f2fs/x86_64-rhel-7.6/200/RAID1/debian-x86_64-2019-11-14.cgz/lkp-ivb-2ep1/sync_disk_rw/aim7/0x42e commit: b145b0eb20 (" ARM and x86 bugfixes of all kinds. The most visible one is that migrating") fe1897eaa6 ("f2fs: fix to update time in lazytime mode") b145b0eb2031a620 fe1897eaa6646f5a64a4cee0e64 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 1208 -60.9% 473.12 aim7.jobs-per-min 992.70 +155.6% 2537 aim7.time.elapsed_time 992.70 +155.6% 2537 aim7.time.elapsed_time.max 1.078e+08 +60.0% 1.724e+08 aim7.time.file_system_outputs 9646301 -4.2% 9238717 aim7.time.involuntary_context_switches 24107 ± 3% +58.2% 38130 ± 17% aim7.time.minor_page_faults 3295 +202.3% 9963 aim7.time.system_time 26.75 +51.4% 40.49 aim7.time.user_time 5.02e+08 +210.5% 1.559e+09 aim7.time.voluntary_context_switches 9294452 ± 4% -15.7% 7835497 ± 16% numa-numastat.node0.local_node 9298417 ± 4% -15.7% 7840991 ± 16% numa-numastat.node0.numa_hit 3.66 ± 3% +4.2 7.82 ± 18% mpstat.cpu.all.idle% 0.01 ± 32% -0.0 0.00 ± 31% mpstat.cpu.all.soft% 0.06 -0.0 0.04 mpstat.cpu.all.usr% 3.85 ± 3% +105.0% 7.89 ± 17% iostat.cpu.idle 89.80 -4.9% 85.36 iostat.cpu.iowait 6.29 +6.6% 6.70 iostat.cpu.system 23968 -48.4% 12356 iostat.md0.w/s 53780 -37.1% 33824 iostat.md0.wkB/s 89.00 -4.8% 84.75 vmstat.cpu.wa 53779 -37.1% 33833 vmstat.io.bo 29311046 -28.6% 20935254 vmstat.memory.free 1026388 +19.4% 1225080 vmstat.system.cs 103289 +8.5% 112038 vmstat.system.in 24013 ± 29% -58.5% 9976 ±120% numa-meminfo.node0.Dirty 119019 ± 10% +44.6% 172139 ± 13% numa-meminfo.node1.Active 3503 ± 56% +464.8% 19786 ± 38% numa-meminfo.node1.Active(file) 40477 ± 16% -38.5% 24896 ± 47% numa-meminfo.node1.Dirty 19966008 ± 2% -29.5% 14084594 ± 44% numa-meminfo.node1.MemFree 13014643 ± 4% +45.2% 18896052 ± 32% numa-meminfo.node1.MemUsed 12018 ± 41% +184.8% 34228 ± 35% numa-meminfo.node1.Shmem 2.09e+10 +127.4% 4.752e+10 ± 4% cpuidle.C1.time 2.558e+08 +140.1% 6.143e+08 ± 4% cpuidle.C1.usage 3.017e+09 ± 10% +301.2% 1.211e+10 ± 9% cpuidle.C1E.time 18225961 ± 9% +430.1% 96624314 ± 6% cpuidle.C1E.usage 4.627e+08 ± 14% +147.5% 1.145e+09 ± 20% cpuidle.C6.time 802166 ± 17% +186.9% 2301023 ± 26% cpuidle.C6.usage 1.799e+10 +148.9% 4.476e+10 cpuidle.POLL.time 2.616e+08 ± 2% +246.4% 9.063e+08 ± 2% cpuidle.POLL.usage 6017 ± 29% -58.5% 2496 ±120% numa-vmstat.node0.nr_dirty 5742 ± 29% -65.0% 2011 ±120% numa-vmstat.node0.nr_zone_write_pending 874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_active_file 10139 ± 16% -38.6% 6224 ± 47% numa-vmstat.node1.nr_dirty 4992144 ± 2% -29.5% 3521200 ± 44% numa-vmstat.node1.nr_free_pages 3003 ± 41% +184.9% 8556 ± 35% numa-vmstat.node1.nr_shmem 874.50 ± 56% +465.6% 4946 ± 38% numa-vmstat.node1.nr_zone_active_file 9775 ± 16% -47.9% 5089 ± 47% numa-vmstat.node1.nr_zone_write_pending 2.558e+08 +140.1% 6.143e+08 ± 4% turbostat.C1 43.77 -4.8 38.98 ± 3% turbostat.C1% 18225642 ± 9% +430.2% 96624037 ± 6% turbostat.C1E 6.31 ± 10% +3.6 9.94 ± 9% turbostat.C1E% 795117 ± 17% +188.4% 2292823 ± 26% turbostat.C6 0.38 ± 42% -63.6% 0.14 ± 13% turbostat.CPU%c6 1.028e+08 +176.7% 2.844e+08 turbostat.IRQ 0.23 ± 33% -69.1% 0.07 ± 22% turbostat.Pkg%pc2 275064 +12.3% 308846 ± 4% meminfo.Active 8036 ± 21% +261.7% 29072 ± 52% meminfo.Active(file) 64400 -45.9% 34850 meminfo.Dirty 16481 +10.8% 18264 meminfo.Inactive(anon) 147067 +12.8% 165858 ± 2% meminfo.KReclaimable 29120417 -28.8% 20740075 meminfo.MemAvailable 29335165 -28.6% 20936804 meminfo.MemFree 36511133 +23.0% 44909491 meminfo.Memused 147067 +12.8% 165858 ± 2% meminfo.SReclaimable 36567 ± 6% +37.2% 50173 meminfo.Shmem 51580 -60.8% 20220 meminfo.max_used_kB 4.27 ± 7% -13.9% 3.67 ± 4% perf-stat.i.MPKI 0.71 ± 3% -0.0 0.67 perf-stat.i.branch-miss-rate% 17.93 +3.5 21.41 ± 2% perf-stat.i.cache-miss-rate% 1029337 +19.1% 1226025 perf-stat.i.context-switches 11421 ± 4% +41.1% 16120 ± 6% perf-stat.i.cpu-migrations 0.42 ± 2% +0.0 0.45 ± 6% perf-stat.i.dTLB-load-miss-rate% 0.11 ± 27% -0.0 0.08 ± 3% perf-stat.i.dTLB-store-miss-rate% 84.04 ± 2% -2.3 81.75 ± 2% perf-stat.i.iTLB-load-miss-rate% 7375571 +7.3% 7917282 perf-stat.i.iTLB-load-misses 3.91 ± 7% -9.2% 3.56 ± 4% perf-stat.overall.MPKI 17.90 +4.1 21.96 ± 3% perf-stat.overall.cache-miss-rate% 0.41 ± 2% +0.0 0.45 ± 6% perf-stat.overall.dTLB-load-miss-rate% 0.10 ± 27% -0.0 0.08 ± 3% perf-stat.overall.dTLB-store-miss-rate% 83.99 ± 2% -2.3 81.72 ± 2% perf-stat.overall.iTLB-load-miss-rate% 1028292 +19.2% 1225532 perf-stat.ps.context-switches 11410 ± 4% +41.2% 16114 ± 6% perf-stat.ps.cpu-migrations 7368087 +7.4% 7914119 perf-stat.ps.iTLB-load-misses 2.558e+13 ± 2% +157.0% 6.575e+13 ± 3% perf-stat.total.instructions 9128 ± 2% -25.3% 6821 ± 4% slabinfo.dmaengine-unmap-16.active_objs 9175 ± 2% -24.4% 6938 ± 4% slabinfo.dmaengine-unmap-16.num_objs 28309 +12.5% 31840 slabinfo.f2fs_extent_node.active_objs 28704 +13.0% 32429 slabinfo.f2fs_extent_node.num_objs 964.00 ± 5% -13.3% 836.00 ± 6% slabinfo.kmem_cache_node.active_objs 1008 ± 5% -12.7% 880.00 ± 6% slabinfo.kmem_cache_node.num_objs 13017 -33.1% 8707 slabinfo.nat_entry.active_objs 13036 -33.2% 8707 slabinfo.nat_entry.num_objs 433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.active_objs 433.75 ± 9% +32.4% 574.50 ± 2% slabinfo.nfs_read_data.num_objs 697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.active_objs 697.50 ± 3% +48.6% 1036 ± 33% slabinfo.numa_policy.num_objs 153098 +22.3% 187296 slabinfo.radix_tree_node.active_objs 2733 +22.3% 3344 slabinfo.radix_tree_node.active_slabs 153098 +22.3% 187296 slabinfo.radix_tree_node.num_objs 2733 +22.3% 3344 slabinfo.radix_tree_node.num_slabs 542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.active_objs 542.00 ± 10% +29.5% 702.00 ± 6% slabinfo.xfrm_state.num_objs 66757 +4.8% 69942 proc-vmstat.nr_active_anon 2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_active_file 96.00 +3.9% 99.75 proc-vmstat.nr_anon_transparent_hugepages 13466177 +60.0% 21542226 proc-vmstat.nr_dirtied 16154 -46.0% 8719 proc-vmstat.nr_dirty 726643 -28.8% 517541 proc-vmstat.nr_dirty_background_threshold 1455064 -28.8% 1036348 proc-vmstat.nr_dirty_threshold 311402 +2.4% 318957 proc-vmstat.nr_file_pages 7332853 -28.6% 5234441 proc-vmstat.nr_free_pages 4120 +10.8% 4565 proc-vmstat.nr_inactive_anon 13939 -1.2% 13772 proc-vmstat.nr_kernel_stack 2845 -1.5% 2803 proc-vmstat.nr_page_table_pages 9142 ± 6% +37.2% 12542 proc-vmstat.nr_shmem 36767 +12.8% 41463 ± 2% proc-vmstat.nr_slab_reclaimable 13387539 +60.3% 21463781 proc-vmstat.nr_written 66757 +4.8% 69942 proc-vmstat.nr_zone_active_anon 2009 ± 21% +261.7% 7267 ± 52% proc-vmstat.nr_zone_active_file 4120 +10.8% 4565 proc-vmstat.nr_zone_inactive_anon 15494 -54.2% 7089 proc-vmstat.nr_zone_write_pending 582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_foreign 13506 ± 3% +114.5% 28974 ± 31% proc-vmstat.numa_hint_faults 8913 ± 6% +148.6% 22154 ± 35% proc-vmstat.numa_hint_faults_local 582146 ±102% +628.1% 4238850 ± 6% proc-vmstat.numa_miss 597885 ± 99% +611.6% 4254585 ± 6% proc-vmstat.numa_other 1014174 ± 20% -46.1% 547135 ± 57% proc-vmstat.pgalloc_dma32 16157005 +22.4% 19769138 proc-vmstat.pgalloc_normal 2516271 +152.8% 6360808 proc-vmstat.pgfault 4866589 +64.4% 8002519 proc-vmstat.pgfree 53553588 +60.3% 85856803 proc-vmstat.pgpgout 1.55 ± 4% +0.1 1.65 perf-profile.calltrace.cycles-pp.__schedule.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback 1.58 ± 4% +0.1 1.69 perf-profile.calltrace.cycles-pp.schedule.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback 1.60 ± 4% +0.1 1.71 perf-profile.calltrace.cycles-pp.io_schedule.wait_on_page_bit.f2fs_wait_on_page_writeback.f2fs_wait_on_node_pages_writeback.f2fs_do_sync_file 0.75 ± 2% +0.2 0.91 ± 2% perf-profile.calltrace.cycles-pp.menu_select.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.70 ± 4% -0.3 0.36 ± 7% perf-profile.children.cycles-pp.__generic_file_write_iter 0.69 ± 4% -0.3 0.35 ± 7% perf-profile.children.cycles-pp.generic_perform_write 0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 0.58 ± 5% -0.3 0.26 ± 7% perf-profile.children.cycles-pp.file_write_and_wait_range 0.57 ± 6% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.do_writepages 0.57 ± 5% -0.3 0.25 ± 6% perf-profile.children.cycles-pp.f2fs_write_data_pages 0.64 ± 5% -0.3 0.34 ± 6% perf-profile.children.cycles-pp.f2fs_write_begin 0.57 ± 5% -0.2 0.32 ± 7% perf-profile.children.cycles-pp.f2fs_put_page 0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_finish_plug 0.31 ± 10% -0.2 0.11 ± 7% perf-profile.children.cycles-pp.blk_flush_plug_list 0.30 ± 11% -0.2 0.11 ± 6% perf-profile.children.cycles-pp.raid1_unplug 0.26 -0.1 0.14 ± 12% perf-profile.children.cycles-pp.f2fs_write_cache_pages 0.22 ± 20% -0.1 0.11 ± 41% perf-profile.children.cycles-pp.brd_do_bvec 0.14 ± 12% -0.1 0.03 ±100% perf-profile.children.cycles-pp.submit_bio_wait 0.21 ± 10% -0.1 0.11 ± 10% perf-profile.children.cycles-pp.submit_bio 0.23 ± 17% -0.1 0.13 ± 31% perf-profile.children.cycles-pp.worker_thread 0.20 ± 3% -0.1 0.11 ± 14% perf-profile.children.cycles-pp.__write_data_page 0.14 ± 15% -0.1 0.05 ± 9% perf-profile.children.cycles-pp.__submit_flush_wait 0.17 ± 12% -0.1 0.09 ± 12% perf-profile.children.cycles-pp.md_make_request 0.16 ± 11% -0.1 0.08 ± 14% perf-profile.children.cycles-pp.md_handle_request 0.15 ± 9% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.raid1_make_request 0.14 ± 17% -0.1 0.06 ± 11% perf-profile.children.cycles-pp.f2fs_issue_flush 0.14 ± 6% -0.1 0.08 ± 19% perf-profile.children.cycles-pp.f2fs_do_write_data_page 0.10 ± 33% -0.0 0.06 ± 11% perf-profile.children.cycles-pp.reschedule_interrupt 0.09 ± 37% -0.0 0.05 ± 8% perf-profile.children.cycles-pp.scheduler_ipi 0.06 ± 6% +0.0 0.08 ± 10% perf-profile.children.cycles-pp.up_read 0.10 ± 4% +0.0 0.12 ± 13% perf-profile.children.cycles-pp.pick_next_task_idle 0.20 ± 2% +0.0 0.23 ± 5% perf-profile.children.cycles-pp.get_next_timer_interrupt 0.16 ± 6% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.irq_exit 0.01 ±173% +0.1 0.07 ± 7% perf-profile.children.cycles-pp.put_prev_task_fair 0.27 ± 3% +0.1 0.33 ± 4% perf-profile.children.cycles-pp.tick_nohz_next_event 0.35 ± 3% +0.1 0.42 ± 3% perf-profile.children.cycles-pp.tick_nohz_get_sleep_length 1.60 ± 4% +0.1 1.72 perf-profile.children.cycles-pp.io_schedule 0.78 ± 2% +0.1 0.92 ± 2% perf-profile.children.cycles-pp.menu_select 0.06 ± 6% +0.0 0.08 ± 10% perf-profile.self.cycles-pp.up_read 0.08 ± 5% +0.0 0.11 ± 11% perf-profile.self.cycles-pp.cpuidle_enter_state 0.35 ± 5% +0.0 0.39 ± 5% perf-profile.self.cycles-pp.f2fs_wait_on_node_pages_writeback 0.35 ± 4% +0.1 0.41 ± 3% perf-profile.self.cycles-pp.menu_select 8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.MIN_vruntime.avg 158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.MIN_vruntime.max 32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.MIN_vruntime.stddev 44053 +192.3% 128774 sched_debug.cfs_rq:/.exec_clock.avg 140530 ± 9% +377.8% 671471 ± 29% sched_debug.cfs_rq:/.exec_clock.max 33108 ± 2% +109.9% 69484 ± 2% sched_debug.cfs_rq:/.exec_clock.min 20291 ± 7% +386.1% 98642 ± 21% sched_debug.cfs_rq:/.exec_clock.stddev 0.25 ± 82% -88.4% 0.03 ±131% sched_debug.cfs_rq:/.load_avg.min 8096 ± 28% +278.9% 30679 ± 20% sched_debug.cfs_rq:/.max_vruntime.avg 158062 ± 17% +299.5% 631442 ± 8% sched_debug.cfs_rq:/.max_vruntime.max 32629 ± 22% +289.7% 127148 ± 13% sched_debug.cfs_rq:/.max_vruntime.stddev 184170 +218.6% 586859 ± 2% sched_debug.cfs_rq:/.min_vruntime.avg 251516 ± 4% +254.5% 891702 ± 11% sched_debug.cfs_rq:/.min_vruntime.max 165614 +131.3% 383025 ± 2% sched_debug.cfs_rq:/.min_vruntime.min 15286 ± 9% +1057.3% 176903 ± 10% sched_debug.cfs_rq:/.min_vruntime.stddev 0.67 ± 6% +45.5% 0.98 ± 9% sched_debug.cfs_rq:/.nr_spread_over.avg 0.74 ± 20% +32.4% 0.98 ± 13% sched_debug.cfs_rq:/.nr_spread_over.stddev 3.43 ± 29% -60.7% 1.35 ±100% sched_debug.cfs_rq:/.removed.load_avg.avg 75.10 ± 34% -84.1% 11.91 ±100% sched_debug.cfs_rq:/.removed.load_avg.max 15.24 ± 23% -75.5% 3.74 ±100% sched_debug.cfs_rq:/.removed.load_avg.stddev 157.58 ± 29% -60.5% 62.19 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.avg 3459 ± 34% -84.0% 552.75 ± 99% sched_debug.cfs_rq:/.removed.runnable_sum.max 700.79 ± 23% -75.4% 172.66 ±100% sched_debug.cfs_rq:/.removed.runnable_sum.stddev 34.81 ± 42% -87.0% 4.53 ±100% sched_debug.cfs_rq:/.removed.util_avg.max 5.89 ± 31% -79.0% 1.23 ±100% sched_debug.cfs_rq:/.removed.util_avg.stddev 59642 ± 58% +506.4% 361657 ± 61% sched_debug.cfs_rq:/.spread0.max 15286 ± 9% +1057.2% 176904 ± 10% sched_debug.cfs_rq:/.spread0.stddev 15.81 ± 15% +28.7% 20.34 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.avg 553.04 ± 15% +42.8% 789.60 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.max 83.37 ± 15% +37.7% 114.78 ± 2% sched_debug.cfs_rq:/.util_est_enqueued.stddev 130721 -17.9% 107295 sched_debug.cpu.avg_idle.avg 32650 ± 7% -16.9% 27145 ± 6% sched_debug.cpu.avg_idle.min 509212 +151.7% 1281639 sched_debug.cpu.clock.avg 509215 +151.7% 1281642 sched_debug.cpu.clock.max 509209 +151.7% 1281636 sched_debug.cpu.clock.min 509212 +151.7% 1281639 sched_debug.cpu.clock_task.avg 509215 +151.7% 1281642 sched_debug.cpu.clock_task.max 509209 +151.7% 1281636 sched_debug.cpu.clock_task.min 480.69 ± 2% +89.2% 909.42 ± 3% sched_debug.cpu.curr->pid.avg 14037 +135.4% 33040 sched_debug.cpu.curr->pid.max 2146 ± 2% +125.6% 4843 sched_debug.cpu.curr->pid.stddev 0.12 ± 5% +10.2% 0.13 ± 3% sched_debug.cpu.nr_running.avg 10174129 +208.2% 31360192 sched_debug.cpu.nr_switches.avg 10849045 +290.1% 42321541 ± 3% sched_debug.cpu.nr_switches.max 564008 ± 15% +1719.4% 10261545 ± 8% sched_debug.cpu.nr_switches.stddev -87.16 +262.4% -315.83 sched_debug.cpu.nr_uninterruptible.min 66.84 ± 14% +143.1% 162.48 ± 21% sched_debug.cpu.nr_uninterruptible.stddev 10172460 +208.3% 31358619 sched_debug.cpu.sched_count.avg 10847113 +290.1% 42319816 ± 3% sched_debug.cpu.sched_count.max 563945 ± 15% +1719.6% 10261603 ± 8% sched_debug.cpu.sched_count.stddev 4957584 +211.1% 15425344 sched_debug.cpu.sched_goidle.avg 5286410 +291.7% 20709361 ± 3% sched_debug.cpu.sched_goidle.max 272603 ± 14% +1717.5% 4954528 ± 7% sched_debug.cpu.sched_goidle.stddev 5120024 +209.4% 15841794 sched_debug.cpu.ttwu_count.avg 58965381 ± 14% +567.2% 3.934e+08 ± 33% sched_debug.cpu.ttwu_count.max 132694 ± 2% -41.0% 78274 ± 5% sched_debug.cpu.ttwu_count.min 11266172 ± 8% +482.6% 65635267 ± 21% sched_debug.cpu.ttwu_count.stddev 114955 +26.1% 144930 ± 3% sched_debug.cpu.ttwu_local.max 83595 ± 4% -53.0% 39284 ± 30% sched_debug.cpu.ttwu_local.min 5967 ± 16% +460.9% 33468 ± 9% sched_debug.cpu.ttwu_local.stddev 509209 +151.7% 1281636 sched_debug.cpu_clk 506629 +152.5% 1279056 sched_debug.ktime 509627 +151.6% 1282053 sched_debug.sched_clk 18876 +138.7% 45051 ± 4% softirqs.CPU0.RCU 131251 ± 5% +153.2% 332324 ± 8% softirqs.CPU0.SCHED 327888 ± 3% +181.5% 923135 ± 10% softirqs.CPU0.TIMER 19003 ± 4% +121.7% 42132 ± 12% softirqs.CPU1.RCU 125587 ± 4% +130.3% 289267 ± 22% softirqs.CPU1.SCHED 331057 ± 2% +205.9% 1012636 ± 18% softirqs.CPU1.TIMER 19567 ± 5% +137.8% 46538 ± 2% softirqs.CPU10.RCU 131681 +161.3% 344047 softirqs.CPU10.SCHED 318774 +177.5% 884714 ± 8% softirqs.CPU10.TIMER 17876 ± 5% +158.8% 46258 softirqs.CPU11.RCU 128299 ± 4% +167.5% 343196 softirqs.CPU11.SCHED 320847 ± 3% +166.4% 854607 ± 8% softirqs.CPU11.TIMER 19129 ± 4% +121.1% 42292 ± 4% softirqs.CPU12.RCU 135251 +172.0% 367942 softirqs.CPU12.SCHED 310663 +201.8% 937514 ± 24% softirqs.CPU12.TIMER 17993 +143.9% 43882 ± 3% softirqs.CPU13.RCU 137747 ± 4% +157.5% 354715 ± 4% softirqs.CPU13.SCHED 359507 ± 23% +126.2% 813048 ± 13% softirqs.CPU13.TIMER 18173 +141.9% 43961 ± 3% softirqs.CPU14.RCU 134564 +163.8% 355019 ± 4% softirqs.CPU14.SCHED 310599 ± 2% +160.3% 808494 ± 13% softirqs.CPU14.TIMER 18180 +139.4% 43524 ± 3% softirqs.CPU15.RCU 134993 +160.8% 352061 ± 6% softirqs.CPU15.SCHED 308373 +164.7% 816152 ± 12% softirqs.CPU15.TIMER 19918 +144.6% 48716 softirqs.CPU16.RCU 134609 +162.6% 353418 ± 4% softirqs.CPU16.SCHED 307916 +163.0% 809772 ± 13% softirqs.CPU16.TIMER 19672 ± 2% +148.8% 48938 ± 3% softirqs.CPU17.RCU 132926 ± 2% +165.9% 353452 ± 4% softirqs.CPU17.SCHED 314316 +157.0% 807880 ± 13% softirqs.CPU17.TIMER 20021 ± 3% +141.2% 48298 ± 2% softirqs.CPU18.RCU 133347 ± 2% +164.0% 352038 ± 5% softirqs.CPU18.SCHED 312605 +158.6% 808358 ± 13% softirqs.CPU18.TIMER 19458 +139.3% 46570 ± 6% softirqs.CPU19.RCU 134001 +150.1% 335170 ± 14% softirqs.CPU19.SCHED 310521 +171.0% 841639 ± 14% softirqs.CPU19.TIMER 19131 ± 3% +138.3% 45586 ± 3% softirqs.CPU2.RCU 129111 ± 4% +154.4% 328419 ± 9% softirqs.CPU2.SCHED 329064 ± 3% +172.7% 897239 ± 10% softirqs.CPU2.TIMER 19249 ± 2% +145.5% 47257 ± 3% softirqs.CPU20.RCU 132532 +167.8% 354983 ± 3% softirqs.CPU20.SCHED 310659 +163.4% 818268 ± 13% softirqs.CPU20.TIMER 19473 +142.2% 47163 ± 4% softirqs.CPU21.RCU 133856 +166.0% 356005 ± 4% softirqs.CPU21.SCHED 308960 +164.3% 816637 ± 13% softirqs.CPU21.TIMER 19216 ± 2% +146.1% 47292 ± 3% softirqs.CPU22.RCU 133039 +165.4% 353140 ± 4% softirqs.CPU22.SCHED 307621 +166.0% 818260 ± 13% softirqs.CPU22.TIMER 19244 ± 2% +145.9% 47314 ± 2% softirqs.CPU23.RCU 133897 +163.1% 352326 ± 5% softirqs.CPU23.SCHED 310036 ± 2% +161.6% 811061 ± 13% softirqs.CPU23.TIMER 18954 ± 4% +143.9% 46235 ± 4% softirqs.CPU24.RCU 128754 ± 3% +157.8% 331927 ± 7% softirqs.CPU24.SCHED 323118 +169.7% 871609 ± 8% softirqs.CPU24.TIMER 18942 ± 6% +136.0% 44695 ± 3% softirqs.CPU25.RCU 129580 ± 3% +162.2% 339786 ± 3% softirqs.CPU25.SCHED 328907 +168.0% 881566 ± 7% softirqs.CPU25.TIMER 18913 ± 2% +147.2% 46757 ± 2% softirqs.CPU26.RCU 124714 ± 2% +176.4% 344692 softirqs.CPU26.SCHED 328463 ± 4% +170.2% 887463 ± 8% softirqs.CPU26.TIMER 19015 +137.2% 45107 ± 5% softirqs.CPU27.RCU 128551 ± 6% +152.0% 323916 ± 13% softirqs.CPU27.SCHED 325384 ± 2% +193.3% 954266 ± 13% softirqs.CPU27.TIMER 19094 +149.9% 47725 ± 2% softirqs.CPU28.RCU 131563 +161.9% 344579 ± 2% softirqs.CPU28.SCHED 319151 +176.7% 883105 ± 9% softirqs.CPU28.TIMER 19050 ± 4% +145.4% 46756 ± 2% softirqs.CPU29.RCU 124211 ± 6% +176.0% 342763 ± 2% softirqs.CPU29.SCHED 329526 ± 4% +158.9% 853085 ± 8% softirqs.CPU29.TIMER 22721 ± 8% +121.7% 50370 ± 4% softirqs.CPU3.RCU 132516 ± 2% +156.9% 340453 ± 4% softirqs.CPU3.SCHED 323552 ± 2% +174.3% 887612 ± 9% softirqs.CPU3.TIMER 19911 ± 3% +137.4% 47276 softirqs.CPU30.RCU 130778 +161.4% 341879 softirqs.CPU30.SCHED 317264 +181.0% 891407 ± 8% softirqs.CPU30.TIMER 19232 ± 4% +144.4% 47012 softirqs.CPU31.RCU 126109 ± 10% +174.9% 346628 ± 2% softirqs.CPU31.SCHED 329439 ± 4% +169.4% 887674 ± 9% softirqs.CPU31.TIMER 18139 ± 4% +141.4% 43795 ± 4% softirqs.CPU32.RCU 125566 ± 4% +170.9% 340112 ± 4% softirqs.CPU32.SCHED 328833 +184.0% 934027 ± 9% softirqs.CPU32.TIMER 19309 ± 10% +128.3% 44077 ± 3% softirqs.CPU33.RCU 128438 ± 3% +167.7% 343879 ± 2% softirqs.CPU33.SCHED 320949 +172.2% 873631 ± 9% softirqs.CPU33.TIMER 18041 ± 4% +146.1% 44399 ± 2% softirqs.CPU34.RCU 129791 +165.0% 343914 ± 2% softirqs.CPU34.SCHED 319403 ± 2% +177.3% 885711 ± 8% softirqs.CPU34.TIMER 17991 ± 5% +146.2% 44290 ± 2% softirqs.CPU35.RCU 137112 ± 7% +148.6% 340824 ± 3% softirqs.CPU35.SCHED 416552 ± 21% +119.5% 914435 ± 8% softirqs.CPU35.TIMER 20036 ± 4% +138.7% 47823 ± 4% softirqs.CPU36.RCU 138048 ± 3% +165.7% 366854 ± 7% softirqs.CPU36.SCHED 359442 ± 22% +196.1% 1064343 ± 19% softirqs.CPU36.TIMER 19421 ± 5% +153.4% 49203 ± 2% softirqs.CPU37.RCU 135462 +158.7% 350500 ± 6% softirqs.CPU37.SCHED 305531 +166.8% 815287 ± 12% softirqs.CPU37.TIMER 19963 +150.5% 50014 softirqs.CPU38.RCU 132257 +167.5% 353848 ± 5% softirqs.CPU38.SCHED 311979 ± 2% +160.3% 812118 ± 12% softirqs.CPU38.TIMER 20119 ± 2% +143.6% 49011 ± 2% softirqs.CPU39.RCU 135450 +161.9% 354713 ± 4% softirqs.CPU39.SCHED 307797 +164.3% 813377 ± 12% softirqs.CPU39.TIMER 20593 ± 9% +132.1% 47796 ± 5% softirqs.CPU4.RCU 129881 +159.2% 336643 ± 3% softirqs.CPU4.SCHED 321940 +175.0% 885297 ± 8% softirqs.CPU4.TIMER 19874 ± 2% +149.6% 49611 softirqs.CPU40.RCU 134660 +161.9% 352636 ± 5% softirqs.CPU40.SCHED 309870 +160.6% 807374 ± 13% softirqs.CPU40.TIMER 19897 ± 2% +147.2% 49186 ± 2% softirqs.CPU41.RCU 131978 ± 2% +167.4% 352864 ± 5% softirqs.CPU41.SCHED 315198 +157.6% 812013 ± 12% softirqs.CPU41.TIMER 19988 ± 2% +143.9% 48749 ± 2% softirqs.CPU42.RCU 133326 ± 2% +165.6% 354135 ± 4% softirqs.CPU42.SCHED 313042 +159.5% 812298 ± 12% softirqs.CPU42.TIMER 19723 ± 2% +141.9% 47704 ± 4% softirqs.CPU43.RCU 131944 +165.7% 350566 ± 5% softirqs.CPU43.SCHED 314428 +162.9% 826595 ± 13% softirqs.CPU43.TIMER 19824 ± 2% +143.0% 48177 ± 4% softirqs.CPU44.RCU 132047 ± 2% +157.8% 340362 ± 12% softirqs.CPU44.SCHED 312615 ± 2% +165.2% 829094 ± 12% softirqs.CPU44.TIMER 20140 ± 2% +138.6% 48049 ± 7% softirqs.CPU45.RCU 133846 +151.7% 336901 ± 14% softirqs.CPU45.SCHED 309086 ± 2% +170.8% 837094 ± 12% softirqs.CPU45.TIMER 20103 ± 3% +138.8% 48009 ± 3% softirqs.CPU46.RCU 133860 +154.1% 340108 ± 11% softirqs.CPU46.SCHED 311323 +166.1% 828498 ± 13% softirqs.CPU46.TIMER 19628 ± 2% +149.0% 48881 ± 2% softirqs.CPU47.RCU 132835 ± 2% +164.6% 351487 ± 5% softirqs.CPU47.SCHED 309264 +162.8% 812666 ± 13% softirqs.CPU47.TIMER 19773 ± 2% +139.0% 47263 ± 2% softirqs.CPU5.RCU 130891 ± 2% +163.7% 345185 softirqs.CPU5.SCHED 323979 ± 2% +182.2% 914392 ± 8% softirqs.CPU5.TIMER 19904 ± 3% +141.4% 48045 ± 4% softirqs.CPU6.RCU 131836 +160.9% 344009 softirqs.CPU6.SCHED 316150 +181.7% 890614 ± 8% softirqs.CPU6.TIMER 19077 ± 3% +145.1% 46751 ± 3% softirqs.CPU7.RCU 127906 ± 2% +157.6% 329548 ± 5% softirqs.CPU7.SCHED 325087 +181.6% 915494 ± 9% softirqs.CPU7.TIMER 20183 ± 2% +114.9% 43380 ± 10% softirqs.CPU8.RCU 131974 +118.3% 288124 ± 30% softirqs.CPU8.SCHED 319626 +190.1% 927161 ± 13% softirqs.CPU8.TIMER 19292 ± 3% +137.6% 45839 ± 3% softirqs.CPU9.RCU 129715 +158.9% 335788 ± 4% softirqs.CPU9.SCHED 321209 +178.0% 893035 ± 8% softirqs.CPU9.TIMER 931061 +140.9% 2242787 softirqs.RCU 6322319 +160.8% 16487173 softirqs.SCHED 15417545 +169.7% 41581783 ± 8% softirqs.TIMER 853.75 ± 12% +139.9% 2048 ± 7% interrupts.35:PCI-MSI.2621441-edge.eth0-TxRx-0 586.00 ± 14% +130.1% 1348 ± 5% interrupts.36:PCI-MSI.2621442-edge.eth0-TxRx-1 651.00 ± 9% +136.3% 1538 ± 9% interrupts.37:PCI-MSI.2621443-edge.eth0-TxRx-2 565.25 ± 14% +129.7% 1298 interrupts.38:PCI-MSI.2621444-edge.eth0-TxRx-3 591.00 ± 14% +123.8% 1322 ± 2% interrupts.39:PCI-MSI.2621445-edge.eth0-TxRx-4 660.25 ± 13% +111.4% 1396 ± 7% interrupts.40:PCI-MSI.2621446-edge.eth0-TxRx-5 529.50 ± 4% +227.1% 1732 ± 14% interrupts.41:PCI-MSI.2621447-edge.eth0-TxRx-6 585.75 ± 16% +163.2% 1541 ± 10% interrupts.42:PCI-MSI.2621448-edge.eth0-TxRx-7 404968 +145.6% 994665 ± 3% interrupts.CAL:Function_call_interrupts 8396 ± 6% +133.7% 19627 ± 6% interrupts.CPU0.CAL:Function_call_interrupts 1988743 +155.3% 5077644 interrupts.CPU0.LOC:Local_timer_interrupts 8698 ± 5% +121.4% 19255 ± 2% interrupts.CPU1.CAL:Function_call_interrupts 1986989 +155.5% 5077206 interrupts.CPU1.LOC:Local_timer_interrupts 8487 ± 5% +132.3% 19713 ± 8% interrupts.CPU10.CAL:Function_call_interrupts 1988375 +155.4% 5077322 interrupts.CPU10.LOC:Local_timer_interrupts 8770 ± 5% +149.6% 21889 ± 3% interrupts.CPU11.CAL:Function_call_interrupts 1987964 +155.4% 5077484 interrupts.CPU11.LOC:Local_timer_interrupts 78678 ± 8% +541.4% 504605 ±122% interrupts.CPU11.RES:Rescheduling_interrupts 8731 ± 9% +119.5% 19167 ± 6% interrupts.CPU12.CAL:Function_call_interrupts 1988222 +155.4% 5077524 interrupts.CPU12.LOC:Local_timer_interrupts 7960 ± 3% +153.5% 20176 ± 9% interrupts.CPU13.CAL:Function_call_interrupts 1988667 +155.3% 5076359 interrupts.CPU13.LOC:Local_timer_interrupts 8518 ± 6% +155.0% 21723 ± 7% interrupts.CPU14.CAL:Function_call_interrupts 1987350 +155.5% 5077482 interrupts.CPU14.LOC:Local_timer_interrupts 37.00 ± 31% +124.3% 83.00 ± 24% interrupts.CPU14.TLB:TLB_shootdowns 8831 ± 4% +137.4% 20964 ± 10% interrupts.CPU15.CAL:Function_call_interrupts 1987501 +155.5% 5077129 interrupts.CPU15.LOC:Local_timer_interrupts 3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.NMI:Non-maskable_interrupts 3708 ± 27% -63.0% 1370 ±103% interrupts.CPU15.PMI:Performance_monitoring_interrupts 23.25 ± 24% +253.8% 82.25 ± 28% interrupts.CPU15.TLB:TLB_shootdowns 8708 ± 8% +132.7% 20268 ± 7% interrupts.CPU16.CAL:Function_call_interrupts 1988418 +155.3% 5076923 interrupts.CPU16.LOC:Local_timer_interrupts 8548 ± 7% +148.3% 21226 ± 7% interrupts.CPU17.CAL:Function_call_interrupts 1988416 +155.3% 5077317 interrupts.CPU17.LOC:Local_timer_interrupts 33.00 ± 18% +147.0% 81.50 ± 21% interrupts.CPU17.TLB:TLB_shootdowns 8456 ± 9% +157.0% 21731 ± 6% interrupts.CPU18.CAL:Function_call_interrupts 1988574 +155.3% 5077430 interrupts.CPU18.LOC:Local_timer_interrupts 30.25 ± 15% +181.8% 85.25 ± 25% interrupts.CPU18.TLB:TLB_shootdowns 8884 ± 6% +144.5% 21720 ± 3% interrupts.CPU19.CAL:Function_call_interrupts 1987747 +155.4% 5077260 interrupts.CPU19.LOC:Local_timer_interrupts 37.00 ± 19% +127.0% 84.00 ± 26% interrupts.CPU19.TLB:TLB_shootdowns 8298 ± 11% +145.3% 20352 ± 9% interrupts.CPU2.CAL:Function_call_interrupts 1988446 +155.4% 5077740 interrupts.CPU2.LOC:Local_timer_interrupts 8594 ± 5% +146.1% 21149 ± 5% interrupts.CPU20.CAL:Function_call_interrupts 1987855 +155.4% 5077226 interrupts.CPU20.LOC:Local_timer_interrupts 8376 ± 8% +151.7% 21084 ± 9% interrupts.CPU21.CAL:Function_call_interrupts 1988171 +155.4% 5077212 interrupts.CPU21.LOC:Local_timer_interrupts 41.25 ± 20% +106.1% 85.00 ± 27% interrupts.CPU21.TLB:TLB_shootdowns 9024 ± 2% +141.1% 21757 ± 3% interrupts.CPU22.CAL:Function_call_interrupts 1987613 +155.4% 5077125 interrupts.CPU22.LOC:Local_timer_interrupts 8095 ± 7% +154.9% 20633 ± 6% interrupts.CPU23.CAL:Function_call_interrupts 1989315 +155.2% 5077451 interrupts.CPU23.LOC:Local_timer_interrupts 853.75 ± 12% +139.9% 2048 ± 7% interrupts.CPU24.35:PCI-MSI.2621441-edge.eth0-TxRx-0 8463 ± 5% +147.0% 20901 ± 7% interrupts.CPU24.CAL:Function_call_interrupts 1988251 +155.4% 5077879 interrupts.CPU24.LOC:Local_timer_interrupts 74848 ± 8% +525.2% 467985 ±128% interrupts.CPU24.RES:Rescheduling_interrupts 586.00 ± 14% +130.1% 1348 ± 5% interrupts.CPU25.36:PCI-MSI.2621442-edge.eth0-TxRx-1 8156 ± 7% +170.0% 22019 ± 2% interrupts.CPU25.CAL:Function_call_interrupts 1988295 +155.3% 5077010 interrupts.CPU25.LOC:Local_timer_interrupts 930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.NMI:Non-maskable_interrupts 930.75 ±123% +260.9% 3359 ± 15% interrupts.CPU25.PMI:Performance_monitoring_interrupts 72678 ± 7% +575.9% 491267 ±120% interrupts.CPU25.RES:Rescheduling_interrupts 651.00 ± 9% +136.3% 1538 ± 9% interrupts.CPU26.37:PCI-MSI.2621443-edge.eth0-TxRx-2 8538 ± 6% +141.5% 20625 ± 6% interrupts.CPU26.CAL:Function_call_interrupts 1988053 +155.4% 5077342 interrupts.CPU26.LOC:Local_timer_interrupts 75367 ± 8% +522.0% 468794 ±133% interrupts.CPU26.RES:Rescheduling_interrupts 565.25 ± 14% +129.7% 1298 interrupts.CPU27.38:PCI-MSI.2621444-edge.eth0-TxRx-3 8157 ± 5% +132.5% 18969 ± 3% interrupts.CPU27.CAL:Function_call_interrupts 1988437 +155.3% 5077428 interrupts.CPU27.LOC:Local_timer_interrupts 591.00 ± 14% +123.8% 1322 ± 2% interrupts.CPU28.39:PCI-MSI.2621445-edge.eth0-TxRx-4 8181 ± 6% +148.1% 20300 ± 3% interrupts.CPU28.CAL:Function_call_interrupts 1987807 +155.4% 5077738 interrupts.CPU28.LOC:Local_timer_interrupts 71071 ± 7% +581.0% 483987 ±128% interrupts.CPU28.RES:Rescheduling_interrupts 660.25 ± 13% +111.4% 1396 ± 7% interrupts.CPU29.40:PCI-MSI.2621446-edge.eth0-TxRx-5 8178 ± 5% +150.4% 20475 ± 7% interrupts.CPU29.CAL:Function_call_interrupts 1987852 +155.4% 5077382 interrupts.CPU29.LOC:Local_timer_interrupts 70205 ± 8% +585.5% 481286 ±127% interrupts.CPU29.RES:Rescheduling_interrupts 17.75 ± 36% +163.4% 46.75 ± 80% interrupts.CPU29.TLB:TLB_shootdowns 8684 ± 5% +151.7% 21860 ± 4% interrupts.CPU3.CAL:Function_call_interrupts 1988786 +155.3% 5077436 interrupts.CPU3.LOC:Local_timer_interrupts 80407 ± 4% +518.6% 497419 ±122% interrupts.CPU3.RES:Rescheduling_interrupts 529.50 ± 4% +227.1% 1732 ± 14% interrupts.CPU30.41:PCI-MSI.2621447-edge.eth0-TxRx-6 8415 ± 4% +138.0% 20031 ± 3% interrupts.CPU30.CAL:Function_call_interrupts 1988609 +155.3% 5077494 interrupts.CPU30.LOC:Local_timer_interrupts 74875 ± 8% +518.8% 463337 ±134% interrupts.CPU30.RES:Rescheduling_interrupts 585.75 ± 16% +163.2% 1541 ± 10% interrupts.CPU31.42:PCI-MSI.2621448-edge.eth0-TxRx-7 7917 +151.5% 19913 ± 3% interrupts.CPU31.CAL:Function_call_interrupts 1988677 +155.3% 5077502 interrupts.CPU31.LOC:Local_timer_interrupts 67596 ± 3% +596.5% 470844 ±134% interrupts.CPU31.RES:Rescheduling_interrupts 8230 ± 4% +161.9% 21551 ± 8% interrupts.CPU32.CAL:Function_call_interrupts 1988317 +155.4% 5077410 interrupts.CPU32.LOC:Local_timer_interrupts 70449 ± 5% +597.2% 491174 ±127% interrupts.CPU32.RES:Rescheduling_interrupts 8684 ± 6% +137.3% 20610 ± 6% interrupts.CPU33.CAL:Function_call_interrupts 1988440 +155.3% 5077281 interrupts.CPU33.LOC:Local_timer_interrupts 77401 ± 5% +495.8% 461151 ±134% interrupts.CPU33.RES:Rescheduling_interrupts 8412 ± 8% +150.3% 21061 ± 4% interrupts.CPU34.CAL:Function_call_interrupts 1988447 +155.4% 5077546 interrupts.CPU34.LOC:Local_timer_interrupts 75351 ± 8% +537.5% 480376 ±128% interrupts.CPU34.RES:Rescheduling_interrupts 7890 +147.7% 19543 ± 5% interrupts.CPU35.CAL:Function_call_interrupts 1988875 +155.3% 5077717 interrupts.CPU35.LOC:Local_timer_interrupts 70158 ± 2% +577.1% 475011 ±135% interrupts.CPU35.RES:Rescheduling_interrupts 7908 ± 2% +184.9% 22530 ± 3% interrupts.CPU36.CAL:Function_call_interrupts 1988403 +155.4% 5077694 interrupts.CPU36.LOC:Local_timer_interrupts 31.75 ± 20% +178.0% 88.25 ± 28% interrupts.CPU36.TLB:TLB_shootdowns 9078 ± 2% +133.8% 21229 ± 6% interrupts.CPU37.CAL:Function_call_interrupts 1986971 +155.5% 5076871 interrupts.CPU37.LOC:Local_timer_interrupts 34.75 ± 36% +154.7% 88.50 ± 25% interrupts.CPU37.TLB:TLB_shootdowns 8574 ± 7% +131.7% 19864 ± 3% interrupts.CPU38.CAL:Function_call_interrupts 1986780 +155.5% 5076769 interrupts.CPU38.LOC:Local_timer_interrupts 29.25 ± 35% +195.7% 86.50 ± 18% interrupts.CPU38.TLB:TLB_shootdowns 8184 ± 7% +158.0% 21112 ± 4% interrupts.CPU39.CAL:Function_call_interrupts 1986972 +155.5% 5077337 interrupts.CPU39.LOC:Local_timer_interrupts 8368 ± 8% +144.7% 20481 ± 12% interrupts.CPU4.CAL:Function_call_interrupts 1987774 +155.4% 5077506 interrupts.CPU4.LOC:Local_timer_interrupts 8165 ± 3% +163.7% 21528 ± 6% interrupts.CPU40.CAL:Function_call_interrupts 1987659 +155.4% 5077421 interrupts.CPU40.LOC:Local_timer_interrupts 32.00 ± 12% +154.7% 81.50 ± 29% interrupts.CPU40.TLB:TLB_shootdowns 8454 ± 6% +158.4% 21847 ± 8% interrupts.CPU41.CAL:Function_call_interrupts 1987504 +155.5% 5077467 interrupts.CPU41.LOC:Local_timer_interrupts 37.75 ± 45% +122.5% 84.00 ± 17% interrupts.CPU41.TLB:TLB_shootdowns 8529 ± 7% +136.4% 20164 ± 8% interrupts.CPU42.CAL:Function_call_interrupts 1988401 +155.4% 5077407 interrupts.CPU42.LOC:Local_timer_interrupts 32.75 ± 39% +171.8% 89.00 ± 37% interrupts.CPU42.TLB:TLB_shootdowns 8102 ± 5% +150.4% 20283 ± 9% interrupts.CPU43.CAL:Function_call_interrupts 1987414 +155.5% 5077903 interrupts.CPU43.LOC:Local_timer_interrupts 32.50 ± 18% +150.8% 81.50 ± 34% interrupts.CPU43.TLB:TLB_shootdowns 8206 ± 7% +157.3% 21112 ± 8% interrupts.CPU44.CAL:Function_call_interrupts 1987278 +155.5% 5077309 interrupts.CPU44.LOC:Local_timer_interrupts 37.00 ± 25% +119.6% 81.25 ± 24% interrupts.CPU44.TLB:TLB_shootdowns 8467 ± 7% +139.3% 20261 ± 5% interrupts.CPU45.CAL:Function_call_interrupts 1987858 +155.4% 5077234 interrupts.CPU45.LOC:Local_timer_interrupts 38.25 ± 22% +113.1% 81.50 ± 27% interrupts.CPU45.TLB:TLB_shootdowns 7773 ± 4% +162.0% 20367 ± 9% interrupts.CPU46.CAL:Function_call_interrupts 1988584 +155.3% 5077417 interrupts.CPU46.LOC:Local_timer_interrupts 37.50 ± 14% +142.7% 91.00 ± 34% interrupts.CPU46.TLB:TLB_shootdowns 8746 ± 4% +138.7% 20880 ± 8% interrupts.CPU47.CAL:Function_call_interrupts 1988495 +155.3% 5077204 interrupts.CPU47.LOC:Local_timer_interrupts 8722 ± 7% +134.1% 20414 ± 10% interrupts.CPU5.CAL:Function_call_interrupts 1987875 +155.4% 5077418 interrupts.CPU5.LOC:Local_timer_interrupts 8435 ± 8% +143.9% 20570 ± 11% interrupts.CPU6.CAL:Function_call_interrupts 1988389 +155.4% 5077630 interrupts.CPU6.LOC:Local_timer_interrupts 8837 ± 5% +139.1% 21133 ± 9% interrupts.CPU7.CAL:Function_call_interrupts 1987853 +155.4% 5077370 interrupts.CPU7.LOC:Local_timer_interrupts 8852 ± 5% +123.8% 19810 ± 3% interrupts.CPU8.CAL:Function_call_interrupts 1988702 +155.3% 5077569 interrupts.CPU8.LOC:Local_timer_interrupts 8273 ± 6% +151.1% 20771 ± 9% interrupts.CPU9.CAL:Function_call_interrupts 1988033 +155.4% 5077588 interrupts.CPU9.LOC:Local_timer_interrupts 70739 ± 6% +578.9% 480243 ±129% interrupts.CPU9.RES:Rescheduling_interrupts 95428175 +155.4% 2.437e+08 interrupts.LOC:Local_timer_interrupts 144.00 +133.3% 336.00 interrupts.MCP:Machine_check_polls 6773889 ± 8% +482.9% 39483682 ± 7% interrupts.RES:Rescheduling_interrupts 1544 ± 5% +91.5% 2957 ± 11% interrupts.TLB:TLB_shootdowns aim7.jobs-per-min 1300 +-+------------------------------------------------------------------+ 1200 +-+.++.+.+.+.+. +.+.+.+. +.+.+ +.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.| | + + + | 1100 +-+ | 1000 +-+ | | | 900 +-+ | 800 +-+ | 700 +-+ | | | 600 +-+ | 500 +-+ O | | O O O O O O O | 400 O-O OO O O O O OO O O O OO O O O O OO O O O O | 300 +-+------------------------------------------------------------------+ aim7.time.user_time 55 +-+--------------------------------------------------------------------+ | | 50 +-+ O | | O | | | 45 +-+ | | O O O O O O O O O O | 40 O-O O OO O O O O O O O O O OO O O O O O | | | 35 +-+ | | | | | 30 +-+ | |.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.+.++.+.+.+.+.+.++.+. .+.+.+.+.++.+.+.| 25 +-+--------------------------------------------------------------------+ aim7.time.system_time 11000 +-+-----------------------------------------------------------------+ O O OO O O O OO O O OO O O O OO O O O OO O O | 10000 +-+ O OO O O OO O | 9000 +-+ | | | 8000 +-+ | | | 7000 +-+ | | | 6000 +-+ | 5000 +-+ | | | 4000 +-+ | | .++. .+. .+.+. .+.+. .++. .+ .+.+.+ .+. .++.+.| 3000 +-+-----------------------------------------------------------------+ aim7.time.elapsed_time 3500 +-+------------------------------------------------------------------+ O O OO O O O O OO O O OO O O O O OO O O O O | 3000 +-+ O | | | | O O O O O O | 2500 +-+ OO | | | 2000 +-+ | | | 1500 +-+ | | | | | 1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.| | | 500 +-+------------------------------------------------------------------+ aim7.time.elapsed_time.max 3500 +-+------------------------------------------------------------------+ O O OO O O O O OO O O OO O O O O OO O O O O | 3000 +-+ O | | | | O O O O O O | 2500 +-+ OO | | | 2000 +-+ | | | 1500 +-+ | | | | | 1000 +-+.++.+.+.+.+.++.+.+.+.++.+.+.+.+.++.+.+.+.++.+.+.+.++.+.+.+.+.++.+.| | | 500 +-+------------------------------------------------------------------+ aim7.time.voluntary_context_switches 1.8e+09 +-+---------------------------------------------------------------+ O O O O OO O OO O O OO O O OO O OO O O | 1.6e+09 +-O O O O O O OO O O O | | | 1.4e+09 +-+ | | | 1.2e+09 +-+ | | | 1e+09 +-+ | | | 8e+08 +-+ | | | 6e+08 +-+ | |.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.| 4e+08 +-+---------------------------------------------------------------+ aim7.time.file_system_outputs 1.8e+08 +-+---------------------------------------------------------------+ O OO O O OO O O OO O O OO O O OO O OO O O OO O O OO O O O | 1.7e+08 +-+ | 1.6e+08 +-+ | | | 1.5e+08 +-+ | | | 1.4e+08 +-+ | | | 1.3e+08 +-+ | 1.2e+08 +-+ | | | 1.1e+08 +-+ | |.++.+.+.++.+.+.++.+.+.++.+.+.++.+.++.+.+.++.+.+.++.+.+.++.+.+.++.| 1e+08 +-+---------------------------------------------------------------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

9 months, 1 week

3
3
0 / 0

[btrfs] 8d47a0d8f7: fio.write_bw_MBps -28.6% regression

by kernel test robot

Greeting, FYI, we noticed a -28.6% regression of fio.write_bw_MBps due to commit: commit: 8d47a0d8f7947422dd359ac8e462687f81a7a137 ("btrfs: Do mandatory tree block check before submitting bio") https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master in testcase: fio-basic on test machine: 72 threads Intel(R) Xeon(R) CPU E5-2699 v3 @ 2.30GHz with 256G memory with following parameters: disk: 2pmem fs: btrfs runtime: 200s nr_task: 50% time_based: tb rw: randwrite bs: 4k ioengine: libaio test_size: 100G cpufreq_governor: performance ucode: 0x3d test-description: Fio is a tool that will spawn a number of threads or processes doing a particular type of I/O action as specified by the user. test-url: https://github.com/axboe/fio Details are as below: --------------------------------------------------------------------------------------------------> To reproduce: git clone https://github.com/intel/lkp-tests.git cd lkp-tests bin/lkp install job.yaml # job file is attached in this email bin/lkp run job.yaml ========================================================================================= bs/compiler/cpufreq_governor/disk/fs/ioengine/kconfig/nr_task/rootfs/runtime/rw/tbox_group/test_size/testcase/time_based/ucode: 4k/gcc-7/performance/2pmem/btrfs/libaio/x86_64-rhel-7.6/50%/debian-x86_64-2018-04-03.cgz/200s/randwrite/lkp-hsw-ep2/100G/fio-basic/tb/0x3d commit: ff2ac107fa ("btrfs: tree-checker: Remove comprehensive root owner check") 8d47a0d8f7 ("btrfs: Do mandatory tree block check before submitting bio") ff2ac107fae2440b 8d47a0d8f7947422dd359ac8e46 ---------------- --------------------------- fail:runs %reproduction fail:runs | | | :4 25% 1:4 dmesg.BUG:unable_to_handle_kernel :4 25% 1:4 dmesg.Kernel_panic-not_syncing:Fatal_exception_in_interrupt :4 25% 1:4 dmesg.Oops:#[##] :4 25% 1:4 dmesg.RIP:cpuidle_enter_state :4 25% 1:4 dmesg.RIP:native_write_msr :4 25% 1:4 dmesg.RIP:perf_prepare_sample 1:4 -25% :4 dmesg.WARNING:at#for_ip_interrupt_entry/0x %stddev %change %stddev \ | \ 0.01 ± 21% +0.3 0.28 ± 20% fio.latency_1000ms% 4.14 ± 13% +1.8 5.95 ± 9% fio.latency_10ms% 8.73 ± 15% +3.0 11.68 ± 8% fio.latency_20ms% 1.91 ± 5% -0.4 1.52 ± 3% fio.latency_250ms% 66.81 -5.9 60.90 ± 2% fio.latency_4ms% 0.18 ± 15% +0.1 0.29 ± 16% fio.latency_750us% 1.017e+08 -28.5% 72701525 fio.time.file_system_outputs 109.75 -26.8% 80.33 ± 2% fio.time.percent_of_cpu_this_job_got 197.97 -26.4% 145.79 ± 2% fio.time.system_time 38.66 ± 4% -24.0% 29.39 ± 3% fio.time.user_time 12388015 -28.5% 8858878 fio.time.voluntary_context_switches 12711059 -28.5% 9083954 fio.workload 247.91 -28.6% 176.96 fio.write_bw_MBps 309248 ± 2% +71.7% 531114 ± 4% fio.write_clat_99%_us 17574 +40.0% 24609 fio.write_clat_mean_us 57989 ± 4% +82.2% 105661 fio.write_clat_stddev 63463 -28.6% 45302 fio.write_iops 564.61 +40.2% 791.54 fio.write_slat_mean_us 9375 ± 7% +96.3% 18406 fio.write_slat_stddev 21.25 +8.5% 23.06 ± 9% boot-time.dhcp 93.32 +1.4% 94.58 iostat.cpu.idle 6.43 -18.6% 5.23 ± 2% iostat.cpu.system 6.48 -1.2 5.27 ± 2% mpstat.cpu.all.sys% 0.24 ± 5% -0.1 0.18 ± 2% mpstat.cpu.all.usr% 93.00 +1.1% 94.00 vmstat.cpu.id 591200 -30.0% 413921 vmstat.io.bo 31653776 ± 2% -10.9% 28196588 ± 2% vmstat.memory.cache 11927341 ± 7% +29.8% 15486805 ± 4% vmstat.memory.free 5.00 -26.7% 3.67 ± 12% vmstat.procs.r 635910 ± 2% -27.6% 460384 vmstat.system.cs 7822384 ± 2% -25.0% 5870498 numa-numastat.node0.local_node 7839861 ± 2% -24.8% 5893785 numa-numastat.node0.numa_hit 224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node0.numa_miss 242465 ± 44% -84.3% 37964 ± 18% numa-numastat.node0.other_node 7762623 ± 12% -26.8% 5681358 ± 6% numa-numastat.node1.local_node 224988 ± 47% -93.5% 14675 ± 48% numa-numastat.node1.numa_foreign 7768618 ± 12% -26.9% 5681573 ± 6% numa-numastat.node1.numa_hit 9.429e+08 ± 10% -35.4% 6.094e+08 cpuidle.C1.time 53504154 ± 6% -31.8% 36504731 cpuidle.C1.usage 1.093e+09 ± 22% -50.3% 5.435e+08 ± 3% cpuidle.C1E.time 16541925 ± 13% -45.0% 9103775 cpuidle.C1E.usage 5.366e+09 ± 36% +71.7% 9.214e+09 ± 18% cpuidle.C3.time 16163664 ± 33% +57.0% 25379738 ± 5% cpuidle.C3.usage 9697055 ± 16% -35.8% 6225168 ± 19% cpuidle.C6.usage 19313595 ± 17% -16.3% 16166123 cpuidle.POLL.time 10291838 ± 9% -26.8% 7537045 ± 7% meminfo.Active 10034782 ± 9% -27.4% 7280710 ± 7% meminfo.Active(file) 31524905 ± 2% -10.9% 28082249 ± 2% meminfo.Cached 115157 ± 4% +61.1% 185560 ± 3% meminfo.CmaFree 6430720 ± 8% +19.1% 7660885 ± 6% meminfo.DirectMap2M 534230 ± 5% -12.8% 465932 ± 6% meminfo.Dirty 11737322 ± 7% +30.2% 15281189 ± 4% meminfo.MemFree 37604344 ± 2% -9.4% 34060483 meminfo.Memused 806.50 ± 37% +137.4% 1914 ± 8% meminfo.Writeback 370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.active_objs 370.50 ± 11% +36.8% 507.00 slabinfo.bdev_cache.num_objs 15147 ± 5% -8.3% 13897 ± 2% slabinfo.blkdev_ioc.active_objs 17839 ± 6% -11.7% 15745 ± 2% slabinfo.blkdev_ioc.num_objs 12178 ± 4% -7.4% 11275 ± 2% slabinfo.kmalloc-512.num_objs 7794 ± 3% +25.4% 9774 slabinfo.proc_inode_cache.active_objs 8017 ± 2% +23.4% 9895 slabinfo.proc_inode_cache.num_objs 2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.active_objs 2366 ± 23% -32.6% 1595 ± 5% slabinfo.task_group.num_objs 263.75 ± 2% -15.1% 224.00 ± 2% turbostat.Avg_MHz 9.46 ± 2% -1.4 8.07 ± 2% turbostat.Busy% 53502248 ± 6% -31.8% 36503898 turbostat.C1 5.86 ± 8% -2.0 3.82 turbostat.C1% 16540396 ± 13% -45.0% 9102733 turbostat.C1E 6.80 ± 22% -3.4 3.40 ± 4% turbostat.C1E% 16163332 ± 33% +57.0% 25379351 ± 5% turbostat.C3 33.55 ± 36% +24.1 57.61 ± 18% turbostat.C3% 9689445 ± 16% -35.8% 6221033 ± 19% turbostat.C6 18.04 ± 26% +49.1% 26.90 ± 9% turbostat.CPU%c3 53.25 ± 3% +10.8% 59.00 ± 4% turbostat.CoreTmp 4.67 ± 14% +62.7% 7.59 ± 26% turbostat.Pkg%pc2 56.50 +9.7% 62.00 ± 2% turbostat.PkgTmp 11.84 -4.8% 11.28 turbostat.RAMWatt 7232478 ± 10% -37.2% 4543037 ± 22% numa-meminfo.node0.Active 7072564 ± 10% -37.6% 4410135 ± 23% numa-meminfo.node0.Active(file) 308167 ± 6% -24.7% 232152 ± 10% numa-meminfo.node0.Dirty 8784028 ± 13% +21.7% 10686327 ± 2% numa-meminfo.node0.Inactive 8669548 ± 12% +20.7% 10465444 ± 2% numa-meminfo.node0.Inactive(file) 572.50 ± 22% +115.9% 1236 ± 25% numa-meminfo.node0.Writeback 15098140 ± 3% -17.7% 12425670 ± 3% numa-meminfo.node1.FilePages 11599549 ± 7% -22.3% 9018051 ± 4% numa-meminfo.node1.Inactive 105067 ± 92% -99.3% 725.33 ± 51% numa-meminfo.node1.Inactive(anon) 11494482 ± 7% -21.6% 9017325 ± 4% numa-meminfo.node1.Inactive(file) 5504 ± 9% -12.5% 4819 ± 2% numa-meminfo.node1.KernelStack 6709938 ± 11% +45.8% 9781880 ± 5% numa-meminfo.node1.MemFree 18029136 ± 4% -17.0% 14957200 ± 3% numa-meminfo.node1.MemUsed 4427 ± 69% -77.5% 996.00 ± 8% numa-meminfo.node1.PageTables 857660 ± 32% -37.2% 538646 ± 8% numa-meminfo.node1.SUnreclaim 109232 ± 88% -96.1% 4274 ± 10% numa-meminfo.node1.Shmem 986740 ± 35% -40.0% 592452 ± 10% numa-meminfo.node1.Slab 575.21 ± 15% +30.1% 748.58 ± 2% sched_debug.cfs_rq:/.runnable_load_avg.max 100.45 ± 15% +28.6% 129.19 ± 5% sched_debug.cfs_rq:/.runnable_load_avg.stddev 830.19 ± 9% +15.7% 960.25 ± 6% sched_debug.cfs_rq:/.util_avg.max 29.97 ± 16% -25.2% 22.43 ± 31% sched_debug.cfs_rq:/.util_est_enqueued.avg 591.17 ± 13% -22.2% 459.83 ± 18% sched_debug.cfs_rq:/.util_est_enqueued.max 107.52 ± 13% -23.0% 82.84 ± 23% sched_debug.cfs_rq:/.util_est_enqueued.stddev 2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock.stddev 2.06 ± 5% -9.3% 1.87 sched_debug.cpu.clock_task.stddev 24.51 ± 7% +21.5% 29.77 ± 8% sched_debug.cpu.cpu_load[0].avg 541.29 ± 15% +38.3% 748.58 ± 2% sched_debug.cpu.cpu_load[0].max 93.97 ± 7% +27.8% 120.09 ± 2% sched_debug.cpu.cpu_load[0].stddev 519.08 ± 6% +40.5% 729.42 ± 4% sched_debug.cpu.cpu_load[1].max 87.45 ± 5% +28.5% 112.39 ± 4% sched_debug.cpu.cpu_load[1].stddev 480.83 ± 7% +39.5% 670.92 ± 13% sched_debug.cpu.cpu_load[2].max 80.92 ± 5% +25.7% 101.75 ± 8% sched_debug.cpu.cpu_load[2].stddev 408.83 ± 4% +54.0% 629.58 ± 19% sched_debug.cpu.cpu_load[3].max 71.62 ± 2% +30.7% 93.61 ± 11% sched_debug.cpu.cpu_load[3].stddev 381.50 ± 8% +66.7% 636.00 ± 18% sched_debug.cpu.cpu_load[4].max 64.40 ± 6% +39.7% 89.96 ± 12% sched_debug.cpu.cpu_load[4].stddev 106.75 ± 20% -88.4% 12.33 ± 10% proc-vmstat.kswapd_high_wmark_hit_quickly 2508660 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_active_file 33315360 -30.4% 23192301 proc-vmstat.nr_dirtied 133447 ± 5% -12.6% 116580 ± 6% proc-vmstat.nr_dirty 7881043 ± 2% -10.9% 7019533 ± 2% proc-vmstat.nr_file_pages 28790 ± 4% +61.1% 46392 ± 3% proc-vmstat.nr_free_cma 2934503 ± 7% +30.2% 3821369 ± 4% proc-vmstat.nr_free_pages 5044168 -3.4% 4872965 proc-vmstat.nr_inactive_file 430282 -6.0% 404589 proc-vmstat.nr_slab_unreclaimable 197.75 ± 38% +136.5% 467.67 ± 10% proc-vmstat.nr_writeback 33243480 -30.4% 23143756 proc-vmstat.nr_written 2508665 ± 9% -27.5% 1819662 ± 7% proc-vmstat.nr_zone_active_file 5044399 -3.4% 4873026 proc-vmstat.nr_zone_inactive_file 134972 ± 5% -12.4% 118262 ± 5% proc-vmstat.nr_zone_write_pending 15632301 ± 5% -25.8% 11598535 ± 3% proc-vmstat.numa_hit 15608826 ± 5% -25.8% 11575029 ± 3% proc-vmstat.numa_local 2300530 ± 10% -31.1% 1585363 ± 10% proc-vmstat.pgactivate 16955745 -26.3% 12499611 ± 2% proc-vmstat.pgalloc_normal 17488000 -25.9% 12959812 ± 2% proc-vmstat.pgfree 1.33e+08 -30.4% 92575440 proc-vmstat.pgpgout 5728177 ± 6% -78.5% 1232433 ± 25% proc-vmstat.pgscan_kswapd 5727967 ± 6% -78.5% 1232244 ± 25% proc-vmstat.pgsteal_kswapd 32760 -7.6% 30285 proc-vmstat.slabs_scanned 1768211 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_active_file 10410188 ± 7% -32.5% 7023939 ± 7% numa-vmstat.node0.nr_dirtied 76995 ± 6% -24.6% 58035 ± 10% numa-vmstat.node0.nr_dirty 2167457 ± 12% +20.7% 2616147 ± 2% numa-vmstat.node0.nr_inactive_file 142.00 ± 24% +120.4% 313.00 ± 24% numa-vmstat.node0.nr_writeback 10308758 ± 7% -32.6% 6949497 ± 7% numa-vmstat.node0.nr_written 1768216 ± 10% -37.7% 1102275 ± 23% numa-vmstat.node0.nr_zone_active_file 2167563 ± 12% +20.7% 2616189 ± 2% numa-vmstat.node0.nr_zone_inactive_file 77786 ± 7% -24.3% 58899 ± 10% numa-vmstat.node0.nr_zone_write_pending 5889240 ± 3% -14.3% 5046874 ± 3% numa-vmstat.node0.numa_hit 5837851 ± 4% -14.0% 5023120 ± 3% numa-vmstat.node0.numa_local 68376 ± 48% -96.9% 2149 ± 46% numa-vmstat.node0.numa_miss 119766 ± 42% -78.4% 25904 ± 3% numa-vmstat.node0.numa_other 30.50 ± 38% -100.0% 0.00 numa-vmstat.node0.workingset_nodes 6829126 ± 6% -22.8% 5269898 ± 10% numa-vmstat.node1.nr_dirtied 3774645 ± 3% -17.7% 3106081 ± 3% numa-vmstat.node1.nr_file_pages 28796 ± 4% +61.1% 46405 ± 3% numa-vmstat.node1.nr_free_cma 1677332 ± 11% +45.8% 2445778 ± 5% numa-vmstat.node1.nr_free_pages 26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_inactive_anon 2873702 ± 7% -21.6% 2254116 ± 4% numa-vmstat.node1.nr_inactive_file 5506 ± 9% -12.5% 4820 ± 2% numa-vmstat.node1.nr_kernel_stack 29031 ± 85% -88.2% 3436 numa-vmstat.node1.nr_mapped 1106 ± 69% -77.5% 249.00 ± 8% numa-vmstat.node1.nr_page_table_pages 27308 ± 88% -96.1% 1068 ± 10% numa-vmstat.node1.nr_shmem 214418 ± 32% -37.2% 134663 ± 8% numa-vmstat.node1.nr_slab_unreclaimable 75.00 ± 35% +145.3% 184.00 ± 18% numa-vmstat.node1.nr_writeback 6761224 ± 6% -23.1% 5202586 ± 10% numa-vmstat.node1.nr_written 26266 ± 92% -99.3% 180.67 ± 51% numa-vmstat.node1.nr_zone_inactive_anon 2873810 ± 7% -21.6% 2254135 ± 4% numa-vmstat.node1.nr_zone_inactive_file 68381 ± 48% -96.9% 2152 ± 46% numa-vmstat.node1.numa_foreign 5035283 ± 15% -26.9% 3679318 ± 7% numa-vmstat.node1.numa_hit 4905249 ± 15% -28.2% 3522229 ± 8% numa-vmstat.node1.numa_local 18.43 ± 6% -28.3% 13.21 ± 3% perf-stat.i.MPKI 2.205e+09 ± 3% +10.5% 2.437e+09 perf-stat.i.branch-instructions 1.83 ± 5% -0.6 1.24 ± 7% perf-stat.i.branch-miss-rate% 37019551 ± 3% -29.4% 26140844 ± 5% perf-stat.i.branch-misses 28932020 ± 6% -31.1% 19933819 ± 3% perf-stat.i.cache-misses 2.03e+08 ± 4% -24.8% 1.526e+08 perf-stat.i.cache-references 645248 ± 2% -27.4% 468301 perf-stat.i.context-switches 1.97 ± 7% -22.1% 1.53 ± 2% perf-stat.i.cpi 1.983e+10 ± 3% -15.1% 1.684e+10 perf-stat.i.cpu-cycles 1.03 ± 5% -0.4 0.64 ± 10% perf-stat.i.dTLB-load-miss-rate% 31879502 ± 6% -37.9% 19807598 ± 9% perf-stat.i.dTLB-load-misses 3.201e+09 ± 3% +18.8% 3.804e+09 perf-stat.i.dTLB-loads 0.13 ± 7% -0.0 0.09 ± 10% perf-stat.i.dTLB-store-miss-rate% 2516181 ± 10% -28.3% 1805272 ± 10% perf-stat.i.dTLB-store-misses 2.143e+09 ± 3% +19.8% 2.568e+09 ± 2% perf-stat.i.dTLB-stores 13781489 ± 3% -28.4% 9866239 ± 3% perf-stat.i.iTLB-loads 1.164e+10 ± 3% +21.6% 1.415e+10 perf-stat.i.instructions 8435 ± 17% +42.5% 12023 ± 14% perf-stat.i.instructions-per-iTLB-miss 0.57 ± 3% +43.6% 0.82 perf-stat.i.ipc 16050361 ± 9% -30.7% 11127721 ± 3% perf-stat.i.node-load-misses 8120610 ± 9% -31.3% 5580996 perf-stat.i.node-loads 2591649 ± 13% -31.1% 1785932 ± 6% perf-stat.i.node-store-misses 1809775 ± 12% -31.2% 1244360 ± 2% perf-stat.i.node-stores 17.45 ± 4% -38.2% 10.79 perf-stat.overall.MPKI 1.68 ± 3% -0.6 1.07 ± 5% perf-stat.overall.branch-miss-rate% 1.70 ± 3% -30.2% 1.19 perf-stat.overall.cpi 689.69 ± 9% +22.6% 845.48 ± 2% perf-stat.overall.cycles-between-cache-misses 0.99 ± 5% -0.5 0.52 ± 10% perf-stat.overall.dTLB-load-miss-rate% 0.12 ± 8% -0.0 0.07 ± 10% perf-stat.overall.dTLB-store-miss-rate% 10.81 ± 14% +2.3 13.09 ± 10% perf-stat.overall.iTLB-load-miss-rate% 7119 ± 14% +35.9% 9679 ± 13% perf-stat.overall.instructions-per-iTLB-miss 0.59 ± 3% +43.1% 0.84 perf-stat.overall.ipc 202912 ± 2% +68.8% 342615 ± 3% perf-stat.overall.path-length 2.195e+09 ± 3% +10.5% 2.425e+09 perf-stat.ps.branch-instructions 36847010 ± 3% -29.4% 26015686 ± 5% perf-stat.ps.branch-misses 28799092 ± 6% -31.1% 19837306 ± 3% perf-stat.ps.cache-misses 2.021e+08 ± 4% -24.8% 1.519e+08 perf-stat.ps.cache-references 642266 ± 2% -27.4% 465968 perf-stat.ps.context-switches 1.974e+10 ± 3% -15.1% 1.676e+10 perf-stat.ps.cpu-cycles 31729472 ± 6% -37.9% 19715594 ± 9% perf-stat.ps.dTLB-load-misses 3.186e+09 ± 3% +18.8% 3.785e+09 perf-stat.ps.dTLB-loads 2504443 ± 10% -28.2% 1797154 ± 10% perf-stat.ps.dTLB-store-misses 2.133e+09 ± 3% +19.8% 2.556e+09 ± 2% perf-stat.ps.dTLB-stores 13717699 ± 3% -28.4% 9816979 ± 3% perf-stat.ps.iTLB-loads 1.159e+10 ± 3% +21.5% 1.408e+10 perf-stat.ps.instructions 15979371 ± 9% -30.7% 11073468 ± 3% perf-stat.ps.node-load-misses 8081205 ± 9% -31.3% 5554114 perf-stat.ps.node-loads 2580198 ± 13% -31.1% 1777169 ± 6% perf-stat.ps.node-store-misses 1801590 ± 12% -31.3% 1238321 ± 2% perf-stat.ps.node-stores 2.579e+12 +20.6% 3.111e+12 ± 2% perf-stat.total.instructions 36149 ± 4% -26.2% 26686 ± 5% softirqs.CPU0.RCU 37255 ± 3% -26.8% 27256 ± 2% softirqs.CPU1.RCU 37834 ± 5% -26.5% 27801 ± 9% softirqs.CPU10.RCU 39075 ± 3% -24.5% 29494 ± 2% softirqs.CPU11.RCU 38137 ± 6% -23.1% 29340 ± 4% softirqs.CPU12.RCU 37480 ± 3% -20.5% 29797 ± 2% softirqs.CPU13.RCU 37690 ± 5% -28.7% 26861 ± 4% softirqs.CPU14.RCU 33863 ± 9% -29.5% 23885 ± 5% softirqs.CPU15.RCU 34306 ± 9% -31.1% 23641 ± 12% softirqs.CPU16.RCU 34374 ± 9% -29.0% 24402 ± 7% softirqs.CPU17.RCU 44013 ± 9% -31.7% 30059 softirqs.CPU18.RCU 43114 ± 8% -29.9% 30219 softirqs.CPU19.RCU 36276 ± 8% -23.3% 27825 ± 5% softirqs.CPU2.RCU 67049 ± 7% +39.1% 93267 ± 18% softirqs.CPU2.TIMER 39721 ± 13% -31.4% 27233 ± 3% softirqs.CPU20.RCU 42340 ± 5% -28.6% 30215 ± 6% softirqs.CPU21.RCU 40896 ± 7% -30.3% 28506 ± 2% softirqs.CPU22.RCU 42422 ± 7% -33.2% 28357 ± 2% softirqs.CPU23.RCU 42330 ± 9% -33.5% 28150 ± 5% softirqs.CPU24.RCU 41406 ± 9% -31.4% 28398 softirqs.CPU25.RCU 39754 ± 5% -29.4% 28063 softirqs.CPU26.RCU 40207 ± 6% -27.9% 28995 ± 12% softirqs.CPU27.RCU 40351 ± 9% -29.4% 28487 ± 3% softirqs.CPU28.RCU 41880 ± 5% -29.6% 29472 softirqs.CPU29.RCU 36293 ± 4% -17.7% 29864 ± 7% softirqs.CPU3.RCU 39341 ± 9% -34.3% 25847 softirqs.CPU30.RCU 39687 ± 9% -33.5% 26400 ± 2% softirqs.CPU31.RCU 38062 ± 14% -39.9% 22878 ± 10% softirqs.CPU32.RCU 39769 ± 13% -33.8% 26314 ± 5% softirqs.CPU33.RCU 39504 ± 10% -37.2% 24824 ± 6% softirqs.CPU34.RCU 39591 ± 11% -38.6% 24300 ± 5% softirqs.CPU35.RCU 40398 ± 8% -27.0% 29488 ± 12% softirqs.CPU36.RCU 39464 ± 12% -25.3% 29478 ± 10% softirqs.CPU37.RCU 41177 ± 9% -34.0% 27190 ± 19% softirqs.CPU38.RCU 38734 ± 13% -24.8% 29145 ± 5% softirqs.CPU39.RCU 36804 ± 5% -23.0% 28354 softirqs.CPU4.RCU 40421 ± 12% -28.8% 28776 ± 6% softirqs.CPU40.RCU 40371 ± 11% -31.1% 27834 ± 8% softirqs.CPU41.RCU 40633 ± 10% -29.9% 28469 ± 10% softirqs.CPU42.RCU 41018 ± 9% -33.6% 27252 ± 6% softirqs.CPU43.RCU 38181 ± 12% -27.0% 27856 ± 10% softirqs.CPU44.RCU 39605 ± 12% -32.3% 26803 ± 12% softirqs.CPU45.RCU 42686 ± 11% -29.7% 30016 ± 9% softirqs.CPU47.RCU 40433 ± 9% -31.8% 27586 ± 11% softirqs.CPU48.RCU 38371 ± 12% -23.0% 29528 ± 7% softirqs.CPU49.RCU 37172 ± 2% -25.9% 27562 ± 3% softirqs.CPU5.RCU 39660 ± 12% -27.5% 28752 ± 11% softirqs.CPU50.RCU 40375 ± 10% -26.7% 29582 ± 3% softirqs.CPU51.RCU 40568 ± 9% -28.8% 28886 ± 6% softirqs.CPU52.RCU 39967 ± 10% -27.3% 29073 ± 11% softirqs.CPU53.RCU 40874 ± 11% -37.1% 25725 ± 6% softirqs.CPU54.RCU 41227 ± 14% -37.9% 25606 ± 6% softirqs.CPU55.RCU 38856 ± 13% -34.7% 25380 ± 8% softirqs.CPU56.RCU 39618 ± 13% -39.1% 24119 ± 4% softirqs.CPU57.RCU 39412 ± 12% -36.6% 24970 ± 3% softirqs.CPU58.RCU 41572 ± 14% -41.3% 24399 ± 10% softirqs.CPU59.RCU 36927 ± 3% -24.4% 27923 ± 22% softirqs.CPU6.RCU 37057 ± 10% -31.8% 25261 ± 2% softirqs.CPU60.RCU 36584 ± 8% -27.8% 26419 ± 2% softirqs.CPU61.RCU 35859 ± 4% -28.2% 25729 ± 5% softirqs.CPU62.RCU 35849 ± 7% -33.2% 23933 ± 5% softirqs.CPU63.RCU 35379 ± 6% -26.4% 26041 ± 4% softirqs.CPU64.RCU 36908 ± 6% -26.5% 27110 ± 2% softirqs.CPU65.RCU 37595 ± 3% -26.1% 27783 ± 4% softirqs.CPU66.RCU 36327 ± 2% -27.0% 26533 ± 2% softirqs.CPU67.RCU 35136 ± 7% -31.7% 23998 ± 9% softirqs.CPU68.RCU 35638 ± 4% -28.6% 25435 ± 4% softirqs.CPU69.RCU 37268 ± 6% -25.4% 27786 ± 4% softirqs.CPU7.RCU 36693 ± 4% -29.2% 25986 ± 5% softirqs.CPU70.RCU 36859 ± 3% -28.8% 26260 ± 4% softirqs.CPU71.RCU 37539 ± 7% -27.7% 27144 ± 4% softirqs.CPU8.RCU 37584 ± 4% -26.0% 27802 ± 15% softirqs.CPU9.RCU 2795313 ± 7% -29.7% 1964363 ± 4% softirqs.RCU 249.50 ± 76% -50.4% 123.67 ± 2% interrupts.41:PCI-MSI.1572870-edge.eth0-TxRx-6 154.50 ± 13% -10.9% 137.67 ± 10% interrupts.42:PCI-MSI.1572871-edge.eth0-TxRx-7 150.50 ± 43% -28.0% 108.33 interrupts.84:PCI-MSI.1572913-edge.eth0-TxRx-49 10393 ± 10% -26.4% 7653 ± 10% interrupts.CPU1.RES:Rescheduling_interrupts 12518 ± 15% -31.0% 8642 ± 21% interrupts.CPU10.RES:Rescheduling_interrupts 945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.NMI:Non-maskable_interrupts 945.25 ± 79% -75.4% 232.67 ±141% interrupts.CPU11.PMI:Performance_monitoring_interrupts 13068 ± 18% -30.6% 9075 ± 23% interrupts.CPU11.RES:Rescheduling_interrupts 11949 ± 11% -49.8% 6001 ± 29% interrupts.CPU12.RES:Rescheduling_interrupts 13711 ± 8% -19.4% 11052 ± 24% interrupts.CPU13.RES:Rescheduling_interrupts 13361 ± 6% -38.8% 8170 ± 25% interrupts.CPU14.RES:Rescheduling_interrupts 7628 ± 12% -29.2% 5401 ± 7% interrupts.CPU18.RES:Rescheduling_interrupts 7909 ± 25% -30.4% 5501 ± 37% interrupts.CPU19.RES:Rescheduling_interrupts 1482 ± 39% -100.0% 0.00 interrupts.CPU2.NMI:Non-maskable_interrupts 1482 ± 39% -100.0% 0.00 interrupts.CPU2.PMI:Performance_monitoring_interrupts 2116 +23.2% 2606 ± 10% interrupts.CPU20.CAL:Function_call_interrupts 194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.NMI:Non-maskable_interrupts 194.75 ±173% +380.1% 935.00 ± 30% interrupts.CPU20.PMI:Performance_monitoring_interrupts 2096 ± 4% +18.0% 2474 ± 9% interrupts.CPU21.CAL:Function_call_interrupts 11313 ± 20% -56.4% 4935 ± 23% interrupts.CPU21.RES:Rescheduling_interrupts 2111 ± 3% +20.1% 2534 ± 11% interrupts.CPU22.CAL:Function_call_interrupts 2109 ± 4% +22.6% 2585 ± 10% interrupts.CPU23.CAL:Function_call_interrupts 10305 ± 16% -40.7% 6115 ± 17% interrupts.CPU23.RES:Rescheduling_interrupts 2128 ± 2% +17.8% 2507 ± 10% interrupts.CPU24.CAL:Function_call_interrupts 12271 ± 20% -39.0% 7488 ± 29% interrupts.CPU24.RES:Rescheduling_interrupts 2123 +19.7% 2541 ± 7% interrupts.CPU25.CAL:Function_call_interrupts 9780 ± 21% -32.9% 6563 ± 25% interrupts.CPU25.RES:Rescheduling_interrupts 980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.NMI:Non-maskable_interrupts 980.75 ± 89% -88.9% 109.00 ±141% interrupts.CPU26.PMI:Performance_monitoring_interrupts 11163 ± 28% -49.8% 5605 ± 13% interrupts.CPU27.RES:Rescheduling_interrupts 11071 ± 12% -56.3% 4835 ± 26% interrupts.CPU29.RES:Rescheduling_interrupts 728.50 ± 66% -100.0% 0.00 interrupts.CPU3.NMI:Non-maskable_interrupts 728.50 ± 66% -100.0% 0.00 interrupts.CPU3.PMI:Performance_monitoring_interrupts 10599 ± 10% -38.7% 6501 ± 16% interrupts.CPU31.RES:Rescheduling_interrupts 2213 ± 4% +14.2% 2527 ± 7% interrupts.CPU32.CAL:Function_call_interrupts 8862 ± 16% -39.6% 5354 ± 28% interrupts.CPU32.RES:Rescheduling_interrupts 2156 ± 6% +20.5% 2599 ± 9% interrupts.CPU33.CAL:Function_call_interrupts 2169 ± 5% +19.9% 2601 ± 9% interrupts.CPU34.CAL:Function_call_interrupts 10483 ± 18% -48.8% 5371 ± 33% interrupts.CPU34.RES:Rescheduling_interrupts 2049 +24.4% 2549 ± 8% interrupts.CPU35.CAL:Function_call_interrupts 10929 ± 15% -49.0% 5578 ± 14% interrupts.CPU35.RES:Rescheduling_interrupts 1993 ± 4% +28.5% 2561 ± 12% interrupts.CPU36.CAL:Function_call_interrupts 161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.NMI:Non-maskable_interrupts 161.75 ±161% +717.9% 1323 ± 43% interrupts.CPU36.PMI:Performance_monitoring_interrupts 1987 ± 4% +30.5% 2594 ± 13% interrupts.CPU37.CAL:Function_call_interrupts 1863 ± 10% +35.7% 2529 ± 14% interrupts.CPU38.CAL:Function_call_interrupts 1984 ± 6% +26.5% 2509 ± 12% interrupts.CPU41.CAL:Function_call_interrupts 12830 ± 30% -26.4% 9445 ± 9% interrupts.CPU43.RES:Rescheduling_interrupts 2067 ± 2% +27.4% 2632 ± 9% interrupts.CPU44.CAL:Function_call_interrupts 2066 ± 3% +27.2% 2629 ± 9% interrupts.CPU45.CAL:Function_call_interrupts 2158 ± 3% +19.6% 2582 ± 9% interrupts.CPU46.CAL:Function_call_interrupts 2036 ± 6% +30.3% 2652 ± 10% interrupts.CPU47.CAL:Function_call_interrupts 150.50 ± 43% -28.0% 108.33 interrupts.CPU49.84:PCI-MSI.1572913-edge.eth0-TxRx-49 11823 ± 12% -36.9% 7460 ± 18% interrupts.CPU49.RES:Rescheduling_interrupts 1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.NMI:Non-maskable_interrupts 1139 ± 58% -75.5% 278.67 ±141% interrupts.CPU5.PMI:Performance_monitoring_interrupts 11748 ± 22% -40.0% 7048 ± 27% interrupts.CPU5.RES:Rescheduling_interrupts 2102 ± 5% +19.6% 2515 ± 7% interrupts.CPU52.CAL:Function_call_interrupts 2115 ± 6% +19.5% 2527 ± 7% interrupts.CPU53.CAL:Function_call_interrupts 11318 ± 17% -20.4% 9004 ± 26% interrupts.CPU53.RES:Rescheduling_interrupts 2177 ± 5% +17.4% 2556 ± 9% interrupts.CPU54.CAL:Function_call_interrupts 10968 ± 23% -62.3% 4134 ± 21% interrupts.CPU54.RES:Rescheduling_interrupts 2161 ± 5% +20.2% 2598 ± 9% interrupts.CPU55.CAL:Function_call_interrupts 9334 ± 19% -55.4% 4158 ± 34% interrupts.CPU55.RES:Rescheduling_interrupts 1443 ± 22% -100.0% 0.00 interrupts.CPU56.NMI:Non-maskable_interrupts 1443 ± 22% -100.0% 0.00 interrupts.CPU56.PMI:Performance_monitoring_interrupts 11377 ± 31% -56.9% 4907 ± 43% interrupts.CPU56.RES:Rescheduling_interrupts 601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.NMI:Non-maskable_interrupts 601.00 ± 99% -79.6% 122.67 ±141% interrupts.CPU57.PMI:Performance_monitoring_interrupts 8295 ± 31% -47.7% 4340 ± 24% interrupts.CPU57.RES:Rescheduling_interrupts 9182 ± 37% -59.7% 3701 ± 46% interrupts.CPU58.RES:Rescheduling_interrupts 249.50 ± 76% -50.4% 123.67 ± 2% interrupts.CPU6.41:PCI-MSI.1572870-edge.eth0-TxRx-6 12473 ± 27% -44.4% 6940 ± 13% interrupts.CPU6.RES:Rescheduling_interrupts 7886 ± 12% -42.2% 4555 ± 31% interrupts.CPU60.RES:Rescheduling_interrupts 971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.NMI:Non-maskable_interrupts 971.50 ± 47% -86.8% 128.33 ±141% interrupts.CPU61.PMI:Performance_monitoring_interrupts 9253 ± 26% -56.4% 4033 ± 16% interrupts.CPU61.RES:Rescheduling_interrupts 2103 ± 5% +19.1% 2505 ± 11% interrupts.CPU63.CAL:Function_call_interrupts 2059 ± 2% +20.6% 2483 ± 10% interrupts.CPU64.CAL:Function_call_interrupts 8545 ± 17% -41.2% 5025 ± 8% interrupts.CPU65.RES:Rescheduling_interrupts 7378 ± 23% -51.0% 3614 ± 30% interrupts.CPU68.RES:Rescheduling_interrupts 8573 ± 28% -50.1% 4277 ± 6% interrupts.CPU69.RES:Rescheduling_interrupts 154.50 ± 13% -10.9% 137.67 ± 10% interrupts.CPU7.42:PCI-MSI.1572871-edge.eth0-TxRx-7 9910 ± 28% -51.2% 4839 ± 40% interrupts.CPU70.RES:Rescheduling_interrupts 8836 ± 26% -52.7% 4180 ± 6% interrupts.CPU71.RES:Rescheduling_interrupts 911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.NMI:Non-maskable_interrupts 911.50 ± 37% -93.2% 61.67 ±141% interrupts.CPU8.PMI:Performance_monitoring_interrupts 12326 ± 15% -32.3% 8350 ± 10% interrupts.CPU8.RES:Rescheduling_interrupts 1392 ± 58% -100.0% 0.00 interrupts.CPU9.NMI:Non-maskable_interrupts 1392 ± 58% -100.0% 0.00 interrupts.CPU9.PMI:Performance_monitoring_interrupts 60185 ± 19% -30.5% 41822 ± 11% interrupts.NMI:Non-maskable_interrupts 60185 ± 19% -30.5% 41822 ± 11% interrupts.PMI:Performance_monitoring_interrupts 749419 -31.9% 510225 ± 3% interrupts.RES:Rescheduling_interrupts 15.05 ± 6% -4.1 10.97 ± 20% perf-profile.calltrace.cycles-pp.normal_work_helper.process_one_work.worker_thread.kthread.ret_from_fork 12.66 ± 5% -4.0 8.68 ± 20% perf-profile.calltrace.cycles-pp.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread.kthread 5.99 ± 8% -1.6 4.37 ± 6% perf-profile.calltrace.cycles-pp.io_submit 5.83 ± 8% -1.6 4.25 ± 6% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.io_submit 5.82 ± 8% -1.6 4.24 ± 6% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit 5.81 ± 8% -1.6 4.23 ± 6% perf-profile.calltrace.cycles-pp.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit 5.76 ± 8% -1.6 4.19 ± 6% perf-profile.calltrace.cycles-pp.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe.io_submit 5.65 ± 6% -1.6 4.08 ± 20% perf-profile.calltrace.cycles-pp.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread 4.52 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work.worker_thread 6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback 6.90 ± 8% -1.5 5.38 ± 10% perf-profile.calltrace.cycles-pp.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode.writeback_sb_inodes 5.58 ± 8% -1.5 4.07 ± 6% perf-profile.calltrace.cycles-pp.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64.entry_SYSCALL_64_after_hwframe 4.50 ± 3% -1.5 2.99 ± 19% perf-profile.calltrace.cycles-pp.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper.process_one_work 5.44 ± 8% -1.5 3.96 ± 7% perf-profile.calltrace.cycles-pp.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit.do_syscall_64 6.63 ± 8% -1.5 5.17 ± 10% perf-profile.calltrace.cycles-pp.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages.__writeback_single_inode 5.34 ± 9% -1.4 3.90 ± 6% perf-profile.calltrace.cycles-pp.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one.__x64_sys_io_submit 5.55 ± 8% -1.2 4.32 ± 10% perf-profile.calltrace.cycles-pp.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages 4.98 ± 8% -1.1 3.86 ± 10% perf-profile.calltrace.cycles-pp.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages 5.00 ± 8% -1.1 3.88 ± 10% perf-profile.calltrace.cycles-pp.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage.extent_write_cache_pages.extent_writepages 2.80 ± 4% -1.0 1.78 ± 22% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper.process_one_work 1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.__btrfs_cow_block.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io 1.45 ± 3% -0.7 0.79 ± 18% perf-profile.calltrace.cycles-pp.btrfs_cow_block.btrfs_search_slot.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper 2.32 ± 3% -0.6 1.72 ± 18% perf-profile.calltrace.cycles-pp.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper 0.91 ± 8% -0.5 0.42 ± 73% perf-profile.calltrace.cycles-pp.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io.normal_work_helper 0.88 ± 8% -0.5 0.41 ± 74% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csum.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io 0.83 ± 5% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums 1.90 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread 1.89 ± 20% -0.4 1.48 ± 29% perf-profile.calltrace.cycles-pp.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space.process_one_work 0.82 ± 4% -0.4 0.42 ± 71% perf-profile.calltrace.cycles-pp.run_one_async_done.normal_work_helper.process_one_work.worker_thread.kthread 0.81 ± 4% -0.4 0.41 ± 71% perf-profile.calltrace.cycles-pp.btrfs_map_bio.run_one_async_done.normal_work_helper.process_one_work.worker_thread 0.74 ± 6% -0.4 0.36 ± 70% perf-profile.calltrace.cycles-pp.push_leaf_right.split_leaf.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks 1.19 ± 3% -0.4 0.83 ± 16% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_insert_empty_items.btrfs_csum_file_blocks.add_pending_csums.btrfs_finish_ordered_io 1.00 ± 6% -0.4 0.65 ± 7% perf-profile.calltrace.cycles-pp.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one 1.27 ± 5% -0.4 0.92 ± 7% perf-profile.calltrace.cycles-pp.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one 1.25 ± 6% -0.3 0.90 ± 7% perf-profile.calltrace.cycles-pp.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage 0.93 ± 8% -0.3 0.59 ± 7% perf-profile.calltrace.cycles-pp.__set_extent_bit.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter 0.94 ± 7% -0.3 0.60 ± 6% perf-profile.calltrace.cycles-pp.lock_extent_bits.lock_and_cleanup_extent_if_need.btrfs_buffered_write.btrfs_file_write_iter.aio_write 0.70 ± 8% -0.3 0.37 ± 70% perf-profile.calltrace.cycles-pp.__lookup_extent_mapping.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter 0.79 ± 6% -0.3 0.47 ± 71% perf-profile.calltrace.cycles-pp.setup_items_for_insert.btrfs_duplicate_item.btrfs_mark_extent_written.btrfs_finish_ordered_io.normal_work_helper 0.71 ± 21% -0.3 0.43 ± 74% perf-profile.calltrace.cycles-pp.__btrfs_free_extent.__btrfs_run_delayed_refs.btrfs_run_delayed_refs.flush_space.btrfs_async_reclaim_metadata_space 1.15 ± 11% -0.3 0.88 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc 1.15 ± 11% -0.3 0.88 ± 8% perf-profile.calltrace.cycles-pp.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage 1.43 ± 13% -0.3 1.17 ± 5% perf-profile.calltrace.cycles-pp.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write.io_submit_one 1.06 ± 5% -0.3 0.81 ± 7% perf-profile.calltrace.cycles-pp.btrfs_drop_extent_cache.create_io_em.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc 1.30 ± 15% -0.2 1.06 ± 4% perf-profile.calltrace.cycles-pp.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter.aio_write 1.03 ± 13% -0.2 0.79 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_csums_range.csum_exist_in_range.run_delalloc_nocow.btrfs_run_delalloc_range 1.20 ± 4% -0.2 0.96 ± 15% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary 1.24 ± 3% -0.2 1.01 ± 15% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 1.05 ± 8% -0.2 0.82 ± 12% perf-profile.calltrace.cycles-pp.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages.do_writepages 0.88 ± 13% -0.2 0.71 ± 2% perf-profile.calltrace.cycles-pp.wait_reserve_ticket.reserve_metadata_bytes.btrfs_delalloc_reserve_metadata.btrfs_buffered_write.btrfs_file_write_iter 1.01 ± 9% -0.2 0.84 ± 9% perf-profile.calltrace.cycles-pp.btrfs_search_slot.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc 1.01 ± 10% -0.2 0.85 ± 9% perf-profile.calltrace.cycles-pp.btrfs_lookup_file_extent.run_delalloc_nocow.btrfs_run_delalloc_range.writepage_delalloc.__extent_writepage 0.72 ± 7% -0.2 0.56 ± 6% perf-profile.calltrace.cycles-pp.btrfs_get_extent.btrfs_dirty_pages.btrfs_buffered_write.btrfs_file_write_iter.aio_write 0.75 ± 8% -0.1 0.60 ± 10% perf-profile.calltrace.cycles-pp.submit_extent_page.__extent_writepage_io.__extent_writepage.extent_write_cache_pages.extent_writepages 0.92 ± 15% +0.2 1.14 ± 3% perf-profile.calltrace.cycles-pp.update_process_times.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt 1.01 ± 15% +0.2 1.25 ± 4% perf-profile.calltrace.cycles-pp.tick_sched_handle.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt 1.15 ± 15% +0.3 1.44 ± 6% perf-profile.calltrace.cycles-pp.tick_sched_timer.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt 0.27 ±100% +0.4 0.63 ± 4% perf-profile.calltrace.cycles-pp.rebalance_domains.__softirqentry_text_start.irq_exit.smp_apic_timer_interrupt.apic_timer_interrupt 1.74 ± 9% +0.4 2.11 ± 2% perf-profile.calltrace.cycles-pp.__hrtimer_run_queues.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state 2.63 ± 10% +0.6 3.26 perf-profile.calltrace.cycles-pp.hrtimer_interrupt.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle 0.00 +0.8 0.75 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook 2.37 ± 19% +0.8 3.19 ± 5% perf-profile.calltrace.cycles-pp.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread.ret_from_fork 2.33 ± 19% +0.8 3.17 ± 5% perf-profile.calltrace.cycles-pp.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread.kthread 0.00 +0.9 0.90 ± 18% perf-profile.calltrace.cycles-pp.map_private_extent_buffer.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook 4.71 ± 11% +1.0 5.74 perf-profile.calltrace.cycles-pp.smp_apic_timer_interrupt.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry 5.04 ± 10% +1.2 6.19 perf-profile.calltrace.cycles-pp.apic_timer_interrupt.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 0.00 +1.3 1.30 ± 19% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range 0.00 +1.4 1.36 ± 19% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents 0.29 ±101% +1.4 1.67 ± 18% perf-profile.calltrace.cycles-pp.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work.worker_thread 0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction 0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction 0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.__filemap_fdatawrite_range.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space 0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space.process_one_work 0.00 +1.4 1.39 ± 19% perf-profile.calltrace.cycles-pp.btrfs_write_marked_extents.btrfs_write_and_wait_transaction.btrfs_commit_transaction.flush_space.btrfs_async_reclaim_metadata_space 0.00 +1.5 1.52 ± 18% perf-profile.calltrace.cycles-pp.btrfs_get_token_64.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio 0.00 +2.2 2.20 ± 14% perf-profile.calltrace.cycles-pp.btrfs_get_token_32.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_writeback.wb_workfn.process_one_work.worker_thread.kthread 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.calltrace.cycles-pp.wb_workfn.process_one_work.worker_thread.kthread.ret_from_fork 7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn 7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work.worker_thread 7.97 ± 7% +3.5 11.44 ± 15% perf-profile.calltrace.cycles-pp.__writeback_single_inode.writeback_sb_inodes.wb_writeback.wb_workfn.process_one_work 49.45 ± 2% +4.3 53.79 ± 4% perf-profile.calltrace.cycles-pp.intel_idle.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary 61.98 ± 2% +4.4 66.38 ± 3% perf-profile.calltrace.cycles-pp.secondary_startup_64 1.07 ± 18% +5.0 6.06 ± 20% perf-profile.calltrace.cycles-pp.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes.wb_writeback 0.93 ± 18% +5.0 5.95 ± 20% perf-profile.calltrace.cycles-pp.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode.writeback_sb_inodes 61.26 ± 2% +5.1 66.36 ± 3% perf-profile.calltrace.cycles-pp.cpu_startup_entry.start_secondary.secondary_startup_64 61.27 ± 2% +5.1 66.37 ± 3% perf-profile.calltrace.cycles-pp.start_secondary.secondary_startup_64 61.21 ± 2% +5.1 66.32 ± 3% perf-profile.calltrace.cycles-pp.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.33 ±100% +5.3 5.68 ± 20% perf-profile.calltrace.cycles-pp.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages.__writeback_single_inode 55.45 ± 2% +5.4 60.88 ± 3% perf-profile.calltrace.cycles-pp.cpuidle_enter_state.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64 0.00 +6.4 6.39 ± 13% perf-profile.calltrace.cycles-pp.check_leaf.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page 0.00 +6.5 6.50 ± 13% perf-profile.calltrace.cycles-pp.btree_csum_one_bio.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb 0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages.do_writepages 0.29 ±100% +6.6 6.92 ± 13% perf-profile.calltrace.cycles-pp.btree_submit_bio_hook.submit_one_bio.submit_extent_page.write_one_eb.btree_write_cache_pages 15.05 ± 6% -4.1 10.97 ± 20% perf-profile.children.cycles-pp.normal_work_helper 12.66 ± 5% -4.0 8.68 ± 20% perf-profile.children.cycles-pp.btrfs_finish_ordered_io 7.84 ± 5% -2.4 5.39 ± 10% perf-profile.children.cycles-pp.btrfs_search_slot 7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_writepages 7.30 ± 7% -1.7 5.59 ± 11% perf-profile.children.cycles-pp.extent_write_cache_pages 7.03 ± 7% -1.6 5.38 ± 12% perf-profile.children.cycles-pp.__extent_writepage 6.19 ± 8% -1.6 4.56 ± 7% perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe 6.17 ± 8% -1.6 4.55 ± 7% perf-profile.children.cycles-pp.do_syscall_64 6.01 ± 8% -1.6 4.39 ± 7% perf-profile.children.cycles-pp.io_submit 5.81 ± 8% -1.6 4.23 ± 6% perf-profile.children.cycles-pp.__x64_sys_io_submit 5.76 ± 8% -1.6 4.19 ± 6% perf-profile.children.cycles-pp.io_submit_one 5.66 ± 5% -1.6 4.09 ± 20% perf-profile.children.cycles-pp.btrfs_mark_extent_written 4.52 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.add_pending_csums 5.59 ± 8% -1.5 4.07 ± 6% perf-profile.children.cycles-pp.aio_write 4.51 ± 3% -1.5 2.99 ± 19% perf-profile.children.cycles-pp.btrfs_csum_file_blocks 5.44 ± 8% -1.5 3.96 ± 7% perf-profile.children.cycles-pp.btrfs_file_write_iter 5.34 ± 9% -1.4 3.90 ± 6% perf-profile.children.cycles-pp.btrfs_buffered_write 5.88 ± 7% -1.4 4.47 ± 11% perf-profile.children.cycles-pp.writepage_delalloc 5.30 ± 8% -1.3 4.01 ± 11% perf-profile.children.cycles-pp.btrfs_run_delalloc_range 5.28 ± 8% -1.3 3.99 ± 11% perf-profile.children.cycles-pp.run_delalloc_nocow 2.55 ± 2% -0.9 1.67 ± 16% perf-profile.children.cycles-pp.btrfs_cow_block 2.54 ± 2% -0.9 1.66 ± 16% perf-profile.children.cycles-pp.__btrfs_cow_block 2.15 ± 13% -0.8 1.31 ± 8% perf-profile.children.cycles-pp.__lookup_extent_mapping 1.87 ± 12% -0.8 1.04 ± 12% perf-profile.children.cycles-pp.__etree_search 0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.printk 0.94 ± 51% -0.7 0.22 ± 34% perf-profile.children.cycles-pp.vprintk_emit 2.60 ± 2% -0.7 1.92 ± 14% perf-profile.children.cycles-pp.btrfs_insert_empty_items 3.00 ± 5% -0.7 2.33 ± 6% perf-profile.children.cycles-pp._raw_spin_lock 1.89 ± 2% -0.7 1.23 ± 7% perf-profile.children.cycles-pp.__set_extent_bit 2.78 ± 4% -0.6 2.14 ± 14% perf-profile.children.cycles-pp.__schedule 1.57 ± 2% -0.5 1.02 ± 9% perf-profile.children.cycles-pp.lock_extent_bits 1.19 ± 14% -0.5 0.70 ± 11% perf-profile.children.cycles-pp.__clear_extent_bit 1.51 ± 7% -0.5 1.04 ± 9% perf-profile.children.cycles-pp.read_block_for_search 2.08 ± 5% -0.5 1.62 ± 18% perf-profile.children.cycles-pp.setup_items_for_insert 2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.btrfs_run_delayed_refs 2.08 ± 17% -0.4 1.67 ± 24% perf-profile.children.cycles-pp.__btrfs_run_delayed_refs 1.60 ± 6% -0.4 1.20 ± 12% perf-profile.children.cycles-pp.schedule 1.95 ± 11% -0.4 1.58 ± 12% perf-profile.children.cycles-pp.try_to_wake_up 1.31 ± 6% -0.4 0.94 ± 9% perf-profile.children.cycles-pp.create_io_em 0.99 ± 4% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.alloc_tree_block_no_bg_flush 1.00 ± 6% -0.4 0.65 ± 7% perf-profile.children.cycles-pp.lock_and_cleanup_extent_if_need 1.27 ± 9% -0.4 0.92 ± 11% perf-profile.children.cycles-pp.__wake_up_common_lock 0.98 ± 3% -0.4 0.63 ± 13% perf-profile.children.cycles-pp.btrfs_alloc_tree_block 1.27 ± 5% -0.4 0.92 ± 7% perf-profile.children.cycles-pp.btrfs_dirty_pages 1.34 ± 7% -0.3 0.99 ± 10% perf-profile.children.cycles-pp.btrfs_map_bio 1.11 ± 9% -0.3 0.77 ± 11% perf-profile.children.cycles-pp.generic_bin_search 1.16 ± 6% -0.3 0.82 ± 9% perf-profile.children.cycles-pp.find_extent_buffer 0.91 ± 7% -0.3 0.57 ± 21% perf-profile.children.cycles-pp.btrfs_lookup_csum 1.21 ± 11% -0.3 0.90 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_csums_range 1.21 ± 11% -0.3 0.91 ± 9% perf-profile.children.cycles-pp.csum_exist_in_range 1.10 ± 10% -0.3 0.80 ± 9% perf-profile.children.cycles-pp.__wake_up_common 0.37 ± 70% -0.3 0.07 ± 18% perf-profile.children.cycles-pp.test_range_bit 1.61 ± 4% -0.3 1.32 ± 15% perf-profile.children.cycles-pp.push_leaf_right 0.60 ± 4% -0.3 0.31 ± 25% perf-profile.children.cycles-pp.btrfs_extend_item 1.02 ± 9% -0.3 0.73 ± 10% perf-profile.children.cycles-pp.autoremove_wake_function 1.12 ± 4% -0.3 0.84 ± 9% perf-profile.children.cycles-pp.btrfs_drop_extent_cache 0.82 ± 6% -0.3 0.55 ± 4% perf-profile.children.cycles-pp.btrfs_release_path 0.85 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_extent_buffer_full 0.99 ± 4% -0.3 0.73 ± 13% perf-profile.children.cycles-pp.btrfs_set_token_32 1.44 ± 13% -0.3 1.18 ± 5% perf-profile.children.cycles-pp.btrfs_delalloc_reserve_metadata 0.84 ± 8% -0.3 0.59 ± 17% perf-profile.children.cycles-pp.copy_page 0.87 ± 11% -0.3 0.62 ± 11% perf-profile.children.cycles-pp.pagecache_get_page 0.84 ± 7% -0.2 0.60 ± 11% perf-profile.children.cycles-pp.kmem_cache_alloc 1.12 ± 7% -0.2 0.88 ± 17% perf-profile.children.cycles-pp.__extent_writepage_io 1.30 ± 15% -0.2 1.06 ± 5% perf-profile.children.cycles-pp.reserve_metadata_bytes 0.82 ± 4% -0.2 0.58 ± 13% perf-profile.children.cycles-pp.run_one_async_done 1.25 ± 3% -0.2 1.01 ± 15% perf-profile.children.cycles-pp.schedule_idle 0.34 ± 53% -0.2 0.10 ± 28% perf-profile.children.cycles-pp.delay_tsc 0.70 ± 13% -0.2 0.47 ± 11% perf-profile.children.cycles-pp.prepare_pages 0.50 ± 22% -0.2 0.28 ± 13% perf-profile.children.cycles-pp.clear_state_bit 0.84 ± 5% -0.2 0.63 ± 6% perf-profile.children.cycles-pp.btrfs_get_extent 0.54 ± 11% -0.2 0.34 ± 4% perf-profile.children.cycles-pp.btrfs_free_path 1.16 ± 11% -0.2 0.96 ± 19% perf-profile.children.cycles-pp.native_queued_spin_lock_slowpath 0.53 ± 9% -0.2 0.33 ± 18% perf-profile.children.cycles-pp.unpin_extent_cache 1.07 ± 10% -0.2 0.88 ± 10% perf-profile.children.cycles-pp.btrfs_lookup_file_extent 0.50 ± 4% -0.2 0.31 ± 15% perf-profile.children.cycles-pp.clear_extent_bit 0.45 ± 13% -0.2 0.26 ± 11% perf-profile.children.cycles-pp.kmem_cache_free 0.40 ± 14% -0.2 0.21 ± 50% perf-profile.children.cycles-pp.__writeback_inodes_wb 0.64 ± 7% -0.2 0.45 ± 10% perf-profile.children.cycles-pp.dequeue_task_fair 0.50 ± 8% -0.2 0.33 ± 23% perf-profile.children.cycles-pp.btrfs_add_delayed_tree_ref 0.88 ± 13% -0.2 0.71 ± 2% perf-profile.children.cycles-pp.wait_reserve_ticket 0.52 ± 9% -0.2 0.35 ± 7% perf-profile.children.cycles-pp.__radix_tree_lookup 0.52 ± 8% -0.2 0.36 ± 8% perf-profile.children.cycles-pp.xas_load 0.78 ± 19% -0.2 0.62 ± 23% perf-profile.children.cycles-pp.__btrfs_free_extent 0.56 ± 17% -0.1 0.42 ± 15% perf-profile.children.cycles-pp.btrfs_del_items 0.51 ± 5% -0.1 0.36 ± 9% perf-profile.children.cycles-pp.dequeue_entity 0.39 ± 7% -0.1 0.24 ± 6% perf-profile.children.cycles-pp.__set_page_dirty_nobuffers 0.82 ± 14% -0.1 0.68 ± 18% perf-profile.children.cycles-pp.ttwu_do_activate 0.55 ± 8% -0.1 0.41 ± 8% perf-profile.children.cycles-pp.btrfs_submit_bio_hook 0.53 ± 5% -0.1 0.40 ± 13% perf-profile.children.cycles-pp.update_load_avg 0.40 ± 8% -0.1 0.27 ± 27% perf-profile.children.cycles-pp.add_delayed_ref_head 0.35 ± 13% -0.1 0.22 ± 3% perf-profile.children.cycles-pp.set_extent_bit 0.52 ± 9% -0.1 0.39 ± 7% perf-profile.children.cycles-pp.btrfs_wq_submit_bio 0.35 ± 3% -0.1 0.23 ± 32% perf-profile.children.cycles-pp.queued_write_lock_slowpath 0.33 ± 6% -0.1 0.21 ± 15% perf-profile.children.cycles-pp.__slab_alloc 0.63 ± 19% -0.1 0.51 ± 13% perf-profile.children.cycles-pp.sched_ttwu_pending 0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode 0.31 ± 11% -0.1 0.19 ± 28% perf-profile.children.cycles-pp.btrfs_update_inode_fallback 0.57 ± 5% -0.1 0.46 ± 10% perf-profile.children.cycles-pp.find_lock_delalloc_range 0.45 ± 5% -0.1 0.34 ± 9% perf-profile.children.cycles-pp.btrfs_cross_ref_exist 0.29 ± 3% -0.1 0.18 ± 7% perf-profile.children.cycles-pp.free_extent_buffer 0.38 ± 3% -0.1 0.26 ± 8% perf-profile.children.cycles-pp.btrfs_mark_buffer_dirty 0.46 ± 5% -0.1 0.35 ± 13% perf-profile.children.cycles-pp.find_delalloc_range 0.41 ± 9% -0.1 0.30 ± 2% perf-profile.children.cycles-pp.btrfs_copy_from_user 0.40 ± 9% -0.1 0.29 ± 2% perf-profile.children.cycles-pp.iov_iter_copy_from_user_atomic 0.38 ± 10% -0.1 0.27 perf-profile.children.cycles-pp.find_get_entry 0.31 ± 12% -0.1 0.20 ± 31% perf-profile.children.cycles-pp.btrfs_remove_ordered_extent 0.30 ± 6% -0.1 0.20 ± 14% perf-profile.children.cycles-pp.___slab_alloc 0.26 ± 8% -0.1 0.16 ± 7% perf-profile.children.cycles-pp.merge_state 0.38 ± 7% -0.1 0.28 ± 3% perf-profile.children.cycles-pp.copyin 0.26 ± 17% -0.1 0.15 ± 26% perf-profile.children.cycles-pp.btrfs_delayed_update_inode 0.25 ± 4% -0.1 0.15 ± 5% perf-profile.children.cycles-pp.insert_state 0.40 ± 13% -0.1 0.30 ± 13% perf-profile.children.cycles-pp.extent_clear_unlock_delalloc 0.36 ± 9% -0.1 0.26 ± 3% perf-profile.children.cycles-pp.alloc_extent_state 0.38 ± 7% -0.1 0.29 ± 3% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string 0.39 ± 3% -0.1 0.29 ± 14% perf-profile.children.cycles-pp.alloc_extent_buffer 0.31 ± 4% -0.1 0.22 ± 11% perf-profile.children.cycles-pp.set_extent_buffer_dirty 0.26 ± 11% -0.1 0.17 ± 23% perf-profile.children.cycles-pp.unlock_up 0.32 ± 3% -0.1 0.24 ± 13% perf-profile.children.cycles-pp.__btrfs_map_block 0.23 ± 13% -0.1 0.15 ± 28% perf-profile.children.cycles-pp.btrfs_add_delayed_data_ref 0.25 ± 12% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.__slab_free 0.47 ± 6% -0.1 0.39 ± 15% perf-profile.children.cycles-pp.__push_leaf_right 0.26 ± 9% -0.1 0.18 ± 19% perf-profile.children.cycles-pp.__test_set_page_writeback 0.32 ± 11% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.btrfs_add_ordered_extent 0.15 ± 8% -0.1 0.07 ± 6% perf-profile.children.cycles-pp.__xa_set_mark 0.23 ± 7% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.release_extent_buffer 0.18 ± 8% -0.1 0.11 ± 26% perf-profile.children.cycles-pp.alloc_extent_map 0.32 ± 10% -0.1 0.24 ± 27% perf-profile.children.cycles-pp.__btrfs_add_ordered_extent 0.32 ± 17% -0.1 0.24 ± 10% perf-profile.children.cycles-pp.btrfs_tree_read_lock_atomic 0.20 ± 13% -0.1 0.13 ± 27% perf-profile.children.cycles-pp.btrfs_inc_extent_ref 0.14 ± 21% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.find_ref_head 0.13 ± 32% -0.1 0.06 ± 19% perf-profile.children.cycles-pp.rb_erase_cached 0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_submit_bio_start 0.53 ± 12% -0.1 0.46 ± 16% perf-profile.children.cycles-pp.btrfs_tree_read_lock 0.21 ± 7% -0.1 0.14 ± 15% perf-profile.children.cycles-pp.btrfs_csum_one_bio 0.17 ± 16% -0.1 0.10 ± 8% perf-profile.children.cycles-pp.rb_erase 0.21 ± 7% -0.1 0.14 ± 16% perf-profile.children.cycles-pp.run_one_async_start 0.22 ± 3% -0.1 0.15 ± 22% perf-profile.children.cycles-pp.btrfs_try_tree_write_lock 0.21 ± 9% -0.1 0.15 ± 21% perf-profile.children.cycles-pp.btrfs_leaf_free_space 0.15 ± 23% -0.1 0.09 ± 35% perf-profile.children.cycles-pp.free_extent_map 0.20 ± 9% -0.1 0.14 ± 20% perf-profile.children.cycles-pp.leaf_space_used 0.19 ± 8% -0.1 0.13 ± 13% perf-profile.children.cycles-pp.check_delayed_ref 0.17 ± 13% -0.1 0.12 ± 31% perf-profile.children.cycles-pp.get_page_from_freelist 0.13 ± 8% -0.1 0.08 ± 26% perf-profile.children.cycles-pp.__update_load_avg_se 0.17 ± 2% -0.1 0.12 ± 18% perf-profile.children.cycles-pp.btrfs_set_range_writeback 0.15 ± 8% -0.1 0.09 ± 20% perf-profile.children.cycles-pp.xas_set_mark 0.18 ± 6% -0.1 0.13 ± 16% perf-profile.children.cycles-pp.___might_sleep 0.20 ± 11% -0.1 0.15 ± 35% perf-profile.children.cycles-pp.__alloc_pages_nodemask 0.10 ± 9% -0.1 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_unlock 0.14 ± 11% -0.1 0.09 ± 28% perf-profile.children.cycles-pp.btrfs_reserve_extent 0.10 ± 33% -0.1 0.05 ± 72% perf-profile.children.cycles-pp.__btrfs_release_delayed_node 0.22 ± 9% -0.0 0.17 ± 25% perf-profile.children.cycles-pp.mark_extent_buffer_accessed 0.09 ± 13% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_tree_read_unlock_blocking 0.18 ± 8% -0.0 0.14 ± 24% perf-profile.children.cycles-pp.__list_del_entry_valid 0.09 ± 13% -0.0 0.04 ± 73% perf-profile.children.cycles-pp.verify_parent_transid 0.11 ± 19% -0.0 0.07 ± 11% perf-profile.children.cycles-pp.entry_SYSCALL_64 0.22 ± 5% -0.0 0.18 ± 12% perf-profile.children.cycles-pp.check_committed_ref 0.11 ± 19% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.select_idle_sibling 0.14 ± 11% -0.0 0.09 ± 5% perf-profile.children.cycles-pp.do_io_getevents 0.15 ± 7% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.__x64_sys_io_getevents 0.08 ± 10% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.btrfs_release_extent_buffer_pages 0.17 ± 18% -0.0 0.13 ± 9% perf-profile.children.cycles-pp.clear_page_dirty_for_io 0.14 ± 5% -0.0 0.10 ± 25% perf-profile.children.cycles-pp.block_group_cache_tree_search 0.18 ± 4% -0.0 0.14 ± 11% perf-profile.children.cycles-pp.btrfs_get_chunk_map 0.11 ± 13% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.btrfs_buffer_uptodate 0.15 ± 15% -0.0 0.11 ± 19% perf-profile.children.cycles-pp.xas_find_marked 0.12 ± 15% -0.0 0.08 ± 20% perf-profile.children.cycles-pp.btrfs_unlock_up_safe 0.12 ± 14% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.btrfs_set_path_blocking 0.11 ± 11% -0.0 0.07 ± 28% perf-profile.children.cycles-pp.find_free_extent 0.24 ± 5% -0.0 0.20 ± 6% perf-profile.children.cycles-pp.btrfs_root_node 0.07 ± 6% -0.0 0.03 ± 70% perf-profile.children.cycles-pp.extent_mergeable 0.13 ± 12% -0.0 0.10 ± 9% perf-profile.children.cycles-pp.__switch_to 0.10 ± 18% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.rb_insert_color_cached 0.07 ± 14% -0.0 0.04 ± 71% perf-profile.children.cycles-pp.rb_prev 0.11 ± 11% -0.0 0.08 ± 12% perf-profile.children.cycles-pp.get_io_u 0.10 ± 14% -0.0 0.07 ± 23% perf-profile.children.cycles-pp.available_idle_cpu 0.17 ± 10% -0.0 0.13 ± 3% perf-profile.children.cycles-pp._raw_write_lock 0.10 ± 11% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.__might_sleep 0.17 ± 7% -0.0 0.14 ± 12% perf-profile.children.cycles-pp._raw_read_lock 0.17 ± 10% -0.0 0.14 ± 12% perf-profile.children.cycles-pp.can_overcommit 0.10 ± 11% -0.0 0.07 perf-profile.children.cycles-pp.__list_add_valid 0.09 ± 7% -0.0 0.06 ± 7% perf-profile.children.cycles-pp.aio_read_events 0.11 ± 10% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.___perf_sw_event 0.11 ± 9% -0.0 0.09 ± 18% perf-profile.children.cycles-pp.get_alloc_profile 0.11 ± 12% -0.0 0.08 ± 14% perf-profile.children.cycles-pp.pick_next_task_idle 0.10 ± 13% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.read_events 0.08 ± 6% -0.0 0.06 ± 13% perf-profile.children.cycles-pp.replace_extent_mapping 0.10 ± 5% -0.0 0.08 ± 10% perf-profile.children.cycles-pp.menu_reflect 0.07 ± 17% +0.0 0.09 ± 5% perf-profile.children.cycles-pp.run_posix_cpu_timers 0.05 ± 58% +0.0 0.08 ± 6% perf-profile.children.cycles-pp.rcu_irq_enter 0.19 ± 11% +0.1 0.26 ± 17% perf-profile.children.cycles-pp.native_irq_return_iret 0.20 ± 14% +0.1 0.27 ± 8% perf-profile.children.cycles-pp.btrfs_comp_cpu_keys 0.38 ± 9% +0.1 0.46 perf-profile.children.cycles-pp.load_balance 0.30 ± 8% +0.1 0.38 ± 15% perf-profile.children.cycles-pp.irq_enter 0.21 ± 24% +0.1 0.30 ± 9% perf-profile.children.cycles-pp.rcu_sched_clock_irq 0.22 ± 3% +0.1 0.32 ± 9% perf-profile.children.cycles-pp.lapic_next_deadline 0.52 ± 12% +0.1 0.63 ± 3% perf-profile.children.cycles-pp.rebalance_domains 1.90 ± 10% +0.3 2.23 ± 2% perf-profile.children.cycles-pp.__hrtimer_run_queues 0.23 ± 12% +0.4 0.67 ± 15% perf-profile.children.cycles-pp.read_extent_buffer 2.88 ± 11% +0.5 3.41 perf-profile.children.cycles-pp.hrtimer_interrupt 0.09 ± 12% +0.7 0.83 ± 15% perf-profile.children.cycles-pp.btrfs_get_token_8 2.37 ± 19% +0.8 3.19 ± 5% perf-profile.children.cycles-pp.btrfs_async_reclaim_metadata_space 2.33 ± 19% +0.8 3.17 ± 5% perf-profile.children.cycles-pp.flush_space 0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_and_wait_transaction 0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.btrfs_write_marked_extents 0.14 ± 34% +1.3 1.39 ± 19% perf-profile.children.cycles-pp.__filemap_fdatawrite_range 0.42 ± 43% +1.3 1.67 ± 18% perf-profile.children.cycles-pp.btrfs_commit_transaction 0.25 ± 5% +1.9 2.17 ± 10% perf-profile.children.cycles-pp.btrfs_get_token_64 1.71 ± 5% +2.1 3.79 ± 5% perf-profile.children.cycles-pp.btrfs_get_token_32 0.43 ± 8% +2.4 2.84 ± 10% perf-profile.children.cycles-pp.map_private_extent_buffer 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_writeback 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.writeback_sb_inodes 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.__writeback_single_inode 8.37 ± 7% +3.3 11.65 ± 16% perf-profile.children.cycles-pp.wb_workfn 50.06 ± 2% +3.8 53.82 ± 4% perf-profile.children.cycles-pp.intel_idle 61.99 ± 2% +4.4 66.39 ± 3% perf-profile.children.cycles-pp.do_idle 61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.secondary_startup_64 61.98 ± 2% +4.4 66.38 ± 3% perf-profile.children.cycles-pp.cpu_startup_entry 8.51 ± 7% +4.5 13.04 ± 12% perf-profile.children.cycles-pp.do_writepages 56.43 ± 3% +4.9 61.30 ± 3% perf-profile.children.cycles-pp.cpuidle_enter_state 61.27 ± 2% +5.1 66.37 ± 3% perf-profile.children.cycles-pp.start_secondary 1.43 ± 7% +6.2 7.64 ± 13% perf-profile.children.cycles-pp.submit_extent_page 1.20 ± 17% +6.2 7.45 ± 13% perf-profile.children.cycles-pp.btree_write_cache_pages 1.05 ± 17% +6.3 7.31 ± 13% perf-profile.children.cycles-pp.write_one_eb 1.11 ± 7% +6.3 7.37 ± 13% perf-profile.children.cycles-pp.submit_one_bio 0.55 ± 16% +6.4 6.95 ± 13% perf-profile.children.cycles-pp.btree_submit_bio_hook 0.00 +6.5 6.48 ± 13% perf-profile.children.cycles-pp.check_leaf 0.00 +6.5 6.53 ± 13% perf-profile.children.cycles-pp.btree_csum_one_bio 2.12 ± 11% -0.8 1.29 ± 7% perf-profile.self.cycles-pp.__lookup_extent_mapping 1.83 ± 12% -0.8 1.03 ± 13% perf-profile.self.cycles-pp.__etree_search 2.06 ± 5% -0.5 1.59 ± 5% perf-profile.self.cycles-pp._raw_spin_lock 0.81 ± 9% -0.3 0.52 ± 10% perf-profile.self.cycles-pp.generic_bin_search 0.84 ± 7% -0.3 0.58 ± 17% perf-profile.self.cycles-pp.copy_page 0.33 ± 54% -0.2 0.10 ± 28% perf-profile.self.cycles-pp.delay_tsc 0.89 ± 9% -0.2 0.69 ± 14% perf-profile.self.cycles-pp.btrfs_set_token_32 0.81 ± 4% -0.2 0.63 ± 14% perf-profile.self.cycles-pp.__schedule 0.51 ± 9% -0.2 0.35 ± 6% perf-profile.self.cycles-pp.__radix_tree_lookup 0.45 ± 8% -0.1 0.32 ± 10% perf-profile.self.cycles-pp.xas_load 0.43 ± 4% -0.1 0.31 ± 6% perf-profile.self.cycles-pp.find_extent_buffer 0.77 ± 5% -0.1 0.66 ± 11% perf-profile.self.cycles-pp._raw_spin_lock_irqsave 0.31 ± 10% -0.1 0.20 ± 28% perf-profile.self.cycles-pp.add_delayed_ref_head 0.39 ± 16% -0.1 0.29 ± 16% perf-profile.self.cycles-pp.try_to_wake_up 0.31 ± 4% -0.1 0.20 ± 26% perf-profile.self.cycles-pp.queued_write_lock_slowpath 0.38 ± 7% -0.1 0.29 ± 3% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string 0.20 ± 17% -0.1 0.11 ± 14% perf-profile.self.cycles-pp.kmem_cache_free 0.25 ± 11% -0.1 0.17 ± 2% perf-profile.self.cycles-pp.__slab_free 0.16 ± 9% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.free_extent_buffer 0.28 ± 17% -0.1 0.20 ± 11% perf-profile.self.cycles-pp.btrfs_search_slot 0.31 ± 4% -0.1 0.24 ± 20% perf-profile.self.cycles-pp.kmem_cache_alloc 0.14 ± 21% -0.1 0.06 ± 19% perf-profile.self.cycles-pp.find_ref_head 0.16 ± 10% -0.1 0.09 ± 5% perf-profile.self.cycles-pp.insert_state 0.12 ± 8% -0.1 0.06 ± 72% perf-profile.self.cycles-pp.__update_load_avg_se 0.16 ± 15% -0.1 0.10 ± 8% perf-profile.self.cycles-pp.rb_erase 0.10 ± 9% -0.1 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_unlock 0.15 ± 13% -0.1 0.10 ± 12% perf-profile.self.cycles-pp.___slab_alloc 0.28 ± 13% -0.1 0.22 ± 5% perf-profile.self.cycles-pp.queued_read_lock_slowpath 0.15 ± 21% -0.1 0.09 ± 35% perf-profile.self.cycles-pp.free_extent_map 0.18 ± 6% -0.1 0.12 ± 11% perf-profile.self.cycles-pp.update_load_avg 0.10 ± 11% -0.1 0.04 ± 70% perf-profile.self.cycles-pp.btrfs_extend_item 0.15 ± 23% -0.1 0.09 ± 9% perf-profile.self.cycles-pp.add_extent_mapping 0.15 ± 8% -0.1 0.09 ± 20% perf-profile.self.cycles-pp.xas_set_mark 0.18 ± 6% -0.1 0.13 ± 16% perf-profile.self.cycles-pp.___might_sleep 0.18 ± 6% -0.0 0.13 ± 22% perf-profile.self.cycles-pp.__list_del_entry_valid 0.18 ± 4% -0.0 0.14 ± 22% perf-profile.self.cycles-pp.pick_next_task_fair 0.09 ± 13% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.verify_parent_transid 0.09 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.btrfs_tree_read_unlock_blocking 0.16 ± 13% -0.0 0.12 ± 24% perf-profile.self.cycles-pp.mark_page_accessed 0.12 ± 14% -0.0 0.08 ± 20% perf-profile.self.cycles-pp.dequeue_task_fair 0.10 ± 19% -0.0 0.07 ± 18% perf-profile.self.cycles-pp.entry_SYSCALL_64 0.08 ± 14% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.unlock_up 0.15 ± 15% -0.0 0.11 ± 19% perf-profile.self.cycles-pp.xas_find_marked 0.24 ± 6% -0.0 0.20 ± 8% perf-profile.self.cycles-pp.btrfs_root_node 0.17 ± 6% -0.0 0.13 ± 10% perf-profile.self.cycles-pp._raw_read_lock 0.10 ± 13% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.btrfs_csum_one_bio 0.16 ± 7% -0.0 0.13 ± 9% perf-profile.self.cycles-pp.dequeue_entity 0.07 -0.0 0.04 ± 71% perf-profile.self.cycles-pp.io_u_queued_complete 0.08 -0.0 0.05 ± 70% perf-profile.self.cycles-pp.mutex_lock 0.11 ± 11% -0.0 0.08 ± 12% perf-profile.self.cycles-pp.get_io_u 0.16 ± 9% -0.0 0.13 perf-profile.self.cycles-pp._raw_write_lock 0.07 ± 17% -0.0 0.04 ± 73% perf-profile.self.cycles-pp.btrfs_tree_read_lock 0.12 ± 8% -0.0 0.09 ± 13% perf-profile.self.cycles-pp.__switch_to 0.10 ± 11% -0.0 0.06 ± 14% perf-profile.self.cycles-pp.__clear_extent_bit 0.07 ± 12% -0.0 0.04 ± 71% perf-profile.self.cycles-pp.rb_prev 0.10 ± 15% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.find_get_pages_range_tag 0.10 ± 12% -0.0 0.07 ± 23% perf-profile.self.cycles-pp.available_idle_cpu 0.10 ± 11% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.__list_add_valid 0.08 ± 13% -0.0 0.06 ± 8% perf-profile.self.cycles-pp.__might_sleep 0.12 ± 17% -0.0 0.09 ± 15% perf-profile.self.cycles-pp.release_extent_buffer 0.09 ± 17% -0.0 0.07 ± 14% perf-profile.self.cycles-pp.___perf_sw_event 0.08 ± 5% -0.0 0.06 ± 13% perf-profile.self.cycles-pp.__push_leaf_right 0.11 ± 7% -0.0 0.09 ± 9% perf-profile.self.cycles-pp.set_extent_buffer_dirty 0.11 ± 4% -0.0 0.08 ± 11% perf-profile.self.cycles-pp.__set_extent_bit 0.09 ± 7% +0.0 0.11 ± 4% perf-profile.self.cycles-pp.hrtimer_interrupt 0.07 ± 17% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.run_posix_cpu_timers 0.07 ± 7% +0.0 0.09 ± 5% perf-profile.self.cycles-pp.load_balance 0.01 ±173% +0.1 0.07 ± 17% perf-profile.self.cycles-pp.lapic_next_deadline 0.00 +0.1 0.06 ± 23% perf-profile.self.cycles-pp.find_next_and_bit 0.19 ± 11% +0.1 0.26 ± 17% perf-profile.self.cycles-pp.native_irq_return_iret 0.19 ± 24% +0.1 0.28 ± 7% perf-profile.self.cycles-pp.rcu_sched_clock_irq 0.50 ± 8% +0.2 0.69 ± 17% perf-profile.self.cycles-pp.cpuidle_enter_state 0.07 ± 11% +0.4 0.46 ± 17% perf-profile.self.cycles-pp.btrfs_get_token_8 0.22 ± 13% +0.4 0.66 ± 16% perf-profile.self.cycles-pp.read_extent_buffer 0.00 +0.6 0.65 ± 14% perf-profile.self.cycles-pp.check_leaf 0.15 ± 11% +1.0 1.13 ± 11% perf-profile.self.cycles-pp.btrfs_get_token_64 1.48 ± 7% +1.0 2.51 ± 5% perf-profile.self.cycles-pp.btrfs_get_token_32 0.41 ± 8% +2.2 2.61 ± 9% perf-profile.self.cycles-pp.map_private_extent_buffer 49.99 ± 2% +3.7 53.74 ± 4% perf-profile.self.cycles-pp.intel_idle fio.write_bw_MBps 300 +-+-------------------------------------------------------------------+ | | 250 +-++.+ +. .+.. .+..+..+.+.. .+..+.. .+ | | : : +. +..+ + +..+ +. + | | : : : : : : | 200 +-+ : : : : : : O O | O O O: O :O O O O O O O O O O: O:O O: O: O O O O O 150 +-+ : : : : : : | | : : : : : : | 100 +-+ : : : : : : | | : : : : : : | | : : : : : : | 50 +-+ :: :: :: | | : : : | 0 +-+---------------------------------------------O-------------O-------+ fio.write_iops 70000 +-+-----------------------------------------------------------------+ |..+.+ +..+..+.+.. .+.+..+.+..+ +.. +..+.+..+..+ | 60000 +-+ : : +. : : + : | | : : : : : : | 50000 +-+ : : : : : : | O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O 40000 +-+ : O: : : : : | | : : : : : : | 30000 +-+ : : : : : : | | : : : : : : | 20000 +-+ : : : : : : | | :: :: :: | 10000 +-+ : : : | | : : : | 0 +-+--------------------------------------------O------------O-------+ fio.write_clat_mean_us 30000 +-+-----------------------------------------------------------------+ | | 25000 O-+O O O O O O O O O O O O O O O O O O O | | O O O O O | | 20000 +-+ | |..+.+ +..+..+.+..+..+.+..+.+..+ +..+ +..+.+..+..+ | 15000 +-+ : : : : : : | | : : : : : : | 10000 +-+ : : : : : : | | : : : : : : | | : : : : : : | 5000 +-+ : : : : :: | | : : : | 0 +-+--------------------------------------------O------------O-------+ fio.write_clat_stddev 160000 +-+----------------------------------------------------------------+ | | 140000 O-+O O O O O O O O O O O O O O | 120000 +-+ O O | | | 100000 +-+ O O O O O O O O | | 80000 +-+ | | .+.. | 60000 +-++.+ +..+.+. +.+..+.+..+..+ +.+ +..+..+.+..+ | 40000 +-+ : : : : : : | | : : : : : : | 20000 +-+ : : : : : : | | :: :: :: | 0 +-+-------------------------------------------O------------O-------+ fio.write_slat_mean_us 900 +-+-------------------------------------------------------------------+ O O O O O O O O O O O O O O O O O O | 800 +-+O O O O O O O 700 +-+ | | | 600 +-++.+ +.+..+..+..+.+..+..+.+..+ +..+ +..+..+..+.+ | 500 +-+ : : : : : : | | : : : : : : | 400 +-+ : : : : : : | 300 +-+ : : : : : : | | : : : : : : | 200 +-+ : : : : : : | 100 +-+ : : :: :: | | : : : | 0 +-+---------------------------------------------O-------------O-------+ fio.write_slat_stddev 25000 +-+-----O---------O-------------------------------------------------+ O O O O O O O O O O O O O O O | | | 20000 +-+ | | O O O O O O O O | | 15000 +-+ | | | 10000 +-++. .+.. .+.. .+..+ .+ .+..+ | |. + +..+..+ +..+ + : +. : +..+.+. | | : : : : : : | 5000 +-+ : : : : : : | | : : : : : : | | :: : :: | 0 +-+--------------------------------------------O------------O-------+ fio.latency_1000ms_ 0.4 +-+------------------------------------------------------------------+ | | 0.35 +-+ O | 0.3 +-+ | | O O | 0.25 +-+ O O | | O O O 0.2 O-+ O O O O O O O O O O | | O O | 0.15 +-+O O O O | 0.1 +-+ | | | 0.05 +-+ | | .+. .+..+..+.+..+.. .+. .+.+..+.. | 0 +-+---------------------------------------------O------------O-------+ fio.workload 1.4e+07 +-+---------------------------------------------------------------+ |..+.+ +..+.+..+.+..+.+..+.+..+ +.. +.+..+.+..+ | 1.2e+07 +-+ : : : : + : | | : : : : : : | 1e+07 +-+ : : : : : : | O O O: :O O O O O O O O O O: O:O O:O : O O O O O O O 8e+06 +-+ : O: : : : : | | : : : : : : | 6e+06 +-+ : : : : : : | | : : : : : : | 4e+06 +-+ : : : : : : | | :: :: :: | 2e+06 +-+ : : : | | : : : | 0 +-+-------------------------------------------O-----------O-------+ fio.time.voluntary_context_switches 1.4e+07 +-+---------------------------------------------------------------+ | | 1.2e+07 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ | | : : : : : : | 1e+07 +-+ : : : : : : | | O O: : O O O : : :O : O O O O O O O 8e+06 O-+ : O:O O O O O O O: O:O O: : | | : : : : : : | 6e+06 +-+ : : : : : : | | : : : : : : | 4e+06 +-+ : : : : : : | | : : : : :: | 2e+06 +-+ : : :: | | : : : | 0 +-+-------------------------------------------O-----------O-------+ fio.time.file_system_outputs 1.2e+08 +-+---------------------------------------------------------------+ | | 1e+08 +-++.+ +..+.+..+.+..+.+..+.+..+ +..+ +.+..+.+..+ | | : : : : : : | | : : : : : : | 8e+07 +-+ : : : : : : O O O O O O O O: O:O O O O O O O O O O: O:O O:O : O O | 6e+07 +-+ : : : : : : | | : : : : : : | 4e+07 +-+ : : : : : : | | : : : : : : | | : : : : :: | 2e+07 +-+ : : :: | | : : : | 0 +-+-------------------------------------------O-----------O-------+ [*] bisect-good sample [O] bisect-bad sample Disclaimer: Results have been estimated based on internal Intel analysis and are provided for informational purposes only. Any difference in system hardware or software design or configuration may affect actual performance. Thanks, Rong Chen

9 months, 1 week

4
4
0 / 0