[sched/fair] 152b0b5300: UBSAN:shift-out-of-bounds_in_kernel/sched/fair.c
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 152b0b530094bc4aa9f2ba10e2046a1cf7c1cd25 ("sched/fair: reduce cases for active balance")
https://git.linaro.org/people/vincent.guittot/kernel.git sched/pelt
in testcase: trinity
version: trinity-static-x86_64-x86_64-1c734c75-1_2020-01-06
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+-------------------------------------------------------+------------+------------+
| | feff2e65ef | 152b0b5300 |
+-------------------------------------------------------+------------+------------+
| boot_successes | 143 | 141 |
| boot_failures | 7 | 21 |
| INFO:rcu_sched_self-detected_stall_on_CPU | 2 | 1 |
| RIP:iov_iter_copy_from_user_atomic | 1 | 1 |
| BUG:soft_lockup-CPU##stuck_for#s![trinity-c4:#] | 2 | 1 |
| Kernel_panic-not_syncing | 2 | 1 |
| IP-Config:Auto-configuration_of_network_failed | 1 | 1 |
| BUG:workqueue_lockup-pool | 4 | |
| RIP:__asan_load8 | 1 | |
| RIP:__asan_load4 | 1 | |
| UBSAN:shift-out-of-bounds_in_kernel/sched/fair.c | 0 | 18 |
| RIP:lock_is_held_type | 0 | 1 |
| WARNING:possible_circular_locking_dependency_detected | 0 | 17 |
| RIP:_raw_spin_unlock_irq | 0 | 1 |
| WARNING:at_fs/read_write.c:#vfs_copy_file_range | 0 | 1 |
| RIP:vfs_copy_file_range | 0 | 1 |
| RIP:__slab_free | 0 | 1 |
| RIP:default_idle | 0 | 4 |
| RIP:rcutorture_one_extend | 0 | 1 |
| RIP:unwind_next_frame | 0 | 1 |
| RIP:__x64_sys_setrlimit | 0 | 1 |
| BUG:KASAN:double-free_or_invalid-free_in_s | 0 | 1 |
| RIP:mutex_spin_on_owner | 0 | 1 |
| RIP:begin_new_exec | 0 | 1 |
| WARNING:at_net/sched/sch_generic.c:#dev_watchdog | 0 | 1 |
| RIP:dev_watchdog | 0 | 1 |
| RIP:clear_page_rep | 0 | 1 |
+-------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 169.729452] UBSAN: shift-out-of-bounds in kernel/sched/fair.c:7683:14
[ 169.731792] shift exponent 188 is too large for 64-bit type 'long unsigned int'
[ 169.733890] CPU: 1 PID: 2259 Comm: trinity-c5 Not tainted 5.9.0-rc1-00141-g152b0b530094b #1
[ 169.736206] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 169.738580] Call Trace:
[ 169.742120] <IRQ>
[ 169.742877] dump_stack+0x9e/0xe0
[ 169.743930] ubsan_epilogue+0x5/0x40
[ 169.745032] __ubsan_handle_shift_out_of_bounds.cold+0x53/0x100
[ 169.749683] ? can_migrate_task+0x410/0x600
[ 169.750945] load_balance.cold+0x18/0x24
[ 169.752100] ? __lock_acquire+0x817/0xef0
[ 169.753381] ? find_busiest_group+0x4e0/0x4e0
[ 169.754675] ? rcu_read_lock_held+0xaa/0xc0
[ 169.755916] ? rcu_read_lock_sched_held+0xe0/0xe0
[ 169.757273] rebalance_domains+0x4c6/0x860
[ 169.758545] ? load_balance+0x1880/0x1880
[ 169.759783] ? _raw_spin_unlock_irqrestore+0x39/0x40
[ 169.761238] ? trace_hardirqs_on+0x1e/0x140
[ 169.762504] __do_softirq+0x100/0x6b6
[ 169.763622] asm_call_on_stack+0xf/0x20
[ 169.764782] </IRQ>
[ 169.765549] do_softirq_own_stack+0x59/0x70
[ 169.766778] irq_exit_rcu+0xb2/0xd0
[ 169.767876] sysvec_apic_timer_interrupt+0x43/0xa0
[ 169.769332] asm_sysvec_apic_timer_interrupt+0x12/0x20
[ 169.770825] RIP: 0010:unwind_next_frame+0x23a/0xb40
[ 169.772263] Code: 4d 85 f6 75 14 48 8d 7b 34 49 c7 c6 40 55 35 83 e8 cb b3 30 00 c6 43 34 01 4d 8d 7e 04 4c 89 ff e8 4b b3 30 00 41 f6 46 04 0f <0f> 84 b7 00 00 00 be 02 00 00 00 4c 89 ff e8 43 bb 30 00 41 0f b6
[ 169.777030] RSP: 0018:ffff8881d8e87a30 EFLAGS: 00000206
[ 169.781848] RAX: 0000000000000000 RBX: ffff8881d8e87b30 RCX: ffffffff81083915
[ 169.783650] RDX: 1ffffffff083c876 RSI: ffffffff841e433c RDI: ffffffff841e43b2
[ 169.785586] RBP: 1ffff1103b1d0f4f R08: ffffffff81083323 R09: fffffbfff083c842
[ 169.787561] R10: 0000000000014010 R11: fffffbfff083c841 R12: 0000000000000001
[ 169.789550] R13: ffff8881d8e87b78 R14: ffffffff841e43ae R15: ffffffff841e43b2
[ 169.791488] ? __orc_find+0x63/0xc0
[ 169.792566] ? unwind_next_frame+0x235/0xb40
[ 169.793851] ? __kasan_kmalloc+0xc2/0xd0
[ 169.795301] ? get_reg+0xd0/0xd0
[ 169.796315] ? check_prevs_add+0x13a0/0x13a0
[ 169.797608] ? __unwind_start+0x2f3/0x370
[ 169.798831] ? stack_trace_save+0xc0/0xc0
[ 169.800057] arch_stack_walk+0x81/0xf0
[ 169.801206] ? __kasan_kmalloc+0xc2/0xd0
[ 169.802647] stack_trace_save+0x8c/0xc0
[ 169.803854] ? irqentry_exit_cond_resched+0x30/0x30
[ 169.805295] ? lock_downgrade+0x370/0x370
[ 169.806492] kasan_save_stack+0x1b/0x40
[ 169.807605] ? kasan_save_stack+0x1b/0x40
[ 169.808746] ? __kasan_kmalloc+0xc2/0xd0
[ 169.810110] ? do_raw_spin_unlock+0x9e/0x130
[ 169.811386] ? _raw_spin_unlock+0x1a/0x30
[ 169.812577] ? deactivate_slab+0x7c6/0x800
[ 169.813856] ? fsnotify_alloc_group+0x30/0x180
[ 169.815173] ? set_track+0x4a/0xc0
[ 169.816256] ? init_object+0x49/0x80
[ 169.817403] ? alloc_debug_processing+0x42/0x160
[ 169.818782] ? ___slab_alloc+0x6b3/0x970
[ 169.820196] ? fsnotify_alloc_group+0x30/0x180
[ 169.821532] ? mark_held_locks+0x23/0x90
[ 169.822717] ? fsnotify_alloc_group+0x30/0x180
[ 169.824022] ? lockdep_hardirqs_on_prepare+0x155/0x250
[ 169.825508] ? __slab_alloc+0x4b/0x60
[ 169.826868] ? kasan_unpoison_shadow+0x33/0x40
[ 169.828183] __kasan_kmalloc+0xc2/0xd0
[ 169.829592] ? fsnotify_alloc_group+0x30/0x180
[ 169.830865] kmem_cache_alloc_trace+0xdc/0x2e0
[ 169.832162] fsnotify_alloc_group+0x30/0x180
[ 169.833463] do_inotify_init+0x2c/0x280
[ 169.834554] __x64_sys_inotify_init1+0x16/0x20
[ 169.835797] do_syscall_64+0x2d/0x40
[ 169.836892] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 169.838398] RIP: 0033:0x463519
[ 169.839384] Code: 00 f3 c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 0f 83 db 59 00 00 c3 66 2e 0f 1f 84 00 00 00 00
[ 169.844369] RSP: 002b:00007ffe787d3648 EFLAGS: 00000246 ORIG_RAX: 0000000000000126
[ 169.846436] RAX: ffffffffffffffda RBX: 0000000000000126 RCX: 0000000000463519
[ 169.848395] RDX: 00000000ffffefff RSI: 00e3cb3139591d83 RDI: 0000000000000000
[ 169.850374] RBP: 00007f6f4f7f4000 R08: 0000000022000028 R09: 28a81d29208010a8
[ 169.852333] R10: 000000000000f774 R11: 0000000000000246 R12: 0000000000000002
[ 169.854317] R13: 00007f6f4f7f4058 R14: 000000000109a850 R15: 00007f6f4f7f4000
[ 169.856329] ================================================================================
[ 169.858715]
[ 169.858719] ======================================================
[ 169.858721] WARNING: possible circular locking dependency detected
[ 169.858724] 5.9.0-rc1-00141-g152b0b530094b #1 Not tainted
[ 169.858727] ------------------------------------------------------
[ 169.858729] trinity-c5/2259 is trying to acquire lock:
[ 169.858730] ffffffff833e95a0 (console_owner){-.-.}-{0:0}, at: console_unlock+0x201/0x730
[ 169.858738]
[ 169.858740] but task is already holding lock:
[ 169.858742] ffff8881e99f8018 (&rq->lock){-.-.}-{2:2}, at: load_balance+0x7c7/0x1880
[ 169.858749]
[ 169.858751] which lock already depends on the new lock.
[ 169.858753]
[ 169.858754]
[ 169.858756] the existing dependency chain (in reverse order) is:
[ 169.858758]
[ 169.858759] -> #4 (&rq->lock){-.-.}-{2:2}:
[ 169.858766] __lock_acquire+0x7f3/0xef0
[ 169.858768] lock_acquire+0x15d/0x550
[ 169.858770] _raw_spin_lock+0x2c/0x70
[ 169.858772] task_fork_fair+0x2e/0x2b0
[ 169.858773] sched_fork+0x146/0x2c0
[ 169.858776] copy_process+0xdd5/0x2ef0
[ 169.858777] _do_fork+0xf4/0x840
[ 169.858779] kernel_thread+0xa3/0xe0
[ 169.858781] rest_init+0x1e/0x2fa
[ 169.858783] start_kernel+0x3a5/0x3c3
[ 169.858785] secondary_startup_64+0xb6/0xc0
[ 169.858786]
[ 169.858787] -> #3 (&p->pi_lock){-.-.}-{2:2}:
[ 169.858795] __lock_acquire+0x7f3/0xef0
[ 169.858797] lock_acquire+0x15d/0x550
[ 169.858799] _raw_spin_lock_irqsave+0x37/0x80
[ 169.858801] try_to_wake_up+0xa2/0x1020
[ 169.858804] autoremove_wake_function+0x10/0x70
[ 169.858806] __wake_up_common+0xbe/0x250
[ 169.858808] __wake_up_common_lock+0xd0/0x130
[ 169.858810] tty_port_default_wakeup+0x16/0x30
[ 169.858812] serial8250_tx_chars+0x20d/0x3e0
[ 169.858815] serial8250_handle_irq+0xdf/0x170
[ 169.858816] serial8250_interrupt+0x83/0xd0
[ 169.858819] __handle_irq_event_percpu+0x80/0x4e0
[ 169.858821] handle_irq_event_percpu+0x6a/0xf0
[ 169.858823] handle_irq_event+0x50/0x7f
[ 169.858825] handle_edge_irq+0xfd/0x380
[ 169.858826] asm_call_on_stack+0xf/0x20
[ 169.858828] common_interrupt+0xea/0x1a0
[ 169.858830] asm_common_interrupt+0x1e/0x40
[ 169.858832] lock_acquire+0x18d/0x550
[ 169.858834] rcu_torture_read_lock+0x28/0x80
[ 169.858836] rcutorture_one_extend+0x26a/0x4b0
[ 169.858838] rcu_torture_one_read+0x30e/0x670
[ 169.858840] rcu_torture_reader+0x13b/0x2d0
[ 169.858842] kthread+0x1f4/0x220
[ 169.858844] ret_from_fork+0x1f/0x30
[ 169.858846]
[ 169.858847] -> #2 (&tty->write_wait){-.-.}-{2:2}:
[ 169.858855] __lock_acquire+0x7f3/0xef0
[ 169.858857] lock_acquire+0x15d/0x550
[ 169.858859] _raw_spin_lock_irqsave+0x37/0x80
[ 169.858861] __wake_up_common_lock+0xb4/0x130
[ 169.858863] tty_port_default_wakeup+0x16/0x30
[ 169.858865] serial8250_tx_chars+0x20d/0x3e0
[ 169.858867] serial8250_handle_irq+0xdf/0x170
[ 169.858869] serial8250_interrupt+0x83/0xd0
[ 169.858872] __handle_irq_event_percpu+0x80/0x4e0
[ 169.858874] handle_irq_event_percpu+0x6a/0xf0
[ 169.858876] handle_irq_event+0x50/0x7f
[ 169.858878] handle_edge_irq+0xfd/0x380
[ 169.858879] asm_call_on_stack+0xf/0x20
[ 169.858881] common_interrupt+0xea/0x1a0
[ 169.858883] asm_common_interrupt+0x1e/0x40
[ 169.858885] lock_acquire+0x18d/0x550
[ 169.858887] rcu_torture_read_lock+0x28/0x80
[ 169.858889] rcutorture_one_extend+0x26a/0x4b0
[ 169.858891] rcu_torture_one_read+0x30e/0x670
[ 169.858894] rcu_torture_reader+0x13b/0x2d0
[ 169.858895] kthread+0x1f4/0x220
[ 169.858898] ret_from_fork+0x1f/0x30
[ 169.858899]
[ 169.858900] -> #1 (&port->lock){-.-.}-{2:2}:
[ 169.858907] __lock_acquire+0x7f3/0xef0
[ 169.858909] lock_acquire+0x15d/0x550
[ 169.858911] _raw_spin_lock_irqsave+0x37/0x80
[ 169.858914] serial8250_console_write+0xfe/0x4f0
[ 169.858916] console_unlock+0x4c9/0x730
[ 169.858918] vprintk_emit+0xe5/0x290
[ 169.858920] printk+0xad/0xde
[ 169.858922] register_console+0x25a/0x3c0
[ 169.858924] univ8250_console_init+0x1f/0x22
[ 169.858926] console_init+0x21a/0x324
[ 169.858928] start_kernel+0x29a/0x3c3
[ 169.858931] secondary_startup_64+0xb6/0xc0
[ 169.858933]
[ 169.858934] -> #0 (console_owner){-.-.}-{0:0}:
[ 169.858941] check_prevs_add+0x325/0x13a0
[ 169.858943] validate_chain+0xde3/0x2860
[ 169.858945] __lock_acquire+0x7f3/0xef0
[ 169.858947] lock_acquire+0x15d/0x550
[ 169.858949] console_unlock+0x25f/0x730
[ 169.858951] vprintk_emit+0xe5/0x290
[ 169.858953] printk+0xad/0xde
[ 169.858955] ubsan_prologue+0x23/0x44
To reproduce:
# build kernel
cd linux
cp config-5.9.0-rc1-00141-g152b0b530094b .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
4 months, 2 weeks
- 1
- 0
[rcu/segcblist] e08055898f: WARNING:at_kernel/rcu/srcutree.c:#cleanup_srcu_struct
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: e08055898fdf4d90caf3e7062fcddbf9ec2fe161 ("[PATCH v7 2/6] rcu/segcblist: Add counters to segcblist datastructure")
url: https://github.com/0day-ci/linux/commits/Joel-Fernandes-Google/Add-suppor...
base: https://git.kernel.org/cgit/linux/kernel/git/paulmck/linux-rcu.git dev
in testcase: kernel-selftests
version: kernel-selftests-x86_64-b5a583fb-1_20201015
with following parameters:
group: kselftests-kvm
ucode: 0xdc
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <lkp(a)intel.com>
kern :warn : [ 76.220665] WARNING: CPU: 2 PID: 10209 at kernel/rcu/srcutree.c:386 cleanup_srcu_struct+0x9d/0x100
kern :warn : [ 76.229759] Modules linked in: xfs libcrc32c sd_mod t10_pi sg snd_hda_codec_hdmi intel_rapl_msr intel_rapl_common snd_hda_codec_realtek snd_hda_codec_generic ledtrig_audio x86_pkg_temp_thermal snd_hda_intel intel_powerclamp coretemp snd_intel_dspcfg crct10dif_pclmul crc32_pclmul snd_hda_codec ahci crc32c_intel mei_wdt snd_hda_core ghash_clmulni_intel dell_wmi libahci snd_hwdep dell_smbios aesni_intel crypto_simd dell_wmi_descriptor wmi_bmof sparse_keymap ipmi_devintf snd_pcm i915 ipmi_msghandler dcdbas snd_timer cryptd mei_me glue_helper pcspkr mei snd libata i2c_i801 i2c_smbus wmi soundcore video intel_pmc_core acpi_pad ip_tables
kern :warn : [ 76.285829] CPU: 2 PID: 10209 Comm: cr4_cpuid_sync_ Not tainted 5.9.0-rc5-00227-ge08055898fdf4 #1
kern :warn : [ 76.294728] Hardware name: Dell Inc. OptiPlex 7040/0Y7WYT, BIOS 1.8.1 12/05/2017
kern :warn : [ 76.302164] RIP: 0010:cleanup_srcu_struct+0x9d/0x100
kern :warn : [ 76.307160] Code: 1c c5 00 f9 6a 82 48 8d bb 10 01 00 00 e8 9b 8a 01 00 48 8d bb 60 01 00 00 e8 6f 11 f8 ff 48 8b 83 c8 00 00 00 48 85 c0 74 b1 <0f> 0b 5b 5d 41 5c 41 5d c3 49 8b 84 24 e0 36 01 00 a8 03 0f 85 e9
kern :warn : [ 76.325999] RSP: 0018:ffffc90001c73db8 EFLAGS: 00010202
kern :warn : [ 76.331257] RAX: 0000000000000001 RBX: ffffe8ffffc8ba40 RCX: 0000000000000001
kern :warn : [ 76.338441] RDX: 0000000000000000 RSI: ffffffff82cc8cc0 RDI: 0000000000000246
kern :warn : [ 76.345605] RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000001
kern :warn : [ 76.352786] R10: 0000000000000001 R11: 0000000000000046 R12: ffffc90001c7dc88
kern :warn : [ 76.359943] R13: ffffc90001c91388 R14: dead000000000122 R15: dead000000000100
kern :warn : [ 76.367111] FS: 0000000000000000(0000) GS:ffff88881dd00000(0000) knlGS:0000000000000000
kern :warn : [ 76.375224] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
kern :warn : [ 76.381015] CR2: 0000562d793bf1a0 CR3: 000000080f3f4004 CR4: 00000000003726e0
kern :warn : [ 76.388171] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kern :warn : [ 76.395357] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
kern :warn : [ 76.402543] Call Trace:
kern :warn : [ 76.405051] kvm_arch_destroy_vm+0x16b/0x1e0
kern :warn : [ 76.409367] kvm_put_kvm+0x1e4/0x400
kern :warn : [ 76.413020] kvm_vm_release+0x1d/0x40
kern :warn : [ 76.416697] __fput+0xb4/0x260
kern :warn : [ 76.419800] task_work_run+0x5f/0xa0
kern :warn : [ 76.423394] do_exit+0x3ee/0xc60
kern :warn : [ 76.426654] ? syscall_enter_from_user_mode+0x22/0x80
kern :warn : [ 76.431766] do_group_exit+0x47/0xc0
kern :warn : [ 76.435374] __x64_sys_exit_group+0x14/0x20
kern :warn : [ 76.439573] do_syscall_64+0x33/0x40
kern :warn : [ 76.443206] entry_SYSCALL_64_after_hwframe+0x44/0xa9
kern :warn : [ 76.448291] RIP: 0033:0x7f02c267b9d6
kern :warn : [ 76.451900] Code: Bad RIP value.
kern :warn : [ 76.455164] RSP: 002b:00007ffd3c505068 EFLAGS: 00000246 ORIG_RAX: 00000000000000e7
kern :warn : [ 76.462791] RAX: ffffffffffffffda RBX: 00007f02c276c760 RCX: 00007f02c267b9d6
kern :warn : [ 76.469991] RDX: 0000000000000000 RSI: 000000000000003c RDI: 0000000000000000
kern :warn : [ 76.477173] RBP: 0000000000000000 R08: 00000000000000e7 R09: ffffffffffffff80
kern :warn : [ 76.484329] R10: 00007ffd3c504f26 R11: 0000000000000246 R12: 00007f02c276c760
kern :warn : [ 76.491498] R13: 0000000000000001 R14: 00007f02c2775428 R15: 0000000000000000
kern :warn : [ 76.498708] irq event stamp: 9799
kern :warn : [ 76.502072] hardirqs last enabled at (9821): [<ffffffff81da9344>] _raw_spin_unlock_irq+0x24/0x60
kern :warn : [ 76.511182] hardirqs last disabled at (9844): [<ffffffff811458a6>] account_kernel_stack+0xe6/0x140
kern :warn : [ 76.520199] softirqs last enabled at (9862): [<ffffffff8200039e>] __do_softirq+0x39e/0x4eb
kern :warn : [ 76.528608] softirqs last disabled at (9875): [<ffffffff81e010d2>] asm_call_on_stack+0x12/0x20
kern :warn : [ 76.537264] ---[ end trace 6b38257f9486b136 ]---
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
lkp
4 months, 3 weeks
- 1
- 0
[mm/writeback] 8d92890bd6: will-it-scale.per_process_ops -15.3% regression
by kernel test robot
Greeting,
FYI, we noticed a -15.3% regression of will-it-scale.per_process_ops due to commit:
commit: 8d92890bd6b8502d6aee4b37430ae6444ade7a8c ("mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK instead")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: will-it-scale
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
nr_task: 100%
mode: process
test: page_fault3
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Will It Scale takes a testcase and runs it from 1 through to n parallel copies to see if the testcase will scale. It builds both a process and threads based test in order to see any differences between the two.
test-url: https://github.com/antonblanchard/will-it-scale
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
compiler/cpufreq_governor/kconfig/mode/nr_task/rootfs/tbox_group/test/testcase/ucode:
gcc-9/performance/x86_64-rhel-8.3/process/100%/debian-10.4-x86_64-20200603.cgz/lkp-csl-2ap4/page_fault3/will-it-scale/0x5002f01
commit:
a37b0715dd ("mm/writeback: replace PF_LESS_THROTTLE with PF_LOCAL_THROTTLE")
8d92890bd6 ("mm/writeback: discard NR_UNSTABLE_NFS, use NR_WRITEBACK instead")
a37b0715ddf30077 8d92890bd6b8502d6aee4b37430
---------------- ---------------------------
fail:runs %reproduction fail:runs
| | |
15:4 -53% 13:4 perf-profile.calltrace.cycles-pp.error_entry.testcase
13:4 -46% 11:4 perf-profile.calltrace.cycles-pp.sync_regs.error_entry.testcase
16:4 -56% 13:4 perf-profile.children.cycles-pp.error_entry
0:4 -1% 0:4 perf-profile.children.cycles-pp.error_exit
1:4 -7% 1:4 perf-profile.self.cycles-pp.error_entry
%stddev %change %stddev
\ | \
397562 ± 2% -15.3% 336574 will-it-scale.per_process_ops
76332041 ± 2% -15.3% 64622457 will-it-scale.workload
39.77 ± 2% -3.3% 38.47 ± 2% boot-time.boot
6743 ± 2% -3.7% 6491 ± 2% boot-time.idle
1181 ± 6% -16.5% 986.75 ± 3% slabinfo.file_lock_cache.active_objs
1181 ± 6% -16.5% 986.75 ± 3% slabinfo.file_lock_cache.num_objs
52683124 -13.2% 45706664 proc-vmstat.numa_hit
52589716 -13.3% 45613288 proc-vmstat.numa_local
52808650 -13.2% 45839676 proc-vmstat.pgalloc_normal
2.291e+10 ± 2% -15.3% 1.94e+10 ± 2% proc-vmstat.pgfault
49686233 ± 7% -15.7% 41876349 ± 5% proc-vmstat.pgfree
13891167 ± 3% -15.0% 11805002 ± 2% numa-numastat.node1.local_node
13912999 ± 3% -15.0% 11829816 ± 2% numa-numastat.node1.numa_hit
13716455 ± 5% -14.8% 11689391 numa-numastat.node2.local_node
13735214 ± 5% -14.7% 11712688 numa-numastat.node2.numa_hit
13726371 ± 4% -14.9% 11684584 numa-numastat.node3.local_node
13757483 ± 4% -14.9% 11712533 numa-numastat.node3.numa_hit
32.74 ± 30% -25.0% 24.56 ± 15% sched_debug.cfs_rq:/.load_avg.stddev
61.79 ± 8% -12.9% 53.80 ± 4% sched_debug.cfs_rq:/.util_avg.stddev
49.46 ± 86% -89.4% 5.25 ± 74% sched_debug.cfs_rq:/.util_est_enqueued.min
21511 ± 3% -9.6% 19450 ± 4% sched_debug.cpu.nr_switches.max
2654 ± 2% +9.7% 2913 ± 4% sched_debug.cpu.sched_goidle.max
372.89 ± 4% -12.4% 326.61 ± 6% sched_debug.cpu.ttwu_local.stddev
8526 ± 4% -9.0% 7758 ± 2% numa-meminfo.node0.KernelStack
13013 ± 3% -8.5% 11911 numa-meminfo.node0.PageTables
110904 ± 7% -12.0% 97557 ± 7% numa-meminfo.node0.SUnreclaim
171034 ± 6% -11.5% 151283 ± 3% numa-meminfo.node0.Slab
7265 ± 4% +9.3% 7941 ± 4% numa-meminfo.node1.KernelStack
87118 ± 8% +13.4% 98831 ± 3% numa-meminfo.node1.SUnreclaim
268655 +8.0% 290122 ± 5% numa-meminfo.node1.Unevictable
8527 ± 4% -9.0% 7758 ± 2% numa-vmstat.node0.nr_kernel_stack
3250 ± 3% -8.3% 2979 numa-vmstat.node0.nr_page_table_pages
27726 ± 7% -12.0% 24389 ± 7% numa-vmstat.node0.nr_slab_unreclaimable
7265 ± 4% +9.3% 7941 ± 4% numa-vmstat.node1.nr_kernel_stack
21779 ± 8% +13.4% 24707 ± 3% numa-vmstat.node1.nr_slab_unreclaimable
67163 +8.0% 72530 ± 5% numa-vmstat.node1.nr_unevictable
67163 +8.0% 72530 ± 5% numa-vmstat.node1.nr_zone_unevictable
8044550 ± 3% -10.9% 7168348 ± 2% numa-vmstat.node1.numa_hit
7970411 ± 3% -10.6% 7125578 ± 2% numa-vmstat.node1.numa_local
7988934 ± 4% -12.3% 7002772 numa-vmstat.node2.numa_hit
7879319 ± 4% -12.6% 6888935 numa-vmstat.node2.numa_local
8041049 ± 4% -13.3% 6974987 numa-vmstat.node3.numa_hit
7919771 ± 4% -13.4% 6856917 numa-vmstat.node3.numa_local
5422 ± 34% +60.2% 8684 interrupts.CPU1.NMI:Non-maskable_interrupts
5422 ± 34% +60.2% 8684 interrupts.CPU1.PMI:Performance_monitoring_interrupts
847.75 ± 7% +10.4% 935.50 ± 6% interrupts.CPU153.CAL:Function_call_interrupts
845.75 ± 7% +10.6% 935.50 ± 6% interrupts.CPU154.CAL:Function_call_interrupts
847.75 ± 7% +10.4% 935.50 ± 6% interrupts.CPU155.CAL:Function_call_interrupts
847.50 ± 7% +10.4% 935.50 ± 6% interrupts.CPU156.CAL:Function_call_interrupts
847.00 ± 7% +10.4% 935.25 ± 6% interrupts.CPU158.CAL:Function_call_interrupts
216.25 ± 17% -55.4% 96.50 ± 58% interrupts.CPU158.RES:Rescheduling_interrupts
848.00 ± 8% +10.1% 933.75 ± 6% interrupts.CPU159.CAL:Function_call_interrupts
847.50 ± 8% +10.4% 935.50 ± 6% interrupts.CPU160.CAL:Function_call_interrupts
847.50 ± 8% +10.4% 935.50 ± 6% interrupts.CPU161.CAL:Function_call_interrupts
847.25 ± 7% +10.4% 935.50 ± 6% interrupts.CPU162.CAL:Function_call_interrupts
847.50 ± 8% +10.4% 935.50 ± 6% interrupts.CPU163.CAL:Function_call_interrupts
370.25 ± 50% -40.1% 221.75 ± 15% interrupts.CPU191.RES:Rescheduling_interrupts
1294 ± 51% -49.8% 649.75 ± 16% interrupts.CPU25.RES:Rescheduling_interrupts
3.38 ± 2% +9.5% 3.70 perf-stat.i.MPKI
3.812e+10 ± 2% -15.2% 3.231e+10 perf-stat.i.branch-instructions
0.30 +0.0 0.31 perf-stat.i.branch-miss-rate%
1.104e+08 ± 2% -12.3% 96774106 perf-stat.i.branch-misses
54.85 -1.7 53.16 perf-stat.i.cache-miss-rate%
3.469e+08 -9.9% 3.124e+08 perf-stat.i.cache-misses
6.312e+08 -7.1% 5.864e+08 perf-stat.i.cache-references
3.14 ± 2% +18.0% 3.71 perf-stat.i.cpi
1697 +11.1% 1885 perf-stat.i.cycles-between-cache-misses
3444920 ± 9% -16.1% 2890587 perf-stat.i.dTLB-load-misses
5.316e+10 ± 2% -15.2% 4.506e+10 perf-stat.i.dTLB-loads
1.278e+09 ± 2% -15.5% 1.08e+09 ± 2% perf-stat.i.dTLB-store-misses
2.735e+10 ± 2% -15.3% 2.318e+10 perf-stat.i.dTLB-stores
71867884 -11.4% 63642234 perf-stat.i.iTLB-load-misses
233257 ± 4% -16.2% 195493 ± 6% perf-stat.i.iTLB-loads
1.873e+11 ± 2% -15.3% 1.587e+11 perf-stat.i.instructions
2611 -4.4% 2497 perf-stat.i.instructions-per-iTLB-miss
0.32 ± 2% -15.2% 0.27 perf-stat.i.ipc
0.35 ± 69% +128.5% 0.79 ± 3% perf-stat.i.metric.K/sec
629.28 ± 2% -15.2% 533.56 perf-stat.i.metric.M/sec
75758540 ± 2% -15.4% 64123701 ± 2% perf-stat.i.minor-faults
16.32 ± 2% +9.5 25.77 ± 2% perf-stat.i.node-load-miss-rate%
6187005 ± 3% +9.5% 6773017 ± 2% perf-stat.i.node-load-misses
32626964 ± 3% -39.1% 19884304 perf-stat.i.node-loads
22.85 -0.3 22.54 perf-stat.i.node-store-miss-rate%
22644910 ± 2% -17.3% 18738027 perf-stat.i.node-store-misses
77237823 ± 2% -15.5% 65254403 perf-stat.i.node-stores
75758541 ± 2% -15.4% 64123701 ± 2% perf-stat.i.page-faults
3.37 ± 2% +9.6% 3.69 perf-stat.overall.MPKI
0.29 +0.0 0.30 perf-stat.overall.branch-miss-rate%
55.00 -1.7 53.31 perf-stat.overall.cache-miss-rate%
3.14 ± 2% +18.0% 3.71 perf-stat.overall.cpi
1694 +11.1% 1882 perf-stat.overall.cycles-between-cache-misses
2606 -4.3% 2493 perf-stat.overall.instructions-per-iTLB-miss
0.32 ± 2% -15.3% 0.27 perf-stat.overall.ipc
15.91 ± 2% +9.5 25.37 ± 2% perf-stat.overall.node-load-miss-rate%
22.67 -0.4 22.31 perf-stat.overall.node-store-miss-rate%
3.801e+10 ± 2% -15.2% 3.223e+10 perf-stat.ps.branch-instructions
1.1e+08 ± 2% -12.3% 96469431 perf-stat.ps.branch-misses
3.46e+08 -9.9% 3.116e+08 perf-stat.ps.cache-misses
6.293e+08 -7.1% 5.846e+08 perf-stat.ps.cache-references
3494841 ± 8% -17.0% 2901338 perf-stat.ps.dTLB-load-misses
5.3e+10 ± 2% -15.2% 4.494e+10 perf-stat.ps.dTLB-loads
1.275e+09 ± 2% -15.5% 1.078e+09 ± 2% perf-stat.ps.dTLB-store-misses
2.727e+10 ± 2% -15.2% 2.312e+10 perf-stat.ps.dTLB-stores
71652742 -11.4% 63468528 perf-stat.ps.iTLB-load-misses
231120 ± 4% -16.5% 192992 ± 5% perf-stat.ps.iTLB-loads
1.868e+11 ± 2% -15.2% 1.583e+11 perf-stat.ps.instructions
75533403 ± 2% -15.3% 63956449 perf-stat.ps.minor-faults
6164201 ± 3% +9.4% 6746436 ± 2% perf-stat.ps.node-load-misses
32581427 ± 3% -39.1% 19854118 perf-stat.ps.node-loads
22575559 ± 2% -17.2% 18684407 perf-stat.ps.node-store-misses
77005596 ± 2% -15.5% 65083018 perf-stat.ps.node-stores
75533404 ± 2% -15.3% 63956450 perf-stat.ps.page-faults
5.653e+13 ± 2% -15.3% 4.788e+13 perf-stat.total.instructions
79.63 -3.5 76.17 perf-profile.calltrace.cycles-pp.page_fault.testcase
8.31 ± 3% -2.1 6.22 ± 2% perf-profile.calltrace.cycles-pp.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
7.96 ± 3% -2.0 5.91 ± 2% perf-profile.calltrace.cycles-pp.shmem_fault.__do_fault.do_fault.__handle_mm_fault.handle_mm_fault
7.37 ± 3% -2.0 5.41 ± 2% perf-profile.calltrace.cycles-pp.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault.__handle_mm_fault
6.80 ± 3% -1.9 4.92 ± 3% perf-profile.calltrace.cycles-pp.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault.do_fault
4.76 ± 3% -1.4 3.39 ± 3% perf-profile.calltrace.cycles-pp.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault.__do_fault
6.30 -0.9 5.35 ± 3% perf-profile.calltrace.cycles-pp.__count_memcg_events.handle_mm_fault.do_user_addr_fault.page_fault.testcase
5.15 ± 2% -0.7 4.41 perf-profile.calltrace.cycles-pp.__mod_lruvec_state.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
1.62 ± 3% -0.7 0.88 ± 3% perf-profile.calltrace.cycles-pp.xas_load.find_get_entry.find_lock_entry.shmem_getpage_gfp.shmem_fault
4.27 ± 3% -0.6 3.66 perf-profile.calltrace.cycles-pp.__mod_memcg_state.__mod_lruvec_state.page_remove_rmap.zap_pte_range.unmap_page_range
5.87 ± 3% -0.5 5.33 perf-profile.calltrace.cycles-pp.__mod_lruvec_state.page_add_file_rmap.alloc_set_pte.finish_fault.do_fault
2.97 -0.5 2.46 ± 3% perf-profile.calltrace.cycles-pp.lock_page_memcg.page_add_file_rmap.alloc_set_pte.finish_fault.do_fault
4.99 ± 4% -0.4 4.60 ± 2% perf-profile.calltrace.cycles-pp.__mod_memcg_state.__mod_lruvec_state.page_add_file_rmap.alloc_set_pte.finish_fault
2.33 ± 2% -0.4 1.95 ± 2% perf-profile.calltrace.cycles-pp.fault_dirty_shared_page.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
1.05 ± 4% -0.3 0.77 ± 3% perf-profile.calltrace.cycles-pp._raw_spin_lock.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
0.94 ± 2% -0.1 0.79 ± 2% perf-profile.calltrace.cycles-pp.__perf_sw_event.do_user_addr_fault.page_fault.testcase
0.66 ± 2% -0.1 0.52 perf-profile.calltrace.cycles-pp.down_read_trylock.do_user_addr_fault.page_fault.testcase
1.04 -0.1 0.92 perf-profile.calltrace.cycles-pp.lock_page_memcg.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas
0.99 ± 2% -0.1 0.88 ± 2% perf-profile.calltrace.cycles-pp.__perf_sw_event.page_fault.testcase
0.96 ± 3% -0.1 0.85 perf-profile.calltrace.cycles-pp.up_read.do_user_addr_fault.page_fault.testcase
0.80 ± 3% -0.1 0.69 ± 2% perf-profile.calltrace.cycles-pp.file_update_time.fault_dirty_shared_page.do_fault.__handle_mm_fault.handle_mm_fault
0.63 ± 2% -0.1 0.53 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__perf_sw_event.do_user_addr_fault.page_fault.testcase
0.73 ± 2% -0.1 0.64 ± 2% perf-profile.calltrace.cycles-pp.swapgs_restore_regs_and_return_to_usermode.testcase
0.61 ± 2% -0.1 0.55 ± 2% perf-profile.calltrace.cycles-pp.___perf_sw_event.__perf_sw_event.page_fault.testcase
84.98 +1.1 86.08 perf-profile.calltrace.cycles-pp.testcase
48.52 +1.3 49.87 perf-profile.calltrace.cycles-pp.do_user_addr_fault.page_fault.testcase
14.57 ± 3% +1.6 16.13 perf-profile.calltrace.cycles-pp.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe
14.57 ± 3% +1.6 16.13 perf-profile.calltrace.cycles-pp.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.__munmap
14.57 ± 3% +1.6 16.13 perf-profile.calltrace.cycles-pp.__vm_munmap.__x64_sys_munmap.do_syscall_64.entry_SYSCALL_64_after_hwframe.__munmap
14.57 ± 3% +1.6 16.14 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__munmap
14.57 ± 3% +1.6 16.14 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__munmap
14.57 ± 3% +1.6 16.14 perf-profile.calltrace.cycles-pp.__munmap
14.56 ± 3% +1.6 16.13 perf-profile.calltrace.cycles-pp.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap.do_syscall_64
14.54 ± 3% +1.6 16.11 perf-profile.calltrace.cycles-pp.unmap_vmas.unmap_region.__do_munmap.__vm_munmap.__x64_sys_munmap
14.54 ± 3% +1.6 16.11 perf-profile.calltrace.cycles-pp.unmap_page_range.unmap_vmas.unmap_region.__do_munmap.__vm_munmap
14.48 ± 3% +1.6 16.07 perf-profile.calltrace.cycles-pp.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region.__do_munmap
44.16 +2.1 46.22 perf-profile.calltrace.cycles-pp.handle_mm_fault.do_user_addr_fault.page_fault.testcase
11.15 ± 4% +2.2 13.37 perf-profile.calltrace.cycles-pp.page_remove_rmap.zap_pte_range.unmap_page_range.unmap_vmas.unmap_region
36.38 +3.2 39.61 perf-profile.calltrace.cycles-pp.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.page_fault.testcase
34.47 ± 2% +3.5 37.98 perf-profile.calltrace.cycles-pp.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault.page_fault
23.42 ± 3% +6.1 29.47 perf-profile.calltrace.cycles-pp.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault.do_user_addr_fault
23.25 ± 3% +6.1 29.34 perf-profile.calltrace.cycles-pp.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault.handle_mm_fault
21.20 ± 4% +6.5 27.72 perf-profile.calltrace.cycles-pp.page_add_file_rmap.alloc_set_pte.finish_fault.do_fault.__handle_mm_fault
8.32 ± 3% -2.1 6.22 ± 2% perf-profile.children.cycles-pp.__do_fault
7.97 ± 3% -2.1 5.92 ± 2% perf-profile.children.cycles-pp.shmem_fault
7.39 ± 3% -2.0 5.43 ± 2% perf-profile.children.cycles-pp.shmem_getpage_gfp
6.86 ± 3% -1.9 4.96 ± 3% perf-profile.children.cycles-pp.find_lock_entry
85.36 -1.6 83.79 perf-profile.children.cycles-pp.testcase
4.80 ± 3% -1.4 3.42 ± 3% perf-profile.children.cycles-pp.find_get_entry
11.04 ± 3% -1.3 9.76 perf-profile.children.cycles-pp.__mod_lruvec_state
65.24 -1.2 64.03 perf-profile.children.cycles-pp.page_fault
9.28 ± 3% -1.0 8.27 perf-profile.children.cycles-pp.__mod_memcg_state
6.30 -1.0 5.35 ± 3% perf-profile.children.cycles-pp.__count_memcg_events
1.63 ± 3% -0.7 0.90 ± 3% perf-profile.children.cycles-pp.xas_load
4.02 -0.6 3.39 ± 2% perf-profile.children.cycles-pp.lock_page_memcg
3.46 ± 3% -0.5 2.99 ± 3% perf-profile.children.cycles-pp.sync_regs
2.38 ± 2% -0.4 1.99 ± 2% perf-profile.children.cycles-pp.fault_dirty_shared_page
1.08 ± 4% -0.3 0.80 ± 2% perf-profile.children.cycles-pp._raw_spin_lock
1.94 ± 2% -0.3 1.68 ± 2% perf-profile.children.cycles-pp.__perf_sw_event
1.31 ± 2% -0.2 1.14 perf-profile.children.cycles-pp.___perf_sw_event
0.88 ± 2% -0.2 0.72 ± 2% perf-profile.children.cycles-pp.page_mapping
0.82 ± 2% -0.2 0.66 ± 2% perf-profile.children.cycles-pp.set_page_dirty
0.66 ± 2% -0.1 0.52 perf-profile.children.cycles-pp.down_read_trylock
0.61 ± 4% -0.1 0.47 ± 3% perf-profile.children.cycles-pp.unlock_page
0.83 ± 2% -0.1 0.72 ± 2% perf-profile.children.cycles-pp.file_update_time
0.96 ± 3% -0.1 0.85 perf-profile.children.cycles-pp.up_read
0.52 ± 2% -0.1 0.42 ± 3% perf-profile.children.cycles-pp.tlb_flush_mmu
0.73 ± 2% -0.1 0.64 ± 2% perf-profile.children.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.39 ± 2% -0.1 0.31 ± 3% perf-profile.children.cycles-pp.release_pages
0.45 ± 2% -0.1 0.38 ± 4% perf-profile.children.cycles-pp.find_vma
0.52 ± 3% -0.1 0.45 perf-profile.children.cycles-pp.current_time
0.62 ± 3% -0.1 0.55 ± 3% perf-profile.children.cycles-pp.__mod_node_page_state
0.38 ± 2% -0.1 0.32 ± 3% perf-profile.children.cycles-pp.vmacache_find
0.41 ± 3% -0.1 0.35 perf-profile.children.cycles-pp.do_page_fault
0.41 ± 2% -0.1 0.35 ± 2% perf-profile.children.cycles-pp.___might_sleep
0.20 ± 4% -0.1 0.14 ± 5% perf-profile.children.cycles-pp.__tlb_remove_page_size
0.23 ± 7% -0.1 0.18 ± 6% perf-profile.children.cycles-pp.xas_start
0.47 -0.0 0.42 ± 2% perf-profile.children.cycles-pp.__unlock_page_memcg
0.33 ± 3% -0.0 0.29 ± 2% perf-profile.children.cycles-pp.prepare_exit_to_usermode
0.26 ± 3% -0.0 0.23 ± 2% perf-profile.children.cycles-pp.mark_page_accessed
0.26 -0.0 0.23 perf-profile.children.cycles-pp.__set_page_dirty_no_writeback
0.14 ± 3% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.perf_swevent_event
0.14 ± 6% -0.0 0.11 ± 7% perf-profile.children.cycles-pp.vm_normal_page
0.21 ± 2% -0.0 0.19 ± 4% perf-profile.children.cycles-pp.__might_sleep
0.18 ± 4% -0.0 0.16 perf-profile.children.cycles-pp._cond_resched
0.13 ± 3% -0.0 0.11 ± 3% perf-profile.children.cycles-pp.PageHuge
0.13 ± 3% -0.0 0.11 ± 6% perf-profile.children.cycles-pp.free_pages_and_swap_cache
0.10 ± 5% -0.0 0.08 perf-profile.children.cycles-pp.rcu_all_qs
0.11 ± 4% -0.0 0.10 perf-profile.children.cycles-pp.page_rmapping
48.60 +1.3 49.95 perf-profile.children.cycles-pp.do_user_addr_fault
14.57 ± 3% +1.6 16.13 perf-profile.children.cycles-pp.__do_munmap
14.57 ± 3% +1.6 16.13 perf-profile.children.cycles-pp.__x64_sys_munmap
14.57 ± 3% +1.6 16.13 perf-profile.children.cycles-pp.__vm_munmap
14.54 ± 3% +1.6 16.11 perf-profile.children.cycles-pp.zap_pte_range
14.56 ± 3% +1.6 16.13 perf-profile.children.cycles-pp.unmap_region
14.62 ± 2% +1.6 16.19 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
14.57 ± 3% +1.6 16.14 perf-profile.children.cycles-pp.__munmap
14.54 ± 3% +1.6 16.11 perf-profile.children.cycles-pp.unmap_vmas
14.54 ± 3% +1.6 16.11 perf-profile.children.cycles-pp.unmap_page_range
14.62 ± 2% +1.6 16.19 perf-profile.children.cycles-pp.do_syscall_64
44.21 +2.0 46.26 perf-profile.children.cycles-pp.handle_mm_fault
11.21 ± 4% +2.2 13.43 perf-profile.children.cycles-pp.page_remove_rmap
36.41 +3.2 39.64 perf-profile.children.cycles-pp.__handle_mm_fault
34.53 ± 2% +3.5 38.03 perf-profile.children.cycles-pp.do_fault
23.44 ± 3% +6.1 29.49 perf-profile.children.cycles-pp.finish_fault
23.28 ± 3% +6.1 29.37 perf-profile.children.cycles-pp.alloc_set_pte
21.24 ± 4% +6.5 27.77 perf-profile.children.cycles-pp.page_add_file_rmap
15.77 ± 2% -2.4 13.39 perf-profile.self.cycles-pp.testcase
9.21 ± 3% -1.0 8.21 perf-profile.self.cycles-pp.__mod_memcg_state
6.29 -1.0 5.34 ± 3% perf-profile.self.cycles-pp.__count_memcg_events
1.39 ± 3% -0.7 0.71 ± 2% perf-profile.self.cycles-pp.xas_load
3.12 ± 3% -0.6 2.49 ± 3% perf-profile.self.cycles-pp.find_get_entry
3.97 -0.6 3.34 ± 3% perf-profile.self.cycles-pp.lock_page_memcg
3.46 ± 3% -0.5 2.99 ± 3% perf-profile.self.cycles-pp.sync_regs
1.42 ± 5% -0.4 0.99 ± 3% perf-profile.self.cycles-pp.find_lock_entry
1.53 ± 2% -0.3 1.21 perf-profile.self.cycles-pp.zap_pte_range
1.07 ± 4% -0.3 0.79 ± 3% perf-profile.self.cycles-pp._raw_spin_lock
1.86 -0.3 1.59 ± 2% perf-profile.self.cycles-pp.__handle_mm_fault
1.14 ± 7% -0.2 0.94 ± 7% perf-profile.self.cycles-pp.__mod_lruvec_state
1.26 ± 2% -0.2 1.10 perf-profile.self.cycles-pp.handle_mm_fault
1.09 -0.2 0.94 ± 2% perf-profile.self.cycles-pp.___perf_sw_event
0.84 ± 2% -0.1 0.69 perf-profile.self.cycles-pp.page_mapping
0.86 ± 2% -0.1 0.72 ± 2% perf-profile.self.cycles-pp.do_user_addr_fault
0.65 ± 2% -0.1 0.51 perf-profile.self.cycles-pp.down_read_trylock
0.59 ± 4% -0.1 0.45 ± 2% perf-profile.self.cycles-pp.unlock_page
0.75 ± 2% -0.1 0.61 perf-profile.self.cycles-pp.alloc_set_pte
0.95 ± 3% -0.1 0.84 perf-profile.self.cycles-pp.up_read
0.63 ± 2% -0.1 0.53 ± 2% perf-profile.self.cycles-pp.__perf_sw_event
0.62 ± 2% -0.1 0.53 perf-profile.self.cycles-pp.page_fault
0.57 ± 3% -0.1 0.48 perf-profile.self.cycles-pp.shmem_fault
0.38 ± 2% -0.1 0.30 ± 4% perf-profile.self.cycles-pp.release_pages
0.60 ± 3% -0.1 0.53 ± 4% perf-profile.self.cycles-pp.__mod_node_page_state
0.53 ± 2% -0.1 0.46 perf-profile.self.cycles-pp.shmem_getpage_gfp
0.39 ± 2% -0.1 0.34 perf-profile.self.cycles-pp.do_page_fault
0.41 ± 4% -0.1 0.35 ± 2% perf-profile.self.cycles-pp.do_fault
0.36 ± 3% -0.1 0.31 ± 3% perf-profile.self.cycles-pp.vmacache_find
0.39 ± 2% -0.1 0.34 perf-profile.self.cycles-pp.___might_sleep
0.22 ± 5% -0.1 0.17 ± 9% perf-profile.self.cycles-pp.xas_start
0.40 ± 3% -0.1 0.35 ± 2% perf-profile.self.cycles-pp.swapgs_restore_regs_and_return_to_usermode
0.45 -0.0 0.40 ± 3% perf-profile.self.cycles-pp.__unlock_page_memcg
0.24 ± 2% -0.0 0.20 ± 3% perf-profile.self.cycles-pp.set_page_dirty
0.31 ± 4% -0.0 0.27 ± 5% perf-profile.self.cycles-pp.file_update_time
0.25 ± 2% -0.0 0.21 ± 2% perf-profile.self.cycles-pp.fault_dirty_shared_page
0.16 ± 2% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.__tlb_remove_page_size
0.28 ± 2% -0.0 0.25 ± 3% perf-profile.self.cycles-pp.prepare_exit_to_usermode
0.26 ± 3% -0.0 0.22 perf-profile.self.cycles-pp.mark_page_accessed
0.16 ± 5% -0.0 0.13 perf-profile.self.cycles-pp.finish_fault
0.23 -0.0 0.21 ± 2% perf-profile.self.cycles-pp.__set_page_dirty_no_writeback
0.16 ± 4% -0.0 0.14 ± 3% perf-profile.self.cycles-pp.__do_fault
0.17 ± 2% -0.0 0.15 ± 4% perf-profile.self.cycles-pp.current_time
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.PageHuge
0.20 ± 2% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.12 ± 4% -0.0 0.11 ± 4% perf-profile.self.cycles-pp.free_pages_and_swap_cache
0.10 ± 4% -0.0 0.09 ± 5% perf-profile.self.cycles-pp.perf_swevent_event
0.12 ± 7% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.vm_normal_page
0.10 ± 5% -0.0 0.08 perf-profile.self.cycles-pp.page_rmapping
4.94 ± 8% +3.1 8.00 ± 3% perf-profile.self.cycles-pp.page_remove_rmap
12.30 ± 6% +7.5 19.84 ± 2% perf-profile.self.cycles-pp.page_add_file_rmap
will-it-scale.per_process_ops
520000 +------------------------------------------------------------------+
500000 |-+ ..+ |
| ..+. : |
480000 |.+ : |
460000 |-+ : |
| : |
440000 |-+ : |
420000 |-+ : |
400000 |-+ : ..+....+.. ..+....+...+.. |
| : ..+... ..+. . ..+...+...+. .|
380000 |-+ +. +. +. |
360000 |-+ |
| O O |
340000 |-+ O O O O |
320000 +------------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
4 months, 3 weeks
- 4
- 5
[softirq] e29b4ce2af: netperf.Throughput_Mbps -44.0% regression
by kernel test robot
Greeting,
FYI, we noticed a -44.0% regression of netperf.Throughput_Mbps due to commit:
commit: e29b4ce2afd4de8c988529909fd7e2bdfc390dbd ("softirq: Provide a softirq_needs_break() API")
https://git.kernel.org/cgit/linux/kernel/git/peterz/queue.git core/softirq
in testcase: netperf
on test machine: 192 threads Intel(R) Xeon(R) Platinum 9242 CPU @ 2.30GHz with 192G memory
with following parameters:
ip: ipv4
runtime: 300s
nr_threads: 200%
cluster: cs-localhost
send_size: 10K
test: TCP_SENDFILE
cpufreq_governor: performance
ucode: 0x5002f01
test-description: Netperf is a benchmark that can be use to measure various aspect of networking performance.
test-url: http://www.netperf.org/netperf/
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/send_size/tbox_group/test/testcase/ucode:
cs-localhost/gcc-9/performance/ipv4/x86_64-rhel-8.3/200%/debian-10.4-x86_64-20200603.cgz/300s/10K/lkp-csl-2ap4/TCP_SENDFILE/netperf/0x5002f01
commit:
52dcd5332b ("softirq: Context aware timeout")
e29b4ce2af ("softirq: Provide a softirq_needs_break() API")
52dcd5332b292a82 e29b4ce2afd4de8c988529909fd
---------------- ---------------------------
%stddev %change %stddev
\ | \
4452 ± 5% -44.0% 2493 netperf.Throughput_Mbps
1709728 ± 5% -44.0% 957615 netperf.Throughput_total_Mbps
306.12 +1.4% 310.40 netperf.time.elapsed_time
306.12 +1.4% 310.40 netperf.time.elapsed_time.max
3.107e+09 ± 2% -44.4% 1.726e+09 netperf.time.involuntary_context_switches
103430 +3.2% 106710 netperf.time.minor_page_faults
9314 -27.7% 6730 netperf.time.percent_of_cpu_this_job_got
27199 -24.8% 20460 netperf.time.system_time
1315 ± 35% -67.0% 433.67 netperf.time.user_time
6.261e+09 ± 5% -44.0% 3.507e+09 netperf.workload
15742 ± 4% +8.9% 17140 ± 3% slabinfo.pde_opener.active_objs
15742 ± 4% +8.9% 17140 ± 3% slabinfo.pde_opener.num_objs
94.25 +1.9% 96.00 vmstat.cpu.sy
46274883 ± 2% -45.7% 25130313 vmstat.system.cs
157333 +14.3% 179783 ± 5% meminfo.Active
157330 +14.3% 179779 ± 5% meminfo.Active(anon)
177904 +14.5% 203717 ± 4% meminfo.Shmem
1.38 -0.4 0.96 mpstat.cpu.all.irq%
25.79 +20.1 45.93 mpstat.cpu.all.soft%
68.10 -17.9 50.24 mpstat.cpu.all.sys%
3.30 ± 35% -2.2 1.09 mpstat.cpu.all.usr%
37716 ± 2% +15.9% 43702 ± 5% numa-vmstat.node3.nr_active_anon
99406 ± 4% +6.4% 105793 ± 3% numa-vmstat.node3.nr_file_pages
41195 ± 4% +17.5% 48400 ± 6% numa-vmstat.node3.nr_shmem
37716 ± 2% +15.9% 43702 ± 5% numa-vmstat.node3.nr_zone_active_anon
8569 ± 2% +23.3% 10568 ± 23% numa-meminfo.node0.Mapped
152539 ± 2% +15.3% 175803 ± 5% numa-meminfo.node3.Active
152539 ± 2% +15.3% 175803 ± 5% numa-meminfo.node3.Active(anon)
399250 ± 4% +6.3% 424492 ± 3% numa-meminfo.node3.FilePages
166411 ± 4% +17.1% 194910 ± 6% numa-meminfo.node3.Shmem
39437 +14.1% 44994 ± 5% proc-vmstat.nr_active_anon
132390 -1.2% 130811 proc-vmstat.nr_anon_pages
98.75 +7.3% 106.00 ± 2% proc-vmstat.nr_dirtied
279821 +2.3% 286242 proc-vmstat.nr_file_pages
11817 ± 3% +3.8% 12269 ± 3% proc-vmstat.nr_mapped
44508 +14.5% 50968 ± 5% proc-vmstat.nr_shmem
31490 -1.5% 31010 proc-vmstat.nr_slab_reclaimable
39437 +14.1% 44994 ± 5% proc-vmstat.nr_zone_active_anon
22466 ± 43% +77.5% 39872 ± 20% proc-vmstat.numa_pages_migrated
72367 ± 4% +20.5% 87235 ± 5% proc-vmstat.pgactivate
22466 ± 43% +77.5% 39872 ± 20% proc-vmstat.pgmigrate_success
5157942 ± 7% -50.2% 2568736 ± 6% sched_debug.cfs_rq:/.MIN_vruntime.avg
36778769 ± 5% -41.5% 21525786 ± 4% sched_debug.cfs_rq:/.MIN_vruntime.max
10763334 ± 7% -41.0% 6351357 ± 4% sched_debug.cfs_rq:/.MIN_vruntime.stddev
152807 ± 10% -28.4% 109454 ± 5% sched_debug.cfs_rq:/.load.avg
284.75 ± 58% +195.6% 841.62 ± 37% sched_debug.cfs_rq:/.load.min
333402 ± 4% -14.2% 286175 sched_debug.cfs_rq:/.load.stddev
5157942 ± 7% -50.2% 2568736 ± 6% sched_debug.cfs_rq:/.max_vruntime.avg
36778769 ± 5% -41.5% 21525786 ± 4% sched_debug.cfs_rq:/.max_vruntime.max
10763334 ± 7% -41.0% 6351357 ± 4% sched_debug.cfs_rq:/.max_vruntime.stddev
26122732 ± 4% -30.3% 18206692 sched_debug.cfs_rq:/.min_vruntime.avg
37760686 ± 4% -41.6% 22048463 ± 5% sched_debug.cfs_rq:/.min_vruntime.max
7314833 ± 16% -65.3% 2534706 ± 34% sched_debug.cfs_rq:/.min_vruntime.stddev
0.36 ± 2% -12.4% 0.31 ± 2% sched_debug.cfs_rq:/.nr_running.stddev
4349 ± 7% +19.0% 5177 sched_debug.cfs_rq:/.runnable_avg.max
7319491 ± 16% -65.4% 2534851 ± 34% sched_debug.cfs_rq:/.spread0.stddev
375.59 ± 4% +12.8% 423.53 ± 4% sched_debug.cfs_rq:/.util_est_enqueued.stddev
153053 -23.0% 117870 sched_debug.cpu.clock_task.avg
155848 -22.6% 120559 sched_debug.cpu.clock_task.max
145339 -24.1% 110271 sched_debug.cpu.clock_task.min
34.46 ± 70% +147.6% 85.33 ± 29% sched_debug.cpu.curr->pid.min
1135 -15.3% 961.77 ± 4% sched_debug.cpu.curr->pid.stddev
36227910 ± 2% -44.8% 19984988 sched_debug.cpu.nr_switches.avg
39978766 ± 2% -46.0% 21578252 sched_debug.cpu.nr_switches.max
30749932 ± 6% -44.4% 17101695 sched_debug.cpu.nr_switches.min
8.278e+10 ± 2% -45.7% 4.498e+10 perf-stat.i.branch-instructions
1.66 +0.1 1.73 perf-stat.i.branch-miss-rate%
1.368e+09 -43.3% 7.752e+08 perf-stat.i.branch-misses
5.55 ± 14% +4.0 9.55 ± 15% perf-stat.i.cache-miss-rate%
1.681e+09 ± 5% -42.5% 9.67e+08 ± 2% perf-stat.i.cache-references
46718602 ± 2% -45.7% 25356221 perf-stat.i.context-switches
1.32 ± 2% +97.8% 2.60 perf-stat.i.cpi
5.401e+11 +7.9% 5.827e+11 perf-stat.i.cpu-cycles
0.02 ± 27% -0.0 0.02 perf-stat.i.dTLB-load-miss-rate%
28895857 ± 25% -62.2% 10909696 ± 2% perf-stat.i.dTLB-load-misses
1.236e+11 ± 2% -45.6% 6.726e+10 perf-stat.i.dTLB-loads
14820827 ± 9% -42.7% 8495419 perf-stat.i.dTLB-store-misses
7.08e+10 ± 2% -45.5% 3.857e+10 perf-stat.i.dTLB-stores
7.879e+08 -47.3% 4.154e+08 perf-stat.i.iTLB-load-misses
1.584e+08 ± 3% -45.3% 86646116 perf-stat.i.iTLB-loads
4.108e+11 ± 2% -45.6% 2.234e+11 perf-stat.i.instructions
530.55 +3.0% 546.53 perf-stat.i.instructions-per-iTLB-miss
0.76 ± 2% -49.3% 0.39 perf-stat.i.ipc
2.81 +7.9% 3.03 perf-stat.i.metric.GHz
0.52 ± 18% -72.1% 0.14 ± 17% perf-stat.i.metric.K/sec
1453 ± 2% -45.6% 791.20 perf-stat.i.metric.M/sec
86.61 +4.2 90.80 perf-stat.i.node-load-miss-rate%
3058643 ± 25% -33.4% 2036257 ± 10% perf-stat.i.node-loads
4.09 ± 3% +5.8% 4.33 ± 2% perf-stat.overall.MPKI
1.65 +0.1 1.72 perf-stat.overall.branch-miss-rate%
5.49 ± 15% +4.1 9.55 ± 15% perf-stat.overall.cache-miss-rate%
1.31 ± 2% +98.4% 2.61 perf-stat.overall.cpi
0.02 ± 28% -0.0 0.02 ± 2% perf-stat.overall.dTLB-load-miss-rate%
521.53 +3.1% 537.90 perf-stat.overall.instructions-per-iTLB-miss
0.76 ± 2% -49.6% 0.38 perf-stat.overall.ipc
86.53 +4.4 90.90 perf-stat.overall.node-load-miss-rate%
8.251e+10 ± 2% -45.7% 4.482e+10 perf-stat.ps.branch-instructions
1.364e+09 -43.4% 7.725e+08 perf-stat.ps.branch-misses
1.675e+09 ± 5% -42.5% 9.632e+08 ± 2% perf-stat.ps.cache-references
46594469 ± 2% -45.7% 25281307 perf-stat.ps.context-switches
5.382e+11 +7.9% 5.808e+11 perf-stat.ps.cpu-cycles
28814016 ± 25% -62.2% 10889628 ± 2% perf-stat.ps.dTLB-load-misses
1.232e+11 ± 2% -45.6% 6.703e+10 perf-stat.ps.dTLB-loads
14764320 ± 9% -42.6% 8472486 perf-stat.ps.dTLB-store-misses
7.057e+10 ± 2% -45.5% 3.844e+10 perf-stat.ps.dTLB-stores
7.851e+08 -47.3% 4.14e+08 perf-stat.ps.iTLB-load-misses
1.58e+08 ± 3% -45.3% 86404685 perf-stat.ps.iTLB-loads
4.095e+11 ± 2% -45.6% 2.227e+11 perf-stat.ps.instructions
3056024 ± 25% -33.6% 2029780 ± 10% perf-stat.ps.node-loads
1.255e+14 ± 2% -44.8% 6.924e+13 perf-stat.total.instructions
54719 ± 81% +117.8% 119196 ± 30% interrupts.CPU116.RES:Rescheduling_interrupts
19136 ± 58% +162.2% 50167 ± 40% interrupts.CPU149.RES:Rescheduling_interrupts
15415 ± 53% +245.6% 53281 ± 39% interrupts.CPU154.RES:Rescheduling_interrupts
12038 ± 78% +286.1% 46482 ± 59% interrupts.CPU156.RES:Rescheduling_interrupts
15166 ± 59% +248.8% 52905 ± 57% interrupts.CPU158.RES:Rescheduling_interrupts
15682 ± 42% +240.5% 53399 ± 55% interrupts.CPU161.RES:Rescheduling_interrupts
21381 ±100% +168.6% 57436 ± 50% interrupts.CPU163.RES:Rescheduling_interrupts
16654 ± 53% +261.0% 60122 ± 37% interrupts.CPU164.RES:Rescheduling_interrupts
25145 ± 91% +598.7% 175698 ± 27% interrupts.CPU168.RES:Rescheduling_interrupts
18367 ± 83% +806.5% 166501 ± 26% interrupts.CPU169.RES:Rescheduling_interrupts
23830 ±121% +580.5% 162160 ± 37% interrupts.CPU170.RES:Rescheduling_interrupts
27840 ±105% +478.6% 161080 ± 25% interrupts.CPU171.RES:Rescheduling_interrupts
23547 ±127% +598.9% 164572 ± 27% interrupts.CPU172.RES:Rescheduling_interrupts
18411 ±114% +799.7% 165648 ± 27% interrupts.CPU173.RES:Rescheduling_interrupts
23357 ±102% +625.0% 169349 ± 32% interrupts.CPU174.RES:Rescheduling_interrupts
4930 ± 34% +52.2% 7502 ± 24% interrupts.CPU175.NMI:Non-maskable_interrupts
4930 ± 34% +52.2% 7502 ± 24% interrupts.CPU175.PMI:Performance_monitoring_interrupts
20831 ±125% +726.2% 172103 ± 29% interrupts.CPU175.RES:Rescheduling_interrupts
13106 ±105% +103.7% 26690 ± 59% interrupts.CPU176.CAL:Function_call_interrupts
19646 ±124% +744.5% 165911 ± 26% interrupts.CPU176.RES:Rescheduling_interrupts
24527 ±135% +621.5% 176975 ± 23% interrupts.CPU177.RES:Rescheduling_interrupts
18107 ±125% +797.1% 162437 ± 34% interrupts.CPU178.RES:Rescheduling_interrupts
9079 ± 82% +160.8% 23678 ± 54% interrupts.CPU179.CAL:Function_call_interrupts
14355 ±109% +1120.8% 175239 ± 27% interrupts.CPU179.RES:Rescheduling_interrupts
20798 ±135% +661.4% 158353 ± 32% interrupts.CPU180.RES:Rescheduling_interrupts
20030 ±125% +709.4% 162133 ± 27% interrupts.CPU181.RES:Rescheduling_interrupts
17512 ±139% +857.1% 167603 ± 30% interrupts.CPU182.RES:Rescheduling_interrupts
21165 ±109% +614.2% 151164 ± 29% interrupts.CPU183.RES:Rescheduling_interrupts
21201 ±133% +647.7% 158529 ± 26% interrupts.CPU184.RES:Rescheduling_interrupts
17319 ±151% +943.9% 180801 ± 29% interrupts.CPU185.RES:Rescheduling_interrupts
25708 ±139% +522.7% 160089 ± 28% interrupts.CPU186.RES:Rescheduling_interrupts
11370 ±103% +108.1% 23663 ± 66% interrupts.CPU187.CAL:Function_call_interrupts
19972 ±140% +770.7% 173901 ± 25% interrupts.CPU187.RES:Rescheduling_interrupts
28954 ±148% +474.9% 166466 ± 33% interrupts.CPU188.RES:Rescheduling_interrupts
11910 ±104% +95.1% 23238 ± 60% interrupts.CPU189.CAL:Function_call_interrupts
18025 ±106% +863.6% 173704 ± 28% interrupts.CPU189.RES:Rescheduling_interrupts
24736 ±145% +600.1% 173187 ± 29% interrupts.CPU190.RES:Rescheduling_interrupts
24202 ±113% +529.6% 152391 ± 25% interrupts.CPU191.RES:Rescheduling_interrupts
44.75 ± 90% -78.8% 9.50 ±125% interrupts.CPU44.TLB:TLB_shootdowns
17195 ± 70% +160.0% 44707 ± 44% interrupts.CPU48.RES:Rescheduling_interrupts
13764 ± 73% +302.3% 55371 ± 34% interrupts.CPU53.RES:Rescheduling_interrupts
19309 ± 66% +203.7% 58647 ± 41% interrupts.CPU58.RES:Rescheduling_interrupts
16459 ± 53% +292.2% 64556 ± 59% interrupts.CPU65.RES:Rescheduling_interrupts
17688 ± 96% +826.6% 163889 ± 21% interrupts.CPU72.RES:Rescheduling_interrupts
8806 ± 77% +153.2% 22300 ± 56% interrupts.CPU73.CAL:Function_call_interrupts
15112 ±113% +914.4% 153294 ± 22% interrupts.CPU73.RES:Rescheduling_interrupts
25161 ±122% +468.6% 143062 ± 30% interrupts.CPU74.RES:Rescheduling_interrupts
22790 ±124% +608.4% 161451 ± 27% interrupts.CPU75.RES:Rescheduling_interrupts
18683 ±140% +667.8% 143452 ± 26% interrupts.CPU76.RES:Rescheduling_interrupts
22175 ±111% +608.6% 157141 ± 28% interrupts.CPU77.RES:Rescheduling_interrupts
24854 ±121% +494.7% 147812 ± 26% interrupts.CPU78.RES:Rescheduling_interrupts
21216 ±126% +671.6% 163711 ± 21% interrupts.CPU79.RES:Rescheduling_interrupts
21746 ±120% +629.0% 158535 ± 24% interrupts.CPU80.RES:Rescheduling_interrupts
20132 ±125% +606.8% 142305 ± 19% interrupts.CPU81.RES:Rescheduling_interrupts
17422 ±132% +742.7% 146826 ± 29% interrupts.CPU82.RES:Rescheduling_interrupts
13882 ±128% +1039.6% 158199 ± 27% interrupts.CPU83.RES:Rescheduling_interrupts
21262 ±140% +598.3% 148467 ± 29% interrupts.CPU84.RES:Rescheduling_interrupts
19095 ±121% +744.2% 161196 ± 25% interrupts.CPU85.RES:Rescheduling_interrupts
8582 ± 85% +122.9% 19133 ± 55% interrupts.CPU86.CAL:Function_call_interrupts
15233 ±123% +911.4% 154069 ± 27% interrupts.CPU86.RES:Rescheduling_interrupts
18025 ±119% +729.3% 149489 ± 24% interrupts.CPU87.RES:Rescheduling_interrupts
18202 ±126% +777.5% 159724 ± 24% interrupts.CPU88.RES:Rescheduling_interrupts
20656 ±130% +644.7% 153829 ± 28% interrupts.CPU89.RES:Rescheduling_interrupts
22908 ±126% +580.4% 155863 ± 23% interrupts.CPU90.RES:Rescheduling_interrupts
18276 ±125% +776.0% 160092 ± 27% interrupts.CPU91.RES:Rescheduling_interrupts
32424 ±117% +327.7% 138689 ± 26% interrupts.CPU92.RES:Rescheduling_interrupts
23456 ± 86% +510.3% 143158 ± 21% interrupts.CPU93.RES:Rescheduling_interrupts
27220 ±104% +483.6% 158870 ± 16% interrupts.CPU94.RES:Rescheduling_interrupts
37493 ± 82% +298.4% 149378 ± 33% interrupts.CPU95.RES:Rescheduling_interrupts
530.50 ± 2% +32.8% 704.25 ± 5% interrupts.IWI:IRQ_work_interrupts
10299076 ± 58% +101.7% 20769167 ± 5% interrupts.RES:Rescheduling_interrupts
14093898 ± 5% -42.8% 8065181 softirqs.CPU0.NET_RX
43844 ± 5% +13.1% 49603 ± 3% softirqs.CPU0.RCU
14264072 ± 5% -42.9% 8145984 softirqs.CPU1.NET_RX
41861 ± 6% +14.7% 48002 ± 3% softirqs.CPU1.RCU
14154662 ± 6% -42.1% 8202544 softirqs.CPU10.NET_RX
40099 ± 6% +19.7% 47989 ± 4% softirqs.CPU10.RCU
14166608 ± 6% -42.8% 8106940 softirqs.CPU100.NET_RX
41623 ± 6% +14.6% 47703 ± 2% softirqs.CPU100.RCU
14120968 ± 7% -42.6% 8099427 softirqs.CPU101.NET_RX
41568 ± 6% +14.2% 47484 ± 3% softirqs.CPU101.RCU
13989788 ± 5% -41.8% 8147215 softirqs.CPU102.NET_RX
41802 ± 6% +13.1% 47262 ± 3% softirqs.CPU102.RCU
13991097 ± 7% -42.0% 8111499 softirqs.CPU103.NET_RX
41363 ± 6% +14.2% 47224 ± 3% softirqs.CPU103.RCU
13896020 ± 6% -41.8% 8087140 softirqs.CPU104.NET_RX
41747 ± 6% +13.0% 47188 ± 3% softirqs.CPU104.RCU
13972231 ± 7% -41.4% 8185649 softirqs.CPU105.NET_RX
41210 ± 5% +14.5% 47172 ± 3% softirqs.CPU105.RCU
14139932 ± 4% -42.6% 8118292 softirqs.CPU106.NET_RX
40411 ± 6% +17.0% 47297 ± 3% softirqs.CPU106.RCU
14088336 ± 6% -42.1% 8159513 softirqs.CPU107.NET_RX
41436 ± 6% +14.1% 47261 ± 3% softirqs.CPU107.RCU
33061 ± 6% +10.4% 36491 softirqs.CPU107.TIMER
14096756 ± 5% -42.4% 8121125 softirqs.CPU108.NET_RX
41377 ± 5% +14.4% 47347 ± 3% softirqs.CPU108.RCU
14112024 ± 5% -42.2% 8151350 softirqs.CPU109.NET_RX
41096 ± 6% +14.4% 47006 ± 3% softirqs.CPU109.RCU
14230448 ± 7% -42.4% 8194257 softirqs.CPU11.NET_RX
41567 ± 5% +15.2% 47880 ± 3% softirqs.CPU11.RCU
14094650 ± 5% -42.3% 8127633 softirqs.CPU110.NET_RX
40911 ± 6% +15.0% 47054 ± 3% softirqs.CPU110.RCU
14545461 ± 3% -44.3% 8107234 softirqs.CPU111.NET_RX
40734 ± 6% +15.9% 47222 ± 4% softirqs.CPU111.RCU
14249001 ± 3% -42.5% 8197045 softirqs.CPU112.NET_RX
14222002 ± 3% -42.9% 8116037 softirqs.CPU113.NET_RX
14444658 ± 4% -43.7% 8135248 softirqs.CPU114.NET_RX
14510431 ± 4% -44.0% 8131566 softirqs.CPU115.NET_RX
14323903 ± 4% -43.2% 8134308 softirqs.CPU116.NET_RX
14264002 ± 6% -42.9% 8151518 softirqs.CPU117.NET_RX
14211232 ± 6% -42.6% 8160897 softirqs.CPU118.NET_RX
14500991 ± 4% -43.8% 8144077 softirqs.CPU119.NET_RX
14001930 ± 6% -42.3% 8078750 softirqs.CPU12.NET_RX
42100 ± 6% +13.9% 47948 ± 3% softirqs.CPU12.RCU
14007320 ± 5% -40.9% 8279623 softirqs.CPU120.NET_RX
14124032 ± 5% -41.6% 8254174 softirqs.CPU121.NET_RX
14156300 ± 7% -41.4% 8291896 softirqs.CPU122.NET_RX
14394225 ± 5% -41.9% 8367757 softirqs.CPU123.NET_RX
14398314 ± 5% -42.4% 8300394 softirqs.CPU124.NET_RX
14196128 ± 6% -41.3% 8338188 softirqs.CPU125.NET_RX
14266003 ± 6% -41.5% 8341322 softirqs.CPU126.NET_RX
14432920 ± 5% -42.4% 8315019 softirqs.CPU127.NET_RX
14442423 ± 4% -42.4% 8317514 softirqs.CPU128.NET_RX
41536 ± 8% +13.6% 47176 ± 4% softirqs.CPU128.RCU
14641276 ± 3% -43.0% 8348890 softirqs.CPU129.NET_RX
41398 ± 8% +13.4% 46938 ± 3% softirqs.CPU129.RCU
14080924 ± 6% -42.3% 8126016 softirqs.CPU13.NET_RX
41640 ± 6% +14.7% 47776 ± 3% softirqs.CPU13.RCU
14523455 ± 5% -42.3% 8384986 softirqs.CPU130.NET_RX
41023 ± 8% +14.7% 47053 ± 4% softirqs.CPU130.RCU
14518687 ± 4% -42.5% 8342814 softirqs.CPU131.NET_RX
40639 ± 7% +15.9% 47097 ± 4% softirqs.CPU131.RCU
14329500 ± 5% -41.8% 8346729 softirqs.CPU132.NET_RX
40728 ± 8% +15.4% 46992 ± 4% softirqs.CPU132.RCU
14480915 ± 4% -42.6% 8318490 softirqs.CPU133.NET_RX
41332 ± 8% +14.1% 47177 ± 4% softirqs.CPU133.RCU
14634110 ± 3% -42.9% 8350721 softirqs.CPU134.NET_RX
41386 ± 8% +14.0% 47180 ± 3% softirqs.CPU134.RCU
14312938 ± 5% -41.7% 8342127 softirqs.CPU135.NET_RX
40983 ± 7% +14.5% 46941 ± 4% softirqs.CPU135.RCU
14490960 ± 3% -42.3% 8366631 softirqs.CPU136.NET_RX
40892 ± 7% +14.8% 46935 ± 3% softirqs.CPU136.RCU
14666633 ± 5% -42.9% 8378802 softirqs.CPU137.NET_RX
41056 ± 8% +14.4% 46968 ± 3% softirqs.CPU137.RCU
14759636 ± 4% -43.4% 8358094 softirqs.CPU138.NET_RX
41214 ± 8% +13.7% 46868 ± 4% softirqs.CPU138.RCU
14649228 ± 4% -42.9% 8358017 softirqs.CPU139.NET_RX
41565 ± 8% +13.7% 47247 ± 3% softirqs.CPU139.RCU
14058610 ± 5% -42.4% 8098862 softirqs.CPU14.NET_RX
41760 ± 5% +14.6% 47844 ± 3% softirqs.CPU14.RCU
14680012 ± 4% -42.6% 8427925 softirqs.CPU140.NET_RX
41115 ± 7% +14.4% 47030 ± 4% softirqs.CPU140.RCU
14580872 ± 5% -42.6% 8365160 softirqs.CPU141.NET_RX
41252 ± 8% +16.3% 47973 ± 6% softirqs.CPU141.RCU
14498859 ± 6% -42.1% 8396045 softirqs.CPU142.NET_RX
41116 ± 8% +14.4% 47049 ± 4% softirqs.CPU142.RCU
14668879 ± 4% -43.0% 8365372 softirqs.CPU143.NET_RX
41202 ± 8% +14.4% 47148 ± 4% softirqs.CPU143.RCU
14651516 ± 3% -43.2% 8322770 softirqs.CPU144.NET_RX
15282552 ± 2% -45.2% 8380380 ± 2% softirqs.CPU145.NET_RX
14838632 ± 4% -43.4% 8395678 softirqs.CPU146.NET_RX
42718 ± 6% +11.5% 47635 ± 4% softirqs.CPU146.RCU
15272356 ± 3% -45.6% 8310939 ± 3% softirqs.CPU147.NET_RX
15020230 ± 2% -43.6% 8463994 softirqs.CPU148.NET_RX
15062861 ± 3% -43.6% 8492603 softirqs.CPU149.NET_RX
42175 ± 6% +12.7% 47514 ± 4% softirqs.CPU149.RCU
14220573 ± 5% -42.7% 8141289 softirqs.CPU15.NET_RX
41393 ± 5% +15.6% 47859 ± 3% softirqs.CPU15.RCU
15338975 ± 4% -44.8% 8465128 softirqs.CPU150.NET_RX
41892 ± 6% +13.5% 47539 ± 4% softirqs.CPU150.RCU
15123374 ± 3% -44.1% 8446595 softirqs.CPU151.NET_RX
14988372 ± 3% -44.0% 8390463 softirqs.CPU152.NET_RX
15189538 ± 2% -44.5% 8430855 softirqs.CPU153.NET_RX
15154727 ± 2% -44.0% 8492379 softirqs.CPU154.NET_RX
15348993 ± 2% -44.9% 8454823 softirqs.CPU155.NET_RX
15193225 ± 3% -45.0% 8352714 ± 2% softirqs.CPU156.NET_RX
15408507 ± 3% -44.9% 8488373 softirqs.CPU157.NET_RX
15313660 ± 2% -44.6% 8479359 softirqs.CPU158.NET_RX
15078110 ± 2% -43.9% 8452170 softirqs.CPU159.NET_RX
42253 ± 7% +12.3% 47452 ± 4% softirqs.CPU159.RCU
14197085 ± 7% -43.0% 8091679 softirqs.CPU16.NET_RX
15372475 ± 3% -45.3% 8411426 ± 2% softirqs.CPU160.NET_RX
15363336 -45.0% 8442884 softirqs.CPU161.NET_RX
15392075 -45.4% 8410219 softirqs.CPU162.NET_RX
15588933 ± 2% -45.7% 8465438 softirqs.CPU163.NET_RX
15143031 ± 4% -43.9% 8497120 softirqs.CPU164.NET_RX
15248016 ± 4% -44.8% 8422805 softirqs.CPU165.NET_RX
15132676 ± 3% -43.4% 8559268 softirqs.CPU166.NET_RX
15374023 ± 2% -44.7% 8497916 softirqs.CPU167.NET_RX
14393917 ± 3% -50.6% 7115777 softirqs.CPU168.NET_RX
5953 ± 31% +55.0% 9229 ± 30% softirqs.CPU168.SCHED
14676634 ± 2% -51.1% 7182396 softirqs.CPU169.NET_RX
5471 ± 25% +64.0% 8972 ± 29% softirqs.CPU169.SCHED
14133800 ± 7% -42.7% 8099136 softirqs.CPU17.NET_RX
14352395 ± 2% -49.8% 7201304 softirqs.CPU170.NET_RX
14467447 ± 4% -50.5% 7166487 ± 2% softirqs.CPU171.NET_RX
14712462 ± 4% -51.1% 7201163 softirqs.CPU172.NET_RX
14428560 ± 3% -50.1% 7206258 softirqs.CPU173.NET_RX
6140 ± 36% +51.5% 9304 ± 28% softirqs.CPU173.SCHED
14457884 ± 3% -50.5% 7156213 softirqs.CPU174.NET_RX
6488 ± 37% +41.5% 9183 ± 28% softirqs.CPU174.SCHED
14465604 ± 3% -50.4% 7182003 softirqs.CPU175.NET_RX
14634475 ± 6% -50.9% 7183693 ± 2% softirqs.CPU176.NET_RX
14749719 ± 2% -51.0% 7221760 softirqs.CPU177.NET_RX
14770564 ± 4% -51.3% 7196719 softirqs.CPU178.NET_RX
5836 ± 35% +49.7% 8739 ± 28% softirqs.CPU178.SCHED
14758451 ± 3% -51.3% 7192323 softirqs.CPU179.NET_RX
5496 ± 25% +65.1% 9075 ± 27% softirqs.CPU179.SCHED
14171509 ± 7% -42.8% 8106768 softirqs.CPU18.NET_RX
14582993 ± 3% -50.7% 7183261 softirqs.CPU180.NET_RX
14856618 ± 3% -51.7% 7181682 softirqs.CPU181.NET_RX
14606433 ± 4% -50.6% 7216821 softirqs.CPU182.NET_RX
5739 ± 36% +43.8% 8252 ± 23% softirqs.CPU182.SCHED
14756474 ± 3% -51.4% 7174683 softirqs.CPU183.NET_RX
14638586 ± 3% -50.9% 7187910 softirqs.CPU184.NET_RX
6016 ± 36% +46.0% 8785 ± 27% softirqs.CPU184.SCHED
14607054 ± 2% -50.9% 7175585 softirqs.CPU185.NET_RX
5675 ± 36% +68.4% 9556 ± 29% softirqs.CPU185.SCHED
14790854 ± 4% -51.5% 7171741 softirqs.CPU186.NET_RX
14756249 ± 4% -51.1% 7213197 softirqs.CPU187.NET_RX
14739441 ± 5% -51.3% 7180193 softirqs.CPU188.NET_RX
14769540 ± 4% -51.2% 7201605 softirqs.CPU189.NET_RX
5548 ± 40% +64.6% 9131 ± 27% softirqs.CPU189.SCHED
14106820 ± 6% -43.0% 8042855 softirqs.CPU19.NET_RX
14521616 ± 4% -50.2% 7238990 softirqs.CPU190.NET_RX
14193832 ± 2% -51.0% 6952260 softirqs.CPU191.NET_RX
14321726 ± 5% -43.2% 8139224 softirqs.CPU2.NET_RX
41945 ± 3% +14.1% 47863 ± 3% softirqs.CPU2.RCU
14260305 ± 4% -43.4% 8077473 softirqs.CPU20.NET_RX
13875693 ± 8% -41.6% 8103278 softirqs.CPU21.NET_RX
13908288 ± 7% -41.4% 8151319 softirqs.CPU22.NET_RX
14050704 ± 5% -42.5% 8078221 softirqs.CPU23.NET_RX
14299293 ± 8% -41.6% 8348457 softirqs.CPU24.NET_RX
14586440 ± 6% -42.4% 8397999 softirqs.CPU25.NET_RX
14458762 ± 8% -42.0% 8378858 softirqs.CPU26.NET_RX
14821581 ± 6% -43.5% 8368849 softirqs.CPU27.NET_RX
14602889 ± 4% -42.2% 8434245 softirqs.CPU28.NET_RX
14690614 ± 6% -42.9% 8393842 softirqs.CPU29.NET_RX
14279490 ± 6% -42.6% 8202016 softirqs.CPU3.NET_RX
41877 ± 5% +14.3% 47882 ± 3% softirqs.CPU3.RCU
14404564 ± 6% -41.8% 8383879 softirqs.CPU30.NET_RX
14287992 ± 5% -41.5% 8364361 softirqs.CPU31.NET_RX
14653249 ± 4% -43.1% 8332469 softirqs.CPU32.NET_RX
41887 ± 8% +14.1% 47786 ± 4% softirqs.CPU32.RCU
14733821 ± 4% -43.1% 8390454 softirqs.CPU33.NET_RX
42137 ± 7% +13.0% 47618 ± 3% softirqs.CPU33.RCU
14414863 ± 4% -41.6% 8423410 softirqs.CPU34.NET_RX
41718 ± 8% +14.5% 47765 ± 4% softirqs.CPU34.RCU
14525373 ± 4% -42.5% 8349493 softirqs.CPU35.NET_RX
41822 ± 7% +14.0% 47695 ± 3% softirqs.CPU35.RCU
14416806 ± 5% -42.1% 8351779 softirqs.CPU36.NET_RX
41644 ± 8% +14.6% 47709 ± 4% softirqs.CPU36.RCU
14370424 ± 6% -42.4% 8282277 softirqs.CPU37.NET_RX
41702 ± 8% +14.6% 47805 ± 4% softirqs.CPU37.RCU
14420991 ± 5% -42.2% 8341414 softirqs.CPU38.NET_RX
41930 ± 7% +13.5% 47604 ± 3% softirqs.CPU38.RCU
14129753 ± 6% -40.9% 8353936 softirqs.CPU39.NET_RX
41537 ± 7% +15.0% 47786 ± 4% softirqs.CPU39.RCU
14440335 ± 5% -43.3% 8191026 softirqs.CPU4.NET_RX
41908 ± 5% +14.4% 47926 ± 3% softirqs.CPU4.RCU
14415511 ± 5% -42.1% 8347099 softirqs.CPU40.NET_RX
41515 ± 7% +14.7% 47620 ± 3% softirqs.CPU40.RCU
14226374 ± 6% -41.6% 8312119 softirqs.CPU41.NET_RX
41944 ± 8% +13.9% 47760 ± 3% softirqs.CPU41.RCU
14239130 ± 5% -41.5% 8329103 softirqs.CPU42.NET_RX
42012 ± 8% +13.6% 47735 ± 3% softirqs.CPU42.RCU
14169469 ± 4% -41.2% 8333072 softirqs.CPU43.NET_RX
41934 ± 8% +13.9% 47783 ± 3% softirqs.CPU43.RCU
14340172 ± 5% -41.9% 8335691 softirqs.CPU44.NET_RX
41704 ± 7% +14.3% 47668 ± 3% softirqs.CPU44.RCU
14370600 ± 6% -41.9% 8350007 softirqs.CPU45.NET_RX
42044 ± 8% +13.1% 47570 ± 3% softirqs.CPU45.RCU
13950873 ± 7% -40.6% 8280960 softirqs.CPU46.NET_RX
41955 ± 8% +13.4% 47567 ± 3% softirqs.CPU46.RCU
14288973 ± 6% -41.5% 8361366 softirqs.CPU47.NET_RX
42209 ± 8% +14.1% 48171 ± 5% softirqs.CPU47.RCU
14856285 ± 3% -43.3% 8416588 softirqs.CPU48.NET_RX
15278345 ± 3% -44.9% 8425881 ± 3% softirqs.CPU49.NET_RX
14210491 ± 5% -42.6% 8154744 softirqs.CPU5.NET_RX
41537 ± 5% +15.3% 47902 ± 3% softirqs.CPU5.RCU
15171625 ± 3% -44.1% 8484672 softirqs.CPU50.NET_RX
15418385 ± 3% -45.5% 8401191 ± 3% softirqs.CPU51.NET_RX
42922 ± 6% +12.3% 48216 ± 4% softirqs.CPU51.RCU
15239623 ± 3% -44.3% 8489481 softirqs.CPU52.NET_RX
15360381 ± 3% -44.7% 8492831 softirqs.CPU53.NET_RX
15337296 ± 4% -44.6% 8490611 softirqs.CPU54.NET_RX
15483347 ± 3% -45.3% 8476515 softirqs.CPU55.NET_RX
42583 ± 7% +12.8% 48017 ± 4% softirqs.CPU55.RCU
15187279 ± 4% -44.1% 8485242 ± 2% softirqs.CPU56.NET_RX
42949 ± 6% +11.6% 47929 ± 4% softirqs.CPU56.RCU
15378214 ± 3% -44.8% 8487067 softirqs.CPU57.NET_RX
15304170 ± 4% -44.4% 8508419 softirqs.CPU58.NET_RX
15240649 ± 5% -44.4% 8472088 softirqs.CPU59.NET_RX
14120271 ± 6% -42.2% 8159546 softirqs.CPU6.NET_RX
41901 ± 6% +14.0% 47756 ± 3% softirqs.CPU6.RCU
15224715 ± 4% -44.7% 8422691 ± 2% softirqs.CPU60.NET_RX
15073381 ± 3% -43.9% 8455897 softirqs.CPU61.NET_RX
15173396 ± 3% -44.0% 8494180 softirqs.CPU62.NET_RX
15404905 ± 3% -45.1% 8451039 ± 2% softirqs.CPU63.NET_RX
42601 ± 6% +11.7% 47600 ± 4% softirqs.CPU63.RCU
15253411 ± 4% -45.0% 8395174 ± 2% softirqs.CPU64.NET_RX
15180111 ± 3% -44.6% 8405733 softirqs.CPU65.NET_RX
14991196 ± 4% -44.0% 8400061 softirqs.CPU66.NET_RX
15149905 ± 4% -44.6% 8397776 softirqs.CPU67.NET_RX
15245707 ± 3% -44.7% 8438356 softirqs.CPU68.NET_RX
15351389 ± 2% -45.4% 8376049 softirqs.CPU69.NET_RX
34302 ± 2% +13.8% 39051 ± 8% softirqs.CPU69.TIMER
13977396 ± 8% -41.7% 8155302 softirqs.CPU7.NET_RX
41868 ± 6% +14.0% 47715 ± 3% softirqs.CPU7.RCU
15046013 ± 4% -44.2% 8401714 softirqs.CPU70.NET_RX
14995529 ± 3% -44.2% 8368974 softirqs.CPU71.NET_RX
14691951 ± 3% -51.0% 7194519 softirqs.CPU72.NET_RX
5691 ± 31% +44.6% 8229 ± 21% softirqs.CPU72.SCHED
14810361 ± 3% -51.4% 7195388 softirqs.CPU73.NET_RX
5191 ± 22% +61.7% 8392 ± 23% softirqs.CPU73.SCHED
14550284 ± 3% -50.2% 7247487 softirqs.CPU74.NET_RX
14723250 ± 3% -50.8% 7248153 ± 2% softirqs.CPU75.NET_RX
14823411 ± 3% -51.3% 7215871 softirqs.CPU76.NET_RX
14878564 ± 3% -51.4% 7236922 softirqs.CPU77.NET_RX
14835191 ± 3% -51.6% 7183765 softirqs.CPU78.NET_RX
14769793 ± 3% -50.9% 7247537 softirqs.CPU79.NET_RX
14009828 ± 7% -42.2% 8104083 softirqs.CPU8.NET_RX
41617 ± 5% +14.6% 47682 ± 3% softirqs.CPU8.RCU
14827414 ± 3% -51.2% 7229340 ± 2% softirqs.CPU80.NET_RX
14860311 ± 3% -51.3% 7235089 softirqs.CPU81.NET_RX
14871427 ± 4% -51.5% 7211067 softirqs.CPU82.NET_RX
5341 ± 33% +52.7% 8156 ± 23% softirqs.CPU82.SCHED
14765879 ± 4% -51.2% 7210555 softirqs.CPU83.NET_RX
5305 ± 28% +57.0% 8328 ± 28% softirqs.CPU83.SCHED
14899555 ± 3% -51.2% 7268538 softirqs.CPU84.NET_RX
15100797 ± 3% -52.1% 7233555 softirqs.CPU85.NET_RX
14833499 ± 3% -51.3% 7224253 ± 2% softirqs.CPU86.NET_RX
5443 ± 28% +51.1% 8227 ± 25% softirqs.CPU86.SCHED
14967915 ± 3% -51.6% 7245622 softirqs.CPU87.NET_RX
14845204 ± 3% -51.3% 7228697 softirqs.CPU88.NET_RX
14857797 ± 4% -51.5% 7210550 softirqs.CPU89.NET_RX
5814 ± 37% +44.6% 8408 ± 25% softirqs.CPU89.SCHED
14073927 ± 7% -42.1% 8154582 softirqs.CPU9.NET_RX
41509 ± 5% +14.6% 47553 ± 3% softirqs.CPU9.RCU
14835607 ± 3% -51.4% 7212011 softirqs.CPU90.NET_RX
14576260 ± 2% -50.8% 7171576 softirqs.CPU91.NET_RX
14463468 ± 3% -50.2% 7201447 ± 2% softirqs.CPU92.NET_RX
14776266 ± 2% -51.3% 7196469 softirqs.CPU93.NET_RX
14533283 ± 3% -50.5% 7194848 softirqs.CPU94.NET_RX
14719006 ± 2% -51.1% 7192903 softirqs.CPU95.NET_RX
13726330 ± 6% -41.7% 8005594 softirqs.CPU96.NET_RX
41265 ± 5% +14.2% 47127 ± 3% softirqs.CPU96.RCU
13942421 ± 6% -42.3% 8051066 softirqs.CPU97.NET_RX
41237 ± 6% +14.0% 47025 ± 3% softirqs.CPU97.RCU
13898254 ± 6% -41.5% 8126094 softirqs.CPU98.NET_RX
40363 ± 5% +17.0% 47222 ± 3% softirqs.CPU98.RCU
14009618 ± 6% -42.0% 8120638 softirqs.CPU99.NET_RX
41440 ± 5% +14.2% 47316 ± 3% softirqs.CPU99.RCU
2.807e+09 ± 2% -45.1% 1.542e+09 softirqs.NET_RX
15.00 -6.0 8.95 perf-profile.calltrace.cycles-pp.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack
15.63 -6.0 9.68 perf-profile.calltrace.cycles-pp.process_backlog.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack
14.18 -5.6 8.55 perf-profile.calltrace.cycles-pp.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action.__softirqentry_text_start
13.41 -5.5 7.93 perf-profile.calltrace.cycles-pp.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog.net_rx_action
13.26 -5.4 7.87 perf-profile.calltrace.cycles-pp.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core.process_backlog
13.18 -5.3 7.83 perf-profile.calltrace.cycles-pp.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv.__netif_receive_skb_one_core
12.91 -5.2 7.70 perf-profile.calltrace.cycles-pp.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver.ip_rcv
10.18 -3.8 6.33 perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.ip_local_deliver
9.90 -3.7 6.17 perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
16.21 -3.7 12.51 perf-profile.calltrace.cycles-pp.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq
8.80 ± 4% -3.6 5.19 perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
8.70 ± 4% -3.6 5.12 perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
6.97 ± 21% -3.1 3.92 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
7.44 ± 2% -2.9 4.54 perf-profile.calltrace.cycles-pp.ret_from_fork
7.44 ± 2% -2.9 4.54 perf-profile.calltrace.cycles-pp.kthread.ret_from_fork
7.43 ± 2% -2.9 4.54 perf-profile.calltrace.cycles-pp.smpboot_thread_fn.kthread.ret_from_fork
6.86 ± 2% -2.8 4.03 perf-profile.calltrace.cycles-pp.schedule.smpboot_thread_fn.kthread.ret_from_fork
6.68 ± 3% -2.7 3.98 perf-profile.calltrace.cycles-pp.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
6.71 ± 4% -2.7 4.02 perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg
6.58 ± 2% -2.7 3.91 perf-profile.calltrace.cycles-pp.__schedule.schedule.smpboot_thread_fn.kthread.ret_from_fork
5.88 ± 4% -2.6 3.28 ± 2% perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg
5.67 ± 4% -2.5 3.17 ± 2% perf-profile.calltrace.cycles-pp.copy_user_enhanced_fast_string.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter
6.91 ± 2% -2.4 4.53 perf-profile.calltrace.cycles-pp.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
5.49 ± 3% -2.4 3.14 perf-profile.calltrace.cycles-pp.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
6.75 ± 2% -2.3 4.44 perf-profile.calltrace.cycles-pp.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
5.22 ± 3% -2.2 2.99 perf-profile.calltrace.cycles-pp.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg.inet_recvmsg
5.94 ± 3% -2.2 3.76 perf-profile.calltrace.cycles-pp.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
5.09 ± 3% -2.2 2.92 perf-profile.calltrace.cycles-pp.schedule.schedule_timeout.wait_woken.sk_wait_data.tcp_recvmsg
6.46 ± 2% -2.2 4.31 perf-profile.calltrace.cycles-pp.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established.tcp_v4_do_rcv
4.94 ± 3% -2.1 2.83 perf-profile.calltrace.cycles-pp.__schedule.schedule.schedule_timeout.wait_woken.sk_wait_data
6.29 ± 2% -2.1 4.23 perf-profile.calltrace.cycles-pp.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable.tcp_rcv_established
4.79 ± 4% -1.7 3.13 perf-profile.calltrace.cycles-pp.generic_file_splice_read.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
4.79 ± 3% -1.6 3.17 perf-profile.calltrace.cycles-pp.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
2.83 ± 2% -1.6 1.24 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
2.77 ± 2% -1.6 1.19 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common.__wake_up_common_lock
3.32 ± 2% -1.6 1.74 perf-profile.calltrace.cycles-pp.try_to_wake_up.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq
4.58 ± 3% -1.6 3.03 perf-profile.calltrace.cycles-pp.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode.entry_SYSCALL_64_after_hwframe
3.94 ± 4% -1.4 2.53 perf-profile.calltrace.cycles-pp.generic_file_buffered_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct.do_sendfile
2.82 ± 3% -1.2 1.66 ± 2% perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.exit_to_user_mode_prepare.syscall_exit_to_user_mode
2.00 ± 3% -1.1 0.88 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.schedule_timeout.wait_woken
2.05 ± 2% -1.1 1.00 perf-profile.calltrace.cycles-pp.pick_next_task_fair.__schedule.schedule.smpboot_thread_fn.kthread
1.50 ± 2% -1.0 0.55 ± 3% perf-profile.calltrace.cycles-pp.dev_hard_start_xmit.__dev_queue_xmit.ip_finish_output2.ip_output.__ip_queue_xmit
1.79 ± 3% -0.8 0.95 ± 2% perf-profile.calltrace.cycles-pp.put_prev_entity.pick_next_task_fair.__schedule.schedule.exit_to_user_mode_prepare
1.55 ± 2% -0.8 0.76 perf-profile.calltrace.cycles-pp.__switch_to_asm
1.83 ± 10% -0.7 1.12 ± 10% perf-profile.calltrace.cycles-pp.tcp_send_mss.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage.inet_sendpage
1.20 ± 3% -0.6 0.59 perf-profile.calltrace.cycles-pp.set_next_entity.pick_next_task_fair.__schedule.schedule.smpboot_thread_fn
1.28 ± 2% -0.6 0.67 perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
1.59 ± 11% -0.6 0.99 ± 12% perf-profile.calltrace.cycles-pp.tcp_current_mss.tcp_send_mss.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage
1.96 -0.6 1.39 perf-profile.calltrace.cycles-pp.sk_stream_alloc_skb.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage.inet_sendpage
1.16 ± 7% -0.6 0.60 ± 2% perf-profile.calltrace.cycles-pp.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg.inet_recvmsg
1.13 ± 14% -0.5 0.59 ± 6% perf-profile.calltrace.cycles-pp.lock_sock_nested.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage
1.44 ± 17% -0.5 0.91 ± 2% perf-profile.calltrace.cycles-pp.ttwu_do_activate.try_to_wake_up.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack
1.04 ± 7% -0.5 0.52 ± 2% perf-profile.calltrace.cycles-pp.__check_object_size.simple_copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg
1.81 -0.5 1.30 perf-profile.calltrace.cycles-pp.__alloc_skb.sk_stream_alloc_skb.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage
0.76 -0.5 0.25 ±100% perf-profile.calltrace.cycles-pp._cond_resched.kmem_cache_alloc_node.__alloc_skb.sk_stream_alloc_skb.do_tcp_sendpages
2.95 ± 5% -0.5 2.45 perf-profile.calltrace.cycles-pp.__dev_queue_xmit.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
1.20 ± 4% -0.5 0.70 ± 2% perf-profile.calltrace.cycles-pp.select_task_rq_fair.try_to_wake_up.__wake_up_common.__wake_up_common_lock.sock_def_readable
1.38 ± 17% -0.5 0.88 ± 2% perf-profile.calltrace.cycles-pp.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__softirqentry_text_start.asm_call_sysvec_on_stack
1.24 ± 8% -0.5 0.78 ± 2% perf-profile.calltrace.cycles-pp.pagecache_get_page.generic_file_buffered_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct
1.02 ± 4% -0.4 0.59 ± 2% perf-profile.calltrace.cycles-pp.select_idle_sibling.select_task_rq_fair.try_to_wake_up.__wake_up_common.__wake_up_common_lock
1.10 ± 8% -0.4 0.68 ± 3% perf-profile.calltrace.cycles-pp.find_get_entry.pagecache_get_page.generic_file_buffered_read.generic_file_splice_read.splice_direct_to_actor
1.31 ± 2% -0.4 0.89 perf-profile.calltrace.cycles-pp.dequeue_task_fair.__schedule.schedule.smpboot_thread_fn.kthread
1.62 ± 2% -0.4 1.21 perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
0.98 ± 3% -0.4 0.60 ± 2% perf-profile.calltrace.cycles-pp.copy_page_to_iter.generic_file_buffered_read.generic_file_splice_read.splice_direct_to_actor.do_splice_direct
0.79 ± 7% -0.4 0.41 ± 57% perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
0.96 ± 2% -0.4 0.60 perf-profile.calltrace.cycles-pp.switch_mm_irqs_off.__schedule.schedule.smpboot_thread_fn.kthread
0.79 ± 12% -0.3 0.45 ± 59% perf-profile.calltrace.cycles-pp.ipv4_mtu.tcp_current_mss.tcp_send_mss.do_tcp_sendpages.tcp_sendpage_locked
1.08 -0.3 0.79 perf-profile.calltrace.cycles-pp.kmem_cache_alloc_node.__alloc_skb.sk_stream_alloc_skb.do_tcp_sendpages.tcp_sendpage_locked
1.15 -0.3 0.87 perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
0.82 ± 2% -0.2 0.58 perf-profile.calltrace.cycles-pp.dequeue_entity.dequeue_task_fair.__schedule.schedule.smpboot_thread_fn
0.82 ± 2% -0.1 0.71 perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__softirqentry_text_start
0.72 ± 4% -0.1 0.65 ± 3% perf-profile.calltrace.cycles-pp.enqueue_entity.enqueue_task_fair.ttwu_do_activate.try_to_wake_up.__wake_up_common
8.62 +1.3 9.92 perf-profile.calltrace.cycles-pp.release_sock.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage
7.83 +1.6 9.47 perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendpage.inet_sendpage.kernel_sendpage
4.14 +2.5 6.61 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.do_tcp_sendpages.tcp_sendpage_locked
3.99 +2.5 6.47 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.do_tcp_sendpages
5.30 +2.5 7.80 perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendpage.inet_sendpage
4.59 +2.6 7.18 perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
3.79 +2.6 6.39 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established
4.57 +2.6 7.17 perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
5.05 +2.6 7.66 perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_sendpage
4.12 +2.7 6.78 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.tcp_rcv_established.tcp_v4_do_rcv
0.00 +3.1 3.09 perf-profile.calltrace.cycles-pp.__x86_indirect_thunk_rax.net_rx_action.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack
4.45 +3.1 7.54 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage
4.91 +3.1 8.02 perf-profile.calltrace.cycles-pp.tcp_write_xmit.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage.inet_sendpage
48.81 +3.4 52.22 perf-profile.calltrace.cycles-pp.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
47.76 +3.7 51.49 perf-profile.calltrace.cycles-pp.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
45.46 +4.4 49.81 perf-profile.calltrace.cycles-pp.do_splice_direct.do_sendfile.__x64_sys_sendfile64.do_syscall_64.entry_SYSCALL_64_after_hwframe
44.96 +4.6 49.52 perf-profile.calltrace.cycles-pp.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64.do_syscall_64
30.75 ± 2% +4.6 35.35 perf-profile.calltrace.cycles-pp.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
30.54 ± 2% +4.6 35.14 perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
31.63 +4.6 36.23 perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.51 +4.6 36.16 perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
88.31 +4.7 93.01 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe
38.75 +6.6 45.38 perf-profile.calltrace.cycles-pp.direct_splice_actor.splice_direct_to_actor.do_splice_direct.do_sendfile.__x64_sys_sendfile64
38.54 +6.7 45.25 perf-profile.calltrace.cycles-pp.splice_from_pipe.direct_splice_actor.splice_direct_to_actor.do_splice_direct.do_sendfile
13.40 ± 2% +6.8 20.20 perf-profile.calltrace.cycles-pp.__tcp_push_pending_frames.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage.inet_sendpage
38.23 +6.8 45.05 perf-profile.calltrace.cycles-pp.__splice_from_pipe.splice_from_pipe.direct_splice_actor.splice_direct_to_actor.do_splice_direct
10.94 ± 3% +6.8 17.77 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.do_tcp_sendpages
13.30 ± 2% +6.8 20.13 perf-profile.calltrace.cycles-pp.tcp_write_xmit.__tcp_push_pending_frames.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage
11.84 ± 3% +7.0 18.87 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames.do_tcp_sendpages.tcp_sendpage_locked
36.73 +7.3 44.03 perf-profile.calltrace.cycles-pp.pipe_to_sendpage.__splice_from_pipe.splice_from_pipe.direct_splice_actor.splice_direct_to_actor
36.22 +7.5 43.77 perf-profile.calltrace.cycles-pp.sock_sendpage.pipe_to_sendpage.__splice_from_pipe.splice_from_pipe.direct_splice_actor
24.71 +7.6 32.34 perf-profile.calltrace.cycles-pp.tcp_sendpage_locked.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage
35.91 +7.7 43.58 perf-profile.calltrace.cycles-pp.kernel_sendpage.sock_sendpage.pipe_to_sendpage.__splice_from_pipe.splice_from_pipe
35.72 +7.8 43.49 perf-profile.calltrace.cycles-pp.inet_sendpage.kernel_sendpage.sock_sendpage.pipe_to_sendpage.__splice_from_pipe
24.30 +7.8 32.14 perf-profile.calltrace.cycles-pp.do_tcp_sendpages.tcp_sendpage_locked.tcp_sendpage.inet_sendpage.kernel_sendpage
80.88 +8.0 88.83 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe
35.11 +8.1 43.16 perf-profile.calltrace.cycles-pp.tcp_sendpage.inet_sendpage.kernel_sendpage.sock_sendpage.pipe_to_sendpage
14.18 ± 2% +9.3 23.47 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit.__tcp_push_pending_frames
10.15 +11.2 21.37 perf-profile.calltrace.cycles-pp.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg
10.78 +11.3 22.09 perf-profile.calltrace.cycles-pp.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom
9.64 +11.3 20.96 perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_recvmsg
11.51 +11.4 22.89 perf-profile.calltrace.cycles-pp.__tcp_transmit_skb.tcp_recvmsg.inet_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
17.41 +11.8 29.20 perf-profile.calltrace.cycles-pp.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb.tcp_write_xmit
22.70 +23.7 46.37 perf-profile.calltrace.cycles-pp.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit.__tcp_transmit_skb
22.48 +23.7 46.21 perf-profile.calltrace.cycles-pp.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output.__ip_queue_xmit
22.00 +24.0 45.97 perf-profile.calltrace.cycles-pp.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2.ip_output
21.56 +24.1 45.61 perf-profile.calltrace.cycles-pp.__softirqentry_text_start.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip
21.83 +24.1 45.89 perf-profile.calltrace.cycles-pp.asm_call_sysvec_on_stack.do_softirq_own_stack.do_softirq.__local_bh_enable_ip.ip_finish_output2
17.70 ± 2% -6.9 10.80 perf-profile.children.cycles-pp.__schedule
16.75 ± 3% -6.6 10.12 perf-profile.children.cycles-pp.schedule
15.04 -6.1 8.97 perf-profile.children.cycles-pp.__netif_receive_skb_one_core
15.69 -6.0 9.71 perf-profile.children.cycles-pp.process_backlog
14.23 -5.7 8.57 perf-profile.children.cycles-pp.ip_rcv
13.44 -5.5 7.95 perf-profile.children.cycles-pp.ip_local_deliver
13.29 -5.4 7.89 perf-profile.children.cycles-pp.ip_local_deliver_finish
13.21 -5.4 7.86 perf-profile.children.cycles-pp.ip_protocol_deliver_rcu
12.96 -5.2 7.75 perf-profile.children.cycles-pp.tcp_v4_rcv
8.81 ± 4% -3.6 5.19 perf-profile.children.cycles-pp.skb_copy_datagram_iter
8.73 ± 4% -3.6 5.14 perf-profile.children.cycles-pp.__skb_datagram_iter
10.41 ± 2% -3.5 6.90 perf-profile.children.cycles-pp.try_to_wake_up
6.99 ± 21% -3.1 3.93 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
6.76 ± 2% -3.0 3.77 perf-profile.children.cycles-pp.pick_next_task_fair
7.44 ± 2% -2.9 4.54 perf-profile.children.cycles-pp.ret_from_fork
7.44 ± 2% -2.9 4.54 perf-profile.children.cycles-pp.kthread
7.44 ± 2% -2.9 4.54 perf-profile.children.cycles-pp.smpboot_thread_fn
6.70 ± 3% -2.7 4.00 perf-profile.children.cycles-pp.sk_wait_data
6.73 ± 4% -2.7 4.04 perf-profile.children.cycles-pp._copy_to_iter
5.88 ± 4% -2.6 3.29 ± 2% perf-profile.children.cycles-pp.copyout
5.83 ± 4% -2.6 3.26 ± 2% perf-profile.children.cycles-pp.copy_user_enhanced_fast_string
6.92 ± 2% -2.4 4.54 perf-profile.children.cycles-pp.sock_def_readable
5.49 ± 3% -2.4 3.14 perf-profile.children.cycles-pp.wait_woken
6.75 ± 2% -2.3 4.45 perf-profile.children.cycles-pp.__wake_up_common_lock
5.23 ± 3% -2.2 3.00 perf-profile.children.cycles-pp.schedule_timeout
5.96 ± 3% -2.2 3.78 perf-profile.children.cycles-pp.exit_to_user_mode_prepare
6.46 ± 2% -2.1 4.32 perf-profile.children.cycles-pp.__wake_up_common
4.19 ± 3% -1.8 2.40 perf-profile.children.cycles-pp.update_load_avg
4.81 ± 4% -1.7 3.14 perf-profile.children.cycles-pp.generic_file_splice_read
16.25 -1.6 14.64 perf-profile.children.cycles-pp.net_rx_action
3.31 ± 2% -1.5 1.77 perf-profile.children.cycles-pp.dequeue_task_fair
4.00 ± 4% -1.4 2.57 perf-profile.children.cycles-pp.generic_file_buffered_read
15.58 -1.3 14.23 perf-profile.children.cycles-pp.tcp_v4_do_rcv
4.66 ± 2% -1.3 3.33 perf-profile.children.cycles-pp.enqueue_task_fair
4.79 ± 2% -1.3 3.46 perf-profile.children.cycles-pp.ttwu_do_activate
2.91 ± 3% -1.3 1.60 perf-profile.children.cycles-pp.update_curr
2.50 ± 2% -1.2 1.35 perf-profile.children.cycles-pp.put_prev_entity
15.07 -1.1 13.94 perf-profile.children.cycles-pp.tcp_rcv_established
2.93 -1.0 1.89 perf-profile.children.cycles-pp.tcp_ack
2.18 ± 2% -0.9 1.24 perf-profile.children.cycles-pp.set_next_entity
2.02 ± 3% -0.8 1.21 perf-profile.children.cycles-pp.select_task_rq_fair
1.60 -0.8 0.82 perf-profile.children.cycles-pp.__switch_to_asm
1.99 -0.8 1.23 perf-profile.children.cycles-pp.dev_hard_start_xmit
1.85 ± 9% -0.7 1.13 ± 10% perf-profile.children.cycles-pp.tcp_send_mss
1.79 ± 2% -0.7 1.08 perf-profile.children.cycles-pp.switch_mm_irqs_off
1.73 ± 4% -0.7 1.02 ± 2% perf-profile.children.cycles-pp.select_idle_sibling
1.54 -0.7 0.85 perf-profile.children.cycles-pp._raw_spin_lock
1.78 -0.7 1.10 perf-profile.children.cycles-pp.loopback_xmit
1.85 ± 5% -0.7 1.17 ± 3% perf-profile.children.cycles-pp.__kfree_skb
1.99 -0.7 1.32 perf-profile.children.cycles-pp.tcp_clean_rtx_queue
1.95 ± 4% -0.7 1.29 perf-profile.children.cycles-pp._cond_resched
2.86 ± 2% -0.7 2.21 perf-profile.children.cycles-pp.enqueue_entity
3.78 -0.6 3.14 perf-profile.children.cycles-pp.__dev_queue_xmit
1.69 ± 11% -0.6 1.06 ± 11% perf-profile.children.cycles-pp.tcp_current_mss
1.58 ± 3% -0.6 0.95 perf-profile.children.cycles-pp.dequeue_entity
1.45 ± 12% -0.6 0.87 ± 4% perf-profile.children.cycles-pp._raw_spin_lock_bh
1.19 ± 6% -0.6 0.62 perf-profile.children.cycles-pp.simple_copy_to_iter
1.96 -0.6 1.40 perf-profile.children.cycles-pp.sk_stream_alloc_skb
1.69 ± 9% -0.6 1.14 ± 3% perf-profile.children.cycles-pp.lock_sock_nested
1.19 -0.5 0.64 ± 2% perf-profile.children.cycles-pp.ktime_get
1.09 ± 6% -0.5 0.55 ± 2% perf-profile.children.cycles-pp.__check_object_size
1.52 ± 4% -0.5 0.99 ± 2% perf-profile.children.cycles-pp.preempt_schedule_common
1.21 ± 3% -0.5 0.68 perf-profile.children.cycles-pp.__update_load_avg_se
1.16 ± 3% -0.5 0.64 perf-profile.children.cycles-pp.load_new_mm_cr3
1.25 ± 6% -0.5 0.72 ± 2% perf-profile.children.cycles-pp.security_file_permission
2.35 -0.5 1.83 perf-profile.children.cycles-pp.__alloc_skb
1.29 -0.5 0.81 perf-profile.children.cycles-pp.__switch_to
0.92 ± 3% -0.5 0.44 perf-profile.children.cycles-pp.sched_clock_cpu
1.22 ± 3% -0.5 0.74 perf-profile.children.cycles-pp.ttwu_do_wakeup
1.26 ± 8% -0.5 0.79 ± 2% perf-profile.children.cycles-pp.pagecache_get_page
0.98 ± 3% -0.5 0.51 perf-profile.children.cycles-pp.sched_clock
0.95 ± 5% -0.5 0.49 perf-profile.children.cycles-pp.reweight_entity
0.87 ± 3% -0.5 0.41 perf-profile.children.cycles-pp.switch_fpu_return
1.05 ± 3% -0.4 0.60 perf-profile.children.cycles-pp.check_preempt_curr
0.93 ± 3% -0.4 0.49 ± 2% perf-profile.children.cycles-pp.native_sched_clock
1.13 ± 2% -0.4 0.70 perf-profile.children.cycles-pp.___might_sleep
1.13 ± 9% -0.4 0.70 ± 2% perf-profile.children.cycles-pp.find_get_entry
0.90 -0.4 0.48 ± 2% perf-profile.children.cycles-pp.pick_next_entity
0.76 ± 15% -0.4 0.35 perf-profile.children.cycles-pp.update_cfs_group
0.83 ± 2% -0.4 0.43 ± 11% perf-profile.children.cycles-pp.__sysvec_apic_timer_interrupt
0.99 ± 3% -0.4 0.60 perf-profile.children.cycles-pp.copy_page_to_iter
0.80 ± 2% -0.4 0.41 ± 12% perf-profile.children.cycles-pp.hrtimer_interrupt
0.90 ± 2% -0.4 0.51 ± 12% perf-profile.children.cycles-pp.sysvec_apic_timer_interrupt
0.94 ± 6% -0.4 0.56 ± 2% perf-profile.children.cycles-pp.common_file_perm
0.82 -0.4 0.45 perf-profile.children.cycles-pp.netif_rx_internal
0.86 ± 2% -0.4 0.48 ± 2% perf-profile.children.cycles-pp.netif_rx
0.70 ± 2% -0.4 0.34 perf-profile.children.cycles-pp.sk_filter_trim_cap
0.89 ± 2% -0.4 0.53 ± 2% perf-profile.children.cycles-pp.check_preempt_wakeup
0.66 ± 2% -0.3 0.32 ± 3% perf-profile.children.cycles-pp.__inet_lookup_established
0.69 ± 2% -0.3 0.35 perf-profile.children.cycles-pp.read_tsc
0.67 -0.3 0.33 ± 2% perf-profile.children.cycles-pp.__netif_receive_skb_core
0.68 -0.3 0.36 perf-profile.children.cycles-pp._raw_spin_lock_irqsave
1.06 ± 3% -0.3 0.74 perf-profile.children.cycles-pp.touch_atime
1.12 ± 2% -0.3 0.81 perf-profile.children.cycles-pp.__update_load_avg_cfs_rq
1.44 ± 2% -0.3 1.13 ± 2% perf-profile.children.cycles-pp.update_rq_clock
0.99 ± 12% -0.3 0.69 ± 18% perf-profile.children.cycles-pp.ipv4_mtu
0.90 ± 5% -0.3 0.59 ± 6% perf-profile.children.cycles-pp.skb_release_all
0.63 ± 4% -0.3 0.33 ± 3% perf-profile.children.cycles-pp.available_idle_cpu
0.79 ± 4% -0.3 0.49 perf-profile.children.cycles-pp.atime_needs_update
0.69 ± 2% -0.3 0.40 ± 2% perf-profile.children.cycles-pp.finish_task_switch
0.74 -0.3 0.46 perf-profile.children.cycles-pp.__might_fault
0.86 ± 5% -0.3 0.58 ± 5% perf-profile.children.cycles-pp.skb_release_head_state
1.22 -0.3 0.94 perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.81 ± 2% -0.3 0.53 perf-profile.children.cycles-pp.___perf_sw_event
0.78 ± 10% -0.3 0.51 ± 8% perf-profile.children.cycles-pp.dst_release
0.48 -0.3 0.22 perf-profile.children.cycles-pp._raw_spin_unlock_irqrestore
0.49 ± 3% -0.3 0.24 perf-profile.children.cycles-pp.__virt_addr_valid
0.59 -0.3 0.34 ± 2% perf-profile.children.cycles-pp.tcp_mstamp_refresh
0.60 ± 6% -0.3 0.35 ± 3% perf-profile.children.cycles-pp.skb_release_data
0.49 ± 4% -0.2 0.26 perf-profile.children.cycles-pp.__calc_delta
0.56 ± 4% -0.2 0.33 ± 2% perf-profile.children.cycles-pp.do_splice_to
0.52 -0.2 0.29 perf-profile.children.cycles-pp.syscall_return_via_sysret
0.41 ± 2% -0.2 0.19 ± 2% perf-profile.children.cycles-pp.copy_fpregs_to_fpstate
0.47 ± 4% -0.2 0.26 ± 3% perf-profile.children.cycles-pp.page_cache_pipe_buf_release
0.60 ± 2% -0.2 0.39 perf-profile.children.cycles-pp.sk_reset_timer
0.42 -0.2 0.22 perf-profile.children.cycles-pp.irqtime_account_irq
0.43 ± 4% -0.2 0.23 ± 2% perf-profile.children.cycles-pp._copy_from_user
0.66 -0.2 0.45 ± 2% perf-profile.children.cycles-pp.__might_sleep
0.46 ± 2% -0.2 0.27 perf-profile.children.cycles-pp.enqueue_to_backlog
0.54 ± 2% -0.2 0.35 ± 2% perf-profile.children.cycles-pp.mod_timer
0.41 ± 3% -0.2 0.23 ± 2% perf-profile.children.cycles-pp.__enqueue_entity
0.51 -0.2 0.33 ± 2% perf-profile.children.cycles-pp.__fget_light
0.40 ± 2% -0.2 0.23 ± 12% perf-profile.children.cycles-pp.__hrtimer_run_queues
0.95 -0.2 0.79 ± 10% perf-profile.children.cycles-pp.asm_sysvec_apic_timer_interrupt
0.38 -0.2 0.21 ± 2% perf-profile.children.cycles-pp.clear_buddies
0.45 ± 3% -0.2 0.29 perf-profile.children.cycles-pp.update_min_vruntime
0.41 ± 2% -0.2 0.24 perf-profile.children.cycles-pp.__wrgsbase_inactive
0.46 ± 5% -0.2 0.30 perf-profile.children.cycles-pp.current_time
0.34 ± 4% -0.2 0.18 ± 2% perf-profile.children.cycles-pp.xas_load
0.33 ± 4% -0.2 0.17 perf-profile.children.cycles-pp._find_next_bit
0.31 ± 5% -0.2 0.16 ± 2% perf-profile.children.cycles-pp.page_cache_pipe_buf_confirm
0.32 ± 2% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.cpuacct_charge
0.32 ± 2% -0.1 0.17 ± 17% perf-profile.children.cycles-pp.tick_sched_timer
0.29 ± 2% -0.1 0.15 ± 2% perf-profile.children.cycles-pp.__list_del_entry_valid
0.32 ± 6% -0.1 0.18 ± 2% perf-profile.children.cycles-pp.cpumask_next_wrap
0.46 ± 3% -0.1 0.34 perf-profile.children.cycles-pp.account_entity_dequeue
0.23 -0.1 0.11 ± 4% perf-profile.children.cycles-pp.security_sock_rcv_skb
0.21 -0.1 0.09 perf-profile.children.cycles-pp.inet_ehashfn
0.48 -0.1 0.36 ± 2% perf-profile.children.cycles-pp.__kmalloc_node_track_caller
0.24 ± 3% -0.1 0.12 ± 4% perf-profile.children.cycles-pp.__rdgsbase_inactive
0.24 ± 7% -0.1 0.12 ± 3% perf-profile.children.cycles-pp.ktime_get_with_offset
0.49 ± 2% -0.1 0.38 perf-profile.children.cycles-pp.tcp_schedule_loss_probe
0.32 ± 3% -0.1 0.21 ± 3% perf-profile.children.cycles-pp.tcp_rcv_space_adjust
0.30 ± 2% -0.1 0.19 ± 2% perf-profile.children.cycles-pp.kfree
0.18 ± 5% -0.1 0.07 ± 11% perf-profile.children.cycles-pp.ktime_get_update_offsets_now
0.22 -0.1 0.11 perf-profile.children.cycles-pp.mark_page_accessed
0.21 ± 2% -0.1 0.11 perf-profile.children.cycles-pp.tcp_release_cb
0.24 ± 4% -0.1 0.14 ± 19% perf-profile.children.cycles-pp.update_process_times
0.54 -0.1 0.44 perf-profile.children.cycles-pp.__kmalloc_reserve
0.34 ± 5% -0.1 0.24 ± 2% perf-profile.children.cycles-pp.account_entity_enqueue
0.24 ± 4% -0.1 0.15 ± 17% perf-profile.children.cycles-pp.tick_sched_handle
0.26 ± 5% -0.1 0.17 ± 2% perf-profile.children.cycles-pp.set_next_buddy
0.19 ± 6% -0.1 0.10 ± 11% perf-profile.children.cycles-pp.clockevents_program_event
0.17 ± 6% -0.1 0.08 perf-profile.children.cycles-pp.ksoftirqd_running
0.22 -0.1 0.13 perf-profile.children.cycles-pp.ip_rcv_core
0.20 ± 2% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.resched_curr
0.17 ± 6% -0.1 0.08 ± 6% perf-profile.children.cycles-pp.tcp_rate_check_app_limited
0.16 ± 5% -0.1 0.07 ± 5% perf-profile.children.cycles-pp.tcp_event_data_recv
0.23 ± 13% -0.1 0.15 ± 7% perf-profile.children.cycles-pp.tcp_add_backlog
0.20 ± 4% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.sanity
0.27 ± 3% -0.1 0.19 ± 2% perf-profile.children.cycles-pp.kmem_cache_free
0.68 ± 5% -0.1 0.59 ± 9% perf-profile.children.cycles-pp.ipv4_dst_check
0.42 ± 2% -0.1 0.34 perf-profile.children.cycles-pp.__fsnotify_parent
0.17 ± 2% -0.1 0.10 ± 5% perf-profile.children.cycles-pp.ip_send_check
0.13 ± 6% -0.1 0.06 ± 9% perf-profile.children.cycles-pp.find_first_bit
0.19 ± 5% -0.1 0.11 perf-profile.children.cycles-pp.rw_verify_area
0.19 ± 5% -0.1 0.11 ± 3% perf-profile.children.cycles-pp.skb_entail
0.18 ± 6% -0.1 0.10 perf-profile.children.cycles-pp.__cgroup_account_cputime
0.21 ± 2% -0.1 0.14 perf-profile.children.cycles-pp.rb_next
0.17 ± 4% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.xas_start
0.17 ± 3% -0.1 0.09 ± 4% perf-profile.children.cycles-pp.__list_add_valid
0.18 ± 2% -0.1 0.11 ± 4% perf-profile.children.cycles-pp.remove_wait_queue
0.21 ± 5% -0.1 0.14 ± 3% perf-profile.children.cycles-pp.__put_user_8
0.29 ± 2% -0.1 0.22 perf-profile.children.cycles-pp.tcp_established_options
0.15 ± 8% -0.1 0.09 ± 5% perf-profile.children.cycles-pp.tcp_queue_rcv
0.17 ± 2% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.tcp_wfree
0.10 ± 4% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.check_stack_object
0.14 ± 8% -0.1 0.08 ± 5% perf-profile.children.cycles-pp.copy_user_generic_unrolled
0.13 ± 8% -0.1 0.07 ± 10% perf-profile.children.cycles-pp.inet_send_prepare
0.12 ± 3% -0.1 0.06 perf-profile.children.cycles-pp.ttwu_queue_wakelist
0.12 -0.1 0.06 perf-profile.children.cycles-pp.tcp_update_pacing_rate
0.10 ± 4% -0.1 0.04 ± 57% perf-profile.children.cycles-pp.tcp_v4_fill_cb
0.26 -0.1 0.20 ± 4% perf-profile.children.cycles-pp.tcp_ack_update_rtt
0.12 ± 3% -0.1 0.06 perf-profile.children.cycles-pp.cpumask_next
0.27 -0.1 0.21 ± 2% perf-profile.children.cycles-pp.fsnotify
0.16 ± 5% -0.1 0.10 perf-profile.children.cycles-pp.timestamp_truncate
0.12 ± 4% -0.1 0.07 perf-profile.children.cycles-pp.tcp_push
0.11 ± 4% -0.1 0.06 perf-profile.children.cycles-pp.__usecs_to_jiffies
0.08 ± 10% -0.1 0.03 ±100% perf-profile.children.cycles-pp.kthread_should_stop
0.20 ± 4% -0.1 0.15 ± 3% perf-profile.children.cycles-pp.rb_insert_color
0.20 ± 4% -0.1 0.15 perf-profile.children.cycles-pp.rcu_all_qs
0.15 ± 3% -0.1 0.10 ± 4% perf-profile.children.cycles-pp.syscall_enter_from_user_mode
0.11 ± 7% -0.0 0.07 ± 7% perf-profile.children.cycles-pp.apparmor_file_permission
0.15 ± 7% -0.0 0.10 ± 7% perf-profile.children.cycles-pp.ktime_get_coarse_real_ts64
0.10 ± 4% -0.0 0.05 perf-profile.children.cycles-pp.woken_wake_function
0.23 ± 3% -0.0 0.18 perf-profile.children.cycles-pp.__ksize
0.21 ± 5% -0.0 0.16 ± 4% perf-profile.children.cycles-pp.aa_file_perm
0.17 ± 6% -0.0 0.12 ± 3% perf-profile.children.cycles-pp.nf_hook_slow
0.14 ± 6% -0.0 0.09 ± 19% perf-profile.children.cycles-pp.scheduler_tick
0.10 ± 9% -0.0 0.05 perf-profile.children.cycles-pp.__tcp_ack_snd_check
0.28 -0.0 0.23 ± 2% perf-profile.children.cycles-pp.__ip_local_out
0.15 ± 4% -0.0 0.11 ± 4% perf-profile.children.cycles-pp.place_entity
0.11 ± 7% -0.0 0.07 perf-profile.children.cycles-pp.tcp_tx_timestamp
0.11 -0.0 0.07 ± 6% perf-profile.children.cycles-pp.kfree_skbmem
0.11 -0.0 0.07 ± 6% perf-profile.children.cycles-pp.wakeup_preempt_entity
0.09 -0.0 0.05 perf-profile.children.cycles-pp.bictcp_acked
0.36 ± 5% -0.0 0.33 ± 6% perf-profile.children.cycles-pp.__ip_finish_output
0.09 ± 4% -0.0 0.05 perf-profile.children.cycles-pp.perf_swevent_event
0.08 ± 10% -0.0 0.05 ± 60% perf-profile.children.cycles-pp.task_tick_fair
0.60 ± 2% -0.0 0.57 perf-profile.children.cycles-pp.__cgroup_bpf_run_filter_skb
0.56 ± 3% -0.0 0.52 perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.14 ± 8% -0.0 0.10 ± 8% perf-profile.children.cycles-pp.eth_type_trans
0.10 -0.0 0.07 ± 7% perf-profile.children.cycles-pp.tcp_stream_memory_free
0.28 -0.0 0.24 perf-profile.children.cycles-pp.rb_erase
0.09 -0.0 0.06 ± 7% perf-profile.children.cycles-pp.lock_timer_base
0.17 ± 5% -0.0 0.14 ± 3% perf-profile.children.cycles-pp.add_wait_queue
0.12 ± 4% -0.0 0.10 ± 4% perf-profile.children.cycles-pp.skb_clone_tx_timestamp
0.09 ± 8% -0.0 0.07 ± 6% perf-profile.children.cycles-pp.skb_network_protocol
0.09 ± 4% -0.0 0.07 perf-profile.children.cycles-pp.tcp_rate_gen
0.06 -0.0 0.04 ± 57% perf-profile.children.cycles-pp.perf_exclude_event
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.children.cycles-pp.__kfree_skb_flush
0.11 ± 4% -0.0 0.09 perf-profile.children.cycles-pp.tcp_cleanup_rbuf
0.07 ± 13% -0.0 0.05 perf-profile.children.cycles-pp.cgroup_rstat_updated
0.30 -0.0 0.29 perf-profile.children.cycles-pp.ip_local_out
0.13 ± 3% +0.0 0.14 ± 3% perf-profile.children.cycles-pp.generic_file_read_iter
0.09 ± 7% +0.0 0.11 perf-profile.children.cycles-pp.bictcp_cwnd_event
0.16 ± 4% +0.0 0.19 ± 4% perf-profile.children.cycles-pp.tcp_update_skb_after_send
0.15 ± 2% +0.0 0.18 ± 2% perf-profile.children.cycles-pp.tcp_rearm_rto
0.05 ± 8% +0.0 0.10 ± 5% perf-profile.children.cycles-pp.hrtick_update
0.06 +0.0 0.10 ± 4% perf-profile.children.cycles-pp.should_failslab
0.08 ± 6% +0.0 0.12 ± 3% perf-profile.children.cycles-pp.tcp_recv_timestamp
0.20 ± 2% +0.1 0.25 perf-profile.children.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.06 ± 7% perf-profile.children.cycles-pp.skb_push
0.00 +0.1 0.06 perf-profile.children.cycles-pp.tcp_options_write
0.10 ± 4% +0.1 0.17 ± 2% perf-profile.children.cycles-pp.validate_xmit_xfrm
0.07 +0.1 0.14 ± 3% perf-profile.children.cycles-pp.__copy_skb_header
0.00 +0.1 0.07 ± 7% perf-profile.children.cycles-pp.jiffies_to_usecs
0.00 +0.1 0.07 ± 31% perf-profile.children.cycles-pp.run_ksoftirqd
0.00 +0.1 0.08 ± 19% perf-profile.children.cycles-pp.apparmor_socket_recvmsg
0.24 ± 8% +0.1 0.32 ± 6% perf-profile.children.cycles-pp.sock_recvmsg
0.00 +0.1 0.08 ± 5% perf-profile.children.cycles-pp.kthread_should_park
0.29 +0.1 0.37 ± 2% perf-profile.children.cycles-pp.entry_SYSCALL_64
0.21 ± 10% +0.1 0.30 ± 6% perf-profile.children.cycles-pp.security_socket_recvmsg
0.07 ± 5% +0.1 0.16 ± 2% perf-profile.children.cycles-pp.tcp_small_queue_check
0.00 +0.1 0.09 ± 4% perf-profile.children.cycles-pp.dev_queue_xmit
0.11 +0.1 0.21 ± 2% perf-profile.children.cycles-pp.__tcp_select_window
0.11 ± 4% +0.1 0.21 ± 2% perf-profile.children.cycles-pp.tcp_rate_skb_sent
0.21 ± 7% +0.1 0.34 ± 10% perf-profile.children.cycles-pp.netif_skb_features
0.41 ± 3% +0.1 0.54 ± 2% perf-profile.children.cycles-pp.ip_finish_output
0.32 ± 2% +0.2 0.51 ± 2% perf-profile.children.cycles-pp.update_irq_load_avg
0.25 +0.2 0.46 ± 2% perf-profile.children.cycles-pp.__skb_clone
0.00 +0.2 0.23 ± 3% perf-profile.children.cycles-pp.splice_from_pipe_next
0.46 ± 5% +0.4 0.82 perf-profile.children.cycles-pp.validate_xmit_skb
9.13 ± 2% +1.2 10.32 perf-profile.children.cycles-pp.release_sock
8.01 +1.6 9.62 perf-profile.children.cycles-pp.__release_sock
0.41 +2.2 2.65 perf-profile.children.cycles-pp.__x86_indirect_thunk_rax
48.87 +3.4 52.25 perf-profile.children.cycles-pp.__x64_sys_sendfile64
47.78 +3.7 51.51 perf-profile.children.cycles-pp.do_sendfile
45.47 +4.3 49.82 perf-profile.children.cycles-pp.do_splice_direct
44.97 +4.6 49.53 perf-profile.children.cycles-pp.splice_direct_to_actor
30.59 ± 2% +4.6 35.19 perf-profile.children.cycles-pp.tcp_recvmsg
30.76 +4.6 35.35 perf-profile.children.cycles-pp.inet_recvmsg
31.65 +4.6 36.26 perf-profile.children.cycles-pp.__x64_sys_recvfrom
31.52 +4.6 36.16 perf-profile.children.cycles-pp.__sys_recvfrom
88.35 +4.7 93.06 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
38.81 +6.6 45.43 perf-profile.children.cycles-pp.direct_splice_actor
38.57 +6.7 45.26 perf-profile.children.cycles-pp.splice_from_pipe
38.27 +6.8 45.08 perf-profile.children.cycles-pp.__splice_from_pipe
36.74 +7.3 44.03 perf-profile.children.cycles-pp.pipe_to_sendpage
36.27 +7.5 43.79 perf-profile.children.cycles-pp.sock_sendpage
24.75 +7.6 32.36 perf-profile.children.cycles-pp.tcp_sendpage_locked
35.97 +7.7 43.63 perf-profile.children.cycles-pp.kernel_sendpage
35.77 +7.7 43.51 perf-profile.children.cycles-pp.inet_sendpage
24.38 +7.8 32.18 perf-profile.children.cycles-pp.do_tcp_sendpages
80.92 +8.0 88.88 perf-profile.children.cycles-pp.do_syscall_64
35.18 +8.0 43.22 perf-profile.children.cycles-pp.tcp_sendpage
18.11 ± 2% +9.3 27.45 perf-profile.children.cycles-pp.__tcp_push_pending_frames
22.94 +12.5 35.42 perf-profile.children.cycles-pp.tcp_write_xmit
28.39 +23.0 51.40 perf-profile.children.cycles-pp.ip_output
27.15 +23.1 50.23 perf-profile.children.cycles-pp.ip_finish_output2
29.75 +23.2 52.98 perf-profile.children.cycles-pp.__ip_queue_xmit
23.12 +23.5 46.61 perf-profile.children.cycles-pp.__local_bh_enable_ip
22.57 +23.7 46.25 perf-profile.children.cycles-pp.asm_call_sysvec_on_stack
22.53 +23.7 46.25 perf-profile.children.cycles-pp.do_softirq
22.07 +24.0 46.07 perf-profile.children.cycles-pp.do_softirq_own_stack
32.07 +24.1 56.22 perf-profile.children.cycles-pp.__tcp_transmit_skb
21.68 +24.3 45.94 perf-profile.children.cycles-pp.__softirqentry_text_start
5.77 ± 4% -2.5 3.23 ± 2% perf-profile.self.cycles-pp.copy_user_enhanced_fast_string
1.92 ± 3% -1.0 0.93 ± 2% perf-profile.self.cycles-pp.update_load_avg
1.03 ±140% -0.9 0.15 perf-profile.self.cycles-pp.syscall_exit_to_user_mode
1.60 ± 2% -0.8 0.82 perf-profile.self.cycles-pp.__switch_to_asm
1.48 -0.7 0.77 perf-profile.self.cycles-pp._raw_spin_lock
1.55 ± 4% -0.7 0.84 ± 3% perf-profile.self.cycles-pp.update_curr
2.08 ± 2% -0.6 1.43 perf-profile.self.cycles-pp.__schedule
1.72 ± 4% -0.6 1.10 perf-profile.self.cycles-pp.do_tcp_sendpages
1.16 ± 3% -0.5 0.63 perf-profile.self.cycles-pp.load_new_mm_cr3
1.12 ± 2% -0.5 0.64 perf-profile.self.cycles-pp.__update_load_avg_se
1.22 ± 4% -0.5 0.76 ± 2% perf-profile.self.cycles-pp._raw_spin_lock_bh
1.21 -0.5 0.75 perf-profile.self.cycles-pp.__switch_to
0.86 ± 3% -0.5 0.41 perf-profile.self.cycles-pp.switch_fpu_return
0.91 ± 2% -0.4 0.47 perf-profile.self.cycles-pp.native_sched_clock
0.94 ± 3% -0.4 0.50 perf-profile.self.cycles-pp.pick_next_task_fair
1.09 ± 2% -0.4 0.68 perf-profile.self.cycles-pp.___might_sleep
0.74 ± 16% -0.4 0.34 ± 2% perf-profile.self.cycles-pp.update_cfs_group
0.70 -0.3 0.35 ± 2% perf-profile.self.cycles-pp.tcp_v4_rcv
0.96 ± 4% -0.3 0.62 ± 3% perf-profile.self.cycles-pp.generic_file_buffered_read
0.73 ± 6% -0.3 0.40 ± 2% perf-profile.self.cycles-pp.common_file_perm
0.65 -0.3 0.33 perf-profile.self.cycles-pp.__netif_receive_skb_core
0.66 ± 2% -0.3 0.34 perf-profile.self.cycles-pp.read_tsc
0.67 -0.3 0.35 perf-profile.self.cycles-pp._raw_spin_lock_irqsave
0.97 ± 11% -0.3 0.67 ± 18% perf-profile.self.cycles-pp.ipv4_mtu
0.61 ± 5% -0.3 0.32 ± 3% perf-profile.self.cycles-pp.available_idle_cpu
0.79 ± 4% -0.3 0.49 perf-profile.self.cycles-pp.copy_page_to_iter
0.61 ± 14% -0.3 0.32 ± 7% perf-profile.self.cycles-pp.tcp_current_mss
0.76 ± 11% -0.3 0.47 ± 3% perf-profile.self.cycles-pp.find_get_entry
1.07 ± 2% -0.3 0.79 perf-profile.self.cycles-pp.__update_load_avg_cfs_rq
1.08 ± 4% -0.3 0.82 ± 3% perf-profile.self.cycles-pp.__dev_queue_xmit
0.60 ± 3% -0.3 0.34 ± 5% perf-profile.self.cycles-pp.ktime_get
0.76 ± 9% -0.3 0.50 ± 9% perf-profile.self.cycles-pp.dst_release
0.53 ± 3% -0.3 0.27 perf-profile.self.cycles-pp.schedule
0.60 ± 2% -0.3 0.34 ± 2% perf-profile.self.cycles-pp.__splice_from_pipe
0.69 ± 2% -0.3 0.43 perf-profile.self.cycles-pp.___perf_sw_event
0.60 ± 6% -0.3 0.34 ± 3% perf-profile.self.cycles-pp.skb_release_data
0.52 ± 10% -0.2 0.27 ± 5% perf-profile.self.cycles-pp.__check_object_size
0.47 ± 3% -0.2 0.23 perf-profile.self.cycles-pp.__virt_addr_valid
0.47 -0.2 0.24 perf-profile.self.cycles-pp.pick_next_entity
0.46 ± 2% -0.2 0.23 perf-profile.self.cycles-pp.inet_sendpage
0.45 ± 3% -0.2 0.22 ± 4% perf-profile.self.cycles-pp.__inet_lookup_established
0.48 ± 3% -0.2 0.26 perf-profile.self.cycles-pp.__calc_delta
0.51 -0.2 0.29 ± 2% perf-profile.self.cycles-pp.syscall_return_via_sysret
0.46 ± 2% -0.2 0.24 perf-profile.self.cycles-pp.pipe_to_sendpage
0.41 ± 4% -0.2 0.19 perf-profile.self.cycles-pp.copy_fpregs_to_fpstate
0.56 ± 2% -0.2 0.34 perf-profile.self.cycles-pp.dequeue_task_fair
0.46 ± 5% -0.2 0.25 ± 3% perf-profile.self.cycles-pp.page_cache_pipe_buf_release
0.41 -0.2 0.20 ± 2% perf-profile.self.cycles-pp._raw_spin_unlock_irqrestore
0.47 ± 2% -0.2 0.26 perf-profile.self.cycles-pp.check_preempt_wakeup
0.57 -0.2 0.37 ± 2% perf-profile.self.cycles-pp.update_rq_clock
0.55 -0.2 0.35 perf-profile.self.cycles-pp.__local_bh_enable_ip
0.45 -0.2 0.25 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.68 ± 2% -0.2 0.48 ± 2% perf-profile.self.cycles-pp.do_sendfile
0.54 ± 2% -0.2 0.35 perf-profile.self.cycles-pp.__skb_datagram_iter
0.62 ± 2% -0.2 0.43 perf-profile.self.cycles-pp.switch_mm_irqs_off
0.39 ± 3% -0.2 0.21 ± 2% perf-profile.self.cycles-pp.tcp_sendpage
0.47 ± 3% -0.2 0.29 perf-profile.self.cycles-pp.tcp_ack
0.50 ± 2% -0.2 0.32 perf-profile.self.cycles-pp.__fget_light
0.51 -0.2 0.34 perf-profile.self.cycles-pp.set_next_entity
0.39 ± 3% -0.2 0.21 ± 2% perf-profile.self.cycles-pp.__enqueue_entity
0.59 -0.2 0.42 ± 2% perf-profile.self.cycles-pp.__might_sleep
0.41 ± 2% -0.2 0.24 perf-profile.self.cycles-pp.__wrgsbase_inactive
0.53 ± 4% -0.2 0.36 perf-profile.self.cycles-pp.select_idle_sibling
0.36 -0.2 0.20 ± 2% perf-profile.self.cycles-pp.loopback_xmit
0.49 ± 3% -0.2 0.33 perf-profile.self.cycles-pp.tcp_clean_rtx_queue
0.32 -0.2 0.16 perf-profile.self.cycles-pp.cpuacct_charge
0.32 ± 4% -0.2 0.16 perf-profile.self.cycles-pp._find_next_bit
0.68 ± 2% -0.2 0.53 ± 2% perf-profile.self.cycles-pp.enqueue_task_fair
0.47 ± 2% -0.2 0.32 perf-profile.self.cycles-pp.finish_task_switch
0.43 ± 4% -0.2 0.28 ± 2% perf-profile.self.cycles-pp.update_min_vruntime
0.30 ± 3% -0.1 0.15 ± 2% perf-profile.self.cycles-pp.do_softirq
0.30 ± 4% -0.1 0.15 ± 2% perf-profile.self.cycles-pp.page_cache_pipe_buf_confirm
0.31 ± 3% -0.1 0.17 ± 3% perf-profile.self.cycles-pp.put_prev_entity
0.40 ± 4% -0.1 0.26 ± 4% perf-profile.self.cycles-pp.enqueue_entity
0.32 ± 8% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.atime_needs_update
0.32 ± 2% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.clear_buddies
0.31 -0.1 0.18 ± 2% perf-profile.self.cycles-pp.ip_output
0.27 ± 5% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.asm_call_sysvec_on_stack
0.29 ± 2% -0.1 0.16 perf-profile.self.cycles-pp.sock_sendpage
0.27 ± 4% -0.1 0.14 perf-profile.self.cycles-pp.splice_from_pipe
0.26 -0.1 0.13 perf-profile.self.cycles-pp.wait_woken
0.29 ± 5% -0.1 0.16 ± 2% perf-profile.self.cycles-pp.sk_wait_data
0.50 ± 3% -0.1 0.38 perf-profile.self.cycles-pp.try_to_wake_up
0.25 ± 3% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.__list_del_entry_valid
0.28 ± 4% -0.1 0.16 ± 4% perf-profile.self.cycles-pp.enqueue_to_backlog
0.40 ± 2% -0.1 0.28 ± 2% perf-profile.self.cycles-pp.tcp_rcv_established
0.24 -0.1 0.12 ± 3% perf-profile.self.cycles-pp.__rdgsbase_inactive
0.18 ± 5% -0.1 0.07 ± 16% perf-profile.self.cycles-pp.ktime_get_update_offsets_now
0.20 ± 2% -0.1 0.09 perf-profile.self.cycles-pp.inet_ehashfn
0.39 ± 3% -0.1 0.28 perf-profile.self.cycles-pp.smpboot_thread_fn
0.22 ± 3% -0.1 0.11 ± 3% perf-profile.self.cycles-pp.tcp_sendpage_locked
0.23 ± 3% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.tcp_send_mss
0.28 ± 2% -0.1 0.17 perf-profile.self.cycles-pp.mod_timer
0.19 ± 2% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.sk_filter_trim_cap
0.42 ± 2% -0.1 0.32 perf-profile.self.cycles-pp.account_entity_dequeue
0.35 -0.1 0.24 ± 2% perf-profile.self.cycles-pp.__alloc_skb
0.21 ± 7% -0.1 0.11 ± 7% perf-profile.self.cycles-pp.tcp_v4_do_rcv
0.29 ± 2% -0.1 0.19 ± 2% perf-profile.self.cycles-pp.kfree
0.20 ± 2% -0.1 0.10 perf-profile.self.cycles-pp.mark_page_accessed
0.20 ± 2% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.tcp_release_cb
0.28 ± 2% -0.1 0.19 ± 2% perf-profile.self.cycles-pp.select_task_rq_fair
0.22 -0.1 0.13 ± 3% perf-profile.self.cycles-pp.ip_rcv_core
0.27 ± 3% -0.1 0.18 ± 2% perf-profile.self.cycles-pp.reweight_entity
0.30 ± 2% -0.1 0.21 ± 2% perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.24 ± 2% -0.1 0.15 ± 2% perf-profile.self.cycles-pp.__might_fault
0.19 ± 2% -0.1 0.10 perf-profile.self.cycles-pp.resched_curr
0.16 ± 2% -0.1 0.07 perf-profile.self.cycles-pp.tcp_event_data_recv
0.24 ± 6% -0.1 0.16 ± 2% perf-profile.self.cycles-pp.set_next_buddy
0.19 ± 5% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.security_file_permission
0.16 ± 6% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.ksoftirqd_running
0.42 ± 2% -0.1 0.33 perf-profile.self.cycles-pp.__fsnotify_parent
0.16 ± 6% -0.1 0.08 ± 6% perf-profile.self.cycles-pp.tcp_rate_check_app_limited
0.14 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.ip_local_deliver
0.16 ± 5% -0.1 0.08 perf-profile.self.cycles-pp.check_preempt_curr
0.30 ± 4% -0.1 0.22 perf-profile.self.cycles-pp.account_entity_enqueue
0.26 -0.1 0.18 ± 2% perf-profile.self.cycles-pp.kmem_cache_free
0.17 ± 4% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.xas_load
0.17 ± 2% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.ip_send_check
0.16 -0.1 0.08 perf-profile.self.cycles-pp.sock_def_readable
0.15 ± 5% -0.1 0.07 perf-profile.self.cycles-pp.do_softirq_own_stack
0.39 -0.1 0.31 ± 2% perf-profile.self.cycles-pp.process_backlog
0.15 ± 3% -0.1 0.07 perf-profile.self.cycles-pp.__netif_receive_skb_one_core
0.18 ± 2% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.kernel_sendpage
0.12 ± 4% -0.1 0.05 perf-profile.self.cycles-pp.find_first_bit
0.30 ± 2% -0.1 0.23 ± 2% perf-profile.self.cycles-pp.__kmalloc_node_track_caller
0.24 -0.1 0.16 ± 6% perf-profile.self.cycles-pp.lock_sock_nested
0.18 ± 2% -0.1 0.10 ± 4% perf-profile.self.cycles-pp.sanity
0.21 ± 2% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.dev_hard_start_xmit
0.21 ± 2% -0.1 0.13 ± 3% perf-profile.self.cycles-pp.rb_next
0.13 ± 6% -0.1 0.06 perf-profile.self.cycles-pp.cpumask_next_wrap
0.11 ± 11% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.__cgroup_account_cputime
0.65 ± 5% -0.1 0.58 ± 9% perf-profile.self.cycles-pp.ipv4_dst_check
0.14 ± 3% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.skb_release_head_state
0.15 ± 6% -0.1 0.08 ± 5% perf-profile.self.cycles-pp.xas_start
0.22 ± 3% -0.1 0.15 ± 3% perf-profile.self.cycles-pp._cond_resched
0.14 ± 3% -0.1 0.07 ± 6% perf-profile.self.cycles-pp.security_sock_rcv_skb
0.09 ± 4% -0.1 0.03 ±100% perf-profile.self.cycles-pp.tcp_v4_fill_cb
0.25 -0.1 0.18 ± 2% perf-profile.self.cycles-pp.release_sock
0.20 ± 3% -0.1 0.14 ± 3% perf-profile.self.cycles-pp.__put_user_8
0.15 ± 3% -0.1 0.09 perf-profile.self.cycles-pp.__list_add_valid
0.11 ± 4% -0.1 0.05 perf-profile.self.cycles-pp.netif_rx_internal
0.87 ± 3% -0.1 0.81 perf-profile.self.cycles-pp.tcp_write_xmit
0.17 ± 4% -0.1 0.11 perf-profile.self.cycles-pp.current_time
0.14 ± 3% -0.1 0.08 perf-profile.self.cycles-pp.tcp_queue_rcv
0.13 ± 8% -0.1 0.07 ± 7% perf-profile.self.cycles-pp.ktime_get_with_offset
0.09 ± 4% -0.1 0.03 ±100% perf-profile.self.cycles-pp.tcp_add_backlog
0.17 ± 4% -0.1 0.11 ± 4% perf-profile.self.cycles-pp.rw_verify_area
0.14 ± 5% -0.1 0.08 perf-profile.self.cycles-pp.irqtime_account_irq
0.39 -0.1 0.33 perf-profile.self.cycles-pp.splice_direct_to_actor
0.13 ± 8% -0.1 0.07 ± 5% perf-profile.self.cycles-pp.copy_user_generic_unrolled
0.10 ± 4% -0.1 0.04 ± 57% perf-profile.self.cycles-pp.woken_wake_function
0.27 -0.1 0.21 ± 2% perf-profile.self.cycles-pp.fsnotify
0.16 -0.1 0.10 ± 4% perf-profile.self.cycles-pp.tcp_wfree
0.12 ± 7% -0.1 0.06 ± 11% perf-profile.self.cycles-pp.inet_send_prepare
0.12 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.ttwu_queue_wakelist
0.12 ± 3% -0.1 0.06 perf-profile.self.cycles-pp.tcp_update_pacing_rate
0.27 -0.1 0.21 ± 2% perf-profile.self.cycles-pp.tcp_established_options
0.31 ± 4% -0.1 0.26 perf-profile.self.cycles-pp.generic_file_splice_read
0.12 ± 4% -0.1 0.07 perf-profile.self.cycles-pp.ip_protocol_deliver_rcu
0.19 ± 4% -0.1 0.14 perf-profile.self.cycles-pp.rb_insert_color
0.14 ± 5% -0.1 0.09 ± 4% perf-profile.self.cycles-pp.timestamp_truncate
0.10 -0.1 0.05 perf-profile.self.cycles-pp.copyout
0.10 -0.1 0.05 perf-profile.self.cycles-pp.__usecs_to_jiffies
0.12 -0.0 0.07 perf-profile.self.cycles-pp.tcp_push
0.15 ± 3% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.syscall_enter_from_user_mode
0.66 ± 2% -0.0 0.61 ± 2% perf-profile.self.cycles-pp.ip_finish_output2
0.14 ± 13% -0.0 0.10 ± 5% perf-profile.self.cycles-pp.aa_sk_perm
0.14 ± 7% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.ktime_get_coarse_real_ts64
0.11 ± 4% -0.0 0.06 perf-profile.self.cycles-pp.ip_rcv_finish
0.14 ± 8% -0.0 0.09 ± 7% perf-profile.self.cycles-pp.skb_entail
0.12 ± 4% -0.0 0.08 perf-profile.self.cycles-pp.ttwu_do_activate
0.10 ± 8% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.apparmor_file_permission
0.17 ± 2% -0.0 0.13 ± 6% perf-profile.self.cycles-pp.tcp_ack_update_rtt
0.26 ± 2% -0.0 0.22 perf-profile.self.cycles-pp.__x64_sys_sendfile64
0.22 -0.0 0.18 perf-profile.self.cycles-pp.__ksize
0.14 ± 3% -0.0 0.10 ± 5% perf-profile.self.cycles-pp.schedule_timeout
0.16 ± 6% -0.0 0.12 ± 3% perf-profile.self.cycles-pp.do_splice_direct
0.13 ± 6% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.pagecache_get_page
0.14 ± 5% -0.0 0.10 perf-profile.self.cycles-pp.place_entity
0.10 ± 4% -0.0 0.06 ± 7% perf-profile.self.cycles-pp.tcp_stream_memory_free
0.09 ± 4% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.ip_rcv
0.19 ± 6% -0.0 0.15 ± 2% perf-profile.self.cycles-pp.aa_file_perm
0.10 ± 4% -0.0 0.07 ± 7% perf-profile.self.cycles-pp.kfree_skbmem
0.10 -0.0 0.06 ± 6% perf-profile.self.cycles-pp.wakeup_preempt_entity
0.09 ± 4% -0.0 0.05 perf-profile.self.cycles-pp.bictcp_acked
0.12 ± 3% -0.0 0.08 perf-profile.self.cycles-pp.direct_splice_actor
0.17 ± 5% -0.0 0.14 ± 6% perf-profile.self.cycles-pp.ttwu_do_wakeup
0.14 ± 11% -0.0 0.10 ± 7% perf-profile.self.cycles-pp.eth_type_trans
0.22 ± 3% -0.0 0.18 ± 2% perf-profile.self.cycles-pp.tcp_schedule_loss_probe
0.12 ± 3% -0.0 0.09 perf-profile.self.cycles-pp.sk_stream_alloc_skb
0.10 ± 5% -0.0 0.06 ± 6% perf-profile.self.cycles-pp.tcp_tx_timestamp
0.08 ± 10% -0.0 0.05 perf-profile.self.cycles-pp._copy_from_user
0.08 ± 5% -0.0 0.05 ± 8% perf-profile.self.cycles-pp.__tcp_push_pending_frames
0.57 ± 2% -0.0 0.54 perf-profile.self.cycles-pp.__cgroup_bpf_run_filter_skb
0.36 ± 2% -0.0 0.33 perf-profile.self.cycles-pp.kmem_cache_alloc_node
0.08 -0.0 0.05 perf-profile.self.cycles-pp.skb_copy_datagram_iter
0.22 -0.0 0.19 perf-profile.self.cycles-pp.dequeue_entity
0.15 -0.0 0.12 ± 3% perf-profile.self.cycles-pp.rcu_all_qs
0.12 ± 5% -0.0 0.09 ± 4% perf-profile.self.cycles-pp.skb_clone_tx_timestamp
0.09 ± 4% -0.0 0.06 perf-profile.self.cycles-pp.tcp_rcv_space_adjust
0.17 ± 7% -0.0 0.14 ± 5% perf-profile.self.cycles-pp.tcp_event_new_data_sent
0.09 ± 8% -0.0 0.07 ± 6% perf-profile.self.cycles-pp.skb_network_protocol
0.09 ± 4% -0.0 0.07 perf-profile.self.cycles-pp.tcp_rate_gen
0.25 -0.0 0.23 perf-profile.self.cycles-pp.rb_erase
0.13 ± 5% -0.0 0.11 perf-profile.self.cycles-pp.sock_put
0.24 -0.0 0.22 perf-profile.self.cycles-pp.touch_atime
0.12 ± 3% -0.0 0.10 ± 4% perf-profile.self.cycles-pp.__x64_sys_recvfrom
0.10 ± 4% -0.0 0.08 ± 5% perf-profile.self.cycles-pp.simple_copy_to_iter
0.09 ± 4% -0.0 0.07 perf-profile.self.cycles-pp.__kfree_skb_flush
0.06 ± 6% -0.0 0.05 perf-profile.self.cycles-pp.cgroup_rstat_updated
0.05 +0.0 0.06 perf-profile.self.cycles-pp.find_next_bit
0.06 ± 7% +0.0 0.07 ± 5% perf-profile.self.cycles-pp.add_wait_queue
0.08 ± 5% +0.0 0.10 ± 5% perf-profile.self.cycles-pp.sock_rfree
0.12 ± 5% +0.0 0.14 ± 3% perf-profile.self.cycles-pp.generic_file_read_iter
0.09 ± 4% +0.0 0.11 perf-profile.self.cycles-pp.bictcp_cwnd_event
0.11 ± 3% +0.0 0.14 ± 3% perf-profile.self.cycles-pp.ip_finish_output
0.05 ± 9% +0.0 0.08 ± 5% perf-profile.self.cycles-pp.__kmalloc_reserve
0.07 +0.0 0.10 ± 4% perf-profile.self.cycles-pp.do_splice_to
0.10 +0.0 0.14 ± 3% perf-profile.self.cycles-pp.__ip_local_out
0.05 +0.0 0.10 ± 5% perf-profile.self.cycles-pp.hrtick_update
0.12 ± 3% +0.0 0.17 ± 3% perf-profile.self.cycles-pp.tcp_rearm_rto
0.07 ± 6% +0.0 0.12 ± 3% perf-profile.self.cycles-pp.tcp_recv_timestamp
0.00 +0.1 0.05 perf-profile.self.cycles-pp.ip_local_out
0.00 +0.1 0.05 perf-profile.self.cycles-pp.ip_queue_xmit
0.00 +0.1 0.05 ± 8% perf-profile.self.cycles-pp.skb_push
0.03 ±100% +0.1 0.08 ± 5% perf-profile.self.cycles-pp.should_failslab
0.07 +0.1 0.13 perf-profile.self.cycles-pp.__copy_skb_header
0.00 +0.1 0.06 perf-profile.self.cycles-pp.jiffies_to_usecs
0.00 +0.1 0.06 perf-profile.self.cycles-pp.tcp_options_write
0.09 ± 9% +0.1 0.15 ± 4% perf-profile.self.cycles-pp.tcp_update_skb_after_send
0.11 ± 3% +0.1 0.18 ± 2% perf-profile.self.cycles-pp.rcu_note_context_switch
0.00 +0.1 0.07 ± 12% perf-profile.self.cycles-pp.sockfd_lookup_light
0.10 ± 4% +0.1 0.17 ± 2% perf-profile.self.cycles-pp.validate_xmit_xfrm
0.00 +0.1 0.08 ± 22% perf-profile.self.cycles-pp.apparmor_socket_recvmsg
0.00 +0.1 0.08 perf-profile.self.cycles-pp.kthread_should_park
0.29 +0.1 0.37 ± 2% perf-profile.self.cycles-pp.entry_SYSCALL_64
0.07 +0.1 0.15 ± 3% perf-profile.self.cycles-pp.tcp_small_queue_check
0.00 +0.1 0.09 ± 4% perf-profile.self.cycles-pp.dev_queue_xmit
0.11 +0.1 0.20 ± 2% perf-profile.self.cycles-pp.__tcp_select_window
0.11 ± 4% +0.1 0.21 ± 2% perf-profile.self.cycles-pp.tcp_rate_skb_sent
0.17 ± 4% +0.2 0.33 ± 3% perf-profile.self.cycles-pp.__skb_clone
0.11 ± 7% +0.2 0.27 ± 15% perf-profile.self.cycles-pp.netif_skb_features
0.29 +0.2 0.45 perf-profile.self.cycles-pp.__x86_indirect_thunk_rax
0.31 ± 2% +0.2 0.50 ± 2% perf-profile.self.cycles-pp.update_irq_load_avg
0.51 +0.2 0.71 perf-profile.self.cycles-pp.__ip_queue_xmit
0.00 +0.2 0.23 ± 4% perf-profile.self.cycles-pp.splice_from_pipe_next
0.22 ± 5% +0.2 0.46 ± 10% perf-profile.self.cycles-pp.validate_xmit_skb
1.31 +0.4 1.71 ± 2% perf-profile.self.cycles-pp.__tcp_transmit_skb
0.41 ± 3% +4.4 4.79 perf-profile.self.cycles-pp.net_rx_action
0.57 +27.5 28.07 perf-profile.self.cycles-pp.__softirqentry_text_start
netperf.Throughput_Mbps
5500 +--------------------------------------------------------------------+
|. .+..+. .+.+.+..+. |
5000 |-+.+ +.+ +.+.+.+.+..+.+.+.+.+.+.+..+.+.+.+ .|
4500 |-+ + .+.+ |
| +.+..+ |
4000 |-+ |
3500 |-+ |
| |
3000 |-+ |
2500 |-+ O O O O |
| |
2000 |-+ |
1500 |-O O O O O O O O O O O O O O O O |
| O O O |
1000 +--------------------------------------------------------------------+
netperf.Throughput_total_Mbps
2e+06 +-----------------------------------------------------------------+
|.+.+.+.+.+.+ +.+.+.+.+.+.+.+. .+.+.+.+.. .+.+ |
1.8e+06 |-+ + + + .+.|
| +.+. .+ |
1.6e+06 |-+ + |
1.4e+06 |-+ |
| |
1.2e+06 |-+ |
| |
1e+06 |-+ O O O O |
800000 |-+ |
| |
600000 |-O O O O O O O O O O O O O O O O |
| O O O |
400000 +-----------------------------------------------------------------+
netperf.workload
8e+09 +-------------------------------------------------------------------+
| |
7e+09 |.+.+.+..+. .+.+.+.+.+.+..+.+.+.+.+.+.+. .+.+.+. .+ |
| + +..+ +.+ + .+.|
6e+09 |-+ +..+.+.+ |
| |
5e+09 |-+ |
| |
4e+09 |-+ |
| O O O O |
3e+09 |-+ |
| O O O O O O O O |
2e+09 |-O O O O O O O O O O O |
| |
1e+09 +-------------------------------------------------------------------+
netperf.time.system_time
32000 +-------------------------------------------------------------------+
| .+. |
30000 |.+.+.+..+.+ +.+.+.+.+..+.+.+.+.+.+.+.+..+.+.+.+.+.+.+ |
28000 |-+ + |
| +..+. .+.+.|
26000 |-+ + |
| |
24000 |-+ |
| |
22000 |-+ |
20000 |-+ O O O O |
| |
18000 |-+ |
| O O O O O O O O O O O O O O O O O |
16000 +-------------------------------------------------------------------+
netperf.time.percent_of_cpu_this_job_got
10500 +-------------------------------------------------------------------+
10000 |.+.+.+..+.+.+.+.+.+.+.+..+.+.+.+.+.+.+.+..+.+.+.+.+.+.+ |
| + |
9500 |-+ +..+.+.+.+.|
9000 |-+ |
8500 |-+ |
8000 |-+ |
| |
7500 |-+ |
7000 |-+ |
6500 |-+ O O O O |
6000 |-+ O O |
| O O O O O O O O O O O |
5500 |-+ O O O O O O |
5000 +-------------------------------------------------------------------+
netperf.time.involuntary_context_switches
4e+09 +-----------------------------------------------------------------+
|.+.+.+.+. .+. +.+.+.+.+.+.+.+.+.+. .+.+.+.+.. .+.+ |
3.5e+09 |-+ +.+ + + + |
| +.+. .+. |
3e+09 |-+ + +.|
| |
2.5e+09 |-+ |
| |
2e+09 |-+ |
| O O O O |
1.5e+09 |-+ |
| |
1e+09 |-O O O O O O O O O O O O O O O O O O O |
| |
5e+08 +-----------------------------------------------------------------+
[*] bisect-good sample
[O] bisect-bad sample
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
Thanks,
Rong Chen
4 months, 3 weeks
- 1
- 0
[HID] 9bdaeff2e2: kmsg.PM:Device_i2c-SYNA2B29:#failed_to_resume_async:error
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 9bdaeff2e281a849350f5e6fb194c9eed4a7abf7 ("[PATCH] HID: i2c-hid: add polling mode based on connected GPIO chip's pin status")
url: https://github.com/0day-ci/linux/commits/Coiby-Xu/HID-i2c-hid-add-polling...
base: https://git.kernel.org/cgit/linux/kernel/git/jikos/trivial.git for-next
in testcase: suspend-stress
version:
with following parameters:
mode: mem
iterations: 10
on test machine: 4 threads Skylake with 8G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
kern :info : [ 15.209413] i2c_hid i2c-SYNA2B29:00: calling i2c_hid_resume+0x0/0x100 [i2c_hid] @ 629, parent: i2c-2
kern :info : [ 15.210766] coretemp coretemp.0: platform_pm_resume+0x0/0x40 returned 0 after 0 usecs
kern :info : [ 15.210783] intel_pmc_core intel_pmc_core.0: calling platform_pm_resume+0x0/0x40 @ 565, parent: platform
kern :info : [ 15.214954] intel_pmc_core intel_pmc_core.0: platform_pm_resume+0x0/0x40 returned 0 after 0 usecs
kern :info : [ 15.216389] i2c_hid i2c-ELAN21EF:00: calling i2c_hid_resume+0x0/0x100 [i2c_hid] @ 632, parent: i2c-6
kern :info : [ 15.279476] i2c_hid i2c-ELAN21EF:00: i2c_hid_resume+0x0/0x100 [i2c_hid] returned 0 after 60240 usecs
kern :info : [ 15.280901] input input14: calling input_dev_resume+0x0/0x40 @ 565, parent: 0018:04F3:21EF.0004
kern :info : [ 15.282271] input input14: input_dev_resume+0x0/0x40 returned 0 after 0 usecs
kern :info : [ 15.283565] input input15: calling input_dev_resume+0x0/0x40 @ 565, parent: 0018:04F3:21EF.0004
kern :info : [ 15.284939] input input15: input_dev_resume+0x0/0x40 returned 0 after 0 usecs
kern :info : [ 15.286231] input input16: calling input_dev_resume+0x0/0x40 @ 565, parent: 0018:04F3:21EF.0004
kern :info : [ 15.287630] input input16: input_dev_resume+0x0/0x40 returned 0 after 0 usecs
kern :info : [ 15.638405] asix 1-1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xC1E1
kern :warn : [ 16.281467] hid-rmi 0018:06CB:77C6.0003: rmi_hid_read_block: timeout elapsed
kern :warn : [ 17.304554] hid-rmi 0018:06CB:77C6.0003: rmi_hid_read_block: timeout elapsed
kern :warn : [ 18.328558] hid-rmi 0018:06CB:77C6.0003: rmi_hid_read_block: timeout elapsed
kern :warn : [ 19.352476] hid-rmi 0018:06CB:77C6.0003: rmi_hid_read_block: timeout elapsed
kern :warn : [ 20.376477] hid-rmi 0018:06CB:77C6.0003: rmi_hid_read_block: timeout elapsed
kern :err : [ 20.377756] rmi4_physical rmi4-00: rmi_driver_reset_handler: Failed to read current IRQ mask.
kern :err : [ 20.379150] PM: dpm_run_callback(): i2c_hid_resume+0x0/0x100 [i2c_hid] returns -11
kern :info : [ 20.380478] i2c_hid i2c-SYNA2B29:00: i2c_hid_resume+0x0/0x100 [i2c_hid] returned -11 after 5047180 usecs
kern :err : [ 20.381966] PM: Device i2c-SYNA2B29:00 failed to resume async: error -11
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
4 months, 3 weeks
- 1
- 0
[PATCH] syscalls/ptrace10: Add new regression test
by Cyril Hrubis
New regression test for a kernel commit:
commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0
Author: Jiri Olsa <jolsa(a)kernel.org>
Date: Mon Aug 27 11:12:25 2018 +0200
perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set
Signed-off-by: Cyril Hrubis <chrubis(a)suse.cz>
CC: Andy Lutomirski <luto(a)kernel.org>
CC: Peter Zijlstra <peterz(a)infradead.org>
CC: Thomas Gleixner <tglx(a)linutronix.de>
CC: Alexandre Chartre <alexandre.chartre(a)oracle.com>
---
This is a follow up for the ptrace08 fixes.
runtest/syscalls | 1 +
testcases/kernel/syscalls/ptrace/.gitignore | 1 +
testcases/kernel/syscalls/ptrace/ptrace10.c | 86 +++++++++++++++++++++
3 files changed, 88 insertions(+)
create mode 100644 testcases/kernel/syscalls/ptrace/ptrace10.c
diff --git a/runtest/syscalls b/runtest/syscalls
index 398145f65..163471bcd 100644
--- a/runtest/syscalls
+++ b/runtest/syscalls
@@ -993,6 +993,7 @@ ptrace05 ptrace05
ptrace07 ptrace07
ptrace08 ptrace08
ptrace09 ptrace09
+ptrace10 ptrace10
pwrite01 pwrite01
pwrite02 pwrite02
diff --git a/testcases/kernel/syscalls/ptrace/.gitignore b/testcases/kernel/syscalls/ptrace/.gitignore
index 7639e1a9f..7ee3b3c47 100644
--- a/testcases/kernel/syscalls/ptrace/.gitignore
+++ b/testcases/kernel/syscalls/ptrace/.gitignore
@@ -5,3 +5,4 @@
/ptrace07
/ptrace08
/ptrace09
+/ptrace10
diff --git a/testcases/kernel/syscalls/ptrace/ptrace10.c b/testcases/kernel/syscalls/ptrace/ptrace10.c
new file mode 100644
index 000000000..b5d6b9f8f
--- /dev/null
+++ b/testcases/kernel/syscalls/ptrace/ptrace10.c
@@ -0,0 +1,86 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * Copyright (C) 2020 Cyril Hrubis <chrbis(a)suse.cz>
+ *
+ * After fix for CVE-2018-1000199 (see ptrace08.c) subsequent calls to POKEUSER
+ * for x86 debug registers were ignored silently.
+ *
+ * This is a regression test for commit:
+ *
+ * commit bd14406b78e6daa1ea3c1673bda1ffc9efdeead0
+ * Author: Jiri Olsa <jolsa(a)kernel.org>
+ * Date: Mon Aug 27 11:12:25 2018 +0200
+ *
+ * perf/hw_breakpoint: Modify breakpoint even if the new attr has disabled set
+ */
+
+#include <stdlib.h>
+#include <stdio.h>
+#include <stddef.h>
+#include <sys/ptrace.h>
+#include <sys/user.h>
+#include <signal.h>
+#include "tst_test.h"
+
+#if defined(__i386__) || defined(__x86_64__)
+
+static pid_t child_pid;
+
+static void child_main(void)
+{
+ raise(SIGSTOP);
+ exit(0);
+}
+
+static void run(void)
+{
+ int status;
+ unsigned long addr;
+
+ child_pid = SAFE_FORK();
+
+ if (!child_pid)
+ child_main();
+
+ if (SAFE_WAITPID(child_pid, &status, WUNTRACED) != child_pid)
+ tst_brk(TBROK, "Received event from unexpected PID");
+
+ SAFE_PTRACE(PTRACE_ATTACH, child_pid, NULL, NULL);
+ SAFE_PTRACE(PTRACE_POKEUSER, child_pid,
+ (void *)offsetof(struct user, u_debugreg[0]), (void *)1);
+ SAFE_PTRACE(PTRACE_POKEUSER, child_pid,
+ (void *)offsetof(struct user, u_debugreg[0]), (void *)2);
+
+ addr = ptrace(PTRACE_PEEKUSER, child_pid,
+ (void*)offsetof(struct user, u_debugreg[0]), NULL);
+
+ if (addr == 2)
+ tst_res(TPASS, "The rd0 was set on second PTRACE_POKEUSR");
+ else
+ tst_res(TFAIL, "The rd0 wasn't set on second PTRACE_POKEUSER");
+
+ SAFE_PTRACE(PTRACE_DETACH, child_pid, NULL, NULL);
+ SAFE_KILL(child_pid, SIGCONT);
+ child_pid = 0;
+ tst_reap_children();
+}
+
+static void cleanup(void)
+{
+ /* Main process terminated by tst_brk() with child still paused */
+ if (child_pid)
+ SAFE_KILL(child_pid, SIGKILL);
+}
+
+static struct tst_test test = {
+ .test_all = run,
+ .cleanup = cleanup,
+ .forks_child = 1,
+ .tags = (const struct tst_tag[]) {
+ {"linux-git", "bd14406b78e6"},
+ {}
+ }
+};
+#else
+TST_TEST_TCONF("This test is only supported on x86 systems");
+#endif
--
2.26.2
4 months, 3 weeks
- 2
- 4
[x86/hotplug] 52d6b926aa: WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_stats_print.cold[rcutorture]
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 52d6b926aabc47643cd910c85edb262b7f44c168 ("x86/hotplug: Silence APIC only after all interrupts are migrated")
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git master
in testcase: rcutorture
version:
with following parameters:
runtime: 300s
test: cpuhotplug
torture_type: rcu
test-description: rcutorture is rcutorture kernel module load/unload test.
test-url: https://www.kernel.org/doc/Documentation/RCU/torture.txt
on test machine: qemu-system-x86_64 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------------------------------------------+------------+------------+
| | d4f07268d0 | 52d6b926aa |
+------------------------------------------------------------------------------+------------+------------+
| boot_successes | 143 | 117 |
| boot_failures | 0 | 22 |
| WARNING:at_kernel/rcu/rcutorture.c:#rcu_torture_stats_print.cold[rcutorture] | 0 | 22 |
| RIP:rcu_torture_stats_print.cold[rcutorture] | 0 | 22 |
+------------------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 70.096042] WARNING: CPU: 0 PID: 498 at kernel/rcu/rcutorture.c:1563 rcu_torture_stats_print.cold+0x342/0x4c0 [rcutorture]
[ 70.098814] Modules linked in: rcutorture torture bochs_drm drm_vram_helper drm_ttm_helper ttm drm_kms_helper sr_mod syscopyarea sysfillrect sysimgblt cdrom fb_sys_fops drm intel_rapl_msr sg ppdev intel_rapl_common crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel ata_generic joydev serio_raw parport_pc parport ata_piix libata ipmi_devintf ipmi_msghandler i2c_piix4 ip_tables
[ 70.108257] CPU: 0 PID: 498 Comm: rcu_torture_sta Not tainted 5.8.0-13576-g52d6b926aabc47 #1
[ 70.110451] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 70.112642] RIP: 0010:rcu_torture_stats_print.cold+0x342/0x4c0 [rcutorture]
[ 70.114605] Code: 00 48 0b 05 29 ff 01 00 48 0b 05 1a ff 01 00 48 0b 05 0b ff 01 00 0f 85 20 fe ff ff 83 fd 01 0f 8e 76 fe ff ff e9 12 fe ff ff <0f> 0b e9 6a fe ff ff e8 d3 dd fc f4 83 3d fc e9 c5 f5 00 0f 85 78
[ 70.119208] RSP: 0018:ffffa24a005d7e00 EFLAGS: 00010202
[ 70.120939] RAX: 0000000000000000 RBX: ffffa24a005d7e20 RCX: 0000000000000000
[ 70.122968] RDX: 0000000000000000 RSI: ffffffffb69beca6 RDI: ffffffffb69bc86c
[ 70.125010] RBP: 0000000000000001 R08: ffffffffb69beca2 R09: 0000000000000004
[ 70.127046] R10: ffffffffb69bf080 R11: 00000000b69beca1 R12: ffffa24a005d7e78
[ 70.129059] R13: ffffffffc0630130 R14: 0000000000000000 R15: 0000000000001997
[ 70.131066] FS: 0000000000000000(0000) GS:ffff8c73ffc00000(0000) knlGS:0000000000000000
[ 70.133255] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 70.135081] CR2: 00007fcdcf159000 CR3: 00000001eca8e000 CR4: 00000000000406f0
[ 70.137126] Call Trace:
[ 70.138410] ? __schedule+0x34b/0x860
[ 70.139514] ? rcu_torture_stats_print+0xe0/0xe0 [rcutorture]
[ 70.141272] rcu_torture_stats+0x2a/0x60 [rcutorture]
[ 70.142961] kthread+0x114/0x160
[ 70.144382] ? kthread_park+0xa0/0xa0
[ 70.145486] ret_from_fork+0x22/0x30
[ 70.146750] ---[ end trace db6662cc642dd991 ]---
To reproduce:
# build kernel
cd linux
cp config-5.8.0-13576-g52d6b926aabc47 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=x86_64 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
4 months, 3 weeks
- 1
- 0
[selftests] 3ca8c91756: kernel-selftests.kvm.make_fail
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 3ca8c91756c3db8101879274075fc0a76b127a52 ("[PATCH v2 4/4] selftests: kvm: Test MSR exiting to userspace")
url: https://github.com/0day-ci/linux/commits/Aaron-Lewis/Test-MSR-exits-to-us...
base: https://git.kernel.org/cgit/virt/kvm/kvm.git linux-next
in testcase: kernel-selftests
version: kernel-selftests-x86_64-e8e8f16e-1_20200807
with following parameters:
group: kselftests-kvm
ucode: 0xdc
test-description: The kernel contains a set of "self tests" under the tools/testing/selftests/ directory. These are intended to be small unit tests to exercise individual code paths in the kernel.
test-url: https://www.kernel.org/doc/Documentation/kselftest.txt
on test machine: 8 threads Intel(R) Core(TM) i7-6770HQ CPU @ 2.60GHz with 32G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
KERNEL SELFTESTS: linux_headers_dir is /usr/src/linux-headers-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52
2020-10-13 13:46:59 ln -sf /usr/bin/clang
2020-10-13 13:46:59 ln -sf /usr/bin/llc
2020-10-13 13:46:59 sed -i s/default_timeout=45/default_timeout=300/ kselftest/runner.sh
2020-10-13 13:46:59 make run_tests -C kvm
make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm'
make --no-builtin-rules ARCH=x86 -C ../../../.. headers_install
make[1]: Entering directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52'
...
HDRINST usr/include/asm/bpf_perf_event.h
INSTALL ./usr/include
make[1]: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52'
gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ilib -Iinclude/x86_64 -I.. -c lib/assert.c -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm/lib/assert.o
gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ilib -Iinclude/x86_64 -I.. -c lib/elf.c -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm/lib/elf.o
gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ilib -Iinclude/x86_64 -I.. -c lib/io.c -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm/lib/io.o
gcc -Wall -Wstrict-prototypes -Wuninitialized -O2 -g -std=gnu99 -fno-stack-protector -fno-PIE -I../../../../tools/include -I../../../../tools/arch/x86/include -I../../../../usr/include/ -Iinclude -Ilib -Iinclude/x86_64 -I.. -c lib/kvm_util.c -o /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm/lib/kvm_util.o
lib/kvm_util.c:1598:3: error: ‘KVM_EXIT_X86_RDMSR’ undeclared here (not in a function); did you mean ‘KVM_EXIT_ARM_NISV’?
{KVM_EXIT_X86_RDMSR, "RDMSR"},
^~~~~~~~~~~~~~~~~~
KVM_EXIT_ARM_NISV
lib/kvm_util.c:1599:3: error: ‘KVM_EXIT_X86_WRMSR’ undeclared here (not in a function); did you mean ‘KVM_EXIT_ARM_NISV’?
{KVM_EXIT_X86_WRMSR, "WRMSR"},
^~~~~~~~~~~~~~~~~~
KVM_EXIT_ARM_NISV
make: *** [Makefile:120: /usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm/lib/kvm_util.o] Error 1
make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-7.6-kselftests-3ca8c91756c3db8101879274075fc0a76b127a52/tools/testing/selftests/kvm'
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
4 months, 3 weeks
- 1
- 0
[net] d18d22ce8f: UBSAN:Undefined_behaviour_in_net/core/dev.c
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: d18d22ce8f62839365c984b1df474d3975ed4eb2 ("net: Fix packet reordering caused by GRO and listified RX cooperation")
https://git.kernel.org/cgit/linux/kernel/git/stable/linux-stable-rc.git linux-5.4.y
in testcase: trinity
version: trinity-i386-4d2343bd-1_20200320
with following parameters:
runtime: 300s
test-description: Trinity is a linux system call fuzz tester.
test-url: http://codemonkey.org.uk/projects/trinity/
on test machine: qemu-system-i386 -enable-kvm -cpu SandyBridge -smp 2 -m 8G
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
+------------------------------------------------------------------+------------+------------+
| | 8bc3025206 | d18d22ce8f |
+------------------------------------------------------------------+------------+------------+
| boot_successes | 0 | 0 |
| boot_failures | 4 | 6 |
| UBSAN:Undefined_behaviour_in_arch/x86/kernel.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbprint.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbutils.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbdata.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbxface.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/workqueue.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/nsaccess.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_lib/string.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_arch/x86/kernel/alternative.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_arch/x86/realmode/init.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_crypto/algapi.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/events/core.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/exit.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/sched/cputime.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/locking/lockdep.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/tbxfload.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/dsmthdat.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/psargs.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/dsinit.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/evgpeinit.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/evregion.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/nsinit.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/utpredef.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/sysfs.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsutils.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsaddr.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rscalc.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsmisc.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/acpi/acpica/rsxface.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_lib/zlib_inflate/inffast.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_crypto/api.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_drivers/base/devres.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/linux/compiler.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/uapi/linux/swab.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/linux/unaligned/access_ok.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/af_inet.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/net/ip.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/ipconfig.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/ip_input.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/udp.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/signal.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/linux/skbuff.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/cgroup/cgroup.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/ip6_offload.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/ip6_input.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/net/ipv6.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/route.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/icmp.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/net/ip6_checksum.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/ndisc.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/addrconf_core.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/addrconf.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv6/ip6_fib.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_fs/readdir.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/unix/garbage.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_include/linux/etherdevice.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/tcp_offload.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/tcp_ipv4.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/linux/tcp.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_include/net/dsfield.h | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/tcp_input.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/ipv4/tcp_minisocks.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_net/sunrpc/xprtsock.c | 4 | 6 |
| UBSAN:Undefined_behaviour_in_kernel/events/hw_breakpoint.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_kernel/capability.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_kernel/time/time.c | 3 | |
| UBSAN:Undefined_behaviour_in_kernel/sys.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_fs/utimes.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_kernel/futex.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_include/net/ndisc.h | 4 | 4 |
| UBSAN:Undefined_behaviour_in_net/ipv6/udp.c | 4 | 4 |
| UBSAN:Undefined_behaviour_in_include/linux/jhash.h | 4 | 4 |
| UBSAN:Undefined_behaviour_in_include/net/neighbour.h | 4 | 4 |
| EIP:check_poison_obj | 1 | 2 |
| EIP:rcutorture_one_extend | 1 | |
| UBSAN:Undefined_behaviour_in_kernel/sched/core.c | 2 | 4 |
| EIP:arch_local_irq_enable | 1 | |
| EIP:delay_tsc | 2 | 1 |
| EIP:arch_local_irq_restore | 2 | 2 |
| EIP:kmem_cache_alloc | 1 | |
| UBSAN:Undefined_behaviour_in_fs/proc/array.c | 1 | |
| EIP:__copy_user_ll | 1 | |
| EIP:update_stack_state | 0 | 1 |
| UBSAN:Undefined_behaviour_in_net/core/dev.c | 0 | 6 |
| UBSAN:Undefined_behaviour_in_fs/ioctl.c | 0 | 2 |
| EIP:copy_user_highpage | 0 | 1 |
| EIP:do_fault | 0 | 1 |
| EIP:native_safe_halt | 0 | 2 |
+------------------------------------------------------------------+------------+------------+
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
[ 439.724028] ================================================================================
[ 439.724993] UBSAN: Undefined behaviour in net/core/dev.c:5144:2
[ 439.725784] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.726595] which requires 8 byte alignment
[ 439.727042] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.727807] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.728754] Call Trace:
[ 439.729035] <SOFTIRQ>
[ 439.729305] dump_stack+0x16/0x18
[ 439.729673] ubsan_epilogue+0x8/0x20
[ 439.730060] ubsan_type_mismatch_common+0x9f/0xed
[ 439.730564] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.731118] netif_receive_skb_list_internal+0xe9/0x3fe
[ 439.731677] ? e1000_clean_rx_irq+0x758/0x94d
[ 439.732145] gro_normal_list+0x43/0xbf
[ 439.732597] napi_complete_done+0xbe/0x1c0
[ 439.733031] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.733563] e1000_clean+0xf32/0x126e
[ 439.733952] ? __lock_is_held+0x57/0x9e
[ 439.734361] net_rx_action+0x15e/0x448
[ 439.734773] __do_softirq+0x277/0x573
[ 439.735164] ? __irqentry_text_end+0x7/0x7
[ 439.735599] do_softirq_own_stack+0x33/0x3a
[ 439.736043] </SOFTIRQ>
[ 439.736314] irq_exit+0x68/0xf4
[ 439.736683] do_IRQ+0xda/0xed
[ 439.737009] common_interrupt+0x11b/0x120
[ 439.737435] EIP: native_safe_halt+0x12/0x13
[ 439.737932] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.739875] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.740568] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.741226] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.741939] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.742381] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.742817] ? native_safe_halt+0x12/0x13
[ 439.743246] ? default_idle+0x2c/0x37
[ 439.743642] arch_cpu_idle+0x9/0xb
[ 439.744007] default_idle_call+0x1d/0x1f
[ 439.744430] do_idle+0x10d/0x296
[ 439.744807] ? schedule+0x14e/0x199
[ 439.745204] cpu_startup_entry+0xd/0xf
[ 439.745631] rest_init+0xd2/0xd5
[ 439.745995] arch_call_rest_init+0x8/0xb
[ 439.746431] start_kernel+0x3f4/0x3fc
[ 439.746859] i386_start_kernel+0x42/0x44
[ 439.747277] startup_32_smp+0x164/0x168
[ 439.747689] ================================================================================
[ 439.750463] ================================================================================
[ 439.751352] UBSAN: Undefined behaviour in net/core/dev.c:5144:2
[ 439.752186] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.753066] which requires 8 byte alignment
[ 439.753541] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.754353] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.755198] Call Trace:
[ 439.755468] <SOFTIRQ>
[ 439.755736] dump_stack+0x16/0x18
[ 439.756110] ubsan_epilogue+0x8/0x20
[ 439.756537] ubsan_type_mismatch_common+0x9f/0xed
[ 439.757050] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.757603] netif_receive_skb_list_internal+0x51/0x3fe
[ 439.758227] ? e1000_clean_rx_irq+0x758/0x94d
[ 439.758691] gro_normal_list+0x43/0xbf
[ 439.759097] napi_complete_done+0xbe/0x1c0
[ 439.759530] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.760045] e1000_clean+0xf32/0x126e
[ 439.760444] ? __lock_is_held+0x57/0x9e
[ 439.760871] net_rx_action+0x15e/0x448
[ 439.761259] __do_softirq+0x277/0x573
[ 439.761648] ? __irqentry_text_end+0x7/0x7
[ 439.762077] do_softirq_own_stack+0x33/0x3a
[ 439.762516] </SOFTIRQ>
[ 439.762787] irq_exit+0x68/0xf4
[ 439.763124] do_IRQ+0xda/0xed
[ 439.763441] common_interrupt+0x11b/0x120
[ 439.763881] EIP: native_safe_halt+0x12/0x13
[ 439.764301] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.766225] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.766879] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.767524] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.768297] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.768771] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.769212] ? native_safe_halt+0x12/0x13
[ 439.769649] ? default_idle+0x2c/0x37
[ 439.770043] arch_cpu_idle+0x9/0xb
[ 439.770422] default_idle_call+0x1d/0x1f
[ 439.770850] do_idle+0x10d/0x296
[ 439.771200] ? schedule+0x14e/0x199
[ 439.771571] cpu_startup_entry+0xd/0xf
[ 439.771970] rest_init+0xd2/0xd5
[ 439.772321] arch_call_rest_init+0x8/0xb
[ 439.772763] start_kernel+0x3f4/0x3fc
[ 439.773148] i386_start_kernel+0x42/0x44
[ 439.773568] startup_32_smp+0x164/0x168
[ 439.773984] ================================================================================
[ 439.774851] ================================================================================
[ 439.775731] UBSAN: Undefined behaviour in net/core/dev.c:5054:2
[ 439.776563] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.777424] which requires 8 byte alignment
[ 439.777924] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.778704] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.779548] Call Trace:
[ 439.779833] <SOFTIRQ>
[ 439.780098] dump_stack+0x16/0x18
[ 439.780492] ubsan_epilogue+0x8/0x20
[ 439.780897] ubsan_type_mismatch_common+0x9f/0xed
[ 439.781389] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.781932] netif_receive_skb_list_internal+0x39d/0x3fe
[ 439.782487] gro_normal_list+0x43/0xbf
[ 439.782897] napi_complete_done+0xbe/0x1c0
[ 439.783336] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.783853] e1000_clean+0xf32/0x126e
[ 439.784249] ? __lock_is_held+0x57/0x9e
[ 439.784705] net_rx_action+0x15e/0x448
[ 439.785089] __do_softirq+0x277/0x573
[ 439.785468] ? __irqentry_text_end+0x7/0x7
[ 439.785886] do_softirq_own_stack+0x33/0x3a
[ 439.786330] </SOFTIRQ>
[ 439.786606] irq_exit+0x68/0xf4
[ 439.786948] do_IRQ+0xda/0xed
[ 439.787268] common_interrupt+0x11b/0x120
[ 439.787698] EIP: native_safe_halt+0x12/0x13
[ 439.788194] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.790264] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.790910] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.791554] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.792262] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.792718] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.793159] ? native_safe_halt+0x12/0x13
[ 439.793585] ? default_idle+0x2c/0x37
[ 439.793979] arch_cpu_idle+0x9/0xb
[ 439.794345] default_idle_call+0x1d/0x1f
[ 439.794763] do_idle+0x10d/0x296
[ 439.795112] ? schedule+0x14e/0x199
[ 439.795488] cpu_startup_entry+0xd/0xf
[ 439.795884] rest_init+0xd2/0xd5
[ 439.796236] arch_call_rest_init+0x8/0xb
[ 439.796682] start_kernel+0x3f4/0x3fc
[ 439.797074] i386_start_kernel+0x42/0x44
[ 439.797487] startup_32_smp+0x164/0x168
[ 439.797957] ================================================================================
[ 439.798825] ================================================================================
[ 439.799688] UBSAN: Undefined behaviour in net/core/dev.c:5054:2
[ 439.800526] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.801371] which requires 8 byte alignment
[ 439.801839] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.802643] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.803535] Call Trace:
[ 439.803802] <SOFTIRQ>
[ 439.804066] dump_stack+0x16/0x18
[ 439.804422] ubsan_epilogue+0x8/0x20
[ 439.804829] ubsan_type_mismatch_common+0x9f/0xed
[ 439.805317] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.805851] netif_receive_skb_list_internal+0x208/0x3fe
[ 439.806405] gro_normal_list+0x43/0xbf
[ 439.806804] napi_complete_done+0xbe/0x1c0
[ 439.807253] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.807782] e1000_clean+0xf32/0x126e
[ 439.808244] ? __lock_is_held+0x57/0x9e
[ 439.808702] net_rx_action+0x15e/0x448
[ 439.809125] __do_softirq+0x277/0x573
[ 439.809513] ? __irqentry_text_end+0x7/0x7
[ 439.809944] do_softirq_own_stack+0x33/0x3a
[ 439.810387] </SOFTIRQ>
[ 439.810656] irq_exit+0x68/0xf4
[ 439.810998] do_IRQ+0xda/0xed
[ 439.811322] common_interrupt+0x11b/0x120
[ 439.811747] EIP: native_safe_halt+0x12/0x13
[ 439.812189] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.814134] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.814799] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.815463] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.816222] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.816679] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.817226] ? native_safe_halt+0x12/0x13
[ 439.817677] ? default_idle+0x2c/0x37
[ 439.818149] arch_cpu_idle+0x9/0xb
[ 439.818521] default_idle_call+0x1d/0x1f
[ 439.818938] do_idle+0x10d/0x296
[ 439.819282] ? schedule+0x14e/0x199
[ 439.819648] cpu_startup_entry+0xd/0xf
[ 439.820050] rest_init+0xd2/0xd5
[ 439.820409] arch_call_rest_init+0x8/0xb
[ 439.820853] start_kernel+0x3f4/0x3fc
[ 439.821250] i386_start_kernel+0x42/0x44
[ 439.821663] startup_32_smp+0x164/0x168
[ 439.822067] ================================================================================
[ 439.822933] ================================================================================
[ 439.823800] UBSAN: Undefined behaviour in net/core/dev.c:5000:2
[ 439.824624] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.825443] which requires 8 byte alignment
[ 439.825891] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.826650] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.827527] Call Trace:
[ 439.827800] <SOFTIRQ>
[ 439.828116] dump_stack+0x16/0x18
[ 439.828520] ubsan_epilogue+0x8/0x20
[ 439.828904] ubsan_type_mismatch_common+0x9f/0xed
[ 439.829397] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.829943] __netif_receive_skb_list_core+0x135/0x154
[ 439.830492] netif_receive_skb_list_internal+0x3e3/0x3fe
[ 439.831044] gro_normal_list+0x43/0xbf
[ 439.831440] napi_complete_done+0xbe/0x1c0
[ 439.831901] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.832427] e1000_clean+0xf32/0x126e
[ 439.832892] ? __lock_is_held+0x57/0x9e
[ 439.833324] net_rx_action+0x15e/0x448
[ 439.833726] __do_softirq+0x277/0x573
[ 439.834118] ? __irqentry_text_end+0x7/0x7
[ 439.834553] do_softirq_own_stack+0x33/0x3a
[ 439.834996] </SOFTIRQ>
[ 439.835282] irq_exit+0x68/0xf4
[ 439.835622] do_IRQ+0xda/0xed
[ 439.835964] common_interrupt+0x11b/0x120
[ 439.836395] EIP: native_safe_halt+0x12/0x13
[ 439.837047] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.839039] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.839708] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.840369] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.841078] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.841489] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.841894] ? native_safe_halt+0x12/0x13
[ 439.842306] ? default_idle+0x2c/0x37
[ 439.842678] arch_cpu_idle+0x9/0xb
[ 439.843028] default_idle_call+0x1d/0x1f
[ 439.843458] do_idle+0x10d/0x296
[ 439.843812] ? schedule+0x14e/0x199
[ 439.844176] cpu_startup_entry+0xd/0xf
[ 439.844605] rest_init+0xd2/0xd5
[ 439.844965] arch_call_rest_init+0x8/0xb
[ 439.845387] start_kernel+0x3f4/0x3fc
[ 439.845789] i386_start_kernel+0x42/0x44
[ 439.846203] startup_32_smp+0x164/0x168
[ 439.846610] ================================================================================
[ 439.847484] ================================================================================
[ 439.848401] UBSAN: Undefined behaviour in net/core/dev.c:5000:2
[ 439.849226] member access within misaligned address ba36ed7e for type 'struct sk_buff'
[ 439.850048] which requires 8 byte alignment
[ 439.850488] CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.15-00023-gd18d22ce8f628 #1
[ 439.851283] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.12.0-1 04/01/2014
[ 439.852106] Call Trace:
[ 439.852364] <SOFTIRQ>
[ 439.852688] dump_stack+0x16/0x18
[ 439.853045] ubsan_epilogue+0x8/0x20
[ 439.853427] ubsan_type_mismatch_common+0x9f/0xed
[ 439.853937] __ubsan_handle_type_mismatch_v1+0x2c/0x32
[ 439.854479] __netif_receive_skb_list_core+0x69/0x154
[ 439.855021] netif_receive_skb_list_internal+0x3e3/0x3fe
[ 439.855574] gro_normal_list+0x43/0xbf
[ 439.855976] napi_complete_done+0xbe/0x1c0
[ 439.856413] ? e1000_clean_jumbo_rx_irq+0xd72/0xd72
[ 439.856956] e1000_clean+0xf32/0x126e
[ 439.857345] ? __lock_is_held+0x57/0x9e
[ 439.857752] net_rx_action+0x15e/0x448
[ 439.858215] __do_softirq+0x277/0x573
[ 439.858615] ? __irqentry_text_end+0x7/0x7
[ 439.859046] do_softirq_own_stack+0x33/0x3a
[ 439.859485] </SOFTIRQ>
[ 439.859755] irq_exit+0x68/0xf4
[ 439.860095] do_IRQ+0xda/0xed
[ 439.860415] common_interrupt+0x11b/0x120
[ 439.860882] EIP: native_safe_halt+0x12/0x13
[ 439.861340] Code: 10 ff 8b 03 a8 08 74 0a 81 25 40 1b df c2 ff ff ff 7f 58 5b 5e 5f 5d c3 a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 fb f4 <c3> a1 58 44 d6 c3 85 c0 7e 07 0f 00 2d 5c f9 5f c2 f4 c3 55 31 c0
[ 439.863238] EAX: 00000000 EBX: 00000000 ECX: 00989680 EDX: 00000006
[ 439.863888] ESI: c2dad280 EDI: c2dad280 EBP: c2d9df38 ESP: c2d9df34
[ 439.864589] DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 EFLAGS: 00200246
[ 439.865335] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.865760] ? rwsem_mark_wake+0x2a9/0x4c2
[ 439.866134] ? native_safe_halt+0x12/0x13
[ 439.866540] ? default_idle+0x2c/0x37
[ 439.866910] arch_cpu_idle+0x9/0xb
[ 439.867259] default_idle_call+0x1d/0x1f
[ 439.867660] do_idle+0x10d/0x296
[ 439.868064] ? schedule+0x14e/0x199
[ 439.868447] cpu_startup_entry+0xd/0xf
[ 439.868878] rest_init+0xd2/0xd5
[ 439.869230] arch_call_rest_init+0x8/0xb
[ 439.869645] start_kernel+0x3f4/0x3fc
[ 439.870020] i386_start_kernel+0x42/0x44
[ 439.870409] startup_32_smp+0x164/0x168
[ 439.870802] ================================================================================
[ 442.758767] sysrq: Emergency Sync
[ 442.759580] sysrq: Resetting
Kboot worker: lkp-worker48
Elapsed time: 480
kvm=(
qemu-system-i386
-enable-kvm
-cpu SandyBridge
-kernel $kernel
-initrd initrd-vm-snb-i386-8.cgz
-m 8192
-smp 2
-device e1000,netdev=net0
-netdev user,id=net0,hostfwd=tcp::32032-:22
-boot order=nc
-no-reboot
-watchdog i6300esb
-watchdog-action debug
-rtc base=localtime
-serial stdio
-display none
-monitor null
)
append=(
ip=::::vm-snb-i386-8::dhcp
root=/dev/ram0
user=lkp
job=/job-script
ARCH=i386
kconfig=i386-randconfig-a015-20200923
branch=linux-stable-rc/linux-5.4.y
commit=d18d22ce8f62839365c984b1df474d3975ed4eb2
BOOT_IMAGE=/pkg/linux/i386-randconfig-a015-20200923/gcc-9/d18d22ce8f62839365c984b1df474d3975ed4eb2/vmlinuz-5.4.15-00023-gd18d22ce8f628
max_uptime=3600
RESULT_ROOT=/result/trinity/300s/vm-snb-i386/debian-i386-20191205.cgz/i386-randconfig-a015-20200923/gcc-9/d18d22ce8f62839365c984b1df474d3975ed4eb2/3
result_service=tmpfs
selinux=0
debug
apic=debug
sysrq_always_enabled
rcupdate.rcu_cpu_stall_timeout=100
net.ifnames=0
printk.devkmsg=on
panic=-1
softlockup_panic=1
nmi_watchdog=panic
oops=panic
load_ramdisk=2
prompt_ramdisk=0
drbd.minor_count=8
systemd.log_level=err
ignore_loglevel
To reproduce:
# build kernel
cd linux
cp config-5.4.15-00023-gd18d22ce8f628 .config
make HOSTCC=gcc-9 CC=gcc-9 ARCH=i386 olddefconfig prepare modules_prepare bzImage
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp qemu -k <bzImage> job-script # job-script is attached in this email
Thanks,
Rong Chen
4 months, 3 weeks
- 3
- 2
[perf bench] 3a25d42fc8: make perf failed
by kernel test robot
Greeting,
FYI, we noticed the following commit (built with gcc-9):
commit: 3a25d42fc87490b61a4c837f6864942e6d479b53 ("[PATCH] perf bench: Use condition variables in numa.")
url: https://github.com/0day-ci/linux/commits/Ian-Rogers/perf-bench-Use-condit...
base: https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git 3dbde69575637658d2094ee4416c21bc22eb89fe
in testcase: perf-sanity-tests
version: perf-x86_64-c85fb28b6f99-1_20201008
with following parameters:
perf_compiler: gcc
ucode: 0xdc
on test machine: 4 threads Intel(R) Core(TM) i5-6500 CPU @ 3.20GHz with 32G memory
caused below changes (please refer to attached dmesg/kmsg for entire log/backtrace):
If you fix the issue, kindly add following tag
Reported-by: kernel test robot <rong.a.chen(a)intel.com>
user :notice: [ 43.399624] 2020-10-12 15:16:24 make ARCH= -C /usr/src/perf_selftests-x86_64-rhel-8.3-3a25d42fc87490b61a4c837f6864942e6d479b53/tools/perf
user :notice: [ 43.416155] make: Entering directory '/usr/src/perf_selftests-x86_64-rhel-8.3-3a25d42fc87490b61a4c837f6864942e6d479b53/tools/perf'
user :notice: [ 43.430643] BUILD: Doing 'make [33m-j4[m' parallel build
user :notice: [ 43.438568] HOSTCC fixdep.o
user :notice: [ 43.443837] HOSTLD fixdep-in.o
user :notice: [ 43.449122] LINK fixdep
user :notice: [ 43.455567] diff -u tools/arch/x86/include/asm/msr-index.h arch/x86/include/asm/msr-index.h
user :notice: [ 43.467802] Auto-detecting system features:
user :notice: [ 43.474731] ... dwarf: [ [32mon[m ]
user :notice: [ 43.483286] ... dwarf_getlocations: [ [32mon[m ]
user :notice: [ 43.491855] ... glibc: [ [32mon[m ]
user :notice: [ 43.500425] ... gtk2: [ [32mon[m ]
user :notice: [ 43.508997] ... libbfd: [ [32mon[m ]
user :notice: [ 43.517568] ... libcap: [ [32mon[m ]
user :notice: [ 43.526145] ... libelf: [ [32mon[m ]
user :notice: [ 43.534720] ... libnuma: [ [32mon[m ]
user :notice: [ 43.543296] ... numa_num_possible_cpus: [ [32mon[m ]
user :notice: [ 43.551882] ... libperl: [ [32mon[m ]
user :notice: [ 43.560458] ... libpython: [ [32mon[m ]
user :notice: [ 43.569039] ... libcrypto: [ [32mon[m ]
user :notice: [ 43.577612] ... libunwind: [ [32mon[m ]
user :notice: [ 43.586170] ... libdw-dwarf-unwind: [ [32mon[m ]
user :notice: [ 43.594742] ... zlib: [ [32mon[m ]
user :notice: [ 43.603329] ... lzma: [ [32mon[m ]
user :notice: [ 43.611918] ... get_cpuid: [ [32mon[m ]
user :notice: [ 43.620487] ... bpf: [ [32mon[m ]
user :notice: [ 43.629090] ... libaio: [ [32mon[m ]
user :notice: [ 43.637643] ... libzstd: [ [32mon[m ]
user :notice: [ 43.646241] ... disassembler-four-args: [ [32mon[m ]
user :notice: [ 43.655766] CC fd/array.o
user :notice: [ 43.661110] CC event-parse.o
user :notice: [ 43.666716] CC exec-cmd.o
user :notice: [ 43.671903] CC core.o
user :notice: [ 43.676942] LD fd/libapi-in.o
user :notice: [ 43.682548] CC fs/fs.o
user :notice: [ 43.687549] CC cpumap.o
user :notice: [ 43.692604] CC help.o
user :notice: [ 43.697707] CC fs/tracing_path.o
user :notice: [ 43.703644] CC fs/cgroup.o
user :notice: [ 43.709079] CC event-plugin.o
user :notice: [ 43.714826] LD fs/libapi-in.o
user :notice: [ 43.720383] CC cpu.o
user :notice: [ 43.725221] CC trace-seq.o
user :notice: [ 43.730559] CC pager.o
user :notice: [ 43.735546] CC debug.o
user :notice: [ 43.740670] CC parse-filter.o
user :notice: [ 43.746332] CC threadmap.o
user :notice: [ 43.751800] CC str_error_r.o
user :notice: [ 43.757385] LD libapi-in.o
user :notice: [ 43.762705] AR libapi.a
user :notice: [ 43.767943] CC parse-options.o
user :notice: [ 43.773625] CC evsel.o
user :notice: [ 43.778826] GEN bpf_helper_defs.h
user :notice: [ 43.784748] MKDIR staticobjs/
user :notice: [ 43.790324] CC staticobjs/libbpf.o
user :notice: [ 43.796482] CC parse-utils.o
user :notice: [ 43.802108] CC kbuffer-parse.o
user :notice: [ 43.807942] CC tep_strerror.o
user :notice: [ 43.813745] CC event-parse-api.o
user :notice: [ 43.819824] LD libtraceevent-in.o
user :notice: [ 43.825881] LINK libtraceevent.a
user :notice: [ 43.831600] CC evlist.o
user :notice: [ 43.836819] CC run-command.o
user :notice: [ 43.842390] CC sigchain.o
user :notice: [ 43.847620] CC mmap.o
user :notice: [ 43.852688] CC subcmd-config.o
user :notice: [ 43.858520] LD libsubcmd-in.o
user :notice: [ 43.864145] AR libsubcmd.a
user :notice: [ 43.869569] MKDIR staticobjs/
user :notice: [ 43.875039] CC staticobjs/bpf.o
user :notice: [ 43.880818] CC zalloc.o
user :notice: [ 43.885939] CC xyarray.o
user :notice: [ 43.891235] GEN common-cmds.h
user :notice: [ 43.896705] CC lib.o
user :notice: [ 43.901763] CC staticobjs/nlattr.o
user :notice: [ 43.907922] CC staticobjs/btf.o
user :notice: [ 43.913797] LD libperf-in.o
user :notice: [ 43.919210] AR libperf.a
user :notice: [ 43.924746] CC staticobjs/libbpf_errno.o
user :notice: [ 43.931595] CC staticobjs/str_error.o
user :notice: [ 43.938104] CC staticobjs/netlink.o
user :notice: [ 43.944645] CC staticobjs/bpf_prog_linfo.o
user :notice: [ 43.951751] CC staticobjs/libbpf_probes.o
user :notice: [ 43.958564] CC staticobjs/xsk.o
user :notice: [ 43.964588] CC staticobjs/hashmap.o
user :notice: [ 43.970946] CC staticobjs/btf_dump.o
user :notice: [ 43.977428] CC staticobjs/ringbuf.o
user :notice: [ 43.983864] HOSTCC pmu-events/json.o
user :notice: [ 43.989998] LD staticobjs/libbpf-in.o
user :notice: [ 43.996594] CC plugin_jbd2.o
user :notice: [ 44.002082] LINK libbpf.a
user :notice: [ 44.007527] CC jvmti/libjvmti.o
user :notice: [ 44.013630] HOSTCC pmu-events/jsmn.o
user :notice: [ 44.019794] LD plugin_jbd2-in.o
user :notice: [ 44.025944] HOSTCC pmu-events/jevents.o
user :notice: [ 44.032199] CC plugin_hrtimer.o
user :notice: [ 44.038406] LD plugin_hrtimer-in.o
user :notice: [ 44.044693] CC plugin_kmem.o
user :notice: [ 44.050661] HOSTLD pmu-events/jevents-in.o
user :notice: [ 44.057246] CC jvmti/jvmti_agent.o
user :notice: [ 44.063645] LD plugin_kmem-in.o
user :notice: [ 44.069629] CC plugin_kvm.o
user :notice: [ 44.075192] LD plugin_kvm-in.o
user :notice: [ 44.081243] CC jvmti/libstring.o
user :notice: [ 44.087452] CC plugin_mac80211.o
user :notice: [ 44.093718] LD plugin_mac80211-in.o
user :notice: [ 44.100100] CC plugin_sched_switch.o
user :notice: [ 44.106687] CC plugin_function.o
user :notice: [ 44.113027] LD plugin_sched_switch-in.o
user :notice: [ 44.119760] CC plugin_futex.o
user :notice: [ 44.125730] LD plugin_function-in.o
user :notice: [ 44.131916] CC plugin_xen.o
user :notice: [ 44.137705] CC jvmti/libctype.o
user :notice: [ 44.143797] LD plugin_futex-in.o
user :notice: [ 44.149914] CC plugin_scsi.o
user :notice: [ 44.155725] LD jvmti/jvmti-in.o
user :notice: [ 44.161772] LD plugin_xen-in.o
user :notice: [ 44.167672] GEN perf-archive
user :notice: [ 44.173231] GEN perf-with-kcore
user :notice: [ 44.179308] CC plugin_cfg80211.o
user :notice: [ 44.185479] CC ui/gtk/browser.o
user :notice: [ 44.191663] LD plugin_cfg80211-in.o
user :notice: [ 44.197922] LD plugin_scsi-in.o
user :notice: [ 44.203991] LINK plugin_jbd2.so
user :notice: [ 44.209807] CC plugin_tlb.o
user :notice: [ 44.215598] LINK plugin_hrtimer.so
user :notice: [ 44.221711] LINK plugin_kmem.so
user :notice: [ 44.227579] LD plugin_tlb-in.o
user :notice: [ 44.233516] LINK plugin_kvm.so
user :notice: [ 44.239213] LINK plugin_mac80211.so
user :notice: [ 44.245483] CC ui/gtk/hists.o
user :notice: [ 44.251172] CC ui/gtk/setup.o
user :notice: [ 44.257230] LINK plugin_sched_switch.so
user :notice: [ 44.263883] LINK plugin_function.so
user :notice: [ 44.269949] LINK plugin_futex.so
user :notice: [ 44.275929] LINK plugin_xen.so
user :notice: [ 44.281705] LINK plugin_scsi.so
user :notice: [ 44.287635] LINK plugin_cfg80211.so
user :notice: [ 44.293821] LINK plugin_tlb.so
user :notice: [ 44.299844] GEN libtraceevent-dynamic-list
user :notice: [ 44.306752] CC builtin-bench.o
user :notice: [ 44.312676] CC ui/gtk/util.o
user :notice: [ 44.318530] CC builtin-annotate.o
user :notice: [ 44.324801] CC ui/gtk/helpline.o
user :notice: [ 44.330974] CC ui/gtk/progress.o
user :notice: [ 44.336981] CC builtin-config.o
user :notice: [ 44.343046] CC builtin-diff.o
user :notice: [ 44.348977] LINK pmu-events/jevents
user :notice: [ 44.355082] CC ui/gtk/annotate.o
user :notice: [ 44.361251] CC ui/gtk/zalloc.o
user :notice: [ 44.367232] CC builtin-evlist.o
user :notice: [ 44.373330] LINK libperf-jvmti.so
user :notice: [ 44.379236] CC builtin-ftrace.o
user :notice: [ 44.385317] CC builtin-help.o
user :notice: [ 44.391032] CC builtin-sched.o
user :notice: [ 44.397025] LD ui/gtk/gtk-in.o
user :notice: [ 44.402840] LD gtk-in.o
user :notice: [ 44.408376] GEN pmu-events/pmu-events.c
user :notice: [ 44.415031] CC builtin-buildid-list.o
user :notice: [ 44.421818] CC builtin-buildid-cache.o
user :notice: [ 44.428624] CC pmu-events/pmu-events.o
user :notice: [ 44.435121] LINK libperf-gtk.so
user :notice: [ 44.441099] CC builtin-kallsyms.o
user :notice: [ 44.447323] GEN python/perf.so
user :notice: [ 44.453022] CC builtin-list.o
user :notice: [ 44.458976] CC builtin-record.o
user :notice: [ 44.465108] LD pmu-events/pmu-events-in.o
user :notice: [ 44.472092] CC builtin-report.o
user :notice: [ 44.477988] CC builtin-stat.o
user :notice: [ 44.483940] CC builtin-timechart.o
user :notice: [ 44.490044] CC builtin-top.o
user :notice: [ 44.495900] CC builtin-script.o
user :notice: [ 44.501826] CC builtin-kmem.o
user :notice: [ 44.508408] CC builtin-lock.o
user :notice: [ 44.514097] CC builtin-kvm.o
user :notice: [ 44.520554] CC builtin-inject.o
user :notice: [ 44.526466] CC builtin-mem.o
user :notice: [ 44.532098] CC builtin-data.o
user :notice: [ 44.537918] CC builtin-version.o
user :notice: [ 44.543867] CC builtin-c2c.o
user :notice: [ 44.549554] CC builtin-trace.o
user :notice: [ 44.555409] CC builtin-probe.o
user :notice: [ 44.561429] CC bench/sched-messaging.o
user :notice: [ 44.567991] CC bench/sched-pipe.o
user :notice: [ 44.574220] CC tests/builtin-test.o
user :notice: [ 44.580528] CC bench/syscall.o
user :notice: [ 44.586514] CC bench/mem-functions.o
user :notice: [ 44.610513] CC bench/futex-hash.o
user :notice: [ 44.621359] CC tests/parse-events.o
user :notice: [ 44.718758] CC bench/futex-wake.o
user :notice: [ 44.832301] CC bench/futex-wake-parallel.o
user :notice: [ 44.954129] CC bench/futex-requeue.o
user :notice: [ 45.060866] CC bench/futex-lock-pi.o
user :notice: [ 45.171348] CC bench/epoll-wait.o
user :notice: [ 45.331626] CC tests/dso-data.o
user :notice: [ 45.350733] CC bench/epoll-ctl.o
user :notice: [ 45.516555] CC tests/attr.o
user :notice: [ 45.530335] CC bench/synthesize.o
user :notice: [ 45.599988] CC bench/kallsyms-parse.o
user :notice: [ 45.670039] CC tests/vmlinux-kallsyms.o
user :notice: [ 45.676941] CC bench/find-bit-bench.o
user :notice: [ 45.688151] CC bench/mem-memcpy-x86-64-lib.o
user :notice: [ 45.719388] CC bench/mem-memcpy-x86-64-asm.o
user :notice: [ 45.733744] CC util/annotate.o
user :notice: [ 45.743182] CC bench/mem-memset-x86-64-asm.o
user :notice: [ 45.763930] CC bench/numa.o
user :notice: [ 45.772258] CC util/block-info.o
user :notice: [ 45.778701] CC tests/openat-syscall.o
user :notice: [ 45.838842] CC tests/openat-syscall-all-cpus.o
user :notice: [ 45.880774] CC tests/openat-syscall-tp-fields.o
user :notice: [ 45.995540] CC tests/mmap-basic.o
user :notice: [ 46.015451] CC tests/perf-record.o
user :err : [ 46.056130] bench/numa.c: In function ‘init’:
user :notice: [ 46.143494] CC util/block-range.o
user :notice: [ 46.149994] CC tests/evsel-roundtrip-name.o
user :notice: [ 46.277777] CC tests/evsel-tp-sched.o
user :notice: [ 46.284176] CC util/build-id.o
user :notice: [ 46.356422] CC tests/fdarray.o
user :notice: [ 46.385339] CC tests/pmu.o
user :notice: [ 46.443435] CC tests/pmu-events.o
user :notice: [ 46.599431] CC util/cacheline.o
user :err : [ 46.653292] bench/numa.c:1461:24: error: ‘struct global_info’ has no member named ‘startup_done_mutex’; did you mean ‘startup_mutex’?
user :err : [ 46.668942] init_global_mutex(&g->startup_done_mutex);
user :err : [ 46.676718] ^~~~~~~~~~~~~~~~~~
user :err : [ 46.684161] startup_mutex
user :notice: [ 46.688754] CC util/config.o
user :notice: [ 46.712970] CC util/copyfile.o
user :notice: [ 46.732870] CC tests/hists_common.o
user :notice: [ 46.814764] CC util/ctype.o
user :notice: [ 46.838251] CC util/db-export.o
user :err : [ 46.992546] bench/numa.c:1462:23: error: ‘struct global_info’ has no member named ‘startup_done_cond’; did you mean ‘startup_cond’?
user :err : [ 47.008124] init_global_cond(&g->startup_done_cond);
user :err : [ 47.015921] ^~~~~~~~~~~~~~~~~
user :notice: [ 47.023389] CC tests/hists_link.o
user :err : [ 47.023811] startup_cond
user :notice: [ 47.087257] CC util/env.o
user :notice: [ 47.121754] CC tests/hists_filter.o
user :notice: [ 47.277321] CC tests/hists_output.o
user :notice: [ 47.284110] CC util/event.o
user :notice: [ 47.319731] CC util/evlist.o
user :notice: [ 47.325558] CC util/sideband_evlist.o
user :notice: [ 47.423521] CC util/evsel.o
user :notice: [ 47.592885] CC tests/hists_cumulate.o
user :err : [ 47.643920] make[4]: *** [/usr/src/perf_selftests-x86_64-rhel-8.3-3a25d42fc87490b61a4c837f6864942e6d479b53/tools/build/Makefile.build:96: bench/numa.o] Error 1
user :notice: [ 47.672415] CC util/evsel_fprintf.o
user :err : [ 47.684400] make[3]: *** [/usr/src/perf_selftests-x86_64-rhel-8.3-3a25d42fc87490b61a4c837f6864942e6d479b53/tools/build/Makefile.build:139: bench] Error 2
user :err : [ 47.711231] make[3]: *** Waiting for unfinished jobs....
user :notice: [ 47.829659] CC util/perf_event_attr_fprintf.o
...
user :err : [ 58.971914] make[2]: *** [Makefile.perf:634: perf-in.o] Error 2
user :notice: [ 58.973854] make: Leaving directory '/usr/src/perf_selftests-x86_64-rhel-8.3-3a25d42fc87490b61a4c837f6864942e6d479b53/tools/perf'
user :err : [ 58.979195] make[1]: *** [Makefile.perf:233: sub-make] Error 2
user :err : [ 59.000676] make: *** [Makefile:70: all] Error 2
user :err : [ 59.007163] make perf failed
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
bin/lkp install job.yaml # job file is attached in this email
bin/lkp run job.yaml
Thanks,
Rong Chen
4 months, 3 weeks
- 1
- 0