Warning on Tempesta restart #2556
Description
A baremetal Tempesta FW setup with plenty of
[tempesta fw] Warning: client 1.2.3.4 requested missing server name, reject connection
messages in dmesg. On Tempesta FW restart I got
[624926.598351] [tempesta fw] Warning: pending active connections for 5s (connections count 0xd7, queues count 0)
[624926.598779] [tempesta fw] Warning: cpu 0(3), backlog size 0, active connections mask 0x56700000000, cntwork queue size 0, close backlog is empty
[624926.599660] [tempesta fw] Warning: cpu 1(3), backlog size 0, active connections mask 0x85300000000, cntwork queue size 0, close backlog is empty
[624926.600650] [tempesta fw] Warning: cpu 2(3), backlog size 0, active connections mask 0x1f000000000, cntwork queue size 0, close backlog is empty
[624926.601751] [tempesta fw] Warning: cpu 3(3), backlog size 0, active connections mask 0xfffffb6c00000000, cntwork queue size 0, close backlog is empty
[624926.602957] [tempesta fw] Warning: cpu 4(3), backlog size 0, active connections mask 0xfffffbdf00000000, cntwork queue size 0, close backlog is empty
[624926.604358] [tempesta fw] Warning: cpu 5(3), backlog size 0, active connections mask 0xfffffe4f00000000, cntwork queue size 0, close backlog is empty
[624926.605690] [tempesta fw] Warning: cpu 6(3), backlog size 0, active connections mask 0xffffff5a00000000, cntwork queue size 0, close backlog is empty
[624926.607141] [tempesta fw] Warning: cpu 7(3), backlog size 0, active connections mask 0xfffffc3900000000, cntwork queue size 0, close backlog is empty
[624931.738348] [tempesta fw] Warning: pending active connections for 5s (connections count 0x69, queues count 0)
[624931.739183] [tempesta fw] Warning: cpu 0(3), backlog size 0, active connections mask 0x55500000000, cntwork queue size 0, close backlog is empty
[624931.740860] [tempesta fw] Warning: cpu 1(3), backlog size 0, active connections mask 0x84400000000, cntwork queue size 0, close backlog is empty
[624931.742590] [tempesta fw] Warning: cpu 2(3), backlog size 0, active connections mask 0x1e100000000, cntwork queue size 0, close backlog is empty
[624931.744534] [tempesta fw] Warning: cpu 3(3), backlog size 0, active connections mask 0xfffffb5e00000000, cntwork queue size 0, close backlog is empty
[624931.746466] [tempesta fw] Warning: cpu 4(3), backlog size 0, active connections mask 0xfffffbd400000000, cntwork queue size 0, close backlog is empty
[624931.748586] [tempesta fw] Warning: cpu 5(3), backlog size 0, active connections mask 0xfffffe4300000000, cntwork queue size 0, close backlog is empty
[624931.750773] [tempesta fw] Warning: cpu 6(3), backlog size 0, active connections mask 0xffffff4d00000000, cntwork queue size 0, close backlog is empty
[624931.753101] [tempesta fw] Warning: cpu 7(3), backlog size 0, active connections mask 0xfffffc2d00000000, cntwork queue size 0, close backlog is empty
[624936.758345] [tempesta fw] Warning: pending active connections for 5s (connections count 0x66, queues count 0)
[624936.759570] [tempesta fw] Warning: cpu 0(6), backlog size 0, active connections mask 0x55500000000, cntwork queue size 0, close backlog is empty
[624936.762001] [tempesta fw] Warning: cpu 1(6), backlog size 0, active connections mask 0x84400000000, cntwork queue size 0, close backlog is empty
[624936.764527] [tempesta fw] Warning: cpu 2(6), backlog size 0, active connections mask 0x1e000000000, cntwork queue size 0, close backlog is empty
[624936.767279] [tempesta fw] Warning: cpu 3(6), backlog size 0, active connections mask 0xfffffb5e00000000, cntwork queue size 0, close backlog is empty
[624936.770045] [tempesta fw] Warning: cpu 4(6), backlog size 0, active connections mask 0xfffffbd400000000, cntwork queue size 0, close backlog is empty
[624936.772778] [tempesta fw] Warning: cpu 5(6), backlog size 0, active connections mask 0xfffffe4300000000, cntwork queue size 0, close backlog is empty
[624936.775598] [tempesta fw] Warning: cpu 6(6), backlog size 0, active connections mask 0xffffff4c00000000, cntwork queue size 0, close backlog is empty
[624936.778452] [tempesta fw] Warning: cpu 7(6), backlog size 0, active connections mask 0xfffffc2c00000000, cntwork queue size 0, close backlog is empty
[624936.781323] ------------[ cut here ]------------
[624936.782732] WARNING: CPU: 6 PID: 812478 at /root/tempesta/fw/main.c:182 tfw_mods_stop+0xa4/0xb0 [tempesta_fw]
[624936.784268] Modules linked in: tcp_diag inet_diag tls tempesta_fw(OE) tempesta_db(OE) sha256_ssse3 sha512_ssse3 tempesta_tls(OE) tempesta_lib(OE) bonding cfg80211 nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 nft_counter xt_mark xt_tcpudp nft_compat nf_tables binfmt_misc nls_iso8859_1 intel_rapl_msr intel_rapl_common x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul ghash_clmulni_intel rapl intel_cstate mgag200 drm_kms_helper ipmi_ssif cec fb_sys_fops syscopyarea sysfillrect sysimgblt dcdbas ipmi_si ie31200_edac ipmi_devintf ee1004 mei_me intel_wmi_thunderbolt acpi_power_meter mei ipmi_msghandler intel_pch_thermal i2c_i801 mac_hid i2c_smbus sch_fq_codel dm_multipath netconsole msr efi_pstore drm nfnetlink dmi_sysfs ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c multipath linear raid0 raid1 crc32_pclmul aesni_intel glue_helper
[624936.784302] crypto_simd cryptd igb tg3 ahci i2c_algo_bit libahci dca wmi video
[624936.797056] CPU: 6 PID: 812478 Comm: sysctl Kdump: loaded Tainted: G W OE 5.10.35.tfw-562f4af #1
[624936.798456] Hardware name: Dell Inc. PowerEdge R230/0MFXTY, BIOS 2.16.0 09/20/2022
[624936.799932] RIP: 0010:tfw_mods_stop+0xa4/0xb0 [tempesta_fw]
[624936.801302] Code: 8b 05 60 3e 05 00 85 c0 75 24 e8 07 c0 b2 eb 85 c0 0f 85 98 d8 01 00 5b 41 5c 5d c3 e8 65 78 00 00 e8 f0 5f 00 00 84 c0 75 c3 <0f> 0b eb bf 0f 0b 90 0f 1f 44 00 00 0f 1f 44 00 00 55 48 c7 c7 e0
[624936.804072] RSP: 0018:ffffb7d4c109fc98 EFLAGS: 00010246
[624936.805511] RAX: 0000000000000000 RBX: ffffffffc0ac8b20 RCX: 0000000000000008
[624936.806866] RDX: 0000000000000008 RSI: 0000000000000000 RDI: ffffffffad54f6a0
[624936.808281] RBP: ffffb7d4c109fca8 R08: 0000000000000000 R09: 0000000000000008
[624936.809596] R10: ffffb7d4c109fa10 R11: ffffffffad3371d0 R12: 0000000000000000
[624936.810892] R13: ffffb7d4c109fdc8 R14: ffffb7d4c109fcf8 R15: ffffb7d4c109fe68
[624936.812247] FS: 00007ff65db8e740(0000) GS:ffff998f4fd80000(0000) knlGS:0000000000000000
[624936.813513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[624936.814764] CR2: 00007ff554ed96f4 CR3: 0000000214fc2006 CR4: 00000000003706e0
[624936.816078] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[624936.817296] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[624936.818491] Call Trace:
[624936.819748] tfw_ctlfn_state_io+0x1be/0x460 [tempesta_fw]
[624936.820914] ? tfw_cleanup+0x30/0x30 [tempesta_fw]
[624936.822053] ? __cgroup_bpf_run_filter_sysctl+0x16d/0x2e0
[624936.823214] proc_sys_call_handler+0x13d/0x250
[624936.824407] proc_sys_write+0x13/0x20
[624936.825497] new_sync_write+0x114/0x1a0
[624936.826566] vfs_write+0x1bd/0x250
[624936.827703] ksys_write+0x67/0xe0
[624936.828721] __x64_sys_write+0x19/0x20
[624936.829718] do_syscall_64+0x38/0x90
[624936.830701] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[624936.831741] RIP: 0033:0x7ff65dcad5a4
[624936.832677] Code: c7 00 16 00 00 00 b8 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 f3 0f 1e fa 80 3d a5 ea 0e 00 00 74 13 b8 01 00 00 00 0f 05 <48> 3d 00 f0 ff ff 77 54 c3 0f 1f 00 55 48 89 e5 48 83 ec 20 48 89
[624936.834569] RSP: 002b:00007ffc06dc79b8 EFLAGS: 00000202 ORIG_RAX: 0000000000000001
[624936.835581] RAX: ffffffffffffffda RBX: 0000564b00a436a0 RCX: 00007ff65dcad5a4
[624936.836496] RDX: 0000000000000005 RSI: 0000564b00a41570 RDI: 0000000000000003
[624936.837391] RBP: 00007ffc06dc79f0 R08: 0000000000000000 R09: 0000000000000000
[624936.838265] R10: 0000000000000001 R11: 0000000000000202 R12: 0000564b00a41530
[624936.839123] R13: 0000000000000005 R14: 00007ff65dd93420 R15: 00007ff65dd92ee0
[624936.840052] ---[ end trace ff24bce73da94fe6 ]---
[624936.862200] [tdb] Close table 'client0.tdb'
[624936.869579] [tdb] Close table 'sessions0.tdb'
[624937.400741] [tdb] Warning: Cannot sync mapping ffff998ecbc00000 of size 524288 pages
[624937.401542] [tdb] Close table 'cache0.tdb'
[624937.446444] [tdb] Close table 'filter0.tdb'
[624937.447362] [tempesta fw] modules are stopped
[624937.531685] [tempesta fw] exiting...
[624937.554546] =============================================================================
[624937.555268] BUG tfw_h2_conn_cache (Tainted: G W OE ): Objects remaining in tfw_h2_conn_cache on __kmem_cache_shutdown()
[624937.556670] -----------------------------------------------------------------------------
[624937.558040] Disabling lock debugging due to kernel taint
[624937.558714] INFO: Slab 0x00000000d303be79 objects=11 used=1 fp=0x00000000a0a46d41 flags=0x17ffffc0010200
[624937.559398] CPU: 0 PID: 812503 Comm: rmmod Kdump: loaded Tainted: G B W OE 5.10.35.tfw-562f4af #1
[624937.560088] Hardware name: Dell Inc. PowerEdge R230/0MFXTY, BIOS 2.16.0 09/20/2022
[624937.560778] Call Trace:
[624937.561464] dump_stack+0x70/0x8b
[624937.562146] slab_err+0xb7/0xdc
[624937.562826] ? flush_cpu_slab+0x4b/0x60
[624937.563502] ? slub_cpu_dead+0xb0/0xb0
[624937.564167] __kmem_cache_shutdown.cold+0x3f/0x139
[624937.564827] kmem_cache_destroy+0x59/0x100
[624937.565491] tfw_sock_clnt_exit+0x21/0x40 [tempesta_fw]
[624937.566157] tfw_exit+0x84/0x97 [tempesta_fw]
[624937.566817] __do_sys_delete_module.constprop.0+0x17f/0x2d0
[624937.567474] ? exit_to_user_mode_prepare+0x35/0x150
[624937.568130] __x64_sys_delete_module+0x12/0x20
[624937.568786] do_syscall_64+0x38/0x90
[624937.569434] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[624937.570085] RIP: 0033:0x7f8344688ccb
[624937.570737] Code: 73 01 c3 48 8b 0d 4d 81 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1d 81 0d 00 f7 d8 64 89 01 48
[624937.572127] RSP: 002b:00007ffdf98e0a78 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[624937.572843] RAX: ffffffffffffffda RBX: 0000555e618f7700 RCX: 00007f8344688ccb
[624937.573567] RDX: 0000000000000000 RSI: 0000000000000800 RDI: 0000555e618f7768
[624937.574296] RBP: 00007ffdf98e0aa0 R08: 1999999999999999 R09: 0000000000000000
[624937.575027] R10: 00007f834470ffc0 R11: 0000000000000206 R12: 0000000000000000
[624937.575757] R13: 00007ffdf98e0cf0 R14: 0000555e618f7700 R15: 0000000000000000
[624937.576470] INFO: Object 0x00000000aa9aba0c @offset=22080
[624937.577168] =============================================================================
[624937.577873] BUG tfw_h2_conn_cache (Tainted: G B W OE ): Objects remaining in tfw_h2_conn_cache on __kmem_cache_shutdown()
[624937.579309] -----------------------------------------------------------------------------
[624937.580792] INFO: Slab 0x00000000e1dd0d31 objects=11 used=3 fp=0x000000000e29e9ae flags=0x17ffffc0010200
[624937.581568] CPU: 0 PID: 812503 Comm: rmmod Kdump: loaded Tainted: G B W OE 5.10.35.tfw-562f4af #1
[624937.582351] Hardware name: Dell Inc. PowerEdge R230/0MFXTY, BIOS 2.16.0 09/20/2022
[624937.583221] Call Trace:
[624937.584010] dump_stack+0x70/0x8b
[624937.584791] slab_err+0xb7/0xdc
[624937.585561] ? printk+0x58/0x6f
[624937.586330] __kmem_cache_shutdown.cold+0x3f/0x139
[624937.587106] kmem_cache_destroy+0x59/0x100
[624937.587892] tfw_sock_clnt_exit+0x21/0x40 [tempesta_fw]
[624937.588671] tfw_exit+0x84/0x97 [tempesta_fw]
[624937.589422] __do_sys_delete_module.constprop.0+0x17f/0x2d0
[624937.590159] ? exit_to_user_mode_prepare+0x35/0x150
[624937.590885] __x64_sys_delete_module+0x12/0x20
[624937.591601] do_syscall_64+0x38/0x90
[624937.592312] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[624937.593014] RIP: 0033:0x7f8344688ccb
[624937.593700] Code: 73 01 c3 48 8b 0d 4d 81 0d 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa b8 b0 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 1d 81 0d 00 f7 d8 64 89 01 48
[624937.595111] RSP: 002b:00007ffdf98e0a78 EFLAGS: 00000206 ORIG_RAX: 00000000000000b0
[624937.595831] RAX: ffffffffffffffda RBX: 0000555e618f7700 RCX: 00007f8344688ccb
[624937.596561] RDX: 0000000000000000 RSI: 0000000000000800 RDI: 0000555e618f7768
[624937.597296] RBP: 00007ffdf98e0aa0 R08: 1999999999999999 R09: 0000000000000000
[624937.598037] R10: 00007f834470ffc0 R11: 0000000000000206 R12: 0000000000000000
[624937.598777] R13: 00007ffdf98e0cf0 R14: 0000555e618f7700 R15: 0000000000000000
[624937.599517] INFO: Object 0x00000000d68f7254 @offset=5520
[624937.600256] INFO: Object 0x00000000a3281e37 @offset=11040
[624937.600991] INFO: Object 0x00000000060863d8 @offset=13800
Metadata
Metadata
Assignees
Labels
No labels