概述
2020年3月14日服务器异常重启,查看/var/log/message日志,如下:
Mar 14 15:04:59 localhost kernel: azx_single_send_cmd: 11 callbacks suppressed
Mar 14 15:05:14 localhost kernel: azx_single_send_cmd: 4 callbacks suppressed
Mar 14 15:07:14 localhost kernel: INFO: task jbd2/dm-1-8:1362 blocked for more than 120 seconds.
Mar 14 15:08:15 localhost kernel: Tainted: G OE 4.17.19-debug.x86_64 #1
Mar 14 15:08:32 localhost kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Mar 14 15:08:43 localhost kernel: jbd2/dm-1-8 D 0 1362 2 0x80000000
Mar 14 15:08:43 localhost kernel: Call Trace:
Mar 14 15:08:43 localhost kernel: ? __schedule+0x28a/0x840
Mar 14 15:08:43 localhost kernel: ? __wake_up_common_lock+0x87/0xc0
Mar 14 15:08:43 localhost kernel: schedule+0x32/0x80
Mar 14 15:08:43 localhost kernel: jbd2_journal_commit_transaction+0x275/0x1a00 [jbd2]
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x40/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to_asm+0x34/0x70
Mar 14 15:08:43 localhost kernel: ? __switch_to+0xa8/0x480
Mar 14 15:08:43 localhost kernel: ? remove_wait_queue+0x60/0x60
Mar 14 15:08:43 localhost kernel: ? try_to_del_timer_sync+0x4d/0x80
Mar 14 15:08:43 localhost kernel: kjournald2+0xc1/0x260 [jbd2]
Mar 14 15:08:43 localhost kernel: ? remove_wait_queue+0x60/0x60
Mar 14 15:08:43 localhost kernel: kthread+0xf5/0x130
Mar 14 15:08:43 localhost kernel: ? commit_timeout+0x10/0x10 [jbd2]
Mar 14 15:08:43 localhost kernel: ? kthread_bind+0x10/0x10
Mar 14 15:08:43 localhost kernel: ret_from_fork+0x35/0x40
原因在于,至少情况下,Linux会使用最多40%的可用内存作为文件系统缓存。 将缓存写入磁盘时,有一个至少120秒的超时时间。出现上述问题的原因是IO的处理速度不够快,不能在120秒将缓存中的数据全部写入磁盘。IO系统响应缓慢,导致越来越多的请求堆积,最终系统内存全部被占用,导致系统失去响应。这是Linux延迟写入机制带来的问题,并且在主机内存重叠时,出现该问题的可能性增大。
我们可以从以下思路进行调优:
- 减少脏数据的比例,避免刷写超时
- 初步脏数据在内存中的存放时间,避免积少成多
修改相应参数
临时修改
# sysctl -w vm.dirty_ratio=10
# sysctl -w vm.dirty_background_ratio=5
# sysctl -p
永久修改
#vi /etc/sysctl.conf
写入
vm.dirty_background_ratio = 5
vm.dirty_ratio = 10
修改完重启服务器
最后
以上就是受伤钢笔为你收集整理的linux---"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message的全部内容,希望文章能够帮你解决linux---"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message所遇到的程序开发问题。
如果觉得靠谱客网站的内容还不错,欢迎将靠谱客网站推荐给程序员好友。
发表评论 取消回复