记一次ZZ的服务器维护

之前爆出的Linux内核的Meltdown漏洞想必已经非常有名了吧,于是咱服务器的提供商进行了一波内核升级(具体原理不大清楚)。

本来是非常开心的事情,毕竟安全性增加了嘛,而且咱收到了他们的Ticket,反复阅读并看了他们的恢复指南咱认为已经不会出什么解决不了的问题了(Flag在此时立下)

然后,然后就没有然后了。在5月1号的维护重启后想登录一手博客——GG,数据库连接出错。

然后咱果断发了个工单过去:

After the recent maintenance(May 1st), the program of mysql seems disappeared... I have no idea of possible reasons. Here are some logs:
=============================
$ sudo service --status-all
...
 [ - ]  mysqld
...
$ sudo update-rc.d mysqld defaults
$ sudo service --status-all
...
 [ - ]  mysqld
...
$ service mysqld start
Failed to start mysqld.service: The name org.freedesktop.PolicyKit1 was not provided by any .service files
See system logs and 'systemctl status mysqld.service' for details.
$ systemctl status mysqld.service
  mysqld.service - LSB: start and stop MySQL
   Loaded: loaded (/etc/init.d/mysqld; generated; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2018-05-01 23:25:35 CST; 2 days ago
     Docs: man:systemd-sysv-generator(8)
=============================
I really need to recovery my database...Hope there's a solution for me.

I forgot the main part...
=============================
$ which mysql
mysql not found
$ which mysqld
mysqld not found
$ apt list | grep mysql
...
mha4mysql-manager/artful,artful 0.55-1 all
mha4mysql-node/artful,artful 0.54-1 all
monodoc-mysql-manual/artful,artful 6.4.3-4 all
mysql-client/artful-updates,artful-updates,artful-security,artful-security 5.7.22-0ubuntu0.17.10.1 all
mysql-client-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-client-core-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-common/artful,artful 5.8+1.0.2ubuntu1 all
mysql-sandbox/artful,artful 3.2.05-1 all
mysql-server/artful-updates,artful-updates,artful-security,artful-security 5.7.22-0ubuntu0.17.10.1 all
mysql-server-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-server-core-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-source-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-testsuite/artful-updates,artful-updates,artful-security,artful-security 5.7.22-0ubuntu0.17.10.1 all
mysql-testsuite-5.7/artful-updates,artful-security 5.7.22-0ubuntu0.17.10.1 amd64
mysql-utilities/artful,artful 1.6.4-1 all
mysql-workbench/artful 6.3.8+dfsg-1build1 amd64
mysql-workbench-data/artful,artful 6.3.8+dfsg-1build1 all
mysqltcl/artful 3.052-2 amd64
mysqltuner/artful,artful 1.7.2-1 all
...
=============================

I've had some tries but I failed. Now it is:
=============================
$ sudo ln -s /usr/local/mysql/bin/mysql /usr/bin
$ sudo ln -s /usr/local/mysql/bin/mysqld /usr/bin
$ sudo service mysqld start
Job for mysqld.service failed because the control process exited with error code.
See "systemctl  status mysqld.service" and "journalctl  -xe" for details.
$ systemctl status mysqld.service
  mysqld.service - LSB: start and stop MySQL
   Loaded: loaded (/etc/init.d/mysqld; generated; vendor preset: enabled)
   Active: failed (Result: exit-code) since Fri 2018-05-04 23:44:53 CST; 32s ago
     Docs: man:systemd-sysv-generator(8)
  Process: 5246 ExecStart=/etc/init.d/mysqld start (code=exited, status=1/FAILURE)
=============================

这三段日志看得我真是心力交瘁…怎么会有这么奇怪的问题啊…按理来说换内核mysql也不至于被删了啊…

客服给我的建议是用这个命令查看另一堆日志sudo journalctl -xe,好吧,然后看了以后跟之前那一堆日志差不了多少。

这时候我已经快放弃了…我开始思考备份服务器然后rebuild来恢复…不过那样的话时间成本就高到不知道哪里去了

在备份完成并保存到本地之后,我脑子一抽顺手看了一下MySQL的错误日志,然后,呵呵呵…

/usr/local/mysql/bin/mysqld: File './mysql-bin.~rec~' not found (Errcode: 13 - Permission denied)
2018-05-01 23:06:08 13510 [ERROR] MYSQL_BIN_LOG::open_purge_index_file failed to open register  file.
2018-05-01 23:06:08 13510 [ERROR] MYSQL_BIN_LOG::open_index_file failed to sync the index file.
2018-05-01 23:06:08 13510 [ERROR] Aborting

2018-05-01 23:06:08 13510 [Note] Binlog end
2018-05-01 23:06:08 13510 [Note] /usr/local/mysql/bin/mysqld: Shutdown complete

看到辣个大大的Permission denied没有!什么鬼啊,权限出错,一手sudo chown -R mysql mysql过去,再次打开,恢复了

这真心是次ZZ的维护啊…

作者: DTSDAO

A C G P - Whatever it takes.

说点什么?

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据