Hello all,
I've contacted Zimbra support about this issue and have been given the number ZBUG-4074.
I am not a database expert, but as far as I have been able to analyse this, it seems to me that the problem is simply due to the old and outdated MariaDB version which, despite setting innodb_max_dirty_pages_pct to 0, will not reduce innodb_buffer_pool_pages_dirty to 0, which is the part of the if statement in the zmstorectl script. It will just lower it to 1 percent and keep it there. And if the number of modified db pages is higher than 0, but at the same time the percentage of dirty pages (LRU & free pages) is lower than 1, the script will very often wait those 600 seconds for no reason, causing unnecessary Zimbra downtime for users.
For MySQL, a similar bug is described here: https://bugs.mysql.com/bug.php?id=62534
I suspect that something like this also happens in MariaDB 10.1, which is used in Zimbra (3.5 years after EoL...).
Also according to the information I've found here:
https://bugzilla.zimbra.com/show_bug.cgi?id=37231
https://www.percona.com/blog/how-to-dec ... own-times/
the whole point of flushing dirty pages is to reduce the time it takes to *actually* shut down and restart the server, by *preparing* it for shutdown. One can keep the application online and responsive for some of that time while dirty pages are flushed from the buffer pool. But instead zmstorectl first stops the application (mailbox) and then does the flushing, so there's no reduction in downtime at all.
Correct me if I am wrong in that but I think the correct order should be
1. Set global innodb_max_dirty_pages_pct=0; to flush dirty pages
2. Monitor the number of dirty pages. If the number approaches 0 or is zero, stop the mailbox.
3. Stop the mysql
instead of the following, which is the current order
1. Stop the mailbox (and from that moment Zimbra is unavailable to users)
2. Set global innodb_max_dirty_pages_pct=0; to flush dirty pages
3. Stop the mysql
I've contacted Zimbra support about this issue and have been given the number ZBUG-4074.
I am not a database expert, but as far as I have been able to analyse this, it seems to me that the problem is simply due to the old and outdated MariaDB version which, despite setting innodb_max_dirty_pages_pct to 0, will not reduce innodb_buffer_pool_pages_dirty to 0, which is the part of the if statement in the zmstorectl script. It will just lower it to 1 percent and keep it there. And if the number of modified db pages is higher than 0, but at the same time the percentage of dirty pages (LRU & free pages) is lower than 1, the script will very often wait those 600 seconds for no reason, causing unnecessary Zimbra downtime for users.
For MySQL, a similar bug is described here: https://bugs.mysql.com/bug.php?id=62534
I suspect that something like this also happens in MariaDB 10.1, which is used in Zimbra (3.5 years after EoL...).
Also according to the information I've found here:
https://bugzilla.zimbra.com/show_bug.cgi?id=37231
https://www.percona.com/blog/how-to-dec ... own-times/
the whole point of flushing dirty pages is to reduce the time it takes to *actually* shut down and restart the server, by *preparing* it for shutdown. One can keep the application online and responsive for some of that time while dirty pages are flushed from the buffer pool. But instead zmstorectl first stops the application (mailbox) and then does the flushing, so there's no reduction in downtime at all.
Correct me if I am wrong in that but I think the correct order should be
1. Set global innodb_max_dirty_pages_pct=0; to flush dirty pages
2. Monitor the number of dirty pages. If the number approaches 0 or is zero, stop the mailbox.
3. Stop the mysql
instead of the following, which is the current order
1. Stop the mailbox (and from that moment Zimbra is unavailable to users)
2. Set global innodb_max_dirty_pages_pct=0; to flush dirty pages
3. Stop the mysql
Statistics: Posted by mocha — Mon May 20, 2024 8:55 am