Hi,
The current advice is to disable HTTP Compression to mitigate BREACH attacks.
In both zimbra 9 and 10, if you do this, it exposes a bug in the /opt/zimbra/jetty_base/etc/jetty.xml.in file that results in the server being unable to start due to malformed XML in the generated /opt/zimbra/jetty_base/etc/jetty.xml file.
From a working 10.0.7 serverJust make the simple change to turn off HTTP Compression, and attempt to restart mailboxd
[zimbra@mail ~]$ zmprov ms mail.server zimbraHttpCompressionEnabled FALSE
[zimbra@mail ~]$ zmmailboxdctl restart
Stopping mailboxd...done.
Starting mailboxd...failed.
[zimbra@mail ~]$ zmprov ms mail.server zimbraHttpCompressionEnabled TRUE
ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)In /opt/zimbra/log/zmmailboxd.out you can see the issue reported The only way to recover the server so that it can start again is to manually edit the /opt/zimbra/jetty_base/etc/jetty.xml.in file and remove the offending comment. In 10.0.x this is line 976 of the file
In 9.0.0.p39 this 1s line 944 of the file
Then restart mailboxd againEDIT: Added line number for Release 9
The current advice is to disable HTTP Compression to mitigate BREACH attacks.
In both zimbra 9 and 10, if you do this, it exposes a bug in the /opt/zimbra/jetty_base/etc/jetty.xml.in file that results in the server being unable to start due to malformed XML in the generated /opt/zimbra/jetty_base/etc/jetty.xml file.
From a working 10.0.7 server
Code:
[zimbra@mail ~]$ zmcontrol statusHost mail.server amavis Running antispam Running antivirus Running ldap Running logger Running mailbox Running memcached Running mta Running opendkim Running proxy Running service webapp Running snmp Running spell Running stats Running zimbra webapp Running zimbraAdmin webapp Running zimlet webapp Running zmconfigd Running[zimbra@mail ~]$ zmprov ms mail.server zimbraHttpCompressionEnabled FALSE
[zimbra@mail ~]$ zmmailboxdctl restart
Stopping mailboxd...done.
Starting mailboxd...failed.
[zimbra@mail ~]$ zmprov ms mail.server zimbraHttpCompressionEnabled TRUE
ERROR: zclient.IO_ERROR (invoke Connection refused, server: localhost) (cause: java.net.ConnectException Connection refused)
Code:
[zimbra@mail ~]$ zmcontrol statusHost mail.server amavis Running antispam Running antivirus Running ldap Running logger Running mailbox Stopped zmmailboxdctl is not running. memcached Running mta Running opendkim Running proxy Running service webapp Stopped zmmailboxdctl is not running. snmp Running spell Running stats Running zimbra webapp Stopped zmmailboxdctl is not running. zimbraAdmin webapp Stopped zmmailboxdctl is not running. zimlet webapp Stopped zmmailboxdctl is not running. zmconfigd RunningCode:
2024-04-19 16:03:24.487:WARN:oejx.XmlParser:main: FATAL@null line:989 col:13 : org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.2024-04-19 16:03:24.488:WARN:oejx.XmlConfiguration:main: java.security.PrivilegedActionException: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.base/java.lang.reflect.Method.invoke(Method.java:568)at org.eclipse.jetty.start.Main.invokeMain(Main.java:218)at org.eclipse.jetty.start.Main.start(Main.java:491)at org.eclipse.jetty.start.Main.main(Main.java:77)Caused by: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)at org.apache.xerces.impl.XMLScanner.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:244)at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:226)at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1881)at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.base/java.lang.reflect.Method.invoke(Method.java:568)at org.eclipse.jetty.start.Main.invokeMain(Main.java:218)at org.eclipse.jetty.start.Main.start(Main.java:491)at org.eclipse.jetty.start.Main.main(Main.java:77)java.lang.reflect.InvocationTargetExceptionat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.base/java.lang.reflect.Method.invoke(Method.java:568)at org.eclipse.jetty.start.Main.invokeMain(Main.java:218)at org.eclipse.jetty.start.Main.start(Main.java:491)at org.eclipse.jetty.start.Main.main(Main.java:77)Caused by: java.security.PrivilegedActionException: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857)... 7 moreCaused by: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)at org.apache.xerces.impl.XMLScanner.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:244)at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:226)at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1881)at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)... 8 morejava.lang.reflect.InvocationTargetExceptionat java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)at java.base/java.lang.reflect.Method.invoke(Method.java:568)at org.eclipse.jetty.start.Main.invokeMain(Main.java:218)at org.eclipse.jetty.start.Main.start(Main.java:491)at org.eclipse.jetty.start.Main.main(Main.java:77)Caused by: java.security.PrivilegedActionException: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)at org.eclipse.jetty.xml.XmlConfiguration.main(XmlConfiguration.java:1857)... 7 moreCaused by: org.xml.sax.SAXParseException; lineNumber: 989; columnNumber: 13; The string "--" is not permitted within comments.at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)at org.apache.xerces.impl.XMLScanner.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanComment(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source)at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:244)at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:226)at org.eclipse.jetty.xml.XmlConfiguration.lambda$main$3(XmlConfiguration.java:1881)at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)... 8 moreUsage: java -jar $JETTY_HOME/start.jar [options] [properties] [configs] java -jar $JETTY_HOME/start.jar --help # for more informationCode:
<!-- Modern UI uses build time compression -->In 9.0.0.p39 this 1s line 944 of the file
Then restart mailboxd again
Code:
[zimbra@mail ~]$ zmmailboxdctl restartStopping mailboxd...mailboxd is not running.Starting mailboxd...done.[zimbra@mail ~]$ zmcontrol statusHost mail.server amavis Running antispam Running antivirus Running ldap Running logger Running mailbox Running memcached Running mta Running opendkim Running proxy Running service webapp Running snmp Running spell Running stats Running zimbra webapp Running zimbraAdmin webapp Running zimlet webapp Running zmconfigd RunningStatistics: Posted by liverpoolfcfan — Fri Apr 19, 2024 3:23 pm