#acl PaulHowarth:read,write,admin,revert,delete All:read

=== Friday 4th February 2022 ===

==== Local Packages ====

 * Updated `dovecot`:
  * Updated `dovecot` to 2.3.18:
   * Removed `mail_cache_lookup_finished` event: this event wasn't especially useful, but it increased CPU usage significantly
   * fts: Don't index inline base64 encoded content in FTS indexes using the generic tokenizer; this reduces the FTS index sizes by removing input that is very unlikely to be searched for
    * See https://doc.dovecot.org/configuration_manual/fts/tokenization for details on how base64 is detected
    * Only applies when using `libfts`
   * lmtp: Session IDs are now preserved through proxied connections, so LMTP sessions can be tracked; this slightly changes the LMTP session ID format by appending "`:Tn`" (transaction), "`:Pn`" (proxy connection) and "`:Rn`" (recipient) counters after the session ID prefix
   * Events now have "`reason_code`" field, which can provide a list of reasons why the event is happening (see https://doc.dovecot.org/admin_manual/event_reasons/)
   * New events are added (see https://doc.dovecot.org/admin_manual/list_of_events/)
   * fts: Added `fts_header_excludes` and `fts_header_includes` settings to specify which headers to index (see https://doc.dovecot.org/settings/plugin/fts-plugin#plugin-fts-setting-fts-header-excludes for configuration details)
   * fts: Initialize the `textcat` language detection library only once per process; this can reduce CPU usage if `fts_languages` setting has multiple languages listed and `service indexer-worker { service_count }` isn't 1
    * Only applies when using `libfts`
   * `lib-storage`: Reduced CPU usage significantly for some operations that accessed lots of emails (e.g. fetching all flags in a folder, `SORT`, ...)
   * lib: `DOVECOT_PREREQ()` - Add micro version that enables compiling external plugins against different versions of Dovecot
   * lmtp: Added new `lmtp_verbose_replies` setting that makes errors sent to the LMTP client much more verbose with details about why exactly backend proxy connections or commands are failing
   * submission: Support implicit `SASL EXTERNAL` with `submission_client_workarounds=implicit-auth-external`; this allows automatically logging in when SSL client certificate is present
   * `*-login`: Statistics were disabled if stats process connection was lost
   * auth: Authentication master user login fails with `SCRAM-*` SASL mechanisms
   * auth: With `auth_cache_verify_password_with_worker=yes`, `passdb` extra fields in the auth cache got lost
   * `doveadm`: Fixed crash if `zlib_save_level` setting was specified, but `zlib_save` was unset (v2.3.15 regression)
   * `doveadm`: Proxying can panic when flushing print output (v2.3.17 regression)
   . Fixes: `Panic: file ioloop.c: line 865 (io_loop_destroy): assertion failed: (ioloop == current_ioloop)`
   * `doveadm`: `stats add --group-by` parameter didn't work
   * fts: Using `email-address` fts tokenizer could result in excessive memory usage with garbage email input, which could cause the `indexer-worker` processes to fail due to reaching the VSZ memory size limit
    * Only applies when using `libfts`
   * imap: A `SEARCH` command timing out while fts returns indexes may timeout returning "`NO [SERVERBUG]`", while it should return "`NO [INUSE]`" instead
   * imap: `LIST-EXTENDED` doesn't return `STATUS` for all folders; sending `LIST .. RETURN (SUBSCRIBED STATUS (...))` did not return `STATUS` for folders that are not subscribed when they have a child folder that is subscribed as mandated by IMAP RFCs
   * `imapc`: Mailbox `vsize` calculation crashed with `Panic: file index-mailbox-size.c: line 344 (index_mailbox_vsize_hdr_add_missing): assertion failed: (mails_left > 0)`
   * `indexer`: If `indexer-worker` crashes, the request it was processing gets stuck in the `indexer` process; this stops indexing for the folder until `indexer` process is restarted (v2.3.14 regression)
   * `indexer`: Process was slowly leaking memory for each indexing request
   * `lib-event`: Unnamed events were wrongly filtered out for event/metric filters like "`event=abc OR something_independent_of_event_name`"
   * `lib-index`: 64-bit big endian CPUs handle `last_used` field in `dovecot.index.cache` wrong
   * `lib-ssl-iostream`: Fix buggy OpenSSL error handling without assert-crashing; if there is no error available, log it as an error instead of crashing
   . The previous fix for this in v2.3.11 was incomplete
   . Fixes: `Panic: file istream-openssl.c: line 51 (i_stream_ssl_read_real): assertion failed: (errno != 0)`
   * lmtp: Out-of-memory issues can happen when proxying large messages to LMTP backend servers that accept the message data too slowly
   * `master`: HAProxy header parsing has read buffer overflow if provided header size is invalid; this happens only if `inet_listener { haproxy=yes }` is configured and only if the remote IP address is in `haproxy_trusted_networks`
   * `old_stats`: Plugin kept increasing memory usage, which became noticeable with long-running imap sessions
   * stats: Dynamically adding same metric multiple times causes multiple stats
   * `submission-login`: Authentication does not accept OAUTH2 token (or other very long credentials) because it considers the line to be too long
   * `submission-login`: Process can crash if `HELO` is pipelined with an invalid domain
   * `submission-proxy`: Don't use `SASL-IR` if it would make the `AUTH` command line longer than 512 bytes
   * `submission`: Service would crash if relay server authentication failed
   * virtual: FTS search in a virtual folder could crash if there are duplicate mailbox GUIDs; this mainly happened when user had both `INBOX` and `INBOX/INBOX` folders and the namespace prefix was `INBOX/`
   . Fixes: `Panic: file hash.c: line 252 (hash_table_insert_node): assertion failed: (opcode == HASH_TABLE_OP_UPDATE)`
   * virtual: If mailbox opening fails, the backend mailbox is leaked and process crashes when client disconnects
   . Fixes: `Panic: file mail-user.c: line 232 (mail_user_deinit): assertion failed: ((*user)->refcount == 1)`
   * virtual: Searching headers in virtual folders didn't always use full-text search indexes, if `fts_enforced=no` or body
  * Updated `pigeonhole` to 0.5.18:
   * `duplicate`: Users without a home directory can crash with Sieve when using duplicate database (v2.3.17 regression)
   * `imapsieve`: When mail was expunged when processing `imapsieve` events, a crash could occur; fixes:
   . `Panic: file mail-index-map.c: line 558 (mail_index_map_lookup_seq_range): assertion failed: (first_uid > 0)`
   * `managesieve-login`: Proxy didn't support forwarding the `forward_*` passdb fields
   * `redirect`: Sieve would crash if redirect after `keep-equivalent` action failed
   * `sieve`: Interpreter crashes when the Sieve index extension is used with index zero
   * `vnd.dovecot.filter`: Envelope sender string may become corrupted when Sieve scripts are using `vnd.dovecot.filter`; this could end up corrupting mbox's `From` line and return wrong envelope sender string in Sieve tests

----