Cyrus IMAP Server: Locking
**************************


Mailboxes
=========

For mailboxes, we lock in this order:

* Mailbox Namelock (shared) <== possibly reversible with
  conversationsdb

* user conversations db

* cyrus.index

If you want to do any mailboxes.db transactions, they need to open and
close without changing any mailbox locking during the transaction.

Likewise seen and statuscache are always done without changing mailbox
locking during their transaction.

Annotations databases are a mess.


Xapian
======

* Xapian per-user NAMELOCK (shared or exclusive)

* xapianactive file lock (shared or exclusive)

Shared namelock holds the following invariants:

* xapianactive file contents are not changed

* directories mentioned in xapianactive are not cleaned up

Xapianactive exclusive lock holds the following invariants:

* owner may write to first database mentioned in xapianactive

Xapianactive shared lock holds:

* all databases in xapianactive are readable and a consistent read
  can be made across all of them, even with multiple requests while
  the lock is held.


Locking orders
--------------

SHARED case:

* user conversations db <=== possibly reversible with SHARED xapian
  namelock

* SHARED xapian namelock

* xapianactive lock (shared to search, exclusive to write)

* cyrus.index may be locked either side of the xapianactive lock,
  because the conversationsdb lock protects it from races.

EXCLUSIVE case:

* EXCLUSIVE xapian namelock That’s it.  While you’ve got this, you
  can add or delete items from the xapianactive file, and delete paths
  on disk for directories that you have removed (either during or
  after locking).  No other locks are permitted.

If you hold a SHARED xapian namelock, you may write to a .NEW folder
for a xapianactive entry that you created without taking any
additional locks, because nothing can clean it under you, and nothing
else can read it.  This is how the repack case works.


Lock lifetime
=============

* Shared mailbox namelock: *   possibly hours

* conversations db and below *   short as possible

Mailbox namelock holds the following invariants:

* cyrus.index may not be repacked, however flags and modseqs may be
  updated

* cyrus.annotations records may change (kind of buggy and bad,
  ideally we’d always write new ones if we changed them and keep the
  old ones)

* cyrus.cache may be appended, but never changed

* spool files may not be deleted (already can’t be changed)
