Nasty MongoDB upsert behaviour

[Menno Smits]

Christian and I have just learned about an unfortunate property of upserts
in MongoDB. It's documented, but unexpected, and I doubt many people know
about it. From the docs:

With a unique index, if multiple applications issue the same update with
> upsert: true, exactly one update() would successfully insert a new document.
>


The remaining operations would either:
> * update the newly inserted document, or

** fail when they attempted to insert a duplicate.*



If the operation fails because of a duplicate index key error, *applications
> may retry the operation* which will succeed as an update operation.


 (
https://docs.mongodb.com/v3.2/reference/method/db.collection.update/#use-unique-indexes
)

This means that an upsert may result in either a new document being
inserted, an existing document being updated, or a duplicate key error
being returned. If the duplicate key error happens it's up to the client to
retry. Yuck!

These semantics exist in all versions of MongoDB that we use with Juju but
under 3.2 with WiredTiger duplicate key violations with upserts seem to be
happening a lot more often. We've been seeing them occasionally with
upserts into txns.stash that mgo/txn does.

Christian and I are about to propose a PR for mgo/txn which deals with this
issue but a check of the Juju state package shows a number of other uses of
upserts which also need to be changed. The most important of these is
probably the upserts for the sequence collection. Christian and I will look
at these next.

Please keep this in mind when interacting with MongoDB.

- Menno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20160627/0a041518/attachment.html>