Automatic environment filtering for DB queries

[Menno Smits]

Team Onyx has been busy preparing the MongoDB collections used by Juju to
support data for multiple environments. For each collection that needs to
store data for multiple environments we have been adding the a "env-uuid"
field to each document and prefixing the environment UUID to the document
id, with the previous document id being moved to a new field. Apart from
the document changes themselves, we've been adjusting the state package
implementation to match the document changes.

Part of this task is ensuring that all DB queries correctly filter by
environment so that we don't end up unintentionally leaking data across
environments. To avoid opportunities for us to forget to add environment
filtering to the DB queries using by Juju, sabdfl would like us to consider
ways to make this filtering happen automatically. To this end, I propose we
add the following methods:

func (s *State) Find(coll *mgo.Collection, sel bson.D) *mgo.Query
func (s *State) FindId(coll *mgo.Collection, id string) *mgo.Query


The idea is that almost all MongoDB queries performed by Juju would use
these 2 methods. They would become the default way that we do queries, used
even on collections that don't contain data for multiple environments.

Both methods would check the collection passed in against a fixed set of
collections that use environment UUIDs. If the collection doesn't use
environment UUIDs then the lookup is passed through to mgo unmodified. If
the collection *does* use environment UUIDs then Find() would automatically
add the appropriate "env-uuid" field to the query selector.  Similarly,
FindId() would automatically call docID() on the supplied id. Pretty simple.

If use of these methods becomes default way the team does DB queries in
Juju code, then we greatly reduce the risk of leaking data between
environments. They also allows us to remove one concern from each
Find/FindId call site - as environment filtering is taken care of by these
methods, it does not having to be repeated all throughout the codebase. To
get us started, I intend to perform a mass-refactoring of all existing Find
and FindId calls to use these new methods.

To make the proposal clearer, here's some examples:

Find call:   err := units.Find(bson.D{{"env-uuid": ...}, {"service":
service}}).All{&docs)
  becomes:   err := st.Find(units, bson.D{{"service": service}}).All{&docs)

FindId call: err = units.FindId(w.st.docID(unitName)).One(&doc)
    becomes: err = w.st.FindId(units, unitName).One(&doc)

Does this sound reasonable? Is there another approach I should be
considering?

- Menno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20141113/351a7a41/attachment.html>