Automatic environment filtering for DB queries

[John Meinel]

I've had this around to think about for a bit. I think it is ok, though
sniffing an input parameter to change behavior seems brittle. Mostly
because the object isn't really design to work that way. Could we wrap the
objects so that we just have Find/FindId do the right thing to start?

I suppose that is a fair amount of more work. I certainly do like the idea
of having a common pattern rather than requiring everyone to know exactly
whether this object is different than the rest.

John
=:->


On Thu, Nov 13, 2014 at 8:10 AM, Menno Smits <menno.smits at canonical.com>
wrote:

> Team Onyx has been busy preparing the MongoDB collections used by Juju to
> support data for multiple environments. For each collection that needs to
> store data for multiple environments we have been adding the a "env-uuid"
> field to each document and prefixing the environment UUID to the document
> id, with the previous document id being moved to a new field. Apart from
> the document changes themselves, we've been adjusting the state package
> implementation to match the document changes.
>
> Part of this task is ensuring that all DB queries correctly filter by
> environment so that we don't end up unintentionally leaking data across
> environments. To avoid opportunities for us to forget to add environment
> filtering to the DB queries using by Juju, sabdfl would like us to consider
> ways to make this filtering happen automatically. To this end, I propose we
> add the following methods:
>
> func (s *State) Find(coll *mgo.Collection, sel bson.D) *mgo.Query
> func (s *State) FindId(coll *mgo.Collection, id string) *mgo.Query
>
>
> The idea is that almost all MongoDB queries performed by Juju would use
> these 2 methods. They would become the default way that we do queries, used
> even on collections that don't contain data for multiple environments.
>
> Both methods would check the collection passed in against a fixed set of
> collections that use environment UUIDs. If the collection doesn't use
> environment UUIDs then the lookup is passed through to mgo unmodified. If
> the collection *does* use environment UUIDs then Find() would
> automatically add the appropriate "env-uuid" field to the query selector.
> Similarly, FindId() would automatically call docID() on the supplied id.
> Pretty simple.
>
> If use of these methods becomes default way the team does DB queries in
> Juju code, then we greatly reduce the risk of leaking data between
> environments. They also allows us to remove one concern from each
> Find/FindId call site - as environment filtering is taken care of by these
> methods, it does not having to be repeated all throughout the codebase. To
> get us started, I intend to perform a mass-refactoring of all existing Find
> and FindId calls to use these new methods.
>
> To make the proposal clearer, here's some examples:
>
> Find call:   err := units.Find(bson.D{{"env-uuid": ...}, {"service":
> service}}).All{&docs)
>   becomes:   err := st.Find(units, bson.D{{"service": service}}).All{&docs)
>
> FindId call: err = units.FindId(w.st.docID(unitName)).One(&doc)
>     becomes: err = w.st.FindId(units, unitName).One(&doc)
>
> Does this sound reasonable? Is there another approach I should be
> considering?
>
> - Menno
>
>
>
> --
> Juju-dev mailing list
> Juju-dev at lists.ubuntu.com
> Modify settings or unsubscribe at:
> https://lists.ubuntu.com/mailman/listinfo/juju-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/juju-dev/attachments/20141118/939cbefa/attachment.html>