[RFC] set based apis...

[Martin Pool]

On 7/18/07, Robert Collins <robertc at robertcollins.net> wrote:
> I'd like to start a pattern of methods that deal with historical data.
> The pattern is to query for sets, and return iterators that are self
> describing so that the returned order can vary from the requested one.
> We should allow the order to be defined only when it really matters -
> but when we allow it we probably want to allow it all the way down.
>
> This is to allow effective read coalescing and index scanning without
> requiring in-memory sort by the lowest layers.
>
> Thoughts?

As a meta-thought, I think these general proposals are more effective
if you give a concrete example.

If you mean 'iterators' specifically then we need to bear in mind the
problems that can occur because we don't know when an iterator is
closed - locking, resource collection, and so on.  It may be ok in
specific cases.  Also, using an iterator rather than an object makes
it a bit harder to evolve the interface.  (Though I guess we can have
an object fulfilling the iterator protocol that does other things
too.)

By 'self describing' I assume you mean that it will be yielding (key,
value) tuples, or something similar - and if the caller provides keys,
they might be returned out of order.

As stated it seems unobjectionable.  I guess the question to ask in
any particular case is: does the producer or the consumer have a
strong preference for any order?  Is there an order that can suit
both?  Will the consumer suffer if the order is left undefined?

-- 
Martin