[MERGE] Integration ordering

[Aaron Bentley]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Robert Collins wrote:
> + * Deprecating versioned files as a supported API: This collaborates with the
> +   Repository API but can probably be done by adding a replacement API for
> +   places where the versioned-file api is used. We may well want to keep a
> +   concept of 'a file over time' or 'inventories over time', so the existing
> +   repository model of exposing versioned file objects may be ok; what we need
> +   to ensure we do is remove the places in the code base where you create or
> +   remove or otherwise describe manipulation of the storage by knit rather than
> +   talking at the level of file ids and revision ids.

This is some nuance I hadn't heard before.  Makes sense to a degree, but
whatever the preferred order for insertion is, we'll probably need
interfaces that make batch insertion easy.

> The current
> +   versioned-file API would be a burden for implementors of a blob based
> +   repository format, so the removal of callers, and deprecation of those parts
> +   of the API should be done before creating a blob based repository format.

I think I should point out that these changes could make knit-based
repositories slower.  Hopefully there won't be fallout, but we should be
aware of the possibility.

> + * _iter_changes based merging: If the current _iter_changes_ API is
> +   insufficient, we should know about that before designing the disk format for
> +   generating fast _iter_changes_ output.

I very much doubt that there's a mismatch here.  But yes, I can work on it.

> + * Working tree disk ordering: Knowing the expected order for disk operations
> +   may influence the needed use case specific APIs, so having a solid
> +   understanding of what is optimal - and why - and whether it is pessimal on
> +   non linux platforms is rather important.

Well, apart from the create-in-limbo-parent change, I haven't found any
particular order of operations faster than any other.  And that's
without repository overhead.

> + * Be able to version files greater than memory in size: This cannot be
> +   achieved until all parts of the library which deal with user files are able
> +   to provide access to files larger than memory. Many strategies can be
> +   considered for this - such as temporary files on disk, memory mapping etc.
> +   We should have enough of a design laid out that developers of repository and
> +   tree logic are able to start exposing apis, and considering requirements
> +   related to them, to let this happen.

I really really hate this change.  It'll make development so much harder.

> + * New container format: Its hard to tell what the right way to structure the
> +   layering is. Probably having smooth layering down to the point that code
> +   wants to operate on the containers directly will make this more clear.

Certainly I would like a higher-level API.  At minimum, I want to store
"compression type" and "parent list" of 99% of my records.

And TBH, it wouldn't suck if names could be generated from revision-id,
file-id and a string.


> +   bundles will become a read-only branch & repository, the smart server wants
> +   streaming-containers, and we are planning a pack based repository, it
> +   appears that we will have three different direct container users. However,
> +   the bundle user may in fact be fake - because it really is a repository.

And indeed, the smart server may also wind up streaming
repositories/bundles.  (Though I suppose your point was that it could
stream other data as well.)

> + * Separation of annotation cache: Making the disk changes to achieve this
> +   depends on the new API being created. Bundles probably want to be
> +   annotation-free, so they are a form of implementation of this and will need
> +   the on-demand annotation facility.

OTOH, bundles may contain multiparent deltas, and those can be used for
annotation quite efficiently.

Aaron
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGd02x0F+nu1YWqI0RAlVJAJ9+5zUp0lp9cR+XHQxoFiUBXKSqfgCfdDVk
bW61+YGAxAYCNwpjikA2404=
=mRnD
-----END PGP SIGNATURE-----