On different ways to combine trees

[Piotr Kalinowski]

Hi,

First of all thanks for taking time to answer my mail. I appreciate
that.

Alexander Belchenko <bialix at ukr.net> writes:

> You have explained scmproj very close to its reality, thanks.
> Actually, when I was started it, I wanted it to support several vcs
> systems by simple config definition. The rudiment support for this you
> can see in vcs.py module of that plugin.

I remember that. That and using a separate control branch for the
project configuration. Unfortunately, at that stage of my life I did not
have time or energy for contributing to open source projects (or writing
my own, for that matter).

> I think starting from the scratch makes sense here, using CLI as API to
> different systems will help you to quickly make a working prototype of
> your idea.

Actually, I want to first make a prototype that works only with bazaar
repositories, and then add support for other systems. But yeah, I'll be
using their CLI at the beginning.

> AFAIR, you can specify revision -r-1 in externals and scmproj configs
> to tell those suystems you want always latest revision of some
> components.

Hmm, interesting. Thanks for the hint.

> For me, the biggest problems with scmproj were:
> 1) I didn't provide a way to override source URL for components while
> getting snapshotted project. So any change in locations of components
> are disastrous.

This is the reason why nested repositories in Mercurial are dangerous,
and not recommended, unless you *really* know what you're doing. There
are ways to recover from unreachable subrepository, but it's messy.

Which is why guestrepo extension was born. The idea is to couple all the
"guests" very loosely with parent project, so that you have to always
use separate commands to get/update/push them. The advantage is that
whatever happens to guests sources, you can work with parent project's
source without any problems, and then you can easily fix source URL.

You can actually do the same with scmproj these days, if you don't use
project-get: just branch the parent project normally, and then update
subrepositories using scmproj commands, possibly after fixing any
locations.

[... snipping some wisdom ...]

> 4) Separate set of commands sometimes is good, but even with
> bzr-externals you don't have proper GUI support, like combined
> status/diff/log over all trees.

I think it's a matter of coupling. If you want tight coupling of parent
to its children, git-subtrees would indeed work a lot better. It's even
resilient to changes in source repositories' locations. With some extra
work, you could even track dependencies that come from different version
control systems, although at this point it becomes slightly tricky (just
wait until import of next version of a dependency overwrites some local
patches ;).

I want very loose coupling, with separate commands. This makes it easy
to get robustness, and encourage thinking of nested projects as
separate entities maintaining some independance. Additionally, it seems
fairly easy to:

* allow to track the same dependency in multiple points under the work
  tree, even using different versions in each location (this is not a
  nice-to-have-feature; this is something guestrepo authors added,
  because they needed it in the company they were working for; I heard
  it also happens/used to happen in Chromium);

* allow to keep only an interesting subset of components in the work
  tree (just make a separate operation to put them into the work tree,
  and update them, like git submodules).

Yes, it's a trade-off: you trade loose coupling and other features for
more commands to use. It may not be best for all scenarios you could
think of, but I have some scenarios where this will work perfectly.

> 5) Big number of components, even divided into subprojects are proved to
> be a bad thing for me. It's hard to work with.

If the components are too small, and/or have many interdependencies, it
may get messy, indeed. Fortunately enough, bazaar performance greatly
improved, so you no longer need *that* small subprojects ;)

On a related note, did you have any specific problems while working with
many components?

> In the end I was very impressed by git-subtrees idea. It solves a lot of
> the problem I had with scmproj in more simpler way.

I've looked it up, and as I mentioned earlier, I can see how it can be
appealing in some scenarios. In my company, we'll be staying with
submodules, at least because our dependencies not necessarily use git,
but a lot more complex configuration, and there are whole products that
do not use any of them. And we're talking about a really large codebase
there in dependencies (whole Chromium, actually), so if you can avoid
tracking it, it pays off not to do it ;)

Now I have to find some time for actually developing the plugin. It's
interesting concept, but I have some outstanding work in the high
priority project of learning the Swedish vocabulary. It'll take some
time then.

Regards,
Piotr Kalinowski

-- 
Intelligence is like a river: the deeper it is, the less noise it makes.