Some plans for scmproj

[Alexander Belchenko]

Michael Gliwinski пишет:
> On Wednesday 25 November 2009 15:49:40 Alexander Belchenko wrote:
>> I'd like to share some of my plans regarding nearest future of scmproj
>> plugin development. As many others I've expected to see nested trees
>> support to materialize in bzr itself, but it's still not here, so I have
>> to continue improve my plugin.
> 
> Thank you for doing so.  I think the plugin can be useful regardless if nested 
> trees support is added to bzr.

Well, in fact the first initial design of scmproj (when it was proof of 
concept) was too complex. And I've made many simplification during first 
versions when Marius has joined me.

Now I see I need even more simplify it to get progress on critical features.

To be honest initial design was too much VCS agnostic. I've dreamed to 
have support for several VCS in scmproj, as it done in config-manager. 
But now I can say this was too ambitious and requires too much efforts. 
Much more than free time I have.

> [snip history and externals ref]
> 
>> So, to unblock further development I decided to remove several features:
>> alts (though they are very useful), subsets (never proved to be very
>> useful) and move project.cfg from separate branch that lived in .scmproj
>> directory to in-tree config, using the latest approved way with .bzrmeta
>> (thanks, Ian). The latter means that I also will remove support for
>> boxed workspaces which is far from ideal (because of limitations of
>> lightweight checkouts) anyway.
>>
>> Removing alts is critical point, because right now it blocks snapshots
>> implementation. Also it will simplify some other things. And also
>> unblocks desire to have several project configs with different number of
>> components (subsets does not help us here, unfortunately).
> 
> That (ability to have project config with different number of components) 
> would be really awesome.  I was just looking for something like that :)

Currently at my work we using separate configs for different number of 
components. For example our main application called "dispatcher" is 
constantly growing in number of supported features and supported 
external devices (we develop specific embedded system). So every time we 
get new external device, we develop new "driver" for it and it becomes 
new component in the project. Current design of scmproj does not support 
simultaneous existence of configurations with different set of 
components. This can be emulated with subsets, but it introduces 
additional complexity.

So given all this in mind, I think just moving project.cfg from separate 
.scmproj control branch to top-level branch of project (into 
.bzrmeta/scmproj) will help to get rid of extra entity (.scmproj) and 
get it closer to natural way of working with project.

Idea to using separate .scmproj branch has its own pros and cons. The 
main pros is VCS agnostic way. The cons as I see it now: harder to track 
all things at once.

> However, do you have any plans to reimplement alt/variant like functionality 
> on top of the new architecture?  I must say at the moment I rely on them 
> heavily.

I'm sorry to say this, but most likely alts will go away.
The main problem for me now is snapshots. Every snapshot should store 
the revision ids of all components tips. If you have several alts, then 
you'll have several snapshots (every snapshot dedicated to every alt). 
And snapshots should be stored in .scmproj branch (in old design). The 
problem wait you when you start to merge 2 diverged .scmproj branches.

If I commit new snapshots to alt A and B, and you commit new snapshots 
to alt A and B, and our snapshots diverge, then one of us will have big 
problems trying to merge those. There will be conflicts, and to solve 
them we need to merge all components corresponding to alt A and B. But 
you can't have 2 different branches with conflicts in one directory.
See?

So we need to merge alt A first, and then merge alt B, and then we can 
solve all conflicts. But bzr works with entire tree, not with separate 
files, and there is problems.

So my new attempt is to use different top-level project directories to 
keep different variants of projects (different alts). It will be similar 
to other concepts existing in bzr.

Removing alts makes the code much much simpler. And I hope it will be 
much simpler for other people to use scmproj.

>> Subprojects will be implemented very soon but in limited fashion
>> (without variants supports, because we deprecate alts).
>>
>> Decision to move project config to .bzrmeta is hard, but it seems my
>> initial plans for separate .scmproj control branch have not used in full
>> potential, so looking at bzr-externals success I'd like to not continue
>> this path.
>>
>> Also, I'm planning to release latest code from trunk as 0.4.6 and then
>> in several steps going to new design as listed above. It will be major
>> design, UI and API break, but I hope it will make scmproj much better.
>>
>> If you're using my plugin and my planned changes will affect you --
>> please speak. I can put the latest verison into maintain mode for some
>> time.

This is done now. 0.4.6 is released, and tagged correspondingly.
The old design code available in lp:bzr-scmproj/0.4.5 branch.

>> If you want to share your ideas, I'll be interested to hear them, but in
>> general the listed approach looks very promising.
>>
>> A.
> 
> I actually see the scmproj plugin as something more than externals/nested 
> trees support.  In fact I was looking into a possibility of adding 
> project/component resources, for example my current use case is creating 
> additional files, hardlinks, softlinks, etc. when the project is checked 
> out/updated, etc.

As I said it looks closer to what bzr-builder doing, or to what 
config-manager do. As I said the initial project clone/update problem is 
not so big problem for me. But snapshots really hurts us too much. 
Imagine our project with 15+ components and if you want to recreate past 
state of all branches? Currently we're using tags. But it's very 
limited. Snapshots should solve this.

> I'm interested in how the architecture will evolve, I'm sure the changes are 
> for the better so keep up the good work!

Thanks!

> I will follow up next week, am a bit short on time today :)

I hope I'll have some progress next week.