Some plans for scmproj

[Michael Gliwinski]

On Friday 27 November 2009 17:39:55 Alexander Belchenko wrote:
> To be honest initial design was too much VCS agnostic. I've dreamed to
> have support for several VCS in scmproj, as it done in config-manager.
> But now I can say this was too ambitious and requires too much efforts.
> Much more than free time I have.

Yes, I know what you mean, VCS abstraction requires much effort.  On the other 
hand since support for accessing foreign formats is getting better and better 
in bazaar I suppose lack of such abstraction wouldn't necessarily be a 
problem.

> > That (ability to have project config with different number of components)
> > would be really awesome.  I was just looking for something like that :)
>
> Currently at my work we using separate configs for different number of
> components. For example our main application called "dispatcher" is
> constantly growing in number of supported features and supported
> external devices (we develop specific embedded system). So every time we
> get new external device, we develop new "driver" for it and it becomes
> new component in the project. Current design of scmproj does not support
> simultaneous existence of configurations with different set of
> components. This can be emulated with subsets, but it introduces
> additional complexity.
>
> So given all this in mind, I think just moving project.cfg from separate
> .scmproj control branch to top-level branch of project (into
> .bzrmeta/scmproj) will help to get rid of extra entity (.scmproj) and
> get it closer to natural way of working with project.

One question I have here is, does this mean that a bzr working tree will have 
to be at the root of the project directory?  Or do you plan to work-around 
this somehow?

> Idea to using separate .scmproj branch has its own pros and cons. The
> main pros is VCS agnostic way. The cons as I see it now: harder to track
> all things at once.

I'm not sure what you mean that it's harder to track things?

> > However, do you have any plans to reimplement alt/variant like
> > functionality on top of the new architecture?  I must say at the moment I
> > rely on them heavily.
>
> I'm sorry to say this, but most likely alts will go away.
> The main problem for me now is snapshots. Every snapshot should store
> the revision ids of all components tips. If you have several alts, then
> you'll have several snapshots (every snapshot dedicated to every alt).
> And snapshots should be stored in .scmproj branch (in old design). The
> problem wait you when you start to merge 2 diverged .scmproj branches.

So, snapshots are in a sense, similar to specifying revision id for a 
component/branch in project.cfg, except that they are done after checking the 
project out instead of before?  (and obviously that there can be multiple 
named snapshots, etc.)

> If I commit new snapshots to alt A and B, and you commit new snapshots
> to alt A and B, and our snapshots diverge, then one of us will have big
> problems trying to merge those. There will be conflicts, and to solve
> them we need to merge all components corresponding to alt A and B. But
> you can't have 2 different branches with conflicts in one directory.
> See?
>
> So we need to merge alt A first, and then merge alt B, and then we can
> solve all conflicts. But bzr works with entire tree, not with separate
> files, and there is problems.

Yes, I see what you mean.  One option would be to make snapshots per-alt but 
IIUC this wouldn't quite work for your use case.

On the other hand if a snapshot recorded both branch and revision info, that 
should work, no?

I know alts as they are (were) could lead to problems but a concept of a 
light 'variant' as a way of specifying multiple branches for a component and 
specifying which one is 'active' (i.e. which one is used when 'getting' the 
project) could still fit in, IMO.

> So my new attempt is to use different top-level project directories to
> keep different variants of projects (different alts). It will be similar
> to other concepts existing in bzr.

Do you mean having separate diverged branches of the control branch for each 
alt?

That has it's own problems (maintaining common things for one, as you'd have 
to cherrypick changes between branches manually), but could work if there's 
no better option.

> > I actually see the scmproj plugin as something more than externals/nested
> > trees support.  In fact I was looking into a possibility of adding
> > project/component resources, for example my current use case is creating
> > additional files, hardlinks, softlinks, etc. when the project is checked
> > out/updated, etc.
>
> As I said it looks closer to what bzr-builder doing, or to what
> config-manager do. As I said the initial project clone/update problem is
> not so big problem for me. But snapshots really hurts us too much.

I did actually have a look at bzr-builder but AFAICT it is more oriented 
towards the process of building packages and similar, i.e. you have a recipe, 
you run it, you get some result.

My use case is more for maintaining a project composed of multiple components, 
so I find scmproj's ability to run e.g. status for all components very 
useful.  The only peculiarity here is that I can't have a branch in project 
root but still need to keep some links there, pointing to files within the 
actual component branches.

> Imagine our project with 15+ components and if you want to recreate past
> state of all branches? Currently we're using tags. But it's very
> limited. Snapshots should solve this.

I don't know if I understand how snapshots should work, but if they record 
revision ids of component branches in a config file in project control 
branch, wouldn't you have problems e.g. when a component is removed from 
project.cfg in control branch?

> > I will follow up next week, am a bit short on time today :)

So I ended up swamped with work :)  Sorry for the delay.


-- 
Michael Gliwinski
Henderson Group Information Services
9-11 Hightown Avenue, Newtownabby, BT36 4RT
Phone: 028 9034 3319

**********************************************************************************************
The information in this email is confidential and may be legally privileged.  It is intended solely for the addressee and access to the email by anyone else is unauthorised.
If you are not the intended recipient, any disclosure, copying, distribution or any action taken or omitted to be taken in reliance on it, is prohibited and may be unlawful.
When addressed to our clients, any opinions or advice contained in this e-mail are subject to the terms and conditions expressed  in the governing client engagement leter or contract.
If you have received this email in error please notify support at henderson-group.com

John Henderson (Holdings) Ltd
Registered office: 9 Hightown Avenue, Mallusk, County Antrim, Northern Ireland, BT36 4RT.
Registered in Northern Ireland
Registration Number NI010588
Vat No.: 814 6399 12
*********************************************************************************