ARM branch maintenance proposal

[Dechesne, Nicolas]

On 08/12/2010 11:29 AM, Tim Gardner wrote:
>> On 08/12/2010 10:13 AM, Andy Whitcroft wrote:
>>
>>      
>>> We have classically taken ARM branches as a stack of patches against
>>> the base release for Ubuntu and rebased those directly onto the tip of
>>> the master branch.  This has allowed us to keep those branches in sync
>>> with our master branch by simply rebasing them as master moves forward.
>>> This means that Security fixes and general Ubuntu goodness propogate to
>>> those branches as they rebase.
>>>
>>> More recently we have started to receive much more complex source trees.
>>> Typically these are based on the target release but they are not simple
>>> patch stacks.  They are complex merged trees pulling in changes from
>>> multiple sources.  The tip of them representing the required tree.
>>> These are not suitable for generating a patch stack from and thus present
>>> a maintenance issue.  We have proposed a merge based system to maintain
>>> these, whereby the source tree, the Ubuntu master branch, and a packaging
>>> branch are merged to generate the result.  This however does impose a
>>> new methodolgy on stable maintenance.
>>>
>>> For other subsystems (such as AppArmor and aufs2) we have yet a third
>>> approach.  Here we take the tip if the upstream tree and literally copy
>>> over the new files into our tree.  These are then committed as a single
>>> commit 'update aufs2 to upstream as at dd-mm-yyyy'.  This has the advantage
>>> that we know our tree is exactly the same as the source tree as well as
>>> allowing us to maintain the tree in the normal way.  The subsystem and
>>> any updates are mearly patches in our stack.
>>>
>>> I am proposing a new mechanism for maintaining ARM branches, specifically
>>> for upstream trees which are very complex and cannot be converted
>>> into a patch stack.  For these I propose we adopt the subsystem update
>>> model above.  That we convert the delta between the branch tip and the
>>> that trees base version into a patch, and apply that patch as normal
>>> to the branch.  This allows maintenance of the tree to continue in the
>>> normal way, via rebase against the master branch.  When updates to the
>>> upstream ARM tree occur we simply can diff the new tip against that
>>> taken in the last update and again apply the delta as an update patch,
>>> this does require we carefully track our sync points when committing them.
>>>
>>> Taking the current ARM ti-omap4 update as an example, we have a tree from
>>> the vendor which is based on v2.6.35.  We therefore can simply diff the
>>> that branch against v2.6.35.  The resulting patch can then be applied to
>>> the tip of the master branch.
>>>
>>>        $ git diff v2.6.35..L29.4>UPDATE
>>>        $ git checkout -b temp v2.6.35
>>>        $ git apply -C1 --index UPDATE
>>>        $ git commit -s -m 'update ARM thingy to SHA1'
>>>        $ git rebase origin/master
>>>
>>> Then we can simply apply the branch packaging and we are ready to upload.
>>> For a subsequent updates we would simply diff from the SHA1 recorded
>>> above to the tip of the branch and again apply the patch.
>>>
>>> By taking this approach we do lose the ability to bisect the upstream
>>> tree.  But where the upsteam tree needs such bisection we would have to
>>> hand generate the trees in order to have the full debian infrastructure
>>> in place.  Which is equally easy to do using the original tree, overall
>>> not a loss.
>>>
>>> Comments?
>>>
>>> -apw
>>>
>>>
>>>        
>> This seems like quite a reasonable approach. As package maintainers we
>> really don't care that much about the content of the ginormous patch
>> that gets the vendor from v2.6.35 to their working tree. We only care
>> that the patch exists and that we can easily rebase against _our_ master
>> in order to maintain ongoing Ubuntu goodness.
>>
>> rtg
>>
>>      
> I like this.  I think it will make my life as the linaro kernel package
> maintainer easier.
>
>    
Andy, as you know, I am the provider of this big giant patch in the 
ti-omap4 branch ;-) i understand what you are trying to achieve here, 
and my fear is that the rebase on ubuntu master might be quite 
problematic... well I just did the test now with the current branch 
states, and the rebase went quite fine (3 merge conflicts out of this 
huge patchset). But how do we manage updates now? Do you expect that we 
continue to put our patches in our tree, and you will apply them on top 
of 'UPDATE' or do you plan to generate UPDATE for each new patch set 
from us? Especially because OMAP3 support is managed in master and OMAP4 
in ti-omap4..

Also this model fits well once the vendor is able to provide a BSP which 
is aligned with the kernel version which is in master which is just 
going to happen now for OMAP4. How do we proceed before (e.g. projecting 
to 11.04) while master is moving at a faster pace than ti branch?

Regarding the linaro kernel, I don't think that might help too much. 
Linaro kernel is mostly based on mainline stuff and it does not get all 
the "fancy" stuff that we put in ti-omap4. at the some point of course 
most of these 1300+ patches might end up in mainline, but until there 
ti-omap4 branch and linaro branch will be a lot different ( i think the 
linaro does not even enable OMAP4 for now). So Linaro kernel should not 
blindly take the big 'UPDATE' and merge it until the code is close to 
get into mainline.  Finally our huge patch set includes patches outside 
of omap (ARM, network, and other). So when merging other ARM vendors 
with the TI patches you might get lots of conflicts, having individual 
patches as commits might help git rebase do a better job, also getting 
the granularity of the commit would ease the debug/bisect as you said.

this is just a thought... in any case we will continue to provide you a 
branch with all TI patches that need to be pulled in ti-omap4. i recall 
that when we started to work together we had decided to maintain a 
branch that would look like: mainline + TI patches + Ubuntu sauce that 
could later become ti-omap4 branch.