ARM branch maintenance proposal

Bryan Wu bryan.wu at canonical.com
Fri Aug 13 05:40:04 UTC 2010


On 08/13/2010 11:51 AM, Bryan Wu wrote:
> On 08/13/2010 05:58 AM, Dechesne, Nicolas wrote:
>> On 08/12/2010 11:29 AM, Tim Gardner wrote:
>>>> On 08/12/2010 10:13 AM, Andy Whitcroft wrote:
>>>>
>>>>      
>>>>> We have classically taken ARM branches as a stack of patches against
>>>>> the base release for Ubuntu and rebased those directly onto the tip of
>>>>> the master branch.  This has allowed us to keep those branches in sync
>>>>> with our master branch by simply rebasing them as master moves forward.
>>>>> This means that Security fixes and general Ubuntu goodness propogate to
>>>>> those branches as they rebase.
>>>>>
>>>>> More recently we have started to receive much more complex source trees.
>>>>> Typically these are based on the target release but they are not simple
>>>>> patch stacks.  They are complex merged trees pulling in changes from
>>>>> multiple sources.  The tip of them representing the required tree.
>>>>> These are not suitable for generating a patch stack from and thus present
>>>>> a maintenance issue.  We have proposed a merge based system to maintain
>>>>> these, whereby the source tree, the Ubuntu master branch, and a packaging
>>>>> branch are merged to generate the result.  This however does impose a
>>>>> new methodolgy on stable maintenance.
>>>>>
>>>>> For other subsystems (such as AppArmor and aufs2) we have yet a third
>>>>> approach.  Here we take the tip if the upstream tree and literally copy
>>>>> over the new files into our tree.  These are then committed as a single
>>>>> commit 'update aufs2 to upstream as at dd-mm-yyyy'.  This has the advantage
>>>>> that we know our tree is exactly the same as the source tree as well as
>>>>> allowing us to maintain the tree in the normal way.  The subsystem and
>>>>> any updates are mearly patches in our stack.
>>>>>
>>>>> I am proposing a new mechanism for maintaining ARM branches, specifically
>>>>> for upstream trees which are very complex and cannot be converted
>>>>> into a patch stack.  For these I propose we adopt the subsystem update
>>>>> model above.  That we convert the delta between the branch tip and the
>>>>> that trees base version into a patch, and apply that patch as normal
>>>>> to the branch.  This allows maintenance of the tree to continue in the
>>>>> normal way, via rebase against the master branch.  When updates to the
>>>>> upstream ARM tree occur we simply can diff the new tip against that
>>>>> taken in the last update and again apply the delta as an update patch,
>>>>> this does require we carefully track our sync points when committing them.
>>>>>
>>>>> Taking the current ARM ti-omap4 update as an example, we have a tree from
>>>>> the vendor which is based on v2.6.35.  We therefore can simply diff the
>>>>> that branch against v2.6.35.  The resulting patch can then be applied to
>>>>> the tip of the master branch.
>>>>>
>>>>>        $ git diff v2.6.35..L29.4>UPDATE
>>>>>        $ git checkout -b temp v2.6.35
>>>>>        $ git apply -C1 --index UPDATE
>>>>>        $ git commit -s -m 'update ARM thingy to SHA1'
>>>>>        $ git rebase origin/master
>>>>>
>>>>> Then we can simply apply the branch packaging and we are ready to upload.
>>>>> For a subsequent updates we would simply diff from the SHA1 recorded
>>>>> above to the tip of the branch and again apply the patch.
>>>>>
>>>>> By taking this approach we do lose the ability to bisect the upstream
>>>>> tree.  But where the upsteam tree needs such bisection we would have to
>>>>> hand generate the trees in order to have the full debian infrastructure
>>>>> in place.  Which is equally easy to do using the original tree, overall
>>>>> not a loss.
>>>>>
>>>>> Comments?
>>>>>
>>>>> -apw
>>>>>
>>>>>
>>>>>        
>>>> This seems like quite a reasonable approach. As package maintainers we
>>>> really don't care that much about the content of the ginormous patch
>>>> that gets the vendor from v2.6.35 to their working tree. We only care
>>>> that the patch exists and that we can easily rebase against _our_ master
>>>> in order to maintain ongoing Ubuntu goodness.
>>>>
>>>> rtg
>>>>
>>>>      
>>> I like this.  I think it will make my life as the linaro kernel package
>>> maintainer easier.
>>>
>>>    
>> Andy, as you know, I am the provider of this big giant patch in the 
>> ti-omap4 branch ;-) i understand what you are trying to achieve here, 
>> and my fear is that the rebase on ubuntu master might be quite 
>> problematic... well I just did the test now with the current branch 
>> states, and the rebase went quite fine (3 merge conflicts out of this 
>> huge patchset). But how do we manage updates now? Do you expect that we 
>> continue to put our patches in our tree, and you will apply them on top 
>> of 'UPDATE' or do you plan to generate UPDATE for each new patch set 
>> from us? Especially because OMAP3 support is managed in master and OMAP4 
>> in ti-omap4..
>>
>> Also this model fits well once the vendor is able to provide a BSP which 
>> is aligned with the kernel version which is in master which is just 
>> going to happen now for OMAP4. How do we proceed before (e.g. projecting 
>> to 11.04) while master is moving at a faster pace than ti branch?
>>
>> Regarding the linaro kernel, I don't think that might help too much. 
>> Linaro kernel is mostly based on mainline stuff and it does not get all 
>> the "fancy" stuff that we put in ti-omap4. at the some point of course 
>> most of these 1300+ patches might end up in mainline, but until there 
>> ti-omap4 branch and linaro branch will be a lot different ( i think the 
>> linaro does not even enable OMAP4 for now). So Linaro kernel should not 
>> blindly take the big 'UPDATE' and merge it until the code is close to 
>> get into mainline.  Finally our huge patch set includes patches outside 
>> of omap (ARM, network, and other). So when merging other ARM vendors 
>> with the TI patches you might get lots of conflicts, having individual 
>> patches as commits might help git rebase do a better job, also getting 
>> the granularity of the commit would ease the debug/bisect as you said.
>>
>> this is just a thought... in any case we will continue to provide you a 
>> branch with all TI patches that need to be pulled in ti-omap4. i recall 
>> that when we started to work together we had decided to maintain a 
>> branch that would look like: mainline + TI patches + Ubuntu sauce that 
>> could later become ti-omap4 branch.
>>
>>
> 
> Andy,
> 
> I totally understand your feeling about the TI L24.9 release tree, which
> contains several merging operations. That will cause the issue when we do
> rebasing on our Ubuntu master tree.
> 
> And the simply updating a large patch will make our Ubuntu tree much easier. But
> I still got some ideas about this method:
> 
> 1. We will lose the git log history about the big patches in our tree. Although
> the history is supposed to be kept in upstream (here is TI L24.9 branch), we
> can't make sure it won't change or lose. If we got some critical issue and we
> need bisect or checking the history information, we might cannot find any
> accurate log. Because we can't control upstream tree, but we can control ours.
> 
> 2. After we applying the large patch file, we are still supposed to get some
> rebasing conflicts when we rebase on our original/master. This won't save us
> much effort, moreover we need to look up upstream tree git logs to decide how to
> solve this conflicts. I guess that will be painful.
> 
> 3. For further TI releases, I believe they will be quite small compare to this
> one. It is not necessary to use this method, it will be easier to just apply
> those additional patches on our tip branch. Just for this first TI 2.6.35
> release, I think we got this merging and rebasing pain.
> 
> 4. As Nico mentioned here, it might be not very difficult to solve the merging
> and rebasing issue. So we can get a 2.6.35 + TI patches + Ubuntu master
> SAUCE/packaging stuff. After a short pain, we can get a tree in good shape. I
> think we can buy it.
> 
> But on the other hand, Nico. Is that possible for TI just release a pure patch
> stack style branch instead of merging from other dev trees?
> 
> Thanks,

I did some simple practice, it is not very difficult for us to handle the rebase.

$ git checkout master
$ git checkout -b testing
$ git rebase ti-release

I got 2 conflicts which are not very difficult for us to solve:
---
Applying: UBUNTU: SAUCE: omap: remove calls to usb_nop_xceiv_register from board
files
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging arch/arm/mach-omap2/board-4430sdp.c
CONFLICT (content): Merge conflict in arch/arm/mach-omap2/board-4430sdp.c
Auto-merging arch/arm/mach-omap2/board-omap3evm.c
Failed to merge in the changes.
Patch failed at 0121 UBUNTU: SAUCE: omap: remove calls to usb_nop_xceiv_register
from board files

Applying: UBUNTU: SAUCE: OMAP: DSS2: enable hsclk in dsi_pll_init for OMAP36XX
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
Auto-merging drivers/video/omap2/dss/dpi.c
CONFLICT (content): Merge conflict in drivers/video/omap2/dss/dpi.c
Failed to merge in the changes.
Patch failed at 0280 UBUNTU: SAUCE: OMAP: DSS2: enable hsclk in dsi_pll_init for
OMAP36XX
---

I pushed a branch [1] for you guys quick review and I still need add our
original -ti-omap4 Ubuntu stuff on top of it. [1] is rebased on [2].

-Bryan

[1]:
http://kernel.ubuntu.com/git?p=roc/ubuntu-maverick.git;a=shortlog;h=refs/heads/ti-release-rebasing-testing
[2]:
http://dev.omapzoom.org/?p=integration/kernel-omap4.git;a=shortlog;h=refs/heads/L24.9




More information about the kernel-team mailing list