[Natty] [ti-omap4] [Pull Request]: 2.6.38.2 based updates

[Bryan Wu]

On Thu, Apr 14, 2011 at 5:41 AM, Andy Whitcroft <apw at canonical.com> wrote:
> On Wed, Apr 13, 2011 at 07:51:00AM -0700, Bryan Wu wrote:
>> Tim,
>>
>> I've got latest updates from Sebastien Jan and Andy Green
>> Please find the branch here:
>>        git://kernel.ubuntu.com/roc/ubuntu-natty.git ti-omap4
>>
>> It's based on latest 2.8.38.2 Linaro release and it is also contains
>> some updates since previous rebasing:
>>   * config updates to fix some booting issues like oops and warnings
>>   * fixed a tracing conflict due LTTng merger in Linaro kernel
>>   * Rebased to 2.6.38.2 based Linaro kernel release
>>   * Updates WLAN/BT/FM/Audio supporting from Andy Green and Sebastien Jan
>>     (u0 tag of for-ubuntu branch in Andy's tree)
>>   * Rebased to Ubuntu master branch from 2.6.38-7.38 to 2.6.38-8.40
>

Thanks Andy for taking almost one day to hep this!

> I have been looking at the ti-omap4 branch we are using, as we would
> prefer for this to be a rebasable branch as-per all of the other
> derivative branches.  However this branch is based off of the Linaro
> mainline branches which I suspect will render this very difficult indeed
> if we wish to maintain any of the history intact (more on this later).
>
> First a little history for completeness.  When Linaro started up we had
> some discussions with them on how they might maintain their kernel trees.
> We did discuss the issues that using a non-rebase model would produce
> downstream, partically for the distro.  The Linaro tree still did end up
> being a merge based tree as it simplifies the upstream management of the
> tree (aiui), and that is their primary focus.
>
> The Linaro v2.6.38.2 base that was used for the currently proposed pull
> request is actually a very different tree to mainline v2.6.38.2.  If I have
> done my maths right there are some 2k commits from v2.6.38.2 to the tip.
> This is in large part due to the large portions of v2.6.39-rc code which
> the updates pull in for arm to get their required functionality, this
> includes tracing updates etc hitting things all over the tree (not just
> in arm/).
>
> As the linaro base tree and its sources are all merge trees there is no
> single guarenteed patch application order for the patches it represents.
> This means that any attempt to flatten this 2k patch stream into a linear
> rebasable chain is very, very likely to throw up essentially false merge
> conflicts which have to be resolved and these in turn engender inverse
> conflicts later in the stack.  With so very many patches and worse so many
> merges this is increasingly likely to occur, and I predict this will make
> it near impossible to convert it into a rebase tree in its current form.
>

Exactly, for Linaro release we found it merges several branches from
upstream and stable tree, while for TI OMAP4 patches (come from Sebjan
and Andy Green) are all linear I think.

> There seem to be two viable approaches (to my eye), though I am fully
> open to other ideas:
>
> 1) squash the delta into a 'omap4' patch and carry that, or
> 2) accept that this is a merge based tree and develop new handling for
>   this kind of tree.
>
> For (1) we would essentially do a 'git diff v2.6.38.2..linaro' and apply
> the non-ubuntu, non-debian part of it as a single patch against the master
> branch.  This would then be semantically a rebaseable tree against master.
> The upside here is that its handling would be indenticle to that of the
> other derivative branches.  The downside is that we would lose all the
> history and would struggle when rebasing should the delta patch fail to
> apply at any time as it will be a humungous patch.  We would also have
> a harder time working out if there are any additional changes in the
> linaro tree needing pulling in (assuming there is any maintenance coming
> from linaro).
>

Right, this is a quick fix and we got a rebase tree, but we will lose
history and might be difficult for bisect in the future. Also we might
got conflicts for cherry pick patches from Linaro or stable or SRU,
that will be very painful to fix those conflicts since we wouldn't
have the history.

> For (2) we would have to accept that this branch type is different,
> that it is not a rebase tree.  Maintenance of this branch would be via
> merges _from_ the master branch.  That is we would work master as normal,
> but to gain the benefits of those changes in the ti-omap4 branch we would
> merge the latest master release tag into the tip of ti-omap4; rather than
> rebase ti-omap4.  The newly merged tip would be packaged up and released
> as normal.  The upside here is that we would be able to merge both from
> linaro to get any fixes from them, and also from master to gain security
> and other goodness.  The downside is the proceedure for doing the updates
> to this branch would be different.
>

Actually, this is exactly what I want. And since the difficulty of
changing current ti-omap4 to a rebasable tree, this method is much
easier for us to manage and update in the future.

But I don't use merge, because I wanna keep the master Ubuntu delta
together for this specific 1208.11 release.
I git format-patch to get all master patches since 2.6.38.2 stable,
since our master is rebasable. I will get maybe several hundreds
patches.
I git am them on top of "for-ubuntu" branch from Andy Green, which
contains TI patches based on Linaro 2.6.38.2.
Then add all ti-omap4 specific Ubuntu things on top of it. So current
tag of master is 2.6.38-8.42, I've pushed it to
http://kernel.ubuntu.com/git?p=roc/ubuntu-natty.git;a=shortlog;h=refs/heads/ti-omap4-dev
And for sure, I fixed some conflicts.

Since this updates is kind of replace the ti-omap4 branch since
1207.10 release, I put master Ubuntu delta together. After 1207.10 is
uploaded, I will just git am further master patches, TI OMAP4 patches
and Linaro patches on top of 1207.10 release, also pack them up.

Instead of merging, I think git am can keep the same linear order and
patches info as master.

> I personally think that with a little work option (2) could work for us,
> although slighly different, the actual amount of work required to perform
> the merge should be similar in order and complexity as that required
> for a rebase.  As I suspect we are going to get more of these derivative
> branches based on Linaro bases going forward given the popularity of ARM
> and the long lead times getting ARM into a more generic form (which seems
> is still 1-2 years out), I suspect having a simple way to maintain them
> will be beneficial.
>
> I think that we should use this oppotunity to trialing method (2) for
> this branch.  Should this turn out to be a mistake for any reason, there
> should be little barrier to converting it to method (1) in the future;
> the effort to convert is minimal and does not become more complex for a
> trial of (2).
>
> In order to test the viability of method (2) I have played about with the
> current ti-omap4 branch as I think that the current rebasing of the Ubuntu
> delta onto the top of the linaro branch simply gains us nothing and indeed
> is rather confusing.  As the linaro tree is already a merge based tree,
> I think it makes much more sense to simply use git merging to integrate
> the ubuntu delta from the ubuntu master branch.  By merging master now,
> we can then use merge to later pull in changes from the master branch.
> To demonstrate this I have rebuilt a new ti-omap4 branch which would be
> more suitable for method (2) above.  I have since then experimentally
> merged it up to 2.6.38-8.41 and -8.42 to test the viability of this
> mechanism.  So far it is looking pretty workable.  I have pushed this
> newly merged branch up as below:
>
>    git://kernel.ubuntu.com/apw/ubuntu-natty ti-omap4
>

That's great, I think it is also workable for me.

> In summary, if we think we are going to have more of these Linaro based
> branches I think we should seriously consider having a methodology for
> handling these, accepting that we may have to handle two different types
> of branch in the short term.  I think the handling of them can be simple,
> though different from our current branches.
>

I'd love to maintain this difference and work on SRU and CVE updates
for ti-omap4 branch of Natty.

> I am happy to document this methodology if we are going this route.
> Likely an in tree marker to make it easy to tell is appropriate.
>
> Comments?
>
> -apw
>

Thanks a lot,
-- 
Bryan Wu <bryan.wu at canonical.com>
Kernel Developer    +86.138-1617-6545 Mobile
Ubuntu Kernel Team
Canonical Ltd.      www.canonical.com
Ubuntu - Linux for human beings | www.ubuntu.com