arm64 Debian/Ubuntu port image available

Thu Feb 28 17:17:08 UTC 2013

Nice work, Wookey!  If experience cross-building for armhf is any guide,
all you need for NSS is a host build of shlibsign; see
https://github.com/mkedwards/crosstool-ng/blob/master/patches/nss/3.12.10/0001-Modify-shlibsign-wrapper-for-cross-compilation.patch.
 There's also scriptage in that repo for the build sequence and cross
parameters:
https://github.com/mkedwards/crosstool-ng/blob/master/scripts/build/cross_me_harder/510-nss.sh.
It's ugly in places (cross pkgconfig was kind of wonky at the time) but may
help you get past NSS and on to apt.

Cheers,
- Michael
On Feb 26, 2013 6:11 PM, "Wookey" <wookey at wookware.org> wrote:

> State of the Debian/Ubuntu arm64 port
> =====================================
>
> *** Arm64 lives! ***
>
> Executive summary
> -----------------
>
>  * There is now a bootable (raring) image to download and run
>  * Everything has been rebuilt against glibc 2.17 so it works
>  * A bit more work is needed to make the rootfs useable as a native buildd
>  * Multiarch crossbuilding and the build-profile mechanism is mature
> enough to cross-build
>     a port from scratch (this is a big deal IMHO)
>  * All packages, sources and tools are in a public repo and this work
> should be reproducible.
>  * This image is fully multiarched so co-installing armhf for a
>     64/32 mix should work nicely, as should multiarch crossbuilding to
>     legacy x86 architectures. :-) (but I haven't tried that yet...)
>
>
>  * Linaro wants 'the distros' to take this work forward from here so
> people interested in
>     Debian and Ubuntu on 64-bit arm hardware need to step up and help out.
>
>
> Bootable images
> ---------------
>
> A milestone was reached this week: Enough packages were built for arm64 to
> debootstrap an
> image which booted to a prompt! After a bit of fettling (and switching to
> multistrap) I got
> an image with all the packages configured which boots with upstart to a
> login prompt (I
> admit, I did get quite excited about this, as it represents the coming
> together of nearly 3
> years work on multiarch, crossbuilding, bootstrapping, cyclic dependencies
> and arm64 :-)
>
> The images are available for download:
> http://wiki.debian.org/Arm64Port#Pre-built_Rootfs
> And there are destructions there for making your own.
>
> All these packages were cross-built on raring, untangling cyclic
> dependencies with build
> profiles (see wiki.debian.org/DebianBootstrap for how that works), making
> this the first
> (non x86) self-bootstrapped debian port ever (so far as I know). All (?)
> previous ports have
> been done using something else like OpenEmbedded (armel, armhf),
> RedHat/HardHat (arm, alpha,
> mips), something IBMy (s390) to get an initial linux rootfs on which
> debian packages are
> built.
>
> The new bootstrap process is (almost) just a list of sbuild commands. In
> practice there are
> still a few rough edges around cross- build-dependencies so of the 140
> packages needed for
> the bootstrap, 9 had to be built manually with 'dpkg-buildpackage -aarm64
> -d' (to skip
> build-dep checks) instead of 'sbuild --host arm64 <package>'.
>
> The current bootstrap packageset status is here:
>
> http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
>
> There is no armv8 (arm64/aarch64) hardware available yet, so this image
> can currently only
> be run in a model. ARM provide a free-beer prorietary 'Foundation model'
> so we do have
> someting to test with. It's sluggish but perfectly useable. Booting the
> images takes a
> couple of minutes on my fairly average machine.
>
> The images are using the Linaro OE release kernels which seem to work fine
> for this purpose.
> Thanks to Marcin for modified bootloader lines in .axf files.
>
>
>
> Image status
> ------------
>
> I was impressed that things basically 'just worked' on first boot. There
> is of course plenty
> of breakage, I'm sure, and I haven't looked very hard yet, but it's a lot
> better than I
> expected after months of just building stuff and testing nothing. (Things
> that are poorly:
> nano can't parse it's own syntax-coluring files for example, and multiarch
> perl has the
> wrong @INC path compiled in; I'm sure there is more). Consider this
> alpha-grade until it's
> been used a bit more.
>
> Things that are not yet built which would make the images a lot more
> useful are apt and a
> dhcp client. apt needs gnupg needs curl needs nss. The nss cross-build
> needs fixing to
> unbung that. A debian chroot without apt turns out to be disappointing
> quite quickly :-)
> Expect an updated image with more packages very soon.
>
>
> Multiarch crossbuilding
> -----------------------
>
> It's really nice to have building and crossbuilding using exactly the same
> mechanisms
> and tools, with all files having one canonical location, and dependency
> mechanisms that
> are reliable. The more I've used this, the more I've been impressed by it.
> There is
> still work to do to expand the set of cross-buildable stuff, but it's a
> solid base to
> work from.
>
> Getting this port working has been 'interesting' because it's attempting 4
> new things all at
> once: multiarch (file layouts and dependencies), crossbuilding (tools and
> packaging support
> in a distro that historically was always natively built), arm64 (aarch64)
> support in
> packages that need it, and build-profiles to linearise the build-order.
>
> The arm64 part of this is a relatively small part as the heavy lifting has
> been done
> upstream (gcc, (e)glibc, binutils, kernel, libffi, autotools and a lot of
> minor fixes in
> various packages). Thanks are due to doko (Matthias Klose) for sterling
> work getting all
> that integrated into the debian and ubuntu toolchain packages, and
> infinity (Adam Conrad)
> for merging various eglibc branches. There were also hordes of very boring
> patches of the
> form 'update config.sub and guess before building'.
>
> Most of the work has been in making things cross-build (exactly the same
> fixes needed for
> armel/hf too so I've had plenty of help there from canonical types who
> want cross-building
> for arm to work nicely), and particular thinks to Neil Williams for taking
> on the perl
> cross-build challenge and creating the debian-perl-cross package to manage
> the
> cross-configury, whilst also working with upstream to make the whole thing
> a bit less 1996.
>
> Multiarchifying has been going on nicely in libraries and -dev packages,
> but things like
> perl and python needed significant work, along with a lot of boring bugs
> saying 'mark this
> package MA: foreign' and 'build-dep on python:any or perl-base:any'.
> Thanks are due to doko
> for the python multiarching and Niko Tyni for the perl multiarchification.
> Getting all 3
> 'aspects' of multiarch perl, cross-built perl and arm64 perl config to
> work at the same time
> was quite hard work, and there are still bugs there. Wider usage of
> multiarched perl would
> no doubt sort this out reasonably quickly. I started a wiki page to track
> the status of
> multiarched cross-buildable perl: http://wiki.debian.org/Multiarch/Perl .
> Help would be
> welcome.
>
> The build-profile work is described on the
> http://wiki.debian.org/DebianBootstrap page.
> Progress has been greatly helped by GSOC projects last year, with good
> work on the tools
> (crossbuild-essential packages, build-profile support) from P.J McDermott
> and an impressive
> contribution from Johannes Schauer on dependency analysis tools around
> libdose, and apt
> build-profile support.
>
>
> All of this apart from multiarch perl, crossbuildable perl and
> build-profile stuff (and
> a few pending patches) is already in raring.
>
>
> Building stuff yourself
> -----------------------
>
> Setting up an arm64 build environment is very simple. Use
> sbuild-createchroot or mk-sbuild
> and point at the bootstrap repo, with a bit of config and some updated
> tools packages from
> the repo (amd64 only supplied). Details are given on
> https://wiki.linaro.org/Platform/DevPlatform/CrossCompile/arm64bootstrap
>
> Once you've created a tarball chroot builds are simply done with
> sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package.dsc> or
> sbuild -c quantal-amd64-sbuild -d quantal --host=arm64 <package>_<version>
>  (I'd love it
>  if sbuild got smart enough to work out the version itself when given a
> distro - Roger
> says he's working on it)
>
> To deal with the chore of 'find version, run sbuild, sign result, upload
> to repo, import to
> repo, deal with reprepro bitching if you re-upload the same version of
> something' for every
> package build, I wrote 'dimstrap' which is a simple-minded tool to wrap
> that up and either do
> one-off builds or run through a list. It is part of the xbuilder package
> here:
> https://launchpad.net/~linaro-foundations/+archive/cross-build-tools/ It
> also includes the
> logfile-parsing script ('generate html') which generates the nice status
> pages:
> http://people.linaro.org/~wookey/buildd/raring-arm64/status-bootstrap.html
>
>
> Image building
> --------------
>
> The config and instructions provided (in
> http://wiki.debian.org/Arm64Port#Building_your_own_rootfs_image ) is
> for multistrap. Debootstrap sort-of produces working images too but
> takes a lot longer to unpack/configure, and misses out various vital
> packages (like libperl5.14). I'm sure it could be kicked into
> submission. In theory multistrap (apt really) should have got all the
> arch all packages from the main repo, but in practice it refused to do
> that so I had to rebuild them or copy them over anyway (grumble).
>
> Any package that installs replaced conffiles seems to generate invalid
> dpkg status entries (ifupdown did this to me). I've not got to the
> bottom of that yet. Deleting the offending line gets you an image that
> works.
>
> Issues
> ------
>
> General:
>
> The build-profile patches for dpkg and apt need to be pushed into the
> distro to make
> that feature permanent. A thread on debian-devel is working on that
> (
> http://debian.2.n7.nabble.com/Bootstrappable-Debian-proposal-of-needed-changes-td2848200.html
> ).
> The main issue is what syntax to use '<>' or '[]' and how to deal with
> multiple overlapping
> profiles. The patches to debian control cannot go in until at least the
> syntax is agreed and
> the tools will parse them without barfing. Johannes ands I will send an
> updated spec
> soonish.
>
> The missing piece of bootstrapping with regard to build-deps is packages
> that build-dep on
> gcc-4.6 or binutils. When cross-building this should be satisfied by
> <triplet>-gcc-4.6 or
> <triplet>-binutils. Nothing makes that happen currently. A scheme has been
> mooted but
> nothing is implemented yet.
>
> There is debate about whether cross-toolchains should build against
> multiarch libraries
> (libgcc, libstdc++) like everything else, or have their own internal
> copies. Doko and I
> disagree on this matter. That will need to be worked out at some point.
>
> We won't get that much further with fixing cross- object-introspection,
> which is a
> non-trivial job.
>
> Image-related:
>
> The images do essentially work but very little has been tested so far.
>
> Multiarch perl still needs work.
>
> nss needs cross-building in order to get apt cross-built
>
> I've not got networking working yet. Info is here:
>
> https://fedoraproject.org/wiki/Architectures/ARM/AArch64/FoundationModel_Networking
> lack of a dhcp client in the image hasn't helped there.
>
>
> More info
> ---------
>
> The canonical arm64 port info page is:
> http://wiki.debian.org/Arm64Port
>
> Full arm64 cross-build status (i.e everything that has been tried) is here:
> http://people.linaro.org/~wookey/buildd/raring-arm64/status.html
>
> All the patches generated so far are here:
> http://people.debian.org/~wookey/bootstrap/patches/
>
> (most that can, have been filed as bugs - there is a backlog of stuff
> filed in Launchpad but not yet forwarded to the Debian BTS - yes I am
> a bad boy - blame the fact that you can't use reportbug or bts from
> inside ARM due to their idiotic email policies).
>
>
> Future work
> -----------
>
> Firstly we should say thank you to Linaro for sponsoring this work in
> various ways over
> the last 3 years. We wouldn't be at this point now if it wasn't for that.
> However
> Linaro has a lot of things to do and is trying hard not to do distro's
> work for them,
> concentrating on upstream things. This makes sense for commercially-backed
> distros like
> Red Hat and Ubuntu, but rather less for Debian where we _are_ the distro
> just as much
> as anyone else is, and ultimately someone has to spend the time to get
> stuff working.
>
> Anyway, I was supposed to stop work on this some time back, but have
> largely failed to
> do so (cross-building is so moreish - there is always one more build to
> try before
> bedtime!) and appreciate being given enough slack to get this to a point
> of actual
> utility. However I expect to have much less time to spend on this from now
> on, except
> insofar as it still co-oncides with things Linaro wants doing. I'd love to
> hear from
> people who actually want to use this, to get more packages built, the
> Debian
> cross-toolchains sorted, build-profiles finalised, and a whole pile of
> stuff fixed once
> Wheezy is released. I'm pretty sure there are quite a lot of people who
> want multiarch
> Debian or Ubuntu on their arm64 machines (or models).
>
> I hear rumours that actual hardware may appear sometime around the middle
> of the year
> with some bagsied for Debian. Setting up the ports infrastructure for that
> would be
> good. I don't know if anyone is interested in building slowly on models in
> the
> meantime, or if we should just carry on crossing and see how far we get.
> This table
> shows that 471 packages in raring can be expected to cross-build already:
> http://people.canonical.com/~cjwatson/cross/armhf/raring/
>
> Todo:
>
> Fix up multiarch/cross perl
> Fix nss
> Build missing packages for apt
> Build missing packages for build-essential
> Build Debian cross-toolchain
> Get all this working in unstable as well as raring
> Setup buildds
> Build all the other packages
> Set up automated bootstraping runs (eventually)
>
>
> Current setup
> -------------
>
> Builds have all been run locally using the sbuild/chroot setup described
> above and on
> the Arm64Port page, which should be easy for anyone to reproduce. The main
> irritation
> is keeping up with raring: out of sync libraries are not MA-installable.
>  Logs are
> uploaded to people.linaro.org (rsync). The reprepro repo is on
> people.debian.org
> (dupload).  This stuff should probably move to ports.debian.org and
> ports.ubuntu.com,
> but neither of those are set up for cross-building so I'm not quite sure
> how this will
> work.
>
> I could go on at great length about the machinery of profiled bootstrap
> builds, and
> interactions between tools, but it's not very exciting, so will resist.
> Suffice it to
> say that whilst it's all pretty slick I'd still like better buildd tools.
>
>
> Build-profile changes
> ---------------------
>
> The build-profile patches are not yet upstreamable so are collecting in
> the repo.
> The patch set so far is here:
> http://people.debian.org/~wookey/bootstrap/patches/profiles/packages/
>
>
> Other thanks:
> Other people who have helped make this happen in various ways but not got
> a mention above:
> Colin Watson, Dmitry Ledkovs, Steve Langasek, Harry Leibel, Thibaut Girka,
> Roger Leigh,
> Marcus Shawcroft, James Morrisey, Jonathan Austin, Steve McIntyre, Peter
> Pearse, Aurelien
> Jarno, and whoever does sysadmin at people.{linaro,debian}.org
>
> I hope I didn't forget anyone, or any important information.
>
> Feedback from anyone attempting to get this working outside my computer is
> very
> welcome. I have almost certainly forgotten to write down some things, and
> upload
> correct versions of some other things.
>
>
> Wookey
> --
> Principal hats:  Linaro, Emdebian, Wookware, Balloonboard, ARM
> http://wookware.org/
>
> _______________________________________________
> cross-distro mailing list
> cross-distro at lists.linaro.org
> http://lists.linaro.org/mailman/listinfo/cross-distro
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20130228/72637a63/attachment-0001.html>