Dulwich C extensions and stand-alone Windows installation of bzr

[Eli Zaretskii]

> From: Martin Pool <mbp at canonical.com>
> Date: Wed, 14 Sep 2011 15:04:44 +1000
> Cc: Alexander Belchenko <bialix at ukr.net>, bazaar at lists.canonical.com
> 
> > Jelmer suggested (off-list) to use this:
> >
> >  python -c "import dulwich._pack"
> >
> > and it does not error out.
> 
> That's great.  So then if you install those .pyd(?) files inside the
> selfcontained bzr install, you should be in business.  (There is some
> chance the compiler or Python is ABI-incompatible but it's worth a
> go.)

Well, it does work.  I've been using it for a couple of days with
several git repositories, and didn't see any problems.  The speedup is
moderate (around 30% to 40%), which is less than what I hoped for;
e.g., the initial branch from gnulib still takes around 1 hour.  But
it's better than 2 ;-)

I made sure that the *.pyd files are indeed loaded by bzr: first, with
a 3-line "plugin" that just imported all three of them, and then by
using the "Find handle" feature of the Process Explorer utility that
clearly shows that bzr.exe has them open when bzr-git is being used.

If there's a good place where I can upload these *.pyd files, perhaps
to the bzr-git page on Launchpad, where others could download them,
please tell me what to do.

Finally, for the record, I'm going to describe here how to build and
install these extensions using MinGW.  Most of the info came from the
advice I got here and from Python docs, but not all of it was correct
or optimal.  I would highly recommend to add what's below to the
dulwich distro, in some README.mingw file or something.

Here's the final recipe I came up with:

 1) Install Python.  It should be the precise version displayed by
    "bzr version", which is currently 2.6.6.  (I tried Python 2.7 at
    first, but bzr would not load the *.pyd files produced by that.)
    There's no link on the Python download page to version 2.6.6, but
    by looking around I found its installer here:

      http://www.python.org/ftp/python/2.6.6/python-2.6.6.msi

 2) Customize Python's distutils to use MinGW.  There are several ways
    of doing that, all of them documented on this page:

      http://docs.python.org/install/index.html#install-index

    The method which was suggested on this list, i.e. to use the
    "-c mingw32" switch to the setup.py script's "build" commands, is
    not the best one, because (a) you need to remember to type it
    every time, and (b) because only the "build" commands accept that
    switch, "install" does not.

    A better way is to add this to the distutils configuration file:

      [build]
      compiler = mingw32

    This can be added either to %HOME%\pydistutils.cfg file (assuming
    you have HOME defined) or to the Lib\distutils\distitils.cfg file
    under the main Python installation directory.  I did the former.
    If the configuration file does not exist, create it.

 3) Make sure to add Python's installation directory to Path.  Then go
    to the directory where you unpacked dulwich -- that's where the
    setup.py script lives -- and type these commands:

      python setup.py build
      python setup.py build_ext -i

    Alternatively, if you have a Make utility, you can type this one
    command:

      make

    Any of these 2 ways creates the 3 *.pyd files in this
    sub-directory of dulwich:

      build\lib.win32-2.6\dulwich

 4) Make sure Python can load the *.pyd files you produced:

      python -c "import dulwich._pack"
      python -c "import dulwich._diff_tree"
      python -c "import dulwich._objects"

 5) Copy those 3 *.pyd files from build\lib.win32-2.6\dulwich to your
    site-packages\dulwich sub-directory of the Bazaar installation.

 6) That's it!  You are set.

Last, but not least: I'd like to thank everyone here who helped by
information and advice.