continuous integration/testing for python packages [Was: Is it worth back porting PEP 3147...]

[Barry Warsaw]

On Apr 27, 2010, at 10:01 AM, Nicolas Chauvat wrote:

>[discussion started at
>http://lists.debian.org/debian-python/2010/04/msg00046
>should we continue or trim some of the cc'ed lists ?]

Is there a better place to discuss Python packaging issues that are of
interest to both Debian and Ubuntu?  Are all Ubuntu developers who care about
shared Python concerns generally members of debian-python?  I don't know these
things. :)

>We are using http://www.logilab.org/project/apycot that is GPL
>software mainly developed and maintained by Logilab, but slowly
>reaching out to a larger audience.

There's also buildbot of course, which I guess it the granddad of Python CI
tools, kind of.

>It uses a web framework to store the information in a db and provide a
>web user interface, plus slave testing bots running on one or more
>hosts that get the next task from the queue, execute it and store the
>results in the db.

Are there things like a API (REST or otherwise) for pulling data out of
apycot?

>> What I like about your display is that a failure in one area does not
>> necessary mean a failure elsewhere.  That way you can better see the overall
>> health of the package.
>
>You may find interesting the following blog posts about apycot and
>ways to display its information http://bit.ly/9dZQQE
>
>> as nearly automatic and effortless packaging in Debian and Ubuntu.
>
>We tried fully automatic packaging of Python programs years (8?) ago
>and did not succeed for distutils and setuptools were too far away
>from Debian packaging concerns.
>
>Introducing in mypackage/__pkginfo__.py and mypackage/setup.py all the
>information needed to generate the debian/* files without the need to
>modify them eventually meant more or less copying their whole content,
>for their is actually not much to generate. It also meant using a less
>efficient toolchain because of the added conversion step.

I'm not familiar with __pkginfo__.py, but I do agree that setup.py makes
automated packaging difficult.  We need a declarative syntax that can be
consumed by more tools, which is why I'm so excited about Tarek's work in
distutils-sig.

>We moved to having tools that check the consistency of the information
>provided by __pkginfo__ and debian/* files and make it easier to build
>the Debian packages. These tools are
>http://www.logilab.org/project/logilab-devtools

Is there a wiki or online documentation documenting these tools, or is it all
in the source?

>Packaging a piece of Python software now requires a bit of (easy) work 
>at first, but following releases only need one or two commands. And
>all the dh_python* helper scripts reduced that work even further.

Is that easy work manual or automated?  What does it take to Debianize
random-simple-pypi-package?  (By that I mean "run a script" or "inspect
setup.py and write the debian/*" or "...?".

>> What I have in mind is defining a set of best practices, embodied as much as
>> possible in tools and libraries, that provide carrots to Python developers, so
>> that if they adhere to these best practices, the can get lots of benefits such
>> ...
>> It's things like 'python setup.py test' just working, and it has an
>> impact on PyPI, documentation, release management, etc.  These best
>> practices can be opinionated and simple.  If they cover only 80% of
>> Python packages, that's fine.  Developers would never be forced to
>> adhere to them, but it would be to their advantage to do so.
>
>Sounds good to me :)

Right now, it's just an idea.  There are a few existing attempts at this, so
it's worth looking at what exists first.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
Url : https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20100427/1979bb5f/attachment.pgp