Loggerhead directions

[Michael Hudson]

On 31/03/10 13:26, Ian Clatworthy wrote:
> Hi all,
>
> After I wrap up improving the Windows installer build process, I'm
> hoping to get more involved in Loggerhead development and maintenance.
> I'm sending out this email now though because I want to raise some
> topics for debate/discussion.

Yay.  I've been a pretty terrible maintainer for loggerhead over the 
last few months...

> At this point, I expect my immediate priorities to be "make it faster
> and get it easier to deploy", i.e.:
>
> 1. Help review and land the dozen or so proposed branches.
> 2. Improve packaging so that Loggerhead becomes a standard part of our
> Windows (and OS X hopefully) installer.

Those both sound great.

> I have some ideas on how to make Loggerhead more beautiful and more
> functional but they can come later. :-)
>
> There's some *great* work going on in this space right now. In
> particular, John Arbash Meinel and Kapil Thangavelu have put together
> some branches that speed things up considerably. IIUIC, John's changes
> leverage recent improvements to bzrlib so, apart from maintaining
> backwards compatibility to earlier bzr versions, they are relatively
> straight forward.

Yes.

> Kapil's branch[1] uses chameleon[2] templates instead
> of simpleTAL. As such, we need to discuss whether that's the best
> direction or not before we can land his changes.

chameleon is just a different implementation of (more or less) the same 
templating language as is already being used, so switching to that isn't 
really a big deal.

> More broadly, I'd like to get some feedback on the current technology
> stack under Loggerhead (simpleTAL, simplejson and Paste).

Loggerhead is /mostly/ implemented as a straight WSGI app, so in some 
sense there isn't really much investment in a technology stack.  Paste 
is used for some generic WSGI helpers that would work in any WSGI app 
and the server/container -- again any server should work.  The 
templating is done almost by hand -- and there's some work to enable 
streaming output, which it would be a bit unfortunate to lose by 
switching to something more generic.

I don't think using simplejson is very controversial?

We use a bit more paste-specific stuff on Launchpad (see 
http://bazaar.launchpad.net/~launchpad-pqm/launchpad-loggerhead/devel/files 
)

> In terms of
> templating languages, both Chameleon and Jinja2[3] are supposedly much
> faster than simpleTAL.

Chameleon implements a much more similar language than jinja2 does, so 
switching to that sounds easy really.  I have this irrational hatred of 
non-tree based templating engines like jinja2, but I can probably get 
over that.

simpleTAL was so much faster than the kid that was being used before 
that I didn't even consider looking for anything faster -- if chameleon 
really is 10x faster, then I'd say switch to that and stop worrying 
about the templating side.

> I also understand that Ian Bicking is refactoring
> the core of Paste into WebOj. So looking forward, some options would be:

WebOb does seem a bit better (and actually has releases etc) than paste.

> * switch to Jinja2 + simplejson + WebOj
> * switch to repoze.bfg (which wraps WebOj and supports both
> Chameleon and Jinja2).

I don't know if you need anything as elaborate as repoze.bfg -- but the 
repoze site seems to be down now, so maybe it's appropriate.

> If we made such a switch, I'd suggest we keep the 1.17 branch on the
> current technology stack and start a "2.0" branch on the new stack.
>
> Thoughts?

I think a new "2.0" (or even "2.2" or whatever) series makes sense.

But, mainly: good luck!

Cheers,
mwh