<div class="gmail_quote">On Tue, Mar 30, 2010 at 8:26 PM, Ian Clatworthy <<a href="mailto:ian.clatworthy@canonical.com">ian.clatworthy@canonical.com</a>> wrote: <blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Hi all, After I wrap up improving the Windows installer build process, I'm hoping to get more involved in Loggerhead development and maintenance. I'm sending out this email now though because I want to raise some topics for debate/discussion. At this point, I expect my immediate priorities to be "make it faster and get it easier to deploy", i.e.: 1. Help review and land the dozen or so proposed branches. 2. Improve packaging so that Loggerhead becomes a standard part of our Windows (and OS X hopefully) installer. I have some ideas on how to make Loggerhead more beautiful and more functional but they can come later. :-) There's some *great* work going on in this space right now. In particular, John Arbash Meinel and Kapil Thangavelu have put together some branches that speed things up considerably. IIUIC, John's changes leverage recent improvements to bzrlib so, apart from maintaining backwards compatibility to earlier bzr versions, they are relatively straight forward. Kapil's branch[1] uses chameleon[2] templates instead of simpleTAL. As such, we need to discuss whether that's the best direction or not before we can land his changes. More broadly, I'd like to get some feedback on the current technology stack under Loggerhead (simpleTAL, simplejson and Paste). In terms of templating languages, both Chameleon and Jinja2[3] are supposedly much faster than simpleTAL. I also understand that Ian Bicking is refactoring the core of Paste into WebOj. So looking forward, some options would be: * switch to Jinja2 + simplejson + WebOj * switch to repoze.bfg (which wraps WebOj and supports both Chameleon and Jinja2). If we made such a switch, I'd suggest we keep the 1.17 branch on the current technology stack and start a "2.0" branch on the new stack. Thoughts? </blockquote><div> Its really not clear to me what value for loggerhead, which is pretty close to barebones wsgi, adopting a framework brings for the goals of making it fast and packaged? As for speed the ideal scenario are figuring out how to make the bzr web browsing experience web speedy for large repositories and effectively O(1) wrt to history. Towards a goal of making the web experience faster, most of the gains will be from reexamining the cache usage and its structure. On the chameleon branch, i've experimented with some fairly trivial changes like the following two which remove the history cache usage from the file diff and revision ui respectively, and result in significant (~3x) performance improvements on request times. <a href="http://bazaar.launchpad.net/~hazmat/loggerhead/speed-rules/revision/408">http://bazaar.launchpad.net/~hazmat/loggerhead/speed-rules/revision/408</a> <a href="http://bazaar.launchpad.net/~hazmat/loggerhead/speed-rules/revision/409">http://bazaar.launchpad.net/~hazmat/loggerhead/speed-rules/revision/409</a> (this one might be a bit more controversial, but the view was previously walking back 10k revisions against the launchpad history and then discarding the result, for no apparent reason). Examining the views one by one and figuring out what they need access to, and to what extent they need to utilize the cache is probably a good first step in this direction. John Meinel's work on the branch is a great step forward in that it updates the cache generation to modern bzr apis and minimizes the time to generate the whole history cache. But even loading cache is relatively expensive, and it seems like its fairly common for views to calculate data from it without using the full result. John mentioned he had some ideas on a new cache structure that might be more efficient which would be great to see. There's also a bit of bit rot i think in the loggerhead code, for example it wasn't clear to me that the directory_ui view was used anywhere. I think if your going to go for 2.0, it might be worth dropping features or at least not making the common default views nesc. depend on them (ie. annotated default file view vs. just a file view). If i were going from the ground up, i'd perhaps even consider dropping revnos from the ui completely which will remove much of the need for a cache. I know it probably won't fly, but i think its worth consideration. i tried it out using trac-bzr as a guinea pig (and also sans the svn directory style change propogration) and it definitely validates at least for myself that fast bzr browsing with O(1) wrt to history and sans cache is possible. Regarding template engines, both jinja2 and chameleon, basically take their respective template languages (django and zope page templates) and compile that out to python code for execution. Given the existing template implementation i'd probably just continue with zpt/chameleon. wrt. to packaging, with a switchout of simpletal, loggerhead is effectively just an easy_install/pip installation away. Not that it obviates the need for binary installers, but its nice for those who don't use them ;-) cheers, Kapil </div></div>