Loggerhead reviews + broad plan

[Max Kanat-Alexander]

On 04/11/2010 04:35 PM, Ian Clatworthy wrote:
> My understanding is that it's basically a swap, give or take some minor
> changes. See
> https://code.edge.launchpad.net/~hazmat/loggerhead/speed-rules for
> Kapil's changes that I'll be reviewing.

	Great!

> Can you elaborate on your thoughts wrt horizontal scaling?

	Sure. There's basically two ways to do it, that I was thinking of:

	1) Create a proxy that sits in front of loggerhead and always has
certain machines handle certain branches. This is nice and easy for us,
relatively--I think it would be mostly changes to launchpad-loggerhead,
and no (or very few) required changes to loggerhead itself. However, it
doesn't handle the problem of one very-popular, very-large branch (which
was where I saw some of the major issues on codebrowse). It seems like
this solution would require some ongoing maintenance from the sysadmin
side of things, unless we came up with a very clever way to
automatically decide which branches should be accessed via which servers.

	2) Store loggerhead's caches in a distributed cache system, perhaps a
NoSQL database or memcached, and then simply put a session-based load
balancer in front of a bunch of machines running loggerhead, all
pointing to the same data. This is mildly complex for us because we'd
have to deal with race conditions in populating the cache. This is
probably a better solution architecturally, because it allows
more-or-less infinite scaling, even for a single very-popular branch.
It's also probably less ongoing maintenance work from the sysadmin side
of things, except that they'd also have to set up the distributed cache
server.

	We could also combine #2 and #1, although I'm not sure what advantages
that would give us.

	Although come to think of it, I can't quite remember why we couldn't
just put a session-based load balancer in front of loggerhead right now
(other than that they'd all create their own caches). There's some
reason, but I can't remember what it was.

> It's a separate plugin. See
> https://lists.ubuntu.com/archives/bazaar/2010q2/068081.html

	Cool! That sounds great!! :-)

	-Max
-- 
Max Kanat-Alexander
Chief Engineer
http://www.everythingsolved.com/
Everything Solved: Complete Computer Management