Options for parallelizing smart server (Was: Re: multiple users at the same time on one repo...)

[Jan Hudec]

On Thu, Aug 24, 2006 at 01:53:38PM -0500, John Arbash Meinel wrote:
> Jan Hudec wrote:
> 
> ...
> 
> >> (3) is something that we are looking to implement for other performance
> >> benefits. It won't be in 0.10, it might be in 0.11. Though I don't know
> >> that it will support concurrency. I don't know the details of the
> >> current implementation, but it certainly could be a single process is
> >> spawned as part of every ssh connection.
> > 
> > Well, I can imagine this being implemented even for non-singleton
> > server, but it's quite a bit of programming.
> > 
> > Basically the idea is, that the repository lock could be somehow made a
> > shared-lock (this seems to me would be the hardest part).
> > 
> > Clients not capable of using fcntl lock would lock it exclusively and do
> > whatever they do now. This means clients using dumb transports and
> > clients accessing over network filesystems that don't support locking.
> 
> The hardest part is making clients using fcntl interact properly with
> clients that can't. Such that clients who are smart about it, realize
> that the branch/repository is locked in shared mode, but other clients
> would only think it was exclusive mode.

Yes, that is the hard part. Basically the lock would have to look as
normal lock, but have something special about it.

IIRC the lock is a directory containing a special file with the locker
info, right? Than a shared lock could be the same directory with one
extra subdirectory. Presence of this subdirectory would indicate a
shareable lock.

Every client wanting to share the lock would create a uniquely named
file in that directory (open with O_EXCL should be good enough).

Every client releasing the shared lock would remove it's file and try to
remove the directory. If that fails, there are more users and this
client is done. If it succeeds, this client is the last user and has to
remove the lock. During the clean-up the lock is exclusive, so no other
client can share it.

> > Clients capable of using them would lock shared and then fcntl lock
> > individual files they need to access. They would either need to lock one
> > knit+index at a time or lock them in defined order (to avoid deadlocks,
> > though posix locks are *supposed* to support deadlock detection).
> 
> This was an initial thought back with Weaves, and a way that we could
> properly support hardlinked knit files. The problem is that it
> distinguishes local branches from remote branches. Which is sort of
> okay, but at a higher level my local branch may be your remote branch,
> just that I'm directly logged into the shared machine.

I don't think it distinguishes local and remote branches. It
distinguishes local and remote access in that that local access can be
parially concurrent with other local access, while remote one can not.

> > They would have to wait for the knit locks to provide a sane semantics.
> > Waiting on the branch lock does not make sense, because we know we will
> > be out of date when it's released.
> > 
> > Than any local clients (which includes smart servers) would than be able
> > to commit to distinct branches in parallel.
> 
> Completely true, but hard to do if you want simple remote filesystem
> support. The sftp v6 draft looks very promising for something like this,
> since it supports OS-level locks over SFTP. But nobody uses a v6
> implementation of the spec. So in the mean time, we basically need a
> custom server that can use local semantics over a remote connection. And
> that is what Martin an Andrew Bennetts are working on.

My proposal is, that simple remote filesystem access would still lock
the whole repository against any other client, local or remote. It only
solves the case of multiple local clients so there may be several
smart-server proecesses accessing a single repository simultaneously.

Another option would be to have a smart server accepting commands on a
socket and cgi/ssh/... access to it would run a small tool, that would
try to connect to that socket and upon failing to do so would start a
smart server and try to connect again. On Unix a unix-domain socket
inside the repository lock would be preferable. I am not sure how to do
this on Windows.

The proposal with shared locking has the advantage that shared locks can
be used in future over sftp (when openssh implements v6 protocol) and
over webdav (apache dav module supports locks, but I am not sure they
are compatible with fcntl locks -- but even if they are not, they could
be made so one day).

The other proposal with sockets on the other hand seems to be easier to
implement. It's probably not less code, but it would be less tricky one.

--------------------------------------------------------------------------------
                  				- Jan Hudec `Bulb' <bulb at ucw.cz>