Yelp Frontpage / Common Questions (was Re: What are we doing wrong?)

[Shaun McCance]

On Wed, 2010-01-20 at 08:22 +0000, Matthew East wrote:
> Hi,
> 
> On Sun, Jan 17, 2010 at 6:36 PM, Dylan McCall <dylanmccall at gmail.com> wrote:
> > Oh, Yelp is also insanely slow and a source of many support questions
> > itself when it starts eating every resource it can find and spitting out
> > nothing in return. A big distro needs to commit serious technical
> > resources to this. There is work going on to port Yelp to use Webkit
> > (which will help)
> 
> At present, I'm not aware of anyone working on yelp development from
> the Ubuntu community, and upstream has only one person working on it.
> I agree that yelp's startup and search speed is a big problem that we
> face in getting users to read documentation and would simply encourage
> anyone with the relevant coding skills who is interested to get in
> touch with upstream and with us - it really would be an exceptionally
> cool contribution.

So there are two things to look at when talking about speed:
1) How long it takes Yelp to start up, and
2) How long it takes Yelp to transform a document.

For (1), we unfortunately had some regressions a few releases
back because of eagerness to kill ScrollKeeper.  Long long ago,
I worked pretty hard to ensure that Yelp doesn't do any sort
of blocking operations on startup.  So stuff like finding all
installed documentation happened in an idle loop.  With the
switch to Rarian, finding installed documentation now blocks
Yelp startup.  This is a problem, and it will be fixed.

Note also that in 3.0, the default front page of Yelp will be
the Desktop Help, which I think is closer to what Ubuntu has
already patched Yelp to do anyway.  So it doesn't really make
sense to block on the list of installed documentation when
you don't even see that page by default.

For (2), if you load up a large document (the Gnumeric manual
is always a good test case), you'll notice that you can get
the front page pretty quickly.  But if you immediately click
to the last page, you'll have to wait.  This is a big problem
if you click a button in the program that takes you directly
to a later page.  To address this, I want to try to process
DocBook chunks on demand.  I'd like to have this in 3.0.

We also have to consider Mallard documents for (2).  Mallard
documents are always processing one page at a time, so that
particular source of problems is gone.  But Yelp does have
to read all the pages to know what links to what.  Lots of
small file reads is never a good idea.  So I want to try
installing Mallard Cache files along with the pages.  This
is not all that difficult.  It just takes a bit of hacking
time on my part.

As for search, Yelp has two backends: one using Beagle, and
one that's basically a glorified grep.  The grep backend is
going to thrash your disk.  I don't know which one is used
by default on Ubuntu.  But Beagle is a dead project, so we
need to look elsewhere for Yelp's future.  I've spoken to
some Tracker developers, and I'd like to work on using it
for Yelp's search backend.  Again, hacking time.


-- 
Shaun McCance
http://syllogist.net/