documentation logo

Dustin Kirkland kirkland at
Fri Sep 5 15:05:40 UTC 2008

On Fri, Sep 5, 2008 at 2:00 AM, Matthew East <mdke at> wrote:
> In this case, there isn't any doubt that the utility is a useful one:
> it's a great idea as those who responded to your initial request for
> comments agreed. But there is one important question: why does this
> project warrant a separate website, as opposed to going directly into

The scripts generate 60,000+ unique HTML pages per distribution
(dapper, feisty, gusty, hardy, intrepid).  Note that I dropped Edgy
since its support is ended.

They also copy the raw, gzip'd manpages to a similar location, which
feeds into phase 2 of this project...  Patching man itself to
optionally download the manpage.N.gz from the
site, if it's not found on the local system.  I have a trivial shell
script wrapper for man, but long-term, I've discussed this thoroughly
with cjwatson, and he's going to help me work it into man itself.

We're talking about 300,000 unique HTML pages, plus 300,000 unique .gz
files, comprising approximately 3.4GB of generated data.

"Generated" being the key word.

The scripts that generate this content are best run in a confined
sandbox.  For one thing, they need local access (NFS-mounted, or some
such) to a full Ubuntu archive mirror.  And secondly, when they run,
they can really impact the performance of the server.  It needs 8-12
hours to generate the initial repository from scratch, and 20-30
minutes per night for the incremental updates.  At 20-30 minutes per
run, we can afford to run them nightly, and it's really nice to pick
up daily changes to the distribution under development (Intrepid,
currently).  This process is essentially a no-op on the stable
distributions (see the code!).

> Starting a new website under the domain has traditionally
> been a big deal in the Ubuntu community.

True, it certainly wasn't trivial!

I wrote most of the code for this in May, shortly after the mailing
list post and during UDS Prague.  I didn't follow up on the list after
my initial post since the reception was somewhat lukewarm, and I
rarely saw technical discussions taking place on ubuntu-doc at .

It was usable on my demo system by June.  Then it needed a security
audit, for which I thank Kees Cook.  (Big thanks to Kees and Jamie
Strandboge, by the way, for the audit, and some code, and generally,
some of the design ideas behind it.)  The security audit was done by
end of July, and it's taken Canonical IT a month or so to get the site
live (Big thanks to LaMont Jones for that!).  I'm working through the
remaining couple of minor bugs now, though it should be online for
production use by next week.  As I said, I'll blog about it and
announce here.

> I remember that when we
> wanted to create a wiki at and move documentation off
>, we spent several months working on the spec, took it
> to the Community Council, who rejected the idea. That was
> disappointing, but I ultimately understood that the reason was because
> we hadn't made a compelling enough case to do it, and they were
> rightly concerned about new websites confusing users and developers.

In the immediate term, there will be a redirect from that points to  I can
open a second IT ticket to get to act as a
reverse proxy, such that all of's content can be
transparently served directly through (with
no redirect).  I'll need to dig through some sites I did years ago to
get the apache conf syntax right, it's very tricky, put perhaps can be
done ;-)

> I'm not saying you need to do something similar here, but I do think
> you need to make a case for a separate website. For me, the manpages
> are documentation, and they should go on the same website as other the
> rest of Ubuntu documentation: Creating a new website
> means that help.u.c will no longer be the unified documentation
> resource that it was intended to be, and users will need to keep two
> sites in mind.

The overwhelming majority of manpages is written by upstream authors,
independent of Ubuntu.  I have a mild concern that some people could
be misled that the Ubuntu Community "wrote" the 300K manpages served.
That's clearly not the case, and very much a distinction from the rest
of the content hosted on and, which is
overwhelmingly written by the Ubuntu Community.

The documentation at is User-contributed.

The documentation at is Documentation-Team-contributed.

This documentation at is Generated from the
manpages already included in each and every .deb shipped in main,
universe, restricted, and multiverse.

To me, those are 3 very distinct forms of documentation.

Furthermore, I think there are far more forms of documentation spread
throughout the Ubuntu Community.  I've posted here before about the
Google Custom Search I've implemented for Ubuntu (yes, there are a
number of others out there):

I consider a wider scope of Ubuntu Documentation to include:
 * Formal Docs
  * Official Docs (
  * Wiki Docs (
  * Man Pages (  <------- N E W
  * Blueprints (
 *  User-level Discussion
  * Answers (
  * Forums (
  * Mailing Lists (
 * Developer-level Discussion
  * IRC Logs (
  * Bugs (
  * Code (

> You've driven this project, so you have a much better understanding of
> its needs than I do. So it may be that there are genuine reasons for a
> separate website with a new subdomain, which we don't know about yet!

Well, this wasn't intended to fork the documentation community in any
way.  Far from it :-)  I think you've stated above that you appreciate
the contribution--and I'm happy to work with anyone who would like to
contribute to the project I established in Launchpad, and in
particular with anyone who wans to help with "Phase 2" (the man

I think there are decent reasons why the code that generates this
belongs on a separate server, which we called out
of a lack of imagination ;-)  But I'd like to tie it to in any way that makes sense.


More information about the ubuntu-doc mailing list