help.ubuntu.com and Google Custom Search Engines

[Doug Smythies]

This e-mail desires responses from old timers, and in particular from Dustin Kirkland.

This e-mail is similar to one Gunnar just sent off list to Dustin, Matt East and Jeremy Bicha, however maybe someone else knows
something.

The issues: 

1.) The Ubuntu Documenataion related google custom search engines (CSEs) are out of date
2.) Trying to make and implement new google custom search engines seems to be difficult.

Note: the "search" tab on our web pages uses Google CSEs
 
What is needed:

We (meaning currently active documentation committers) need to have edits rights added to the existing google custom search
engines by whomever owns them and / or has the ability to do so.

What else is needed:

If we are unable to fix this via simple editing of the CSEs, once we get access then:
Someone that understands this stuff needs to volunteer to take on the task of updating it.
It seems incredibly complicated and there is a dearth of information about what was done and why.

I have also listed this on the agenda for the next team meeting, but Gunnar and I got into it in the last few days.

The big picture:

Gunnar has been doing some excellent behind the scenes work on help.ubuntu.com. The web crawler directives file, robots.txt, has
been updated so as to much better direct crawlers on indexing our content. The .htaccess file has been updated, amongst other
fixes it should also aid in crawlers being returned appropriate content. However our own custom search pages use very obsolete
CSEs, and as a result the tool is currently virtually useless, and as the search engine databases gradually flush old content they
will become even more useless if not updated.

Other:

@ Dustin Kirkland: You mention "Pay Google to remove the directed advertisements" on the wiki [1]. Is google being paid? I think
no, because there are ads.

References:
[1] https://wiki.ubuntu.com/DocumentationSearch
https://lists.ubuntu.com/archives/ubuntu-doc/2008-April/010786.html 
https://www.google.com/cse/home?cx=004599128559784038176:vj_p0xo-nng
https://www.google.com/cse/home?cx=003883529982892832976:e2vwumte3fq
http://bazaar.launchpad.net/~ubuntu-core-doc/ubuntu-docs/help.ubuntu.com/view/head:/search.html