State of the Ubuntu Error Tracker for August 29th

Wed Aug 29 16:25:28 UTC 2012

This is a periodic update of the status of the Ubuntu Error Tracker project
(whoopsie-daisy).

*Get involved*

We're looking for people to help with the Ubuntu Error Tracker project.
There are lots of interesting tasks that need to be done, spanning a wide
range of technologies and skills.

This document will get you started:
https://wiki.ubuntu.com/ErrorTracker#How_you_can_help

See my previous email for a list of tasks that need your help:
https://lists.ubuntu.com/archives/ubuntu-devel/2012-August/035636.html

Equally, do feel free to get in touch if you would like to help out, but
are confused or need more information.

*What we're working on*

   - Evan is investigating why the retracers are exiting multiple times a
   day and fixing the coloring of the most common problems table when Ubuntu
   12.04 is selected.
   - Brian is trying to fix apport sending incomplete reports to
   daisy.ubuntu.com and is going to start working on getting package
   installation failure reports in the database.
   - Matthew has tweaked the
text<https://code.launchpad.net/~mpt/apport/warmer-text>of the error
dialogs.

*New features*

There are a number of changes to https://errors.ubuntu.com queued up behind RT
55322 <https://portal.admin.canonical.com/55322>. *All of these have not
landed on the production website yet.*

*Permanent URLs for most common problems view (bug
1020580<https://bugs.launchpad.net/errors/+bug/1020580>
)*

You can be very specific and specify that you want the most common errors
that occurred in Ubuntu 12.04 in the 5.2.5 version of software-center from
July 1st to August 26th:
https://errors.ubuntu.com/?release=Ubuntu%2012.04&package=software-center&from=20120701&to=20120826&version=5.2.5

If you want to see...

All the versions for that period:
https://errors.ubuntu.com/?release=Ubuntu%2012.04&package=software-center&from=20120701&to=20120826

The most common errors in the Ubuntu 12.04 versions of software-center for
just July 1st:
https://errors.ubuntu.com/?release=Ubuntu%2012.04&package=software-center&from=20120701

The most common errors for software-center on July 1st:
https://errors.ubuntu.com/?package=software-center&from=20120701

*Better colorization of the most common problems table*

Matthew, Emily, and I have reworked the coloring of the most common
problems table to be more obvious in its intent:

http://people.canonical.com/~evand/tmp/errors-design-proposed-20120823-1.png

Pink lines are potential regressions. These are problems that have been
marked as fixed or invalid, but are still showing up in the most recent
published version. Grey lines are problems which do not yet exist in the
most recent published version. Strike-through lines are problems that are
marked as complete.

A couple of you have asked for a key at the bottom of the page, but I would
like to first ensure that we can get this as straightforward as possible
before resorting to lots of help text.

*More accurate listing of the most common problems*
*
*
The most common problems table no longer hides entries with less than 20
instances. This was causing confusion when looking at the Ubuntu 12.10 view
before there were any problems above this threshold, and thus seeing an
empty result.

*Faster loading of the most common problems*

The load time of the table has been considerably sped up. We fetch results
from Launchpad without SSL since we're behind the firewall
(r167<http://bazaar.launchpad.net/~ev/errors/trunk/revision/167>).
We now grab five rows at a time
(r161<http://bazaar.launchpad.net/~ev/errors/trunk/revision/161>).
There is more that can and will be done.

*More accurate average errors per day graph*

The calculation for the graph is now the number of crashes in the day
period divided by the number of unique systems seen in the 90 days prior to
and including the day period. This gives us a view of the computers that
would be likely to report a crash.

Previously, we were simply dividing the number of crashes in the day by the
number of unique systems seen in the same day. This meant that the graph
would never go below 1.

*Most common problems table improvements*

It is now possible to select a specific version of a binary package for the
most common problems table
(r151<http://bazaar.launchpad.net/~ev/errors/trunk/revision/151>).
The inability to select a date range twice and other form oddities should
now be fixed (r159<http://bazaar.launchpad.net/~ev/errors/trunk/revision/159>).
The Frequency column uses a linear scale, rather than the confusing
logarithmic one (r163<http://bazaar.launchpad.net/~ev/errors/trunk/revision/163>
).

*New API methods*

You can obtain the full report from an instance of a problem by using the
following URL, replacing id= with the instance identifier:
https://errors.ubuntu.com/api/1.0/instance/?id=ffbcf7ca-7cc6-11e1-889b-e4115b0f8a4a&format=json

You can obtain the reported instances by a particular machine by using the
following URL, replacing id= for the machine identifier:
https://errors.ubuntu.com/api/1.0/reports-for-system/?id=deadbeefdeadbeefdeadbeef&format=json

*Recording the installation version with new reports*

Brian has changed whoopsie to include the Live CD version with each new
report, giving us a data point for the to-be-created "what is interesting
about this problem?" section on the relevant page.

*Unresolved issues*

1043229 <https://bugs.launchpad.net/daisy/+bug/1043229> - The color coding
in the Ubuntu 12.04 specific view is inaccurate.
1036927 <https://bugs.launchpad.net/daisy/+bug/1036927> - The retracers are
exiting multiple times a day. We thought we had fixed this last night by
correcting the configuration for the retracers to see more than a single
cassandra node, but this is still occurring.
1040190 <http://pad.lv/1040190> 984823 <http://pad.lv/984823>
974428<http://pad.lv/974428>
 968157 <http://pad.lv/968157> 964696 <http://pad.lv/964696> - Some crashes
are improperly identified. This is a nice bite-sized problem. If anyone
wants to pick it up, let me know.
989800 <http://pad.lv/989800> - Error reports stop appearing after the
third time, skewing statistics. I think we should aggregate the reports and
display them in bulk, but there issome
disagreement<http://irclogs.ubuntu.com/2012/08/20/%23ubuntu-devel.html#t11:51>.
This would be addressed by the multiple simultaneous errors
branch<https://code.launchpad.net/~ev/apport/multiple-simultaneous-errors>
of
apport.
1042970 <http://pad.lv/1042970> - Block reports of DBus no-reply messages.

*Tickets*

55650 <https://portal.admin.canonical.com/55650> - Please add a cronjob for
90-day unique users calculation. This is support for the daily calculation
of 90-day unique systems. As this is an expensive operation (~2-4 minutes
of getting all the system identifiers seen in 90 days and then finding the
unique set), this cannot be done in real time. By only showing the previous
day and back, we indirectly solve the problem of the current day's totals
being inaccurate because they did not take into account the proportion of
the day remaining.
52598 <https://portal.admin.canonical.com/52598> - Reinstall cassabanana as
another retracer machine. This adds a second retracer machine, also running
amd64 and i386 instances.
55518 <https://portal.admin.canonical.com/55518> - Hadoop for
daisy.ubuntu.com. This will let us write ad-hoc queries that are balanced
against all the Cassandra nodes.
55322 <https://portal.admin.canonical.com/55322> - Setup django-openid-auth
backed by a database for errors.ubuntu.com. This caches OpenID logins,
fixes the issues some of you were seeing when trying to log in, and paves
the way for team-specific views of the most common problems table.
53325 <https://portal.admin.canonical.com/53325> - Need an instance of
jmxtrans talking to the crash database cassandra ring and outputting to a
(new?) graphite server. This gives us a better view into the day-to-day
operation of Cassandra and the errors.ubuntu.com API.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/ubuntu-devel/attachments/20120829/ee165a3b/attachment-0001.html>