Stormified importer has introduced unintended change in data being written to sqlite

[James Westby]

On Sun, 22 Apr 2012 01:52:16 +0100, Max Bowsher <_ at maxb.eu> wrote:
> I've repaired the database, mainly by making lots of use of:
> 
> UPDATE foo SET bar = CAST(bar AS text) WHERE typeof(bar) = 'blob';
> 
> However, in two tables (revids and should_retry), it wasn't that simple,
> since the above technique would give a key uniqueness violation.
> 
> For those, I manually consulted the data and merged it appropriately. In
> the revids table, this only affected 4 packages (6tunnel aalib acct ace).

Thanks for going through that to reconstruct the data.

> Because of the above woes and because of the deadlocking issue, and some
> discussion with Jelmer on #bzr, I've reverted Storm out of lp:udd.

I'm fine with that, but we should aim to push it back in, as it is a
step to moving away from sqlite, which is one of the causes of these
issues.

> I guess we can put it back in, in time, but this time let's be really
> really sure we don't create operational issues.

How do we go about doing that. We have only the production environment
available to give us good feedback on the reliability of the code it
seems.

The revert should have happened sooner, and I would have done it
immediately if the issues hadn't takes more than 24 hours to show up in
the first place.

I am genuinely interested in ideas for handling this better though.

> With the above accomplished, the importer is now happily alive, and
> chugging through the extreme backlog from the previous malfunction at a
> concurrency level of 12 threads.

Thanks for ensuring good service.


James