Moving udd away from sqlite

[Jonathan Lange]

On 15 June 2012 09:56, Vincent Ladeuil <vila+udd at canonical.com> wrote:
>>>>>> James Westby <james.westby at canonical.com> writes:
>
>    > Hi,
>
>    > We're interested in moving our deployment of udd away from sqlite
>    > to postgres, and we're interested in doing the same thing for the
>    > package importer deployment.
>
> There is a work in progress regarding the jubany package importer
> deployment, let's try to no step on each other feet ;)
>

What's the work in progress?

...
>    > After lots of head-scratching I believe I've worked out why
>    > changing to storm made this happen when the underlying db was the
>    > same. Storm forces sqlite to operate at a higher isolation level,
>    > so udd was taking locks more frequently or holding them for
>    > longer, leading to more contention and eventually the deadlock.
>
> But we already know we can have deadlocks with sqlite
> (https://bugs.launchpad.net/udd/+bug/724893) so I'm not convinced
> changing the db will magically fix the issues.
>

It's not magic. It's moving from a database that's not designed for
concurrent use to one that is designed for concurrent use.

> <snip/>
>
>    >   4) Leave the package importer on sqlite and do something else for our
>    >      instance (fork basically)
>
> As I mentioned before, I favor a friendly fork that can be merged back
> after thorough tests.
>

Even once the correctness and safety of the code change is
demonstrated, you'll still have to pick something from Options 1 - 3.
And I'm very reluctant to fork without an actual plan for merging
back: how to know when it's safe & how to actually achieve it.

When: We won't be running on sqlite, so our own experience in
production won't be valid. AIUI, the package importer doesn't have a
staging server, doesn't have anything in its automated test suite that
demonstrates the locking problem that James saw and doesn't have a
test plan.

How: We'll still need to pick something from Options 1 - 3. Degraded
performance? Storm hack? Postgres?

>    > Are there any others?
>
> Setup a test environment with a launchpad test instance where you can
> demonstrate that the importer won't break when this change is deployed ?
>

James has already done this in an EC2 environment. I don't know how
closely it mirrors the production environment, but it demonstrated the
locking problem and is how he came up with those options.

We have had a lot of experience recently working with Canonical IS to
get new servers and new staging servers deployed. If you want a
staging server, we'd be happy to help you and would gladly advocate
for you in their priority queue.

cheers,
jml