Package branch freshness

Tue Jul 19 14:33:39 UTC 2011

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

...

>> I think there are a couple of factors:
> 
>> 1) You're on a pretty impressive link. For me, the bzr+ssh handshake
>> takes about 2.8s. (time echo hello | ssh bazaar....) So adding even 1s
>> to that isn't terrible.
> 
> Averages out to about 1.3s for me.
> 

I'm curious what your ping time is. As here it is pretty consistently
2.5s+. And my ping is pretty good at 34ms.

>> 2) debian was *much* slower than Ubuntu branches. This is because
>> apparently Launchpad doesn't track "Published" for debian. So when
>> making an api request for "ubuntu/bzr" we can ask for what the single
>> most recent Published state is. For Debian we have to ask for the
>> Pending list. Now, we limit the list to 1 entry, and Launchpad sorts in
>> newest-first. However, I imagine at the app/DB level, Launchpad has to
>> read in all the rows and sort them.

Turns out they also have a SQUID cache in front of the app servers, so
repeat-testing may or may not be valid. For the real use case, I think
it actually isn't valid, since most of the requests are going to be
cold, and aren't going to be repeated in within the cache lifetime.

On devpad, with SQUID it takes 50ms to get the response. Explicitly
skipping the cache (by setting an Auth header), I get cold 750ms, then
190-360ms. (for debian). 197-250ms for ubuntu. Though again, there is
still some DB level caching when repeating these queries.

So regardless of latency, the "best" worst-case time is still 750ms. So
adding that to your 1.3s ssh handshake is certainly going to show up.

I believe these queries haven't been particularly optimized on the LP
side, either.

I also ran into a very strange python bug. Which is that making the
first SSL connection takes about 600ms on my machine, but making a
second SSL connection (not shared, not even to the same host), takes
only 200ms. I'm guessing something with openssl caching the certificate
files or something.

> 
>> 3) The API request is a lot slower today than when I was testing it
>> during development. I don't have any idea why that would be. But pulling
>> out the setup code into:
> 
> Interesting.

I'm wondering if it is the second-ssl connection thing. It shouldn't be,
because there should have still only been a single https connection, but
maybe I'm missing something.

> 
>> Great. I think it would certainly be reasonable to have a knob that
>> issues short form for people who know what everything means and what it
>> to be as easy to parse as possible.
> 
> +1, and thanks!
> 
> Something else I've been wanting to do, but which probably isn't within
> bzr/udd mission.  I want an uber-rmadison which tells me all the versions of a
> package in both Ubuntu and Debian, *and* in the packaging branches, without
> having to download anything.  Yeah, I should just go write the thing.
> 
> -Barry

which inter-mixes series.

Its pretty easy to get the tags from the packaging branches. I'd be
happy to work with you on something like that. Especially on working out
ways to avoid round-trips to expand the info. It would be *really* nice
if Launchpad tracked Published for debian the way they do for Ubuntu:
bzr | 2.4.0~beta5-2ubuntu1      | oneiric          | main
bzr | 2.3.3-0ubuntu1            | natty-proposed   | main
bzr | 2.3.1-1ubuntu1            | natty            | main
bzr | 2.2.4-0ubuntu1            | maverick-updates | main
bzr | 2.2.0-1                   | maverick         | main
bzr | 2.1.4-0ubuntu1            | lucid-updates    | main
bzr | 2.1.1-1                   | lucid            | main
bzr | 2.0.2-0ubuntu1            | karmic-updates   | main
bzr | 2.0.0-0ubuntu1            | karmic           | main
bzr | 1.13.1-1                  | jaunty           | main
bzr | 1.6.1-1                   | intrepid         | main
bzr | 1.3.1-1ubuntu0.1          | hardy-updates    | main
bzr | 1.3.1-1                   | hardy            | main
bzr | 1.3.1~rc1-0ubuntu1~gutsy1 | gutsy-backports  | main
bzr | 0.90-1                    | gutsy            | main
bzr | 0.8.2-1ubuntu3            | dapper           | main

Anyway, I'll start poking at the verbosity stuff next.

John
=:->
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Cygwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk4llcMACgkQJdeBCYSNAAMopQCfdDAdBOX6ZnJbWtUP+Dr/rA6K
zMsAn1iK6UxmxovtxlCyWU2U1GcI6+do
=PWm/
-----END PGP SIGNATURE-----
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: up_to_date.py
URL: <https://lists.ubuntu.com/archives/ubuntu-distributed-devel/attachments/20110719/4b9b9c4e/attachment.ksh>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: up_to_date.py.sig
Type: application/octet-stream
Size: 72 bytes
Desc: not available
URL: <https://lists.ubuntu.com/archives/ubuntu-distributed-devel/attachments/20110719/4b9b9c4e/attachment.obj>