System call analysis report

[Robert Collins]

On Thu, 2007-05-10 at 12:28 +1000, Martin Pool wrote:
> On 5/10/07, Ian Clatworthy <ian.clatworthy at internode.on.net> wrote:
>         All,
>         
>         Attached is one of the reports I'm putting together for the
>         sprint in
>         London next week. The idea is to run a 'user script'
>         containing various
>         Use Cases and to measure and report on various things: time
>         taken, disk 
>         space used, system calls made, network traffic, disk io, etc.
>         The script
>         can be run on different source code trees so we can see how
>         well we
>         scale or otherwise. As we address bottlenecks over time, we
>         can also
>         check we haven't pushed the problem elsewhere - the good ol'
>         whack-a-mole tuning challenge. :-)

It looks nice.

>         This particular report shows data from a script that was run
>         on the
>         source code tree created using this command in the 0.16
>         branch:
>         
>            bzr export bzr0.16.tar.gz
>         
>         The resulting tree contains 613 files. I'll have reports on
>         much larger
>         trees (e.g. 50k files) real soon.
>         
>         If you have any suggestions on making this report more useful
>         ( e.g. by
>         filtering the set of system calls to just the 'useful' ones),
>         let me
>         know. I'm keen to turn the vast amounts of data being
>         collected by
>         strace and other tools into information we can action.

For individual analysis I plan on using the strace module within bzrlib
because of its ability to remove the python startup overhead between
runs. To be able to assess scaling on tree size it would be nice if you
can generate a report on an entirely empty tree with no history: that
should let us determine how the overhead scales as we add files/history.

> That does look pretty interesting -- I guess the real test will be
> when someone tries to use it in evaluating a change to the code, or if
> we catch some performance regressions by looking at it.

Indeed. I dont think it will help us with e.g. disk head seeking at this
point. I wonder what we can do to get such information. Head thrashing
is one of the things the hg folk have reported grave issues with in the
past.

> I have a couple of random thoughts -- may or may not be useful so you
> don't need to do them:
> 
>  * I like the way it displays the deltas rather than all absolute
> values; you could maybe make them red or green depending. 

That would be nice. Please be sure to vary the saturation of the
colours.

>  * strace has a mode (-T) where it can show the time spent in system
> calls, or if you used -c to generate this table then you should have
> the time already.  It might be good to include that as a second row in
> each cell. 

I wonder if dstat can give use some additional figures on the same run.
We'll want to have the rest of the system at rest of course.

Another interesting thing would be to sort the table of syscalls by
frequency within the current codebase. 

-- 
GPG key available at: <http://www.robertcollins.net/keys.txt>.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : https://lists.ubuntu.com/archives/bazaar/attachments/20070510/d50fd851/attachment.pgp