system-tap support in Ubuntu kernel

[Chase Douglas]

On Sun, 2010-06-27 at 20:50 +0300, Amit Kucheria wrote:
> On 10 Jun 25, Sebastien Jan wrote:
> > Just a short query regarding kernel config.
> > Some of our teams are regularly making performance tests and some monitoring and use system-tap for this task.
> > The Ubuntu config already includes some flags enabling a basic system-tap support out of the box.
> > However, for most of our system-tap based tests, an additional config flag has to be activated: CONFIG_BLK_DEV_IO_TRACE
> > This flag activates several additional tracers into the kernel config (see below for detailed config impact).
> > 
> > We are looking at if we shall include these additional flags in our default kernel images, or if we shall make special builds dedicated to system-tap usage (including this additional flag).
> > 
> > The criterion behind this choice are:
> >  - kernel size: I checked - increase of a bit less than 5%
> >  - performances: I have no idea
> > 
> > => So the questions are: 
> > 1) I know that the ubuntu kernel team makes reviews of the flags to integrate into the ubuntu kernel. Were these flags investigated, and was there a rational for not taking them?
> 
> Yes they are reviewed at UDS. I think someone on the kernel team will know
> the details about this specific decision, the mailing list is now cc'ed.
> Chase and Andy have been looking at the tracing problem in general.

I've been watching perf and ftrace for the Ubuntu kernel. I had
questions about the usage of some config options, so I inquired
upstream:

http://lkml.org/lkml/2010/5/25/382

> > 2) Would you take these flags into the ti-omap4 branch of the maverick tree?
> 
> I am curious to know if there is something in systemtap that cannot be
> measured using ftrace? As I understand the winds blowing in kernel-land,
> ftrace is the new favourite amongst all the tracing subsystems and is already
> upstream.
> 
> From personal experience, systemtap has always been very cumbersome to use,
> ftrace is a lot easier. Also, the runtime overhead of ftrace is close to zero (really
> zero?) when it is not enabled.

The problem here is that ARM still cannot dynamically enable and disable
ftrace. Thus, enabling it for ARM has a performance hit.

The only thing I am aware of that systemtap has over ftrace and perf
probe (new in maverick) is the ability to modify code execution (more
than just printing values of variables and registers). However, my gut
tells me this isn't used very often.

> > Complete changes implied by activating the CONFIG_BLK_DEV_IO_TRACE flag:
> > -ENABLE_DEFAULT_TRACERS n
> >  BINARY_PRINTF n -> y
> >  BLK_DEV_IO_TRACE n -> y
> > +CONTEXT_SWITCH_TRACER y
> > +EVENT_TRACING y
> > +FTRACE_STARTUP_TEST n
> > +GENERIC_TRACER y
> > +NET_DROP_MONITOR n
> > +NOP_TRACER y
> > +STACKTRACE y
> > +TRACEPOINTS y
> > +TRACING y

See the following email for what's enabled in our common builds:

http://lkml.org/lkml/2010/6/8/319

We then have architecture and flavour overrides, so it's best to check
our git tree for all the relevant config options.

-- Chase