[ubuntu-mythtv] Performance tuning for Mythbuntu

[Michael Haas]

Hello,

a  few days ago, Daniel Kristjansson (a MythTV developer), wrote an
interesting email to mythtv-dev with some notes for distributors. Here's
an excerpt:

> Along with some notes for distributors like: run xfs_fsr nightly or
> the equivalent for whatever file system you standardize on, install
> LIRC and make include setserial in the distro, tune mysql at least
> a little bit and delete the binary logs once in a while, no one wants
> a 200 GB log with weeks of DB queries filling up /var, etc, etc.
> 

Alright, I'll break down his statement into smaller, interesting pieces
and add some thoughts of my own:


> run xfs_fsr nightly

xfs_fsr is used to defragment XFS file systems. It makes sense to run
this once in a while because XFS can become fragmented, which will
result in a performance penalty, among other things. If you want to find
out how fragmented your XFS is, read "man xfs_db" and skip to the "frag"
option.

Here's what I have on my box:

> laga at screwless:~$ sudo xfs_db -r /dev/mapper/datengrab-aufnahmen
> xfs_db> frag
> actual 186374, ideal 811, fragmentation factor 99.56%
> xfs_db>                

Oops, this is more than I actually expected it to be, but it's a nice
example. Anyways, this file system was created in January and has since
served as the main directory for recordings.

Another approach to avoid xfs fragmentation is to use the "allocsize="
mount option. Excerpt from "man mount":

> Mount options for xfs
>        allocsize=size
>               Sets the buffered I/O end-of-file preallocation size when doing  delayed  allocation
>               writeout (default size is 64KiB).  Valid values for this option are page size (typi‐
>               cally 4KiB) through to 1GiB, inclusive, in power-of-2 increments.

See [1] and [2] (long!) for more information.

Implementation:

we could either run xfs_fsr as a cron job (maybe coupled with ionice,
see below) or mount file systems with allocsize=<SomethingsSane>. While
this is not a problem on our 'dedicated' Mythbuntu install disks, people
using mythbuntu-control-centre to convert existing boxes might not want
us to mess with their fstab.

--------------------------------------------

> tune mysql at least a little bit

Alright, once again the MythTV wiki can help us out: [3]. The thread
linked from that wiki article is also interesting, it includes an
explanation for some settings.



> delete the binary logs once in a while, no one 
> wants a 200 GB log with weeks of DB queries filling up /var, etc, etc.

Binary logs can be disabled by commenting out a few lines in my.cnf:

> #server-id              = 1
> log_bin                 = /var/log/mysql/mysql-bin.log
> # WARNING: Using expire_logs_days without bin_log crashes the server! See README.Debian!
> expire_logs_days        = 10
> max_binlog_size         = 100M
> #binlog_do_db           = include_database_name
> #binlog_ignore_db       = include_database_name

I guess that log_bin, expire_logs_days and max_binlog_size should be
commented out. I'm not sure if binary logs have any impact on
performance, so i guess I should find out :)

Implementation: A little bit of sed magic can customize my.cnf to "our"
needs, but does the user want that? Maybe we can add a tab with
"advanced settings" to the control centre. Just hope it won't break
often.. Maybe we should just copy a new my.cnf over while making a
backup of the older one.

-------------------------------------------------

Ok, I'll add some of my thoughts/suggestions:

* use ionice to lessen I/O load caused by mythcommflag/mythtranscode

I doubt that mythcommflag or mythtranscode are actually causing I/O
problems in many systems out there, but reducing the priority for these
i/o intensive tasks where a minor speed loss is negligible could improve
 system responsiveness.


Implementation:
I'm not sure how we could implement this. We could use a cron job to
regularly adjust the priority of all running mythcommflag/mythtranscode
instances (similar to "and"). OTOH, we could create wrapper scripts
similar to what we're doing for mythfrontend now, e.g.

#!/bin/sh
ionice -c2 -n7 /usr/bin/mythcommflag $@

In this example, -c2 puts the process in the "best effort" scheduling
class, -n7 makes sure sets it to the "slowest" priority available in
that class.

Using -c3 for "idle io priority" ("The impact of idle io processes on
normal system activity should be zero.") might be even better, but this
is not available to normal users ("ioprio_set: Operation not permitted").

In the ionice man page, it says "Programs inherit the CPU nice setting
for io priorities.". I wonder how nice levels are translated to ionice
levels. Does anyone here know more?


* install and use -rt kernel

In Gutsy, there's a linux-rt kernel available which includes "Ingo
Molnar's full real time preemption patch (2.6.22.1-rt8)". I'm not sure
if switching from -generic to -rt will be noticeable, though, because
-generic already includes some optimizations (preemptible kernel, etc).

At the time of writing, there's no fglrx/nvidia.ko in the l-r-m package
for the -rt kernel which renders it useless for a lot of MythTV boxes,
but a bug is already filed for that problem and I hope it's resolved soon.


* enable RTC video timing

There's already a blueprint:
https://blueprints.edge.launchpad.net/mythbuntu/+spec/rtc-video-timing

I'll try to get this into the kernel provided by Ubuntu. If that's not
possible, we can still add that line to sysctl.conf ourselves.


* enable realtime priority for the video display thread

I just saw that we already have a blueprint for that, too:

https://blueprints.edge.launchpad.net/mythbuntu/+spec/mythtv-process-realtime

It's not enough to give the system user the right to use realtime
priority, this setting also has to be turned on in the UI (playback
settings). Should we enable this by default? It could result in unstable
systems.



-------------------------------------------------------------


Alright, that's enough for today. Please let me know what you think
and/or if you have additional optimization ideas.

Regards,

Michael

[1]
http://www.mythtv.org/wiki/index.php/Optimizing_Performance#Combat_fragmentation
[2] http://www.gossamer-threads.com/lists/mythtv/users/287356
[3] http://www.mythtv.org/wiki/index.php/Optimizing_Performance#MySQL_Tweaks