If you could attach the output generated from running valgrind (see https://wiki.ubuntu.com/Valgrind) when you start rhythmbox to when you see the memory usage change significantly, that may be of some help once you've narrowed down how you're using rhythmbox. -- Possible memory leak https://launchpad.net/bugs/58372