Nvidia and vesafb

[refdoc]

I am running x.org on hoary and use the nvidia driver.

For reasons which I do not fully understand (lack of knowledge) the
nvidia driver will not run with teh normal kernel, but asked on
compiling it requires a kernel without vesafb support.

If started , indeed X will not start up.
After modprobe vesafb to remove vesafb I can start X with nvidia active.

At the moment I am doing this by hand when I want to play 3d games - not
a good solution. How can I permanently disable vesafb at teh start - I
do not want to recompile a Kernel as this would need to be re-done with
every iteration of new kernel coming out. I'd rather have a config file
somewhere which keeps vesafb out of my system. 
What shall I do?

Thanks !

Peter
-- 
refdoc <refdoc at gmx.net>