I had the same problem. Though seems fixed in kernel > 2.6.20, as I haven't had this problems since upgrading the kernel to 2.6.20. -- kernel panics after modprobe https://launchpad.net/bugs/50372