[Bug 414885] Re: KVM crashes when -vga is set to vmware.

[Roland Dreier]

I figured out the cause of the crash, at least in my system.  With SDL,
qemu-kvm advertises the cursor setting capability to the guest, and
therefore gets a DEFINE_CURSOR command.  In the crashing case, it gets a
request for a 64x64 cursor at 32bpp, which requires 64*64 = 4096 32-bit
words to store.  However struct vmsvga_cursor_definition_s.image[] is
declared to have size only 1024, so handling the cursor request overruns
the array.

I'm attaching a patch that enlarges the image array, which fixes the
issue (for me at least -- I am able to run a Lucid guest with the vmware
X.org driver on a Karmic host with "-vga vmware" with this applied,
which used to crash).  I've not checked if upstream has fixed this yet.

** Attachment added: "Patch to fix cursor pixmap array size in qemu's vmware-vga implementation"
   http://launchpadlibrarian.net/36895573/qemu-vmware-vga-cursor-fix.diff

-- 
KVM crashes when -vga is set to vmware.
https://bugs.launchpad.net/bugs/414885
You received this bug notification because you are a member of Ubuntu
Server Team, which is subscribed to qemu-kvm in ubuntu.