x11vnc without X

[Tim Henderson]

I have a workstation setup in a remote location.  I have it set to a 
runlevel which does not start X or gdm (customization).  Keep in mind 
this is not single user, this is multi-user.

I am trying to vnc into the workstation while it is in non-X multi-user 
mode, however the connection keep getting dropped.  The x11vncserver 
command I am using is below:

/usr/bin/x11vnc -usepw -display :0 -create
(and other variants, this is basic)

However when a connection comes in the errors are as follows:

24/07/2010 16:35:28 Got connection from client IP
24/07/2010 16:35:28   other clients:
24/07/2010 16:35:28 incr accepted_client=1 for IP:60302  sock=4
24/07/2010 16:35:28 wait_for_client: got client
24/07/2010 16:35:28 client progressed=0 in 15/10 0.009926 s
24/07/2010 16:35:28 client_set_net: 216.92.130.16  0.0028
24/07/2010 16:35:28 wait_for_client: running: env X11VNC_SKIP_DISPLAY='' 
  /bin/sh /tmp/x11vnc-find_display.MxfLUl
xauth:  creating new authority file /home/USER/.Xauthority
24/07/2010 16:35:28 wait_for_client: find display cmd failed.
24/07/2010 16:35:28 wait_for_client: FINDCREATEDISPLAY cmd: /bin/sh 
/tmp/x11vnc-find_display.MxfLUl Xvfb
trying N=20 ...
24/07/2010 16:35:28 wait_for_client: read failed: /bin/sh 
/tmp/x11vnc-find_display.MxfLUl Xvfb
24/07/2010 16:35:28 fgets: Bad file descriptor

(snipped out username and IP)

Then the server stops and a connection is never "really" made.  Any 
ideas on how to set this up, to start X only during x11vnc sessions?

Thanks,
Tim H.