speech recognition to Linux text injector

[Eric S. Johansson]

Henrik Nilsen Omma wrote:
> That looks very interesting. I tried to do something similar last year 
> using a VNC connection instead, with mixed results. This looks much more 
> elegant. I'll be very interested to try it. It seems building and 
> installing is a major issue with it at the moment (because everyone has 
> different environments). We should clearly get it packaged for Ubuntu.
> 
> I don't know anything about Xnee, but I know that Chris wrote some 
> simple C code to inject keystrokes for onBoard, without using Xnee or 
> AT-SPI.

http://www.gnu.org/software/xnee/

"""Xnee is a suite of programs that can record, replay and distribute 
user actions under the X11 environment. Think of it as a robot that can 
imitate the job you just did.
"""

I agree packaging is a really important step but packaging what is there 
is not so good.  I really think that we should consider eliminating the 
xnee hack, before distributing.  I think simple keystroke injection 
while a good "let's get something working and help some level of 
disabled user" step, it may make it difficult to move to an environment 
where there is full command and control of gnome.  for example, one 
simple problem is awareness of the currently active window.  I use a 
Python based macro environment for all of my command and control macros. 
  When the grammar recognizes "Emacs" is active, it turns on the Emacs 
specific grammars so I can work in Emacs was painfully than I can by 
hand.  Simple keystroke injection makes that difficult.  Keystroke 
injection with active window state feedback makes it less so.

Then there is the ever popular select and say feature of 
NaturallySpeaking.  Again with direct injection, not very possible.  But 
with the right at-spi interfaces, we could read the window, find a 
section to be selected and do whatever magic is necessary.  But these 
things are not happen all at once.  I just wanted to highlight a couple 
of issues that will come in the future.

I was hoping to get to try it out today but I'm running out of time. 
Please keep me up to date on what ever you do.  I'm even willing to be a 
guinea pig.  Since I use Linux in a virtual machine, I can create a new 
test bed as long as I have disk space.

My goal is to eventually get to the place where Windows is only a host 
OS and a speech recognition platform.  Linux would run in a virtual 
machine full-screen and I can get my work done.

---eric