[sok] Simple On-screen Keyboard planning

[Henrik Nilsen Omma]

Hi everyone,


The application deadline for SoC students is nearing an end (though it 
has been extended by Google by a day or so) and I'm pleased to see a 
good number of applications for 2 of the 3 AT projects (nobody applied 
for the config panel, which can understandably seem boring).

I won't go so far as to post all the applications here ;) but I'll post 
some more detailed thougths and invite discussion on the planned 
features and roadmaps (yes, this is in part me being lazy and not 
wanting to have 12 identical email conversations fleshing out spec 
details -- but I also think that an open process works better). I've 
copied in the SOK applicants on Bcc. Feel free to jump in with comments 
and ideas or just hang back and read the suggestions. To sign up: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-accessibility

The advantage in this open approach is that there are many people 
reading this list who know a great deal about accessibility and others 
who know much about ubuntu and development. The spec ideas still need a 
fair bit of fleshing out and input from experienced people. The sooner 
we start that process the better positioned we will be when it is time 
to start coding. I'll be doing the same with the XGL-magnifier later.


The Simple On-screen Keyboard
--------------------------------------------------

Let's start by dividing this up into milestones:

M0: Select The language and libraries we will use
  * I'm leaning towards python with pyGTK, but I'd like to hear other 
suggestions with reasons (where is python's xlib support, do we need it, 
can we use C for some aspects?).
  * I want to look at rendering the keys directly with Cairo instead of 
using a widget toolkit. That will give us more flexibility in the 
display and make porting to other platforms easier (or?).
  * Make sure these have bindings we need, esp. to AT-SPI
  * Decide on keyboard definition format. Can the existing GOK format be 
used or do we need to extend it with additional features (regarding key 
placement, colour, etc.)?

M1: Proof of concept keyboard
  * Make a 3-letter ABC keyboard that can pass keystrokes to the active 
application
  * Should be able to be always-on-top without being active
  * Keys rendered in Cairo (if we go that way)
  * All settings are hard coded

M2: A full QWERTY keyboard read from a file
  * Read a key layout from an XML file
  * Support for modifier keys such (Shift, Ctrl, Alt), probably without 
using Sticky Keys

M3: General configuration handling
  * Any configurations that that the user might want to make should be 
sepparated out from the code and read from a file
  * The scheme used should be compatible with the Common AT config approach

M4: Plug-in framework
  * The core idea of SOK is to keep it lean and simple, with extra 
features added as plug-ins.
  * Possible plug-ins might include: Word prediction, switch activation 
support, X-10 home automation support, ...

M5: Non-latin and Tablet PC support testing
  * The keyboard should be unicode from the ground up, so this should 
just work, but needs testing

Optional features:

I list these here partly as a braindump area for new ideas, but also to 
draw a clear line. These features will not be part of the initial 
implementation (to keep it simple) and may only ever be implemented as 
plug-ins.

 * Transparency and animation - these are cool and potentially useful 
things, but clearly non-core
 * Scriptability - Orca has shown us the advantage of adapting the AT 
tools to different applications. That may well have benefits for SOK as 
well.
 * Word prediction - very useful but not everyone needs it. Should be 
implemented as a plug-in or separate utility. Some will want prediction 
without an on-screen keyboard.
 * Graphical layout generator - Many will appreciate the ability to make 
their own layouts easily.
 * Configuration GUI - Should be seen in context with other AT apps, 
ideally as part of the common config panel (still just in planning)

-----

Also, while I'm writing this, let me clear up a common misconception 
that came up in the applications:

The main focus of this project is enhanced accessibility for disabled 
users. I mentioned tablet PC and multi-language input because I think it 
is strategically wise to factor in these use cases. If we make a working 
on-screen keyboard that can also be used in other ways we will get more 
testing and development. And if we don't then alternatives will 
eventually appear anyway. When one of the big PC makers decides they 
want to push tablet PCs for Linux they could either use our solution and 
help improve it or build their own non-AT version, which would be a 
small tragedy.


After we get some feedback on this and nail down some ideas we should 
flesh out the spec in the wiki: 
https://wiki.ubuntu.com/Accessibility/Specs/SOK

- Henrik