[xgl-mag] XGL/compiz screen magnifier planning
Henrik Nilsen Omma
henrik at ubuntu.com
Tue May 9 21:37:35 BST 2006
The application deadline for SoC students has just ended. In the same
way that I posted some new thoughts about SOK I will also post some
musings on the XGL magnifier. See:
Again I've copied in the applicants on Bcc. Feel free to jump in with
comments and ideas or just hang back and read the suggestions. To sign
The advantage in this open approach is that there are many people
reading this list who know a great deal about accessibility and others
who know much about ubuntu and development. The spec ideas still need a
fair bit of fleshing out and input from experienced people. The sooner
we start that process the better positioned we will be when it is time
to start coding.
XGL/compiz screen magnifier
I'm afraid most student applicants have underestimated this project,
thinking we are simply looking for a magnifier window to follow the
mouse cursor around. That is trivial to implement, but not very helpful.
This misunderstanding is largely my fault for not writing up the project
in greater detail before the SoC application process. So let me try to
explain better here.
The best way to get a feel for what we are looking for is to play with
gnopernicus in magnifier mode. It has the basic features we need, but
performance is not great and the usability could also do with some love.
You'll notice the vital cursor tracking feature. If you open Gedit with
the magnifier on, you can see that the zoom window will stay focused on
the input cursor as you type along. If you try the same thing with kmag
you will see that this feature is missing.
The must-have features:
* Fixed zoom factors - This could be pre-set by the user, but it should
also be possible to switch zoom levels on the fly with a hotkey
* Ability to follow the text-entry cursor - This requires AT-SPI magic,
but is a vital feature.
* Configurability - As many settings as possible should be configurable
by the user (but it can be a simple config file to start with).
Some desirable features:
* Variable zoom factor - Fixed factors may not be exactly want you
want. It would be cool to be able to adjust the zoom level precisely
with the mouse wheel, say.
* Scriptability - Again Orca has shown that being able to tailor the
behaviour of the AT app to your working application can be very useful.
This might well apply to the magnifier as well.
* Fit window to zoom area - If you are working in Gedit and you have
zoomed in to a comfortable level, you now want to make sure that the
whole window fits into the viewable area. You then press a hot key and
Gedit's window is resized so that it fits.
* Jumping between window sections - If you zoom to a fixed level of 3x
you then have in a sense 9 sections on the screen. You could float
around the desktop with the mouse, but there might be cases where you
want to know exactly where you are. On Gnome you can switch between
desktops with the Hotkeys Ctr-Alt-Right/Left/Up/Down. We could do the
same with zoomed sections.
* Interaction with screen readers and braille systems - This adds
another level of complexity but could potentially be extremely powerful.
You could browse at a moderate zoom level and then have sections of text
read out to you.
Let's look at some use cases to get an idea of the kind of features we
need to provide:
Use case - Reading a news website:
Take this news page: http://news.bbc.co.uk/1/hi/uk/4752819.stm
It has some text and pictures and a great deal of non-essential
navigation items (most other newspapers are worse). For the sake of
example, let's consider a user who might use 2x zoom to get a general
overview of a desktop of webpage layout but would need 8x zoom on normal
text to read it.
It would be useful to be able to zoom in on the different elements
separately. Imagine that you could select the picture first. Press some
hotkey to skip to the picture, and have the picture by itself displayed
at 8x zoom where you can pan around. Another keystroke would give you
the caption overlayed at the bottom of the picture in a high contrast
colour or it could be read out via the screen reader.
You then want to move on to the main body text. You return to the 2x
overview page. You press another hot key to highlight some text. The
first block of text appears with a heavy bright red (customisable of
course) frame around it or with a full rectangle at 50% opacity so you
can pick out the block you want. the next hotkey or mouse command would
then zoom that text up to 8x, filling most of the screen. It would be
wrapped to fit nicely and could be scrolled if needed. It would be as if
you selected the text, copied and pasted it into a special app that
would magnify it to your liking or read it out if you preferred. You
could also select your own fonts.
Use case - Document composition:
In using a word processor like OpenOffice you can set the font size very
large or the zoom to a high factor in OpenOffice itself, which might be
sufficient to read the text. However there are several problems:
* The text entry cursor is narrow and difficult to spot
* The toolbar buttons, slider widgets, etc. are small and in low
contrast grey tones
* The menubar menus and context menus are small
You could go for a large high-contrast theme, but in fact you might
prefer to keep the buttons and widgets small so they don't take up too
much space and distract from the text you are working on.
Solution: You should be able to hover over the toolbar and have the
buttons zoomed by 8x and the contrast enhanced. You move the mouse over
to the right and the scrolling widget grows a large handle in high
contrast that you can easily grab and scroll. The context menus also
Both of these cases need a fair bit of information from the application
via AT-SPI and there needs to be custom script knowledge about the
application and individual user preferences. In the case of the BBC
website the system would also need to know where the useful information
usually is located. But if you have 10-20 websites you visit often you
could teach the magnifier to behave a certain way on those domains, and
you could share those custom scripts with others.
It soon becomes apparent that this is a great deal more complicated than
simply providing a moveable magnification window. But then simple
problems are boring really :) With the combination of the new XGL
technology, an experienced user group in our community and fresh
programming talent we have an opportunity to do some great stuff!
More information about the Ubuntu-accessibility