[u-a-dev] Phone?

[Luke Yelavich]

On Thu, Sep 24, 2015 at 08:41:11PM AEST, Ken Perry wrote:
> That is amazing.  That the accessibility would not already be started.
> Putting it in after the fact is much harder than adding it when it is built.
> Just look at the difference between apple who put it in when they made the
> OS verses google who is still trying to fix all their access problems with
> Android.  Is the Orca stuff not working with it at all?

Since this is a development related list, I'm going to go into some
technical details here.

Firstly, the phone project does not use X, but a Canonical developed
display server called Mir. Mir is similar to Wayland in what it does,
however the accessibility stack doesn't even support Wayland yet, let alone
mir. The accessibility stack, specifically the at-spi registry daemon has
to work with the lower level parts of the display and input stacks to be
able to work properly with widget screen coordinates, moving the mouse
and monitoring mouse movement, keyboard events, and touch screen events,
which again, at-spi doesn't yet support, even in X.

Secondly, there is the toolkit chosen to build phone applications, and
the new Unity 8 shell. The toolkit in use is Qt5. Qt5 has cross-platform
accessibility support, however this varies from platform to platform. Even
using the Qt4 linux accessibility work as a base, the Qt5 linux
accessibility support is still very imature, and needs both testing, and
further development work. Qt5 also supports a form of markup for developing
application interfaces called qml. The QML library and language also need
to support accessibility, but this is only partially done. In addition,
QML is so abstract, that one has to specifically add relevant accessibility
metadata into the interface code to make accessibility with Orca et al a
possibility. THis is something that could be done in the SDK, and developers
are encouraged to use the SDK for widgets etc, but a developer doesn't have
to at all, and it cannot be forced on them. One also has to add metadata
for keyboard navigation as well, to move between text fields, buttons, etc.

There is also the web browsing experience to consider. Canonical were
originally using Qt's webkit library for the web engine, but have since
moved to using the Chromium/blink web engine, with a wrapper on top of
that for integration with the phone and relevant platform libraries. This
wrapper would need to be extended to enable accessibility for use with
Orca. Blink/Chromium does have internal code for doing this, but its
connecting up with that infrastructure that requires work.

I have talked a few times with folks about getting things moving, but
its not a priority for them, and they have other things that they have
to work on. Of course they are happy to help those who want to work on
things etc. As it is, I currently have my hands full just making sure the
existing desktop remains accessible.

Hope this sheds some light on the situation.

Luke