Digging into TalkBack's source code

Braille

October 11, 2021

For a while now, I've been curious about which platform's accessibility is, at its foundation, more secure, more "future proof", and better able to be extended. Today, I'm looking into the TalkBack source code that is currently on Github, which I cloned just today. I'll go through the source, to see if I can find anything interesting.

First of all, according to this file:

https://github.com/google/talkback/blob/master/braille\src\main\java\com\google\android\accessibility\braille\service\translate\LouisTranslation.java#L2

This whole project was started in 2015. Of course, we then have this one:

https://github.com/google/talkback/blob/master/braille\src\main\java\com\google\android\accessibility\braille\service\translate\TranslateUtils.java#L2

Which shows that it is copyright 2020. The first just seems to wrap Liblouis in Java, but what about this one?

Ah, it seems to be the thing that translates the table files and such into Java things. So that's kind of where the Braille keyboard gets its backend. Now let's look at the front-end.

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\analytics\BrailleAnalytics.java#L1

So, this was made in 2019. I do like seeing that they have been working on this stuff. Now, here, we have:

/** Stub implementation of analytics used by the open source variant. */

Yeah, figured I wouldn't get much out of this file.

Now here's where we might just get something:

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\dialog\ContextMenuDialog.java#L4

This part was made in 2020. When they need to crank out a feature, they really get rolling. I just hope they give us a good few features *this* year.

Oh, now this is pretty considerate, a dialog will show if the Braille keyboard is opened and TalkBack is off:

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\dialog\TalkBackOffDialog.java#L27

And one for if a device doesn't support enough touch points (like an iPhone!):

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\dialog\TooFewTouchPointsDialog.java#L26

Okay, so this next one seems to allow the braille keyboard to grab braille from a file, then turn it into something else:

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\input\AutoPerformer.java#L51

This could lead to a sort of substitutions list, or spell checking or braille correction facility.

Ah, now this:

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\input\BrailleInputPlane.java#L53

is a good summary of the interface. Also yeah, looks like lots of geometry.

Okay, this part is rather interesting:

/** Reads saved points from SharedPreference. */

So, does that mean it can remember where I most put my fingers for typing?

And here's the jackpot:

https://github.com/google/talkback/blob/master/brailleime\src\main\java\com\google\android\accessibility\brailleime\keyboardview\KeyboardView.java#L29

Here, we learn how Google is working around explore by Touch to give a braille keyboard that bypasses TalkBack's own touch interaction model.