RFP: Kakoune, Accessibility and say

Plugins
#1

I am losing my vision, this has motivated me to consider the best way to write code while blind. There are a lot of perspectives on this in the accessibility community, but generally it involves the popular tools (VoiceOver and NVDA) and using heavy doses of assistance from autocomplete with tools like vscode.

This works well enough and I an exploring it. But the more I considered it the more I thought possibly using a light scripted layer with “say” (available on mac) which lets you pipe stuff to it to be read aloud with a focus on code might be worth more. Certain things could be done easily, like calling out indent depth, custom names for code-specific things that you might not want elsewhere. It would allow things like “on selection 1/9 content foo” “rotated selection” “applied to 7 locations” “opened insert on 7 lines” and woudl allow new commands useful to blind people, like “read all selected lines”. I think terminal editors have often been overlooked by the accessibility community, as they have a tendancy to just fill screens of text.

I am very new to this way of thinking, so I am mostly thinking out loud and do not speak for the accessibility community, but I am curious what ideas the community might have on how to how to use kakoune if your monitor was turned off…

Brainstorming ideas?

3 Likes
#2

Hey @robertmeta, sad to hear you are losing your vision.

While I never designed Kakoune with blind coding in mind, I think the fundamental editing and extension model could be well adapted to that use case. The focus on a simple, predictable and scriptable set of normal mode commands should already go a long way, as Kakoune is already usable for complex editing tasks with reduced feedback.

Feedback is obviously still needed, for that I hope the existing extension model can enable Kakoune to interact with tools such as a braille reader or voice based system. Where we might still be lacking good alternatives is for non-buffer content information, which is not always easily available or easy to send to external tools.

This is not something I thought for a long time about, so those are just initial thoughts. In any case, if we can make it easier to use Kakoune without relying on display, I think that’s worthwhile.

1 Like
#3

The simple command on MacOS “say” uses the voice and intonation, so just piping text to it works great. The hard spots are around handling like help and popup lists. The basic framing of the plugin will be just hooking events and piping data to say. Even idle can be used to do neat stuff like emit status information. The more I look at it, the more I think Kakoune could be near the ideal editor for the blind as the model of knowing what you are working on is key. Additionally, the menu system could be ideal for screen readers as well… as you are given the top level context first, so you can hit the key if you know it, or you can wait for it to be read. I might be going too far, but I think the gap between where it is and near ideal isn’t that far… some things I am curious about the best way to do, file finding, etc. But possibly the best way for a blind user is to simply populate a file with a grep style output and let it be treated normally to open files, also makes going back to it easy. Lots of gaps, but I think I am going to start building out the basic tooling and see how it goes, would love Kakoune to be the default editor of the vision impaired.

1 Like
#4

Wow that’s rough, I’m sorry.

It looks like the Orca screen reader reads whatever text is drawn on the terminal.
For example whenever the modeline is redrawn, it reads that (doing that on every movement quickly gets annoying so I set an empty modeline).
Orca also reads the completion list and sometimes even info boxes.

Proper Kakoune integration sounds much better though.
A plugin probably wants hooks that have a widget’s text as argument. Luckily, there are only few relevant widget types (I think):

  • info boxes
  • prompt line (echo/fail or navigating prompt history)
  • modeline (though it’s probably better to have shortcuts the individual attributes)
  • insert/prompt completer
  • menu (not used that much AFAIK)

there are also highlighters like line-flags but I can’t imagine them being necessary.

Somewhat related: here is someone who hacked Emacs to work impressively well with voice input. I found that really inspiring.

Another exciting idea is https://oskars.org/ , a braille keyboard for smartphones.

FWIW I use a (grayscale) eink screen because it’s easier on the eyes. I’ve been using this colorscheme lately:

# Black-on-white colorscheme for minimal distraction & maximal contrast.
# Works well with e-ink screens, also in sunlight.

# For Code
face global value default
face global type default
face global variable default
face global module default
face global function default
face global string default
face global keyword default
face global operator default
face global attribute default
face global comment default
face global documentation comment
face global meta default
face global builtin default

# For markup
face global title default
face global header default
face global mono default
face global block default
face global link default
face global bullet default
face global list default

# builtin faces
face global Default default,default
face global PrimarySelection black,rgb:cccccc+fg
face global SecondarySelection black,rgb:e0e0e0+fg
face global PrimaryCursor default,default+rfg
face global SecondaryCursor white,rgb:777777+fg
face global PrimaryCursorEol black,rgb:777777+fg
face global SecondaryCursorEol black,rgb:777777+fg
face global LineNumbers default,default
face global LineNumberCursor default,default+r
face global MenuForeground white,black
face global MenuBackground black,rgb:dddddd
face global MenuInfo default
face global Information black,white
face global Error white,black
face global DiagnosticError default
face global DiagnosticWarning default
face global StatusLine default,default
face global StatusLineMode default,default
face global StatusLineInfo default,default
face global StatusLineValue default,default
face global StatusCursor default,default+r
face global Prompt default,default+r
face global MatchingChar default,default+b
face global Whitespace default,default+fd
face global BufferPadding default,default
#5

Yeah, the idea would be while in kakoune to disable the screen reader because Kakoune would provide all the needed feedback. Making it able to be more tuned to the task. So, only data piped to say would be heard. So hooking things like what buffer has the input.

I don’t want to change the UI too much either, because when pairing it is good for the other developer to be able to see it regularly.

The other thing I am struggling with is the best way to REPL and do a test cycle, might finally force me into using proper test driven development! Because generally I just un another terminal right next door and send commands to it, but with the screen reader off those changes would be invisible. Maybe just routing them to a file and reading the file would work.

#6

I use :make for that, it’s very convenient because I can quickly cycle trough stack traces.

#7

Hi @robertmeta, I came across Microsoft Accessibility resources a while back and given your predicament you might find it useful in your quest for kakoune and programming with Accessibility services in mind.

I have not read the title but is rather a resource for when my programming matures past algorithms and data structures. The below title its a high level overview for software design with a focus on .Net framework.

You may enjoy it or not and that’s ok, it’s an idea from the other side of the fence which may help in your quest.

Microsoft 2009, ‘Engineering Software for Accessibility’, Microsoft Press, viewed 04 April 2022, https://www.microsoft.com/en-us/download/details.aspx?id=19262

I also have access to a large multinational database network at my university if you need anything (research material) just ask. Even ‘pm’ me with a bunch of defined keywords and I can screenshot you back a list of resources or a specific title to broaden your knowledge base.

Looking forward to Kakoune Accessibility, bye :wave:.