A survey of newer terminal editor projects

Like many of you probably are, I’m very interested in terminal text editors. There’s an inexplicable draw to working with pure text in a pure text environment that just feels right to me. I did a little looking around for some experimental terminal editors that are in the vein of kakoune, or at least vim, which might have interesting properties. Here’s a list of ones that caught my eye along with short descriptions of the features that stood out to me.

  • Breeze - Compiles to WebAssembly, plugins in WebAssembly, |-shaped cursors, directly kakoune-inspired (breaks a little from kakoune in providing cursor movement without selection).
  • Zee - Tree-sitter highlighting, rope data structure for buffers (like Xi), targets 100 FPS even with millions of lines. Emacs bindings though.
  • Pepper - Directly kakoune-inspired, custom syntax highlighter, lua command prompt and config, intends direct lsp integration
  • Amp - Easymotion mode, sublime text syntax compatibility with jump to symbol based on it, yaml config.
  • Sapling - Pure AST editing and highlighting. Very early stage but interesting.
6 Likes

For decades programmers have had to write structured data (“programs”) in plain, unstructured text files. To make life easier, editors grasped at what few structural markers were generally available: editors like ed let you insert, delete, or change entire lines; editors like Microsoft’s Notepad let you move around by word or by page; probably the most complete are editors like Vim and Kakoune that provide all sorts of fancy operations for working with sentences or string-literals or matching grouping symbols, but at this point we’re grasping for structures that might reasonably appear in a generic text file, or even a generic source-code file — anything more requires the editor to have syntax-specific understanding.

Vim and Kakoune actually do have that, in a very simple way. In many situations, rather than using a predefined movement or selection pattern, you can write a regular expression that describes a small part of the file’s structure, and navigate like that. Regexes are not easy, especially beyond the most basic uses, but it’s a lot simpler to write a regex occasionally than it would be to update Kakoune’s C++ code and recompile it for every file-type you wanted to edit. This combination of built-in support for generic structures, and ad-hoc support for specific structures through regexes, is very powerful and is a good part of the reason why Vi-lineage editors are still around nearly fifty years after Vi was first invented.

Powerful as it is, there’s still significant limitations. For example, the “jump to matching bracket” operation is frequently confused by brackets inside string literals. Trying to work on text inside comments frequently gets the comment markers mixed up with the content. Are < and > grouping symbols or mathematical operators? Properly handling such things requires the editor to understand the specific file-format in question.

At this point a lot of people start talking about giving the editor deep knowledge of syntax, like an AST editor, or a JetBrains IDE, but it turns out this is a lot of work. For all the experience and institutional knowledge JetBrains has about editing structured programs, they currently sell eleven specialised IDEs — there must be thousands or even millions of file-formats and syntaxes in the world, it’ll never be possible to write a JetBrains-tier IDE for every single one. A tool like Tree Sitter is a lot simpler, but developing a Tree Sitter grammer still requires writing a grammar in JavaScript, compiling it to C, then compiling the C into your program — orders of magnitude more effort than writing a regex.

I definitely think the next killer feature of Vi-lineage editing will be structural knowledge, but I think it needs to be more like writing a regex or a Kakoune syntax highlighter than writing “real” code. I’d like to tell my editor that comment = "//" not("\n")* and then anywhere I write a regex I can write $comment to mean “text spans tagged as comments”. I’d like to try various structure patterns interactively, and when I find the ones I like, paste them directly into a file to make a grammar, like making a Kakoune plugin. I’d like to run an external program to generate a syntax tree in case the built-in structure pattern language isn’t flexible enough, in the same way that many Kakoune plugins generate range-specs highlighters.

I’m pretty sure it would be possible to make an editor like that, and I’m pretty sure it would be great, but I think it would be too radical a change for Kakoune. And writing a brand new editor would take a long time to reach the maturity and polish and ecosystem Kakoune already has. So I’m not sure that editor will ever arise, but I’m keeping an eye out just in case.

4 Likes

Kinda sounds like you want a Parsing Expression Grammar (PEG) based editor.

I hope these two things will become more common across all editors.

I concur, and from what I’m seeing in that small list, and with neovim, it seems to be a lot of people want to make it happen. I’m pretty bullish about tree-sitter because it’s used across projects so the likelihood someone will come along and improve a parser for your language is much higher than say a PEG for vis, or breeze’s custom lua matchers, or even kak’s highlighting groups.

Went looking for more recently, this time by browsing go packages. It feels like there haven’t been many successful editor projects in go in general, and those that made it were pretty standard. Interesting contrast to rust.

Anyway I did find one that has some interesting ideas, though I haven’t run it to see if it’s really that good:

  • Glom - annotation of documents, structural drag and drop. Actually a gui editor, but including for lack of terminal editors in go, and interesting features.
2 Likes

I disagree that this is too radical for kakoune. Imo, this is totally feasible as a plugin that integrates with tree-sitter. Sure, you have to solve the distribution problem, and the pain threshold for adding a brand new grammar is higher, but I don’t think those are wildly difficult.

There’s some required ui work for good integration.

  1. You could provide a locked structured mode which uses all the usual binds, but in a structured context. e.g. sf for select function decl/def. if the user wants, you could swap the modes to make structured the default and normal mode bindings in a user mode.

  2. tree-sitter parse trees aren’t asts and they differ from my mental model of the language, but that might be solvable in the pattern language.

  3. The pattern language of tree-sitter could be massaged in the same way that kakoune massages structured regular expressions to be interactive.

Kakoune is the editor that’s, imo, best equipped for this and the effort required to integrate kakoune with tree sitter seems likely to be far less than developing a new editor with that natively.

Additionally, if I think about an editor, I think I’d prefer the vanilla editor to not require language specific information to be useful, e.g. regex.

2 Likes

Like a chronic illness, once every few months I get an urge to find a structural code editor. Most of the languages I use primarily are less mainstream so I have yet to enjoy and explore kak-tree.
I rarely limit that search to terminal editor projects though. Like I think code editing shouldn’t be limited to movements conceived around text operations; we should also not be limited by what a char-grid can represent to edit codes.

Just found this on orange site:

https://helix-editor.com/

Looks like pre-alpha stage, but it’s interesting since it seems to follow Kakoune editing model pretty closely.

6 Likes

Important addition from the linked site: " Syntax highlighting and code editing using Tree-sitter.

No more slow regex highlighting! Tree-sitter parses code into syntax trees much like a compiler, giving us a lot more information about code structure. We can track local variables, calculate indentation and manipulate selection to select syntax nodes."

honestly the main problem I have with terminal based editors is the lack of dynamic windows(like the hovering docs from vscode), clippy(or other tool) showing over the top of the file being edited definitely gets in the way (especially with smaller window sizes)

1 Like