I have been using Emacs at work for whole week

I’m asking because tools like tree sitter provide coordinates. This code:

fn foo(x: i32) {
    println!("x is {}", x);
}

Produces such tree in tree-sitter’s playground:

source_file [0, 0] - [3, 0])
  function_item [0, 0] - [2, 1])
    name: identifier [0, 3] - [0, 6])
    parameters: parameters [0, 6] - [0, 14])
      parameter [0, 7] - [0, 13])
        pattern: identifier [0, 7] - [0, 8])
        type: primitive_type [0, 10] - [0, 13])
    body: block [0, 15] - [2, 1])
      macro_invocation [1, 4] - [1, 26])
        macro: identifier [1, 4] - [1, 11])
        token_tree [1, 12] - [1, 26])
          string_literal [1, 13] - [1, 22])
          identifier [1, 24] - [1, 25])

On every keypress code is being reparsed by tree-sitter down the tree from modified leaf and coordinates change. We should update all changed region highlighters. If we could do so on every keypress we could have very nice highlighter with tree-sitter. There’s plugin already that uses tree-sitter to provide structural selections by @ulis, but I believe it doesn’t re-parse on every keypress, am I right?

It doesn’t on every keypress but it does on every tree-* command invocation. This plugin is stateless and doesn’t leverage incremental parsing.

However I use it to highlight scopes which means it’s called and does re-parse on NormalIdle and I didn’t notice any problems with it, tree-sitter is fast enough even in non-incremental mode for my case.

Here is interesting talk about Xi editor design. Its author mentions that syntax highlighter is regex based and acknowledges that it’s probably not the best technique.

This is a nice talk. What stood out for me is JSON RPC syntax highlighters, which might be something we could use.

I think that ultimate goal should be not just syntax higlighting, but editor understanding the code. Indentation, folding, and syntax are close friends. Local variable highlighting can be based on syntax tree as well, and so forth. Emacs does this for several languages and I find it super convenient.

We do have tree based highlighting using the region and group system, I am conflicted about exposing this outside of highlighting. It would solve some issues but greatly complexify the mental model of Kakoune commands (if we decide that text objects need to start/end in the same region type).

Improving the design of the region system to make it easier to express grammars would be worthwhile though.

3 Likes