Rewriting/reorganising Kakoune's documentation


#1

Kakoune has generally good documentation, and pretty much everything you need to know to use Kakoune interactively and write scripts is in there somewhere, but sometimes it can be hard to find the thing you want to know, or even to discover that it exists in the first place. For example, the commands page documents most commands, except for things like execute-keys, evaluate-commands, hook, and map, which have their own pages, except for exec and eval which share a page. None of those pages actually talk about how command arguments work, though - that’s command-parsing. command-parsing mentions expansions, but doesn’t go into detail since the expansions page is all about them, although it mentions parsing in return.

Just today somebody made a PR adding a bunch of details to the expansions page, and when I pointed out that it duplicated some of the information in the command-parsing page, they said:

I didn’t realize the command-parsing page contained that (I don’t think I’ve ever actually read it to be honest). The information [in this PR] is what I though was missing from expansions (and ended up learning from looking at the source code).

It seems to me that Kakoune’s documentation has grown organically, with things being added and amended over time as Kakoune has added and changed features, but without much attention to the larger structure.

So:

  • does anybody else agree that it can be difficult to find things in Kakoune’s documentation?
  • would anybody else like to see the documentation reorganised?
  • is anybody willing to contribute financially to reorganising the documentation?

If nobody has a problem with the state of Kakoune’s documentation, then fair enough. On the other hand, if there’s consensus that Somebody Should Do Something, I’ll start thinking about what a reorganisation might look like, when I can find the time. After all, I’ve already contributed to Kakoune’s documentation, and I’ve written long-form documentation before. If people are willing to contribute financially, I’m willing to make the time.


#2

When I first started Kakoune I just browsed through all doc pages, so now I know where to find stuff. I usually do that if I find a new tool.

Maybe some improvement is needed, but I would say that interconnecting the pages a bit more, like refer to the execute-keys , evaluate-commands , hook , and map pages from the commands page, would be a simple step to make it clearer.


#3

I think the overall organization of the pages is satisfactory, except when it comes to command-parsing, execeval and expansions.

The list of values in “expansions” should be moved to a “values” page, similar to options and registers. This page would simply explain the available values and their type
“command parsing” should be renamed to “strings” since it’s really what it’s all about. It would document how all the string types work and what are their usecases.
Then “expansions” should just explain what the various %…{} expand to and in which context, and link to options/registers/values.

Expansion and string could be merged into one page since the two topics are essentially interlinked

edit: just saw the part about financial contribution only now, I’m willing to do it


#4

Same here. But I would like to have ability to search over doc in a way how Vim does it. Vim has best doc system in my opinion.


#5

I’ve definitely found myself wondering about the organization of the docs pages. Reorganizing all the docs would be a worthy effort, but maybe an intermediate solution - create an “index” page and have :docs with no arguments start there.

It would serve as the top level documentation page that lists all of the current documentation pages, and short blurbs about what they cover. Once those blurbs are written, I think it’d become pretty clear where the overlaps are, or what pages need to be merged.

I realized I’m talking about creating a documentation page for documentation pages - more things to maintain - but it might be a lower effort means of making the existing docs more accessible to newer user’s who don’t know where to begin. Additionally, while the autocomplete listing of docs are nice, it’s not the most accessible for folks who don’t even know what things like “expansions” means.


#6

I like this plan. If you don’t have time to try it out in the near future, I might give it a shot.

Yeah, that’d be great. Of course, one reason Vim’s documentation is great is because they use a custom syntax with different markup for different cross references - so :foo is always a command, 'foo' is always a runtime option, +foo is always a compile-time option, and *foo* is a generic cross-reference. All that stuff is picked up by Vim’s documentation-indexer.

Kakoune uses Asciidoc syntax, which isn’t quite as convenient. As best I can tell, the syntax for making something a cross-reference anchor is [[like-this]], or even worse, anchor:like-this[]. That means Kakoune’s internal AsciiDoc renderer would need to filter out that text, and that means that jumping to a particular anchor gets more complex because you have to load the buffer and do the jump before it gets reformatted.

It should be possible, but it feels like it would be a lot of work.

That would be rather nice. It’d also be a good place to document features like “you can hit Enter on a cross reference to follow the link”.


#7

Vim uses tags for navigating in documentation. It generates ctags compatible tags per doc file, and later these tags are used in documentation for jumping. What is really cool is that Kakoune uses asciidoc, and ctags can uderstand asciidoc. And Kakoune understands ctags (ahem), so I wonder if Kakoune could utilize ctags for better navigation in docs. Basically we have everything we need for that.


#8

POSIX only requires ctags to support C and Fortran, and even Exuberant Ctags doesn’t support AsciiDoc. Universal Ctags apparently does, although it’s not listed in the docs.

Meanwhile, although Kakoune understands ctags, “This script requires the readtags command available in ctags source but not installed by default”.

While Kakoune depends on these third-party tools for good reasons, I don’t think it’s a good idea to have Kakoune’s built-in documentation system require them. Generally, Kakoune’s basic functionality requires only standard POSIX tools.

The ctags format is simple enough that we could probably write our own indexer, and the resulting tags file should be small enough that industrial-strength readtags isn’t needed for good performance, but we’d still have the problems of updating Kakoune’s AsciiDoc renderer, and making sure that looking up a tag jumps to the right spot before formatting is applied.


#9

Well, ctags support of asciidoc isn’t really needed in this case - tag files can be generated by kakoune itself, and for parsing we basically need a tool that can do regex search over file, such as awk or grep, which fits POSIX requirements.

True. I was astonished when I’ve compared readtags with grep performance on 607MiB file (4 million lines) - it is in order of magnitude faster compared to grep. But this difference isn’t so great for small files, and I suppose documentation tags will be relatively small.

I also think that this requirement can be wrapped by using posix tools combination in case readtags isn’t installed but that’s another topic.


#10
  • does anybody else agree that it can be difficult to find things in Kakoune’s documentation?

Exceptionally hard, and documentation is the lifeblood of a long-lived editor.

  • would anybody else like to see the documentation reorganised?

Yes, not even that picky about how – I feel most reworks would be an improvement.

  • is anybody willing to contribute financially to reorganising the documentation?

If it ended with inline, searchable, referable documentation, absolutely.

My question is simply, what would it cost: just to an order of magnitude? You could start a bountysource today if @mawww basically pre-agreed to merge the result (with obviously some information on where it was going).


#11

This is because readtags can do a binary search if the tags file is sorted, which is why Kakoune relies on readtags in the first place instead of grep its way through the tags.

More generally, we already support cross references in the documention (just hit enter on the links), we also have a rudimentary index system through the :doc command auto completion. I guess this could be expanded to support additional syntax, say :doc :command-name for prompt commands (with auto completion, as soon as you type :doc : the completion shows all commands in the documentation pages, same if you type :doc <key> to get a key documentation, and we just need to find a syntax for options…).

I am pretty proud that doc.kak provides the current system in only 165 lines of kak script (rendering a subset of asciidoc included), we have a bit of room to improve it before it becomes horribly complex.

As to the general organization of the pages, this grew up in an organic fashion, so I am all for a good rationalization of all that.

All that to say, I believe we are not that far from a good help system, we need to reorganize and cleanup the pages, and add a few features to the doc command completion system so that we can easily search for a command, key or option.


#12

Yeah, it should be easy enough to write a ctags-compatible indexer, and a custom lookup tool, but it does mean that Kakoune’s existing ctags support isn’t much help.

I originally mentioned financial support because I’ve done documentation writing before and felt pretty confident I could bang something out.

Since that time, however, I’ve been convinced by @occivink’s description of how the docs could be reorganised without a full rewrite, and the general tone here seems to be that searchability is a more pressing need. I’m a lot less confident about my ability to build a documentation searching system in Kakoune-script, so I don’t really want to take people’s money when I might not have anything to show for it at the end.


#13

Why it is so? The only change needed for current implementation to work with documentation tags (provided by some indexer) is to replace readtags with grep or awk, in case readtags isn’t installed. The rest could be used as is.


#14

Hi all,
I know this problem not as a programmer but a report writer. Over the past five years I have constructed 47 reports which equates to over 110,000 words using mainly tex/latex to pdf and MSWord/MacTeX/TexWorks as editors. Which are all no longer in the tool chain thanks to your efforts on the Kakoune modal editing, done right.

My problems are exactly the same as described in this thread. With multiple documents poor cross-referencing with duplication of attempts to find subject matter dependant on the day/month/year. I am currently believe that my solution lies with scribble not as a racket documentation tool but as a preprocessor for a PMBOK (<- make mum proud moment).

I am still in my first steps of scribble and will be continuing my education this semester with more reports giving greater opportunity to grasp scribble. Thought it was worth mentioning as a tool in this discussion thread and with your programming experience determining if it provides a solution for Kakoune documentation.

Whatever the out come is what’s one more report, sign me up. Happy to help on the Kakoune documentation. I am a part-time programmer with just three years’ experience. Make that two years, damn you, Haskell.

This would be my first experience contributing to an open source project so training wheels are tightly secure, with git man page on hotkey.

Obligatory links below:

srcibble
scribble/bnf
BNF Grammer examples
scribble/xref

The reference below is an overview of the tool chain: mouse event on the heading ‘6.1.2 Layer Roadmap’ to see section link. (<- mum that means click the mouse button.)

Flatt M, & Barzilay, E 2019, Scribble: The Racket Documentation Tool, Racket documentation series, v7.2, viewed 24 Feburary 2019,
https://docs.racket-lang.org/scribble/layers.html#(part._.Layer_.Roadmap).


#15

The bar for me is fairly low. I want to be able to do :h def or :doc def or something similar.


#16

You raise a good point @Screwtapello, I would like to donate to a collaborative expense that promotes the usability of Kakoune. I’m investing a considerable amount of time with the editor and finding myself with a list of features that I’m unable to quickly implement. Has there been any discussion on financially backing Kakoune and community-driven features?

I currently support Polar-Bookshelf on Open Collective which promotes open source projects and would subscribe monthly to Kakoune development.

If you’re unfamiliar with the site: “Open Collective is an online funding platform for open and transparent communities. We provide the tools to raise money and share your finances in full transparency.”

Yes, the site is open source with a github repo.

Asciidoctor is a good example on open-collective that everyone can relate to with the use of asciidocs for Kakoune documentation.

@robertmeta yes the sentiment of documentation as a secondary concern is widespread. This can be seen on thousands of github readme.md landing pages and with no considerable investment coming from Github (or a competitor) to inspire design. A user’s first experience with your hard work is the bland flat and unimaginative dull landscape of markdown in either light or dark grey and now continuously looping gifs. Sorry, bit of a rant and I too prefer reading an applications documentation inside the environment or pdf.