Hi, @scr!
I remember the experience I had when I learned Lua many years ago. Back then, I was mainly a C programmer with some skills on Perl and Python scripting. But nothing of that prepared me to Lua: it was the first time I was seeing lambdas, closures, coroutines and modules being first class values I could store in a variable and manipulate as I wish (much like OCaml modules, but even more flexible). It took me a long time until I fully understand all of that concepts.
Then I started learning purely functional languages (Haskell first, then Elm some years later). It was another breakthrough. At first I was fascinated but also very frustrated because I couldn’t manage to do even simple things like putting a printf to inspect some variable.
These two experiences were very important to me because they taught me to be a better person (not a better programmer but a better person). And they did so because I learned something valuable: programming is all about community, and when you enter a new community you may have to face a new culture, and whenever you face a new culture, you must leave behind whatever you think you know about the world, otherwise you won’t understand the new culture and you will never be able to embrace it.
Different cultures have different views of the world, have different cosmogonies, take different paths, and we shouldn’t expect they solve problems the same way we are used to.
Now you are entering the Kakoune community. Welcome! But be aware that to enjoy the experience at full, you must leave behind the old you and start to learn anew. Otherwise, you will experience to much pain to do even simple things, like the old me trying to learn new programming languages.
Writing scripts in Kakoune doesn’t need to be that hard, you just need to learn how to do it differently, from the perspective of another culture. For example, it’s true that Kakoune doesn’t have loops, but functional languages also don’t have them, and yet they are very powerful and elegant.
It’s also true that Kakoune doesn’t have a built-in programming language, but on the other hand it has a very expressive text editing language, in par with the one provided by the Vis editor, but way more powerful than what Vim and Emacs provide, let alone Atom, VSCode and Sublime Text. Together with its commands and expansions, it’s capable of doing a lot of sophisticated things without too much burden. But it takes time to fully appreciate it, because it demands a cultural change, and every cultural change demands a cognitive change, and that takes time. And it’s fine, since we are humans, not machines.
So let me present you some of the capabilities of such a sophisticated text editing language. And, since I just talked about functional languages, let’s see what we can learn from them that can be used here. You don’t need to know the concepts I’m going to mention here though, specially because the concepts we are going to see are already present in many imperative languages that took inspiration from the functional world, like SmallTalk, Pharo, Rust and Ruby.
As we are about to see, more often than not loops are an unnecessary complication.
Selections are functors
Sometimes, it’s hard to conceive a way to iterate over every element of a data structure to transform it. But, what if I simply don’t need to? What if I could just find a way to operate on a single element and the system magically uses this information to transform all of them? That’s what a functor is all about. Functors let me focus on the transformation of a single element extracted from the data structure, and its machinery apply this transformation to all of the elements inside the data structure for me, without the need for loops. It’s an operation called mapping.
Kakoune selections are functors. Let’s see how.
Mapping using keys
Suppose I want to surround every word construir with quotation marks in the following verse from João Cabral de Melo Neto:
A arquitetura como construir portas,
de abrir; ou como construir o aberto;
construir, não como ilhar e prender,
nem construir como fechar secretos;
construir portas abertas, em portas;
casas exclusivamente portas e tecto.
Well, the first thing I need to do is select (executing %sconstruir) all of these words:
A arquitetura como [construir] portas,
de abrir; ou como [construir] o aberto;
[construir], não como ilhar e prender,
nem [construir] como fechar secretos;
[construir] portas abertas, em portas;
casas exclusivamente portas e tecto.
For now on I’ll use the notation [something] to mean that something is selected. Now comes the question: how do I iterate over all of these selections to quote everyone of them? You know, Kakoune doesn’t have loops. Does it mean I must resort to %sh{} and use a shell loop over the $kak_selections expansion? Fortunately not. I can just forget for a moment I have a lot of selections and pretend I have just one. And I know how to surround a single selection with quotes: inserting a quote (i") at the beginning of the selection, leaving insert mode (<esc>), then appending a quote (a") at its end. And, suddenly, every selection is magically surrounded by quotes:
A arquitetura como “[construir”] portas,
de abrir; ou como “[construir”] o aberto;
“[construir”], não como ilhar e prender,
nem “[construir”] como fechar secretos;
“[construir”] portas abertas, em portas;
casas exclusivamente portas e tecto.
No loops required.
Mapping using an external tool
Now something a bit trickier. Suppose I have a list of numbers I would like to increment:
[12], [16], [18], [22], [28], [30]
Kakoune doesn’t know how to count. How can I do that? The secret is the | key. It allows me to pipe each selection to an external program and write the result in its place. So let’s try executing |xargs echo "1 + " | bc. Here, the idea is to append each selection to the expression 1 + and then send this string to the bc calculator. Does it work?
[13], [17], [19], [23], [29], [31]
Wow! Prime numbers!
Note that the external tool didn’t need to iterate over the values of the selections. From the bc point of view, there’s only one expression, not a list of them. Kakoune managed to extract one item at a time and send it the pipeline.
Let’s try the same thing using Lua this time: |lua -e 'print(tonumber(io.read()) + 1)'. Same thing: it just works. No loops.
Functors are nice.
As an aside, judging only by the text in your first post in this topic, this is exactly the functionality you were looking for.
Selections can be filtered
Suppose I have the following code with all lines selected:
[zip : Tree node leaf -> Zipper node leaf]
[zip tree =]
[ ( tree, [] )]
[]
[]
[unzip : Zipper node leaf -> Tree node leaf]
[unzip =]
[ goToRoot >> subtree]
[]
[]
[subtree : Zipper node leaf -> Tree node leaf]
[subtree ( tree, _ ) =]
[ tree]
I want to keep only the selections with a type annotation. How do I do that?
In the imperative way of thinking, I would need to iterate over each selection, matching its text against the regex ^\w+ : and keeping the ones that match. Those with a functional background know that I can use a function like filter to do the job. The equivalent in Kakoune is the <a-k> key. Let’s try it executing <a-k>^\w+ ::
[zip : Tree node leaf -> Zipper node leaf]
zip tree =
( tree, [] )
[unzip : Zipper node leaf -> Tree node leaf]
unzip =
goToRoot >> subtree
[subtree : Zipper node leaf -> Tree node leaf]
subtree ( tree, _ ) =
tree
Success!
And there’s also <a-K>, that clears all the selections that match the provided regex. It’s the complement of <a-k>.
Filtering using an external tool
But what if I have the following text from Guimarães Rosa:
[Só] [outro] [silêncio]. [O] [senhor] [sabe] [o] [que] [o] [silêncio] [é]? [É] [a] [gente] [mesmo], [demais].
and want to keep selected only the words with at least six characters? Complicated… Kakoune not just can’t count, but it’s even worse: it can’t decide, meaning it can’t use ifs and elses to make decisions and take branches. What now?
Let me present you the excelent $ key. With it, I can pipe each selection to an external tool and Kakoune will keep those selections for which the shell returned 0. Let us try: $lua -e 'if #io.read() >= 6 then os.exit(0) else os.exit(1) end'.
Só outro [silêncio]. O [senhor] sabe o que o [silêncio] é? É a gente mesmo, [demais].
Selections are monads (sort of)
We have just seen how mapping applies automatically on all selections operations that take some text (inside a selection) and give some text back. But what if my operation, instead of returning some text, returns even more selections? Monads to the rescue!
Let’s first see an example. Consider the following verses from Fernando Pessoa:
[O poeta é um fingidor.]
[Finge tão completamente]
[Que chega a fingir que é dor]
[A dor que deveras sente.]
Now, if I do S , I’m communicating to Kakoune that I want to split each selection on its spaces. Kakoune then goes over each selection for me and apply this operation as if I was operating on a single selection. But the result of such an operation is itself a collection of selections. So, I’d expect the result to be something like this:
[[O] [poeta] [é] [um] [fingidor.]]
[[Finge] [tão] [completamente]]
[[Que] [chega] [a] [fingir] [que] [é] [dor]]
[[A] [dor] [que] [deveras] [sente.]]
But Kakoune knows how to deal with that, and instead removes these nested structures, giving back a flat collection of selections. This allows us to select inside other selections, and that’s an incredible powerful concept!
Let me illustrate how powerful this concept is. Consider I have the following scenario:
- a Markdown file containing many sections;
- at each section, I have many code snippets in Elm and Lua;
- inside a section named “Meus algoritmos maravilhosos”, I want to select all the comments from Lua snippets, but not from Elm snippets.
It’s tricky, because both Lua and Elm comments start with -- . Here are some steps to do it using selections inside selections:
- select the whole buffer:
%;
- select the right heading inside the whole buffer:
s^# Meus algoritmos maravilhosos;
- extend the selection until the next heading:
?^# ;
- select the beginning of all Lua snippets inside this selection:
s^```lua ;
- extend the selections until the closing of the snippet:
?^``` ;
- split the selections on line boundaries:
<a-s>;
- keep those starting with a Lua comment mark:
<a-k>--
Selections are monoids
Selections can be merged together in several ways. There’s the <a-_> key and all the ways marks can be combined. Any data structure that has a notion of merging is called a Monoid. A monoid is useful for many things, but let’s try to find an interesting and simple example.
Suppose I have the following Lua code:
local x = 17
if something then
x = 19
end
local t = { key = x }
if y < 21 then
z = something or 3
t.other_key = y + z
end
Now I want to select these two if blocks. Using marks, I can do the following:
- select the whole buffer (
%);
- select every
if keyword (sif );
- mark these selections (
Z);
- again, select the whole buffer (
%);
- select every
end keyword (send);
- keep a union of the current selections with the ones previously marked (
<a-z>u).
Done!
Selections can be manipulated like lists
Lists are ubiquitous in functional languages. It’s because they have some nice properties, among which the fact that they can be easily splitted in a head (the first element) and a tail (the remaining elements), something that plays nicely with recursive algorithms.
The interesting thing is that selections also have a head (the main selection) and a tail (the remaining selections) and so it should be possible to implement recursive algorithms with them.
Let’s try to build one, just for fun!
I want to define a command called reverse, that reverses all the lines of a buffer (I mean: the first line becomes the last one and so on). How hard can it be?
This is my attempt:
define-command revert -docstring "Os últimos serão os primeiros!" %{
execute-keys <percent><a-s> # Select the whole buffer and split on line boundaries
revert-the-lines # The actual recursive command
}
define-command -hidden revert-the-lines %{
# Use a `try` to avoid raising an error when there are no more selections left
try %{
# - run `execute-keys` in a draft context to avoid loosing the current selections
# - reduce selections to just the main one (`<space>`)
# - cut its content (`d`)
# - go to the end of the buffer (`gj`)
# - paste the line there (`p`)
execute-keys -draft <space>dgjp
# remove the head (the main selection)
execute-keys <a-space>
# recurse, this time with only the remaining selections
revert-the-lines
}
}
For a great explanation of why the -draft switch was needed, let @alexherbo2 enlighten us.
Can this recursive command possibly work? Well, let’s try it: :reverse
}
}
revert-the-lines
# recurse, this time with only the remaining selections
execute-keys <a-space>
# remove the head (the main selection)
execute-keys -draft <space>dgjp
# - paste the line there (`p`)
# - go to the end of the buffer (`gj`)
# - cut its content (`d`)
# - reduce selections to just the main one (`<space>`)
# - run `execute-keys` in a draft context to not loose the current selections
try %{
# Use a try to avoid raising errors when there are no more selections left
define-command -hidden revert-the-lines %{
}
revert-the-lines # The actual recursive command
execute-keys <percent><a-s> # Select the whole buffer and split on line boundaries
define-command revert -docstring "Os últimos serão os primeiros!" %{
Q.E.D.
That’s… amasing!! It really works!!
Keep your mind open
First of all, I must enphasize that everything we achieved here was done without any %sh{} or lua %{} blocks. And we achieved a lot. Kakoune’s text editing language is so powerful we simply didn’t need a programming language for the tasks we’ve presented here.
Then, a very personal commentary: I don’t see %sh{} as the Kakoune’s programming language. I like to see it as a message. With it, Kakoune is saying: “I desire to communicate with my surroundings”. Kakoune is saying it doesn’t want to be a selfish black box and instead want to be a part in something bigger. The %sh{} expansion is a brigde to the outside world. And that’s beautiful.
From within Vim we can use external tools to manipulate text, but no editor I’m aware of has such a flexible mechanism to integrate it to its surroundings as Kakoune has. I’m not capable of writing luar in another editor spending less than 100 lines of code, exactly because they lack something like %sh{}.
I know you don’t like POSIX shell scripting. Me neither. But that’s OK, because we can use tools like luar. I would never write the logic of a plugin using shell scripting, but just because people do things in a way I’d never do doesn’t mean they are ugly, barbarians and evil. We must keep our mind open to be able to embrace new cultures. That’s the beauty of diversity.
Practical considerations
I’m not an experienced plugin developer and can’t offer good advices on that regard. But I like to observe and listen, and I’d say a good cultural practice inside the Kakoune community is relying on the powerful features of the Kakoune API, resorting to a programming language just on some special cases where the API is not enough. Take for instance this plugin from @mawww and see how little of %sh{} usage it needs. Maybe this way you can avoid much of the burden you are currently having.
). I think lua is a good middle ground between the two worlds. I hope the use of lua will be more used in the kakoune community ( mostly plugin makers ) and that one day it would be part of kakoune in a more official way.
But I’ll take your suggestion into consideration.
