A :lua command for Kakoune

Plugins
1

Kakoune doesn’t have an embeded scripting language. Instead, it defines a clever extension model in which you rely on external tools to do all sorts of work. In particular, to execute some programming logic, it has the built-in %sh{} expansion.

As a user of the fish shell, I’m not used to the idiosyncrasies of standard Unix shell programming. So I’ve been missing a programming language I’m more familiar with when writing my kakrc.

This is why I’ve written a very simple plugin to Kakoune called luar. It defines just a single :lua command to execute lua code from within Kakoune. It’s not meant to be a bloated binding to Kakoune’s internals. It’s just a simple complement to the %sh{} expansion to be used whenever you need to write some logic.

7 Likes
2

This is really cool! We had a thread previously discussing the need of real scripting language: Kakoune needs a real scripting language and in it there was several lua examples you may find interesting.

I’ve defined this abomination https://github.com/andreyorst/dotfiles/blob/master/.config/kak/defperl.kak but never actually used it, because I mostly fine with shell script. Some of my plugins do use Perl, but I just call functions from files directly via shell. But your solution seems like a quite nice library

3

Thanks!

One thing I’ve found curious about that thread is that people started to mix two different decisions:

  1. Have a scripting language built in.
  2. Expose traditional bindings (like Emacs and Vim do) to Kakoune’s internals using that scripting language.

This plugin shows they can be thought independent of each other.

I don’t think Kakoune would benefit from an implementation of the second decision. For me, this is clear.

But I would like to see a good discussion about the pros and cons of the first decision. Perhaps, during that discussion, the community find that this is not a good path to take, but I still didn’t see good arguments in favour or against it. Just arguments mixing the two different decisions.

4 Likes
4

I am becoming less fine with them by the day, as for reasons I have yet to track down they will just suddenly stop working. There was a great spelling one I got from here, that suddenly stopped working and I haven’t yet dug in to debug.

3 Likes
5

Multiple great points.

Agreed, and as you pointed out, they are best carried on as two completely seperate discussions. The reasoning (as I have understood it) for not embedding a script language is well – the platform has script languages.

One interesting thing is how differently those two editors approach this, vim having a kakoune alike (the command is the command) and emacs having “the command calls the API” – which in and of itself is really a 3rd discussion!

Very cool binding, as someone who was a huge fan of tier 1 support for Lua in Neovim, this is something I will have to dig into.

6

I like sh as a glue script to external tools. It encourages to write Kakoune and text editor independant tools.

I don’t like sh because of the string hell escaping and slowness. Talking back to Kakoune with perfect escaping is difficult, and running multiple shell commands to have their outputs is slow.

I’m not a fan of Lua, but it elegantly solves the two problems above.

What I’m afraid of, is having a real programming language encourages users to extend the editor with it instead of using it as a glue to external tools.

sh is ugly for sure, but it is a good constraint – and rail – for the creativity and composability (because it would be too ugly and slow to implement that and that in pure sh :D, we rely on external and language independant applications). On another topic, the same idea can be transposed for synchronous vs. asynchronous highlighters.

2 Likes
7

debugging shell scripts is really painful indeed. The most complex plugins I wrote were plug.kak and fzf.kak, and I’m really scared to change those much

8

ahem, shit, ahem

9

sh is ugly for sure, but it is a good constraint – and rail – for the creativity and composability (because it would be too ugly and slow to implement that and that in pure sh :D, we rely on external and language independant applications).

This is an odd reasoning, but a legitimate one. So, apparently, these are the arguments in favor of using a POSIX shell for scripting:

  • It’s portable to any POSIX OS.
  • Being a glue language, it encourages you to rely on external tools to do all sorts of work, thus enforcing composability.
  • Being such a horrible language, no one wants to use it except for the bare minimum. Again, enforcing composability. In fact, the claim that Kakoune does not have a scripting language of choice is not very accurate, since it does have one (the POSIX shell), although not built-in; it’s just that it is such a pain to use in complex programs development that developers prefer to use something else.

So, let’s start to thing about alternative ways to enforce the same discipline without to much burden for plugin developers.

At the moment, I can see two possible alternatives I want to discuss with you.

Restrict the environment

Suppose Kakoune embeds a scripting language. It, as the host environment, can restrict what a script is able to see. I’m going to use Lua as an example, but I thing the same arguments hold for other scripting languages.

One very nice property of Lua is that it’s designed from the beginning not to be an interpreted language, but a scripting language. It means its design reflects the fact that it is conceived to be embedded in a host environment, being the Lua interpreter just an example of such a host. This is one of the reasons for its minimalism (the entire Lua C library has just a hundred kilobytes). As such, a Lua script can access the outside world only through the host, and the host has total control of what is exposed to the script.

For example, by default, its C API doesn’t expose any Lua module to the script. The host has to explicitly tell what it want the plugin to see.

So, one way to have Lua as a scripting language while still enforcing a Unix discipline is to load only the minimum set of modules necessary for the plugin to communicate with the POSIX environment (like the io and os modules). Or even providing a custom (more POSIX-like) module for accessing the OS, since the desire to be extremely portable and minimalist makes Lua do not provide many tools for interacting with the operating system.

Another interesting aspect is that the require statement used to import modules is not really a statement, but a function. Being a function: it can be configured to just load modules from a specific PATH Kakoune has control of; it can be replaced by a custom one; or we can even remove the require function altogether, making the plugin unable to use any module not explicitly provided by the host.

This way, we gain:

  • a better scripting language;
  • a way faster one;
  • a portable one, because it’s embedded in Kakoune itself;
  • a programming environment that enforces a composability discipline as much as we want.

Again, I’m using Lua as an example because I know it better (and I know it enough to say that it also has many aspects I don’t like at all). I’m sure you can list other languages with similar characteristics, and I really don’t care if another one would be chosen (in the case Kakoune decides to embed a scripting language). I don’t have a beloved programming language and actually like to learn new ones (provided they are not as arcane as bash scripting =D).

Provide a client-server model

A very interesting scripting model is the one adopted by bspwm. It has two programs: the actual window manager (bspwm) and a configuration program (bspc). The later communicates with the former using a socket, and it is used to send to bspwm whatever messages necessary to control it.

It has some subcommands, among which bspc query, to inspect bspwm’s state, bspc config, to set configuration options, and bspc subscribe, to listen to events.

A configuration file is just an executable file bspwm executes at initialisation. Here is mine:

#! /usr/bin/fish

sxhkd &
picom --experimental-backends &

bspc monitor eDP1 -d 1 2 3 4
bspc monitor HDMI1 -d 5 6 7 8

bspc config initial_polarity first_child
bspc config border_width 1
bspc config presel_feedback_color "#0b2f4f"
bspc config focused_border_color "#eff0f1"
bspc config normal_border_color "#2f353a"
bspc config window_gap 6
bspc config gapless_monocle true
bspc config click_to_focus true
bspc config pointer_follows_monitor true
bspc config remove_disabled_monitors true
bspc config remove_unplugged_monitor true

bspc rule -a plasmashell state=floating border=off layer=normal
bspc rule -a krunner border=off state=floating layer=above

My bspwmrc is very basic and declarative, but it can have arbitrary logic in it if needed.

One nice property is that bspwm is idempotent regarding the configuration. So, to reload the config file, one just needs to execute it again.

This models allows bspwm to adhere more closely to the Unix philosophy. For instance, it doesn’t process keyboard shortcuts, relying on external programs (such as sxhkd) to do so. So, a sample sxhkdrc looks like this:

# Alternate between the tiled and monocle layout
super + m
	bspc desktop -l next

# Swap the current node and the biggest node
super + g
	bspc node -s biggest.local

# Cancel the preselection for the focused desktop
super + ctrl + shift + space
	bspc query -N -d | xargs -I id -n 1 bspc node id -p cancel

Note how every command is just a simple shell command line invocation. Also, note how clever this model is: since the configuration file is just an executable, bspwm doesn’t need to parse it; it doesn’t even need to read it, just execute it. From bspwm perspective, its content is irrelevant. As such, I can write my config file in whatever language I want, even compiled ones. It’s written in fish, but could also be in Lua:

#! /usr/bin/env lua

-- A bspwmrc written in lua

os.execute("sxhkd &")

-- ...

os.execute("bspc config initial_polarity first_child")
os.execute("bspc config border_width 1")

-- ...

Or, perhaps, Ruby:

#! /usr/bin/env ruby

# A bspwmrc written in ruby

`sxhkd &`

# ...

`bspc config initial_polarity first_child`
`bspc config border_width 1`

# ...

This is truly a language agnostic plugin model. Kakoune, instead, is tied to POSIX shell scripting more than we usually admit.

It also has the benefit to abstract away both the handling of communication with the server and the protocol to do so. If it uses json, ZeroMQ, raw strings, etc., for the user it doesn’t matter, and frees the bspwm developers to change to another implementation if needed without compromising the API.

I don’t know if this is a model that can be used by Kakoune, but if it could, it would be a big improvement.

3 Likes
11

I’m not sure I’ve understood it…

12
13

Not quite. Many scripts that work in posix sh on Linux will not work in poxis sh on BSD due to bugs in sh itself. I had some experience with my plugins not working on BSD because of how its shell put processes to background

14

Well done plugin! I have to come up with some nice uses for it.

We kinda have it with Kakoune - echo "cmd" | kak -p SESSION. You can execute this command from any programming language skipping shell (and avoiding painful shell escaping issues). Transferring data out of Kakoune is cumbersome unfortunately. Env variables are clunky and most reliable way (I think) is echo -to-file since it avoids shell altogether.

15

Then there’s yet another unanticipated problem in the current situation…

16

Hi, @TeddyDD! Thanks for the suggestion. Unfortunately, it works just until you realise you need the SESSION information in the first place. How do you get it? Additionally, I can’t write my kakrc in whatever language I want (like with my bspwmrc), because Kakoune parses it and so it needs to understand it. But let’s try to elaborate it a bit further to see what we get.

Perhaps what we need is an opaque kakrc together with opaque command definitions.

Suppose for a moment that, like bspwm, we have an hypothetical kakc client, used to communicate with the server (it can be a kak -p with some additional features). Also like bspwm, suppose Kakoune doesn’t read kakrc, just executes it. In this scenario, kakrc is opaque to Kakoune and so can be written in any language, say, Julia:

#!/usr/bin/env julia

# This is `kakrc` written in Julia

`kakc set-option global grepcmd ag`
`kakc set-option global aligntab true`

`kakc define-command a-new-command -params 2 mycommand`

If, then, the command a-new-command is invoked like this: :a-new-command \d+ 17, Kakoune does something along these lines:

export KAKOUNE_SESSION="the session number"
export KAKOUNE_CLIENT="the client from which the command was invoked"
mycommand '\d+' '17'

It allows us to write the custom command as an executable file called mycommand written, say, in Lua:

#!/usr/bin/env lua

-- This is file `mycommand`

local param1, param2 = arg[1], arg[2]
local register = io.popen("kakc query register 1"):read()
local text = register:gsub(param1, param2)
os.execute("kakc add-highlighter window/mycommand dynregex " .. text .. " 0:default,+u")

There are many informations in the above examples. Let’s start with the first one:

  • kakc subcommands are named after the commands they invoke;
  • a custom command is nothing more than an external executable (in this example, called mycommand).

When the new command is invoked (see the second example):

  • Kakoune executes the associated executable, setting some environment variables needed to identify the client and session from which the command was invoked;
  • whatever parameters are passed to :a-new-command are then passed to the executable as command line arguments (like luar currently does).

In the mycommand file:

  • parameters can be accessed as command line arguments;
  • Kakoune’s state is accessible via a new kakc query subcommand;
  • the plugin doesn’t need to manage session information; since everything is already set in environment variables, kakc itself can automatically read them, freeing plugin authors from handling them.

By doing so, both kakrc and command definitions would be opaque to Kakoune and, as such, all code related to parsing .kak files could be deleted from Kakoune, thus reducing the code base.

Is it enough? Or am I missing some complications? More importantly, is it feasible?

2 Likes
17

There’s a $kak_session environment variable that is being exported via Kakoune inside %sh{} blocks

The problem is, that there is no truly crossplatform solution. Our best bet is virtual machines, that encapsulate common part between all systems while providing the same interfaces for all different operating systems thus sacrificing performance. But even java has cross platform problems.

VM based languages generally better at cross platform apps though, and that’s why I mostly use Perl for my plugins, since it is available for majority of systems where Kakoune runs, and seems to be pretty the same across those. I don’t know how good Lua in this regard, but I know that it uses libraries written in C, which probably can be a portability problem. I also think that this is why python is popular for plugin development across various editors.

Given that we have a way to escape Kakoune to shell we can use any language for scripting, as long as we escape strings well. Luckely for us Kakoune supports arbitrary delimiters for strings so we can easily use non-ascii characters like: echo %sh🦀 echo "{}[]()''\"\""🦀 and don’t worry much about escaping things.

18

Lua is written in Clean C (a subset of ANSI C) exactly to enable porting it to as many platforms as possible. This is why its os library is so restricted. This deliberate decision probably makes it a better solution than alternatives regarding portability.

I mean we still need to rely on %sh{} to get expansions, so we can’t approach the problem the bspwm way even with kak -p

19

You can save session by doing echo -to-file /tmp/kak-session %val{session}. I get your point though.

20

The truth is that I have little experience writing plugins for Kakoune. You are at a much better position to say what works and what doesn’t :smile: I was just trying to take into consideration @alexherbo2’s concerns. For my personal needs, the :lua command is the missing piece, everything else works well enough.

That said, I must say I like the idea of having opaque commands .

That’s the main problem with Lua. One can’t expected it to be shipped by default on any OS. On the other hand, since it’s so small (the interpreter for Linux 64 is ~ 250 KB), if this turns out to be a problem, I can always ship it bundled with any plugin I make, or even download it on demand.

This is something very useful indeed.

1 Like
21

You’ll be good at it in no time :smiley: