A :lua command for Kakoune

Plugins
11

I’m not sure I’ve understood it…

12
13

Not quite. Many scripts that work in posix sh on Linux will not work in poxis sh on BSD due to bugs in sh itself. I had some experience with my plugins not working on BSD because of how its shell put processes to background

14

Well done plugin! I have to come up with some nice uses for it.

We kinda have it with Kakoune - echo "cmd" | kak -p SESSION. You can execute this command from any programming language skipping shell (and avoiding painful shell escaping issues). Transferring data out of Kakoune is cumbersome unfortunately. Env variables are clunky and most reliable way (I think) is echo -to-file since it avoids shell altogether.

15

Then there’s yet another unanticipated problem in the current situation…

16

Hi, @TeddyDD! Thanks for the suggestion. Unfortunately, it works just until you realise you need the SESSION information in the first place. How do you get it? Additionally, I can’t write my kakrc in whatever language I want (like with my bspwmrc), because Kakoune parses it and so it needs to understand it. But let’s try to elaborate it a bit further to see what we get.

Perhaps what we need is an opaque kakrc together with opaque command definitions.

Suppose for a moment that, like bspwm, we have an hypothetical kakc client, used to communicate with the server (it can be a kak -p with some additional features). Also like bspwm, suppose Kakoune doesn’t read kakrc, just executes it. In this scenario, kakrc is opaque to Kakoune and so can be written in any language, say, Julia:

#!/usr/bin/env julia

# This is `kakrc` written in Julia

`kakc set-option global grepcmd ag`
`kakc set-option global aligntab true`

`kakc define-command a-new-command -params 2 mycommand`

If, then, the command a-new-command is invoked like this: :a-new-command \d+ 17, Kakoune does something along these lines:

export KAKOUNE_SESSION="the session number"
export KAKOUNE_CLIENT="the client from which the command was invoked"
mycommand '\d+' '17'

It allows us to write the custom command as an executable file called mycommand written, say, in Lua:

#!/usr/bin/env lua

-- This is file `mycommand`

local param1, param2 = arg[1], arg[2]
local register = io.popen("kakc query register 1"):read()
local text = register:gsub(param1, param2)
os.execute("kakc add-highlighter window/mycommand dynregex " .. text .. " 0:default,+u")

There are many informations in the above examples. Let’s start with the first one:

  • kakc subcommands are named after the commands they invoke;
  • a custom command is nothing more than an external executable (in this example, called mycommand).

When the new command is invoked (see the second example):

  • Kakoune executes the associated executable, setting some environment variables needed to identify the client and session from which the command was invoked;
  • whatever parameters are passed to :a-new-command are then passed to the executable as command line arguments (like luar currently does).

In the mycommand file:

  • parameters can be accessed as command line arguments;
  • Kakoune’s state is accessible via a new kakc query subcommand;
  • the plugin doesn’t need to manage session information; since everything is already set in environment variables, kakc itself can automatically read them, freeing plugin authors from handling them.

By doing so, both kakrc and command definitions would be opaque to Kakoune and, as such, all code related to parsing .kak files could be deleted from Kakoune, thus reducing the code base.

Is it enough? Or am I missing some complications? More importantly, is it feasible?

2 Likes
17

There’s a $kak_session environment variable that is being exported via Kakoune inside %sh{} blocks

The problem is, that there is no truly crossplatform solution. Our best bet is virtual machines, that encapsulate common part between all systems while providing the same interfaces for all different operating systems thus sacrificing performance. But even java has cross platform problems.

VM based languages generally better at cross platform apps though, and that’s why I mostly use Perl for my plugins, since it is available for majority of systems where Kakoune runs, and seems to be pretty the same across those. I don’t know how good Lua in this regard, but I know that it uses libraries written in C, which probably can be a portability problem. I also think that this is why python is popular for plugin development across various editors.

Given that we have a way to escape Kakoune to shell we can use any language for scripting, as long as we escape strings well. Luckely for us Kakoune supports arbitrary delimiters for strings so we can easily use non-ascii characters like: echo %sh🦀 echo "{}[]()''\"\""🦀 and don’t worry much about escaping things.

18

Lua is written in Clean C (a subset of ANSI C) exactly to enable porting it to as many platforms as possible. This is why its os library is so restricted. This deliberate decision probably makes it a better solution than alternatives regarding portability.

I mean we still need to rely on %sh{} to get expansions, so we can’t approach the problem the bspwm way even with kak -p

19

You can save session by doing echo -to-file /tmp/kak-session %val{session}. I get your point though.

20

The truth is that I have little experience writing plugins for Kakoune. You are at a much better position to say what works and what doesn’t :smile: I was just trying to take into consideration @alexherbo2’s concerns. For my personal needs, the :lua command is the missing piece, everything else works well enough.

That said, I must say I like the idea of having opaque commands .

That’s the main problem with Lua. One can’t expected it to be shipped by default on any OS. On the other hand, since it’s so small (the interpreter for Linux 64 is ~ 250 KB), if this turns out to be a problem, I can always ship it bundled with any plugin I make, or even download it on demand.

This is something very useful indeed.

1 Like
21

You’ll be good at it in no time :smiley:

22

Is there an equivalent to io.popen to run a process with the command and arguments?

Something similar to Process.run of Crystal.

23

You can just pass the full command (including arguments) to the first argument of io.open:

local buffers = io.popen("kcr get %val{buflist}"):read("a")
24

Passing the full command (including arguments) means to correctly escape the arguments.

I would rather like to not deal with the escaping.

25

Hmm… I didn’t get it. Could you give me an example?

26

If you have a file named, say, read me first.txt, and (for some reason) you try to count the lines by feeding it through wc:

io.popen("wc -l read me first.txt")

…then you’ll get a bunch of errors:

wc: read: No such file or directory
wc: me: No such file or directory
wc: first.txt: No such file or directory

To actually read the file, you need to quote the filename:

io.popen("wc -l 'read me first.txt'")

…or escape the spaces:

io.popen("wc -l read\\ me\\ first.txt")

That’s perfectly possible, but it’s kind of a frustrating API, since it makes doing the wrong thing easy and the right thing difficult. I don’t know what facilities Lua has available, but for example Python’s equivalent API lets you pass a list of strings and avoid the quoting issue entirely:

subprocess.Popen(["wc", "-l", "read me first.txt"])

If you get the filename read me first.txt from somewhere, you can stick it straight on the end of the list of command-line arguments, and be sure it won’t be molested or misparsed and wc will receieve it exactly as intended, no extra quoting or escaping required.

1 Like
27

I see… this is where the lua focus on being a lightweight embedded scripting language pays its price. There are many useful packages you can install to make scripts communicate with the outside world. I like this one: luachild - LuaRocks. Simple and effective. But the built-in tools are quite limited.

One workaround is to define a wrapper:

-- Define execute as a varargs function
function execute(...)
    local args = {}
    -- Iterate over the arguments
    for _, arg in ipairs({...}) do
        -- The %q format quotes strings as needed
        args[#args+1] = string.format("%q", arg)
    end
    return io.popen(table.concat(args, " ")):read("a")
end

Now, you can use it this way:

local lines = execute("wc", "-l", "read me first.txt")
28

Yes, some thing like that would definitely do the trick — although as a minor nitpick, %q isn’t exactly right: it quotes things according to Lua’s quoting rules, not POSIX shell’s quoting rules. For example, it doesn’t escape $ or backticks, which are important. I’m sure it wouldn’t be difficult to add a custom shell-quoting helper to your Lua plugin, though, just like I’m sure you already have a custom Kakoune-quoting helper.

29

and these rules are kinda stupid tbh…

30

You are right :sweat_smile:

It quotes each argument of a command. As far as I can anticipate, it’s enough.

@alexherbo2, going back to your question, perhaps the best advice I can give you is to keep in mind that the lua command is meant to be a complement of the %sh{} expansion, not a replacement of it.

Personally, I tend to use the lua command when I want to run some logic in my scripts (something I find cumbersome to do with shell scripting) and %sh{} when I want to run external tools, except for simple enough cases. Using it in this manner, the lua language is a good fit, for its speed, small footprint and portability. You can even combine both in many cases:

lua %sh{wc -l "read me first.txt"} %{
    local count = arg[1]
    ...
}

Unless you want to get realtime expansions from within a lua block using kcr, like @scr once asked me, in which case we need to find a good solution.

1 Like