Kakoune needs a real scripting language

I don’t know many Perl based, but kakoune-snippets and kakoune-gdb use perl. I’m using Perl in some of my plugins, for instance langmap.kak uses it to generate mappings, and plug.kak to sort paths by depth. Kaktree uses Perl quite heavily too

How does this adhere to the philosophy? I actually think the philosophy is adhered to very clearly by the way Kakoune is doing it now. That said, I am not sure the adherence to that philosophy is always a good decision in the face of pragmatism.

Which violates another Kakoune design goal of Unified interactive usage and scripting – if you want there to be a command like indent-cursor-line – or perhaps you just meant the commands to do that would go there?

Agreed. That said, I am not sure we are in a worse place than viml (vimscript) but it can be a dark, confusing, escape oriented slog.


I think rather than focusing on trying it go against the design goals, we should try to improve on the pain points. For example, if you like Lua or Python – how could we make it easier to interact with them in a performant way? Where are the wins we can do without violating the core of Kakoune?

One idea could be to allow declaring custom expansion types.

declare-expansion lua '/usr/bin/lua'

evaluate-commands %lua{ print("set-option makecmd '" .. os.getenv'kak_buffile' .. "'") }

Of course, this is just a shorter and more efficient version of %sh{!#/usr/bin/lua ...}, so maybe the value add is not large enough…

1 Like

Would be interesting in languages that have repl or some way of keeping the process running. If we could use a command like that to avoid startup cost of the process might be worth having, which would make it a thicker wrapper than just %sh{!#…}

1 Like

I honestly think there are more options to consider before adding a scripting language to Kakoune. Since the idea here has been to leverage POSIX, and POSIX has a scripting language, a better idea would be to give shells more access, via POSIX-y means.

For example, if Kakoune started storing its registers in $XDG_RUNTIME_DIR/kak/<session-name>/registers/<regname>, and reading these files when it needed the values, then shell scripts (and other external programs!) could manipulate the register contents. This could remove features from the kakscript (presuming it is done well), which is more inline with the stated design philosophy of minimalism and integration with other tools.

Registers, specifically, might be terrible idea; however, it is clear that if we are creative, we can come up with solutions in this space without adding a built-in scripting language. In fact, I find Kakoune inspiring in the interesting ways it has leveraged POSIX to do less with more.

And why am I opposed to a built-in scripting language? Because it would kill Kakoune, as a project. There are only a few developers to maintain the code here, and they could easily get sucked into maintaining a scripting language when they should be thinking about how to edit text. Just as in business, a project should pick one thing to do and do it well, and the rest should be outsourced.

4 Likes

I have been for a long time on the fence about a suggestion by @alexherbo2: have alternate names for normal mode keys, such as <keep-matching>, <split-on-regex>, <align>

I think this would go a long way into making normal mode scripts much easier to understand, but would unfortunately decorelate the interactive keystroke from the scripting command name (although using the key directly should still work, the idea would not be to introduce an indirection from a to <append>, they are the same key.

As said initially, I am not sold on this, but maybe it would be a good direction to explore.

Isn’t the next natural step to have alternative names for everything… then you might as well bind keys to those names? I hate to invoke the slippery slope argument, and I am not even convinced it is the wrong decision, it just seems like doing it just for normal commands would be an inconsistent line in the sand.

I think the argument for there being one true definition of a thing is still valid, and I am not sure if it is actually the most confusing thing. Speaking personally, I am far more likely to get stuck on escaping (or forgetting to escape) something than anything else.

I think scripting the editor is an issue, but obviously not a crippling one, we have a lot of great plugins.

@eraserhd’s very P9 inspired idea of allowing a lot of work to be done via a mock filesystem is interesting and possibly very complete solution to a lot of hard problems, from history management to recent files to register handling to (you name it) – again following the P9 path.

Another interesting idea (I think) would be building on @bugabinga’s declare-expansion idea, possibly allowing the thing to be run persistently for languages with like a repl, so your plugin could do declare-expansion lua-for-plugin '/usr/bin/lua' and that would basically just shove what is run onto the repl and get whatever the result is back. Would like the script hide some state in the running lua.

Yeah I kinda liked declare-expansion idea, which would turn a some of %sh{perl -e ' ... ' } into simple %perl{ ... } which also will allow to use single quotes inside Perl expansion without pain and properly highlight Perl code inside in the same way how it is done for %sh{} expansions.

@eraserhd you look at http://man.cat-v.org/plan_9/4/acme – a lot of great ideas to be borrowed.

EDIT: The more I think about it – the more I think @eraserhd has hit on a fundamentally amazing idea that would make communicating with Kakoune from the shell amazing rich, powerful and shell friendly. I think a rich file system interface is the solution to almost every problem I have had.

I think adding scripting language to Kakoune is bad idea. Sometimes when I see some crazy escaping i really miss Lua (or other embeddable language) but at the end of the day I think it’s better to stick to unix philosophy.

Hopefully we will avoid creating huge, Kakoune-specific plugins this way. Take a look at Parinfer rust. It’s great example of plugin implemented as external tool that is usable from Kakoune and Vim (probably other editors could use it as well).

Kakoune as 9p server is thrilling idea but I see one issue: portability. Currently you need POSIX enviroment to run Kakoune, this model would require 9p client (or FUSE). Honestly something more mundane like built-in http API or extending JSON RPC would be much more portable (but less cool).

Basically this. It’s obvious that implementing some things in shell is crazy so we should think how to improve integration with external languages and tools.

I’m not sure what to think about declare-expansion idea yet. It’s interesting direction (especially persistent interpreter connected to Kakoune) but it doesn’t solve all the problems (you still need to escape }, need to communicate via env variables, what happens if you call blocking function, etc).

1 Like

There’s work going to make a Emacs package that will use it! Can’t wait

Let’s face it. If we want to write scripts in different languages without escaping hell, and use these scripts in plugins we need a system that will be able to track where plugin was sourced from so we could have consistent relative paths to our scripts from that directory. My plug.kak could do it, I suppose, but that’s not the right place to fix this problem.

For example. I see no problem implementing something in Perl, because we can use perl -e in shell expansion. But if this Perl script will get complex it is more appropriate to move it to some script.pl file and call it instead. But how? We need to know a path to the sctipt, so we could call it in shell expansion like so: perl /any/path/can/be/here/script.pl. What solution could be used here?

Maybe some variable that Kakoune will manage for us, like $kak_script_path, for example let it be ~/.config/kak/autoload/plugin/rc/, so we could write %sh{ perl "$kak_script_path/../perl/script.pl" }?
This way we can have proper file structure in plugins with any kind of language used in it.

Same goes for Python, Lua, LISP, and other languages, even compiled ones because we could call executables. Sure we can place those scripts in our PATH, like it is currently done for some Rust based plugins like kak-lsp or Parinfer-rust, but I don’t know if it’s suitable in terms of security, and name conflicts if all plugins will place their script.pl, script.py, script.scm in PATH.

Well it is already possible with $kak_config/perl/script.pl, isn’t it?

no. I can put script.kak anywhere and source it with source /home/user/kakscripts/somescript.kak in kakrc for example. It is not located under $kak_config and nothing prevents me from it.

ah I see, yeah that would be useful

Also it is the same principle that Kakoune uses. We know that kakoune uses relative paths from it’s location to get its scripts. I think this could be exposed for scripts too.

I don’t believe a persistent interpreter should be implemented in kak.
There are just too many out there, all with their own special version of hell.
Imagine a persistent lua interpreter and how to stop each script/expansion snippet from trampling over each other’s global state.

The problem of “having to escape {}” is solved by kak itself, IMO: %sh{...}, %sh(...), %sh[...],%sh|...| are all the same to kak.
Simply pick one that does not conflict at the time.
The beenfits of declare-expansion I see are:

  1. Efficiency
  2. Semantic clarity

Although, as said, I am not yet sure, both those benefits are large enough to warrant changing kak’s code.

I am not sure I understand the benefit here.
Would that mean, that files in PATH are not accessible anymore in shell expansions?
Is this to keep the plugin ecosystem clean?
Otherwise, where is the difference to an environment variable KAK_PATH, that I manage myself?


Another idea to mitigate the “escape craziness” in kak meta-commands (commands emitted in shell expansions) is for us to only use the long string syntax (%{...}, %#...#) in those meta-commands.
That way one could always resolve conflicting escape characters *1.
It doesn’t help, that the % is already meaningful in printf, but hey … it is manageable.

@eraserhd Thanks for providing thoughtful push back. I appreciate the attitude of being careful with the accretion of features. Your suggestion reminded me of a blog post where someone outlined an idea of a programming language, that would basically store its state (stack, program counter, variables,…) as directories and files. http://250bpm.com/blog:153

*1: Except, when you write shell expansions, that return commands, that use shell expansions … :face_with_monocle:

$kak_source contains the path of the currently being sourced kak script. I think bundling helper perl/python/… scripts in a well known relative location and sourcing that. I think I discussed it some time ago about kakoune-gdb.

I didn’t knew about $kak_source but it seems it is a thing that doesn’t available when we running commands defined in the script unless we store it some option:

filename vaiv.kak:

echo -debug %sh{ echo $kak_source } # prints path to vaiv.kak in debug buffer

declare-option str vaiv_path %val{source} # stores that path

define-command vaiv %{
    echo -debug %sh{ echo $kak_source } # prints nothing
    echo -debug %sh{ echo $kak_opt_vaiv_path } # prints correct path again
}

Documentation says:

path of the file currently getting executed (through the source command)

Are there any caveats? Like, can I use this method in my plugins to separate them into files on different languages? Or there is a way in which this variable can be left empty? I guess it’s a question that should be addressed in this Manual++ topic (which seems to be lacking some value expansions, @robertmeta!)


@mawww

Hm. I was experimenting with my langmap.kak plugin, and the source path part works fine - I can execute my Perl script no matter where plugin is placed. But the Perl script itself doesn’t work anymore.

I’ve pushed it to test branch. Not sure why it isn’t working.

Previously I used STDOUT from Perl to set options and execute map commands, but now, when perl isn’t inline Perl (perl -e ' ... ') I thought that I should pass $kak_client into it, wrap all inside commands into eval -client $passed_client, and pipe its output to | kak -p $kak_session, but that’s not working for some reason. I’m not asking you to dig in my code, but maybe I just missed something obvious.

Edit: nwm, it was my mistake.

evaluate-commands %sh{
    perl -e '
        # welcome to single-quote escape hell
        # also you need to escape any delimiter which you've
        # used to declare the shell expansion here, 
        # and perl can use them all, unless you've used
        # something like this: %sh£ perl ... £
    '
}

The benefit is that you can have access to any file relative to the script you’re currently using.

Nope.

Yep.

It will be different for every script.