Hooks, concurrency, race conditions

Can hooks interrupt each other? For example, if I’m setting a window option (= variable, because we have no variables) in hook #1, can I be sure that no other hook #2 will interrupt and change that option until the first one is done?

Kakoune server is single threaded. For a single event, all hooks run in sequence (that should be in the order they were added). However, if hook 1 triggers hook 2 (say hook 1 changes an option value and hook 2 is a BufSetOption hook), then hook 2 will indeed run nested in hook 1.

Hooks need to be well behaved and cooperate.

Thanks you, @mawww, that sounds like system interrupts. Does that still mean that hook code needs to be re-entrant? I.e. can this sequence happen (including user hooks):

: hook1
::: hook2
::::: hook1

My problem is simple, I want to echo -to-file to a temporary file with a unique name, under /run, without calling sh. So I was thinking of using a global counter option:

echo -to-file ".../something_%val{session}_%opt{cnt}"
# use, add NormalIdle hook to remove, whatever
set -add global cnt 1

Then it occurred to me that I can’t protect that option from race conditions.

Kakoune keeps track of the currently executing (hook name, hook param) pair and will reject already running hooks, but that does not prevent the same hook handler from running multiple times nested with different parameters.

Regarding your use case, my first question is going to be why do you want to avoid the shell ? Hooks are not very common operation and shell overhead should not be noticeable. That said, I am not sure which race condition you are referring to here, do you expect to have some malicious hook that voluntarily changes cnt to a previous value ? How is that different from whatever happens to cnt between multiple runs of your hook ?

So it rejects hooks that have the exact same name and params.

I’m not worried about malicious code, but about my own code, if I use the counter mechanism in a lot of places. For example, here’s a with-tmpfile mechanism:

def defp -params .. %{
  def -override -params .. %arg{@}
} -override

declare-option int tmpcnt
defp with-tmpfile %{
  set -add global tmpcnt 1
  %arg{@} "/tmp/kak_tmp_%opt{tmpcnt}"
  hook -group tst -once add NormalIdle .* %{
    nop %sh{ rm -f "/tmp/kak_tmp_%opt{tmpcnt}" }

defp tst-cmd-hist %{
  echo -debug %arg{1}
  echo -to-file %arg{1} -quoting shell -- set -- %reg{:}
  nop %sh{
    . "$1"  # load prologue,  overwrites $@
    echo 1>&2 "Second command was $2"

with-tmpfile tst-cmd-hist

This code avoids the issue with potentially limited environ size on various systems, as pointed out by @Screwtapello in a previous topic, by saving them to a temp file (which should be a RAM-only operation) and loading the file inside the shell into shell (not environment) variables.

Regarding the shell, I know it’s a controversial topic. I actually love the shell, as is probably apparent from my posts here. Now, a lot of people say calling the shell doesn’t matter — and it doesn’t, on a desktop, or for infrequent hooks. On a laptop, for “on-every-keystroke”, or “on-timer-msecs” hooks, there’s battery life and thermal (and possibly fan → unbearable noise) to consider.

Also, TBH, it’s fun to figure out how to do things differently :slight_smile:

You might be interested in the command-fifo branch that I intend to merge soon-ish and that solves the environ size limit by allowing to run commands from inside a %sh{...} block

I am not sure forking a couple shells on every keystroke is going to make a big difference on battery life (well provided KAKOUNE_POSIX_SHELL points to lightweight one), on my laptop CPU usage is usually at idle levels when I use Kakoune.

The fun to try to solve the problem with that constraint I can relate to much more, I am not sure how to solve your issue in a general way, we would need some kind of stack data structure, which history registers already almost provide. I could get convinced that we need a general purpose stack register and some user accessible ways to get/set just the “main” values of register (instead of the whole list) which would mean pop/push on that register (the same way it means push to history/get last history entry on history registers like :)

I’ll take a look at command-fifo, that’s what others have recommended too.

Somewhat OT, I found a way to create a stack of tokens (not arbitrary strings), but it requires my %arg{@:N} extension branch and it’s… nasty:

decl str stk *EMPTY*
defp stack-push %{  # args: 1=optname 2=new
  eval set global '%arg{1}' """%arg{2} %%opt{%arg{1}}"""
defp stack-pop %{  # args: optname
  # substitute $1; substitute %opt; deparse
  eval eval _stack-pop1 %{'%arg{1}'} "%%opt{%arg{1}}"
defp _stack-pop1 %{  # args: 1=optname 2..=values
  set global %arg{1} "%arg{@:3}"
stack-push stk xx; echo %opt{stk}
stack-pop stk; echo %opt{stk}

There are probably other ways to do it (short of sh) like spawning a draft buffer and using “normal-mode script”. Not sure what the overhead for a new buffer is.

Hi almr,
Nice topic. I am still in the learning stage so it’s hit and miss. Check the below reference for the legitimate source.

Different way of thinking about the same problem. Break the problem apart and check for equality. In light of race conditions.

Two containers / files


if N==N then do it else don’t

let N=cnt

[T|N] -> [N|C]
check N = N


[N|T] -> [T|C]
check T = T

source adapted: Appendix C. Core Data Structures p.702

Structure Order Unique Associative Dynamic Locality
Dictionary no    yes   key=value
Heap       no    yes   key-priority  no      yes
Hash table no    yes   key=value     yes     no

Dynamic - container can resize on insertion/removal.
Locality - if elements are stored in a single uninterrupted block of memory.

The below book, discount, and article are all by the La Rocca as an incentive to purchase. Which is entirely up to you. I have no affiliation with Manning Publications or La Rocca the author.

La Rocca, M 2021, Advanced Algorithms and Data Structures, Manning Publications, viewed 02 July 2021, https://www.manning.com/books/advanced-algorithms-and-data-structures

Take 42% off the entire book. Just enter code slrocca into the discount code box at checkout at manning.com. https://freecontent.manning.com/slideshare-expand-your-programming-skill-set/

Explaining MapReduce (with Ducks) https://freecontent.manning.com/explaining-mapreduce-with-ducks/

@mawww thanks for sharing the knowledge. Bye :wave:.