Kakhist: persist command history, history UI

I’ve added kakhist to mru-files.kak. It’s a separate module, so those who have their own MRU’s can use kakhist independently (it does, however, require the sh library module in the same repo). There’s also a vim q: replacement which you can map to g: (command history buffer).

kakhist saves & restores command (:) history to/from a single kakhist.txt file. It works with multiple, overlapping sessions without clobbering unrelated histories — it figures out which commands have been loaded and which are new for the session (only new ones are appended to kakhist.txt when the client exits).

Note: if you have have a single kak server for all kak's (single session), you may wish to stick with @Screwtapello’s kakoune-state-save (state-save-reg-save colon). kakhist hands the command history over to the sh, so it can do (by default when kak starts / exits) command filtering and avoid multiple-session clobbering.

Filtering

You can define arbitrary sh code to filter out commands that shouldn’t be saved (option mru_files_ignore_sh). The default code filters out quit-related commands and sets up $2 to contain the base command name, stripped of args and ! ($1 always contains the full command). You can replace it completely, or append to it. To add exclude patterns (e.g. don’t save pwd):

set -add global kakhist_ignore_sh %{
  case "$2" in
    pwd|w|write) return 0 ;;  # reject
  esac; false  # don't exclude for now
}

Such code can be further appended to. kakhist-save only checks the return code of the last command.

Command history buffer

kakhist-buf-show shows a list of previous commands, like vim’s q:. Press <ret> on any line to execute. I personally map it to “g:”:

map global goto ':' '<esc>: kakhist-buf-show<ret>' \
  -docstring 'show command history'

Implementation notes

kakhist-{load,save} add a special marker (: kakhist-save, which should never enter history normally as it begins with a space) to the colon register. kakhist-save appends everything after the last marker to kakhist.txt. This way, different sessions never step on each others’ toes.

To save, the shell script does

eval set -- "$kak_quoted_reg_colon"

… which loads all list elements into sh “$@” positionals. I believe this is the safest approach in general to loading an “array” into shell, e.g.

decl str-list tst
set -add buffer tst 'a "b" c'
set -add buffer tst 1
set -add buffer tst "d 'e' f"
echo -to-file /tmp/tst -quoting shell %opt{tst}
nop %sh{
eval set -- "$(cat /tmp/tst)"
printf 1>&2 ':%s:\n' "$3" "$2" "$1"
}

processes list members as expected.

Extra

I’ve also added persistent session (= currently open buffers) support to mru-files.kak. mru-files-session-save generates a list of edit commands that recreate the current session, in the spirit of vim’s mksession.

If you think this might be useful, please test & enjoy. Comments, suggestions, GitLab issues etc all welcome.

4 Likes

Ohhh, that’s a neat trick.

When I first made kakoune-state-save, I tried to be clever about merging the history of concurrent sessions. However, Kakoune automatically deduplicates history registers, and also caps them at 100 items, so there’s no guarantee that the command history of two concurrent sessions share any common suffix. As a result, I gave up and let concurrent sessions stomp on each other — it was simplest to implement, the risk of losing history is low, and the consequences of losing history are also pretty low.

I hadn’t thought of sticking a marker into the history, however. That should make computing the common suffix quite reliable, although it would also make saving/loading history registers more complex,

Looking at your code, have you considered putting the history-buffer mappings in a comment at the top of the buffer, rather than in a :info box?

Also, it looks like your code would have difficulty with commands containing newlines, or a command history longer than can fit in an environment variable, but those things are pretty rare so it’s a reasonable engineering tradeoff to ignore them.

2 Likes

Thanks. I’ve fixed the newline issue by surrounding the escaped string in sh, not sed (__apos_quote()). I would have used sed -z from the beginning, but who knows if it’s available. I should probably write something along the lines of your kakquote's, but for sh. For context,

:reg colon %sh{printf '%s\n' 'echo 1' 'echo 2'}

adds an “ugly” command to history, then you can actually execute it with <up>. In kak it shows up as having a tiny LF unicode char, but on a single line.

Regarding env var limitations, that could probably be fixed via echo -to-file -quoting shell %reg{colon} with a mktemp file. Though I’ve used env vars in too many places already, I’ll have to check.

Regarding UI, thanks, that’s a good idea. Maybe comments at the end of the buffer, since the user should probably start there, like vim's q:. Probably still keep the infobox around (or do you think it’s annoying?)

Hi almr, enjoyed going through your project really good job. I have given it a play and here is some feedback first impressions:

# bug
delete-buffer=db
map buffer normal <esc> ': bd<ret>'
kak kakhist/rc/kakhist.kak +43

# would be nice to exclude users .hidden files out-of-the-box
str mru_files_ignore_sh %{
  */.[A-Za-z]*) return 0 ;;
}

# parse user file as a branch of `str mru_files_ignore_sh`
.ignore

# also not sure but alot of kak scripts use `edit!` might continuously show up.
'w'|'w!'|'e'|'e!'|'edit!'

# *kakhist* keys feature request
clear or delete a line from kakhist.txt within the *kakhist* buffer

${TMPDIR:-/tmp}/user/$(id -un)}
# no real need to call $(id -un) instead why not just
${TMPDIR:-/tmp}/mru-files}

# same sort of thing with
.local/share/kak/
.local/share/kak-mru-files/

First impression, nicely written code base and project layout. I’m looking forward to playing with it more. Thanks almr. Bye :wave:

@duncan, thanks for reviewing and for your comments.

  1. private alias ‘db’: fixed
  2. I, at least, work on dotfiles, so I want .bashrc in my MRU; but see 3 and 4.
  3. ignorefile — this is easily implemented, and I don’t want to add further complexity. The default ignore would be annoying to replace if it over-reaches or breaks. However, I’ll add examples to the README’s and I’ll soon enable a wiki. I’ve already added an example on how to add a regex ignore. See 4.
  4. I’ve added no-args edit!/write[!] to the default kakhist ignore. Note that with the default ignore_sh$2” is the command basename without “!”, so if you -add to it you have it easy.
  5. I’ve implemented saving *kakhist* to the history file on ‘>’ (just like for *mru*). However, it’s based on kakhist-save, so any edits to already-loaded / already-saved commands are ignored. So, undocumented for now, needs work, and I’ve opened a gitlab issue.
  6. id -un: note that Unix is multi-user, even if a lot of users / authors implicitly assume otherwise
  7. It’s configurable (as per README), but yeah, probably should be changed before 0.1.3

@Screwtapello:

  • added comments-as-help to *mru* and *kakhist* (bit of a b**ch to get right, but it helped me understand eval, exec, etc better)
  • added __sh_quote{,_single} to the shlib module, which should properly fix the newline issue
  • haven’t been able to find conclusive limitations for environment var size, and command history is truncated anyway, but in any case the new code passes the colon register as sh function arguments.

Thanks guys!

When the kernel creates a new process, it reserves space in the memory map for the “aux area”, which contains information like the effective user and group IDs (see getauxval(3) as well as the command line and environment. The size of this area is platform-dependent, but if you have GNU xargs installed it will tell you:

$ echo | xargs --show-limits
Your environment variables take up 2228 bytes
POSIX upper limit on argument length (this system): 2092876
POSIX smallest allowable upper limit on argument length (all systems): 4096
Maximum length of command we could actually use: 2090648

On my Linux system, the maximum space available to command-line arguments and environment variables is 2,092,876 bytes, but on some systems could be as little as 4,096 bytes.

In particular, I believe macOS has a notably smaller aux area than Linux.

Yeah, I’ve seen something like that on stackoverflow. Can someone with macOS run echo | xargs --show-limits?

This limit probably doesn’t apply to sh variables (x=... or set -- ...). If so — then ideally, kak should be configurable to pass $kak_* and $n to shell scripts via a specially crafted prologue (e.g. save the values in a script in mktemp, and source the script in an invisible %sh{} prologue). Otherwise, every user script has to apply this workaround separately, which seems like an awful amount of duplication.

Kakoune doesn’t currently do that, but mawww has recently been working on the command-fifo branch as a way for scripts to obtain data from Kakoune without the limits of the auxiliary area.

Suppose I tried to implement a parallel mechanism to pass options / registers to %sh in a temporary file in /run. I would need to generate a unique filename from within kakscript, without resorting to a separate %sh call (e.g. to call date).

How can I generate a unique ID, from within kakscript, that will not conflict among concurrent sessions / hooks / user-triggered commands? I’ve looked at %val{client}, session, history_id, timestamp (useless) but I’m still not sure combining these would yield a unique ID, if hooks can run concurrently.

Really like this plugin. I was at first sceptical to try mru-files but omg loved so much I implemented my own! Well maybe not my own fd. Thanks for the great job and inspiring me on kowsky.kak. Going have a deeper play with kakhist on the weekend with some predetermined commands for the text file. I’ll check out your repos updates.

Yes, you are right I can do that myself. Sometimes I just need to hear it from someone else :grinning_face_with_smiling_eyes: .

Did not know this. A familiar occurrence. Last minute thought (always a worry :innocent:) have a look at this. A roll your own approach to cd history:

Ramey, C & Unknown, A 2002, cdhist.bash.gz, www.fifi.org, viewed 26 March 2021, http://www.fifi.org/doc/bash-doc/examples/scripts.v2/cdhist.bash.gz

It’s not a request or feature just thought you might like it for an idea. I owed you one at least. Bye :wave:

1 Like

Thanks almr, gave kakhist.kak a run and am hooked. Bye :wave:

Fisher 2018, Losing it, Catch & Release, viewed 09 July 2021, 
             <https://open.spotify.com/track/62WEkOD8TUO7wzkolOQW9v?si=4c35a17e6c7f44ac>
             <https://youtu.be/u31thuMehjM>

kakhist-almr-jrename-gradle-02

kakhist-almr-jrename-gradle-03

I’m sorry, but I most be confused as to the configuration you’ve provided in the README, because I’m getting this error in *debug*:

plug.kak: Error occured while loading 'https://gitlab.com/kstr0k/mru-files.kak.git' plugin:
          2:5: 'evaluate-commands': 13:1: 'evaluate-commands': 3:2: 'mru-files': no such command

Here’s what I have:

plug "https://gitlab.com/kstr0k/mru-files.kak.git" config %{
	set-option global mru_files_history %sh{ echo "$HOME/.local/share/kak/mru.txt" }
} demand mru-files %{ # <- this is apparently the issue
	require-module kakhist; kakhist-init 
} config %{
	map global goto ':' '<esc>: kakhist-buf-show<ret>' -docstring "show command history"
    # `files` is a user mode I defined else where before this
	map global files 'r' '<esc>: mru-files ' -docstring "recent files"
}

Your snippet works in my setup, if I pre-declare files mode; note that Andy has recently reworked demand, it had some bugs before.

In the readme, though, precisely because multi-module plug.kak usage can get confusing / fragile, I suggested putting the kakhist commands after plug, in a separate top-level block. So a single demand and a single config arg to the plug command.