ShellCheck integration for Kakoune

ShellCheck is a static-analysis tool for shell scripts that detects a whole bunch of bugs and potential portability issues.

kakoune-shellcheck is a plugin that automatically sets the lintcmd option appropriately when you edit a shell-script, so you can use :lint to check for errors and :lint-enable (or :lint-next-message, or the *lint-output* buffer) to see them.

It also makes use of the new :lint-selections command that recently landed. If you are editing a Kakoune script (filetype=kak), the :lint command is redirected to a helper command that will select all the %sh{} blocks in the file, and only check those with ShellCheck, instead of trying to validate the entire file.

Basically, if you have ShellCheck and this plugin installed, then running :lint in a shell script or Kakoune script should Just Work.

4 Likes

Nice! I’m slowly adjusting my shell coding style to make shellcheck happy, and while it’s a bit of a wrench at times I think it’s making my code a lot more maintainable.

POSIX shell is one of those languages that’s designed to keep the simple things really simple, at the expense of letting moderately-complex things become difficult or impossible. To make things worse, a lot of common shells have extensions to smooth off the rough corners, so you can try something that makes sense and have it work for you, but it might not work for other people.

There’s a long list of "things that don’t work the way you expect"¹ and an even longer list of “things that should work the way you expect but don’t on some platforms”, and while a lot of them are documented in things like the Bash FAQ and the Portable Shell chapter of the Autoconf manual it takes some effort to learn to recognise those problems and what to do about them. Having a bunch of those guidelines encoded into a tool like ShellCheck is a great start.

¹: For example, consider this fragment:

foo=a

if [ -n $foo ]; then
    echo non-empty
else
    echo empty
fi

If you run that script, it will echo non-empty, exactly as it should. But if you edit the first line to say foo="", then you might expect the test to fail with an error like “missing operand”. Instead, it will still echo non-empty, because $foo is removed at expansion time, and so the -n is checking whether “]” is non-empty. The solution is, as usual, to exhaustively quote variable expansions wherever they appear, like "$foo".

That’s why shell is expected to be kind of a glue between programs written in languages designed to do certain task. Shell is hell for programming, and though one could make it less painful by creating a better shell, I don’t think one really should. Complex platform independent code should be done with appropriate language, that ensures that it will always work on different platforms.

I do not want to be heard as “we don’t need a better shell”, because we do need a better shell, but not as a programming environment, but as a better glue, which is consistent across platforms and has as small as possible performance penalties.

There’s another thing though, which corresponds with previous paragraph. In Emacs there’s a shell, called eshell. This is a shell, written in Emacs Lisp, which is kind of a REPL for Emacs Lisp, and it can be used as a shell. What’s amazing about it, is that you have a shell, which uses Lisp, a real language, as it’s extension language. Emacs Lisp is not the best of all Lisps, but I think it is much better compared to POSIX sh, or bash scripting. Because it is a real language that you can write programs in. And it equally works on Linux and Windows, because it is a lisp interpreter, what makes it cross platform.

Which gets me thinking that, perhaps, we should abandon shell as it is seen today, and provide a REPL like shell, with real and powerful language instead of ad hoc one, with well established semantics and behavior. This is something, to what Oil Shell does, though I think that Python2 as implementation language is a poor choice, and Oil implements a new shell language, aimed towards writing maintainable scripts, which simply is a new shellscript, and not a real general purpose/problem-solving language, such as Python, Lisp, or Perl. I think that eshell is more a step forward, even if eshell is not really good as a shell.

2 Likes

Awesome idea and perfect timing!

What would the state-save.kak you are referencing in the README be? I did a search for it and found a couple of references.

You might be interested in Elvish, which aims to be a pleasant glue language (with support for arrays, hashmaps, closures, and passing JSON streams from one command to another) as well as a pleasant interactive environment.

The last I heard about Oil Shell, they’d given up on writing a whole new shell language and decided to let scripts “opt in” to more sensible behaviour for the traditional Bourne-like syntax. It’s probably the right move, overall. Also, while Oil was originally written in Python, they’ve forked the interpreter heavily customised it, and they’re investigating ways to automatically convert it to C/C++ — so it’s not really “implemented in Python” in the same way that most things are.

Oops! Can you guess which project’s README I copied and edited when I set up this project? kakoune-state-save is a plugin to automatically restore the cursor position when you re-open a file, and to help restore previous sessions’ command history and search history when you start a new session. It has its own thread, too.

1 Like

Yeah, I’ve heard about Elvish, it is kinda cool, but the fact that it mixes some kind of s-expressions, with C-like expressions is kinda weird. Like if $true { } else { } and (+ 1 2)

its the same as posix shell, just () instead of $(). and curly braces instead of fi.

not quite. It’s not about parentheses, but about that in posix shell $((1 + 2)) is what (+ 1 2) in elvish, which resembles me of s-expressions from lisps. My concern is that you either do if predicate {true branch} else {false branch} and 1 + 2 or (if predicate (true branch) (false branch)) and (+ 1 2), but not mixed.

well, because its not s-expressions

sure, but why (+ 1 2) and not (1 + 2) then?

sh equivalent:

add() {
  echo $(($1 + $2))
}

alias +=add

When you write (+ 1 2) in Elvish, you write $(+ 1 2) in sh.
It is not an expression, just a builtin function.

It is not an expression, just a builtin function.

What is expression if not an invocation of a (built-in) function with some arguments? :slight_smile:

I’m just nitpicking. I understand that this is syntax for a subshell. For me having + as a function and not an operator means that perhaps one could use it in higher order procedures, like map. This certainly will not work with sh's +, but may work with elvish's +.

$ square() { echo $(($1 * $1)); }
$ map() {
>     f=$1
>     shift
>     if [ $# -gt 0 ]; then
>         first=$1
>         shift
>         echo -n "$(eval '$f $first') "
>         map $f $@
>     else
>         echo ""
>     fi
> }
$ map square 1 2 3 4
1 4 9 16

Also, why limit yourself with two arguments?

$ add() { while [ $# -gt 0 ]; do res=$((${res:-0} + $1)); shift; done; echo $res; }
$ alias +=add
$ echo $(+ 1 2 3)
6

Now that I got to refining my scripts, I was copying shell blocks into Emacs to run it trhough the shellcheck plugin there, and recently I’ve remembered that I saw a plugin for Kakoune too, so I’ve decided to try and use it, and I’ve noticed something.

The shellcheck in the Emacs reports way more warnings than this plugin. E.g. here:

echo %sh{
    echo $kak_opt_filetype
}

If I run lint command, I’ll see only the SC2086 Double quote to prevent globbing and word splitting. warning. But in Emacs I also get the SC2154 kak_opt_filetype is referenced but not assigned..

I’ve looked down the code, and saw that you explicitly disable this warning with --exclude SC2154 for %sh{} blocks. Why? I’ve actually caught some bugs recently by making scritps more strict with the ${var:?} syntax, that ensures that variable is present and not empty when script runs. This makes it easier to write code that will fail in a known way if the surroundings change.

Last I checked, ShellCheck only supports “all variables must be declared” and “no variables must be declared” modes. For most shell-scripts, the false-positive rate for SC2154 would be very low, so it makes sense for that warning to be on by default. However, because made-up variable names are part of Kakoune’s official scripting API, the false-positive rate for SC2154 in Kakoune scripts is very high. While disabling SC2154 probably hides a few legitimate bugs, I expect leaving it enabled would teach people to ignore all kinds of ShellCheck warnings since “ah, it’s probably just another false positive for SC2154”.

Ideally, ShellCheck would support a list of variables assumed to be defined externally, and I’d ship the plugin with a list of as many Kakoune expansions as I could think of. Unfortunately, the ShellCheck authors haven’t gotten around to that yet, and my Haskell isn’t good enough to write a PR, so I figured the best option was just to turn off that specific warning.

alternatively you can post process the output and exclude all lines that match warning 2154 and contain a variable that starts with kak_, although this should be mentioned explicitly in the readme if you decide to do it