# Pipe-map, a tool to map out a shell pipeline

**URL:** <https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430>\
**Category:** Resources\
**Created:** [December 14, 2020, 11:46am UTC](https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430 "2020-12-14T11:46:41Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Screwtapello](https://discuss.kakoune.com/user_avatar/discuss.kakoune.com/screwtapello/32/12_2.png) [@Screwtapello](https://discuss.kakoune.com/u/Screwtapello)\
**Post date:** [December 14, 2020, 11:46am UTC](https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430/1 "2020-12-14T11:46:41Z")

</div>

The other day I was working on a Kakoune plugin that tried to do something clever with shell pipelines and FIFO buffers, but it just silently hung and I couldn’t figure out why. I suspected I’d somehow set up the pipeline incorrectly, but I didn’t have a way to inspect it and see what I got wrong.

So I threw together a hacky Python script to scrape the Linux `/proc` directory for processes and their open file descriptors, pick out a process of interest, and map out the pipeline it’s part of. Now I can run a command like:

```
$ (cat | tr a-z A-Z | sort) 2>/dev/null

```

…and automatically generate a map of the pipeline like:

 ![Automatically generated pipeline map](https://discuss.kakoune.com/uploads/default/original/1X/1f90f9f50bfe2ba3d19766c62ec9511c796ac7d9.png)

I don’t know if this might be useful to anybody else, or even if I’ll ever use it again myself, but I figured I’d toss it into a git repo just in case:

> **[Screwtape / pipe-map · GitLab](https://gitlab.com/Screwtapello/pipe-map)**
>
> A tool to generate a GraphViz description of a shell pipeline

---

<div class="post-metadata">

**Author:** ![scr](https://discuss.kakoune.com/user_avatar/discuss.kakoune.com/scr/32/486_2.png) [@scr](https://discuss.kakoune.com/u/scr)\
**Post date:** [December 14, 2020, 4:58pm UTC](https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430/2 "2020-12-14T16:58:11Z")

</div>

Wow ! that’s super cool !  
Could you specify which python version you use exactly, I didn’t work for me with python3.6 ( ` __future__ ` error ), neither with 3.7:

```auto
Traceback (most recent call last):
  File "./pipe-map.py", line 139, in <module>
    graph = Graph.from_proc_tree(pathlib.Path("/proc"))
  File "./pipe-map.py", line 84, in from_proc_tree
    proc = Process.from_path(each)
  File "./pipe-map.py", line 48, in from_path
    file = File.from_path(each.readlink())
AttributeError: 'PosixPath' object has no attribute 'readlink'

```

Also, if I understand well one cannot render the graph of a finished pipeline, the pipeline should be “on going”. So is it necessary to use, for example:  
`( cat | my | pipeline)`  
instead of:  
`(cat my_input | my | pipeline )` ?

---

<div class="post-metadata">

**Author:** ![Screwtapello](https://discuss.kakoune.com/user_avatar/discuss.kakoune.com/screwtapello/32/12_2.png) [@Screwtapello](https://discuss.kakoune.com/u/Screwtapello)\
**Post date:** [December 15, 2020, 1:16am UTC](https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430/3 "2020-12-15T01:16:17Z")

</div>

I happen to be using Python 3.9. I avoided using the `datatypes` standard library module which I know was added only recently, but I guess there were other dependencies I missed.

The ` __future__ ` import might be a bit of a pain to remove, but if it works on Python 3.7 that’s probably Close Enough.

I am very suprised that `.readlink()` was only added to `pathlib` in Python 3.9. If you add `import os` at the top of the file (with the rest of the imports) you should be able to change that line to:

```
file = File.from_path(pathlib.Path(os.readlink(each)))

```

PRs welcome!

You can’t render the graph of a finished pipeline, because when the pipeline finishes all the information about it gets cleaned away. Using `cat` to “hold it open” definitely works, but hitting `<c-z>` to pause the pipeline while it’s running, sending SIGSTOP to one of the processes, or just getting lucky with the timing all work too.

---

<div class="post-metadata">

**Author:** ![scr](https://discuss.kakoune.com/user_avatar/discuss.kakoune.com/scr/32/486_2.png) [@scr](https://discuss.kakoune.com/u/scr)\
**Post date:** [December 15, 2020, 2:43am UTC](https://discuss.kakoune.com/t/pipe-map-a-tool-to-map-out-a-shell-pipeline/1430/4 "2020-12-15T02:43:35Z")

</div>

> file = File.from\_path(pathlib.Path(os.readlink(each)))

That worked on python3.7 🙂 Thanks I could not have found it myself.