Helix magic

I’ve been using as daily driver the Helix text editor for 3 months now. Coming from 5 consistent years of Neovim usage, I switched fairly quickly. The kakoune-based modal editing intrigued me, and I wanted to give it a try. But what swayed me is the great out-of-the-box user experience. Going from 200 lines of fennel config and 20 dependencies to 0 config and 0 dependency is a big plus.

I’m still more comfortable with neovim than helix, but Helix has capabilities that I’ve yet to take advantage of.

Here is an example of what I mean.

The problem

I was asked to create a criterion report for changes I made to the bevy implementation of query iterator size hints.

After familiarizing myself with criterion and running the benchmarks, criterion generated a report in the form of a HTML file tree. Each benchmark has its own html file with a variety of plots, the reports also contain a bunch of json and csv files I’ve no idea how to use.

Criterion also generates automatically difference plots if you run it twice in a row. But again, does nothing else but embed them in the benchmark-specific html page.

However, what I needed was a summary of all the changes. It was impossible to tell what the changes I made entailed at a glance, since I would need to open each html benchmark report individually and look at the “Changes” section at the very end of the file.

How did Helix help me solve it

What I wanted was a unique page that displays a single plot per benchmark and a summary of the changes for it.

Looking at the file tree generated by criterion, I found that each report had a subdirectory called both with a few plots in them. The one I chose was pdf.svg (the probability density function), because it does both a good job of expressing the accuracy of the benchmark and the effects of the change.

I also needed the table of comparison from each html report files (as follow):

	Lower bound	Estimate	Upper bound
Change in time	-0.0149%	+0.0193%	+0.0543%	(p = 0.33 > 0.05)

Ideally, I’d also make it clear which benchmark was an improvement and which one was a regression by having a different color for each comparison result.

How I did it

I first used the fd tool to list all pdf.svg files within a both directory. I then put the result in a file.

(btw I strongly recommend you use fd, it’s a better version of find, which paleolithic UI I absolutely never once remembered how to use, even at five minutes interval)

fd --full-path both/pdf.svg target/criterion > all_changes.html

I then opened all_changes.html with helix. The content of the all_changes.html file was as follow:

target/criterion/for_each_iter/report/both/pdf.svg
target/criterion/for_each_par_iter/threads/1/report/both/pdf.svg
target/criterion/for_each_par_iter/threads/16/report/both/pdf.svg
target/criterion/for_each_par_iter/threads/2/report/both/pdf.svg
target/criterion/for_each_par_iter/threads/32/report/both/pdf.svg
target/criterion/for_each_par_iter/threads/4/report/both/pdf.svg
...

Note that <ESC> means that I press the escape key.

I created a h2 html element by copying the head of the file name and copying it into a h2 tag. The command sequence would be:

111C (create a cursor per line in the file)
gs2f/d (go to start of line, select all characters between it and the 2nd next slash and delete them)
gl (go to line ending)
3F/h (go back 3 slashes in the line + one more character back)
v (enter selection extension mode)
gs (go to start of the line)
"iy (yank selection into register i)
O<ESC> (create a new line before this one, exit edit mode)
_iP (insert in before the cursor the content of register i)
I<h2><ESC> (insert the string “<h2>” at the beginning of the line)
A</h2> (finally, insert “</h2>” at the end of the line)

<h2>for_each_iter</h2>
for_each_iter/report/both/pdf.svg
<h2>for_each_par_iter/threads/1</h2>
for_each_par_iter/threads/1/report/both/pdf.svg
<h2>for_each_par_iter/threads/16</h2>
for_each_par_iter/threads/16/report/both/pdf.svg

I then embedded the image in the file by translating each line into a img element.

o<img src="<ESC> (start new line with img tag beginning)
_iP (past content of register i before the cursor)
i/report/both/pdf.svg"><ESC> (insert the end of the img tag)

<h2>for_each_iter</h2>
<img src="for_each_iter/report/both/pdf.svg">
for_each_iter/report/both/pdf.svg
<h2>for_each_par_iter/threads/1</h2>
<img src="for_each_par_iter/threads/1/report/both/pdf.svg">
for_each_par_iter/threads/1/report/both/pdf.svg

Now the difficult thing was embedding the html change report table from the benchmark page. For this, I used xidel, a xpath tool to read xml and html. It’s very similar to jq, but for xml rather than json.

FYI xidel is a single pascal file of 2000 lines.

I first tested in my shell how I would do it in pure bash. I came up with the following function:

function get-report-table {
    file="${1%%both/pdf.svg}"
    xidel ./target/criterion/$file/index.html \
        --xpath3 '//*[@class="additional_stats"]' \
        --printed-node-format=html
}

This returned (almost) the HTML I wanted in my final file:

<div class="additional_stats">
  <!-- The report data for the last benchmark, which
       we don't care about for the summary -->
</div>
<!-- What we want: the comparison table between the last benchmark and
     the current one (with a diagnostic such as "Performance has improved.")
     -->
<div class="additional_stats">
  <h4>Additional Statistics:</h4>
  <table>
    <thead>
      <tr>
        <th></th>
        <th title="0.95 confidence level" class="ci-bound">Lower bound</th>
        <th>Estimate</th>
        <th title="0.95 confidence level" class="ci-bound">Upper bound</th>
        <th></th>
      </tr>
    </thead>
    <tbody>
      <tr>
        <td>Change in time</td>
        <td class="ci-bound">-17.097%</td>
        <td>-14.185%</td>
        <td class="ci-bound">-11.391%</td>
        <td>(p = 0.00 &lt; 0.05)</td>
      </tr>
    </tbody>
  </table>
  Performance has improved.
</div>

Helix has the “|” command to pass the currently selected lines to a shell command and replace them in place with the output of the command.

I had difficulties with this one, because xidel only accepts files as argument, also parameter expansion (such as ${1%%both/pdf.svg} used to remove the trailing both/pdf.svg from the input) does only work with variables, not standard input.

The trick was to:

Replace the $file in the command by $(cat /dev/stdin)
Remove in helix the trailing both/pdf.svg (gl2F/d)
Now I can just press | and copy the command.

And BANG! I had now the table in my html file for each benchmark.

I then selected the extraneous report div by using helix’s awesome tree-sitter based motions (I bound them to ( and ) on my keyboard), and pressed d to remove it.

Now for clarity, I just needed to wrap the “Performance has improved” bit into a <a style="color:red/green"> based on the message.

This is a classic usage of the helix editor.

Select the whole document with %
with s enter “select mode”
type improved, then enter. You now have all the occurrences of improved
expand selection to parent syntax node (I set this to ])
Add in front the anchor node opening with i
Add at the back the node closing delimiter with a
Repeat with regressed.

Done.

I now had the full report, I uploaded it on my site and linked it in the PR.