make.ts

      sounds familiar? This is how I've historically run benchmarks and other experiments that require repeated sequences of commands - type them in manually once, then rely on shell history (and maybe some terminal partitioning) for reproduction. Over the years I have arrived at a much better workflow pattern - <code>make.ts</code>. Once I started working with multiprocess applications I was forced to adapt to it, where entering commands manually is impossible. In retrospect, I should have optimized the workflow years ago.
    </p><div id="The-Pattern">

      <p>



        Use (gitignored) file for interactive scripting. Instead of entering a command directly into the terminal, first write it to a file, and then run the file. For me, I type stuff <code>make.ts</code> and then run <code>./make.ts</code> in my terminal (well, I want <em>One</em> <kbd><kbd>enter above</kbd></kbd>    for him).





        I want to be clear here, I'm not advocating writing "proper" scripts, I'm simply capturing your interactive, ad-hoc commands into a persistent file. Of course any command you want to execute
        <em>Frequently</em> Belongs to build system. Surprisingly, even the more complex one-off commands benefit from walking through the file, as it would take you several tries to get them right!





        has many advantages over <kbd><kbd>up up up</kbd></kbd>



        Workflow:

      <ul>
        <li>



          The actual commands become larger, and it's great to use a real 2D text editor instead of the shell's line editor.
        </li>
        <li>



          If you need more than one command, you can write multiple commands, and still run them all with the same key (first <code>make.ts</code>I was prone to creating rather horrendous && combinations for this reason).
        </li>
        <li>



          With the sequence of commands outlined, you motivate yourself to sequentially improve them, make them inactive, and otherwise invest in your own workflow for the next few minutes, without falling into the YAGNI pit from the beginning.
        </li>
        <li>



          At some point you may realize that after running a series of ad-hoc benchmarks interactively, you will write a proper script that executes a collection of benchmarks with different parameters. With the file approach, you already have the main body of the script implemented, and you just need to wrap it in some ifs and ifs.
        </li>
        <li>



          Finally, if you work with multi-process projects, it will be easier for you to manage concurrency declaratively, by generating a tree of processes from the same script, rather than switching between terminal splits.
        </li>
      </ul>
    </div><div id="Details">





        Use a consistent file name for the script. I use <code>make.ts</code>and so there is a <code>make.ts</code> At the core of most of the projects I work on. Accordingly, I have <code>make.ts</code> line in project <code>.git/info/exclude</code>
        - The <code>.gitignore</code> A file that is not shared. Fixed name reduces fixed costs - whenever I need complex interactivity I don't need to come up with a name for a new file, I just open my already existing file. <code>make.ts</code>Erase whatever was there and start hacking. Similarly, I have <code>./make.ts</code> In my shell history, that's why fish auto-suggestions work for me. At one point, I had a VS Code task to run <code>make.ts</code>However now I use the terminal editor.





        Start the script with hash bang,
        <span class="display"><code>#!/usr/bin/env -S deno run
            --allow-all</code></span>



        In my case, and
        <span class="display"><code>chmod a+x make.ts</code></span>



        To make it easier to play the file.

      Write the script in a language that:
      <ul>
        <li>



          You are comfortable with it,
        </li>
        <li>



          Doesn't require a huge setup,
        </li>
        <li>



          Makes it easier to generate subprocesses,
        </li>
        <li>



          There is good support for concurrency.
        </li>
      </ul>




        To me, that's TypeScript. Modern JavaScript is ergonomic enough, and structural, that sequential typing is a sweet spot that gives you reasonable code completeness, but still allows any problem to be solved by throwing sufficiently stringent instructions at it.





        JavaScript's tagged template syntax is great for scripting use-cases:


      <figure class="code-block">
        <pre>

function $(literal, ...interpolated) {
console.log({ literal, interpolated });
}

const dir = "hello, world";
$ls <span class="hl-subst">${dir}</span>;

print

      <figure class="code-block">
        <pre>

{
literal: [ "ls ", "" ],
interpolated: [ "hello, world" ]
}

        what happens here happens <code>$</code> Gets a list of literal string pieces inside the backticks, and then, separately, gets a list of values ​​to interpolate between. it <em>can</em>
        Concatenate everything into just one string, but it's not necessary to do so. This is exactly what is needed for process spawning, where you want to pass an array of strings <code>exec</code>
        syscall.





        Specifically, I use the Dax library with Deno, which is excellent as a single-binary battery-involved scripting environment (see <3 Deno). Bun's Box has a Dax-like library and it's a good alternative (although I personally stick with Deno
        <code>deno fmt</code> And <code>deno lsp</code>). You can also use the famous zx, although keep in mind that it uses your shell as a middleman, which I consider sloppy (clarification).





        Whereas <code>dax</code> Makes it convenient to create a single program, <code>async/await</code> Excellent for organizing various processes:


      <figure class="code-block">
        <pre>

await Promise.all([
$sleep 5,
$sleep 10,
]);

        Here's how I implemented this pattern earlier today. I wanted to measure how the TigerBeetle cluster recovers from the primary crash. The manual way to do this would be to create a bunch of ssh sessions to multiple cloud machines, format the datafiles, start the replicas, and then do some load. I
        <em>About</em> I started out partitioning my terminal, but then I discovered I could do it smarter.





        The first step was to cross-compile the binary, upload it to the cloud machines, and run the cluster (using my box from the other week):


      <figure class="code-block">
        <pre>

#!/usr/bin/env -S deno run --allow-all
import $ from "jsr:@david/dax@0.44.2";

await $./zig/zig build -Drelease -Dtarget=x86_64-linux;
await $box sync 0-5 ./tigerbeetle;
await $box run 0-5</span></span>
<span class="line"><span class="hl-string"> ./tigerbeetle format --cluster=0 --replica-count=6 --replica=?? 0_??.tigerbeetle
;

await $box run 0-5</span></span>
<span class="line"><span class="hl-string"> ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle
;

        Running the above a second time, I realized I needed to kill the old cluster first, so two new commands were "interactively" entered:


      <figure class="code-block">
        <pre>

await $./zig/zig build -Drelease -Dtarget=x86_64-linux;
await $box sync 0-5 ./tigerbeetle;

await $box run 0-5 rm 0_??.tigerbeetle.noThrow();
await $box run 0-5 pkill tigerbeetle.noThrow();

await $box run 0-5</span></span>
<span class="line"><span class="hl-string"> ./tigerbeetle format --cluster=0 --replica-count=6 --replica=?? 0_??.tigerbeetle
;

await $box run 0-5</span></span>
<span class="line"><span class="hl-string"> ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle
;

        At this point, my investment in writing this file and not just entering commands one by one has already paid off!





        The next step is to run the benchmark load in parallel on the cluster:


      <figure class="code-block">
        <pre>

await Promise.all([
$box run 0-5 ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle,
$box run 6 ./tigerbeetle benchmark --addresses=?0-5?,
])

        I don't need two terminals for two processes, and I get to mostly copy-paste-edit the same commands.





        For the next step, I want to actually kill one of the replicas, and also capture live logs to see how the cluster reacts in real time. right here <code>0-5</code> The box's multiplexing syntax is trivial, but, given that it's JavaScript, I can just write a for loop:


      <figure class="code-block">
        <pre>

const replicas = range(6).map((it) =>
$box run <span class="hl-subst">${it}</span></span></span>
<span class="line"><span class="hl-string"> ./tigerbeetle start --addresses=?0-5? 0_??.tigerbeetle</span></span>
<span class="line"><span class="hl-string"> &amp;&gt; logs/<span class="hl-subst">${it}</span>.log

.noThrow()
.spawn()
);

await Promise.all([
$box run 6 ./tigerbeetle benchmark --addresses=?0-5?,
(async () => {
await $.sleep("20s");
console.log("REDRUM");
await $box run 1 pkill tigerbeetle;
})(),
]);

replicas.forEach((it) => it.kill());
await Promise.all(replicas);

        At this point, I need two terminals. one runs <code>./make.ts</code> And shows the log from the benchmark itself, runs another <code>tail -f logs/2.log</code> To see the next replica to become primary.





        I've definitely crossed the line where writing a script makes sense, but the good thing is that there's been gradual development up to this point. There is no bottleneck where I need to spend 15 minutes trying to shape various ad-hoc commands from five terminals into a coherent script that was in the file at the beginning.





        And then the script is easy to develop. Once you realize it's a good idea to run the same benchmark against a different, baseline version of TigerBeetle, you change it to <code>./tigerbeetle</code>
        with
        <code>./${tigerbeetle}</code> and wrap everything in


      <figure class="code-block">
        <pre>

async function benchmark(tigerbeetle: string) {

}

const tigerbeetle = Deno.args[0]
await benchmark(tigerbeetle);

      <figure class="code-block">
        <pre>

$ ./make.ts tigerbeetle-baseline
$ ./make.ts tigerbeetle

        A little more hacking, and you end up with a repeatable benchmark schedule for a matrix of parameters:


      <figure class="code-block">
        <pre>

for (const attempt of [0, 1])
for (const tigerbeetle of ["baseline", "tigerbeetle"])
for (const mode of ["normal", "viewchange"]) {
const results = $.path(
./results/<span class="hl-subst">${tigerbeetle}</span>-<span class="hl-subst">${mode}</span>-<span class="hl-subst">${attempt}</span>,
);
await benchmark(tigerbeetle, mode, results);
}

        This is its essence. Don't let shell history be your source, capture it to a file first!

    </div>



<a href

Leave a Comment