Sterling has too many projects

Blogging about Raku programming, microcontrollers & electronics, 3D printing, and whatever else...

Supply Blocks

When you have a stream of data flowing through your Raku application that needs to be accessed safely among threads, you want a Supply. Today we’re going to discuss one particular way of working with supplies, the supply block. If you are familiar with sequences, aka Seq objects, a supply block works in a very similar fashion, but lets you pull work as it arrives and easily do something else in the meantime.

multi a(1) { 1 }
multi a(2) { 2 }
multi a($n where $n > 2) { a($n - a($n-1)) + a($n - a($n-2)) }

my $hofstadter-generator = supply {
    for (1 ... *).map(-> $n { a($n) }) -> $v {
        emit $v;
    }
}

react {
    whenever $hofstadter-generator -> $v {
        note $v;
    }
    whenever Supply.interval(1) {
        say "Waiting...";
    }
}

So we have three sections of code here:

  1. There is a function a() which generates values of the Hofstadter Q-sequence1 (in a particularly inefficient way).

  2. There is a Supply object generated using a supply block which outputs values of of the Hofstadter Q-sequence using the function we defined, starting at the beginning and continuing until the program quits.

  3. And then there is a react block, outputting the values whenever they are available and outputting a waiting message every second in between values. I will go into react blocks in more another day. For now, just know that react blocks allow us to synchronize asynchronous work by pulling the results back into a single thread.

Regarding our central subject, the supply block, it returns a Supply object that can be “tapped.” A Supply is tapped by calling either the .tap or .act methods or by using it in a whenever block as part of a react or another supply block.

In the case of a supply block, the kind of Supply object we get is called an “on demand Supply.” This means that each time the Supply is tapped, the code associated with the block is run. Each time an emit is encountered, the block passed to .tap or the whenever statement will be run and given the value emitted as the argument (named $v in the example code above). Execution continues until either block given to supply exits or the done statement is reached, which causes the supply block to exit (this operates very similar to last for loops).

For example, if we want our sequence to end automatically after 100 iterations, we could rewrite our supply like so:

my $stopping-hofstadter-generator = supply 
    for (1 ... *).map(-> $n { a($n) }) -> $v {
        emit $v;
        done if $n > 100;
    }
}

One very important factor to remember while using a supply block is that the emit command blocks until every tap has finished working.2 This means that that supply block “pays for” the compute time of its taps. This delay provides a form a back-pressure which prevents the generated Supply from running faster than the taps can process. Therefore, if you want your streams of events to run at light speed and don’t care if the tools processing can keep up, you need to either make sure that the taps immediately start new tasks to run to avoid blocking or you want a different mechanism from a plain Supply to handle the workload.

Cheers.


  1. For background on the following integer sequence, see https://oeis.org/A005185.  ↩

  2. This is a feature of all Supply objects, not just ones generated by supply blocks or on–demand supplies.  ↩