Best practices for cpu-intensive actors

[cristiano-belloni]

How do you approach CPU-intensive tasks with actors? The first idea is creating an actor that manages a worker pool and asynchronously round-robins it, but:

1. Does it even make sense? Actors are probably sequential, so the pool will never be used apart from the first thread.
2. How would it scale?
3. I assume it wouldn’t work with certain runtimes (e.g. Cloudflare)

Is there any meaningful pattern to handle this with Rivet?

[NathanFlurry]

Does it even make sense? Actors are probably sequential, so the pool will never be used apart from the first thread.

Under the hood, actors are just stateful tasks with a unique network address for each one. Vanilla actors do not have the same single-message concurrency that you might be thinking of with other actor engines.

There should be no issue with this part.

I assume it wouldn’t work with certain runtimes (e.g. Cloudflare)

Yep – Cloudflare has a lot of limitations we can't control.

---

We're launching the Rivet Engine in a couple weeks which will make this a bit more clear how this works at scale, but I'll do my best to answer here first.

In Rivet, the process that runs the actors is called the "runner." You're correct that they're single threaded, so you'd have to set up a worker pool to run CPU-intensive applications.

There's 3 approaches to this:

Option 1: Child process

The first option is to create a child process for every individual actor. Inside your actor's actions, you can send messages to the child process.

For example:

const { fork } = require('child_process');

const cpuIntensiveActor = actor({
  createVars: () => ({
    worker: undefined,
  }),
  onStart: c => {
    c.vars.worker = fork('./worker.js');
    c.vars.worker.on('message', (msg) => {
      // Handle worker responses
    });
  },
  onStop: c => {
    c.vars.worker?.kill();
  },
  actions: {
    doSomethingWithWorker: (c, input: unknown) => {
      c.vars.worker?.send(input);
    }
  }
});

It looks a bit like this:

Note that this has 2 things we don't support but are on the roadmap:

• onStart does not have access to c.vars
• onStop is not exposed yet

This would take a few minutes to ship – let me know if this is the option you go with and I'll unblock you.

Option 2: Fork RivetKit and add native actors as child processes

What you're describing has been considered a few times. It'd be really nice to have native process isolation as an opt-in option – similar to what Vitest does.

It' look like this:

If you're up to contribute, I'd love to work with you on this.

Option 3: Run individual runners per thread

The last option is to just run a bunch of individual runners that are idle when there's no work going on.

When using RivetKit with the Rivet Engine, you'll have the option to configure the maximum actors per a runner. Setting this to 1 would mean that you can do CPU intensive work without having multiple actors on the same runner.

It'd look like this:

---

I'd recommend option 1 for now. Option 3 will be better at scheduling workloads when Rivet Engine is released since it will allow you to cap it to 1 actor per core.

[cristiano-belloni]

Thanks for the detailed response! 1) would be certainly the best option (for now at least). I supposed you had a process in the example but it can also be a worker (depending on the engine I'm running on).
I also guess there's no easy way to spawn a pool, to remove the worker / process creation overhead.

[NathanFlurry]

I also guess there's no easy way to spawn a pool, to remove the worker / process creation overhead.

You can certainly use cluster and maintain your own pool. The main issue here is that until Rivet Engine is released (which allows you to cap the number of actors running), there's no guarantees that the number of actors on a runner will be smaller than your pool size.

I supposed you had a process in the example but it can also be a worker (depending on the engine I'm running on).

Can you provide more details on what you're looking for? This is certainly a use case we want to support.

[cristiano-belloni]

Literally using https://bun.com/docs/api/workers (or the Node counterpart) to offload calculations. Initialising them in a pool at server start instead of creating one of them when a query comes removes the spawning overhead.
From what I understand, Rivet creates a dynamic quantity of actors per runner, so there are no guarantees of when an actor is started or shutdown or how much it stays alive (correct me if I'm wrong), so spawning a pool of workers on actor start is something essentially hard to control in terms of quantity / overhead?