Thread Pools with Rayon

19 min read

Spawning a fresh OS thread for every small unit of work is wasteful: thread creation has real cost, and a thousand threads fighting over eight cores thrash more than they compute. A thread pool keeps a fixed set of worker threads alive and feeds them tasks. In Rust the de-facto pool is Rayon, which gives you a ready-made global pool, a join primitive for divide-and-conquer parallelism, and builders for dedicated custom pools.

Quick Overview

A thread pool reuses a small number of long-lived worker threads to run many short tasks, avoiding the per-task cost of std::thread::spawn. Rayon provides this out of the box with a work-stealing scheduler: idle workers steal tasks from busy ones, so the cores stay saturated without you hand-balancing the load. For a TypeScript/JavaScript developer this is the closest analogue to a worker_threads pool (like Piscina) — but Rayon’s tasks are plain Rust closures sharing memory safely, not separate Workers exchanging serialized messages.

Note: This file is about the pool machinery — the global pool, rayon::join, and building custom pools. The high-level par_iter() API that runs on top of these pools has its own file: see parallel-iterators.md.

TypeScript/JavaScript Example

JavaScript runs your code on a single thread. To use multiple cores you reach for worker_threads, and because spawning a Worker is expensive, real apps pool them. A pool maintained by hand looks like this:

1
// Node.js v22 — a hand-rolled fixed worker pool (the shape Piscina automates)
2
import { Worker } from "node:worker_threads";
3
import { availableParallelism } from "node:os";
4

5
interface Task<T> {
6
  payload: unknown;
7
  resolve: (value: T) => void;
8
  reject: (err: Error) => void;
9
}
10

11
class WorkerPool {
12
  private idle: Worker[] = [];
13
  private queue: Task<number>[] = [];
14

15
  constructor(size: number, private readonly script: string) {
16
    for (let i = 0; i < size; i++) {
17
      this.idle.push(this.makeWorker());
18
    }
19
  }
20

21
  private makeWorker(): Worker {
22
    const worker = new Worker(this.script);
23
    worker.on("message", (result: number) => {
24
      const task = (worker as any).currentTask as Task<number>;
25
      task.resolve(result);
26
      this.idle.push(worker); // return to the pool
27
      this.drain();
28
    });
29
    return worker;
30
  }
31

32
  private drain(): void {
33
    if (this.queue.length === 0 || this.idle.length === 0) return;
34
    const worker = this.idle.pop()!;
35
    const task = this.queue.shift()!;
36
    (worker as any).currentTask = task;
37
    worker.postMessage(task.payload); // payload is *copied/serialized*
38
  }
39

40
  run(payload: unknown): Promise<number> {
41
    return new Promise((resolve, reject) => {
42
      this.queue.push({ payload, resolve, reject });
43
      this.drain();
44
    });
45
  }
46
}
47

48
const pool = new WorkerPool(availableParallelism(), "./score-worker.js");
49
const scores = await Promise.all(
50
  documents.map((doc) => pool.run(doc)),
51
);
52
console.log("total:", scores.reduce((a, b) => a + b, 0));

Key points:

You manage the worker count, the idle list, and the queue yourself (or pull in Piscina to do it).
Every postMessage copies the payload (structured clone) across the thread boundary — workers do not share your heap.
Sharing memory at all requires SharedArrayBuffer + Atomics, which only works for raw numeric buffers, not arbitrary objects.

Rust Equivalent

Rayon ships a global pool that is already sized to your machine, so the “score every document” workload above is one line. For divide-and-conquer you use rayon::join, which forks two closures and only actually parallelizes them if a worker is free to pick up the second one.

1
// Cargo.toml: run `cargo add rayon`  (this pulls rayon 1.12)
2
use rayon::prelude::*;
3

4
fn is_prime(n: u64) -> bool {
5
    if n < 2 {
6
        return false;
7
    }
8
    (2..=((n as f64).sqrt() as u64)).all(|d| n % d != 0)
9
}
10

11
// Divide-and-conquer with rayon::join: split the slice until chunks are small.
12
fn sum_of_primes(slice: &[u64]) -> u64 {
13
    if slice.len() <= 1000 {
14
        // Base case: small enough that sequential is faster than forking.
15
        return slice.iter().filter(|&&n| is_prime(n)).sum();
16
    }
17
    let mid = slice.len() / 2;
18
    let (left, right) = slice.split_at(mid);
19
    // Run both halves *potentially* in parallel; returns when both finish.
20
    let (left_sum, right_sum) =
21
        rayon::join(|| sum_of_primes(left), || sum_of_primes(right));
22
    left_sum + right_sum
23
}
24

25
fn main() {
26
    let numbers: Vec<u64> = (0..200_000).collect();
27

28
    // The GLOBAL pool: par_iter() borrows it automatically.
29
    let count = numbers.par_iter().filter(|&&n| is_prime(n)).count();
30
    println!("primes below 200000: {count}");
31

32
    let total = sum_of_primes(&numbers);
33
    println!("sum of those primes: {total}");
34

35
    // How many worker threads back the global pool?
36
    println!("global pool threads: {}", rayon::current_num_threads());
37
}

Running this on an 8-core machine prints the real output:

1
primes below 200000: 17984
2
sum of those primes: 1709600813
3
global pool threads: 8

Key points:

No pool to set up, no queue to manage: par_iter() and rayon::join both use the global pool.
Closures borrow numbers directly — no copying, no serialization. The borrow checker proves the data outlives the parallel work.
The pool defaults to one worker per logical core (8 here), discovered the same way Node finds availableParallelism().

Detailed Explanation

The global pool

The first time you call any Rayon API (par_iter, join, spawn, …), Rayon lazily creates a single process-wide pool with N worker threads, where N is the number of logical cores. Those threads live for the rest of the program and are shared by every Rayon call, so you never pay thread-creation cost per task. rayon::current_num_threads() reports its size.

This is the big structural difference from the Node example: there is no per-task Worker, no idle list, no message queue in your code. Rayon owns one queue per worker thread and balances them with work-stealing.

`rayon::join` and work-stealing

rayon::join(a, b) is the core primitive. It says “these two closures are independent; run them in parallel if it pays off.” The implementation is subtle and worth understanding:

The current worker pushes task b onto its own local deque and starts running a immediately on the current thread.
If another worker is idle, it steals b and runs it concurrently.
If no worker is free, the current thread simply runs b itself after a — sequentially, with almost zero overhead.

That self-tuning behavior is why sum_of_primes can recurse all the way down without a manual “is this chunk big enough to be worth a thread?” heuristic on the thread side. The only tuning you supply is the base-case cutoff (<= 1000 here) that stops the recursion; below it, the forking overhead would outweigh the parallelism. join returns a tuple (A, B) of both closures’ results, so you compose results by returning values, not by mutating shared state.

Custom pools

Sometimes you do not want the global pool: a latency-sensitive request handler should not be starved by a giant background batch job, and a CPU-bound batch should not fight your async runtime’s threads. Build a dedicated pool with ThreadPoolBuilder:

1
// cargo add rayon
2
use rayon::prelude::*;
3
use rayon::ThreadPoolBuilder;
4

5
fn main() {
6
    // A custom pool: fixed size, named threads (named threads show up in
7
    // profilers and panic messages, which is gold when debugging).
8
    let pool = ThreadPoolBuilder::new()
9
        .num_threads(4)
10
        .thread_name(|i| format!("worker-{i}"))
11
        .build()
12
        .expect("failed to build thread pool");
13

14
    // install() runs the closure ON this pool; par_iter inside uses these
15
    // 4 threads instead of the global pool.
16
    let sum: u64 = pool.install(|| (1..=1_000_000u64).into_par_iter().sum());
17
    println!("sum = {sum}");
18
    println!("custom pool threads = {}", pool.current_num_threads());
19

20
    // join can be scoped to a specific pool by wrapping it in install().
21
    let (a, b) = pool.install(|| {
22
        rayon::join(|| (0..500u64).sum::<u64>(), || (500..1000u64).sum::<u64>())
23
    });
24
    println!("a + b = {}", a + b);
25

26
    // scope(): spawn an arbitrary number of tasks that borrow local data and
27
    // are all guaranteed to finish before scope() returns.
28
    let data = vec![1, 2, 3, 4];
29
    let mut results = vec![0; data.len()];
30
    pool.scope(|s| {
31
        for (slot, &value) in results.iter_mut().zip(&data) {
32
            s.spawn(move |_| {
33
                *slot = value * value;
34
            });
35
        }
36
    });
37
    println!("squares = {results:?}");
38
}

Real output:

1
sum = 500000500000
2
custom pool threads = 4
3
a + b = 499500
4
squares = [1, 4, 9, 16]

Three ways to feed a custom pool:

pool.install(closure) — runs closure on the pool and blocks until it returns its value. Any Rayon call inside (par_iter, join, …) uses this pool. This is the workhorse.
pool.scope(|s| ...) — opens a scope where s.spawn(...) launches tasks that may borrow local variables (slot, data); the scope does not return until every spawned task completes, which is what makes the borrows sound.
pool.spawn(closure) — fire-and-forget: the task runs on the pool whenever a worker is free, and spawn returns immediately. The closure must be 'static + Send because nothing waits for it.

Note: A custom pool is a real resource. When the ThreadPool value is dropped, Rayon signals its worker threads to finish and joins them. Keep the pool alive for as long as you submit work to it — usually store it in a struct or a OnceLock.

Configuring the global pool

You can also resize the global pool, but exactly once and before first use:

1
// cargo add rayon
2
use rayon::prelude::*;
3
use rayon::ThreadPoolBuilder;
4

5
fn main() {
6
    // Configure the GLOBAL pool — must happen before any Rayon call uses it.
7
    ThreadPoolBuilder::new()
8
        .num_threads(2)
9
        .build_global()
10
        .expect("global pool already initialized");
11

12
    println!("global threads = {}", rayon::current_num_threads());
13

14
    // A SECOND build_global fails: the global pool can be set only once.
15
    let second = ThreadPoolBuilder::new().num_threads(8).build_global();
16
    println!("second build_global is_err = {}", second.is_err());
17

18
    let total: u64 = (1..=100u64).into_par_iter().sum();
19
    println!("total = {total}");
20
}

Real output:

1
global threads = 2
2
second build_global is_err = true
3
total = 5050

build_global() returns a Result and errors if the pool was already initialized — either by a prior build_global or by any earlier Rayon call that triggered lazy creation. Without calling build_global, the environment variable RAYON_NUM_THREADS also controls the default size, which is handy for ops to tune in production without recompiling.

Key Differences

Concern	Node.js `worker_threads` pool	Rayon thread pool
Default pool	None — you build it (or use Piscina)	Global pool, auto-sized to cores
Task unit	A separate `Worker` + message	A plain closure
Data sharing	Copied via structured clone; `SharedArrayBuffer` for raw bytes only	Closures borrow your heap directly, checked at compile time
Load balancing	Your queue/idle-list logic	Built-in work-stealing scheduler
Result delivery	`postMessage` + `Promise`/event	Return value of `join` / `install`, or a channel
Divide-and-conquer	Manual chunking + many Workers	`rayon::join`, forks only when a core is free
Failure mode	Worker `error` event, pool may wedge	A panicking task propagates the panic out of `join`/`install`

The mental-model shift: in Node, a “thread pool” is a fleet of isolated VMs you message; in Rust, it is a set of threads that run your closures against shared, borrow-checked memory. You parallelize a computation by splitting a borrow (split_at_mut) and handing each half to a worker — something that is simply impossible in JavaScript’s isolate model.

Tip: Rayon is for CPU-bound work (parsing, hashing, image processing, number crunching). For I/O-bound concurrency (HTTP, DB queries), reach for an async runtime like Tokio instead — see ../11-async/README.md. Mixing them is fine: run blocking CPU work on a Rayon pool so it never blocks Tokio’s async workers.

Common Pitfalls

Pitfall 1: Trying to mutate shared state from both `join` closures

A TypeScript developer’s instinct is to have both tasks write into a shared accumulator. Rust’s borrow checker rejects this at compile time, because two closures holding &mut to the same variable is a data race waiting to happen:

1
fn main() {
2
    let mut counter = 0;
3
    // does not compile (error[E0499]: cannot borrow `counter` as mutable
4
    //    more than once at a time)
5
    rayon::join(|| counter += 1, || counter += 1);
6
    println!("{counter}");
7
}

The real compiler error (the line/column refer to the rayon::join call itself):

1
error[E0499]: cannot borrow `counter` as mutable more than once at a time
2
 --> src/main.rs:3:34
3
  |
4
3 |     rayon::join(|| counter += 1, || counter += 1);
5
  |     ----------- -- -------       ^^ ------- second borrow occurs due to use of `counter` in closure
6
  |     |           |  |             |
7
  |     |           |  |             second mutable borrow occurs here
8
  |     |           |  first borrow occurs due to use of `counter` in closure
9
  |     |           first mutable borrow occurs here
10
  |     first borrow later used by call

Fix: have each closure return its contribution and combine the results. This is the whole point of join returning a tuple:

1
fn main() {
2
    // Each half computes its own value; the parent combines them.
3
    let (a, b) = rayon::join(|| 1, || 1);
4
    println!("counter = {}", a + b);
5
}

This prints counter = 2. (If you genuinely need shared mutation, you would use an atomic or a Mutex — see atomic-operations.md — but for divide-and-conquer, returning values is faster and simpler.)

Pitfall 2: Assuming `join` always uses two threads

rayon::join(a, b) does not guarantee a and b run on different threads. If every worker is busy, b runs sequentially after a on the same thread. Never write code whose correctness depends on the two closures overlapping in time — join is an optimization hint, not a concurrency guarantee. (For guaranteed concurrent execution with explicit threads, see threads.md.)

Pitfall 3: Forgetting the base-case cutoff in recursive `join`

Recursing with join all the way down to single elements drowns the real work in forking overhead. Always stop forking once chunks are small enough to process sequentially (the slice.len() <= 1000 check in sum_of_primes). Without it, a parallel version can be slower than the sequential one.

Pitfall 4: Letting a custom pool be dropped too early

Because dropping a ThreadPool shuts down its workers, this is a classic mistake:

1
// Conceptually broken: the pool is dropped at the end of the function,
2
// so the spawned task may be cut short.
3
use rayon::ThreadPoolBuilder;
4

5
fn start_background() {
6
    let pool = ThreadPoolBuilder::new().num_threads(2).build().unwrap();
7
    pool.spawn(|| { /* long-running work */ });
8
    // `pool` drops here → its workers are signaled to finish. Bad for spawn().
9
}

Keep the pool alive (store it in a long-lived struct or a static OnceLock) for the lifetime of the work you submit to it. install and scope block until their work finishes, so they are safe; spawn is the one to watch.

Best Practices

Default to the global pool. For ordinary CPU-bound work, just call par_iter() or rayon::join and let Rayon size and manage the pool. Reach for a custom pool only when you need isolation (background batch vs. request path) or a non-default size.
Compose with return values, not shared mutation. join and reduce hand you results; prefer them over Mutex/atomics inside the hot loop.
Tune the cutoff, not the thread count. The lever for recursive join performance is the sequential base-case size, not how many threads you spawn.
Name your pool threads. thread_name(|i| format!("scorer-{i}")) makes panics, profilers, and top output readable.
Propagate errors cleanly. A parallel pipeline of fallible work can collect() into a Result<Vec<T>, E> (stops conceptually at the first error) or use try_reduce/try_for_each:

1
// cargo add rayon
2
use rayon::prelude::*;
3

4
fn main() {
5
    // Collecting Results: any Err short-circuits to a single Err.
6
    let inputs = vec!["10", "20", "30", "oops", "40"];
7
    let parsed: Result<Vec<i64>, _> =
8
        inputs.par_iter().map(|s| s.parse::<i64>()).collect();
9
    println!("parse result is_err = {}", parsed.is_err());
10

11
    // try_reduce: fold in parallel, bailing on the first error.
12
    let good = vec!["10", "20", "30"];
13
    let sum: Result<i64, _> = good
14
        .par_iter()
15
        .map(|s| s.parse::<i64>())
16
        .try_reduce(|| 0, |acc, n| Ok(acc + n));
17
    println!("sum = {sum:?}");
18
}

Real output:

1
parse result is_err = true
2
sum = Ok(60)

Keep Rayon away from blocking I/O. A worker blocked on a socket cannot steal other tasks. Do I/O on an async runtime and feed only CPU work to Rayon.

Real-World Example

A document-scoring batch job — the Rust counterpart to the Node WorkerPool at the top. It runs on a dedicated pool (so a huge batch never starves the rest of the service) and combines per-document scores with a parallel reduce:

1
// cargo add rayon
2
use rayon::prelude::*;
3
use rayon::ThreadPoolBuilder;
4
use std::collections::HashMap;
5

6
/// A CPU-bound job: score a document by word-frequency (sum of squared counts).
7
fn score_document(text: &str) -> u64 {
8
    let mut counts: HashMap<&str, u64> = HashMap::new();
9
    for word in text.split_whitespace() {
10
        *counts.entry(word).or_insert(0) += 1;
11
    }
12
    counts.values().map(|&c| c * c).sum()
13
}
14

15
fn main() {
16
    let documents: Vec<String> = (0..10_000)
17
        .map(|i| format!("doc {i} alpha beta beta gamma gamma gamma"))
18
        .collect();
19

20
    // A dedicated pool isolates this batch from the global pool / request path.
21
    let pool = ThreadPoolBuilder::new()
22
        .num_threads(4)
23
        .thread_name(|i| format!("scorer-{i}"))
24
        .build()
25
        .expect("pool");
26

27
    // install() runs the whole pipeline on the 4 dedicated workers.
28
    let (total, max) = pool.install(|| {
29
        documents
30
            .par_iter()
31
            .map(|doc| score_document(doc))
32
            .map(|score| (score, score))
33
            // Parallel reduce: an associative combine of per-document results.
34
            .reduce(
35
                || (0, 0),
36
                |(sum_a, max_a), (sum_b, max_b)| {
37
                    (sum_a + sum_b, max_a.max(max_b))
38
                },
39
            )
40
    });
41

42
    println!("documents scored: {}", documents.len());
43
    println!("total score: {total}");
44
    println!("max single-doc score: {max}");
45
}

Real output:

1
documents scored: 10000
2
total score: 160000
3
max single-doc score: 16

Compared to the Node version, there is no message queue, no idle-worker bookkeeping, and no serialization: par_iter() borrows documents directly, the 4 named workers steal chunks from each other, and reduce folds the results in parallel. The pool is dropped at the end of main, cleanly joining its workers.

Exercises

Exercise 1: Parallel quicksort with `rayon::join`

Difficulty: Medium

Objective: Use rayon::join for genuine divide-and-conquer parallelism over a mutable slice.

Instructions: Write fn parallel_quicksort<T: Send + Ord + Copy>(slice: &mut [T]) that partitions the slice around a pivot, then sorts the two partitions with rayon::join. Use split_at_mut to hand each half a disjoint mutable borrow. Verify the result is sorted.

Solution

1
// cargo add rayon
2
fn parallel_quicksort<T: Send + Ord + Copy>(slice: &mut [T]) {
3
    if slice.len() <= 1 {
4
        return;
5
    }
6
    let pivot_index = partition(slice);
7
    // split_at_mut gives two disjoint &mut halves — no aliasing, so join is sound.
8
    let (left, right) = slice.split_at_mut(pivot_index);
9
    rayon::join(
10
        || parallel_quicksort(left),
11
        || parallel_quicksort(&mut right[1..]), // skip the pivot at right[0]
12
    );
13
}
14

15
fn partition<T: Ord + Copy>(slice: &mut [T]) -> usize {
16
    let len = slice.len();
17
    let pivot = slice[len / 2];
18
    slice.swap(len / 2, len - 1);
19
    let mut store = 0;
20
    for i in 0..len - 1 {
21
        if slice[i] < pivot {
22
            slice.swap(i, store);
23
            store += 1;
24
        }
25
    }
26
    slice.swap(store, len - 1);
27
    store
28
}
29

30
fn main() {
31
    let mut data = vec![9, 3, 7, 1, 8, 2, 6, 5, 4, 0, 11, 10];
32
    parallel_quicksort(&mut data);
33
    println!("sorted = {data:?}");
34
    assert!(data.windows(2).all(|w| w[0] <= w[1]));
35
}

Output:

1
sorted = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

The key insight is that split_at_mut proves to the compiler that the two halves never overlap, which is exactly what lets rayon::join run them in parallel safely.

Exercise 2: A bounded custom pool

Difficulty: Medium

Objective: Build a custom ThreadPool of a chosen size and run a parallel reduction on it.

Instructions: Write fn batch_max(values: &[u64], threads: usize) -> u64 that builds a ThreadPool with threads workers and uses install + par_iter().reduce to compute the maximum. Confirm it returns the right answer with threads = 2.

Solution

1
// cargo add rayon
2
use rayon::prelude::*;
3
use rayon::ThreadPoolBuilder;
4

5
fn batch_max(values: &[u64], threads: usize) -> u64 {
6
    let pool = ThreadPoolBuilder::new().num_threads(threads).build().unwrap();
7
    // reduce needs an identity (u64::MIN) and an associative combiner (u64::max).
8
    pool.install(|| values.par_iter().copied().reduce(|| u64::MIN, u64::max))
9
}
10

11
fn main() {
12
    let m = batch_max(&[3, 99, 42, 7, 88], 2);
13
    println!("max = {m}");
14
    assert_eq!(m, 99);
15
}

Output:

1
max = 99

reduce’s identity element (u64::MIN) must be neutral for the operation — max(x, MIN) == x — and the combiner must be associative so partial results from different workers compose correctly.

Exercise 3: Fire-and-forget tasks with a result channel

Difficulty: Hard

Objective: Use pool.spawn for detached tasks and collect their results over an mpsc channel.

Instructions: Build a 4-thread pool, spawn 8 jobs that each compute job_id * job_id, and send (job_id, result) down an mpsc channel. After spawning, drop the original sender and drain the receiver into a sorted Vec. Explain why dropping the sender matters.

Solution

1
// cargo add rayon
2
use rayon::ThreadPoolBuilder;
3
use std::sync::mpsc::channel;
4

5
fn main() {
6
    let pool = ThreadPoolBuilder::new()
7
        .num_threads(4)
8
        .thread_name(|i| format!("task-{i}"))
9
        .build()
10
        .unwrap();
11

12
    let (tx, rx) = channel();
13
    for job_id in 0..8u32 {
14
        let tx = tx.clone();
15
        // spawn detaches the task; it runs whenever a worker is free.
16
        pool.spawn(move || {
17
            let result = job_id * job_id;
18
            tx.send((job_id, result)).unwrap();
19
        });
20
    }
21
    // Drop the original sender: rx.iter() ends only once EVERY sender is gone.
22
    drop(tx);
23

24
    let mut results: Vec<(u32, u32)> = rx.iter().collect();
25
    results.sort();
26
    println!("results = {results:?}");
27
}

Output:

1
results = [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)]

rx.iter() blocks until all senders are dropped. Each spawned task holds a cloned tx that drops when the task finishes, but the original tx would keep the channel open forever — so we drop(tx) explicitly after the spawn loop. The pool stays alive through the end of main, so every detached task gets to run. See channels.md for more on this pattern.

Thread Pools with Rayon

Quick Overview

TypeScript/JavaScript Example

Rust Equivalent

Detailed Explanation

The global pool

rayon::join and work-stealing

Custom pools

Configuring the global pool

Key Differences

Common Pitfalls

Pitfall 1: Trying to mutate shared state from both join closures

Pitfall 2: Assuming join always uses two threads

Pitfall 3: Forgetting the base-case cutoff in recursive join

Pitfall 4: Letting a custom pool be dropped too early

Best Practices

Real-World Example

Further Reading

Exercises

Exercise 1: Parallel quicksort with rayon::join

Exercise 2: A bounded custom pool

Exercise 3: Fire-and-forget tasks with a result channel

`rayon::join` and work-stealing

Pitfall 1: Trying to mutate shared state from both `join` closures

Pitfall 2: Assuming `join` always uses two threads

Pitfall 3: Forgetting the base-case cutoff in recursive `join`

Exercise 1: Parallel quicksort with `rayon::join`