Instrument, not marketing
jev-curate benchmark: 24.0 rows/sec measured
This page reports the numbers the repository actually measures. Run the same benchmark yourself with cargo run --release --example bench_mock. No network, no TypeSafe credits, nothing hidden.
jev-curate benchmark (mock server, no network)
machine cores: 4
rows: 120, concurrency 1 and 32
sequential: 24.0 rows/sec (42 ms/row)
concurrent 32: 24.0 rows/sec (42 ms/row)
note: client rate limiter defaults to 20 req/sec
(TypeSafe 1,200 req/min policy)
What was measured
The end to end pipeline, exactly as shipped: the token bucket rate limiter, the HTTP client, and one Jev evaluation per row against an in-process wiremock server. One hundred twenty rows, run twice, on release build.
| Run | Rows | Concurrency | Rows/sec | ms/row |
|---|---|---|---|---|
| Sequential | 120 | 1 | 24.0 | 42 |
| Concurrent | 120 | 32 | 24.0 | 42 |
Sequential and concurrent land on the same number. That is the finding, not a rounding accident: throughput is bound by the rate limiter, not by CPU or IO.
Why the ceiling is there
jev-curate sends one fan-out request per row. Jev accepts 1,200 requests per minute, so the client's token bucket refills at 20 tokens per second. One row per token means a hard ceiling near 20 rows per second on a single worker, with short bursts above it. Measured: 24.0 rows/sec, matching a 20-token burst plus the steady refill.
Raising that ceiling is a batching problem, not a compute problem. Rows are independent, so the pipeline scales horizontally: at 24 rows/sec per node, the 1,500 rows/sec figure in the project target is reachable with roughly 63 worker nodes. The target did not misreport a single-node speed; this page exists so nobody has to guess which one is true.
The method, so it is reproducible
The bench lives at examples/bench_mock.rs and runs with cargo run --release --example bench_mock. It boots a wiremock server, mounts one canned successful response at the systemone path, and runs 120 rows twice: once with concurrency 1 and once with concurrency 32. The client is the real JevClient with the shipped token bucket limiter and speculative fan-out. Rows are identical payloads, which removes sampling noise and also means the number measures the pipeline, not the dataset. Budget: 120 rows at about 42 ms each, so the whole run takes roughly ten seconds.
Hardware for this run: an Intel Core i3-1115G4, 2 cores 4 threads, 8 GB RAM, Windows 11, release build. A faster machine will not move the reported number much, because the limiter, not the CPU, is the pinch point. What the number does not claim: it is not a public-API throughput guarantee, it is not a Parquet IO benchmark, and it does not cover a mixed dataset. It is the honest ceiling of this pipeline against a zero-latency judge, and that is enough to separate the target from the measurement.
Scaling, in round numbers
Rows are independent, so the pipeline scales by worker count. The table is arithmetic on the measured 24.0 rows/sec per node, not a promise.
| Nodes | Expected rows/sec | Rough daily volume |
|---|---|---|
| 1 | 24 | 2.1M |
| 4 | 96 | 8.3M |
| 16 | 384 | 33.2M |
| 63 | 1,512 | 130M |
1,500 rows/sec is a cluster target and this page says so. When a single node in a real deployment lands a different number, the right move is to publish that number next to this one, not to adjust the table.