Sandbox milestones
Concurrency by phase
Loading latency scatter…
| Phase | P50 | P95 | P99 | Mean | Count |
|---|---|---|---|---|---|
Allocate sandbox.create() | 14.10s | 17.57s | 18.03s | 12.98s | 109,720 |
Readiness node -v after allocate | 189ms | 441ms | 615ms | 222ms | 109,720 |
| Segment | Idx range | Count ok | P50 | P95 | P99 | Max | Mean |
|---|---|---|---|---|---|---|---|
| First 25% | 0-27,709 | 27,430 | 6.54s | 14.35s | 14.90s | 15.96s | 7.05s |
| Middle 50% | 27,710-82,569 | 54,860 | 15.41s | 17.79s | 18.16s | 18.77s | 15.23s |
| Last 25% | 82,570-119,839 | 27,430 | 14.20s | 17.37s | 17.85s | 18.27s |
What we test
Each provider gets one attempt to hold 100,000 sandboxes concurrently from a cold start. The one variable that we successfully measured (without confounding variables) is that each provider successfully held 100,000 live sandboxes concurrently during the test. Some may have failures caused by quota/rate limits (not a true failure) or by us sending more than 100k create() requests.
Execution
One thousand coordinator processes run in parallel — each on its own Namespace VM, each responsible for up 140 sandboxes. All 100,000 sandbox.create() calls fire simultaneously; there is no stagger or ramp. Sandboxes are numbered (0-indexed) in submission order so ordering effects can be measured.
Lifecycle
Each sandbox moves through four steps: allocate (create returns), readiness probe (node -v inside it — failure → readiness_failed), alive hold (all 100,000 sandboxes held simultaneously while fleet-wide peak concurrency is recorded), then liveness check (a second node -v — pass → success, fail → partial). The Invitational score is the fleet-wide peak concurrent count at the hold.
Measurement
Timing is recorded by the coordinator at the moment each API call returns or errors — it does not include client-side retry logic.
Benchmark phases
sandbox.create() until the API confirms the sandbox exists.node -v executes and returns successfully inside it.partial.VM metrics
Each VM managed up to 140 sandboxes. One thousand ran in parallel to produce the full 100,000+ sandbox burst. System metrics shown here reflect a single coordinator VM's resource usage — all 1,000 coordinators ran on identical hardware.
| 14.42s |
/proc/net/sockstatSandbox milestones
Concurrency by phase