Capacity is a boundary.
A fit estimate does not predict throughput. A larger memory budget can avoid an out-of-memory condition without making a job faster.
Give the workload
some room.
Change the assumptions. Inspect the budget. Keep the plan.
Reference size at FP32
1 GiB64 GiB
164
Example device capacity
ESTIMATED WORKING SETWITHIN BUDGET
GiB
15% RESERVE
42% USED Working set Available Reserve
- Parameters at chosen precision
- GiB
- Activations × batch
- GiB
- Runtime allowance
- 2.0 GiB
- Available after reserve
- GiB
This example fits within the planning budget.
ESTIMATE, NOT A BENCHMARK.Read the formula and limits ↗
Leave deliberate headroom.
This model reserves 15% of the chosen capacity. Allocator behaviour and temporary peaks still require measurement on the intended stack.
Keep devices separate.
Multiple devices are not treated as one memory pool. Distribution and communication must be designed for the workload.