squid cache dir

Squid-1.0 used 100 first-level directories for each 'cache_dir'. For very large caches, this meant between 5,000-10,000 files per directory, which isn't good for performance on any unix system. As well as the directory search times being slow, the amount of disk traffic due to directory operations was quite large (due to directory fragmentation (variable length filenames) each directory was about 100k in size).

To reduce the number of files per directory it was necessary to increase the number of directories used. If this was done using a single level directory structure we would have a single 'cache_dir' with an excessive number of directories in it. Hence we went to a 2 level structure. We wanted to keep each directory smaller than a filesystem block (usually 4-8k), and also wanted to be able to accommodate 1M+ objects. Assuming approximately 256 objects per directory, we settled on 16 first-level (L1) and 256 second-level (L2) directories for a total of 16x256x256 = 1,048,576 objects.

The number of L1 and L2 directories to use is configurable in the squid.conf file (swap_level1_dirs, swap_level2_dirs). To estimate the optimal numbers for your installation, we recommend the following formula:

given:
	DS = amount of 'cache_swap' / number of 'cache_dir's
	OS = avg object size = 20k
	NO = objects per L2 directory = 256

calculate:
	L1 = number of L1 directories
	L2 = number of L2 directories

such that:
	L1 x L2 = DS / OS / NO

x=Size of cache dir in KB (i.e. 6GB=~6,000,000KB) y=Average object size
(just use 13KB z=Number of directories per first level directory

(((x / y) / 256) / 256) * 2 = # of directories

As an example, I use 6GB of each of my 13GB drives, so:

6,000,000 / 13 = 461538.5 / 256 = 1802.9 / 256 = 7 * 2 = 14

So my cache_dir line would look like this:

cache_dir 6000 14 256

http://www.linofee.org/~jel/proxy/Squid/rel-notes-1_1.html