Too many files in one folder? Try a dirtree.
Too many files in a single folder/directory - 1,000 or more - causes severe performance problems on most file systems. If the files are already linked to in your HTML, simply moving them can be a hassle. Here's a solution that moves your files and uses mod_rewrite to let you leave the old links in place.
Let's say you have a dir called "files1" that has 1,000 files in it and you decide you want to move these into a dirtree (that's my name for a deep directory tree). You can do this with a consistent algorithm based on the filename so that you can systematically figure out which dir any file is in.
First, you need to move the files in "files1" to the deep directory tree, "dirtree." For this example to work, you need files in a dir called "files1" and you need an empty directory called "dirtree" to exist. This step takes an md5 hash of each filename and use the first 2 characters for the first dir and the second 2 characters for the second dir. In other words, if you have a file called 6.txt in the "files1" dir, the script will first make a "dirtree/99/af" dir, then move the 6.txt file to it.
move_files_to_dir_tree.php
<?php
# BEGIN Config
$singledir = "files1";
$dirtree = "dirtree";
# END Config
if ( !is_dir($singledir) || !is_dir($dirtree) )
die("ERROR: $singledir and $dirtree must exist as directories already\n");
$handle = opendir($singledir);
$files = array();
while ($file = readdir($handle))
$files[] = $file;
closedir($handle);
foreach ($files as $file)
{
if ( ! is_file("$singledir/$file") )
continue;
$md5 = md5($file);
$dir1 = substr($md5, 0, 2);
$dir2 = substr($md5, 2, 2);
if ( !is_dir($dirtree."/$dir1") )
mkdir($dirtree."/$dir1");
if ( !is_dir($dirtree."/$dir1/$dir2") )
mkdir($dirtree."/$dir1/$dir2");
rename("$singledir/$file", "$dirtree/$dir1/$dir2/$file");
print "Moved $singledir/$file to $dirtree/$dir1/$dir2/$file\n";
}
Now, your thousands of files are spread out across up to 65K directories inside "dirtree". Now you can store hundreds of thousands of files and only have a few files in each folder. Pretty cool, eh?
But, what if you originally had all your files in "files1" and had tons of HTML with links to these files, maybe even more in forums and blogs? It would be a tremendous task to find and fix all of these. Fortunately, you don't have to. You can use mod_rewrite to do a redirect to your new files.
To do this, you need an .htaccess file in your original "files1" directory. You're going to detect any request for the /files1/filename pattern and have dirtree.php process it.
.htaccess
RewriteEngine On
RewriteBase /
RewriteRule ^files1/([^/]+)$ /dirtree.php?file=$1 [L]
Finally, you need a dirtree.php file containing the following code in the "files1" directory. This sends a redirect header back to the browser to tell it where the files are now.
dirtree.php
<?php
# BEGIN Config
$dirtree = "/dirtree";
# END Config
$file = $_GET['file'];
$md5 = md5($file);
$dir1 = substr($md5, 0, 2);
$dir2 = substr($md5, 2, 2);
header("Location: $dirtree/$dir1/$dir2/$file");
exit;
