Thursday, March 06, 2014

Improve performance: Organize your files into subfolders - Drupal 7

Having too many files in a single folder can be detrimental to your site's performance. It is advisable to segregate files into multiple folders for various benefits. Your sysadmin would surely thank you later :-)

This Acquia post introduces the problem of file organization and suggests two token based modules. Personally, having tried both, File Entity Paths is my choice as it allows you to create folder structures based on properties of the file per-se and not the parent entity. This is useful if you would not like files from changing their location every time they get reused in another entity and need to be in folders based on that entity's properties. By ensuring a folder structure based on file id (fid) for example, you can also guarantee a more even spread and can limit the number of folders created.

For example, to store up to 1,000,000 files with not more than 1000 in a folder, it means we do not need more than 1000 folders. For an even distribution of files, this folder structure could be: [0-9]/[00-99] where the [0-9] is the third digit from the end of the fid, and the [00-99] digits are based on the last two digits of the fid (left pad fids with '0' for fids < 99). So a file 'image.jpg' having fid 123456 would be in files/4/56/image.jpg.

To implement the above logic, we can modify the helpful Drupal 7 drush script to relocate files based on their fid, instead of the timestamp.

Here's the script (Drupal 7):

For new files uploaded via the UI, it's easy to create a file token based on the above logic, to be used in File Entity Paths settings.

/** * Implements hook_token_info(). */ function mymodule_token_info() { $info['tokens']['file']['fid_split'] = array( 'name' => t('Split based on File id'), 'description' => t('Creates two directories based on fid'), ); return $info; }
/** * Implements hook_tokens(). */ function mymodule_tokens($type, $tokens, $data = array(), $options = array()) { $replacements = array(); if (($type == 'file') && !empty($data['file'])) { $fid = str_pad($data['file']->fid, 10, '0', STR_PAD_LEFT); $len = strlen($fid); // Top level dir to be 0 - 9 $top_dir = $fid[$len - 3]; // Sub level dir to be 00 - 99 (100 per top level dir) $sub_dir = $fid[$len - 2] . $fid[$len - 1]; $value = $top_dir . '/' . $sub_dir; $replacements[$tokens['fid_split']] = $value; } return $replacements; }

At the time of this writing, you need to use the latest dev version of fe_paths module (commit dated 16 Oct 2012).

1 comment:

ark bug repellent id said...

I would recommend adding the verbiage "wordpress" in your title. Those who are creating sites would appreciate the clarification before the click on your video and start watching it. Also, it might help those who are using wordpress find this video. Take care.