Drupal Load Balancing File Sync

When being responsible for more than one site you being to worry about the time when things go wrong and how to avoid downtime of sites you are in charge of. I read a pretty good article about clustering and scaling a drupal website (http://www.johnandcailin.com/blog/john/scaling-drupal-open-source-infrastructure-high-traffic-drupal-sites) and I have been implementing some of the things that are suggested. I have 3 linux web servers to work with, all of varying age and ability. If one of them goes down, eventually I want the the others to take over.

Notes on File Syncing

While going through a few things and reading the comments on the above page I decided to use unision (http://www.cis.upenn.edu/~bcpierce/unison/) to check for changes to the /files directory and sync them between the webhosts. After the initial first run, it is very robust and i will have multiple locations that have an exact copy of the data files. I run two different processes from the same host like this…

unison -silent -repeat 030 /var/www/drupal/files/ ssh://www1.example.org//var/www/drupal/files/  -log -logfile /var/log/unison/www1.log &
unison -silent -repeat 030 /var/www/drupal/files/ ssh://ww2.example.org//var/www/drupal/files/ -log -logfile /rvar/log/unison/www2.log &

These lines above will make unison wait 30 seconds before checking for changes. I just Added these to the /etc/rc.local file so that they will start on boot up and used ssh rsa key authentication to avoid the scripts from asking for a password. (This page covers this pretty well, just adjust it to your environment: http://backuppc.sourceforge.net/faq/ssh.html ) To make sure that the two processes are running on the server I used the webmin Server Status module to add a check that looks for at least two unison processes.

Notes on load balancing with apache mod_proxy

There were a few things that I had to figure out on my own with going throught step 2. I was turning a single server/database setup into a front end server. On the front end server make sure to remove the <DocumentRoot> directives or you will not be able to get to the balancer manager ui. (I just commented mine out in case I needed to go back.)

Since I have several virtual hosts on my server. I opted to leave the /etc/apache2/mods-available/proxy.conf file alone. I did not want to change “Deny from all” to “Allow for all” globally. Thus my config in the front end’s virtual host ended up looking like this. I’ll use http://www.cswtech.com as an example…
(changes in bold)

<Location /balancer-manager>
    SetHandler balancer-manager

    Order Deny,Allow
    Deny from all
    Allow from

<Proxy balancer://cswtechcluster>
    Allow from All  
      # cluster member 1
      BalancerMember http://wwwl.cswtech.com:80 route=cswtech1

      # cluster member 2
      BalancerMember http://www2.cswtech.com:80 route=cswtech2

ProxyPass /balancer-manager !
ProxyPass / balancer://cswtechcluster/ lbmethod=byrequests stickysession=CSWTECHID
ProxyPassReverse / http://www1.cswtech.com/
ProxyPassReverse / http://www2.cswtech.com/
A Note on Drupal configuration.

In the settings.php on each balancer in addition to setting the variable $cookie_domain = 'www.cswtech.com'; I also had to set $base_url = 'http://www.cswtech.com'; as well. If I did not then when I used the site search the links to the results had either cswtech1 or cswtech2 in them.



About Craig Szymanski

Systems Administrator Photography Enthusiast
This entry was posted in Drupal and tagged , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s