Setting Up Varnish & Memcache with Aegir

I had the hardest time finding resources & documentation on how to get this configured properly so I am adding this to the documentation for those of you who may need it.

A couple of notes: I am using Ubuntu 10.04 LTS. I set my apache to port 8082 because there was a conflict on port 8080 on my system. You should make a copy of each file before editing it. Memcache settings are drupal 7.

Varnish

Setting up varnish isn't too hard but it could end up being kind of tedious if you configure something incorrectly & you have alot of sites.

First you install it sudo apt-get install varnish

Then you have to go to configure it. I use vim. sudo vim /etc/varnish/default.vcl

These are my settings.

backend default {
    .host = "127.0.0.1";
    .port = "8082";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}
sub vcl_recv {
  // Remove has_js and Google Analytics __* cookies.
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
  // Remove a ";" prefix, if present.
  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
  // Remove empty cookies.
  if (req.http.Cookie ~ "^\s*$") {
    unset req.http.Cookie;
  }
 

// Skip the Varnish cache for install, update, and cron
  if (req.url ~ "install\.php|update\.php|cron\.php") {
    return (pass);
  }
 

// Cache all requests by default, overriding the
  // standard Varnish behavior.
  // if (req.request == "GET" || req.request == "HEAD") {
  //   return (lookup);
  // }
}
sub vcl_hash {
  if (req.http.Cookie) {
    set req.hash += req.http.Cookie;
  }
}

Now you have to edit /etc/default/varnish sudo vim /etc/default/varnish

Here are my settings

# Should we start varnishd at boot?  Set to "yes" to enable.
START=yes
# Maximum number of open files (for ulimit -n)
NFILES=$(ulimit -n)
#Malloc size depend on your memory server
DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,512M \
             -u www-data \
             -g www-data"

Now we have to configure apache's listening ports

sudo vim /etc/apache2/ports.conf

change

NameVirtualHost *:80
Listen 80

So that it looks like this

NameVirtualHost *:8082
Listen 8082

Start varnish services varnish start

Now login to aegir Edit your server & change the port from 80 to 8082

Aegir will re-verify the server, as well as all platforms & sites on the server.

If that doesnt work, you need to go to /var/aegir/config/server_master/apache/vhost.d and edit all of your virtual hosts so that the port shows 8082 instead of 80 then do the same thing with /var/aegir/config/server_master/apache.conf Reverify your server.

Now add this to the local.settings.php of the sites that you want to leverage varnish

<?php
# Tell Drupal it's behind a proxy
$conf['reverse_proxy'] = TRUE;
# Tell Drupal what addresses the proxy server(s) use
$conf['reverse_proxy_addresses'] = array('127.0.0.1','your-ip-address-here');
# Bypass Drupal bootstrap for anonymous users so that Drupal sets max-age &gt; 0
$conf['page_cache_invoke_hooks'] = FALSE;
?>

Memcache

There is hardly any current documentation for setting up memcache on D7, especially not for aegir.

FIrst you install memcache & its dependencies Download & install memcache module into the platform of the site/sites in which you will be using memcache

Then you have to edit /etc/default/memcached sudo vim ./etc/init.d/memcached

Here are my settings

#! /bin/sh
#

PORT=11211
USER=nobody
MAXCONN=1024
OPTIONS=""
DAEMON=/usr/bin/memcached

RETVAL=0
prog="memcached"

start_instance() {
        echo -n $"Starting $prog ($1): "
        start-stop-daemon --start --quiet --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON -- -d -p $PORT -u $USER  -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid $OPTIONS
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch /var/lock/memcached.$1
        PORT=`expr $PORT + 1`
}

stop_instance() {
        echo -n $"Stopping $prog ($1): "
        start-stop-daemon --stop --quiet --oknodo --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON 
        RETVAL=$?
        echo
        if [ $RETVAL -eq 0 ] ; then
            rm -f /var/lock/memcached.$1
            rm -f /var/run/memcached/memcached.$1.pid
        fi
}
start () {
        # insure that /var/run/memcached has proper permissions
        mkdir -p /var/run/memcached
        if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
                chown $USER /var/run/memcached
        fi

        start_instance default 64;
        start_instance block 16;
        start_instance content 128;
        start_instance filter 128;
        start_instance form 32;
        start_instance menu 16;
        start_instance page 8;
        start_instance update 8;
        start_instance views 8;
        start_instance path 8;
        start_instance field 8;
        start_instance rules 8;
        start_instance token 8;
        start_instance image 8;
        start_instance apachesolr 16;
}
stop () {
        stop_instance default;
        stop_instance block;
        stop_instance content;
        stop_instance filter;
        stop_instance form;
        stop_instance menu;
        stop_instance page;
        stop_instance update;
        stop_instance views;
        stop_instance path;
        stop_instance field;
        stop_instance rules;
        stop_instance token;
        stop_instance image;
        stop_instance apachesolr;
}

restart () {
        stop
        start
}

# See how we were called.
case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  status)
        status memcached
        ;;
  restart|reload|force-reload)
        restart
        ;;
  *)
        echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
        exit 1
esac

exit $?

Start memcache sudo services memcached start If you see errors u may have an issue where a one line got split into two lines. Check and make sure that none of your lines got broken off.

Place this code in your local.settings.php file inside your site folder

// the path to the core cache file
  include_once('./includes/cache.inc');
  // the path to the memcache cache file
  include_once('./sites/all/modules/memcache/memcache.inc');
  // make MemCacheDrupal the default cache class
  $conf['cache_default_class'] = 'MemCacheDrupal';
  
  # Key Prefix: edit this for multisite use.
  $conf['memcache_key_prefix'] = "$_SERVER[db_user]";
  
  $conf['memcache_servers'] = array(
  '127.0.0.1:11211' => 'default',
  '127.0.0.1:11212' => 'menu',
  '127.0.0.1:11213' => 'filter',
  '127.0.0.1:11214' => 'form',
  '127.0.0.1:11215' => 'block',
  '127.0.0.1:11216' => 'update',
  '127.0.0.1:11217' => 'views',
  '127.0.0.1:11218' => 'content',
  '127.0.0.1:11219' => 'apachesolr',
  '127.0.0.1:11220' => 'path',
  '127.0.0.1:11221' => 'field',
  '127.0.0.1:11222' => 'rules',
  '127.0.0.1:11223' => 'token',
  '127.0.0.1:11224' => 'image',    
);
$conf['memcache_bins'] = array(
  'cache' => 'default',
  'cache_menu'   => 'menu',
  'cache_filter' => 'filter',
  'cache_form'   => 'form',
  'cache_block'  => 'block',
  'cache_update' => 'update',
  'cache_views'  => 'views',
  'cache_views_data'  => 'views',
  'cache_content'  => 'content',
  'cache_apachesolr'  => 'apachesolr',
  'cache_path'  => 'path',
  'cache_field'  => 'field',
  'cache_rules'  => 'rules',
  'cache_token'  => 'token',
  'cache_image'  => 'image',
);

And that should be basically it.