Developer Manual


This section of the documentation is aimed at developers interested in the Aegir project.

We use Git as a version control system for managing the project code, so we have specific instructions on how to use git.

Developers wishing to collaborate directly with the core team will want to read the release engineering guide.

Developers working on third-party modules should read the guide to extend Aegir and the API documentation.

Developers interested in developing on, for or with Aegir might be interested in Aegir-up, which provides a fully installed and configured virtual machine on Virtualbox/Vagrant. The Puppet module on which it is based supports an $aegir_dev_build option, that will install Provision and Hostmaster from the git repos, preserving the .git repos.

Git repositories locations and instructions


The source code of Aegir is located on, as with the issue queues and release nodes.

Aegir is also made up of several separate components designed to work together, that are grouped in two main git repositories: hostmaster and provision. This page details those locations.

Newcomers to git should probably consult our git crash course while people already familiar with git can have a reminder on how to use it within aegir through the basic git workflow page.

Repositories locations

There are the URLs you can use to pull the repositories:

git clone
git clone

Developers with push access to the repositories should pull via SSH.

git clone ssh://[username]
git clone ssh://[username]

There is a web interface for your perusal at:

The git repositories were hosted for a while on Koumbit's servers, but were migrated back to during the great migration. Do not use as it doesn't have the most recent commits and may be shutdown at any time.

A crash course in Git

Clone a repo

Let's take hostmaster:

git clone

If successful, you will have a hostmaster directory at the last state of development ('HEAD' in CVS speak, usually 'master' branch in git speak, though in our case this is now 6.x-1.x).

Check the status of the local git repository:

git status

See the commit messages:

git log

Check all 'releases'

git tag

Create a new branch for your local development:

git checkout -b <new_branch_name>

Create a new branch for development, based on an existing remote branch or tag

git checkout -b <new_branch_name> origin/<branch_name>

Basic git workflow: committing and pushing

So say you have some work you want to do on the hostmaster module. You first need to clone the repository:

git clone

That will create a hostmaster directory with the whole history of the project, and a checkout of the "master" branch (which is the equivalent of CVS's "HEAD" in git).

You make some modifications to the code to fix a bug, and then, you make a commit. That commit will only be local for now, that's the way git works:

vi hostmaster.profile
git commit -m "#12345 fix the install profile to do X" hostmaster.profile

Note that contrarily to subversion or CVS, you need to explicitly mention which file(s) you want to commit. You can also use the -a flag to commit all changes.

If you have write access to the repository, you can now push your changes straight into git. However, the git:// url above is readonly, so you will want to use SSH to push your changes. For that, we will have had to add your SSH public key to our repositories. You may also want to add a special "remote" repository to push your changes. You will need to do this only once (per repository):

git remote add drupalorg ssh://[username]

You can now push your changes to that repository.

git push drupalorg

If you do not have write access to the repository, you can send the Aegir developers patches by submitting them for review in the Issue queues (see the patch contributor guide on for more details) or by uploading them to your personnal sandbox.

Branch and tagging conventions

Aegir development makes a heavy use of branches to keep development stable. We use feature branches to do large development work and have release branches to keep stable releases maintained while development happen on the development branches.

1. Summary

  • The current development branch is: 7.x-3.x
  • The current stable branch is: 6.x-2.x

2. Branch naming conventions

All branches are either development branches, stable branches or feature branches. Development and stable branches have the same naming convention: 6.x-2.x or 7.x-3.x. The only different between the two is that a stable branch is the branch where stable releases are published from.

Note: since the great migration, we are back on and have been forced to change some of our naming convention, especially to include the core release number in the tag (so 6.x-2.x), which is a source of confusion for our provision module which supports any Drupal core version. See this issue on for followup.

2.1. Main development branch

The main development branch is created after a fully stable release is published. For example, the 6.x-2.x branch has been created when the 1.0 release was published. It is a development branch until it is considered feature-complete and goes through the alpha/beta/rc release cycle, at which point it will be declared a stable branch and a new development branch will be created (7.x-3.x at that point).

The development branch is not stable and should not be used in production environments.

Note that we have dropped the master branch as it is not well supported on for releases.

2.2. Stable release branches

Stable releases are performed from stable release branches. A branch is declared stable only after it went through the complete alpha/beta/rc release cycle, as detailed above..

Once the branch is declared stable, new releases are quickly done as "hotfixes", without intervening alpha/beta/rc releases. So for example, 6.x-1.0 will come after 6.x-1.0-alpha1, 6.x-1.0-beta1, 6.x-1.0-rc1 at least, and then be followed directly by 6.x-1.1.

Releases on the same stable branch are API-compatible with each other (so 1.1 is compatible with 1.0). The upgrade path is supported between major stable releases (so you can upgrade from 1.0 to 1.1 or 2.0).

Special case: during the 0.x release cycle, releases were not API-compatible with each other, ie. 0.4 was not compatible with 0.3, and you could barely upgrade from 0.3 to 0.4.

Only one stable branch is maintained at a time.

2.3. Feature branches

More complex development goes into development or "feature" branches, to avoid making master too unstable. Feature branches must be prefixed with "dev-" and use the issue number (if available or relevant) and a short name. So for example, there could be work on a dev-12345-mybug branch or dev-dns branch for a larger feature (DNS, in that case) which doesn't fit in a single issue.

Feature branches should be based of a known stable baseline as much as possible. For example, you should branch off the last alpha release if you want to make merging with other dev branches and master easier. See this article for a thorough discussion on this practice

3. Tag naming convention

Every release is tagged with a 6.x-x.y tag (e.g. 6.x-0.4, 6.x-0.4-alpha15 or 6.x-1.0). Those tags are usually laid on the stable branches, except for alpha/beta/rc releases which are laid on the development branch.

4. CVS and git history

The CVS tags and branches are still accessible in git and will be kept there for posterity. They respect the traditional naming convention.

They have also been converted to the new conventions during the great migration.

Since that migration (february 2011), history has been rewritten to clean up the authors in the history.

The great migration


After thorough discussions where we weighted the pros and cons of profiting from the Great Git Migration, I think it's pretty clear that we have a great opportunity to migrate our source repositories back to We will get:

  • more visibility - people will find us through easily
  • more consistency - versions in the issue queue matching the real releases
  • even more consistency - people will find the source better from
  • and then more consistency - usage stats will work again!

We will be able to:

  • host our module and theme within the install profile - the major reason why we migrated away back then
  • use whatever naming convention we want for tags and branches - although only some will be used for releases for now, see this issue for follow-up
  • keep our history - the git team was nice enough to clone directly from our repositories (although see below, we're taking the opportunity of the migration to clean up our history)

What it means for you

This does have an impact on your existing repositories. Since we migrated to, the repositories have changed location. The other thing is that the commit IDs have changed, and you will need to clone new checkouts before you can work with them effectively.

So in short:

  • repos will move to
  • you need to clone again
  • make sure your identity is set correctly

URL changes

So the repositories have moved. Everyone has to fetch the source code from now. At some point in the future, Koumbit will close Concretely, Koumbit will keep the mirror running readonly for a while just to be on the safe side, but you shouldn't expect to work and should switch URLs, as detailed below.

This documentation is taken from the git workflows documentation.

Readonly repositories URL change

The readonly access will change from:



See also the git patch maintainer guide.

Core committers repositories URL change

The read/write access will change from:




... notice how you will need to set your username in the URL. has good documentation on how authentication works on See also the project maintainer git guide.


The migration to also introduces sandboxes, something we didn't have before, and which allows you to host your own branches of our modules (even if you're not a core committer) or contrib/test modules, directly on See the sandbox collaboration guide for more information on that.

What if I pull/push from/to the wrong one?

It could happen before/during/after the migration that you pull or push from or to the wrong repository. git can rebase with the new repository, but unfortunately, the old commits will be mingled with the new ones.

To be able to push to the new repositories, you will need to follow instructions below to cleanup your repositories.

Rewriting history

I have taken the liberty of rewriting our collective history. There were a lot of commits with inconsistent authors: some had invalid emails, some had no real names, some had multiple different email addresses for the same person. I narrowed the list down, through extensive detective work, to a complete list of 14 distinct authors.

I have rewritten all commit authors that didn't seem right to follow a consistent pattern:

Firstname Lastname <email>

The firstname/lastname is the combination you have used in certain commits. It's not the most popular way (ie. most commits had the nickname instead of first/last), but it seems the most logical one.

The email was your most often used email or the one you have registered as such at, if you didn't have an email address specified (in case you committed your code back in the days of CVS). (There are 6 commits from someone with a box at that I could track, and that I can only guess are Miguel's, but I'm not sure so I have let those one fly. :)

You should make sure you have your identity set straight in your git configuration, especially your email address, if you want your contributions to be properly credited in the future on In short, this means:

git config --global "User Name"
git config --global

Again, see the manual for more information, especially how this maps to your account.

A fresh clone is necessary!!!!

Note that, because we rewrote history, you will need to clone your repository from new remote. So yes, contrarily to what was documented here before, you will need to clone from scratch.

We understand this may be annoying if you work on a bunch of branches or have local branches you want to keep. Unfortunately, our original tests didn't cover all use cases and were misleading in thinking we could just rebase existing repositories.

Other documentation

People unfamiliar with git or specifically git on should read the growing git handbook on

People that are familiar with git should contribute to that manual. :)

Under the hood

The history rewrite was performed through git-fast-export and git-fast-import magic, with a fairly simple perl script of my own. I have cloned the current repo, filtered it and published a new version for to pull. You can see the results here:

The exact calling sequence is this:

mkdir export orig
cd export
git init --bare hostmaster.git
cd ../orig
git clone --bare git://
cd hostmaster.git
# look at the authors in the original repo
git fast-export --all |  grep -a '^author [^<]* <[^>]*> [0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$' | sed 's/[0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$//' | sort | uniq -c
git fast-export --all | perl /home/anarcat/bin/ | ( cd ../../export/hostmaster.git ; git fast-import )
cd ../../export/hostmaster.git
# look at the authors in the new repo
git fast-export --all |  grep -a '^author [^<]* <[^>]*> [0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$' | sed 's/[0-9][0-9]* [+-][0-9][0-9][0-9][0-9]$//' | sort | uniq -c
# rinse, repeat until the mapping is right
# push repositories online
git remote add origin ssh://
git push --all
git push --tags
# repeat with provision

This is the script source, without all the mappings:

#! /usr/bin/perl -w

%authors = ( 
    'anarcat ' => 'Antoine Beaupré ',
   # ... 

sub remap {
    my $a = shift;
    if ($authors{$a}) {
        return $authors{$a};
    } else {
        return $a;

while (<>) {
    s#^author ([^<]+ <[^>]+>) ([0-9]+ [+-][0-9][0-9][0-9][0-9])$#'author ' . remap($1) . ' ' . $2#e;

Migration checklist

  1. serve exported repositories to migration team done!
  2. update the repositories with change authors done!
  3. the great git migration done! is now readonly, commits are pushed to and releases are performed on
  4. update documentation in handbook in progress!
  5. update the home page links done - I have changed the link in the body from to provision/hostmaster project pages, and pointed the "Get the source" link in the bottom to the install manual instead
  6. update makefiles and files in docs/* in provision to fetch from done!
  7. make at least one other release done - 0.4-rc2 release coordination
  8. turn off done

Architecture Guide


This section deals with providing coherence and understanding of various aspects of Aegir - what it is, and what it is not.

It tries to explain some of the concepts with which Aegir is developed for and how all the bits hook together.

Aegir design and terminology


@TODO - this document is out of date and needs updating.

This page documents all the different terms used when referring to components of the Hostmaster system, and how the different entities relate to each other.

Front End

The user interface used to administrate your sites. The front end is provided by the Hostmaster install profile, and the Hosting contributed module. It defines a complete Drupal based data model for managing the various aspects of your installation.


These entities are used primarily in the front end to manage the configuration and data. Upon calling the back end, the system breaks down the properties of these entities to be passed as options for the command line, so the back end only has a flat data structure to work with. During this process, all relationships are automatically retrieved, making it a one step process for the developer.

The person or group that runs the site. This information is usually required for billing and access purposes, to assure that only certain people are able to view the information for sites they run. If you do not intend on having more than one client access the system, you will not need to create any additional clients for your purposes.
An instance of a hosted site. It contains information relating to the site, most notably the domain name, database server and platform it is being published on. A site may also have several aliases for additional domains the site needs to be accessible on.
The mechanism whereby Hostmaster keeps track of all changes that occur to the system. Each task acts as a command for the back end, and contains a full log of all changes that have occurred. If a task should fail, the administrator will be notified with an explanation of exactly what went wrong, and how to fix it.
The file system location on a specific web server on which to publish sites. Multiple platforms can co-exist on the same web server, and need to do so for upgrades to be managed, as this is accomplished by moving the site to a platform hosting an updated release. Platforms are most commonly built for specific releases of Drupal.
The server that provides various services.
The service that runs on the server. This might be HTTP or a Database server. Each web server hosts one or more platforms, which act as publishing points for the hosted sites. If you are not intending to use Hostmaster in a distributed fashion, you will not need to create additional web servers for your purposes. Most web servers and database servers are on the same machine, but for performance reasons external database servers might be required. It is not uncommon for one database server to be shared amongst all site instances. If you are not intending to use an external database server, or multiple database servers, you will not need to create any additional database servers for your purposes.
Service type
The type of service that runs on the server. In the example of 'HTTP' as a service, a service *type* is Apache or Nginx, for example. MySQL is a *type* of service 'Database'.
An installable component for a site. Hostmaster keeps tracks of all the modules, themes and install profiles installed across your entire network. These packages most commonly link to their project on, but might not in the case of custom modules. Users are not able to create nodes of this type, as they are generated automatically during verify and sync tasks.
Package Release
The release of a specific package. Each package on your network has at least one installed release, and each platform (including Drupal itself) may have several releases installed across your network.
Package Instance
Where a package release has been installed. A package release may be installed in multiple places on a platform, and sites may have access to multiple versions of packages. This entity also tracks which modules are enabled, and what version of the schema (if any) they have installed.

Task types

These are "hosting commands" that the server can accomplish. These are mapped to back end commands by the task queue processor.

Verify platform
Verify that a platform is correctly installed. Automatically created when new platforms are added, to ensure that they are capable of having new sites installed on them. Operates on: platforms
Import sites
Import existing sites onto a platform. Operates on: platforms
Install site
Install a new site. This is automatically generated when a site node is created. Operates on: sites
Sync site
Synchronize changes to the site record with the back end. Automatically generated on editing of the site node. Operates on: sites
Backup site
Generate a backup of an existing site. This backup contains everything needed to run the site. Operates on: sites
Restore backup
Restore a site to a previous backed up version. Operates on: sites
Disable site
Disable a running site. This is commonly used in the case of non-payment from clients. Operates on: sites
Enable site
Enable a disabled site. The opposite of Disable. Operates on: sites

This is an incomplete list, and will be updated as more tasks are added


The front end uses and maintains several queues, which are then processed through the drush.php hosting dispatch command. More queues can be added by optional contributed modules, and are often used to provide regularly scheduled back end commands, such as backing up sites.

Task Queue
The primary and most important queue. Whenever a change is made to the data set, the front end creates a "Task" node. These task relate to such things as installing new sites, regenerating the configuration files of sites, verifying that a platform has been correctly configured and importing existing sites on a newly added platform.
Cron Queue
This queue is used to manage the regular execution of the cron process required by most sites. It is configurable with a frequency you want all the sites to be cronned within. This is commonly 1 hour, but might be higher or lower. Higher frequency will cause higher system load. This command will iterate through all enabled sites, and batch the cron calls, based on how many sites are available, and how frequently it is running.
Backup Queue
This queue operates in the same way as the cron queue, in that it iterates through all installed sites, based on a configurable frequency. This executes the same functionality as requesting a site backup through the user interface. You are able to revert back to a previous backup of your sites.
Statistics Queue
This queue operates on the same mechanism as the cron queue, and alows you to retrieve statistics of your installed sites. This will retrieve such metrics as number of registered and active users, and number of posts / comments, which will then be viewable on the front end.

Back End

The back end is a command line interface provided by the Provisioning Framework, of all the tasks it is capable of performing.

It is designed to operate separately from the front end, so that you can have multiple back ends on one or more servers.

The back end handles the server level configuration of the system, such as creating new databases and virtual host records and restarting the apache process for new installations to take effect.

When the queues are processed by the front end, these are turned into calls to the back end, for whichever server they happen to be on. The back end communicates to the front end through a very simple mechanism, whereby it returns a serialized string containing state information and log messages / error return codes.

Aegir Architecture


This wiki page documents the Aegir architecture in schematic form. It is intented to complement a related page on the typical file system structure on an Aegir installation.

Click on image for a larger version.

Aegir file system structure


This page documents the typical file system structure on an Aegir installation. It is intended to complement a related wiki page on Aegir Architecture. The following paths are based on an Aegir 0.4-x installation and assume that all directories are within the /var/aegir folder.

Scripts and Configuration Files

/backupssite-specific tar balls containing a database dump and folders under path/to/platform/sites/
/config/server_master/apache/conf.dnon-aegir or non-drupal virtual hosts files
/config/server_master/apache/platform.dcontains .htaccess information for each aegir platform
/config/server_master/apache/vhost.dapache virtual host files for aegir platforms
/config/server_master/apache/vhost.d/aegir.example.comvirtual host file for aegir web site - specifies path to platform directory and site database settings (so that database credentials are not exposed directly in site settings.php file)
/config/server_master/apache/vhost.d/site-1.comvirtual host file for deployed web site
/drushdrush folder
/drush/drush.php drush script
/.drushdrush extensions and server, platform and site aliases
/.drush/drush_makedrush_make project folder - used for building web sites from .make files
/.drush/provisionprovision folder
/.drush/server_master.alias.drushrc.phpsettings for the master server where the main aegir database, hosting platform and aegir site reside
/.drush/platform_hostmaster.alias.drushrc.phpsettings for the hostmaster platform on which the aegir site is based
/.drush/hostmaster.alias.drushrc.phpsettings for the aegir site
/.drush/platform_platform1.alias.drushrc.phpsettings for platform1 on which is based
/.drush/ for

Hostmaster Platform and Aegir (front-end) Site

/hostmaster-0.xhostmaster platform
/hostmaster-0.x/profiles/hostmasterhostmaster profile
/hostmaster-0.x/profiles/hostmaster/modules/hostinghosting module
/hostmaster-0.x/profiles/hostmaster/modules/install_profile_apiinstall_profile_api – facilitates provisioning of sites based on a non-default profile
/hostmaster-0.x/profiles/hostmaster/themes/eldireldir theme – provides Aegir front end look and feel
/hostmaster-0.x/profiles/hostmaster/hostmaster.profileprofile file – used in site provisioning to configure a drupal database
/hostmaster-0.x/profiles/hostmaster/hostmaster.makemake file – used to include modules, themes, libraries etc. from various sources
/hostmaster-0.x/sites/aegir.example.comaegir web site folders

Deployed Platforms

Note: the directory /platforms is optional but can be useful to separate deployed platforms from directories for scripts, config files and hostmaster platform.

/platforms/platform-1/profiles/custom-profile/custom.profileprofile file – used in site provisioning to configure a drupal database
/platforms/platform-1/profiles/custom-profile/custom.makemake file – used to include modules, themes, libraries etc. from various sources
/platforms/platform-1/sites/ drupal configuration file
/platforms/platform-1/sites/ site-specific aegir-specific configuration file

Platform permissions

Path Directory              File                   Notes
./* aegir:aegir
The webserver has no business writing or moving files in the Drupal codebase.
Inside ./sites/
drushrc.php aegir:aegir
Web server shouldn't be able to read drushrc.php, it's not a component of the Drupal site.
settings.php aegir:www-data
Web server can read this file, but otherwise tight control over this file which can contain sensitive information.
modules themes
Because of the sticky bit (s) on the parent directory, each child directory will inherit attributes of the parent. The attribute that is consistently inherited is the group. This means that a developer in the "aegir" group can add files which will retain the "aegir" group ownership of the parent directory.
Aegir sets these directories with a sticky bit (s) so that under certain conditions new folders and files will inherit parent permissions. There are only a few cases where this happens though.
The permissions shown here are how files and directories created by www-data will look. When verifying a platform, Aegir won't "correct" these files and directories to match the parents. If you have trouble with permissions/ownership on these directories, you can safely run the following commands (in this case on the files directory):
# chown -R aegir:www-data /path/to/site/files/*
# chmod -R 775 /path/to/site/files/*

Provision - the Aegir backend.


Each of the major entities managed by Aegir (namely servers, platforms and sites) are represented by nodes in the front end, which in turn generate Drush 'named contexts' in the backend. Named contexts are similar to what Drush 3.0 refers to as 'site aliases', but are more structured and cover more than just sites.

Whenever you create or edit a node on the front end, Aegir will create or update the named context on the backend, and any tasks run on the entity will be run on that named context, ie:

drush @server_master provision-verify

All of the actual work related to managing your system is done by the backend, and the front end primarily serves as a way to manage these aliases. You can very easily use the backend on it's own if you so choose.

Creating or updating named contexts.

Contexts are created and updated through the provision-save command. The format of the command is :

drush provision-save @contextname --context_type=type --property=value --other_property=value

Every time this command is called, it will update the named context and only modify the properties that have been specified. It is important to note that the provision-save command never has a site alias or context name before the command, but requires you to specify which context you are modifying as the first argument.

Deleting named contexts.

To delete an existing named context, you simply need to pass the 'delete' option to the provision-save command, ie:

drush provision-save @contextname --delete

Using named contexts

Named contexts are used in 3 ways in provision: as aliases, arguments or options.

Aliases : Aliases appear to the left of the drush command, and are making use of the Drush 'site alias' functionality.

# Verify the site represented by
drush provision-verify 

Options : Certain options to provision-save require you to pass the alias of another named context of a specific type, and uses this information to build relationships between objects.

 # generate a new site context in the directory represented by @platform_test
 drush provision-save --context_type=site --platform=@platform_example

Arguments : A select few provision commands require you to pass the alias of another specific type of named context as an argument.

# Migrate the site to the platform represented by @platform_newversion
drush provision-migrate @platform_newversion

Context types

There are (currently) 3 different entities represented as named contexts in the aegir backend. You determine which type of entity you are working with by specifying the context_type option to the provision-save command. You dont need to specify it every time, but it is best to always explicitly specify it, just in case.

Because the namespace of all contexts is shared, they each have different naming conventions to avoid collisions. There are certain named contexts of each type which are always present in the system.

Each of the context types have different semantics and properties, and certain provision commands will only operate on certain types of entities.

Server contexts

Naming guideline : Server with a hostname of becomes @server_serverexamplecom.

Special contexts : @server_master always refers to the local server where aegir is installed.

Required properties :

  • remote_host : This should always be set to the FQDN of the server being managed.

Services : Servers are the only entities which have services associated to them. Services are things such as http, db or dns.

Each service may select one service_type, which chooses which implementation to load for this service. Examples of these are the 'mysql' implementation of the 'db' service, or the 'apache' and 'apache_ssl' implementations of the http service.

All services that are found by Aegir are enabled, but unless you specifically select one of the implementations, the 'null' service type is loaded. Each service type has additional required properties and values that can be passed to it, but describing them all is outside of the scope of this document.

Important commands :

  • provision-verify : Verify that the configuration is correct and fix them if they aren't.

Example :

# Enable a new remote server hosting sites with apache on port 8080
drush provision-save @server_webexamplecom --context_type=server \ \
  --http_service_type='apache' \

# Verify that the settings are correct
drush @server_webexamplecom provision-verify --debug

Platform contexts

Platforms are how aegir refers to a specific copy of Drupal checked out on a file system. Platforms are hosted on web servers and sites are hosted on platforms.

Naming guideline : A platform in /var/aegir/codebase-6.1 becomes @platform_codebase61 , but this is not strictly enforced. We only use that naming convention when we do not have a node representing the platform available. If a node is available, we use the title of the node to generate the context name.

Basically, you can name your platform anything but it should still be prefixed with 'platform_'.

Special contexts : There are no special contexts per se, but the hostmaster-install and hostmaster-migrate commands generate aliases for the hostmaster platform in the form of '@platform_hostmaster$version'. If the front end is present, one or more of these contexts are usually available too.

Required properties :

  • root : The absolute path to a Drupal directory, generally in the form /var/aegir/platformdir.

Optional properties :

  • web_server : The name of a server context that provides the http service. The default is to @server_master
  • makefile : The absolute path or url to a drush make makefile. If the root directory does not exist and the makefile property does, provision will call drush_make to attempt to create the directory.

Important commands :

  • provision-verify : This will ensure that all the files are the correct permissions, and are all present on the correct remote servers. Also rebuild package database. Run this command whenever the contents of the platform changes.

Example :

# New open atrium platform in /var/aegir/atrium-head that will be built from a remotely hosted
# makefile and published on an external web server.
drush provision-save @platform_atriumhead --context_type=platform \
        --root=/var/aegir/atrium-head \
        --web_server=@server_webexamplecom \

# Verify settings and generate the directory with drush make, pushing to the remote server after.
drush provision-verify @platform_atriumhead

Site contexts

Naming guidelines : A site with the url '' becomes The context name is always the url.

Special contexts : @hostmaster always refers to the site running the hostmaster front end. This helps with debugging and makes it more easily scriptable.

Required properties :

  • uri : The FQDN of the site. should match the context name.
  • platform : The name of a platform context that the site will be hosted on.

Optional properties :

  • db_server : The name of a server context which provides the db server. defaults to @server_master.
  • client_email : The email address to send the login information for the admin user.
  • profile : The installation profile the site is running. dependent on the platform supporting it.
  • language : The ISO language code of the site. dependent on the platform and profile supporting it.

Important commands :

Almost all of the provision commands are related to sites, so this is just the very core of them. Check drush help for more.

  • provision-install : Install the site represented by the context. Just creating the context doesn't install the site yet.
  • provision-verify : Verify the site is running correctly, and ensure the files are all published to the correct servers.
  • provision-delete : Delete the site and all it's information. The context will still be there.

Example :

# Set up a new site to be installed on the open atrium platform (hosted on a remote server),
# using an external database server , with the openatrium profile and localize it into spanish.
drush provision-save --context_type=site \ \
        --platform=@platform_atriumhead \
        --db_server=@server_dbexamplecom \ \
        --profile=openatrium \

# Install the new site to the specifications requested above.
drush provision-install --debug

# Verify the site to make sure everything is ok.
drush provision-verify --debug

Importing contexts into the front end

Hostmaster provides a drush command which will attempt to generate or updates front end nodes that match your defined contexts. To import a context into the front end, just run :

drush @hostmaster hosting-import @contextname

This will import that context, and every context it depends on into the front end. This is useful for integration with other systems such as the puppet-aegir module for Puppet, which can add newly configured servers directly to the front end.

Messaging systems evaluation

So we've been talking seemingly forever about possible queuing system instead of our crappy "PHP/Vixie Cron based" one, but we have yet to document pro/cons of possible alternatives. Here are the goods (or the beginning of em).

Note that this document was written before (in november 2009) the current multiserver implementation was designed and completed.

Current design

Tasks are basically serialized in a MySQL database and polled at regular interval by a cron job, running drush hosting dispatch, which bootstraps Drupal, fetches the object, and fires up provision commands.

The possibility here is to have another dispatcher that would always run, and maybe not even talk to the mysql database, to fire up the provision commands.

Another requirement is that the queuing system may be network-aware. We want multi-server support and that is a critical issue for the next release (0.4 at the time of writing). One thing to consider is how we pass files around...

Roll our own


  • full control over the API, functionality and whatnot (ie. function of manpower invested)


  • rolling our own is hard! let's go shopping!

Meta: AMQP vs JMS

So yeah, there's a standard in development for all this stuff: . There is also a Drupal module being developed to work with it:

JMS on the other hand is the "Java Messenging Service", that creates a language- (as opposed to network-) based API.

Maybe we should just not care about which messaging system you use in the backend and allow our frontend to talk to any backend, with the stupid cronjob mysql poller as a default...


Pros (mostly out of the home page):

  • supports PHP in frontend and backend
  • Multi-language - There are interfaces for a number of languages, and this list is growing. You also have the option to write heterogeneous applications with clients submitting work in one language and workers performing that work in another.
  • Flexible - You are not tied to any specific design pattern. You can quickly put together distributed applications using any model you choose, one of those options being Map/Reduce.
  • Fast - Gearman has a simple protocol and interface with a new optimized server in C to minimize your application overhead.
  • Embeddable - Since Gearman is fast and lightweight, it is great for applications of all sizes. It is also easy to introduce into existing applications with minimal overhead.
  • No single point of failure - Gearman can not only help scale systems, but can do it in a fault tolerant way.
  • Production ready - ran on digg, yahoo, livejournal, ...
  • Gearman uses an efficient binary protocol and no XML. There's an a line-based text protocol for admin so you can use telnet and hook into Nagios plugins.
  • Persistent queues are available using mysql, memcached or sqlite
  • There's pretty good PHP support and it's easy to get up and running quickly.


  • "somewhat higher latency, signal-and-pull architecture"
  • The system makes no guarantees. If there's a failure the client is told about the failure and the client is responsible for retries.
  • gearman-specific binary/network protocol
  • Notice how maintains both gearman and the shwartz, below


  • Mostly written in Perl
  • Increasing year-over-year development activity
  • Established codebase
  • Few source code comments



  • redis-based (therefore somehow language agnostic deep in there, but certainly network-centric)
  • github's (aka: it's got to work at least somewhere in production)


  • ruby-centric
  • I haven't actually looked at the thing or tested it

Active MQ

Apache ActiveMQ:


  • Supports a variety of Cross Language Clients and Protocols from Java, C, C++, C#, Ruby, Perl, Python, PHP
  • Supports many advanced features such as Message Groups, Virtual Destinations, Wildcards and Composite Destinations
  • Supports pluggable transport protocols such as in-VM, TCP, SSL, NIO, UDP, multicast, JGroups and JXTA transports
  • REST API to provide technology agnostic and language neutral web based API to messaging
  • Persistent


  • uses too many acronyms on the frontpage (really.)
  • uses a lot of Java jargon
  • production-ready?


  • Mostly written in Java
  • Large, active development team




  • "high reliability, availability and scalability along with good throughput and latency performance that is predictable and consistent"
  • "based on the emerging AMQP standard"
  • in debian
  • Drupal module - AMQP
  • apparently very fast, according to dellintosh


  • erlang? do you know erlang? (Arguably not really necessary, since all publishers and consumers can be written in other languages that have AMQP support)


  • Mostly written in Erlang
  • Large, active development team
  • Well-commented source code

Open Messenging Queue


  • SOAP/HTTP interface
  • Scalable


  • Java/C specific

The Schwartz

The Schwartz


  • Perl (come on, perl is everywhere... isn't it?)
  • "reliable job queue"
  • has retry
  • "lightweight"


  • Perl (aka write only)
  • "library, so some assembly required"
  • "light on documentation"
  • Notice how maintains both the shwartz and gearman, above

Open AMQ

Open AMQ


  • Multi-language (but not PHP)
  • "learn in a day or so" then "another day or two for wireAPI", development will take "some weeks"...
  • "remote admin tools, one-line failover, instant federation, protection against slow clients, detailed logging"
  • AMQP standard
  • "Linux, AIX, Solaris, Mac OS/X, other UNIX"


  • "for C/C++ and JMS"


  • Mostly written in C
  • Well-commented source code
  • Short source control history
  • Only a single active developer
  • Apache Software License may conflict with GPL
  • Apache License 2.0 may conflict with GPL


Drupal Module for Drupal 6 and 7 is at

Pros: * used in production ("causes" application in facebook)


  • C

  • Mostly written in C
  • Large, active development team
  • Few source code comments



  • persistent or in-memory


  • java-only.

Zero MQ


  • fast
  • lightweight messaging implementation.
  • supports different messaging models.
  • already very fast. We're getting 13.4 microseconds end-to-end latencies and up to 4,100,000 messages a second today.
  • very thin. Requires just a couple of pages in resident memory.
  • provides C, C++, Java, Python, .NET/Mono, Ruby, Fortran, COBOL, Tcl, Lua and Delphi language APIs.
  • supports different wire-level protocols: TCP, PGM, AMQP, SCTP.
  • runs on AIX, FreeBSD, HP-UX, Linux, Mac OS X, OpenBSD, OpenVMS, QNX Neutrino, Solaris and Windows.
  • supports i386, x86-64, Sparc, Itanium, Alpha and ARM microarchitectures.
  • fully distributed: no central servers to crash, millions of WAN and LAN nodes.


  • no PHP whatsoever
  • production?


  • Mostly written in C++


There's a good vibe around using Jenins (formerly known as Hudson) in place of cron for those kind of things. In our scenario, Aegir would queue up jobs to Jenkins, which would take care of dispatching them to remote servers.


  • supports tons of different notification modes
  • scales well
  • supports remote servers


  • application-specific (unclear?) protocol to queue up tasks
  • Java (so a bit heavy)

OHLOH: * has had 63,860 commits made by 708 contributors * representing 893,049 lines of code * is mostly written in Java * with an average number of source code comments * has a well established, mature codebase * maintained by a very large development team * with decreasing year-over-year commits

Others to evaluate?

Others mentionned here: ActiveMessaging, BackgroundJob, DelayedJob, and Kestrel.


All the above must have:

  • Open Source - It's free as in beer and speech

Provision contexts

This section is currently being written, so its probably going to be full of errors, and possibly not that useful. Feel free to edit.

I'm probably going to pop a nice table of contents here, for now look to the right...


Contexts are simply a way for Aegir to store certain pieces of information that it thinks may be useful later. They are named so that they can be referred to easily.

Every server, platform and site in Aegir has an associated context. Each one of these contexts stores information about the associated entity. So, for example, a site context stores the base URI of the site. Later, the code responsible for informing the web server of the site's existence can look at this context and determine the requested URI.

In theory contexts allow most of the back-end commands in Aegir to operate just on the name of the context. This is an important concept: the alternative would be to try to guess everything that the command being invoked might want to know and pass it in as a series of command line arguments.

For example, the command that Aegir uses to clone a site just needs to be given the context of the site to clone and the platform to clone it to. From there the code can work out where the destination platform actually is, what servers it needs to talk to, and with what credentials.

Drush contexts

Note that Drush also has the concept of a context, but this is not to be confused with Aegir contexts which are different. Aegir contexts are closer to, and stored as, Drush 'aliases'.


Contexts are always named to start with a '@' symbol, except for in the filename of the file in which they are stored.

Contexts in Aegir are stored as a simple array of data in a file within the backend of Aegir. However this array of data is accessed and modified with a set of objects and accessors. Here is an example of what a context file looks like:

$aliases['hostmaster'] = array (
  'context_type' => 'site',
  'platform' => '@platform_hostmaster',
  'server' => '@server_master',
  'db_server' => '@server_localhost',
  'uri' => '',
  'root' => '/var/aegir/hostmaster-0.4-beta2',
  'site_path' => '/var/aegir/hostmaster-0.4-beta2/sites/',
  'site_enabled' => true,
  'language' => 'en',
  'client_email' => '',
  'aliases' =>
  array (
  'redirection' => false,
  'profile' => 'hostmaster',

These files are stored in the /var/aegir/.drush directory on the master Aegir server.

In the above example we can see the properties of the context, like the 'uri' of the site, and the 'profile' that the site is running.

There are also a number of more interesting properties in the example, note the 'db_server' property, which has a value that is another context. We shall see later that these properties are special, and allow developers to easily access functions of the 'db_server' without needing to care which db server they are talking to.

Contexts are used within drush commands as objects, which subclass provision_context. This allows much more flexibility and cleverness, though can make them very confusing to use sometimes! As a developer you will only need to worry about the context objects, as Aegir handles storing them in the files for you. But it is important to note that each context must be representable in a flat text file (or be prepared to do some serious leg-work) by that I mean you probably don't want to be storing massive amounts of relational data in them, use a database for that!

Using contexts in your code

Getting the current context is really easy, just call the 'd' accessor:

= d();
What the current context is will depend on the drush command that is running and the context that was passed into that drush command. For example if you ran a site verify task, then the caller would name a site context, and thus initially calls to d() would return the site context. The values of the context are accessible as simple properties of this object.

So, suppose a verify task is started for the main Aegir frontend, which has a context that is always called '@hostmaster', the drush command invocation would look like this:

drush @hostmaster provision-verify

Then the drush command for provision-verify could do this:

function drush_provision_verify() {
$context = d();
drush_print('Passed context of type: ' . d()->type);
Which would print the type of the context passed to the verify command, in this case 'site'.

The d accessor is not to be feared! It is used all over the place in Aegir and if you're not sure what context you're going to get then you can always print d()->name and you'll get the name of the context that you're dealing with.

Loading a named context

You can pass an optional argument to the d accessor of the name of the context that you want. So, if you feel compelled to access a property about the main Aegir frontend site anywhere you can do this:

= d('@hostmaster');

though most of the time you'll be dealing with the current context, and contexts that it references.


Here's an example context:

$aliases['hostmaster'] = array (
  'context_type' => 'site',
  'platform' => '@platform_hostmaster',
  'server' => '@server_master',
  'db_server' => '@server_localhost',
  'uri' => '',
  'root' => '/var/aegir/hostmaster-0.4-beta2',
  'site_path' => '/var/aegir/hostmaster-0.4-beta2/sites/',
  'site_enabled' => true,
  'language' => 'en',
  'client_email' => '',
  'aliases' =>
  array (
  'redirection' => false,
  'profile' => 'hostmaster',

Notice that some of the properties are the name of contexts, such as 'db_server' which has a value of '@server_localhost'.

Suppose now that I have some code that wants to get a property of the db_server associated with the @hostmaster context, I can simply do this:

= d('@hostmaster')->db_server;
// $db_server_context is now a fully populated context object, not the string '@server_localhost'
drush_print('DB server on port: . $db_server_context->db_port);

The provisionContext object will return full context objects for properties that store the names of other contexts. The how and why of determining which to return as strings and which to return as a context object will be covered later. It is entirely possible to have the name of a context stored as a property in a context, that when accessed returns the string, rather than the context named in the string.

Properties and services

Services are the way in which new properties are added to contexts, in fact if you wish to indicate you want to store additional properties in a context, this must be done by a service. There is no other way to store data in a context. One might assume that you can just ask provision to save an additional value on a context, but this will not work, and you'll probably get very frustrated indeed!

For example, the provisionService_pdo service adds a 'master_db' property to the server context, it does this in a method thus:

function init_server() {

This means that the 'master_db' property can now be set in a provison-save drush command, and will be persisted when the context is saved.

A service may also define properties on the context as actually representing another context, so that you may access that subcontext directly. You can see an example of using this subcontext in the 'Implementation' section of this Provision Contexts guide, but the an example of how you let provision know that a property name is not just a string, but a named context is with a method on the service:

   * Register the http handler for platforms, based on the web_server option.
static function subscribe_platform($context) {
$context->setProperty('web_server', '@server_master');
$context->service_subscribe('http', $context->web_server->name);
Here the provisionService_http is being asked to 'subscribe' to a platform, it sets a property on the context, as above, but additionally uses the is_oid() method to indicate that the property is a named context. It then uses that immediately when it gets the name of the web_server context: $context->web_server->name. We'll worry about what 'subscribing' to a platform means later,.

Using services from contexts

This is stub page that will describe the way that you can use the service() method on a context.

Provision services

We should add some documentation about Provision services, what they are and how they work and interact with contexts.

UNIX group limitations

While working on the security enhancements, I stumbled upon a few fundamental UNIX limitations with groups. These limitations similarly apply to GNU/Linux.

1. Group name length limits

A group name (e.g. dialout) has a length limit, usually hardcoded in the operating system. This limit varies across different UNIX systems and implementations.

In general, we can probably assume that 16 characters is a safe limit, and that's what's being used in hosting_client_sanitize.

2. Group membership limits

A UNIX user has a primary group, defined in the passwd database, but can also be a member of many other groups, defined in the groups database. UNIX has a hardcoded limit of 16 groups a user can be a member of (source). This means that if we translate Aegir clients into UNIX groups and you have access to multiple Aegir clients, you'll be able to access only 16 of those clients at a time. Rather annoying.

The problem is related with NFS, or rather AUTH_SYS (source), which has an in-kernel data structure where the groups a user has access to is hardcoded as an array of 16 identifiers (gids to be more precise, see this post for details). Even though Linux now supports 65536 groups, it is still not possible to operate on more than 16 from userland, through NFS - on a regular ext3 filesystem, it works without problems (tested in Debian Squeeze).

Solutions for this are not obvious. The blog post on proposes a technical enhancement to OpenSolaris. It's unclear whether that fix was implemented. This SAGE post explains that using GSS_API (used in NFSv4) should (should!) resolve the issue.

This more thorough analysis of the problem is more direct, to shamelessly quote it:

  • Use NFSv4
  • Use RPCSEC_GSS authentication
  • Use ACLs

As mentionned above, this only applies to NFS mounts. NFSv4 should (should!) be simple enough and available on Linux. It is also unclear exactly how to use RPCSEC_GSS (is that Kerberos?). As for ACLs, we are fully supporting this now with the provision ACL extension, so that shouldn't be an issue.

Running drush on Aegir sites as a regular user

N.B. The ACL-based approach, described below, has been implemented as an extension to Provision in

What's the problem here?

As things stand, you cannot run tasks as a regular UNIX user on your Aegir Drupal sites:

anarcat@marcos:~$ drush @hostmaster cc all
A Drupal installation directory could not be found                                            [error]
The drush command '@hostmaster cc all' could not be found.                                    [error]

The reason the site is not found is because the alias cannot be loaded. If we change our home directory to find the alaises, we then get permission problems:

anarcat@marcos:~$ env HOME=/var/aegir drush @hostmaster cc all
include(/var/aegir/.drush/platform_hostmaster.alias.drushrc.php): failed to open stream:      [warning]
Permission denied
include(): Failed opening '/var/aegir/.drush/platform_hostmaster.alias.drushrc.php' for       [warning]
inclusion (include_path='.:/usr/share/php:/usr/share/pear')
include(/var/aegir/.drush/hostmaster.alias.drushrc.php): failed to open stream: Permission    [warning]
include(): Failed opening '/var/aegir/.drush/hostmaster.alias.drushrc.php' for inclusion      [warning]
A Drupal installation directory could not be found                                            [error]
The drush command '@hostmaster cc all' could not be found.                                    [error]

... so that doesn't seem to be a good approach, although one could just fix the permissions on the alias files for the site (which don't contain the DB password, oddly enough) and platforms (which have been safe since the "server" seperation):

cd /var/aegir/.drush
chmod a+r ls *alias* | grep -v &#039;server&#039;

Then we end up with another problem:

Cannot open drushrc "/var/aegir/hostmaster-HEAD/sites/",      [warning]
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring.                       [warning]
Cannot open drushrc "/var/aegir/hostmaster-HEAD/sites/",      [warning]
include_once(/var/aegir/hostmaster-HEAD/sites/ failed to   [warning]
open stream: Permission denied
include_once(): Failed opening './sites/' for inclusion      [warning]
Cannot modify header information - headers already sent by (output started at                 [warning]
Cannot modify header information - headers already sent by (output started at                 [warning]
Drush command could not be completed.                                                         [error]

... and this is really the core of the issue here: we need to allow more than one user to access those files, and aegir must be able to fix those permissions when the site is created/verified.

So to sum up:

  • a regular user doesn't have permissions to access the drushrc.php and settings.php files
  • the aliases are not accessible unless you change your home directory (note that -i doesn't work either, for some reason)

Hackish solution: add user to more groups

Miguel indicates here that files should be writable by the Aegir group, and, if we start with this idea, we can actually get pretty far. The drushrc.php file is owned by the Aegir group, so adding yourself to that group and fixing up the file to be group-readable actually goes a long way. You also need to add the user to the www-data group since drush expects to be able to open the settings.php file also:

adduser anarcat aegir
adduser anarcat www-data
chmod g+r drushrc.php

With the above changes, we can actually run drush commands!

anarcat@marcos:~$ env HOME=/var/aegir drush @hostmaster cc all
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring.                       [warning]
WD php: include(/var/aegir/.drush/server_localhost.alias.drushrc.php): failed to open stream: [error]
Permission denied in /usr/share/drush/includes/ on line 433.
WD php: include(): Failed opening '/var/aegir/.drush/server_localhost.alias.drushrc.php' for  [error]
inclusion (include_path='.:/usr/share/php:/usr/share/pear') in
/usr/share/drush/includes/ on line 433.
WD php: include(/var/aegir/.drush/server_master.alias.drushrc.php): failed to open stream:    [error]
Permission denied in /usr/share/drush/includes/ on line 433.
WD php: include(): Failed opening '/var/aegir/.drush/server_master.alias.drushrc.php' for     [error]
inclusion (include_path='.:/usr/share/php:/usr/share/pear') in
/usr/share/drush/includes/ on line 433.
include(/var/aegir/.drush/server_localhost.alias.drushrc.php): failed to open stream:         [warning]
Permission denied in /usr/share/drush/includes/ on line 433.
include(): Failed opening '/var/aegir/.drush/server_localhost.alias.drushrc.php' for inclusion[warning]
(include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drush/includes/
on line 433.
include(/var/aegir/.drush/server_master.alias.drushrc.php): failed to open stream: Permission [warning]
denied in /usr/share/drush/includes/ on line 433.
include(): Failed opening '/var/aegir/.drush/server_master.alias.drushrc.php' for inclusion   [warning]
(include_path='.:/usr/share/php:/usr/share/pear') in /usr/share/drush/includes/
on line 433.
'all' cache was cleared                                                                       [success]

We get a bunch of warnings, because provision can't load the server aliases (for a good reason - they contain DB passwords), but the command runs fine! In fact, if we want, we can avoid the luxury of using the aliases and just run the drush command in the site directory:$ drush cc all
Cannot open drushrc "/var/aegir/hostmaster-HEAD/drushrc.php", ignoring.                       [warning]
'all' cache was cleared                                                                       [success]

That warning can be removed by simply fixing the permissions on the drushrc.php file to be group-readable.

However, the problem with that approach is that the user has access to all sites, from there on. No regard to what client the user has access to. Obviously, this can work for a dedicated server (and it would need patches to provision so that drushrc.php files and optionnally aliases are group-readable) but it will not work for shared hosting, which is my prime objective here.

Patches necessary:

  • group-readable drushrc.php (changes in provision)
  • add users to the aegir and www-data groups manually
  • optional: group-readable site aliases (so that users can access the aliases)


  • all or nothing: the user has access to all sites or none as we have a single group

Traditionnal UNIX permission-based approach

In order for this to work really correctly, we would need the following permissions setup. Instead of having the typical following:

-r--r-----  1 aegir aegir    55456 16 mar 13:35 drushrc.php
drwxrws--- 10 aegir www-data  4096 16 mar 13:34 files
drwxrwsr-x  2 aegir aegir     4096 16 mar 13:34 modules
-r--r-----  1 aegir www-data  3276 16 mar 13:35 settings.php

(I am skipping libraries, themes and private directories for simplicity - libraries and themes are the same as modules and private is the same as files.)

... We would need to have the following:

-r--r-----  1 aegir    client   55456 16 mar 13:35 drushrc.php
drwxrws--- 10 www-data client    4096 16 mar 13:34 files
drwxrwsr-x  2 aegir    client    4096 16 mar 13:34 modules
-r--r-----  1 www-data client    3276 16 mar 13:35 settings.php

That is, all files and directories are readable by the "client" group (which allow inter-client isolation) and some would be readable or writable by the webserver (which allows for uploads and the webserver to read the settings.php and so on).

In this scenario, the webserver has access to all sites all the time: it can read/write uploaded files of other modules, but, because of database cloaking, it can't access other sites' databases. If we want to fix this we would have to go with ACLs.

Note that this setup doesn't require the webserver running any special patch or daemon to run the PHP process as a normal user: it still runs as the www-data user.

Patches necessary:

  • patches mentionned above (group-readable drushrc and aliases)
  • creating the client group and adding users to it
  • adding the aegir user to the client group
  • patching provision so it changes the group of some files to the client group
  • patching provision so it changes the owner of some files to the www-data user (this actually requires root: ouch!)


  • requires aegir to run as root
  • aegir need to be a member of every group

The ACL approach

This is based on this excellent comment from malclocke.

This is one of the simplest and elegant approach, as it:

  • doesn't require root access
  • fits well with the n to n client access list model of aegir
  • works without having to change the existing permissions (so would fit well in a contrib module)

To configure ACLs, you need to add the option to your mountpoint in fstab and remount the mountpoint (or reboot):

apt-get install acl
mount -o remount,acl /var # and edit /etc/fstab to add the acl option to the mountpoint

The rest is fairly simple: we just need to add access to the required groups to the drushrc and settings.php files:

setfacl -m g:client:r-- settings.php drushrc.php

... where "client" is the client group.

Optionnally, we can also give access to the alias files the same way, but we end up with the same warning on the server level files.

Note: this is a good introduction to ACLs, in Redhat

Patches necessary:

  • make platform-level drushrc readable by all (or add an ACL for each client with access to the platform..)
  • patch provision to add relevant ACLs on verify and install, to drushrc.php and settings.php:
  • system-level configuration of mountpoints and extra package
  • creating the client group and adding users to it


Other similar research

This is actually a well-discussed topic in the community, although nobody came up with a clean and complete solution for this.

Extending Aegir


Aegir is designed to be easily extendable by developers. As it is made with Drupal and Drush, it is made of the hooks and command you know and love. If you are a user or admin looking to deploy contrib modules, you should look into the contrib modules list and user documentation instead.

What can I extend exactly?

These extensions may come in the form of

This area is be devoted to teaching you how to extend and develop for Aegir to encourage contributions to the Aegir project or to help you modify Aegir to suit your unique use case.

Aegir API documentation

The inline documentation is a good start to understand the various hooks and internals that allow you to extend and customize Aegir to your liking. The documentation is rendered on daily.

See also the developer cheat sheet.

Please submit any suggestions or bug reports to the Aegir Project issue queue of your choice, under the "documentation" component.

Example modules

The contrib modules page documents all known Aegir extensions in existence. There are also specific developer documentation pages below.

Contrib developer documentation and roadmaps

Contrib developers are free to use this space to document the internals of their modules or their roadmaps. There is already this documentation:

Aegir developer cheat sheet

1. Locations and paths

  • Site URI: d()->uri; // ex: $base_url = 'http://' . d()->uri;
  • Drupal root: d()->site // returns: /var/aegir/platform/drupal-6.20/
  • Site path: d()->site_path // returns: /var/aegir/platform/drupal-6.20/sites/

2. File permissions and ownership

 $ctools_path = d()->site_path . '/files/ctools';

 provision_file()->chmod($ctools_path, 0770, TRUE)
   ->succeed('Changed permissions of @path to @perm')
   ->fail('Could not change permissions @path to @perm')

 provision_file()->chgrp($ctools_path, d('@server_master')->web_group, TRUE)
   ->succeed('Change group ownership of settings.php to @path')
   ->fail('Could not change group ownership of settings.php to @path')

3. Enabling/disabling drush/provision extensions from hostmaster

Every drush extension can implement a hook_drush_load(), and check if a module is enabled in the hostmaster site. See provision.api.php for an example.

4. References

Extending Aegir and communicating with install profiles


This article has been republished from mig5's website. Original URL

Aegir is a pretty powerful tool that allows you to very quickly provision new Drupal sites out of the box and manage them throughout their lifespan through a variety of tasks.

Its ability to understand different install profiles and thus 'distributions' shoots the awesome factor through the roof once you can deploy instances of OpenAtrium, Pressflow, ManagingNews or your own custom distro in just a couple of clicks.

Inevitably, though, your requirements will grow more complex. Aegir is about automation, and install profiles that prompt users for information through a series of tasks, and fail without that information, aren't much help. You might resort to a bunch of hacks to get that data into your client's site. You might try and insert some custom stuff into the site's vhost only to whimper in dismay when Aegir does a routine Verify sanity check and obliterates your changes.

If you get to this point, the simplicity of Aegir will be a distant memory and you'll be shaking your fist at us developers for forgetting about your corner case. Curses!

Rest assured, despite the constantly shifting development in Aegir and its early alpha stages, we try to maintain an ability to hook into Aegir and extend it as easily as possible, and it will only continue to get easier, especially once our API stabilises and is published.

Today I'm going to show you:

  • how to add a custom Hosting submodule to Aegir
  • how to alter the site form to add custom fields of data
  • how to send that data to the backend
  • how to apply that data to site vhosts and inside your custom install profile and site database

There are a few files here, because we're creating a custom module and install profile, and a few other things. For the impatient, I've attached a tarball to this article called 'cheesy.tar.gz' that contains all the relevant components and a README.txt instructing where to put it all.

Those of you who follow me on Twitter know I like to perpetuate the semi-French stereotype and rant and rave about cheese. I spent some time trying to think of a non-cheesy example to use in this tutorial, before realising what a silly idea that was. You can never have enough cheese :)

Part one - Adding a cheesy module

We'll add a custom module to the Hosting frontend called 'Cheese'.

Hosting is the 'frontend' bit of the Aegir system that allows you to plug data in via a web interface and have that information, if necessary, communicated to the backend Drush/Provision system by way of 'tasks'.

Our new module is simple: we're going to invoke a hook_form_alter() to add a textfield to the site form. The module will add a table to the Aegir database that stores this value, linked to the site nid. Keeping it out of the hosting_site table means we can disable and uninstall this module later and clean up properly.

There's nothing new or unusual about a Hosting submodule than any other Drupal module. Make the directory /var/aegir/hostmaster-0.x/profiles/hostmaster/modules/hosting/cheese

Info file

Here's our

name = Cheese
description = Cheesemongers need to define cheese in their Drupal sites.
package = Hosting
dependencies[] = hosting_site

core = 6.x

You can see we depend on hosting_site, because hosting_site is the module that provides the site form for installing sites!

Install file

Now let's add an .install file that defines our database table schema and how to install/uninstall it. Standard Drupal stuff.

* Implementation of hook_schema().
function hosting_cheese_schema() {
$schema['hosting_cheese'] = array(
'fields' => array(
'vid' => array(
'type' => 'int',
'not null' => TRUE,
'default' => 0,
'nid' => array(
'type' => 'int',
'unsigned' => TRUE,
'not null' => TRUE,
'default' => 0,
'cheese' => array(
'type' => 'varchar',
'length' => 128,
'not null' => TRUE,
'indexes' => array(
'vid' => array('vid'),
'cheese' => array('cheese'),




hosting_cheese_install() {
// Create tables.


hosting_cheese_uninstall() {
// Remove tables.

The module

Here's our simple hosting_cheese.module file.

It's pretty straightforward: our first function is a hook_form_alter() of hosting_site_form(). We are adding a single textfield to the form called 'Cheese'. Because it's my favourite, the default value is going to be 'camembert' if it hasn't already been set on the node (if we're updating a site rather than creating a new one).

The next few functions are invocations of common database hooks such as insert, update and delete. They will insert the 'cheese' value from our form into the hosting_cheese table.

I call hook_nodeapi() to do $stuff on those actions. There's validation to check that a cheese was entered. We invoke hook_load(), which allows various modules and features to return their bits and pieces from various parts of the database as 'additions' to the core node attributes. With this we can call $node->cheese when we might need to.

* Implementation of hook_form_alter()
function hosting_cheese_form_alter(&$form, $form_state, $form_id) {
  if (
$form_id == 'site_node_form') {
$form['cheese'] = array(
'#type' => 'textfield',
'#title' => t('Cheese'),
'#description' => t('What sort of cheese?'),
'#default_value' => $form['#node']->cheese ? $form['#node']->cheese : 'camembert',
'#required' => TRUE,
'#weight' => 0,

* Implementation of hook_insert()
function hosting_cheese_insert($node) {
  if (
$node->cheese) {
db_query("INSERT INTO {hosting_cheese} (vid, nid, cheese) VALUES (%d, %d, '%s')", $node->vid, $node->nid, $node->cheese);

* Implementation of hook_update()

function hosting_cheese_update($node) {
db_query("UPDATE {hosting_cheese} SET cheese = '%s' WHERE nid = %d", $node->cheese, $node->nid);

* Implementation of hook_delete()
function hosting_cheese_delete($node) {
db_query("DELETE FROM {hosting_cheese} WHERE nid=%d", $node->nid);

* Implementation of hook_delete_revision()
function hosting_cheese_delete_revision($node) {
db_query("DELETE FROM {hosting_cheese} WHERE vid=%d", $node->vid);

* Implementation of hook_nodeapi()
function hosting_cheese_nodeapi(&$node, $op, $a3 = NULL, $a4 = NULL) {
  if (
$node->type == 'site') {
    switch (
$op) {
'delete' :
'delete revision':
'validate' :
        if (!
$node->cheese) {
form_set_error('cheese', t('You must enter a cheese!'));
$additions['cheese'] = db_result(db_query("SELECT cheese FROM {hosting_cheese} WHERE vid=%d", $node->vid));

At this point, you can go to /admin/build/modules and enable the Cheese feature. The database table will get installed, and you can now go to Create Content > Site and see our cheese field is present in the site form!

Part two - sending the cheese to the backend

Yes, it's very exciting that we have cheese in Aegir now. But don't provision a site yet! The fun hasn't even started.

In the same directory as the hosting_cheese module, create a new file called This is a Drush file that is going to be responsible for catching when an Install or Verify task is being executed against a site, and to send the cheese value along for the ride to the backend.

* Implementation of drush_hook_pre_hosting_task()
* Send the site's cheese attribute to the backend for processing.
function drush_hosting_cheese_pre_hosting_task() {
$task =& drush_get_context('HOSTING_TASK');
  if (
$task->ref->type == 'site' && ($task->task_type == 'install' || $task->task_type == 'verify')) {
$task->options['cheese'] = $task->ref->cheese;

In this code, $task->ref represents the node object (the site).

Because of our additions in our implementation of hook_node_load(), the cheese value is fetched and available for us to use.

Part three - an install profile (just for fun)

I'm demonstrating all this to show you how to get data from the frontend (in the site form) to the backend. We've achieved that, but you probably want to actually *do* stuff with the data.

I'll demonstrate two things you can do with this data. They are just examples - Aegir isn't limited to this at all.

First let's create a simple install profile called 'Cheesy sites'.

* Return an array of the modules to be enabled when this profile is installed.
* @return
*  An array of modules to be enabled.
function cheesy_sites_profile_modules() {
  return array(
/* core */ 'block', 'color', 'filter', 'help', 'menu', 'node', 'system', 'user', 'path',
/* other contrib */ 'install_profile_api', 'token', 'pathauto', 'views',

* Return a description of the profile for the initial installation screen.
* @return
*   An array with keys 'name' and 'description' describing this profile.
function cheesy_sites_profile_details() {
  return array(
'name' => 'Cheesy site',
'description' => 'Select this profile to install a really cheesy site.'


cheesy_sites_profile_tasks(&$task, $url) {
// Install dependencies
// Fetch and set the cheese attribute from Drush, sent originally
  // from the site form in the Aegir frontend.
variable_set('cheese' , drush_get_option('cheese', 'camembert'));

This is not meant as an exercise in learning how to write install profiles so I'll gloss over some of the details here. Basic version: we define our modules that the profile depends on (core + some contrib), we install those modules, and finally we do a basic variable_set to store a 'cheese' variable in the site's database, fetching the value from the Drush/Provision backend (sent by the frontend per above), defaulting to 'camembert' if there was none.

Obviously in a normal install profile, this file would be bigger, as you'd be doing a bunch of other stuff such as setting up node types, setting up menus, blocks, users etc.

I provide a Drush makefile here to fetch the contrib for you. This is totally out of the scope of the tutorial, but it's just fun to use Drush Make and I encourage you to get used to it :)

core = 6.x
api = 2

; Contrib modules
projects[token][version] = "1.15"
projects[pathauto][version] = "1.5"
projects[views][version] = "2.11"
projects[install_profile_api][version] = "2.1"

Download Drupal into /var/aegir/drupal-6.19 and copy the cheesy_sites directory into the profiles directory of drupal-6.19, or use a Drush stub makefile to do it all.

If you downloaded core manually, you'll want to cd drupal-6.19; drush make profiles/cheesy_sites/cheesy_sites.make --no-core. The contrib will be downloaded to /var/aegir/drupal-6.19/sites/all/modules

Add the platform

Add a new Platform in Aegir with the Publish Path being that which we just created: /var/aegir/drupal-6.19. A Verify task will be spawned, and Aegir will pick up all the modules and also the 'cheesy_sites' profile, which you can check by viewing the 'Installation profiles' section of the 'Packages' menu tab on the platform node, after the Verify task has completed.

You could also add this profile and its dependencies to an existing platform and just re-Verify the platform, and the new profile will be synced up and added to this platform's package registry.

Part four - make the backend knowledgeable about cheese

Now we've set up the frontend to get the cheese, set up a drush file to send the cheese to the backend, and we have an install profile that expects to find cheese and set a variable about it in a database when installing a site.

We've left out one crucial stage, and that is to actually prepare the backend to tell it that cheese is coming!

Telling the backend about the incoming cheese is very similar to what we did earlier about capturing the cheese info on specific tasks and do $stuff with it.

To tell Drush and Provision (Aegir's Drush extension) about the cheese, create a file in ~aegir/.drush called

We'll invoke a Drush hook that sets the value for the backend on installing a site.

* Implementation of hook_pre_provision_install()
function drush_cheese_pre_provision_install($url = NULL) {
drush_set_option('cheese', drush_get_option('cheese', 'camembert'), 'site');

This saves the value into the site context, where it's usable by the install profile. This occurs in the pre hook, prior to when Aegir starts executing the tasks outlined by the install profile.

Part five - using Aegir hooks to inject settings into the site vhost

In Part four, we'll have accomplished what is needed to set the cheese value for the backend, which in turn means you can now create a site in the Aegir frontend, choose the 'Cheesy site' profile, and expect the install profile to pick that cheese setting up and store it in the new site's database as a variable.

But! While I'm here, let me show you another Aegir hook that allows you to safely inject extra configuration sent from the frontend into the site's Apache vhost config file, persistently, without it being overwritten next time the site is Verified.

We'll invoke an Aegir hook called provision_apache_vhost_config() to insert a SetEnv into the vhost file, just for the hell of it.

Another example where you might want to do this is, say, to set a htpasswd password in the site form, and use the hook to inject mod_auth htpasswd protection for the site (perhaps a demonstration for another time!).

Add this to your

* Implementation of hook_provision_apache_vhost_config()
function cheese_provision_apache_vhost_config($uri, $data) {
"  SetEnv cheese  " . drush_get_option('cheese', 'camembert');

The return value is inserted into the vhost. You can return multiple lines by wrapping them in an array, or even more elegantly, simply return a path to a template .tpl.php file and do your customisations in the template!

Part six - check the results!

If you've done everything up to Part four and five prior to installing a site using the 'Cheesy site' install profile, do so now!

I'm going to create a site called '' and set my cheese type to be 'cheddar'.

Now let's check to see those settings in their end result:

The variable in the new site's database

Run this command: drush vget cheese

You should see something like this:

Great! Now let's look at the Apache vhost file for this site, and we should see a 'SetEnv' parameter injected into the vhost.


I know these have been silly examples, but I hope you've taken some important lessons out of all this:

  • How to extend the frontend of Aegir and hook into things like the site form and site node
  • How to capture data on task 'events' like install and send it to the backend
  • How to recognise that data in the backend and use it in install profiles or invoke our hooks to inject the data into configuration files
  • How much I really like cheese

Cheesy tarball

As mentioned at the start of this article, you can download all these components I've been talking about here: cheesy.tar.gz. Remember to read the README.txt

Further reading

A great article by hadsie on g.d.o on sending data to an install profile via the Signup form covers similar information on capturing data sent to the backend in an install profile.

Steven Jones has published a HTTP mod_auth feature extension for injecting htpasswd password protection on sites over at

E-Commerce Integration Roadmap

Related links:

  • For installation and configuration instructions, see the Administrator Manual.
  • There was some valuable discussion on use cases on

Don't put feature requests in here. Submit them to the uc_hosting or hostmaster issue queues instead.

1. Drupal 6 / uc_hosting 1.0

Aiming for compatibility with:

  • Hostmaster 1.0
  • Ubercart 2.4


  • uc_hosting: This implements shared functions and classes, including the function to create a client based on an ubercart order and product.
  • uc_hosting_products: This is where we put the ubercart function calls, attribute generation, etc... This module is intended to be used alone for the "my ubercart is on my aegir site" scenario.

1.1. Goals

1.1.1. Single site products


1.1.2. Multi site packages


1.1.3. Recurring payments

We need to test the module using uc_recurring.

1.2. Nice-to-haves

Given where we are at now, this stuff will most likely not make it in until an eventual 1.1 release. The exception being Hadsie's work on "try before you buy".

1.2.1. Purchasing wizard

Stills needs to be developed. The create my site now option is a start and is already in.

1.2.2. Make it unnecessary to log single-site purchasers into the aegir site

Needs some testing but should be doable. Maybe just needs a little documentation. I believe Hadsie's work is related to this.

1.2.3. Deferred payments

Ergonlogic is working on this.

2. Drupal 7 / uc_hosting 2.0

We are still unsure about whether to support commerce, ubercart, or both with this module. We will most likely wait until development on the D7 port of aegir starts before beginning this work so we will see then what the landscape looks like.

Aiming for compatibility with:

  • Hostmaster 2.x
  • Ubercart 3.x, or Commerce 1.x (maybe both?)

2.1. Goals

2.1.1. Remote storefronts

Hopefully, this will be supported by plans for new xmlrpc methods in the hostmaster signup module.

2.2. Nice-to-haves

2.2.1. Desjardins recurring payments

This is a seperate module being developed by Koumbit.

2.2.2. Importable demo products

This would be really great.

2.2.3. Simple procedure to deny user 1 to certain clients

Ergonlogic has published a module that should make this possible:

3. Wishlist:

4. Unanswered Questions

  1. Will the Drupal 7 version use Ubercart or Commerce? Both? Neither?
  2. How and when will people be billed? And what degree of control will uc_hosting offer over that?
  3. Aegir has clients & UC has clients. How are they related/distinct? How do we keep this clear to users/admins?

Creating product features using uc_hosting 1.x


This document assumes you want to implement a product feature using uc_hosting's database tables and exiting infrastructure. For a more complete example of an Ubercart 2.x product feature implementation, check out uc_file in the Ubercart core.

The Basics

Any module implementing a uc_hosting feature should include both uc_hosting and uc_hosting_products as dependencies. Include these lines in your .info file:

dependencies[] = uc_hosting
dependencies[] = uc_hosting_products

Declaring product features to Ubercart 2.x

The first step is to implement Ubercart's hook_product_feature. Here is an example implementation for the platform access feature in uc_hosting_products:

 * Implementation of hook_product_feature().
function uc_hosting_products_product_feature () {
  $features = array();

  // Set the feature for the platforms
  $features[] = array(
    'id' => 'hosting_platform',
    'title' => t('Access to a platform'),
    'callback' => 'uc_hosting_products_platform_form',
    'delete' => 'uc_hosting_products_feature_delete',
    'settings' => 'uc_hosting_products_platform_settings',

  return $features;

An implementation of hook_product_feature returns a numerical array of associative arrays. Feature arrays must include five keys:

  • id will be used to identify your feature at the database level, in the uc_product_features table, and also in urls. Think of it as the type of your feature.
  • title is the name of your feature as it will be displayed to site admins.
  • callback will be used to create and modify instances of your feature attached to specific products. This callback should return a drupal form passed through the uc_product_feature_form function.
  • delete is a simple function called on the removal of a feature instance from a product.
  • settings is a callback that provides additional settings for the feature, and is not used presently in any uc_hosting product feature.

Of note is that, unlike Drupal's core hook_menu and hook_theme, hook_product_feature does not include a 'file' key. uc_hosting works around this by using include_once at the beginning of the hook_product_feature implementation.

If you wish to use uc_hosting's shared feature functions, make sure to include the file "products/inc/" (path relative to the uc_hosting root) at the beginning of your implementation of hook_product_feature.

The callback form

This form is called everytime someone adds a product feature to a product. It allows you to set any options for your feature that should be delivered on a per-product basis. Lets take a look at this function for the platform access feature:

 * Callback to add the platform form on the product feature page
function uc_hosting_products_platform_form ($form_state, $node, $feature) {
  $form = array();

  // Get default values and shared form elements for all uc_hosting features
  $hosting_product = _uc_hosting_products_feature_fetch_product($feature);
  if (empty($feature)) {
    $feature = array('id' => 'hosting_platform');

_uc_hosting_products_feature_fetch_product attempts to retrieve existing values for this instance of a product feature from the database.

  _uc_hosting_products_feature_shared_elements(&$form, $node, $feature, $hosting_product);

_uc_hosting_products_feature_shared_elements declares some form elements that either uc_hosting or Ubercart itself depend on to provide the product feature functionality.

  // @see _hosting_get_platforms()
  if (user_access('view locked platforms')) {
    $platforms = _hosting_get_platforms();
  else if (user_access('view platform')) {
    $platforms = _hosting_get_enabled_platforms();
  else {
    $platforms = array();
    drupal_set_message('You must have permission to access platforms to enable this feature.', 'warning');

  $form['platform'] = array(
    '#type' => 'select',
    '#title' => t('Platform'),
    '#description' => t('Select the platform to associate with this product.'),
    '#default_value' => $hosting_product->value,
    '#options' => $platforms,
    '#required' => TRUE,

This code block generates a list of platforms to allow the admin to select which platform to bind to his product. This code is specific to the platform access implementation - this is where you should include your own form elements relevant to your feature.

  $form['#validate'][] = 'uc_hosting_products_single_feature_validate';

For many implementations, uc_hosting_products_single_feature_validate ensures that a given uc_hosting product feature can only be enabled once on any given product. All of the exising uc_hosting_products features make use of this function.

  return uc_product_feature_form ($form);

Finally, uc_product_feature_form adds some required form elements from Ubercart.

You also need to make sure to code a submit function for your callback, of the form callback_submit:

 * Save the platform feature settings.
function uc_hosting_products_platform_form_submit($form, &$form_state) {
  $hosting_product = array(
    'pfid' => $form_state['values']['pfid'],
    'model' => $form_state['values']['model'],
    'type' => 'platform',
    'value' => $form_state['values']['platform'],

  $platform_node = node_load($hosting_product['value']);
  $description = t('<strong>SKU:</strong> !sku<br />', array('!sku' => empty($hosting_product['model']) ? 'Any' : $hosting_product['model']));
  $description .= t('<strong>Platform:</strong> !platform', array('!platform' => $platform_node->title));

  $data = array(
    'pfid' => $form_state['values']['pfid'],
    'nid' => $form_state['values']['nid'],
    'fid' => 'hosting_platform',
    'description' => $description,

  $form_state['redirect'] = uc_product_feature_save($data);

We manually save the product_feature to ubercart's database tables here. This is so that we can then retrieve the index of the entry in the uc_product_features table for later use in our own data.

  // Insert or update uc_hosting_products table
  if (empty($hosting_product['pfid'])) {
    $hosting_product['pfid'] = db_last_insert_id('uc_product_features', 'pfid');

  $existing_prod = db_fetch_object(db_query("SELECT hpid, data FROM {uc_hosting_products} WHERE pfid = %d LIMIT 1", $hosting_product['pfid']));

  $key = NULL;
  if ($existing_prod->hpid) {
    $key = 'hpid';
    $hosting_product['hpid'] = $existing_prod->hpid;
    $hosting_product['data'] = unserialize($existing_prod->data);
    $hosting_product['data']['platform'] = $hosting_product['value'];
  // If necessary build a data array from scratch
  else {
    $hosting_product['data'] = array(
      'platform' => $hosting_product['value'],

  $hosting_product['data'] = serialize($hosting_product['data']);

  drupal_write_record('uc_hosting_products', $hosting_product, $key);

Note that the uc_hosting_products table has both an integer column value, and a longtext column data for storing serialized information about the feature.

Taking action in Aegir based on product features

uc_hosting assumes that most purchases made via Ubercart are intended to be expressed as changes to an Aegir client node. So it provides some code to help you create new clients or modify exiting ones when an order containing your feature is made. Any other actions needed to make your feature work you will need to implement yourself. This will involve implementing the Ubercart hook, hook_order and your own callback function.

Lets start by taking a look at uc_hosting_products own implementation of hook_order:

 * Implementation of hook_order
 * Actions t$form_state['values']['create_later']o take on order changes involving an aegir product
 * @param $op string
 *   Provided by ubercart on invocation
 * @param &$arg1
 *   Different data depending on the op
 * @param $arg2
 *   Different data depending on the op
function uc_hosting_products_order ($op, &$arg1, $arg2) {
  switch ($op) {
    case 'update':
      if ($arg2 == 'completed') {
        foreach ($arg1->products as $product) {
          if (_uc_hosting_products_has_feature ($product)) {
            // Make the changes necessary to the hosting client
            $client = uc_hosting_update_client($arg1, $product, 'uc_hosting_products_client_update');

_uc_hosting_products_has_feature is defined in uc_hosting_products.module, and provides some simple logic to detect uc_hosting product features. Rather than use this function, you will most likely want to define your own conditions.

uc_hosting_update_client does the work of matching up an incoming ubercart order to an existing Aegir client, if possible. If not possible, it creates a new client node. It then calls the function provided as a third argument, in this case uc_hosting_products_client_update, and passes to that function the affected client node, as well as the product and order objects from Ubercart.

This callback function is the place to take actions in Aegir or pass commands. Take a look at uc_hosting_products for a fully fleshed out example.

Class auto-loading


Provision has been refactored in 6.x-2.x to use class auto-loading. This simplifies use of classes, as it obviates the need to explicitly include the files in which the class and it's parent classes are defined. In addition, it standardizes the placement of class definitions within a hierarchical file-system structure and will make name-spacing simpler if/when we begin to adopt Symfony components.

While not strictly required in contributed modules, it is highly encouraged. If you maintain a contributed Provision extension, some re-factoring will be required to support class auto-loading.

First off, you'll need to add a registration function that will enable Provision to autoload your extensions classes. Here's an example from provision_civicrm:

* Register our directory as a place to find Provision classes.
* This allows Provision to autoload our classes, so that we don't need to
* specifically include the files before we use the class.
function civicrm_provision_register_autoload() {
  static $loaded = FALSE;
  if (!$loaded) {
    $loaded = TRUE;
    $list = drush_commandfile_list();
    $provision_dir = dirname($list['provision']);
    include_once($provision_dir . '/');
    include_once($provision_dir . '/');
    provision_autoload_register_prefix('Provision_', dirname(__FILE__));

Next, your extension will need to call your new function in a hook_drush_init(), so that it is run as early as possible during a Drush bootstrap.

* Implements hook_drush_init().
function provision_civicrm_drush_init() {
  // Register our service classes for autoloading.

Finally, you can move your class definitions into the proper file-system structure. In provision_civicrm, we defined a new (stub) service to allow saving data from the front-end into a site context (entity/alias). So, we created a file at Provision/Service/civicrm.php that included the class definition:

* The civicrm service base class.
class Provision_Service_civicrm extends Provision_Service {
  public $service = 'civicrm';


   * Add the civicrm_version property to the site context.
  static function subscribe_site($context) {

Customizing Platform Templates

Non-standard Aegir installs may require adjustments to the default configurations for a platform that are defined by Aegir during the creation of a platform. These templates can be found in /home/aegir/.drush/provision.

Editing the actual templates prevents Aegir from overwriting your customizations when the verify task is run on an live platform.

Below are examples of customizations made to non-standard Aegir installs

Running an Aegir platform on a unix socket for php-fpm on Nginx

First let's find out where provision has the default config templates for our platfom.

[root@host]cd /home/aegir/.drush; grep -nir "fastcgi_pass" --exclude=*.svn* *
provision/http/nginx/nginx_advanced_include.conf:227:        fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:360:          fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:213:        fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:346:          fastcgi_pass; ### php-fpm listening on port 9000
We can see that the config templates for Nginx direct all php requests to port 9000 on the localhost I.P. Let's change that to a unix socket (which avoids the slight delay associated with tcp connections). After commenting out those lines and adding our custom lines which will use a unix socket instead of a tcp port, we get..
provision/http/nginx/nginx_advanced_include.conf:227:    #    fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:228:        fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_advanced_include.conf:361:          #fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_advanced_include.conf:362:          fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_simple_include.conf:213:        #fastcgi_pass; ### php-fpm listening on port 9000
provision/http/nginx/nginx_simple_include.conf:214: fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;
provision/http/nginx/nginx_simple_include.conf:347:#          fastcgi_pass; ### php-fpm lis#tening on port 9000
provision/http/nginx/nginx_simple_include.conf:348:     fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock;

Remember to do the same to your platform config files in /home/aegir/config/includes. Next time you verify your platform your customizations will remain after verification.

Passing values into the site drushrc.php file during migrations

It's possible rely to on manipulating the drush context for certain tasks and using hook_post_command to make use of that data. In this case, I wanted to save values to the site's drushrc.php file.

Add the following code to hook_post_provision_verify:

if (d()->type == 'site') {
$val = drush_get_option('my_custom_option');
drush_set_option('my_custom_option', $val, 'site');

This makes sure that my_custom_option perpetuates through these potentially destructive operations. Since it is provision-verify that writes the drushrc.php file for the site, drush options set in the site context during hook_post_provision_verify will be saved.

The final step is to make sure the verify hook is run with the appropriate options at the end of my migrate task, so add this to hook_post_provision_migrate.

if (d()->type == 'site') {
$options = array('my_custom_option' => drush_get_option('my_custom_option'));
$target = drush_get_option('target_name');
provision_backend_invoke($target, 'provision-verify', array(), $options);

I got this idea from looking at the drush_provision_drupal_provision_migrate function in platform/

I used this approach to create a patch for provision_civicrm

This was originally documented in the provision issue queue.

Integrating Aegir


This section of the documentation deals with integrating Aegir components with other services (as opposed to extending or overriding direct Aegir functionality itself).

Examples might include using Aegir's database to authenticate users based on their presence as 'clients' in the Aegir system, e.g authenticating FTP access to sites for clients that have those sites managed in Aegir.

Integrating Aegir with OpenSSH

Adam has written an article that explains how to hook up OpenSSH and PAM on your Linux server, so that you can authenticate your clients via the Aegir database and have them SFTP/SSH into your server and be chrooted (locked) into just seeing their sites (or a platform) with a restricted command(s) shell.

Read the article here.

Integrating Aegir with PureFTP

Cafuego has written an excellent article that explains how to hook up PureFTPd and PAM on your Linux server, so that you can authenticate your clients via the Aegir database and have them FTP into your server and be chrooted (locked) into just seeing their sites.

Read the article here.

Continuous Integration

Testing server URL:

Recently, mig5 put together a testing server based on Jenkins.

So what? In doing so, he's produced a tool that will allow us to spin up a temporary virtual server, drag in our codebase from Drush Make, other codebases as platforms, and do test installs of sites on it. This should become a routine start-to-finish test of our applications, features, modules, etc. All at the click of a button (or less, by scheduling it!)

Currently, we have a build project that:

  • Provisions a new VPS at Rackspace Cloud
  • Installs Aegir 6.x-1.x branch
  • Builds a Drupal 6, Drupal 7 and OpenAtrium platform using Drush Make
  • Installs a site (with respective install profiles) on all three platforms

If anything fails here, an alert will come through to #aegir on Freenode. This will help us to test the Aegir install (especially to ensure our compatibility with Drush and Drush Make releases), and to identify any regressions in our own project.

Every 15 minutes, Jenkins will check if there's a new commit to hostmaster and provision, and if there is, fire up a new install and test a few things to find regressions.

We also have a build project for the most recent official release of Aegir (@TODO: test .deb installation method of Aegir for this), as well as a build that installs an earlier Aegir and then upgrades to the latest 6.x-1.x to test the upgrade path.

Also, steven has created a sandbox to host the code that runs the tests on the jenkins server.

Despamming guide

We have seen numerous waves of spam on this site. At first we setup mollom, in 2011 but sometimes it is not enough.

Current practices

The very first step is to identify the user account and disable it (not delete it, we need it to find the content to remove).

Views Bulk Operations views used for despamming:

Note that this will flood your screen with drupal_set_message for every page that is deleted, which may make the site difficult to use for a while. To top that, the final page will list the name of every page that was removed, just to make sure you had enough spam for the day.

You can also use the regular comment view to delete content and report it to mollom if you are patient enough.

You can also delete the account if you are motivated and want to grind that specific axe. Look at the regular user listing for the latest user accounts created.

Other ideas