It is possible to import existing sites into the Aegir hosting system, so that they can be managed by Aegir.

These instructions assume you've set up a new server with Aegir on it, and you want to import sites into Aegir from another server, or even from the same server, using backups.

There are basically three ways of importing sites in Aegir:

• from Aegir backups - simplest and fastest, scriptable
• from any single site, manually - most reliable, can import even non-aegir sites, hard to automate
• from a complete Drupal install - fairly reliable, can import non-aegir sites, could be automated but requires access to the database servers currently in use by the site

This section also regroups troubleshooting and other less orthodox techniques.

If you need to import a site from an existing Aegir system, this can be done from an Aegir site backup.

Requirements:

1. New Aegir version 0.4 alpha15 or later.
2. Access to Aegir backups from the old server.
3. A platform set up and verified that matches the platform on the old Aegir server.

1. Copy backups

First log into the new server via ssh - switch to the aegir user. Create a temporary directory somewhere, for example:

Run a command to syncronize the new directory with the backups directory of the old Aegir hosting system. This command does the entire directory, but you may want to modify this if you have a lot of unneeded backups on the old system.

This command also assumes that you can shell into the other server as the aegir user. Modify as required.

Now you can re-run the above command just before you are going to migrate a site. This is handy if you have a lot of sites to migrate and each site requires attention. Simply make a new backup of the site in the old system, and rerun the rsync command.

2. Create the site context and deploy the backup

For brevity, I assume that drush can be executed as drush. Replace with php /var/aegir/drush/drush.php if needed.

Pay attention to these two commands, they require modification to match the name of the site, the alias of the platform, and the name of the backup archive. These commands should each be on one line. You also need to know the platform alias. You can see all the platform aliases available by typing drush sa | grep platform.

First tell Aegir about the site, which creates a site alias.

Then run the provision-deploy:

Note: if you receive an error, you might also try adding the site to your new Aegir platform (using the same platform and site name, of course). Then run the provision-deploy again.

3. Verify the Platform

The final step is to import the site into Aegir. This is easily done by logging into the Aegir front-end and verifying the platform, which then will trigger an import task for the new site.

4. Caveats

The provision-save command will assume the database server is the same as the master server and that you are not using any install profile. So if your DB lives in another server or if you are using an install profile like openatrium you will probably need to edit the file created at /var/aegir/.drush/example.com.alias.drushrc.php

5. Setting a Non-Default Install Profile

If you are using a non-default install profile, you may want to specify your install profile by adding --profile=profilemachinename to the drush provision-save command above.

Using the example above but using an install profile for Open Atrium, the command would look like:

If you already have a platform setup on Aegir with EXACTLY the same codebase as your existing site, then you don't need to transfer the entire old codebase - you can just transfer the sites/example.com directory. However, you also need to make sure any dependencies on contrib modules that your site has, are covered on the codebase or Platform that you're importing it into.

In general it may be considered safer/less prone to confusing errors to transfer the entire old codebase into Aegir as a whole Platform, whereby the site will be imported automatically under that Platform. You can then migrate it in Aegir to one of your existing Platforms later. See importing from a complete Drupal platform for this method.

Also, it is much faster to just "deploy" an Aegir backup than go through the manual procedure here, so you should generally follow that procedure instead of the one defined here unless you have a very hairy setup.

To transer a site manually into Aegir, those are the steps you should follow:

1. Create the site database, user and directory

In order to import the site, you need to create a database and database user for the site, along with a directory in the platform.

The simplest way to do this is to create an empty site in the right platform and overwrite it. Through the commandline:

Through the UI, this can be done simply by creating a site in Create content.

If this is not working for some reason, you can create the mysql user and database manually using the following:

2. Transfer the files

You need to copy the sites/example.com directory from your old server to the new Aegir server.

To do this, first create a tarball of the site on the old server. Within the sites/example.com directory type:

Then, on the Aegir server, you can create a directory for it in the codebase (platform) you want to put it under:

Instead of using wget you could of course use SCP or FTP to transfer the tarball onto the Aegir server.

In the above we assume that you have created an empty site as directed above, if not, you will also need to extract the settings.php file, so that last command should instead be simply:

Then you will need to modify the settings.php file to follow the user and database created in the first step. Again, this is not necessary if you let Aegir create the site for you.

Once the file is unpacked, check the ownership and group of each file by using ls -al. It should be either 'aegir:aegir' or 'aegir:www-data'. Change it if necessary. In particular, files that may have been uploaded on your site via modules like imagecache, upload, Profile pics etc, may need to have their ownership changed:

3. Transfer the Database

Make a backup of the database on your old server (Backup and Migrate module is good for this, or you can use phpmyadmin or your favourite MySQL management tool. It's a good idea to truncate the cache, search and watchdog tables first to reduce the size of the database. The "Backup and Migrate" module does this for you.

If you are importing a site from a previous Aegir installation somewhere, such as from a backup tarball, the database.sql will already be within the tarball you unpacked. You can use this to import into a new database on your new server.

You can import the database with drush using the following command:

4. Verify In Aegir

In the Aegir web interface click on the name of the platform you have added your site to. You can access the list of platforms via the 'Platforms' tab in the primary links menu.

Execute a new Verify task on this platform via the 'Verify' button in the task list.

Aegir will now re-verify the platform. In the process of this, it will discover your new site and spawn a new 'Import' task for the site.

Once it's done this (it may take a couple of cron runs to dispatch these tasks in the queue) the tasks should turn green (if not, see 'Troubleshooting' below).

At this point, presuming you have updated DNS or are overriding DNS via an entry in your /etc/hosts file to access the site via the new Aegir server, you can visit the site in a browser and check around to see that it has worked. Pay particular attention to any links in node content that pointed to paths referencing /sites/community.aegirproject.org if your site was not part of a multisite setup. (Drupal has a habit of storing these paths in the database, or they may have been hard-coded into nodes by users).

Aegir will not go through all your database and update all URLs, so some images or links may be broken. This will need to be done manually. Aegir does attempt to update paths stored in tables such as 'files' though.

It's a good idea to clear the caches, and you may need to get imagecache to rebuild its thumbnails if you use it.

This way of importing sites is often considered the safest whereby you transfer across an entire codebase containing Drupal core, and define it as a Platform first. This makes sure you bring any dependencies, contrib modules etc, in exactly the same version and configuration that is referenced in the database for the site.

1. Transfer the codebase

Create a tarball of the codebase on the old server. Within the drupal root directory type: tar -zcvf example.com.tar.gz * .htaccess

Note that we have to specify including the .htaccess because it isn't picked up by the * wildcard due to being prefixed with a full stop. It is important to bring along the .htaccess for a platform so that Aegir can consider these settings when it generates Apache configuration for this platform.

Then, on the new server you can create a directory for it in the codebase you want to put it under:

Instead of using wget you could of course use SCP or FTP to transfer the tarball onto the Aegir server.

Once the file is unpacked, check the ownership and group of each file by using ls -al. It should be either 'aegir:aegir' or 'aegir:www-data'. Change it if necessary. We have to assume you have basic knowledge of Linux and POSIX permissions - we can't be responsible for teaching you sysadmin skills :)

In particular, files that may have been uploaded on your site via modules like imagecache, upload, Profile pics etc, may need to have their ownership changed: sudo chown -R aegir $platform/sites/*/files

If your site previously resided in sites/defau1t you need to move it because Aegir ignores the default directory (it has no way of understanding what URL this would be accessed by, so it is impossible to 'manage' it). Each site needs its own directory with the correct domain in typical Drupal multisite design:

If you do this, you may need to also update your filepath's stored in the files table by running the following query on your database:

There are many other places in the database that can suffer the same problem, the node_revisions table for example. If you have phpMyAdmin available you can easily search the entire database for 'sites/defau1t'. Select the database, then click the Search tab. Once you know which tables are affected you can run a variation of the above query to correct these records.

2. Transfer the Database

Note: This step is not necessary if you are moving your site from one directory on your server (e.g., /var/www/html) to a newly created Aegir directory on the same server (e.g., /var/aegir).

Make a backup of the database on your old server (Backup and Migrate module is good for this, or you can use phpmyadmin or your favourite MySQL management tool. It's a good idea to truncate the cache, search and watchdog tables first to reduce the size of the database (Backup and Migrate does this for you).

Aegir does not provide support for importing databases with prefixed tables so it is best to remove prefixes form the database. However, there is work to bring this support in (http://drupal.org/node/483346) and a solution if you want to keep prefixes on your tables (http://drupal.org/node/483346#comment-3113516)

Unlike traditional Drupal projects the database settings cannot be found in settings.php. The database credentials are stored in the Apache vhost config of the associated site. This is a security measure implemented by the Aegir project.

In the vhost directory on your old server, /var/aegir/config/server_master/apache/vhost.d/, is a vhost file for every site on your server. The contents of the file can be displayed in your terminal with the command cat thenameofthefile. At the top of the file you will find the credentials needed to create a database on the new server.

On your new server, manually create a new database and upload the .sql file from your backup.

Then create the mysql user that your site accesses the database as, and grant it all permissions on that database except 'GRANT'.

Connect to the database on the old server: mysql -u root -p

Create the new database: create database yourdatabasename

Add the user: CREATE USER 'yourdatabaseuser'@'localhost' identified by 'youruserpassword';

Give the necessary privileges: GRANT ALL PRIVILEGES ON yourdatabasename.* TO 'yourdatabaseuser'@'localhost';

Import the database: mysql -u yourdatabaseuser -p -h localhost yourdatabasename < thenameofthedatabase.sql

3. Setup Platform in Aegir

In the Aegir web interface click on 'Create Content' and 'Platform'. This step is often easiest when using the admin-menu on the top of your screen.

Enter the name you want to use for the platform (eg 'My Platform import'), and the path to the platform on the server (eg '/var/aegir/platforms/your-new-platform')

Click Save to submit the form, and Aegir will spawn a Verify task for this platform into the queue.

Once the task is complete this the task should turn green (click through to the homepage to refesh and check. If not, see 'Troubleshooting' below).

4. Import and Verify Sites In Aegir

Upon completion of the above Platform verification task, Aegir will discover your new site and spawn a new 'Import' task for the site.

Once it's done this (it may take a couple of cron runs to dispatch these tasks in the queue) the tasks should turn green (if not, see 'Troubleshooting' below).

At this point, presuming you have updated DNS or are overriding DNS via an entry in your /etc/hosts file to access the site via the new Aegir server, you can visit site in a browser and check around to see that it has worked. Pay particular attention to any links in node content that pointed to paths referencing /sites/community.aegirproject.org if your site was not part of a multisite setup. (Drupal has a habit of storing these paths in the database, or they may have been hard-coded into nodes by users).

It's a good idea to clear the caches, and you may need to get imagecache to rebuild its thumbnails if you use it.

5. Migrate To Another Platform

Now that you have your sites under Aegir's control you can take advantage of its power, and easily migrate them to another platform. In the event that your sites were on old versions of Drupal core and a new one is available and present as another Platform on your Aegir server, you can use the Migrate process to upgrade those sites onto the new core.

For details on how to Migrate sites, consult the Migrate documentation.

6. Troubleshooting

For those who are migrating existing sites that live with their files in the sites/community.aegirproject.org folder, things can get tricky. However, you can follow this tutorial and successfully get things moved over rather easily.

We have already seen how to import sites in an Aegir install, but what if we want to test the "hostmaster" site itself, ie. setup a development or staging environment of an Aegir install itself?

This page explains how. The basic steps are:

1. Installing aegir

The first step should be fairly simple and intuitive if you got that far: you have to install aegir itself on the staging server. You should install the specific version that is currently in production.

Maybe that also involves uninstalling the existing install.

2. Backup the production site

Run a backup on the production site, on the production server. This can be done in the frontend or in the backend using:

3. Deploy the production site in staging

Copy the resulting backup to the staging server. Then you should be able to deploy it with the following command:

In this case, we use deploy to also rename the site (--old_uri=aegir.koumbit.net).

4. Tweaking the install for the staging server

The environment on the staging server could be very different from the production server. The server name will be different, maybe there will not be a dedicated mysql server, platforms will not exist, etc.

It is recommended to review the "Servers" page to change the domain names to the staging environment. In Koumbit's case, this means:

1. editing the master server to change the hostname and IPs
2. renaming the DB server to "localhost" and changing the database credentials
3. renaming the main hostmaster site:
   drush @hostmaster sqlq "UPDATE node n JOIN hosting_context h ON h.nid = n.nid SET n.title='hostmaster.koumbit.net' WHERE name='hostmaster';"
drush @hostmaster sqlq "UPDATE node_revisions nr JOIN hosting_context h ON h.nid = nr.nid JOIN node n ON n.vid = nr.vid SET nr.title='hostmaster.koumbit.net' WHERE h.name='hostmaster';"
4. change the sites to point to the good web and db servers:
   UPDATE hosting_site SET db_server=3631 WHERE db_server <> 3631;
UPDATE hosting_platform SET web_server = 3 WHERE web_server <> 3;

Note that this will have the added benefit of reverifying all platforms and therefore create them if they are makefile-based.

5. Troubleshooting

I had weird issues with failing to run updatedb on the hostmaster site (while deploying to another alias would work). My workaround was to suspend the process right before it would run drush updatedb, then run updatedb manually, twice:

First things to check:

Does the aegir user have proper permissions?

sudo chown aegir -R PLATFORM/sites/yoursite

Is the database properly set up? Try drush sql-cli to see if the site db user has access and if there are tables (try SHOW TABLES).

If something goes wrong with importing a site you might end up with a node for a non-working site. In such a case it can be good to delete the node for this non-working site and verify the underlying platform again. Since there is no direct way to delete such a node from the user interface you have change the URL from node/NUMBER/edit into node/NUMBER/delete to do this.

Aegir supports multiple 'server' entities. These servers have 'services' such as 'Web' or 'Database', and 'service types' which are implementations of that service, such as 'Apache' or 'MySQL'.

Some previous experience in the configuration of Apache and MySQL will help if you want to use remote servers with Aegir. If you haven't had any experience in setting up and maintaining Apache or MySQL before you might want get familiar with the basic concepts first.

1. Remote web servers

1.1. System dependencies

On the remote server, install these packages

1.2. Aegir user

Any number of remote web servers may be configured. The remote server needs an aegir user created on the system.

1.3. Web server configuration

You'll also need to prepare the web server in the same way you did for the master Aegir server during installation:

Don't restart Apache even when it prompts you. This will be done by the Verify task you'll spawn for the server from the Aegir frontend later.

1.4. Sudoers

Add the aegir user to sudoers so it can restart Apache (e.g. sudo visudo -f /etc/sudoers.d/aegir):

1.5. Login shell

The remote aegir user will also need a login shell, which can be modified using the chsh command.

1.6. MySQL access

Aegir connects directly to the remote server's MySQL to create databases and users. Therefore, you need to log into MySQL in the remote server as root and create a user with the following command:

Where

• 'root' is the username on the Aegir server that will be connecting. Leave 'root';
• 'aegir_server_ip' is the IP address of your Aegir server. For example '123.123.123.123';
• 'password' is the password that will be used by your Aegir 'root' user to connect to the remote server's MySQL to create databases and users. It is suggested to store that password in a secure location. To increase security, it is recommended to choose a password with 6 to 16 alphanumeric and or punctuation characters.

If MySQL returns something like "Query OK", that means the command was successful. For example:

Query OK, 0 rows affected (0.00 sec)

Note the comment about "bind-address = 127.0.0.1" on http://community.aegirproject.org/installing/manual#Database_configuration

When the server is being verified, Aegir will attempt to create a database and grant privileges to a user for that database. If any of these two fails, verify that your MySQL configuration is correct and that there is no firewall blocking your MySQL port.

You can confirm that MySQL is configured correctly on the remote server by manually connecting from the command line on your Aegir machine:

If your MySql database is running on the same server as the Aegir master and is configured with the name "localhost", you may run an issue where the remote web server tries to access MySql via a local socket rather than the hostname. To resolve this, just change the server hostname for the MySql server in Aegir from "localhost" to the hostname of the server.

1.7. SSH keys

SSH public/private keys should be set up so the main Aegir server's aegir user can access remote web aegir users with no passwords required.

Example: on main Aegir server:

Do not use a passphrase when you create your key. Simply press Enter to leave the passphrase blank. Otherwise, SSH will insist that you enter the passphrase everytime you try to SSH using the key. We don't want SSH to present any prompts.

Put the public key's contents into /var/aegir/.ssh/authorized_keys on the remote web server. The easiest way to do this is to use the ssh-copy-id command.

You should manually login for the first time from your Aegir master server to your remote server as the aegir user, so that the remote web server is added to the known_hosts file in /var/aegir/.ssh on your Aegir master server. Verifying the remote webserver will fail until this has been done.

You can confirm that SSH is set up correctly by manually connecting from your aegirmaster server to your remote server as aegir@ user to aegir@ user:

This should directly log you into the remote shell command line without any additional keys pressed.

There are many, many tutorials online for setting up ssh keys, and various mistakes can be made by inexperienced users such as permissions etc. Aegir isn't a 'Linux beginner's practice tool', so setting these up is really out of the scope of this document and users are encouraged to research this on their own. (For Ubuntu, see this article).

If you have followed the instructions above, and your SSH connection gets closed immediately after you try to connect to the remote server as the aegir user, you may not have given the aegir user a shell correctly. Check the /etc/passwd file on the remote server to make sure that the aegir user has a shell.

1.8. Verify the server

Now you can add a new server node in Aegir, set the hostname and/or IP and set the service type to be 'Apache' (or Apache_SSL if this site is to handle SSL sites).

A verify task will be spawned and added to the Task queue ready for dispatching. During a server verification task, various configurations will be set on the Aegir master server and also synced to the remote web server, restarting Apache.

Now when you add a new Platform node in Aegir, you have the option of setting which web server to host it on. If you are not using a makefile but are downloading a platform manually, you must still do this on the main Aegir server. The contents will then be synced across to the web server.

You don't choose a web server when installing a new site. Because a site depends on a platform, its web server is implied by which platform has been chosen. In other words, all sites on a platform are hosted on the same server. You cannot host two sites on the same platform on different servers.

Any number of remote web servers may be configured. They need an aegir user and webserver configuration, with the same user name and directory paths. SSH public/private keys should be set up so hostmaster's Aegir user can access remote web Aegir users with no passwords. The above Apache configuration needs to be performed too.

Additionally, the remote web server needs the 'mysql' client binary installed so that Aegir can test that it can successfully connect to the database server. On (at least) Debian-based systems, this can be installed via the 'mysql-client' package.

They will also need a login shell, which can be modified using the chsh command.

Shell commands as root::

apt-get install mysql-client
adduser --system --group --home /var/aegir aegir
adduser aegir www-data    #make aegir a user of group www-data
chsh -s /bin/sh aegir
apt-get install rsync apache2 php5 php5-cli php5-mysql
mkdir /var/aegir/.ssh
cat > /var/aegir/.ssh/authorized_keys <<EOF
ssh-rsa AAAAB3NzaC1yc2EAAAADAQAB...UF aegir@filer01
EOF
chown aegir:aegir /var/aegir/.ssh -R
chmod 750 /var/aegir/.ssh
chmod 640 /var/aegir/.ssh/authorized_keys
a2enmod rewrite
ln -s /var/aegir/config/apache.conf /etc/apache2/conf.d/aegir.conf
cat > /etc/sudoers.d/aegir <<EOF
Defaults:aegir  !requiretty
aegir ALL=NOPASSWD: /usr/sbin/apache2ctl
EOF
chmod 440 /etc/sudoers.d/aegir

For clusters, use the aegir-cluster-slave package to operate the above and NFS mounts for you. See also web cluster configuration documentation.

Especially if you have been using Aegir for a while simply because it handles updates, moving to new releases, cloning, aliases and everything Drupal with such elegance on a single server - you might not be entirely clear on what changes when you make the move to multiple servers. Hopefully this will help.

Understanding the process

Multiple servers are great even if you do not have to manage a server farm. Having a server for development that you are free to break, one for testing that you hope will not break, one for approvals so your clients can go to see how their "real" website will look, and one for production that serves the daily grind of live sites is a wonderful thing. This DTAP (Development, Testing, Acceptance, and Production) structure works and it is surprisingly easy to accomplish under Aegir.

If you are managing a farm, creating massive numbers of servers is quick and easy with Aegir - especially if you use virtual machines like Xen where a server image can be cloned or generated using LVM disks that can be migrated (even while running) to other physical machines.

When it is time break out an Aegir master controller and remote slaves you have to remember that Aegir is the place that it (almost) all happens. Everything is stored in Aegir - including your remote platforms and your sites - and then distributed to the slave servers by Aegir in the Verify process.

Remember too that sites reside on platforms, and platforms reside on servers. You are turning your Aegir install into an Aegir Master as the hub, and the Aeger Remote Slaves as the spokes. To move into this "hub and spoke" architecture, you must build the remote server, tell the Aegir Master about that server and verify it, install a Drupal platform for that server on the Aegir Master and finally tell the Aegir Master about that Drupal install and verify that. You then create the sites on the platform you want based upon the install profile you select for that site. Need more? How about an example:

The mechanics

(1) Build a server image on your network to become your Aegir Slave. This can be anything from a separate headless server to a Virtual Machine image on physical machine with a dozen other Virtual Machines on it. As long as it has its own IP and is capable of running a Drupal image, it will work.

(2) Ensure that your new server image is resolved in your DNS by adding it to your zone files and incrementing the serial number in the zone file. Aegir can manage that for you now to some extent, but you might want to wait until that feature leaves the experimental status. Make sure the entries in /etc/hostname and /etc/hosts all match your zone files (including reverse DNS). Reload/restart the DNS server. Any DNS slaves should automatically get notified and the incremented serial number should make sure they reload automatically. Test the resolution and ping the new Aegir Slave server image from the Aegir Master to Ensure that the IPs all resolved properly.

(2) Follow the steps in the Aegir Handbook page at http://community.aegirproject.org/node/30 carefully. You should be able to use cut and paste if you are using Debian Stable or the like. In any case, you will need those packages. You do not need Aegir on the slave, but you do need the aegir user and that user must have a shell. I prefer the bash shell, but the sh shell in the instructions will do.

(3) You need to have the "passphraseless" ssh working and proven before you continue. A web search on the terms "passphraseless ssh" plus your operating system will generally get you great instructions, but putting those instructions here for every possible combination is beyond the scope of this example. In terms of helpful hints, however, you do not need to have a password for the aegir user on the Aegir Slave. Your access will be this certificate. By simply hitting enter when it asks for the passphrase, you make the certificate passphraseless. You should carefully set the permissions and back these certificates up to a secure location. You MUST log in manually from the Aegir Master to the Aegir Slave the first time or your verify will fail. This is because the system has to add the ssh keys to the known hosts and it will ask about about it before it will allow the connection. Aegir nows nothing about this and will not answer, so the verify will fail. To be absolutely sure it works, after exiting the initial login, you should retry the login to verify it does not ask for a password and then exit the login and try an rsync (which is the process by which the Aegir Master controls the Aegir Slaves). All of these commands should work from the Aegir Master to the new Aegir Slave while logged into the Aegir Master as the aegir user with no request for a password, proving that the systems are exchanging keys (do not proceed until you can do this flawlessly without having to enter a response):

(4) Go to Add Content/Server and put in the server name, leave the Database set as None (Your sites will use the Aegir Master database and none is required on the Remote Slave) and under Web select apache or apache ssl (to match the example) then click Save. After the process completes you should see that it found the IP address properly on its own, plugged it in all by itself and then Verify quite nicely. If you look on the Aegir Slave remote server you will see that you now have directories named config and platform.

(5) Now that hard decision for platform choice comes in play. My preference is a Drupal install that is the same release as the Aegir Master server or higher. Since sites are eventually destined to be Drupal 7 one day, and the Aegir system is not Drupal 7 ready as of this writing but heading that direction, making at least one Drupal 7 slave is a good idea so that you can migrate sites to that platform as they are ready. If you are migrating sites off of your existing Aegir and onto slaves, migrating to platforms of the same Drupal version makes that process much easier. In any case, firm up your decision about which Drupal version you want to install on your new Aegir Slave Platform.

(6) Time to create that platform. The big thing here is to do it right the first time and make certain that the name of the platform is unique to any platform on your Aegir Master server. If you look in your /var/aegir directory you will notice that Aegir has its own naming convention. For instance, you should see /var/aegir/hostmaster-0.4-rc1 as the platform for the Aegir 0.4 release candidate 1. You need to consider doing the same. For instance, name your platform something like: /var/aegir/slavename-drupal-version and you will make it easier to distinguish which server the platform resides upon and the drupal version so that you can manage updates and migrations more easily. Make certain that you are not destroying one platform when creating another. Be aware that at some point Aegir started organizing platforms into a separate directory: /var/aegir/platforms and you should change to that structure if you have an older original install. Again, be careful you are not overwriting an existing install when you create the new one. For instance, if you already have a Drupal install in /var/aegir/platforms/drupal-7.14 the code below will attempt to recreate it on top of the existing install. If you are using a Makefile, be sure that the same problem does not occur in that makefile. To create a Drupal 7 Aegir Remote Slave Platform, log into the Aegir Master as the aegir user and run this simple code (of course edit it to match the platform and name you want to create:

(7) The final step in creating this platform is to actually tell the Aegir Master about it so that it can distribute it (via rsync) to the new Aegir Remote Slave. Simple enough. Just go to Create Content/Platform and the Create Platform screen comes up. Under Name write something descriptive and unique. In the example so far, this will be something like 'slavename Drupal 7.14' or in some way adhere to your chosen naming convention. I use the directory name I created above since that serves two purposes. Under Publish Path you will put the path that holds this Drupal distribution. In the example above that will be /var/aegir/platforms/slavename-drupal-7.14 but the path will be reflected from the Aegir Master to the Aegir Slave and will be the same on both. If you are using a Makefile you need to enter the full path and name of that Makefile here as well, but the creation and use of the Makefile is beyond the scope of this example. Lastly choose the server that this platform will reside upon. When you are finished, double check the screen and then click on Save. The Verify step should execute flawlessly and when it does, your Aegir Remote Slave will be complete and ready to accept sites.

(8) Lather, rinse and repeat for each Aegir Remote Slave.

Creating sites on the remote

The only difference you will notice when creating a new site under Aegir will be the choice of platform. The Create Site screen is quite self explanatory, but the main thing is to select the Install Profile that you require and then the list of Platforms that support that install profile will be populated.

As usual, Aegir makes it all faster and easier than you expect.

Cluster module

The cluster module is a simplified solution for maintaining platforms across multiple web servers. A cluster server node does not require a physical machine to be present on the network. Simply create web servers as usual and create a server node that has "Cluster" selected under the web field set of the server node add form. Select the servers that should be part of the new web cluster by clicking the check box next to each web server in the "Servers" section.

The cluster module will rsync the platforms between each web server and keep them up to date.

Pack module

A Pack consists of a single server node that is specified as "pack" and any number of server nodes specified as "apache" or "nginx". The main difference between the Pack functionality and Web Cluster functionality is that Aegir rsync's configuration (Apache & Nginx) to "all" nodes in the Pack and rysnc's the platform and site code to only the "master" node in the Pack. All other "slave" nodes will see the platform and site code via NFS.

Configuring the Pack server node:

The "pack" server node will not be used by Aegir to physically deploy sites or platforms on, so consider it more of a 'virtual server group' for logistical purposes only. When creating the pack node, you do not need to supply a valid Server hostname or IP address, so choose a naming convention that makes sense in your environment.

Next, select the "pack" radio button option under the web configuration when creating or editing the server node, in order to designate this server node as the "pack" server.

Now select the "Master" server from the list of Master servers. The Master server will be the server node that Aegir rsync's the platform and site to. Typically, you would choose the Aegir server itself as the 'master'.

Finally, select the "Slave" servers from the list of Slave servers. The Slave servers will have the Apache or Nginx config rsync'd to them but not the platform or site code, since those will be available to all Slave servers via the NFS share.

Configuring the Web server nodes (Master and Slaves):

Configure all web server nodes using these instructions. Take care to ensure the 'aegir' user and group on your NFS client machines, have the same UID/GID as that of the NFS server, or else you may run into permissions issues with NFS.

Then mount the files on the remote server.

On the NFS server:

sudo apt-get install nfs-kernel-server
echo "/var/aegir/platforms    10.0.0.0/24(rw,no_subtree_check)" >> /etc/exports
sudo service nfs-kernel-server reload

(Replace the subnet here or add specific /32 hosts as necessary for your environment)

Then on the web servers (Master, if it wasn't also the NFS server, and the Slaves):

sudo apt-get install nfs-client
sudo mount 10.0.0.1:/var/aegir/platforms /var/aegir/platforms

(Replace the NFS server's IP here with that of your master server/Aegir server)

Add this to your fstab on the servers that mount the NFS share, so that the share is mounted on boot:

10.0.0.1:/var/aegir/platforms /var/aegir/platforms nfs rw 0 0

Creating a Platform on a Pack:

When configuring a Platform to be deployed on a Pack, choose the "Pack" server node from the Web server radio group during the Platform node creation. Then you choose this Platform as usual when adding a site, and that site will be served from any servers within the Pack.

Caveats

Relying on an NFS share to serve your entire /var/aegir/platforms can be a single point of failure if the NFS share becomes unavailable. You may want to look into providing some sort of failover for NFS (google for things like DRBD and Heartbeat), or using some other form of redundancy for your NFS (Netapp filer clusters etc)

There have also been reports of issues with MySQL GRANT statements not being provided to all webservers in the pack - especially where the master server is not the Aegir hostmaster server. This issue is ongoing - see the Talk page here or this ticket

Also attached is an example diagram of a Pack cluster known to be functioning in production (with an optional MySQL-MMM cluster out of scope for this documentation), which may help you visualise the Pack and how it can be put together.

I had the hardest time finding resources & documentation on how to get this configured properly so I am adding this to the documentation for those of you who may need it.

A couple of notes: I am using Ubuntu 10.04 LTS. I set my apache to port 8082 because there was a conflict on port 8080 on my system. You should make a copy of each file before editing it. Memcache settings are drupal 7.

Varnish

Setting up varnish isn't too hard but it could end up being kind of tedious if you configure something incorrectly & you have alot of sites.

First you install it sudo apt-get install varnish

Then you have to go to configure it. I use vim. sudo vim /etc/varnish/default.vcl

These are my settings.

backend default {
    .host = "127.0.0.1";
    .port = "8082";
    .connect_timeout = 600s;
    .first_byte_timeout = 600s;
    .between_bytes_timeout = 600s;
}
sub vcl_recv {
  // Remove has_js and Google Analytics __* cookies.
  set req.http.Cookie = regsuball(req.http.Cookie, "(^|;\s*)(__[a-z]+|has_js)=[^;]*", "");
  // Remove a ";" prefix, if present.
  set req.http.Cookie = regsub(req.http.Cookie, "^;\s*", "");
  // Remove empty cookies.
  if (req.http.Cookie ~ "^\s*$") {
    unset req.http.Cookie;
  }
 

// Skip the Varnish cache for install, update, and cron
  if (req.url ~ "install\.php|update\.php|cron\.php") {
    return (pass);
  }
 

// Cache all requests by default, overriding the
  // standard Varnish behavior.
  // if (req.request == "GET" || req.request == "HEAD") {
  //   return (lookup);
  // }
}
sub vcl_hash {
  if (req.http.Cookie) {
    set req.hash += req.http.Cookie;
  }
}

Now you have to edit /etc/default/varnish sudo vim /etc/default/varnish

Here are my settings

# Should we start varnishd at boot?  Set to "yes" to enable.
START=yes
# Maximum number of open files (for ulimit -n)
NFILES=$(ulimit -n)
#Malloc size depend on your memory server
DAEMON_OPTS="-a :80 \
             -T localhost:6082 \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
             -s malloc,512M \
             -u www-data \
             -g www-data"

Now we have to configure apache's listening ports

sudo vim /etc/apache2/ports.conf

change

NameVirtualHost *:80
Listen 80

So that it looks like this

NameVirtualHost *:8082
Listen 8082

Start varnish services varnish start

Now login to aegir Edit your server & change the port from 80 to 8082

Aegir will re-verify the server, as well as all platforms & sites on the server.

If that doesnt work, you need to go to /var/aegir/config/server_master/apache/vhost.d and edit all of your virtual hosts so that the port shows 8082 instead of 80 then do the same thing with /var/aegir/config/server_master/apache.conf Reverify your server.

Now add this to the local.settings.php of the sites that you want to leverage varnish

<?php
# Tell Drupal it's behind a proxy
$conf['reverse_proxy'] = TRUE;
# Tell Drupal what addresses the proxy server(s) use
$conf['reverse_proxy_addresses'] = array('127.0.0.1','your-ip-address-here');
# Bypass Drupal bootstrap for anonymous users so that Drupal sets max-age &gt; 0
$conf['page_cache_invoke_hooks'] = FALSE;
?>

Memcache

There is hardly any current documentation for setting up memcache on D7, especially not for aegir.

FIrst you install memcache & its dependencies Download & install memcache module into the platform of the site/sites in which you will be using memcache

Then you have to edit /etc/default/memcached sudo vim ./etc/init.d/memcached

Here are my settings

#! /bin/sh
#

PORT=11211
USER=nobody
MAXCONN=1024
OPTIONS=""
DAEMON=/usr/bin/memcached

RETVAL=0
prog="memcached"

start_instance() {
        echo -n $"Starting $prog ($1): "
        start-stop-daemon --start --quiet --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON -- -d -p $PORT -u $USER  -m $2 -c $MAXCONN -P /var/run/memcached/memcached.$1.pid $OPTIONS
        RETVAL=$?
        echo
        [ $RETVAL -eq 0 ] && touch /var/lock/memcached.$1
        PORT=`expr $PORT + 1`
}

stop_instance() {
        echo -n $"Stopping $prog ($1): "
        start-stop-daemon --stop --quiet --oknodo --pidfile /var/run/memcached/memcached.$1.pid --exec $DAEMON 
        RETVAL=$?
        echo
        if [ $RETVAL -eq 0 ] ; then
            rm -f /var/lock/memcached.$1
            rm -f /var/run/memcached/memcached.$1.pid
        fi
}
start () {
        # insure that /var/run/memcached has proper permissions
        mkdir -p /var/run/memcached
        if [ "`stat -c %U /var/run/memcached`" != "$USER" ]; then
                chown $USER /var/run/memcached
        fi

        start_instance default 64;
        start_instance block 16;
        start_instance content 128;
        start_instance filter 128;
        start_instance form 32;
        start_instance menu 16;
        start_instance page 8;
        start_instance update 8;
        start_instance views 8;
        start_instance path 8;
        start_instance field 8;
        start_instance rules 8;
        start_instance token 8;
        start_instance image 8;
        start_instance apachesolr 16;
}
stop () {
        stop_instance default;
        stop_instance block;
        stop_instance content;
        stop_instance filter;
        stop_instance form;
        stop_instance menu;
        stop_instance page;
        stop_instance update;
        stop_instance views;
        stop_instance path;
        stop_instance field;
        stop_instance rules;
        stop_instance token;
        stop_instance image;
        stop_instance apachesolr;
}

restart () {
        stop
        start
}

# See how we were called.
case "$1" in
  start)
        start
        ;;
  stop)
        stop
        ;;
  status)
        status memcached
        ;;
  restart|reload|force-reload)
        restart
        ;;
  *)
        echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
        exit 1
esac

exit $?

Start memcache sudo services memcached start If you see errors u may have an issue where a one line got split into two lines. Check and make sure that none of your lines got broken off.

Place this code in your local.settings.php file inside your site folder

// the path to the core cache file
  include_once('./includes/cache.inc');
  // the path to the memcache cache file
  include_once('./sites/all/modules/memcache/memcache.inc');
  // make MemCacheDrupal the default cache class
  $conf['cache_default_class'] = 'MemCacheDrupal';
  
  # Key Prefix: edit this for multisite use.
  $conf['memcache_key_prefix'] = "$_SERVER[db_user]";
  
  $conf['memcache_servers'] = array(
  '127.0.0.1:11211' => 'default',
  '127.0.0.1:11212' => 'menu',
  '127.0.0.1:11213' => 'filter',
  '127.0.0.1:11214' => 'form',
  '127.0.0.1:11215' => 'block',
  '127.0.0.1:11216' => 'update',
  '127.0.0.1:11217' => 'views',
  '127.0.0.1:11218' => 'content',
  '127.0.0.1:11219' => 'apachesolr',
  '127.0.0.1:11220' => 'path',
  '127.0.0.1:11221' => 'field',
  '127.0.0.1:11222' => 'rules',
  '127.0.0.1:11223' => 'token',
  '127.0.0.1:11224' => 'image',    
);
$conf['memcache_bins'] = array(
  'cache' => 'default',
  'cache_menu'   => 'menu',
  'cache_filter' => 'filter',
  'cache_form'   => 'form',
  'cache_block'  => 'block',
  'cache_update' => 'update',
  'cache_views'  => 'views',
  'cache_views_data'  => 'views',
  'cache_content'  => 'content',
  'cache_apachesolr'  => 'apachesolr',
  'cache_path'  => 'path',
  'cache_field'  => 'field',
  'cache_rules'  => 'rules',
  'cache_token'  => 'token',
  'cache_image'  => 'image',
);

And that should be basically it.

It can be convenient to do stuff as the aegir user. The default shell of the aegir user is /bin/false, so you have to make sure to use something more usable:

sudo su - aegir -s /bin/bash

GNU Screen is a super useful window manager for the console. When launching screen you will usually get errors like

Cannot open your terminal '/dev/pts/6' - please check.

You can get around this by running

script /dev/null

All in all this works nicely:

sudo su aegir -s /bin/bash -c "script -q /dev/null"

It can be convenient to put it in a little script as /usr/local/bin/suaegir

Allowing other users besides root on your system to run commands as aegir

If you wish to allow other users on your system to run commands as the aegir user using sudo, without allowing them to use sudo generally, you can add the following two lines to your /etc/sudoers file using visudo:

If you want to allow these users to use aegir without entering a password, simply change the second line to this:

If you connect to your Hostmaster server and look at ~aegir/config/server_examplecom/apache.conf, you can see that this file includes files from pre.d and post.d directories. The includes are done in the following order:

1. pre.d
2. vhost.d
3. platform.d
4. post.d

The earlier files take precendence over the later files for <VirtualHost> blocks. The latter files take precedence over earlier files for <Directory> blocks.

If we wanted to add a phpini directive that can only be set on a PHP_INI_PERDIR-for example, post_max_size-we could create a file called upload.conf and add it either to pre.d or post.d.

Here's a possible upload.conf:

If we put upload.conf in pre.d, our setting would always take precedence, because it occurs higher up the chain of include files.

If we put upload.conf in post.d, our setting would be applied only if the same setting is not defined higher up the chain by the aegir code. As Antoine states here post.d is mainly useful when migrating in sites. It would also be useful if you wanted to set a server-wide default which could be overridden on by a virtual host or platform config setting.

Related: http://community.aegirproject.org/node/73

Introduction

SSL support was significantly improved in Aegir 0.4 alpha9 and subsequent releases have further refined the SSL functionality. Here are the current steps to configure SSL support in Aegir and apply it to your web sites.

Prepare Your Server

1. Make sure port 443 is open for SSL traffic.
2. From the command line, install SSL software for your web server (e.g. on Debian/Ubuntu you can use sudo apt-get install openssl).
3. Enable SSL support (e.g. sudo a2enmod ssl). You will need to restart Apache at this point.

Enable SSL Support in Aegir

1. You have to enable SSL support in Aegir as it is off by default. Assuming the URL of your Aegir front end is aegir.example.com, browse to aegir.example.com/admin/hosting/features
2. Click on Experimental to reveal experimental features
3. Check SSL support
4. Click Save configuration

Configure Your Aegir Server

1. Click on the Servers tab
2. Click on the server that you wish to enable SSL support
3. Click Edit to change the server configuration
4. Click apache_ssl (this will reveal an additional field: SSL port, which should be already populated with 443).
5. You may also have to add an IP address to the IP addresses field. The field is used as an "IP pool" for SSL certificates: IPs are assigned to SSL sites when they are created on a first come, first served basis. (Note that there are known issues with this process, see issue #1126640 for details.)
6. Click Save - this will start various tasks beginning with a verify task on the server followed by verify tasks on all platforms that are associated with that server
7. If all goes well you will see the following changes in your Aegir file system structure: a new /var/aegir/config/ssl.d directory and a new /var/aegir/config/server_master/ssl.d directory.

The /var/aegir/config/ssl.d directory is where you will be able to manipulate SSL keys and certificates, for example by importing commercial SSL certificates or generating a new key. Do not manually edit the /var/aegir/config/server_master/ssl.d directory as changes to that directory will be overwritten when the server or site are verified.

Configure Your Aegir Site

1. You must enable SSL on your sites that are on those platforms associated with the server. Browse to aegir.example.com/hosting/c/example.com
2. Click Edit to change the site configuration
3. Choose the type of Encryption required and the Encryption key (see the explanatory notes below each option). NOTE:, Alternatively, you may specify a directory under /var/aegir/config/server_master/ssl.d where your own certificate and key is to be stored (see Apache notes below).
4. Click Save. Aegir will then generate a certificate and private key for your web site and insert these into a new VirtualHost directive in your vhost file. (This file is typically at /config/server_master/apache/vhost.d/example.com).
5. If all goes well the VirtualHost directive will now have these important elements:

Now, when you navigate to https://example.com you should see that your site is SSL enabled. It will, however, generate warnings in your browsers because it is a self-signed certificates. See below for how to use commercial certificates to remove this warning.

Notes for Apache users with Commercial Certificate File(s)

If you wish to use your own commercial certificate and key you will need to do the following:

1. Follow the directions above, using the "Generate new encryption key" option and using your site's domain name for the "New encryption key". This will create a site directory under /var/aegir/config/ssl.d/example.com. With this step, you have created a self-signed certificate, and your site is now configured to use it.
2. This generated a 2048 bit RSA key for you along with a CSR (Certificate Signing Request). If you prefer to generate your own RSA key, replace the files (openssl.key and openssl.csr) in the /var/aegir/config/ssl.d/example.com directory with your RSA key and associated CSR.
3. Copy and paste the .csr file into the form for the issuing Certificate Authority (CA) to create your certificate.
4. When your certificate has been generated, download the files from the issuing authority and place in your temporary folder on your PC. You may have more than one .crt files, in this case you have a "bundle" or what we call a "certificate chain" that you need to add in aegir (see below).
5. Transfer all the files to /var/aegir/config/ssl.d/example.com. Rename the site .crt file to openssl.crt. If you have a certificate chain, install it in openssl_chain.crt. You should have at least three files in the directory (openssl.crt, openssl.key, openssl.csr, and optionnally openssl_chain.crt).

Verify your site from aegir's frontend.

You should now be able to access your site via https:// using your commercial certificate.

Notes for Nginx users:

It is recommended to allow Aegir to create a default self-signed certificate and key first, and then replace the contents of both files (not the files itself) with your real key and certificate. Any chained certificates (bundles) should be included in the same file, directly below your own certificate - there is no need for extra files/lines like it is for Apache configuration.

Debian/Ubuntu

There is preliminary support for DNS in Aegir's backend that allows it to manage zonefiles dynamically when sites are created. There are still a few major bits missing (described in this meta-ticket) so we generally recommend people use the wildcard approach for now.

That said, while perhaps not yet ready for production use, Aegir's DNS functionality can be quite useful for local development environments. If you run Aegir locally, you're probably in the habit of hacking /etc/hosts regularly, or if you're more adventurous, you've set up a local DNS forwarder, such as dnsmasq. With Aegir's DNS feature, you can avoid this hassle, and let Aegir handle it transparently.

While Aegir supports both BIND and dnsmasq, this documentation will focus on the former. The first step is to install BIND. On Debian-based distros, this will look like:

Next we need to allow the aegir user to run rndc (the name server control utility). On recent Debian systems, this will involve editing /etc/sudoers.d/aegir, whereas on older releases this would be /etc/sudoers. Modify the aegir line to read:

Then we need to make sure the bind user has access to read Aegir's files, so we'll add bind to the aegir group:

We're now ready to activate the feature in Aegir's frontend. Go to Hosting >> Features (admin/hosting/features) and select "DNS Support" under the "Experimental" fieldset, and save the configuration.

Finally, we need to activate DNS on the Aegir server. Visit the server node (hosting/c/server_master) and click the edit tab (which should bring you to node/2/edit). Then under "DNS Service", select "Bind" and then save.

This should trigger a verify of the server (and then probably the platforms on it too), after which Aegir will be managing your local BIND server, and it's zonefile(s). If you have installed Aegir in /var/aegir, put this line in the file /etc/bind/named.conf.local:

Aegir is now a full-fledged DNS server that you can use to resolve addresses locally. This can be accomplished on Debian-like systems by editing /etc/resolv.conf to include "nameserver 127.0.0.1". If you're running Aegir in a VM, you can just point this to the address of the virtual machine.

CentOS/RHEL

You need to have a working DNS/bind/named server for AEgir to take control of. What follows is rough documentation to reach that stage. It is currently UNTESTED in a production environment.

Additional Notes

Additional documentation can be found in the Provision DNS module: http://drupalcode.org/project/provision.git/blob/HEAD:/dns/NOTES.

Why run a wildcard DNS server?

A wildcard DNS setup lets you automatically have subdomains for a given domain. For example, say you own the domain widgets.com and you want to setup an unlimited number of subdomains like dev.widgets.com, test.widgets.com, customers.widgets.com, etc.... Typically you would have to set these all up individually. A wildcard DNS can let you bypass a lot of configuration. In a development environment it can let you setup any number of test/development sites very quickly and easily. Drupal developers in particular can leverage Drupal's multisite installation feature to setup lots of sites for development or production very quickly.

Creating a DNS wildcard with hosted DNS

Many of the popular domain registration companies allow you to manage your DNS records through a web interface. If you have access to such a facility, adding a DNS wildcard is easy: you just create an A record called '*', pointing to your server's IP address.

For convenience, here are instructions for some popular hosting providers: GoDaddy, hosting.com (which uses the popular cPanel system) and 123-reg.co.uk. It is likely that your host will have a similar system to one of these.

How to configure DNS wildcards on your own server

Ubuntu/Debian

OS X Snow Leopard

Setting up a Wildcard DNS configuration on Ubuntu/Debian

Overview of Steps on Ubuntu/Debian

I. Install Bind 9 II. Add a zone to /etc/bind/named.conf.local. Check the syntax of named.conf.local with named-checkconf III. Add a zone file & Check the syntax of your zone files for errors with named-checkzone IV. Edit /etc/resolv.conf V. Start up Bind VI. Check setup with dig VII. Troubleshoot if needed

I. Install Bind 9

II. Edit /etc/bind/named.conf.local to add a zone.

We need to add a zone to /etc/bind/named.conf.local. A zone is a record for a domain. In this case we'll use a fake domain for development purposes. Our domain can be called anything. Let's call it mydev. (I was really tempted to call the domain bullshit - would have been great for screencasts). So at a shell prompt do:

Add this to the end of the file:

Save the file with Control - O and enter. Exit from nano with Control X.

Check the syntax of the file as follows:

If the file is syntactically correct the shell prompt will return nothing. If there is an error - make sure you did not miss a quote or semicolon and recheck the file.

III. Add a zone file & Check the syntax of your zone files for errors with named-checkzone

Add this to the file replacing 10.0.2.15 with your IP address

Save the file with Control - O and enter. Exit from nano with Control X. Check the syntax with the following:

If the syntax of the file is correct you should see something like:

If not review the file for errors. In particular whenever referring to a domain, it must end in a . , thus mydev.

IV. Edit /etc/resolv.conf

resolv.conf tells applications like browsers where to look for DNS info. By default it may have your ISPs info. We need to tell it to check our local DNS server first then the ISP.

Add/change the following

If you are receiving an IP address via DHCP then you need to edit /etc/dhcp3/dhclient.conf too.

Look for the lines send host-name, supersede domain-name, and prepend domain-name-servers and set them as follows (and make sure they are uncommented ) :

Whew! Almost there.

V. Start up Bind (or Restart)

VI. Check setup with ping and dig

If ping works then you should be good to go. If not try dig and part V. To stop the ping action press [Ctrl]+[c].

VII. Troubleshoot

I found the section on Troubleshooting Bind in O'Reilly's "Linux System Administration" helpful. Fortunately, this part of the book is viewable on Google books as of this writing (10/29/2009): http://books.google.com/books?id=-jYe2k1p5tIC&lpg=PP1&dq=Linux%20System%... . See page 66

Getting a Wildcard DNS running on OSX Snow Leopard

Note: it took me several tries to get this working. If you are new to DNS be patient with yourself - you'll get it, but may take a few tries.

In this example I will concentrate on setting a development environment with OSX using wildcard DNS

Overview of Steps

I. Edit /etc/named.conf to add a zone.
II. Add a zone file at /var/named/ III. Check the syntax of named.conf and your zone files for errors IV. Edit /etc/resolv.conf V. Set your computers network settings to use 127.0.0.1 as a name server VI. Start up Bind VII. Check setup with dig VIII. Reboot if needed

Before we start a few more notes.

I tend to use the nano text editor to edit Unix configuration files you could use Emacs, VI, Textmate, BBEdit or the editor of your choice.

Backup all these files we are editing so you can start over if you mess up. I didn't do this and it added more time to the project. For example to backup /etc/named.conf do:

Last, most of the files we need to edit are owned by root so you will need to use sudo to edit these files. If you get tired of typing sudo you can become root by doing this:

Be careful when working as root or using sudo. You can mess up your system so make sure to backup. All example here are run as root.

I. Edit /etc/named.conf to add a zone

We need to edit named.conf to add our zone.

I called my zone vmdev so I added this to named.conf

I added this right before the zone 0.0.127.inaddr.apra and saved the file. So we told Bind to look in /var/named/db.vmdev for this zone.

II. Add a zone file at /var/named/

You can just copy this but be sure to change 192.168.0.199 to you Mac's IP address

III. Check the syntax of named.conf and your zone files for errors

Run this to check your named.conf file:

If it returns nothing, your named.conf file is at least syntactically correct. If there is an error, then well you have to diagnose and fix the error.

Now run this to check your zone file:

It should return something like this:

If there are errors then diagnose and fix them.

IV. Edit /etc/resolv.conf

V. Set your computers network settings to use 127.0.0.1 as a name server

Do this at System Preferences -> Network. You may want to use your ISPs Name server as the second name server

VI. Start up Bind

The -w option tells OSX to enable Bind at startup

VII. Check setup with dig

Should return something like this:

The key here is status NOERROR; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45640 If you get an error then check previous steps.

Now try:

should return:

but if it does not got to VIII.

VIII. Reboot if needed.

I needed to reboot to get everything to take. Whoila! Have a cookie or something.

Because Aegir is capable of installing, deploying and moving your sites around, it has to manage the various configuration files that keep your sites running.

These configurations include the standard Drupal settings.php for a site, .htaccess overrides for your HTTP server, as well as HTTP 'VirtualHost' or 'vhost' configuration files that tell your HTTP server where your site is.

A caveat of this system is that Aegir regularly re-generates these configuration files to apply new changes that have been made to the sites or platforms, as well as doing so as a safety mechanism to ensure these files are 'sane'.

For example, if you make a modification to a site vhost or settings.php and it breaks your site, running a 'Verify' task on the site should restore the file back to how it was.

However, sometimes it is necessary to add custom configuration or overrides to these files, and you can't do that if Aegir is regularly wiping your changes.

Fortunately, Aegir provides a series of hooks and inclusions for overriding or injecting customisations into these files safely and persistently.

This chapter shows you how each of these hooks or inclusions work.

Changing maximum filesize upload is common when setting up sites in Drupal. As described in Overriding site-specific PHP values you can change this value for each site created with Ægir by adding a .drush.inc file in /var/aegir/.drush directory. But wouldn´t it be nice to be able to do it side-wide?

Here below is the same code from ergonlogic demonstrating how to create a domainname.drush.inc file using the domain name as a condition before the code injection.

Apparently you can do this side-wide by omitting the IF statement and create a file with a .drush.inc extension.

For instance, you could create a file called global_settings.drush.inc and put in the following code:

* the closing ?> syntax should not be used

Make sure you Verify your site after you create the file. Then scroll through the log and find the message you added in the code:

Overriding PHP file size values. See .drush/global_settings.drush.inc

Using a .htaccess with an Allow Override all directive in Apache can be a major performance killer, because it requires Apache to stat each subdirectory of the codebase looking for overrides in .htaccess.

As a result, Aegir disables the reading of the Drupal .htaccess in the runtime environment.

This does not mean that the .htaccess is not needed. Instead, when you run the Verify task against a Platform, the .htaccess is studied by Aegir and its contents are copied into the platform-wide Apache vhost configuration, typically located in /var/aegir/config/server_master/apache/platform.d

Need to make a modification to the .htaccess? Simple: you can simply edit it in-place as you normally would, but you must re-Verify the platform in Aegir afterward, in order for those new or modified settings to be 'loaded in' to the platform vhost file.

The end result is improved performance for your sites, without losing any functionality, as you can still customise the .htaccess to your liking.

Aegir provides some hook functions in its API, one of which allows you to inject extra configuration snippets into your Apache vhost files for sites.

When would I want to do this?

A good example for this is when you may need to inject a custom Rewrite rule that goes beyond what the Aegir Aliases 'redirection' feature provides.

Or, perhaps you have to inject some htpasswd mod_auth password protection for your site, or perhaps a CustomLog definition. The http_basic_auth module uses this. (code)

Typically you'd just add what you need to the vhost file, but the problem is that Aegir manages these vhosts, and on every Verify task, will rewrite the config from a template, blowing away your changes in the process. Ouch!

Fortunately there is a very easy and elegant solution to this problem to save your configurations persistently across Verify tasks and the like, by means of invoking the Provision hook provision_apache_vhost_config(), or, if you are using Nginx, provision_nginx_vhost_config() (and just replace the below examples with 'nginx' instead of 'apache' where necessary). Below "mig5" is just an example, you can replace this with anything as drush looks for all *.drush.inc files in ~aegir/.drush

A simple example

In this example I'll inject a simple 'ErrorLog' apache definition into a vhost to save the site error logs to a file.

1. Create a file in ~aegir/.drush called mig5.drush.inc. (Choose <any> prefix for the file name <any>.drush.inc).
2. Add this snippet of PHP to the file:
   
<?php
  function mig5_provision_apache_vhost_config($uri, $data) {
    return "ErrorLog /var/aegir/" . $uri . ".error.log";
  }
?>
   (Choose <any> prefix for the function name <any>_provision_apache_vhost_config).
3. drush cache-clear drush
4. Install a site or verify an existing one

Check your site's vhost config file (in /var/aegir/config/server_master/apache/vhost.d/) and you'll see the line has been injected into the '#Extra configuration' area of the vhost

It's that simple! You can see that via the hook, we pass $uri and the drush data to the function, allowing me to abstract the site url so that each site will get its own error log. You could do extra PHP conditionals to ensure certain data only gets inserted into certain sites of a specific name.

To inject multiple lines instead of one, use an array, i.e

The key point of this is that the file ~aegir/.drush/mig5.drush.inc file will never be touched by Aegir, so you can rest assured your changes will be respected.

If you want to only inject code into a specific site, wrap your code with an if statement, i.e

A more advanced example

Managing multiple versions of a production site can be a tricky proposition, even in Aegir. This is particularly true when you want to use the same canonical domain name to allow users to access one such site of your choosing at any given moment. For instance, in Aegir I might have a site named www.example.com (with an alias of example.com,) and a couple of alternates with the site names of test1.example.com and test2.example.com. If I suddenly decide that I want my primary domain to access test1.example.com, Aegir forces me into a tedious process that ultimately results in site downtime – I have to delete or migrate the existing www.example.com site to a new unused site name (or clone it to an unused site name, and delete the original) and then I have to migrate test1.example.com to the vacated site name www.example.com. Alternatively, I could add the www.example.com and example.com aliases to test1.example.com but then my users would just be redirected to test1.example.com.

Given the way Aegir manages aliases, it would actually be easier to make this switch if the desired address was never used as a site name to begin with. Instead of having the site name www.example.com alongside of the two other test sites, we might have live.example.com, with in-use aliases of www.example.com and example.com, and a rewrite instruction in the vhost that rewrites live.example.com to www.example.com. If we could easily change the rewrite instructions in that vhost, all we would need to do is move the aliases from one site to the next in the GUI (with the requisite verify tasks executed on each site in question) in order to quickly change which site was being loaded at any given moment by www.example.com. For example, if I wanted to move to test2.example.com, I would remove the www.example.com and example.com aliases from live.example.com and verify, add those aliases to test2.example.com and verify, and change my vhost so that test2 .example.com is rewritten as www.example.com. Since as previously mentioned Aegir overwrites changes to the vhost during the verify process, the solution is to use the provision_apache_vhost_config hook in a drush.inc file to selectively add the rewrite information to the vhost when verifying the site that we want our canonical domain to refer to.

When we want to switch again, say to test1.example.com, We could update the code to look for the test1.example.com uri.

Sometimes it is useful to override certain PHP values on a per-site basis, but changes to php.ini are generally server-wide. Depending on the value you want to override, a couple of options present themselves.

First, let's consider where PHP values can be changed. http://www.php.net/manual/en/ini.list.php lists php.ini directives, and under the "Changeable" column, indicates where a configuration setting may be set. If your value shows either PHP_INI_USER or PHP_INI_ALL, then the easiest way to change this value would be using ini_set() in a local.settings.php:

On the other hand, if the changes mode is either PHP_INI_PERDIR or PHP_INI_SYSTEM, php_ini() won't work. In this case, the solution is to inject the value into the vhost. Since vhosts are managed by Aegir, manually adding an override to /var/aegir/config/server_master/apache/vhost.d/<uri> would get blown away the next time that the site is verified.

As described in Injecting into site vhosts, we can inject values into vhosts using a Drush hook. For example, to raise the file upload size limit on http://ergonlogic.com, I added the following code in /var/aegir/.drush/ergonlogiccom.drush.inc:

This results in the insertion of the following lines into /var/aegir/config/server_master/apache/vhost.d/ergonlogic.com:

Also, in the verify task log I get the following informative message:

One challenge this technique may present is inspecting the values of the parameters passed into this function. It appears that the Hostmaster site doesn't get bootstrapped, and so common debugging tools (such as devel.module's dd()) aren't available. However drush_log() is, and when called, will push arbitrary messages back into your Aegir site's verify task log.

So sticking the following into the function above can help:

Introduction

Every web site in an Aegir environment has a Drupal configuration file settings.php in /sites/example.com directory. Web administrators often need to make changes to this file; however, the Aegir system also manages this file and any manual customizations will be lost when a site is verified.

Fortunately, there are two mechanisms to ensure that your customizations can be preserved. If you look in the bottom of an Aegir settings.php file you will see references to two files local.settings.php and global.inc.

If these files exist they are loaded at run time by Drupal. As you can probably surmise from the paths to these files, local.settings.php is for site-specific customizations and global.inc is for Aegir-wide customization.

Let's look at these files in more detail. We'll use customization of user session cookies as an example. If you look at the settings.php file generated by Aegir you see that it sets more conservative php settings for cookies (@ini_set('session.cookie_lifetime',  0); i.e. cookies expire immediately) than are in the default.settings.php packaged with Drupal (@ini_set('session.cookie_lifetime',  2000000); i.e. 2 million seconds, which is just over 23 days).

Site-specific Customization

The local.settings.php file by default does not exist in a new Aegir site installation so you have to create it. Continuing with our example of user session cookies, let's override the Aegir default.

Note that because local.settings.php is included after the variables are set in the main settings.php it's customizations takes precedence.

Now, whenever you clone a site or migrate it between platforms, Aegir moves a copy of local.settings.php as well.

Using drush_hook_provision_drupal_config

You can also use the API to add module-specific site configurations with hook_provision_drupal_config

For example it could look like this

That is used to add the site settings added by the UI in the hosting backend implemented in https://github.com/EugenMayer/hosting_site_settings

Aegir-wide Customization

In some situations you may want to implement the same configuration settings on all your Aegir sites. This is where global.inc comes in. Note that global.inc is now included in settings.php before local.settings.php, so that Aegir system administrators no longer retain the ability to override configuration changes in local.settings.php, but instead it is possible to override global settings per site (read why this has been changed: http://drupal.org/node/1044938 - note: this change is available since 0.4-rc1 release).

For example, say the system administrator wanted to limit users' session lifetimes to a maximum of one day they could create a global.inc as follows:

You can even set more granular policy within global.inc (however it makes more sense to keep site-specific overrides in the local.settings.php):

If you are using Aegir to manage multiple remote webservers, you will need to run the Verify task on the webserver in order to push global.inc to the remote machine.

Note: on some Aegir installations (e.g. alpha14) the permissions on /var/aegir/config are insufficent for Apache to properly include the global.inc file. If you find that the configuration settings in global.inc are being ignored, change the permissions with chmod a+x /var/aegir/config. This only has to be applied to the config directory as the sub directory and file permissions should be correct.

If you have other configuration examples, create a wiki and reference it here.

The drushrc.php file can be changed in two ways:

1. by changing the templates used for the Drushrc context (see the hook_provision_config_load_templates() hook)
2. by creating a local.drushrc.php file in the site-specific folder (e.g. sites/example.com/local.drushrc.php

This is a stub. You can put more stuff here and it would be prettier.

1. Aegir 7.x-3.x

Contrib modules for the upcomming version are listed on a sub page

2. Extensions to Hostmaster (frontend)

2.1. Verified in Aegir 2.x

Aegir HTTP basic authentication (now included in hosting_tasks_extra)

This module allows you to protect sites with passwords.

Hosting server titles

This is a really simple module that allows you to change the names of servers that are displayed in lists in the Hosting frontend when choosing what server to use for something.

Hosting site Backup Manager

Add a backups tab to a site node. It extends the default backup functionality with the ability to download a backup over HTTP.

Now also includes the functionality from Hosting backup garbage collection and Hosting backup queue

Hosting site Git

This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.

Hosting platform Git

Provides integration with Aegir that allows provisioning a platform from a git repository (also allows specifying the branch or tag name)

Hosting tasks extra

This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.

Now also includes the functionality from Aegir HTTP basic authentication

Hosting database login

Provides a link to login into a PHPMyAdmin instance (or other db management web software) configured for the current site database server. Requires dblogin to be installed on the site.

Aegir Piwik integration

Makes it easy to connect an Aegir instance and a Piwik instance. When you create a site in Aegir, the site is registered in Piwik. When you verify the site in Aegir, new site aliases (if any) get added to Piwik. Also adds system variables to settings.php in order to integrate properly with the Piwik module.

Note: There's a good chance that other Aegir 1.x contrib modules also work on 2.x

2.2. Aegir 1.x

Aegir Ubercart Integration

This module allows you to sell an Ægir client, with quotas and access to specific platforms, through Ubercart . It automates a number of set-up tasks from the moment the order ...

Aegir HTTP basic authentication

This module allows you to protect sites with passwords.

Aegir HTTP basic LDAP authentication

A fork of the above to allow you to specify an LDAP server and Require (ldap filter) string for authentication of 'sites'.

Aegir feeds

This is a very basic module that provides feeds integration with Aegir . Features Provides mapping targets for Aegir site nodes, allowing you to create new sites from a feed or file upload.

Aegir Piwik integration

Makes it easy to connect an Aegir instance and a Piwik instance. When you create a site in Aegir, the site is registered in Piwik. When you verify the site in Aegir, new site aliases (if any) get added to Piwik. Also adds system variables to settings.php in order to integrate properly with the Piwik module.

Aegir Rules

Aegir Rules provides Rules integration for the Aegir.

Aegir Drush Make Platform working-copy

This feature allows you to specify 'working-copy' when adding a platform using a Drush makefile. Working-copy means your .git subdirectories are preserved in the platform, which may be useful for development purposes.

Aegir Subfolders

Support for 'example.com/subsite1' style sites in Aegir (experimental as of July 2012)

Hosting Platform Pathauto

This is an add-on module for the Aegir hosting system: http://aegirproject.org/ This module simply makes a little time-saving tweak when creating a new platform: ...

Hosting queue runner

This module allows the Hosting tasks queue for the Aegir project to be 'daemonized' so that tasks are run as soon as possible instead of waiting for a cron run. This makes Aegir appear much more responsive. This module was inspired by the Waiting queue project. Requirements You will need ...

Hosting Reinstall

The Hosting Reinstall module provides a frontend for the Provision Reinstall module so that you can reinstall any sites via Hostmaster (Aegir).

Hosting backup garbage collection

This is an add-on module for the Aegir hosting system: http://aegirproject.org Introduction This module allows you to specify how long Aegir should retain backups for. You can specify, ...

Hosting backup queue

This is a simple module for the Aegir project that allows scheduling of backups. E.g. backup sites every hour, day or month. The module allows for the general settings to be applied to all sites, ...

Hosting site Backup Manager

Add a backups tab to a site node. It extends the default backup functionality with the ability to download a backup over HTTP.

Hosting site Git

This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.

Hosting upload

This module allows administrators to add platforms to aegir and extend platforms and sites with modules, themes, libraries without the need of ssh access.

Hosting Profile Roles

Hosting Profile Roles extends Aegir to enable more control over what role(s) a client is assigned during site install. Since, by default, Aegir client's are given UID1 during site installs, this module also allows UID1 to be assigned to another user.

Hosting server titles

This is a really simple module that allows you to change the names of servers that are displayed in lists in the Hosting frontend when choosing what server to use for something.

Services API Support

Integrates the Services API framework into the Hostmaster suite of tools. It allows Aegir managers the ability to build Services API connections over any of the available servers for Services API.

Hosting Advanced Cron

This module allows an administrator to set the default cron interval globally and to override that interval for individual sites.

Hosting Features

This module provides a Task to Hostmaster that provides integration with a site's Features. (In beta as of June 1,2012)

Hosting Notifications

This is a module that integrates with the Notifications framework. This allows you to receive notifications about task execution in various formats.

Hosting Task Garbage Collection (merged into Aegir 7.x-3.x)

This module will delete rows in the {hosting_task} and {hosting_task_log} tables. (In alpha as of June 1,2012)

Hosting Remote Import

This Drupal module provides a UI for fetching sites from remote Aegir servers. (In development as of June 1, 2012)

Hosting Varnish

Allows the Varnish cache for Aegir-managed sites to be purged via the user interface. (In alpha as of June 1,2012)

Hosting CivicCRM Cron

This module provides the ability to trigger CiviCRM cron jobs on sites hosted on the Aegir Hosting System.

Hosting tasks extra

This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.

Hosting Filemanager

This module provides a web file browser for viewing files under sites and platforms. (In beta as of August 29,2012)

Hosting Signup Settings

This module is dependent on hosting_signup module in the hosting package of Aegir. It allows the Signup form at /hosting/signup to be configured. (Sandbox project, October 22, 2012)

Hosting Client Roles

Client Roles module integrates Aegir user roles into hosting_products as product feature. With this module you can assign Aegir user roles to Ubercart products, and sell them as hosting products. This module is part of the "Hosting & Ubercart Integration" package, and depends on uc_hosting and uc_hosting_products modules

Hosting Solr

Works with Provision Solr to give Aegir the ability to manage Solr servers and and gives sites a Solr database.

3. Extensions to Provision (backend)

Provision boost support

the Aegir project and therefore depends on the provision backend and expects the Aegir environment to be functional. Quickstart Download the provision_boost code into your /var/aegir/.drush/ Download ...

Provision ACL support

This drush command aims to provide preliminary ACL support for the Aegir backend . The objective here is to allow administrators to set more granular access policies on files managed by Aegir and resolve more hairy access permissions requirements than what is usually allowed by the Aegir core. ...

Project Status

Project status is an extension for drush which takes a list of drupal platforms as arguments and returns a list of modules which are not in use by any site within these platforms. It is also optionally able to list modules which are use, by platform and site.

Provision CiviCRM

A Drush module to automatically setup Drupal instances with the CiviCRM Constituent Relationship Management solution. It is specifically aimed towards the Aegir project.

Provision Backup Platform

This hooks into deleting a platform and creates a simple tar backup.

kPlatforms

Koumbit's "standard Drupal platforms", a set of makefiles to build and maintain development and production platforms easily.

Provision Git

Provision Git is a backend Drush module. It provides 4 drush commands. (In alpha as of June 1,2012)

Provision Merge Log

This Aegir extension allows you to automate fetching and merging logs from multiple servers part of the same cluster in a single logfile for later analysis with various analysis tools

Provision CDN

This is a simple module and drush script for that allows you to enable CDN support per site in Aegir. (In development as of June 1,2012)

Provision profile settings

Incredibly simple extension for provision that allows install profiles to easily inject settings into settings.php for provisioned sites.

Provision New Relic

This is a simple extension that exposes all sites in New Relic, so you can monitor each sites performance.

Provision Solr

Works with Hosting Solr to give Aegir the ability to manage Solr servers and and gives sites a Solr database.

4. Themes

• Koumbit has provided it's custom Aegir theme as a sample subtheme of Eldir.
• Saeven, a clone of the core theme Seven, based on Omega 4 or 5, clean, simple and responsive.

5. Puppet modules and Chef cookbooks

Those projects allow you to manage Aegir instance(s) through Puppet. Chef recipes also welcome, we don't discriminate.

• Aegir Puppet module
• Drush Puppet module
• Aegir Chef cookbook

6. Others

Aegir up

Aegir-up is a Drush extension that deploys a local instance of the Aegir Hosting System atop Vagrant and Virtualbox, for development and testing purposes.

DevShop

DevShop is a Drupal Development Environment Manager built on Aegir.

DevShop creates Aegir platforms and sites automatically from Git URLs. It tracks multiple Projects and allows multiple environments to be created for each Project, such as dev, test, and live. It provides tools to Pull Code, Sync Data, Commit Features, and Run Tests on these environments, and provides a dashboard with useful links and information for developers.

7. Your module here?

Developers: Please add your contributed module here, and feel free to put install instructions or other documentation in a child page. If you are looking for ways of extending Aegir or have already done so and want to document the internals of your module, head over to the extending Aegir page.

Aegir Services integrates the Services API framework into the Hostmaster suite of tools. It allows Aegir managers the ability to build Services API connections over any of the available servers for Services API.

Using Services 3.x we are aiming at full CRUD support for clients, tasks and sites, as well as read access to profiles, platforms, and eventually, packages.

The 2.x branch of Aegir Ubercart Integration will take advantage of this support to provide a remote storefront functionality.

Please report issues and request support via the issue queue on Drupal.org

Installation

1. Install Aegir
2. Install the dev version of services 3.x, including at least one server and, ideally, an authentication server.
3. Install this module.
4. Configure your endpoint, it's resources and authentication, normally.
5. Optionally, test your rest server using the included bash script, hosting_services.rest_test.sh

1. Extensions to Hostmaster (frontend)

Hosting Git

This is a simple module for the Aegir project that adds a 'Git pull', 'Git checkout' and 'Git clone' task for sites.

The successor of Hosting site Git & Hosting platform Git

Hosting Logs

This is a simple module for the Aegir project that adds a 'Logs' tab to sites and platforms. Showing Apache error, Git commit and watchdog logs.

Hosting Platform Pathauto

This is an add-on module for the Aegir hosting system: http://aegirproject.org/ This module simply makes a little time-saving tweak when creating a new platform: ...

Hosting site make

Allows a site to have its modules built from a makefile in the sites directory.

Hosting tasks extra

This module extends Aegir hostmaster (and drush/provision) with some additional tasks. Such as: cache-clear and registry-rebuild.

Now also includes the functionality from Aegir HTTP basic authentication

2. Extensions to Provision (backend)

Starting from Aegir 7.x-3.x the Drush component can be included in a 'drush' directory in the same git repository as the hosting module.

Therefore this list will be shorter then for previous versions.

Provision STS

Adds the Strict Transport Security header to hosts that require SSL.

3. Themes

• Saeven
• Eldir. (The default theme)

4. Puppet modules and Chef cookbooks

Those projects allow you to manage Aegir instance(s) through Puppet. Chef recipes also welcome, we don't discriminate.

• Aegir Puppet module
• Drush Puppet module
• Aegir3 Chef Cookbook

5. Others

Aegir up

Aegir-up is a Drush extension that deploys a local instance of the Aegir Hosting System atop Vagrant and Virtualbox, for development and testing purposes.

DevShop

DevShop is a Drupal Development Environment Manager built on Aegir.

DevShop creates Aegir platforms and sites automatically from Git URLs. It tracks multiple Projects and allows multiple environments to be created for each Project, such as dev, test, and live. It provides tools to Pull Code, Sync Data, Commit Features, and Run Tests on these environments, and provides a dashboard with useful links and information for developers.

Aegir Pathologic Files

A tiny Drupal Module to simplify file paths in content. This helps prevent broken images when the site directory name changes. Requires an apache rewrite rule to point /files to /sites/example.com/files, which Aegir provides by default.

This module is not for hostmaster, but for the sites hosted under Aegir.

6. Your module here?

Developers: Please add your contributed module here, and feel free to put install instructions or other documentation in a child page. If you are looking for ways of extending Aegir or have already done so and want to document the internals of your module, head over to the extending Aegir page.

Homepage: http://drupal.org/project/devshop

DevShop Provision

Drupal development infrastructure made easy.

This module provides the backend commands needed to deploy and manage sites using the DevShop git and features based development workflow.

About DevShop

The goals of DevShop are...

1. to simplify management of multiple environments for multiple Drupal projects.
2. to provide web-based tools that streamline the Drupal site building workflow.
3. to provide a common, open-source infrastructure for Drupal development shops.

Installation

To install devshop_provision, simply use drush:

$ drush dl devshop_provision

Or, you can download the source code to any available drush commands folder, such as /usr/share/drush or ~/.drush

Usage

DevShop Provision works by providing a new Provision class called "Project".

To start, you must have a Project drush alias. Using the DevShop+Hostmaster system will make this much easier, but if you wish to only use the backend, you can create a project alias with provision-save, for project NAME:

Commands

DevShop contains a set of features that make Drupal site building within a version-controlled code workflow quick and easy.

Currently you must create new platforms within a project in hostmaster. Once created, you can execute the following commands:

1. Pull Code

This task runs on the dev platform for this project. It runs git pull, and optionally runs new updates, reverts features, and clears caches. It is used to keep the dev server up to date on every commit via the devshop_pull module, and can be used as the deployment task.

• Git Pull the code for your site's platform.
• Then, all optionally:
  • Run update.php.
  • Revert all Features modules
  • Clear caches

2. Commit Features

This task integrates with Features.module to make it very easy to commit your changes to your features.

• Calls drush features-update-all
• Commits the result, with a part automated and part customized commit message.
• (Optionally) pushes the commits.
• (Optionally) force-reverts after a commit.

3. Sync Content

This task makes it easy to syncronize the database and files down from other environments within the project.

WARNING: This will DESTROY the destination site's database!

This task:

• (optionally) Pulls code
• Drops the @destination database.
• Creates an SQL dump from @source.
• Copies the SQL dump to the local system (if @source is a remote).
• Imports the SQL dump into @destination database.
• (optionally) Runs update.php.
• (optionally) Runs features-revert-all.
• (optionally) Clears all caches.

$ drush --filter=devshop_provision

$ drush provision-devshop-commit --help

$ drush provision-devshop-pull --help

$ drush provision-devshop-sync --help

$ drush provision-devshop-test --help

This page documents the Provision ACL extension to Aegir which allows more granular access control over your sites files and directories.

1. Install instructions

First, you'll need a running Aegir install (1.0-rc3 or later), see http://community.aegirproject.org/installing. Most (if not all) of these commands will have to be run as root (or using sudo, etc.)

1.1. Download and install provision

drush dl provisionacl-6.x

1.2. Enable ACLs on your filesystem

mount -o remount,acl /

Here we assume everything is under the root (/) filesystem here, otherwise run this command for every filesystem Aegir will work on (e.g. /srv, /var or /home).

You also need to edit your /etc/fstab for this configuration to survive reboots.

1.3. Install ACL support package

apt-get install acl

1.4. Create a UNIX group

In this case we choose a group called "devs" but you can choose another name.

groupadd devs

1.5. Add users to the group

Add one or more UNIX users that you want to give access to that group. For an existing user (socrates32), this would look like:

usermod -a -G devs socrates32

For a new user (ergonlogic), this would look like:

useradd -G devs ergonlogic

1.6. Create a client

Create a client (should be called "devs" for this example) in the frontend at /node/add/client.

1.7. Create a site

Create a site for the client in the frontend at /node/add/site.

2. What it does

When the site is installed, members of the "devs" group will be able to write to the sites' directories (e.g. upload files and modules) and run drush commands on the site (yes, including site aliases, although see caveats below).

This works also for existing sites; make sure you create a group matching the internal name of the existing client and reverify the site.

3. LDAP integration

Provisionacl supports LDAP groups as well. Ensure that an LDAP client is running and that the 'aegir' user can see the LDAP-provided groups:

getent groups

You may need to restart the Name Service Cache Daemon (nscd):

/etc/init.d/nscd restart

4. API - how to add ACL support to your Aegir extension

To change ACLs on files, you should use something like this:

if (function_exists('provisionacl_set_acl')) {
    provisionacl_files_acls(d()->site_path . '/mysettings.php');
}

You can optionnally pass a group as an argument, but it will guess that from the client name of the site. Also note that this will raise a drush error if setfacl fails, but just set a warning if the group doesn't exist.

5. Caveats (ie. what it does not)

Giving shell access to users in Aegir is still insecure, see this upstream issue: #762138.

We aim to refactor this into the Aegir core in 2.x, but in the meantime this should provide a good workaround for the limitations of the existing permission system.

This will only work in 1.0 and above, as it needs the "client_name" field to be populated.

You will need to change your $HOME variable for aliases to work, because of this bug in drush: #1104438. Example:

See also this post for context and design.

6. Debugging

If for some reason you have lost the ACLs on the directories and you need to restore them, use the following commands, which is basically what the module does:

7. Notes

This page is the reference documentation for the Provision ACL module page on Drupal.org - keep this in mind when editing please.

This module provides several features to ubercart products on an Aegir platform.

Requirements

• Hostmaster 6.x-1.0 or higher
• Ubercart 6.x-2.4 or higher
• Patience

Installation

1. Download the module and its dependencies into sites/all/modules on your hostmaster platform (drush @hostmaster dl uc_hosting ubercart should work)
2. Activate the clients feature for hostmaster (admin/hosting/features)
3. Enable the module uc_hosting_products. This should enable all dependencies (drush @hostmaster en uc_hosting_products)

Creating your first site product

N.B. A video introduction covering configuration is available from the DrupalCon London session Aegir-based Business Models (starting at 26:30).

1. Create a product. Make sure it is not shippable. (node/add/product)
2. Edit the product. Go to the features tab.
3. Choose the "Create a site and adjust quotas accordingly" feature. It should use the sku of your product automatically.
4. For single-site products, make sure, on the product feature form, to check off "Force clients to create their site on purchase" if you want to be sure people create their site.
5. Now all you need is a drupal platform that any user can create sites on and you are ready to go. (node/add/platform)

Creating more complex offerings using product kits

The method we recommend for creating complex offerings, including multiple site quotas and access to restricted platforms, is via Ubercart product kits.

1. Enable uc_product_kit.
2. Create one or more uc_hosting products.
3. Create a product kit with these products in them in the appropriate quantities.

Support

You can report bugs, request support and propose patches via our issue queue on Drupal.org.

Development

If you would like to use uc_hosting to develop your own Aegir dependant product features, be sure to read the relevant documentation.

Check out the roadmap.

Other Resources

• Aegir community site
• Ubercart Documentation
• #aegir IRC channel on freenode

provision_civicrm helps to manage the installation and upgrades of CiviCRM in Drupal sites managed with Aegir.

CiviCRM is not a typical Drupal module. It is a third-party system that integrates with Drupal, WordPress and Joomla for its front-end and permission system. CiviCRM has its own installer, upgrade system and cron. The "provision_civicrm" module helps to manage any CiviCRM+Drupal site as any other Drupal site managed in Aegir.

Requirements

• Aegir 2.x or 3.x
• hosting_civicrm must be installed in the hostmaster front-end (~/hostmaster-x-y/sites/example.org/modules).

Installation

Download the provision_civicrm module as the "aegir" user:

drush dl provision_civicrm-6.x-2.x

This should install the module in ~/.drush/provision_civicrm.

NB: provision_civicrm 6.x-2.x is a Drush module, and works with both Aegir 2 and Aegir 3. However, for Aegir 3, you must use the 7.x-3.x branch of the hosting_civicrm module.

Creating a CiviCRM platform

provision_civicrm assumes that if you have a CiviCRM installation in your sites/all/modules/civicrm, then this is a CiviCRM platform and CiviCRM should be installed automatically.

(A cleaner way, in theory, would be to have an install profile and let the user configure it through the front-end, patches welcome! -- however, you do not want any site to simply enable CiviCRM from the Drupal admin/build/modules, so having dedicated platforms for CiviCRM makes sense in any case.)

A sample drush makefile is provided with provision_civicrm.

Assuming you are running as the user 'aegir', whose $HOME is /var/aegir/, save this in a file such as ~/makefiles/drupal-6.22-civicrm-3.3.4.make

Then create the new platform:

mkdir ~/platforms/drupal-6.22-civicrm-3.3.4
cd ~/platforms/drupal-6.22-civicrm-3.3.4
drush make ~/makefiles/drupal-6.22-civicrm-3.3.4.make

You can now add your platform in the Aegir front-end and then create new sites.

Sometimes you will need to administer your server beyond what you can do as the Aegir user.

Examples include installing additional software or editing PHP configuration.

It's not recommended to use the root user for administering things as it can be easy to make mistakes that could break your system.

Instead of running as root, it is recommended to create a user for yourself that has sudo privileges.

Create a user with SUDO privileges

In ubuntu/debian systems, creating a user and giving them the ability to run commands with sudo is easy with the adduser command.

Think of a username and password for yourself, then run the following commands as root:

root@aegir:~# adduser bob
Adding user `bob' ...
Adding new group `bob' (1001) ...
Adding new user `bob' (1001) with group `bob' ...
Creating home directory `/home/bob' ...
Copying files from `/etc/skel' ...
Enter new UNIX password: 
Retype new UNIX password: 
passwd: password updated successfully
Changing the user information for bob
Enter the new value, or press ENTER for the default
    Full Name []: Bob
    Room Number []: 
    Work Phone []: 
    Home Phone []: 
    Other []: 
Is the information correct? [Y/n] y
root@aegir:~#

Then add them to the "sudo" user group:

root@aegir:~# adduser bob sudo
Adding user `bob' to group `sudo' ...
Adding user bob to group sudo
Done.
root@aegir:~# 

Then you can switch to the "bob" user using sudo su, or login via ssh with your password.

root@aegir:~# sudo su - bob
bob@aegir:~$

These notes describe a possible way of installing Apache with fastcgi using php5-fpm. The below configuration works but is experimental.

On a Ubuntu 12.04 box:

Install Aegir2 from Debian packages

Follow http://community.aegirproject.org/installing/debian. Below is a shortened form.

Add the Aegir package "Software Source" repository:

Install aegir2:

If the installation fails, it might be that your FQDN is not set or your postfix is not configured.

Verify your aegir site is working

Navigate your browser to the FQDN of your aegir server. You need to see the "Welcome to Aegir" page.

Modify the configuration to use fastcgi / php5-fpm

Edit /etc/apt/sources.list and enable the multiverse repository (it's needed for the libapache2-mod-fastcgi package).

Then install php5-fpm and make apache use it:

Modify /etc/php5/fpm/pool.d/www.conf Make it listen on socket, instead of IP:

Create /etc/apache2/conf.d/php-fpm.conf with this content:

Enable apache modules:

Restart php5-fpm and apache to load the new configs:

Verify that you can still reach your Aegir site

Navigate your browser to the FQDN of your aegir server. This time, you are using php5-fpm. Verify e.g. via phpinfo().

--
Page author: Marji Cermak marji@morpht.com, http://morpht.com

While building (and re-building) platforms using makefiles and the Aegir front-end (node/add/platform) is great, it has a number of minor annoyances. They're slow, and consume lots of bandwidth, but worse still, even a small hiccup in drupal.org (or other) infrastructure can break a platform build, which requires starting from scratch each time.

When you maintain platforms with upwards of 300 modules, themes and libraries, such as with kPlatforms, and you start to deploy these to multiple clients' Aegir servers, those minor annoyances can become significant challenges.

There is good documentation on setting this up for use with Drush Make already, such as this blog post. However, there are some gotchas when doing so in Aegir. Most of the suggested solutions involve something like setting environment variables in ~/.bashrc, which works fine if you're running 'drush make' yourself in a login shell. However, since the Aegir is running in a non-interactive shell, there isn't a opportunity to set environment variables.

Thankfully, Drush Make supports curl, and opts for it automatically if it's installed. Careful reading of curl's manpage indicates that:

[curl] always [...] checks for a default config file and uses it if found.

So, in the end, all that's required is a /var/aegir/.curlrc file containing:

proxy = "http://<proxy_host>[:port]"

Thus a basic configuration on Debian would involve installing a proxy, such as Squid:

# apt-get install squid3

Then add an acl in /etc/squid3/squid.conf allowing access from your local network, e.g.:

acl localnet src 192.168.0.0/16
http_access allow localnet

Restart Squid and you're good to go:

# /etc/init.d/squid3 restart

Testing that Aegir is actually using your new proxy cache, and that Squid is actually caching the makefile projects involves taking a quick look at the logs of the Squid server:

# tail -f /var/log/squid3